Voice data processing method and electronic equipment

Abstract

The invention discloses a voice data processing method and electronic equipment, and the method comprises the steps: carrying out the frequency division processing of first voice data if the initial sampling rate of the first voice data in to-be-processed multi-frame voice data is greater than the target sampling rate of a set model, and obtaining first low-frequency data and first high-frequency data; performing down-sampling processing on the first low-frequency data according to the target sampling rate and the initial sampling rate to obtain first reference low-frequency data; inputting the first reference low-frequency data into a set model to obtain a first masking value set of the first reference low-frequency data; calculating a second masking value set according to the first masking value set; and generating first target voice data based on the first reference low-frequency data, the first masking value set, the first high-frequency data and the second masking value set. Therefore, one model is suitable for voice data with different sampling rates, high-frequency components are not lost, and the voice quality is not reduced; an installation package of one model is small, and installation resources of the mobile terminal are reduced.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method and electronic device for processing voice data. Background Art

[0002] In the field of speech data processing, this can be achieved through deep learning. Usually, the sampling rate supported by the deep learning model has been determined in the training phase. In the application phase, only speech data sampled at the same sampling rate as the sampling rate supported by the model can be processed. If speech data sampled at other sampling rates is processed, the processed speech data may be unusable.

[0003] There are usually two processing methods in the prior art. In one processing method, multiple models that support different sampling rates can be trained, but this method requires preparing training samples with different sampling rates, which is a lot of work, and integrating multiple models at the same time will also result in a large installation package. In another processing method, when the sampling rate of the input voice data is greater than the sampling rate supported by the model, the input voice data is downsampled, and then after being processed by the model, the downsampled voice data is upsampled to the original sampling rate. However, this processing will lose high-frequency components and reduce voice quality. Summary of the invention

[0004] In an exemplary embodiment of the present application, a method for processing voice data and an electronic device are provided to implement the use of a trained model for voice data input at different sampling rates without losing high-frequency components or reducing voice quality; the installation package of a model is smaller, reducing the installation resources on the mobile terminal.

[0005] According to a first aspect of an exemplary embodiment, a method for processing speech data is provided, comprising:

[0006] Obtain multiple frames of voice data to be processed;

[0007] If the initial sampling rate of the first voice data is greater than the target sampling rate of the set model, the first voice data is subjected to frequency division processing to obtain first low-frequency data and first high-frequency data; wherein the set model is a model for optimizing the first voice data; and the first voice data is any one frame of the multiple frames of voice data to be processed;

[0008] Downsampling the first low-frequency data according to the target sampling rate and the initial sampling rate to obtain first reference low-frequency data; wherein the sampling rate of the first reference low-frequency data is the same as the target sampling rate;

[0009] Inputting the first reference low-frequency data into the setting model to obtain a first masking value set of the first reference low-frequency data; wherein the first masking value set is used to describe the amplitude spectrum characteristics of the first low-frequency data;

[0010] Calculating a second masking value set according to the first masking value set; wherein the second masking value set is used to describe the amplitude spectrum characteristics of the first high-frequency data;

[0011] First target speech data corresponding to the first speech data is generated based on the first reference low-frequency data, the first masking value set, the first high-frequency data, and the second masking value set.

[0012] In the embodiment of the present application, for any frame of the multi-frame speech data to be processed, taking the first speech data as an example, when the initial sampling rate of the first speech data is greater than the target sampling rate of the set model, if the downsampling process is directly performed and then input into the set model, and then the upsampling process is performed according to the output of the set model, the processing result obtained in this way may lose high-frequency components and reduce the number of voices. Therefore, the first speech data can be subjected to frequency division processing to obtain first low-frequency data and first high-frequency data, and then different operations are performed on the first low-frequency data and the first high-frequency data. Among them, for the first low-frequency data, the downsampling process can be performed according to the target sampling rate and the initial sampling rate to obtain the first reference low-frequency data with the same target sampling rate, and then the first reference low-frequency data is input into the set model to obtain the first masking value set of the first reference low-frequency data, and the first masking value set is used to describe the amplitude spectrum characteristics of the first low-frequency data. For the first high-frequency data, in order not to lose the high-frequency component, the first high-frequency data does not need to be input into the set model, and the second masking value set is directly calculated according to the first masking value set. In this way, the first target voice data corresponding to the first voice data can be generated based on the first reference low-frequency data, the first masking value set, the first high-frequency data and the second masking value set. The same model can support multiple sampling rates. By applying the method of the embodiment of the present application, a model can be applicable to input voice data of different sampling rates, and high-frequency components will not be lost, and voice quality will not be reduced. In addition, the method of not needing to train a model for each sampling rate also reduces the model training time and the time to prepare training data, reduces the size of the installation package, and reduces the installation resources of the mobile terminal.

[0013] In an optional implementation, performing frequency division processing on the first voice data to obtain first low-frequency data and first high-frequency data includes:

[0014] Inputting the first speech data into a low-pass filter to obtain first low-frequency data;

[0015] Inputting the first speech data into a high-pass filter to obtain first high-frequency data;

[0016] The cutoff frequencies of the low-pass filter and the high-pass filter are determined according to the target sampling rate.

[0017] In the above embodiment, in order not to lose the high-frequency components, the first voice data is preferentially subjected to frequency division processing, so that different subsequent processing methods can be adopted for the obtained first high-frequency data and first low-frequency data.

[0018] In an optional implementation, the first reference low-frequency data is input into a setting model to obtain a first masking value set of the first reference low-frequency data, including:

[0019] The first low-frequency data and the second low-frequency data are spliced ​​to obtain low-frequency spliced ​​data; wherein the second low-frequency data is the low-frequency data corresponding to the second voice data, and the second voice data is the previous frame of voice data of the first voice data; if the first voice data is the initial frame, the second voice data is the first preset voice data;

[0020] Performing a windowing operation and a fast Fourier transform (FFF) operation on the low-frequency spliced ​​data to obtain first transformed low-frequency data;

[0021] The first transformed low-frequency data is input into the setting model to obtain a first mask value set of the first reference low-frequency data.

[0022] In the above embodiment, the first masking value set of the first reference low-frequency data and the second masking value set of the first high-frequency data are calculated in different ways. In the process of calculating the first masking value set, the delay, calculation complexity and effect are comprehensively considered, and the last frame of speech data is selected for splicing, and then the low-frequency spliced ​​data is operated accordingly. The first masking value set obtained in this way can more accurately describe the amplitude spectrum characteristics of the first reference low-frequency data.

[0023] In an optional embodiment, the method further includes:

[0024] The first high-frequency data and the second high-frequency data are spliced ​​to obtain high-frequency spliced ​​data; wherein the second high-frequency data is the high-frequency data corresponding to the second voice data, and the second voice data is the previous frame of voice data of the first voice data; if the first voice data is the initial frame, the second voice data is the second preset voice data;

[0025] A windowing operation and an FFT operation are performed on the high-frequency spliced ​​data to obtain first transformed high-frequency data.

[0026] In the above embodiment, considering the delay, computational complexity and effect, the splicing with the previous frame of speech data is selected, and then the high-frequency splicing data is operated accordingly. The second masking value set is calculated using the first transformed high-frequency data obtained in this way, and the obtained second masking value set can more accurately describe the amplitude spectrum characteristics of the first high-frequency data.

[0027] In an optional implementation, generating first target speech data corresponding to the first speech data based on the first reference low-frequency data, the first masking value set, the first high-frequency data, and the second masking value set includes:

[0028] Multiplying the first transformed low-frequency data by the first masking value set to obtain first masked low-frequency data, and performing an inverse fast Fourier transform IFFT operation and a weighted folding and adding WOLA operation on the first masked low-frequency data to obtain first target low-frequency data;

[0029] Performing interpolation processing on the first target low-frequency data; wherein the interpolation multiple is determined according to the initial sampling rate and the target sampling rate;

[0030] Inputting the interpolated first target low-frequency data into a low-pass filter to obtain first filtered low-frequency data;

[0031] Multiplying the first transformed high-frequency data by the second mask value set to obtain first masked high-frequency data, and performing IFFT operation and WOLA operation on the first masked high-frequency data to obtain first target high-frequency data;

[0032] The first filtered low-frequency data and the first target high-frequency data are synthesized to obtain first target voice data corresponding to the first voice data.

[0033] In the above embodiment, since the frequency division process is performed, the mixing process is also required. Therefore, in the process of the mixing process, the first reference low-frequency data and the first high-frequency data are multiplied by the first masking value set and the second masking value set corresponding to themselves, respectively, and since the calculation process of the second masking value set does not need to go through the setting model, it is ensured that the synthesized first target speech data does not lose high-frequency components. In addition, the IFFT and WOLA operations can further ensure the accuracy of the synthesized first target data.

[0034] In an optional implementation, calculating the second masking value set according to the first masking value set includes:

[0035] Selecting some masking values ​​in the first masking value set for calculation to obtain a reference masking value; wherein the number of masking values ​​in the first masking value set is the number of frequency domain points of the first reference low-frequency data;

[0036] A second masking value set is calculated based on a reference masking value, a set constant, and the number of frequency domain points of the first high-frequency data; wherein the number of frequency domain points of the first high-frequency data is determined based on the number of frequency domain points of the first reference low-frequency data, and the relationship between the initial sampling rate and the target sampling rate.

[0037] In the above embodiment, the first masking value set of the first reference low-frequency data and the second masking value set of the first high-frequency data are calculated in different ways. In the calculation process of the second masking value set, in order not to lose the high-frequency components, it is not necessary to input the first high-frequency data into the set model, but the first masking value set is used to calculate the second masking value set. In this process, the relationship between the number of frequency domain points of the first reference low-frequency data and the number of frequency domain points of the first high-frequency data is considered. In this way, the obtained second masking value set can better restore the high-frequency components when mixing.

[0038] In an optional implementation, each second masking value in the second masking value set is calculated by the following formula;

[0039] index = constcoeff + i / len;

[0040] highfreq_mask[i]=powf(base_mask,index);

[0041] Among them, highfreq_mask[i] is the i-th second mask value, powf is the power function, base_mask is the base mask value, constcoeff is the set constant, LEN is the number of frequency domain points of the first high-frequency data, and i is the number of any second mask value.

[0042] In the above embodiment, a reference masking value is calculated by the first masking value set, and the reference masking value can be used when calculating each second masking value in the second masking value set. The second masking value set calculated in this way does not need to be set by a model, does not lose high-frequency components, and ensures voice quality.

[0043] According to a second aspect of the exemplary implementation, there is provided an electronic device, including a processor and a memory;

[0044] Memory, configured to execute:

[0045] Storage setting model;

[0046] A processor configured to execute:

[0047] Obtain multiple frames of voice data to be processed;

[0048] If the initial sampling rate of the first voice data is greater than the target sampling rate of the set model, the first voice data is subjected to frequency division processing to obtain first low-frequency data and first high-frequency data; wherein the set model is a model for optimizing the first voice data; and the first voice data is any one frame of the multiple frames of voice data to be processed;

[0049] Downsampling the first low-frequency data according to the target sampling rate and the initial sampling rate to obtain first reference low-frequency data; wherein the sampling rate of the first reference low-frequency data is the same as the target sampling rate;

[0050] Inputting the first reference low-frequency data into the setting model to obtain a first masking value set of the first reference low-frequency data; wherein the first masking value set is used to describe the amplitude spectrum characteristics of the first low-frequency data;

[0051] Calculating a second masking value set according to the first masking value set; wherein the second masking value set is used to describe the amplitude spectrum characteristics of the first high-frequency data;

[0052] First target speech data corresponding to the first speech data is generated based on the first reference low-frequency data, the first masking value set, the first high-frequency data, and the second masking value set.

[0053] According to a third aspect of an exemplary implementation, there is provided a device for processing speech data, comprising:

[0054] The processing unit is used to: obtain multiple frames of voice data to be processed;

[0055] The processing unit is further used for: if the initial sampling rate of the first voice data is greater than the target sampling rate of the set model, performing frequency division processing on the first voice data to obtain first low-frequency data and first high-frequency data; wherein the set model is a model for optimizing the first voice data; and the first voice data is any one frame of the multiple frames of voice data to be processed;

[0056] A resampling unit, configured to: perform downsampling processing on the first low-frequency data according to a target sampling rate and an initial sampling rate to obtain first reference low-frequency data; wherein the sampling rate of the first reference low-frequency data is the same as the target sampling rate;

[0057] The processing unit is further used to: input the first reference low-frequency data into the setting model to obtain a first masking value set of the first reference low-frequency data; wherein the first masking value set is used to describe the amplitude spectrum characteristics of the first low-frequency data;

[0058] The processing unit is further used to: calculate a second masking value set according to the first masking value set; wherein the second masking value set is used to describe the amplitude spectrum characteristics of the first high-frequency data;

[0059] The synthesis unit is used to generate first target speech data corresponding to the first speech data based on the first reference low-frequency data, the first masking value set, the first high-frequency data and the second masking value set.

[0060] According to a fourth aspect of the exemplary implementation, a computer storage medium is provided, in which computer program instructions are stored. When the instructions are executed on a computer, the computer executes the method for processing speech data as described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0062] Figure 1 An application scenario diagram of a method for processing voice data provided by an embodiment of the present application is exemplarily shown;

[0063] Figure 2 A flowchart of a method for processing voice data provided by an embodiment of the present application is exemplified;

[0064] Figure 3 An amplitude-frequency characteristic diagram of a filter applied when the first voice data provided by an embodiment of the present application has a sampling rate of 48KHz is exemplarily shown;

[0065] Figure 4 An amplitude-frequency characteristic diagram of a filter applied when the first voice data provided by an embodiment of the present application has a sampling rate of 32KHz is exemplarily shown;

[0066] Figure 5 A flowchart of a method for calculating a first masking value set provided by an embodiment of the present application is exemplarily shown;

[0067] Figure 6 A flowchart of a method for calculating a second masking value set provided by an embodiment of the present application is exemplarily shown;

[0068] Figure 7 A flowchart of a method for processing first high-frequency data provided by an embodiment of the present application is exemplarily shown;

[0069] Figure 8 A flowchart of a method for generating first target voice data provided by an embodiment of the present application is exemplarily shown;

[0070] Fig. 9 The following is a schematic diagram showing the structure of a voice data processing device provided by an embodiment of the present application;

[0071] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is exemplarily shown. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0073] For ease of understanding, the terms involved in the embodiments of the present application are explained below:

[0074] (1) Sampling rate: Also known as sampling speed or sampling frequency, it defines the number of samples extracted from a continuous signal to form a discrete signal per unit time. In layman's terms, sampling rate refers to how many signal samples can be collected per unit time. For example, if the sampling rate is 16KHz, 16,000 data can be collected in one second.

[0075] (2) Resampling: Resampling is a data processing technology that is mainly used to deal with imbalanced data sets. Resampling can be divided into two methods: upsampling and downsampling. The resampling sampling rate is compared with the original sampling rate of the digital signal. If it is greater than the original signal, it is called upsampling, and if it is less than the original signal, it is called downsampling. The essence of upsampling is interpolation or interpolation, and the essence of downsampling is the extraction of the signal.

[0076] (3) Upsampling: Increase the number of minority class samples to balance the categories in the data set. Common upsampling methods include random repeated sampling and synthetic minority class oversampling techniques. The advantage of upsampling is that it maintains the information of majority class samples and increases the model's ability to recognize minority classes, but the disadvantage is that it may lead to overfitting because repeated samples or synthetic samples may introduce redundant information.

[0077] (4) Downsampling: Reduce the number of majority class samples to balance the various categories in the data set. Common downsampling methods include random downsampling and ensemble methods. The advantage of downsampling is that it reduces the amount of data, making training faster and balancing the various categories in the data set, but the disadvantage is that valuable information may be lost, resulting in a decrease in the model's ability to recognize the majority class.

[0078] (5) Mask is a common operation in deep learning. Simply put, it is equivalent to covering the original tensor with a mask (which can be called a mask value or mask value) to shield or select some specific elements, so it is often used to construct tensor filters. For example, the nonlinear activation function Relu (a simple and rough binary division based on the positive and negative intervals of the output) and the dropout mechanism (binary division based on probability) can be understood as generalized mask operations. In an embodiment of the present application, after the model is set to process the input data, the output result is called the mask value.

[0079] (6) Weighted Overlap and Add (WOLA) filter bank is a highly efficient digital signal processing filter. WOLA filter bank has been widely used in speech signal processing, audio coding and decoding, wireless communication, radar signal processing and other fields. It can perform weighted overlap and add operations on input data.

[0080] (7) Fast Fourier Transform (FFT) is one of the most basic methods in time-domain to frequency-domain transformation analysis. FFT is used to convert time-domain signals into frequency-domain signals.

[0081] (8) Inverse Fast Fourier Transform (IFFT) is the inverse process of FFT. IFFT is used to convert frequency domain signals back to time domain signals.

[0082] (9) The amplitude response of an IIR filter refers to the gain or attenuation characteristics of the filter at different frequencies. The amplitude response graph is usually expressed as a function of frequency to show the gain of the filter at each frequency point.

[0083] In video conferencing, the speech codec is usually determined through negotiation of the Session Description Protocol (SDP). Different conferences may use different codecs, and the sampling rates used may also be inconsistent. Therefore, speech pre-processing algorithms usually need to support multiple sampling rates, such as acoustic echo cancellation (AEC) algorithm, noise reduction (NS) algorithm, and dereverberation. In recent years, with the development of deep learning, more and more speech processing algorithms have adopted deep learning to improve the algorithm effect. A large part of the algorithms use deep learning to find a set of masking values, and then multiply this set of masking values ​​in the frequency domain to obtain the enhanced signal. However, the sampling rate supported by the deep learning model is usually determined during the model training stage. If the actual sampling rate is inconsistent with the sampling rate used for model training, the result will be abnormal and cannot be used.

[0084] In the related art, there are two ways to overcome the above problems:

[0085] In one processing method, multiple models that support different sampling rates can be trained, but this method requires preparing training samples with different sampling rates, which is a lot of work, and integrating multiple models at the same time will also result in a large installation package.

[0086] In another processing method, when the sampling rate of the input voice data is greater than the sampling rate supported by the model, the input voice data is downsampled, and then after being processed by the model, the downsampled voice data is upsampled to the original sampling rate. However, this processing will lose high-frequency components and reduce voice quality. For example, if the sampling rate supported by the model is 16KHz, when input voice data with a sampling rate of 48KHz, the 48KHz voice is first downsampled to 16KHz, and then after being processed by the model, the 16KHz voice is upsampled to 48KHz. This method will lose high-frequency components and reduce voice quality.

[0087] To this end, an embodiment of the present application provides a method for processing speech data, in which, when the sampling rate of the input speech data is greater than the sampling rate supported by the model, the speech data can be subjected to frequency division processing to obtain low-frequency data and high-frequency data. Then the low-frequency data is subjected to down-sampling processing, and the high-frequency data is not resampled. The model is used to calculate a masking value set for describing the amplitude spectrum characteristics of the down-sampled low-frequency data, and then the masking value set of the high-frequency data is calculated based on the masking value set of the low-frequency data. Finally, according to the down-sampled low-frequency data and the corresponding masking value set, the high-frequency data and the corresponding masking value set, a mixing process is performed to obtain the target speech data. The model can process speech data whose sampling rate is greater than the sampling rate supported by the model, so that the target speech data obtained will not lose high-frequency components and will not reduce the speech quality.

[0088] After introducing the design ideas of the embodiments of the present application, the following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application and are not limited. In specific implementation, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0089] refer to Figure 1 , shows an application scenario diagram of a method for processing voice data, the method is implemented by an electronic device 11, a setting model is deployed in the electronic device 11, the setting model may be a deep learning model, and the sampling rate supported by the setting model may be 16KHz. In the field of video conferencing, commonly used sampling rates include 16KHz, 32KHz, and 48KHz. Through the method of the embodiment of the present application, the setting model supporting the 16KHz sampling rate can process voice data of 16KHz, 32KHz, and 48KHz.

[0090] When voice data input is detected, the electronic device processes it through the set model, and the obtained target voice data will not lose high-frequency components and will not reduce the voice quality.

[0091] To further illustrate the technical solution provided by the embodiment of the present application, this is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiment of the present application provides the method operation steps shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided in the embodiment of the present application.

[0092] Combine the following Figure 1 The application scenario shown is shown in Figure 2 A flowchart of a method for processing voice data is shown to illustrate the technical solution provided by an embodiment of the present application.

[0093] S201: Acquire multiple frames of speech data to be processed.

[0094] S202: If the initial sampling rate of the first speech data is greater than the target sampling rate of the set model, frequency division processing is performed on the first speech data to obtain first low-frequency data and first high-frequency data.

[0095] S203: Down-sampling the first low-frequency data according to the target sampling rate and the initial sampling rate to obtain first reference low-frequency data.

[0096] S204: Input the first reference low-frequency data into a set model to obtain a first masking value set of the first reference low-frequency data.

[0097] S205: Calculate a second masking value set according to the first masking value set.

[0098] S206: Generate first target speech data corresponding to the first speech data based on the first reference low-frequency data, the first masking value set, the first high-frequency data and the second masking value set.

[0099] In the embodiment of the present application, for any frame of the multi-frame speech data to be processed, taking the first speech data as an example, when the initial sampling rate of the first speech data is greater than the target sampling rate of the set model, if the downsampling process is directly performed and then input into the set model, and then the upsampling process is performed according to the output of the set model, the processing result obtained in this way may lose high-frequency components and reduce the number of voices. Therefore, the first speech data can be subjected to frequency division processing to obtain first low-frequency data and first high-frequency data, and then different operations are performed on the first low-frequency data and the first high-frequency data. Among them, for the first low-frequency data, the downsampling process can be performed according to the target sampling rate and the initial sampling rate to obtain the first reference low-frequency data with the same target sampling rate, and then the first reference low-frequency data is input into the set model to obtain the first masking value set of the first reference low-frequency data, and the first masking value set is used to describe the amplitude spectrum characteristics of the first low-frequency data. For the first high-frequency data, in order not to lose the high-frequency component, the first high-frequency data does not need to be input into the set model, and the second masking value set is directly calculated according to the first masking value set. In this way, the first target voice data corresponding to the first voice data can be generated based on the first reference low-frequency data, the first masking value set, the first high-frequency data and the second masking value set. The same model can support multiple sampling rates. By applying the method of the embodiment of the present application, a model can be applicable to input voice data of different sampling rates, and high-frequency components will not be lost, and voice quality will not be reduced. In addition, the method of not needing to train a model for each sampling rate also reduces the model training time and the time to prepare training data, reduces the size of the installation package, and reduces the installation resources of the mobile terminal.

[0100] In S201, for example, in a video conference or voice conference scenario, in order to optimize the voice data, the voice data can be collected at a certain sampling rate and input into a pre-trained set model for optimization. In an embodiment of the present application, multiple frames of voice data are obtained in real time.

[0101] In S202, a frame of multiple frames of voice data is taken as an example to illustrate the processing of voice data, and the frame of voice data is represented by first voice data. The sampling rate of the first voice data is called the initial sampling rate.

[0102] If the initial sampling rate of the first voice data is greater than the target sampling rate of the set model, the first voice data may be subjected to frequency division processing to obtain first low-frequency data and first high-frequency data.

[0103] Frequency division can be achieved in the following ways:

[0104] The first speech data is input into a low-pass filter to obtain first low-frequency data; the first speech data is input into a high-pass filter to obtain first high-frequency data. The cutoff frequencies of the low-pass filter and the high-pass filter are respectively determined according to the target sampling rate.

[0105] Optionally, since the voice data is insensitive to phase information, an infinite impulse response (IIR) digital filter can be used in the frequency division process. For example, the initial sampling rate of the first voice data is 48KHz, and the target sampling rate supported by the model is set to 16KHz, then the cutoff frequencies of the low-pass filter and the high-pass filter are 8KHz respectively.

[0106] Figure 3 The amplitude-frequency characteristic diagram of a filter applied when the first voice data has a sampling rate of 48KHz is provided in an embodiment of the present application. Figure 4 An amplitude-frequency characteristic diagram of a filter applied when the first voice data has a sampling rate of 32KHz is provided in an embodiment of the present application.

[0107] In S203, downsampling processing (also called decimation processing) can be performed on the first low-frequency data after the frequency division processing. For example, if the initial sampling rate of the first voice data is 48KHz and the target sampling rate supported by the setting model is 16KHz, the first low-frequency data is decimated three times; if the initial sampling rate of the first voice data is 32KHz and the target sampling rate supported by the setting model is 16KHz, the first low-frequency data is decimated twice. The sampling rate of the obtained first reference low-frequency data is 16KHz.

[0108] In step S204, the first reference low-frequency data is input into the set model to obtain a first masking value set of the first reference low-frequency data. Figure 5 accomplish, Figure 5 It includes the following steps S204-1 to S204-3.

[0109] S204-1: Splice the first low-frequency data and the second low-frequency data to obtain low-frequency spliced ​​data.

[0110] Taking into account the delay, computational complexity and effect, the first low-frequency data may be spliced ​​with a frame of other low-frequency data. In actual application, the number of frames of the spliced ​​other low-frequency data may be adjusted.

[0111] Optionally, the previous frame of voice data of the first voice data is selected, called the second voice data, and the second voice data is frequency-divided to obtain corresponding second low-frequency data and second high-frequency data. Therefore, the first low-frequency data and the second low-frequency data are spliced ​​to obtain low-frequency spliced ​​data.

[0112] In addition, if the first voice data is an initial frame, the second voice data may be replaced by the first preset voice data, and the first preset voice data may be zero.

[0113] S204-2: Perform a windowing operation and a fast Fourier transform (FFF) operation on the low-frequency spliced ​​data to obtain first transformed low-frequency data.

[0114] Among them, the windowing method and FFT method in the prior art can be used to process the low-frequency spliced ​​data to obtain the first changed low-frequency data.

[0115] S204 - 3 : Input the first transformed low-frequency data into a set model to obtain a first masking value set of the first reference low-frequency data.

[0116] The first mask value set is used to describe the amplitude spectrum characteristics of the first reference low-frequency data.

[0117] In S205, in order to avoid loss of high frequency components, when calculating the second masking value set of the first high frequency data, it can be calculated based on the first masking value set. The second masking value set here is used to describe the amplitude spectrum characteristics of the first high frequency data.

[0118] Figure 6 A process for calculating a second masking value set provided in an embodiment of the present application, Figure 6 At least includes steps S205-1 to S205-2.

[0119] S205-1: Select some masking values ​​in the first masking value set for calculation to obtain a reference masking value.

[0120] The number of masked values ​​in the first masked value set is the number of frequency domain points of the first reference low-frequency data. For example, the sampling rate of the first reference low-frequency data is 16KHz, and 10ms is used as an acquisition cycle, that is, one frame of speech data is generated every 10ms, then the number of frequency domain sampling points of the first reference low-frequency data is 160. In this way, the number of masked values ​​in the first masked value set is 160.

[0121] In actual application, some of the masking values ​​may be selected to calculate the reference masking value, for example, the average value of the last 80 masking values ​​may be selected as the reference masking value.

[0122] S205-2: Calculate a second masking value set according to a reference masking value, a set constant, and the number of frequency domain points of the first high-frequency data.

[0123] The number of frequency domain points of the first high-frequency data is determined based on the number of frequency domain points of the first reference low-frequency data and the relationship between the initial sampling rate and the target sampling rate. For example, if the number of frequency domain points of the first reference low-frequency data is 160, the initial sampling rate is 48KHz, and the target sampling rate is 16KHz, then the number of frequency domain points of the first high-frequency data is 480. The set constant can be 1.5. Optionally, the formula for calculating the second masking value is as follows:

[0124] index = constcoeff + i / len;

[0125] highfreq_mask[i]=powf(base_mask,index);

[0126] Among them, highfreq_mask[i] is the i-th second mask value, powf is the power function, base_mask is the base mask value, constcoeff is the set constant, LEN is the number of frequency domain points of the first high-frequency data, and i is the number of any second mask value.

[0127] In this way, 480 second masking values ​​are calculated in sequence according to the above formula.

[0128] Regarding S206, after the first masking value set and the second masking value set are calculated, the first reference low-frequency data and the first high-frequency data can be mixed by the first masking value set and the second masking value set to obtain the first target voice data after the first voice data is processed.

[0129] Before mixing, you can also pass Figure 7 Steps S20601-S20602 in the above process the first high frequency data:

[0130] S20601: Splice the first high-frequency data and the second high-frequency data to obtain high-frequency spliced ​​data.

[0131] The second high-frequency data is the high-frequency data corresponding to the second voice data, and the second voice data is the previous frame of voice data of the first voice data; if the first voice data is the initial frame, the second voice data is the second preset voice data. The implementation method of this step can refer to step S204-1, which is not repeated here.

[0132] S20602: Perform windowing and FFT operations on the high-frequency spliced ​​data to obtain first transformed high-frequency data.

[0133] The implementation method of this step can refer to step S204-2, which will not be repeated here.

[0134] After processing the first high-frequency data, first target speech data corresponding to the first speech data may be generated based on the first reference low-frequency data, the first masking value set, the first high-frequency data, and the second masking value set. Figure 8 Steps S206-1 to S206-5 in the embodiment are implemented as follows:

[0135] S206-1: multiply the first transformed low-frequency data by the first masking value set to obtain first masked low-frequency data, and perform an inverse fast Fourier transform IFFT operation and a weighted folding and adding WOLA operation on the first masked low-frequency data to obtain first target low-frequency data.

[0136] Since the FFT operation is performed before, an IFFT operation is required. In addition, in order to accurately restore the original low-frequency data, a WOLA operation can also be performed to obtain the first target low-frequency data.

[0137] S206-2: Perform interpolation processing on the first target low-frequency data.

[0138] The interpolation multiple is determined according to the initial sampling rate and the target sampling rate, and the interpolation process can also be called upsampling process. In the embodiment of the present application, the interpolation multiple can be 3. The interpolation method can directly adopt the zero-filling method.

[0139] S206 - 3 : Input the interpolated first target low-frequency data into a low-pass filter to obtain first filtered low-frequency data.

[0140] Optionally, the low-pass filter here may be a low-pass filter applied during frequency division processing. In an embodiment of the present application, the interpolated first target low-frequency data is input into the low-pass filter to obtain first filtered low-frequency data.

[0141] S206-4: Multiply the first transformed high-frequency data by the second mask value set to obtain first masked high-frequency data, and perform IFFT operation and WOLA operation on the first masked high-frequency data to obtain first target high-frequency data.

[0142] The implementation method of this step can be found in S206-1 and will not be described in detail here.

[0143] S206-5: Synthesize the first filtered low-frequency data and the first target high-frequency data to obtain first target voice data corresponding to the first voice data.

[0144] Here, synthesis can also be called addition. In the embodiment of the present application, the synthesis method of low-frequency data and high-frequency data in the relevant technology can be applied to obtain the first target voice data after the first voice data is processed.

[0145] It should be noted that, considering that there may be a delay when the setting model is processed, the delay also needs to be considered when multiplying with the mask value set.

[0146] In addition, when the initial sampling rate of the first voice data is less than the target sampling rate of the set model, the first voice data is upsampled, and the upsampled first voice data is input into the set model to obtain the masking value set output by the set model, and then upsampled again based on the masking value set to obtain the first target voice data for the first voice data processing result.

[0147] For example, if the initial sampling rate of the first speech data is 8KHz, it is increased to 16kHz by upsampling, and then processed by the set model, and then downsampled to 8KHz based on the obtained masking value set, and finally output directly.

[0148] like Fig. 9 As shown, based on the same inventive concept, an embodiment of the present application provides a speech data processing device, including a processing unit 91, a resampling unit 92 and a synthesis unit 93.

[0149] The processing unit 91 is used to: obtain multiple frames of voice data to be processed;

[0150] The processing unit 91 is further used to: if the initial sampling rate of the first voice data is greater than the target sampling rate of the set model, perform frequency division processing on the first voice data to obtain first low-frequency data and first high-frequency data; wherein the set model is a model for optimizing the first voice data; and the first voice data is any one frame of the multiple frames of voice data to be processed;

[0151] The resampling unit 92 is used to: perform downsampling processing on the first low-frequency data according to the target sampling rate and the initial sampling rate to obtain first reference low-frequency data; wherein the sampling rate of the first reference low-frequency data is the same as the target sampling rate;

[0152] The processing unit 91 is further used to: input the first reference low-frequency data into the setting model to obtain a first masking value set of the first reference low-frequency data; wherein the first masking value set is used to describe the amplitude spectrum characteristics of the first low-frequency data;

[0153] The processing unit 91 is further used to: calculate a second masking value set according to the first masking value set; wherein the second masking value set is used to describe the amplitude spectrum characteristics of the first high-frequency data;

[0154] The synthesis unit 93 is used to generate first target speech data corresponding to the first speech data based on the first reference low-frequency data, the first masking value set, the first high-frequency data and the second masking value set.

[0155] In an optional implementation, the processing unit 91 is specifically configured to:

[0156] Inputting the first speech data into a low-pass filter to obtain first low-frequency data;

[0157] Inputting the first speech data into a high-pass filter to obtain first high-frequency data;

[0158] The cutoff frequencies of the low-pass filter and the high-pass filter are determined according to the target sampling rate.

[0159] In an optional implementation, the processing unit 91 is specifically configured to:

[0160] The first low-frequency data and the second low-frequency data are spliced ​​to obtain low-frequency spliced ​​data; wherein the second low-frequency data is the low-frequency data corresponding to the second voice data, and the second voice data is the previous frame of voice data of the first voice data; if the first voice data is the initial frame, the second voice data is the first preset voice data;

[0161] Performing a windowing operation and a fast Fourier transform (FFF) operation on the low-frequency spliced ​​data to obtain first transformed low-frequency data;

[0162] The first transformed low-frequency data is input into the setting model to obtain a first mask value set of the first reference low-frequency data.

[0163] In an optional implementation, the processing unit 91 is further configured to:

[0164] The first high-frequency data and the second high-frequency data are spliced ​​to obtain high-frequency spliced ​​data; wherein the second high-frequency data is the high-frequency data corresponding to the second voice data, and the second voice data is the previous frame of voice data of the first voice data; if the first voice data is the initial frame, the second voice data is the second preset voice data;

[0165] A windowing operation and an FFT operation are performed on the high-frequency spliced ​​data to obtain first transformed high-frequency data.

[0166] In an optional implementation, the synthesis unit 93 is specifically used for:

[0167] The first transformed low-frequency data is multiplied by the first masking value set to obtain first masked low-frequency data, and an inverse fast Fourier transform IFFT operation and a weighted folding and adding WOLA operation are performed on the first masked low-frequency data to obtain first target low-frequency data;

[0168] Performing interpolation processing on the first target low-frequency data; wherein the interpolation multiple is determined according to the initial sampling rate and the target sampling rate;

[0169] The interpolated first target low-frequency data is input into a low-pass filter to obtain first filtered low-frequency data; the first transformed high-frequency data is multiplied by the second masking value set to obtain first masked high-frequency data, and the first masked high-frequency data is subjected to IFFT operation and WOLA operation to obtain first target high-frequency data;

[0170] The first filtered low-frequency data and the first target high-frequency data are synthesized to obtain first target voice data corresponding to the first voice data.

[0171] In an optional implementation, the processing unit 91 is specifically configured to:

[0172] Selecting some masking values ​​in the first masking value set for calculation to obtain a reference masking value; wherein the number of masking values ​​in the first masking value set is the number of frequency domain points of the first reference low-frequency data;

[0173] A second masking value set is calculated based on a reference masking value, a set constant, and the number of frequency domain points of the first high-frequency data; wherein the number of frequency domain points of the first high-frequency data is determined based on the number of frequency domain points of the first reference low-frequency data, and the relationship between the initial sampling rate and the target sampling rate.

[0174] In an optional implementation, the processing unit 91 is specifically configured to calculate each second masking value in the second masking value set by using the following formula:

[0175] index = constcoeff + i / len;

[0176] highfreq_mask[i]=powf(base_mask,index);

[0177] Among them, highfreq_mask[i] is the i-th second mask value, powf is the power function, base_mask is the base mask value, constcoeff is the set constant, LEN is the number of frequency domain points of the first high-frequency data, and i is the number of any second mask value.

[0178] Since the device is the device in the method in the embodiment of the present application, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0179] like Fig.10 As shown, based on the same inventive concept, an embodiment of the present application provides an electronic device, including a processor 101 and a memory 102 .

[0180] The memory 102 is configured to execute:

[0181] Storage setting model;

[0182] The processor 101 is configured to execute:

[0183] Obtain multiple frames of voice data to be processed;

[0184] If the initial sampling rate of the first voice data is greater than the target sampling rate of the set model, the first voice data is subjected to frequency division processing to obtain first low-frequency data and first high-frequency data; wherein the set model is a model for optimizing the first voice data; and the first voice data is any one frame of the multiple frames of voice data to be processed;

[0185] Downsampling the first low-frequency data according to the target sampling rate and the initial sampling rate to obtain first reference low-frequency data; wherein the sampling rate of the first reference low-frequency data is the same as the target sampling rate;

[0186] Inputting the first reference low-frequency data into the setting model to obtain a first masking value set of the first reference low-frequency data; wherein the first masking value set is used to describe the amplitude spectrum characteristics of the first low-frequency data;

[0187] Calculating a second masking value set according to the first masking value set; wherein the second masking value set is used to describe the amplitude spectrum characteristics of the first high-frequency data;

[0188] First target speech data corresponding to the first speech data is generated based on the first reference low-frequency data, the first masking value set, the first high-frequency data, and the second masking value set.

[0189] In an optional implementation, the processor 101 is specifically configured to execute:

[0190] Inputting the first speech data into a low-pass filter to obtain first low-frequency data;

[0191] Inputting the first speech data into a high-pass filter to obtain first high-frequency data;

[0192] The cutoff frequencies of the low-pass filter and the high-pass filter are determined according to the target sampling rate.

[0193] In an optional implementation, the processor 101 is specifically configured to execute:

[0194] The first low-frequency data and the second low-frequency data are spliced ​​to obtain low-frequency spliced ​​data; wherein the second low-frequency data is the low-frequency data corresponding to the second voice data, and the second voice data is the previous frame of voice data of the first voice data; if the first voice data is the initial frame, the second voice data is the first preset voice data;

[0195] Performing a windowing operation and a fast Fourier transform (FFF) operation on the low-frequency spliced ​​data to obtain first transformed low-frequency data;

[0196] The first transformed low-frequency data is input into the setting model to obtain a first mask value set of the first reference low-frequency data.

[0197] In an optional implementation, the processor 101 is further configured to execute:

[0198] The first high-frequency data and the second high-frequency data are spliced ​​to obtain high-frequency spliced ​​data; wherein the second high-frequency data is the high-frequency data corresponding to the second voice data, and the second voice data is the previous frame of voice data of the first voice data; if the first voice data is the initial frame, the second voice data is the second preset voice data;

[0199] A windowing operation and an FFT operation are performed on the high-frequency spliced ​​data to obtain first transformed high-frequency data.

[0200] In an optional implementation, the processor 101 is specifically configured to execute:

[0201] Multiplying the first transformed low-frequency data by the first masking value set to obtain first masked low-frequency data, and performing an inverse fast Fourier transform IFFT operation and a weighted folding and adding WOLA operation on the first masked low-frequency data to obtain first target low-frequency data;

[0202] Performing interpolation processing on the first target low-frequency data; wherein the interpolation multiple is determined according to the initial sampling rate and the target sampling rate;

[0203] Inputting the interpolated first target low-frequency data into a low-pass filter to obtain first filtered low-frequency data;

[0204] Multiplying the first transformed high-frequency data by the second mask value set to obtain first masked high-frequency data, and performing IFFT operation and WOLA operation on the first masked high-frequency data to obtain first target high-frequency data;

[0205] The first filtered low-frequency data and the first target high-frequency data are synthesized to obtain first target voice data corresponding to the first voice data.

[0206] In an optional implementation, the processor 101 is specifically configured to execute:

[0207] Selecting some masking values ​​in the first masking value set for calculation to obtain a reference masking value; wherein the number of masking values ​​in the first masking value set is the number of frequency domain points of the first reference low-frequency data;

[0208] A second masking value set is calculated based on a reference masking value, a set constant, and the number of frequency domain points of the first high-frequency data; wherein the number of frequency domain points of the first high-frequency data is determined based on the number of frequency domain points of the first reference low-frequency data, and the relationship between the initial sampling rate and the target sampling rate.

[0209] In an optional implementation, the processor 101 is specifically configured to execute, calculating each second masking value in the second masking value set by the following formula;

[0210] index = constcoeff + i / len;

[0211] highfreq_mask[i]=powf(base_mask,index);

[0212] Among them, highfreq_mask[i] is the i-th second mask value, powf is the power function, base_mask is the base mask value, constcoeff is the set constant, LEN is the number of frequency domain points of the first high-frequency data, and i is the number of any second mask value.

[0213] The embodiment of the present application also provides a computer storage medium, in which computer program instructions are stored. When the instructions are executed on a computer, the computer executes the steps of the above-mentioned voice data processing method.

[0214] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0215] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0216] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0217] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0218] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for processing speech data, characterized in that: include: Obtain multiple frames of voice data to be processed; If the initial sampling rate of the first voice data is greater than the target sampling rate of the set model, the first voice data is subjected to frequency division processing to obtain first low-frequency data and first high-frequency data; wherein the set model is a model for optimizing the first voice data; and the first voice data is any one frame of the multiple frames of voice data to be processed; Downsampling the first low-frequency data according to the target sampling rate and the initial sampling rate to obtain first reference low-frequency data; wherein the sampling rate of the first reference low-frequency data is the same as the target sampling rate; Inputting the first reference low-frequency data into the setting model to obtain a first masking value set of the first reference low-frequency data; wherein the first masking value set is used to describe the amplitude spectrum characteristics of the first low-frequency data; Calculating a second masking value set according to the first masking value set; wherein the second masking value set is used to describe the amplitude spectrum characteristics of the first high-frequency data; First target speech data corresponding to the first speech data is generated based on the first reference low-frequency data, the first masking value set, the first high-frequency data, and the second masking value set.

2. The method according to claim 1, characterized in that The performing frequency division processing on the first voice data to obtain first low-frequency data and first high-frequency data includes: Inputting the first speech data into a low-pass filter to obtain first low-frequency data; Inputting the first speech data into a high-pass filter to obtain first high-frequency data; The cutoff frequencies of the low-pass filter and the high-pass filter are respectively determined according to the target sampling rate.

3. The method according to claim 1, characterized in that The step of inputting the first reference low-frequency data into the setting model to obtain a first masking value set of the first reference low-frequency data includes: The first low-frequency data and the second low-frequency data are spliced ​​to obtain low-frequency spliced ​​data; wherein the second low-frequency data is the low-frequency data corresponding to the second voice data, and the second voice data is the previous frame of voice data of the first voice data; if the first voice data is the initial frame, the second voice data is the first preset voice data; Performing a windowing operation and a fast Fourier transform (FFF) operation on the low-frequency spliced ​​data to obtain first transformed low-frequency data; The first transformed low-frequency data is input into the setting model to obtain a first masking value set of the first reference low-frequency data.

4. The method according to claim 3, characterized in that The method further comprises: The first high-frequency data and the second high-frequency data are spliced ​​to obtain high-frequency spliced ​​data; wherein the second high-frequency data is the high-frequency data corresponding to the second voice data, and the second voice data is the previous frame of voice data of the first voice data; if the first voice data is the initial frame, the second voice data is the second preset voice data; A windowing operation and an FFT operation are performed on the high-frequency spliced ​​data to obtain first transformed high-frequency data.

5. The method according to claim 4, characterized in that The generating first target speech data corresponding to the first speech data based on the first reference low-frequency data, the first masking value set, the first high-frequency data and the second masking value set includes: Multiplying the first transformed low-frequency data by the first masking value set to obtain first masked low-frequency data, and performing an inverse fast Fourier transform IFFT operation and a weighted folding and adding WOLA operation on the first masked low-frequency data to obtain first target low-frequency data; Performing interpolation processing on the first target low-frequency data; wherein the interpolation multiple is determined according to the initial sampling rate and the target sampling rate; Inputting the interpolated first target low-frequency data into a low-pass filter to obtain first filtered low-frequency data; Multiplying the first transformed high-frequency data by the second masking value set to obtain first masked high-frequency data, and performing IFFT operation and WOLA operation on the first masked high-frequency data to obtain first target high-frequency data; The first filtered low-frequency data and the first target high-frequency data are synthesized to obtain first target voice data corresponding to the first voice data.

6. The method according to any one of claims 1 to 5, characterized in that: The calculating a second masking value set according to the first masking value set comprises: Selecting some masking values ​​in the first masking value set for calculation to obtain a reference masking value; wherein the number of masking values ​​in the first masking value set is the number of frequency domain points of the first reference low-frequency data; A second masking value set is calculated based on the benchmark masking value, a set constant, and the number of frequency domain points of the first high-frequency data; wherein the number of frequency domain points of the first high-frequency data is determined based on the number of frequency domain points of the first reference low-frequency data, and the relationship between the initial sampling rate and the target sampling rate.

7. The method according to claim 6, characterized in that Calculate each second masking value in the second masking value set by the following formula; index = constcoeff + i / len; highfreq_mask[i]=powf(base_mask,index); Among them, highfreq_mask[i] is the i-th second mask value, powf is the power function, base_mask is the benchmark mask value, constcoeff is the set constant, LEN is the number of frequency domain points of the first high-frequency data, and i is the number of any second mask value.

8. An electronic device, characterized in that: including a processor and a memory; The memory is configured to execute: Storage setting model; The processor is configured to execute: Obtain multiple frames of voice data to be processed; If the initial sampling rate of the first voice data is greater than the target sampling rate of the set model, the first voice data is subjected to frequency division processing to obtain first low-frequency data and first high-frequency data; wherein the set model is a model for optimizing the first voice data; and the first voice data is any one frame of the multiple frames of voice data to be processed; Downsampling the first low-frequency data according to the target sampling rate and the initial sampling rate to obtain first reference low-frequency data; wherein the sampling rate of the first reference low-frequency data is the same as the target sampling rate; Inputting the first reference low-frequency data into the setting model to obtain a first masking value set of the first reference low-frequency data; wherein the first masking value set is used to describe the amplitude spectrum characteristics of the first low-frequency data; Calculating a second masking value set according to the first masking value set; wherein the second masking value set is used to describe the amplitude spectrum characteristics of the first high-frequency data; First target speech data corresponding to the first speech data is generated based on the first reference low-frequency data, the first masking value set, the first high-frequency data, and the second masking value set.

9. The electronic device according to claim 8, characterized in that: The processor is specifically configured to execute: Inputting the first speech data into a low-pass filter to obtain first low-frequency data; Inputting the first speech data into a high-pass filter to obtain first high-frequency data; The cutoff frequencies of the low-pass filter and the high-pass filter are respectively determined according to the target sampling rate.

10. The electronic device according to claim 8, characterized in that: The processor is specifically configured to execute: The first low-frequency data and the second low-frequency data are spliced ​​to obtain low-frequency spliced ​​data; wherein the second low-frequency data is the low-frequency data corresponding to the second voice data, and the second voice data is the previous frame of voice data of the first voice data; if the first voice data is the initial frame, the second voice data is the first preset voice data; Performing a windowing operation and a fast Fourier transform (FFF) operation on the low-frequency spliced ​​data to obtain first transformed low-frequency data; The first transformed low-frequency data is input into the setting model to obtain a first masking value set of the first reference low-frequency data.

Images