Tibetan speech data generation method based on fast Fourier transform

Through fast Fourier transform and automated acquisition technology, the complexity and cost of Tibetan speech data generation are solved, and efficient and accurate speech data generation is achieved, suitable for Tibetan and other low-resource languages.

CN119091857AInactive Publication Date: 2024-12-06UNIV OF ELECTRONICS SCI & TECH OF CHINA
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411590707.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-08
Publication Date
2024-12-06
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

As a low-resource language, Tibetan language faces problems such as difficulty in data collection, high cost and cumbersome process. Its complex pronunciation characteristics and dialect diversity lead to challenges in speech data generation technology.

Method used

Using the fast Fourier transform method, the Tibetan voice data is collected through automated crawler code and robot process automation software, window preprocessing, short-term fast Fourier transform, adaptive frequency domain filtering and gradient optimization, and reduce calculations. Complexity, improve data generation quality and accuracy.

Benefits of technology

It realizes efficient and accurate frequency domain analysis, captures the differences in Tibetan pronunciation characteristics and dialects, improves the efficiency and quality of speech data generation, reduces data collection costs, and enhances the generalization ability and robustness of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091857A_ABST
    Figure CN119091857A_ABST
Patent Text Reader

Abstract

The invention discloses a Tibetan speech data generation method based on fast Fourier transform, which comprises the following steps of: acquiring speech data, applying a Hamming window function on a time domain to reduce spectrum leakage, and converting an acquired audio signal from the time domain to a frequency domain by using a fast Fourier transform algorithm to realize efficient frequency analysis. A Butterworth band-pass filter is designed to filter voice data in a frequency domain so as to extract specific frequency components, the similarity of signals before and after filtering is calculated by using a cosine distance so as to evaluate a filtering effect, a gradient descent algorithm is adopted to search and optimize filter parameters, and the optimal filtering effect is achieved by iteratively optimizing and adjusting each parameter. And converting the final frequency domain signal into a time domain signal by using inverse fast Fourier transform to generate a high-quality audio file. According to the scheme of the invention, by controlling the parameters in the time-frequency conversion and filtering process, the generation quality and accuracy of the Tibetan speech data are effectively improved, and the cost of data acquisition is greatly reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of speech generation, and in particular to a Tibetan speech data generation method based on fast Fourier transform. Background Art

[0002] In the field of computer science and artificial intelligence, speech generation technology is gradually becoming an important bridge connecting different languages ​​and cultures. This technology can not only improve the accuracy of speech recognition and natural language processing, but also create a richer and more convenient interactive experience for multilingual users. Although in common languages ​​such as Chinese and English, speech recognition, speech synthesis, speech translation and other fields have made significant progress due to the development of technologies such as deep learning, which has promoted the widespread popularization and technological innovation of related applications, however, for minority languages, especially languages ​​such as Tibetan, the research and application of these technologies face a series of challenges.

[0003] There are significant differences between Tibetan and Chinese in terms of phonetics, grammar, vocabulary, etc. These differences require that Tibetan speech data generation technology needs to be specially designed and adjusted to adapt to its unique language characteristics. In addition, the diversity of Tibetan dialects also brings additional complexity to data collection. Tibetan dialects in different regions vary in pronunciation and intonation, which requires researchers to not only master standard phonetic rules, but also understand the characteristics of various dialects. High-quality speech data is the key to the success of speech generation technology. However, for low-resource languages ​​such as Tibetan, data collection faces many difficulties. There are relatively few Tibetan speakers, resulting in a limited amount of data available for training; secondly, the professionalism and complexity of Tibetan require that professionals must participate in the collection process to ensure the accuracy and consistency of the data. These factors have led to high data collection costs and cumbersome processes. Summary of the invention

[0004] In the face of the specific challenges of Tibetan as a low-resource language, as well as its complex speech features and dialect diversity, the present invention provides a Tibetan speech data generation method based on fast Fourier transform, aiming to solve the challenges faced by Tibetan as a low-resource language, and effectively capture its complex speech features and pronunciation differences in dialect diversity through fast and accurate frequency domain analysis. The fast Fourier transform FFT algorithm adopted by the present invention is based on butterfly operation, which achieves efficient calculation by decomposing the calculation of discrete Fourier transform into multiple smaller discrete Fourier transforms. Compared with direct discrete Fourier transform, FFT greatly reduces the calculation complexity and improves the calculation speed, while ensuring high-resolution and accurate frequency domain analysis of the signal. By controlling the parameters in the conversion and filtering process, this method can effectively improve the generation quality and accuracy of Tibetan speech data, and is suitable for the three major Tibetan dialects of Wei-Zang, Kangba and Amdo, and can be widely used in complex scene tasks.

[0005] To achieve the above-mentioned object of the invention, the present invention proposes a method for generating Tibetan speech data based on fast Fourier transform, which comprises the following steps:

[0006] Step S1: Obtain Tibetan speech data, use Python, MacOS system automation software and robotic process automation software RPA to design automated crawler code and processes, and collect speech samples of the three major Tibetan dialects of U-Tsang, Amdo and Kham;

[0007] Step S2: Perform windowing preprocessing on the speech data, segment the audio according to the window size, control the overlap between windows, apply the Hamming window function to each segment in the time domain, and perform endpoint detection on the speech sample to remove the silent segment;

[0008] Step S3: frequency domain signal conversion, using short-time fast Fourier transform to convert the Tibetan time domain speech signal into a frequency domain signal, The length of the sequence decomposition is Two subsequences of, each subsequence recursively applies the same decomposition steps, performs discrete Fourier transform calculation, and combines the discrete Fourier transform results of the subsequences to obtain a complete frequency domain signal after short-time fast Fourier transform;

[0009] Step S4: filtering the frequency domain signal, designing an adaptive frequency domain bandpass filter to filter the frequency domain signal to extract the signal within the required frequency range;

[0010] Step S5: Iterative optimization of filter parameters. Within a preset number of iterations, the filter parameters are dynamically adjusted using a gradient descent algorithm according to the similarity of the signals before and after filtering.

[0011] Step S6: Generation of Tibetan audio data. After the iteration is completed, the final optimized filter parameters and the filtered frequency domain signal are output, the final filtered frequency domain signal is inversely transformed by Fourier transform, the frequency domain signal is converted into a time domain signal, new Tibetan speech data is generated, and it is written into an audio file.

[0012] Further, in step S2, the Hamming window function is set to , the windowed signal obtained after windowing the collected speech data is set as , , , where N is the length of the window and t is the index in the window, i.e. the time variable.

[0013] Furthermore, the transfer function of the bandpass filter designed in step S4 is By coefficient and Sure:

[0014]

[0015] Among them, the coefficient and The initial value of is obtained through experiments. lie in [ f low , f high ] When the frequency is within the range, the filter allows the signal to pass without attenuation; when the frequency Less than or greater than , the filter attenuates the signal to zero.

[0016] Furthermore, the similarity judgment of the signals in step S5 is realized by calculating the cosine distance of the signal frequency domain before and after filtering. Expressed as ,in and Represent the FFT vectors before and after filtering, respectively. is the cosine similarity, ranging from [-1,1]. The smaller the value of the cosine distance d is, the more similar the key features of the signal frequency domain of the original audio and the generated audio before and after filtering are, and the better the generation effect is.

[0017] Furthermore, the gradient optimization in step S5 aims to guide the update direction and step size of the parameters by calculating the gradient of the objective function relative to the parameters, so as to gradually approach the extreme point of the function. The objective function is an objective function based on the difference between the frequency domain signal and the original speech signal. Cut-off frequency and The calculation optimization method is as follows:

[0018]

[0019]

[0020] in is a small perturbation value in the range of [0.001, 0.005], which is used to calculate the approximate value of the gradient. The cutoff frequency is updated as follows:

[0021]

[0022]

[0023] in is the learning rate, which controls the step size of parameter update.

[0024] The present invention adopts a Tibetan speech data generation method based on fast Fourier transform, which can efficiently convert time domain signals into frequency domain signals in a short time. The fast Fourier transform (FFT) algorithm adopted by the present invention is based on butterfly operation. By decomposing the calculation of discrete Fourier transform into multiple smaller discrete Fourier transforms, efficient calculation is achieved. Compared with direct discrete Fourier transform, FFT greatly reduces the calculation complexity and improves the calculation speed, while ensuring high-resolution and accurate frequency domain analysis of the signal; at the same time, through accurate filtering of frequency domain signals and optimization adjustment of filtering parameters, more useful speech features can be effectively extracted from limited Tibetan data, ensuring high-quality speech output, and ensuring the diversity and complexity of audio data to adapt to different dialects and speech features; by evaluating the similarity of signals before and after filtering, it is judged whether the filtering effect is improved. The scheme of the present invention not only improves the generalization ability and robustness of the model, but also improves the generation efficiency and quality of speech data, while greatly reducing the cost of additional data collection, effectively narrowing the gap between Tibetan and common languages ​​in technical applications, and bringing new impetus to the technical development of Tibetan and other low-resource languages. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a flowchart of a method for generating Tibetan speech data based on fast Fourier transform provided by an embodiment of the present invention;

[0026] Figure 2 is a schematic diagram of a fast Fourier transform algorithm provided by an embodiment of the present invention;

[0027] Figure 3 is a schematic diagram of spectrum analysis of original audio provided by an embodiment of the present invention;

[0028] Figure 4 is a schematic diagram of spectrum analysis of audio generated by the method of the present invention provided in an embodiment of the present invention;

[0029] Figure 5 is a schematic diagram of time spectrum analysis of original audio provided by an embodiment of the present invention;

[0030] Figure 6 It is a schematic diagram of time spectrum analysis of audio generated by the method of the present invention provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. For ordinary technicians in this field, other drawings can also be obtained based on these drawings without paying creative work. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for which protection is claimed, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0032] Taking into account the complexity of Tibetan audio signal processing, in order to solve the technical problems existing in the prior art, the present invention proposes a Tibetan speech data generation method based on fast Fourier transform, so as to obtain the required audio signal processing results without having a large data sample, thereby obtaining better quality audio with the help of the processing results.

[0033] like Figure 1 As shown, the method for generating Tibetan speech data based on fast Fourier transform proposed by the present invention comprises the following steps:

[0034] Step 1: Get Tibetan speech data

[0035] In this disclosure, the purpose of step 1 is to obtain basic Tibetan speech data for input of this method, and use Python, MacOS system automation software and robotic process automation software RPA to design automated crawler code and process, and collect about 2000 hours of speech samples of the three major Tibetan dialects of U-Tsang, Amdo and Kham. These data will be the object of processing of the present invention, and will be enhanced or generated into speech data through the processing of the present invention, thereby improving the quality of the audio signal and meeting specific application requirements. In the example, the audio signal is set to ,in is a time variable.

[0036] Step 2: Preprocess the speech data by adding windows

[0037] Windowing is to perform windowing on audio segments that are segmented into 25 millisecond segments in the time domain to reduce the sudden changes at both ends of the signal and the spectrum leakage effect, thereby improving the accuracy of frequency domain analysis and the quality of audio signal processing. In one implementation, the Hamming window function is selected. Windowing is performed to improve the accuracy of spectrum estimation by smoothing the start and end of the signal and to improve the periodicity of the signal, making it more suitable for Fourier analysis. In this example, the Hamming window function is set to , the windowed signal obtained after windowing is set to , , w(t) is defined as:

[0038]

[0039] Where N is the length of the window and t is the index in the window, i.e. the time variable.

[0040] Step 3: Perform a short-time fast Fourier transform on the windowed time domain signal to obtain a frequency domain signal

[0041] The fast Fourier transform algorithm recursively decomposes the signal sequence and applies butterfly operation in each level of recursive decomposition. Specifically, the fast Fourier transform algorithm first decomposes the discrete Fourier transform DFT of length N into two lengths of DFTs:

[0042]

[0043] in, and are the DFT of the even-indexed and odd-indexed samples of the original signal, respectively, is the complex form of the Nth root of unity. and The same decomposition process is applied recursively until the length of the sub-DFT becomes 1. At each level of recursion, a butterfly operation is generated, and its basic form is:

[0044]

[0045]

[0046] Among them, A and B are the two parts of the input signal, and are two parts of the output signal. In this way, the FFT algorithm can efficiently calculate the frequency domain representation of the signal, providing accurate frequency domain information for the spectrum analysis of Tibetan speech signals and subsequent audio processing steps. Perform Fourier transform to get its spectrum The process is as Figure 2 As shown:

[0047]

[0048] in is the frequency domain signal, is a frequency variable, Is an imaginary unit.

[0049] Step 4: Design a frequency domain bandpass filter

[0050] In this disclosure, designing a frequency domain bandpass filter is a crucial step that involves creating a specific filter , which can accurately control the frequency components passing through according to its transfer function. The function of the frequency domain bandpass filter is to selectively allow frequency signals within a certain range to pass through, while suppressing or reducing signals of other frequencies, which is of great significance for extracting and enhancing target speech signals, such as key speech features in Tibetan dialects. In the example, a Butterworth bandpass filter is used to filter the frequency domain signal. The transfer function of the bandpass filter is By coefficient and Determine, its mathematical expression is expressed as:

[0051]

[0052] Among them, the coefficient and The initial value of is obtained through experiments, and then it is iteratively updated through the gradient descent algorithm to optimize the objective function based on the difference between the frequency domain signal and the original speech signal. lie in [ f low , f high ] When the frequency is within the range, the filter allows the signal to pass without attenuation; when the frequency Less than or greater than , the filter attenuates the signal to zero.

[0053] Step 5: Apply the designed bandpass filter to the frequency domain signal

[0054] In the frequency domain, a bandpass filter is applied to filter the frequency domain signal. Specifically, the frequency response of the filter is multiplied by the spectrum of the signal. In this way, only the signal components within the set frequency range of the filter can pass through, while other frequency components are attenuated or blocked. This process helps to remove noise and unnecessary frequency components while retaining and enhancing key features of Tibetan speech, such as pitch, duration, and timbre. In the example, the transfer function of the filter is Application Get the filtered spectrum like Figure 3 As shown:

[0055]

[0056] Step 6: Iteratively optimize the filter parameters

[0057] Set the number of iterations. The number of iterations is obtained through experiments. The preferred number of iterations is 30. Within the preset number of iterations, the filter parameters are dynamically adjusted using the gradient descent algorithm based on the similarity of the signals before and after filtering. Specifically, the following are included:

[0058] (1) Perform similarity judgment on the frequency domain signal after filtering and the frequency domain signal before filtering

[0059] Similarity judgment refers to the process of evaluating the similarity between two or more signals or data sets in signal processing and data analysis. In the present disclosure, similarity judgment is achieved by calculating the cosine distance of the signal frequency domain before and after filtering. This distance is a measurement method for measuring the difference in the directions of two vectors. By minimizing the cosine distance, it can be ensured that the optimized audio signal retains the important features of the original signal. In the example, the cosine distance It is expressed as:

[0060]

[0061] in and Represent the FFT vectors before and after filtering, respectively. is the cosine similarity, ranging from [-1,1]. The smaller the cosine distance d value is, the more similar the key features of the signal frequency domain of the original audio and the generated audio before and after filtering are, and the better the generation effect is. Through this metric, the performance of the filter can be evaluated, and the filter parameters can be further adjusted to minimize the objective function based on the difference between the frequency domain signal and the original speech signal, thereby improving the quality and accuracy of the generated audio.

[0062] (2) Gradient optimization

[0063] The filter parameters are then iteratively updated using a gradient descent algorithm to optimize an objective function based on the difference between the frequency domain signal and the original speech signal.

[0064] Gradient optimization aims to guide the update direction and step size of parameters by calculating the gradient of the objective function with respect to the parameters, so as to gradually approach the extreme points of the function. In the process of gradient optimization disclosed in the present invention, the gradient is a vector, each component of which is the partial derivative of the objective function with respect to each parameter. The direction of the gradient points to the direction in which the objective function grows fastest for the minimization problem. By updating the parameters along the negative direction of the gradient, the value of the objective function can be gradually reduced, and finally a local minimum or a global minimum is found. Objective function Cut-off frequency and The calculation optimization method is as follows:

[0065]

[0066]

[0067] in is a small perturbation value in the range [0.001, 0.005], used to calculate the approximate value of the gradient. The formula for updating the cutoff frequency is:

[0068]

[0069]

[0070] in is the learning rate, which controls the step size of parameter update.

[0071] Then the updated and optimized bandpass filter is applied to the frequency domain signal to filter the frequency domain signal. If the number of iterations does not reach the termination condition, the similarity between the newly filtered frequency domain signal and the original frequency domain signal before filtering is judged, the cosine distance is recalculated and its value is evaluated to see whether it has become smaller, so as to judge whether the filtering effect has been improved, and the gradient optimization is performed again to calculate the objective function. The gradient of the filter parameters is obtained, and the filter parameters are updated according to the gradient information until the iteration ends, and the final optimized filter parameters and the filtered frequency domain signal are output.

[0072] Step 7: Perform inverse Fourier transform on the final filtered signal

[0073] The inverse Fourier transform is the inverse process of the Fourier transform, which is used to convert the frequency domain signal into the time domain signal. This process involves the inverse operation of complex numbers and the superposition of frequency components. In the example, the filtered spectrum Perform inverse Fourier transform to obtain the newly generated time domain signal :

[0074]

[0075] in, is the amplitude and phase information of the discrete frequency domain signal, is the length of the signal, is the time variable, Is an imaginary unit.

[0076] Step 8. Save the newly generated signal

[0077] In this example, the newly generated audio signal Save as a new audio file.

[0078] Figure 3The original audio spectrum in shows a significant high energy peak in the low frequency region (0-2000 Hz), indicating that the original audio has a large dynamic range. At the same time, although the energy gradually decreases in the high frequency region, the overall energy is still dispersed. In contrast, Figure 4 The spectrum of the generated audio has more concentrated and balanced energy, and its spectrum amplitude is up to about 20dB, which is conducive to improving the efficiency and accuracy of subsequent speech recognition or analysis work. Through this comparison, it can be seen that the scheme of the present invention uses the efficient calculation of FFT and optimizes the filter parameters to successfully adjust the characteristics of the audio spectrum, so that the generated audio spectrum is more in line with the processing target. In particular, it performs well in retaining high-frequency information, which is very critical for capturing the complex details of Tibetan language audio. In addition, by reducing the dynamic range of audio and improving spectrum balance, the scheme of the present invention improves the availability and applicability of audio data, and is particularly suitable for complex application scenarios that require accurate speech feature extraction.

[0079] Figure 5 and Figure 6 They are the original audio time domain diagram and the generated audio time domain diagram of the sample speech of the Amdo Tibetan dialect. Figure 5 ) has large amplitude fluctuations in the time domain, ranging from -10000 to 10000, which may cause volume control problems on different playback devices. Figure 6 ) shows that the amplitude is greatly compressed to the range of -3000 to 3000, which effectively realizes the normalization and control of the amplitude. Although the waveform features in the two figures have different amplitudes, they show similar patterns, indicating that the processing of the present invention not only retains the core features of the original audio, but also realizes its normalization. In addition, the consistency of the waveforms of the two on the time axis indicates good time alignment. The scheme of the present invention makes the generated audio more matched with the original audio in auditory and technical processing, which is particularly suitable for Tibetan, a low-resource language, and retains its complex speech features and dialect diversity while performing data enhancement.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating Tibetan speech data based on fast Fourier transform, characterized in that: The method comprises the following steps: Step S1: Obtain Tibetan speech data, use Python, MacOS system automation software and robotic process automation software RPA to design automated crawler code and processes, and collect speech samples of the three major Tibetan dialects of U-Tsang, Amdo and Kham; Step S2: Perform windowing preprocessing on the speech data, segment the audio according to the window size, control the overlap between windows, apply the Hamming window function to each segment in the time domain, and perform endpoint detection on the speech sample to remove the silent segment; Step S3: frequency domain signal conversion, using short-time fast Fourier transform to convert the Tibetan time domain speech signal into a frequency domain signal, The length of the sequence decomposition is Two subsequences of, each subsequence recursively applies the same decomposition steps, performs discrete Fourier transform calculation, and combines the discrete Fourier transform results of the subsequences to obtain a complete frequency domain signal after short-time fast Fourier transform; Step S4: filtering the frequency domain signal, designing an adaptive frequency domain bandpass filter to filter the frequency domain signal to extract the signal within the required frequency range; Step S5: Iterative optimization of filter parameters. Within a preset number of iterations, the filter parameters are dynamically adjusted using a gradient descent algorithm according to the similarity of the signals before and after filtering. Step S6: Generation of Tibetan audio data. After the iteration is completed, the final optimized filter parameters and the filtered frequency domain signal are output, the final filtered frequency domain signal is inverse Fourier transformed, the frequency domain signal is converted into a time domain signal, new Tibetan speech data is generated, and it is written into an audio file.

2. The method according to claim 1, characterized in that In step S2, the Hamming window function is set to , the windowed signal obtained after windowing the collected speech data is set as , , , where N is the length of the window and t is the index in the window, i.e. the time variable.

3. The method according to claim 1, characterized in that The transfer function of the bandpass filter designed in step S4 is By coefficient and Sure: , Among them, the coefficient and The initial value of is obtained through experiments. lie in When the frequency is within the range, the filter allows the signal to pass without attenuation; when the frequency Less than or greater than , the filter attenuates the signal to zero.

4. The method according to claim 1, characterized in that: The similarity judgment of the signals in step S5 is realized by calculating the cosine distance of the signal frequency domain before and after filtering. Expressed as ,in and Represent the FFT vectors before and after filtering, respectively. is the cosine similarity, ranging from [-1,1]. The smaller the value of the cosine distance d is, the more similar the key features of the signal frequency domain of the original audio and the generated audio before and after filtering are, and the better the generation effect is.

5. The method according to claim 1, characterized in that The gradient optimization in step S5 aims to guide the update direction and step size of the parameters by calculating the gradient of the objective function relative to the parameters, so as to gradually approach the extreme point of the function. The objective function is an objective function based on the difference between the frequency domain signal and the original speech signal. Cut-off frequency and The calculation optimization method is as follows: , , in is a small perturbation value in the range of [0.001, 0.005], which is used to calculate the approximate value of the gradient. The cutoff frequency is updated as follows: , , in is the learning rate, which controls the step size of parameter update.

Citation Information

Patent Citations

  • Fan fault feature identification method of adaptive Mel filter in strong noise environment

    CN115064182A

  • Tibetan dialect automatic identification method

    CN118762687A

  • Meta-learning for adaptive filters

    US20230343350A1

  • Earphone and audio processing method and apparatus therefor, and storage medium

    US20240323586A1