Human voice style recognition method based on time-frequency refinement analysis

Abstract

This invention discloses a method for recognizing vocal styles based on time-frequency refined analysis. The process is as follows: First, a fundamental frequency estimation and harmonic labeling step is performed. The short-duration vowel portion of the vocal signal is first used for fundamental frequency estimation. This estimation employs a frequency estimation algorithm combining time-domain autocorrelation and narrowband spectral energy. Then, based on the estimated fundamental frequency, adaptive harmonic labeling is performed on the spectrum to accurately identify all harmonics. Next, a time-frequency refined analysis step is performed. The vocal signal is refined and features are extracted from both the time and frequency domains, with a focus on periodic variations and harmonic structures. Finally, a support vector machine (SVM) model training and recognition step is performed. The extracted features and corresponding vocal style labels are used to train the SVM model. After training, the model can be used for vocal style recognition. The style is obtained by using the feature vectors extracted from the vocal signal as input.

Description

Technical Field

[0001] This invention relates to the field of language signal processing technology, specifically to a method for recognizing human voice style based on time-frequency refined analysis. Background Technology

[0002] Voice style refers to the personalized characteristics and stylistic expression of a person's voice in terms of timbre, pitch, and intonation. It is a relatively abstract concept without specific measurement standards. Previous voice recognition studies have mainly focused on attributes such as the emotion conveyed during speech, the speaker's age, and gender. However, research specifically targeting voice style recognition is relatively limited. Voice style recognition can be applied to screening announcers, presenters, actors, etc., or to intelligent voice assistants and speech translation software. It can also be used to create voice style datasets, providing support for research in this area.

[0003] Currently, some researchers categorize human voices into six styles based on gender and frequency: soprano, mezzo-soprano, contralto, tenor, baritone, and bass. Others categorize them into eight styles based on age and gender: loli voice, teenage girl voice, mature woman voice, middle-aged woman voice, young boy voice, youthful voice, middle-aged man voice, and elderly voice. Both classification methods are comprehensive, basically covering all population groups, but the definitions are rather general, lacking specific classifications for the voice styles of professionals in film, television, and broadcasting industries. Referring to the style terms provided by existing technical platforms and the characteristics of broadcasting styles, we categorize human voice styles into the following types: cute and sweet, youthful and fashionable, mature and intellectual, soft-spoken, steady and magnetic, grand and deep, passionate and powerful, etc. This classification method can provide a reference standard and basis for selecting voice styles for professionals in these industries.

[0004] Voice style recognition is a challenging task because it requires identifying subtle features of audio signals, such as the speaker's voice style, pitch, and pronunciation. These features are often highly subjective and difficult to measure. Furthermore, large-scale style-labeled datasets are lacking in voice style recognition. This is because the concept of voice style is very broad; different people may have different feelings and evaluations of the same voice. Therefore, constructing an accurate and representative voice style dataset is extremely challenging.

[0005] Traditionally, voice feature extraction has primarily relied on speech signal processing. These methods typically extract overall or macroscopic features of the signal, such as pitch, spectral centroid, and cepstral coefficients. However, these extractions do not encompass all the information in the signal and lack fine-grained feature extraction, such as harmonics. Most extracted features perform poorly, especially in voice recognition problems against complex backgrounds.

[0006] Therefore, accurately identifying each harmonic, performing refined time-domain and frequency-domain analysis of the human voice signal, and extracting refined features for human voice style recognition are of great significance. Summary of the Invention

[0007] The purpose of this invention is to address the subjective limitations of existing voice style recognition methods and provide a voice style recognition method based on refined time-frequency analysis. This invention preprocesses the voice signal, extracts short-time vowel portions for more accurate fundamental frequency estimation and harmonic labeling, and extracts refined features such as the time-domain single-cycle fundamental frequency sequence variation trend and frequency-domain harmonic ratio. These refined features are then input into the model to obtain the style output. The combination of traditional features such as spectral centroid and MFCC with refined features further improves the recognition accuracy. This method has wide applications, including screening announcers, hosts, actors, intelligent voice assistants, and voice translation software.

[0008] The objective of this invention can be achieved by adopting the following technical solutions:

[0009] A method for recognizing vocal styles based on time-frequency refined analysis, the method comprising the following steps:

[0010] S1, Collect human voice signal S voice And label them, build a human voice style dataset, and analyze S voice Preprocessing is performed to obtain the preprocessed human voice signal S. pvoice ;

[0011] S2, the preprocessed human voice signal S pvoice The fundamental frequency f is obtained by estimating the fundamental frequency f from the short-time vowel portion, and then the human voice signal S is analyzed. pvoice A fast Fourier transform is performed to obtain a spectrum, and adaptive harmonic marking is applied to the spectrum using the fundamental frequency f.

[0012] S3, for human voice signal S pvoice A refined analysis is performed in the time domain to extract refined time-domain features, including the standard deviation f of the single-cycle fundamental frequency sequence. std Median f of a single-period fundamental frequency sequence med Mean f of a single-period fundamental frequency sequence mean Distance f between the median and mean of a single-period fundamental frequency sequence dist ;

[0013] S4. Perform frequency domain refinement analysis on the harmonic-marked spectrum and extract refined frequency domain features, wherein the refined frequency domain features include the amplitude ratio R of the first harmonic to the second harmonic. fir_sec The amplitude ratio R of odd-order harmonics to even-order harmonics odd_even The amplitude ratio R of prime harmonics to non-prime harmonics prim_noneHarmonic amplitude ratio R harm_spect The amplitude ratio R of lower harmonics to higher harmonics low_high Harmonic variation rate R dec Harmonic peak-to-valley ratio R peak_valley Then, the time-domain refined features and the frequency-domain refined features are combined into a time-frequency refined feature vector V. fine ;

[0014] S5. Combining traditional features and time-frequency refined feature vector V fine The feature vector V is used to train a support vector machine model, wherein the traditional features include the spectral centroid C. spect Mel cepstral coefficients C MFCC ;

[0015] S6. Use a trained support vector machine model to identify the style of the human voice signal under test.

[0016] Further, step S1 is as follows:

[0017] S101, Collecting human voice signals S voice The system categorizes individuals and women into seven types of tags: cute and sweet, young and fashionable, mature and intellectual, soft-spoken, steady and magnetic, dignified and profound, and passionate and energetic. This categorization method can provide a good reference standard and basis for the selection of professionals in broadcasting, film and television and other industries.

[0018] S102, regarding the collected human voice signal S voice After filtering with an FIR high-pass filter to remove low-frequency noise, the pre-processed human voice signal S is obtained. pvoice This improves the quality of human voice signals and reduces noise interference.

[0019] Furthermore, step S2 is as follows:

[0020] S201, For the pre-processed human voice signal S pvoice The vowel portion of the signal is roughly extracted by taking the maximum value of the time-domain amplitude;

[0021] First, find the location of the maximum amplitude, denoted as t. max , see t max The subsequent signal is S pvoice The vowel part is then used for subsequent analysis and processing;

[0022] S202. The fundamental frequency f0 is roughly estimated by performing short-time autocorrelation in the time domain on the above vowel part through frame division.

[0023] Signal processing algorithms deal with stationary signals, so analyzing an entire segment of human voice is meaningless. Since human voice signals can be considered stable over a short period, it's important to analyze the signal S... pvoice Frame segmentation is performed. The vibration period of human voice is approximately 5-10ms. There are multiple periods within a frame, resulting in a frame length of typically 20-50ms. Half of this frame length is then used as the frame shift. The first frame, S, is extracted. pvoice (m) Perform short-time autocorrelation and record the start position t of the frame. start The frame length is N, a is the number of x-coordinate points of the autocorrelation function, and the autocorrelation function R(a) is:

[0024]

[0025] The autocorrelation function has a maximum value at a = 0. Then, find the index of the maximum value among the peaks excluding zero, denoted as maxb. Assuming the signal sampling rate is Fs, the fundamental frequency can be roughly estimated. This is subsequently used to calculate the multiple relationship between the frequency represented by the maximum spectral peak and the fundamental frequency;

[0026] S203. The fundamental frequency is precisely estimated using the narrowband spectral energy method and the iterative method to reduce the fundamental frequency estimation error;

[0027] First, the frequency represented by the maximum spectral peak is estimated using the narrowband spectral energy method. For S202, the frequency represented by the peak at position t is calculated. start Furthermore, the spectrum X is obtained by performing a Fast Fourier Transform on a frame of signal with a frame length of N. k k is the subscript of the spectrum, |X k | represents the spectral amplitude corresponding to the subscript k, and k0 is the subscript corresponding to the maximum spectral peak. For human voice signals, the narrowband bandwidth is generally taken at three points, one point to the left and one point to the right of the maximum spectral peak. The narrowband range at k0 is [k0-1, k0+1]. NDD is the number of points in the FFT. denoted by the spectral interval, representing the frequency difference between two points in the spectrum. The frequency expression based on narrowband spectral energy estimation is:

[0028]

[0029] The frequency f represented by the maximum spectral peak estimated by the narrowband spectral energy method max The coarse estimate of the fundamental frequency f0 calculated by S202 is used to calculate the multiple relationship between the maximum spectral peak frequency and the fundamental frequency. The round() function is used for rounding to the nearest integer.

[0030] Use the maximum spectral peak frequency f in the spectrum max Based on the multiple relationship t between the coarse estimate of the fundamental frequency f0 and the fundamental frequency, the fundamental frequency is estimated again. At this point, x is 1;

[0031] Then, an iterative method is applied. It is determined whether the fundamental frequency f1 estimated in the first iteration of the narrowband spectral energy method and the fundamental frequency f0 roughly estimated by autocorrelation satisfy the condition that the error is less than one-thousandth, i.e., |f1-f0| < 0.001f1. If this condition is met, the iteration is terminated, and the fundamental frequency f = f1 is determined. If not, the signal S is adjusted using a sampling rate transformation. pvoice The vowel portion is resampled, where the resampled frequency is the current fundamental frequency f. x The number of iterations x is increased by 1, and the fundamental frequency f is estimated again using the narrowband spectral energy method. x The iteration continues until the error between the current fundamental frequency and the previous fundamental frequency is less than one-thousandth or the number of iterations is greater than 50, i.e., |f x -f x-1 |<0.001f x ∪x>50, determine the fundamental frequency f=f x Iteration can reduce fundamental frequency error and make the estimated fundamental frequency closer to the true frequency.

[0032] S204. Using an adaptive method to change the harmonic spacing, a fast Fourier transform is performed in S203 to obtain the spectrum X. k In the middle, mark the amplitude values ​​and subscripts of all harmonics;

[0033] For the first three harmonics, i.e., when the harmonic order i <= 3, the fundamental frequency f is used as the interval between adjacent harmonics to mark the FFT spectrum, and the amplitude value and index of the peak point with the largest amplitude value in a certain range nearby are used as the amplitude value H of that harmonic. i and subscript HI i The left boundary l i and right boundary r i The values ​​of HI are shown in formulas (3) and (4), respectively. i-1 This indicates the position of the previous harmonic in the FFT spectrum. When i = 1, it means that the first harmonic is currently being marked. HI0 is set to 0 to calculate the range of l1 and r1.

[0034]

[0035]

[0036] To address the issue of harmonic spacing constantly changing with increasing frequency, an adaptive method for adjusting the harmonic spacing is adopted. Starting from the fourth harmonic (i>3), the harmonic spacing is dynamically changed. The average of the previous two harmonic spacings is used as the predicted value for the current harmonic spacing. Furthermore, the amplitude and index of the peak point with the largest frequency spectrum within a certain range are identified as the amplitude value H of that harmonic. i and subscript HI i The left boundary l i and right boundary ri The values ​​are shown in formulas (5) and (6) respectively:

[0037]

[0038]

[0039] When the peak value is found within this range, the harmonic amplitude value H is obtained. i and subscript HI i ;

[0040] When no peak value is found within this range, the right boundary range will be expanded to continue searching for harmonics and marking them. Each time, the range will be expanded by one point. However, the expansion of the right boundary range is not unlimited and must also satisfy formula (7):

[0041]

[0042] If no harmonic is found after expanding the right boundary range, then let HI i =2HI i-1 -HI i-2 H i =0, then proceed to the next harmonic marking;

[0043] Finally, the amplitude sequence of harmonics [H1, H2, ..., H] is obtained. i and the index sequence [HI1,HI2,…,HI] i ], where H1, H2, ..., H i These represent the amplitudes of the 1st, 2nd, ..., ith harmonics, HI1, HI2, ..., HI2, respectively. i These represent the subscripts of the 1st, 2nd, ..., ith harmonics, respectively.

[0044] Furthermore, step S3 is as follows:

[0045] S301, Regarding human voice signal S pvoice The fundamental frequency sequence is obtained by performing a single-cycle cross-correlation operation in the time domain.

[0046] First, the fundamental frequency f is estimated based on S203, and the time-domain start position t of the short-time vowel used for frequency estimation is determined. start The period length at this position is determined as follows: For each point, first calculate the period to the right of that position, then pass through that interval. The maximum cross-correlation value between the right and adjacent intervals determines the length L of the next period, from which the fundamental frequency of the next period can be obtained. By calculating sequentially, the period lengths and fundamental frequencies of all the cycles on the right side can be obtained. Then, the period lengths and fundamental frequencies of all the cycles on the left side can be calculated in the same way, and finally the entire signal S can be obtained. pvoiceThe fundamental frequency sequence is denoted as [f t1 ,f t2 ,…,f tm ], where f t1 f t2 ... f tm These represent the fundamental frequencies of the 1st, 2nd, ..., mth periods, respectively; arranging this fundamental frequency sequence in ascending order yields: f (1) ≤f (2) ≤…≤f (m) .

[0047] S302. Calculate and extract refined time-domain features, including the mean value f of the single-period fundamental frequency sequence. mean Standard deviation f of single-period fundamental frequency sequence std Median f of a single-period fundamental frequency sequence med Distance f between the median and mean of a single-period fundamental frequency sequence dist The calculation method is as follows:

[0048]

[0049]

[0050]

[0051] f dist =f med -f mean Formula (11)

[0052] The time-domain refinement features reflect the variation characteristics of the fundamental frequency of the entire signal.

[0053] Furthermore, step S4 is as follows:

[0054] S401. Perform frequency domain refinement analysis on the spectrum diagram after harmonic marking in step S204, and extract the frequency domain refinement features respectively.

[0055] Frequency domain refinement features include the amplitude ratio R of the first harmonic to the second harmonic. fir_sec As shown in formula (12): the ratio of the amplitude of the first harmonic to the second harmonic will affect the style and listening experience of the human voice. When the amplitude of the first harmonic to the second harmonic is relatively low, the human voice will sound fuller and clearer.

[0056]

[0057] The amplitude ratio R of odd-order harmonics to even-order harmonics odd_even H odd H represents the sum of the amplitude values ​​of all odd-order harmonics. evenThe sum of the amplitude values ​​of all even-order harmonics is represented by the formula (13). When the amplitude of odd-order harmonics is higher, the human voice will sound sharper and more tense; while when the amplitude of even-order harmonics is higher, the human voice will sound fuller and more rounded.

[0058]

[0059] The amplitude ratio R of prime harmonics to non-prime harmonics prim_none H prim H represents the sum of the amplitude values ​​of all prime harmonics. none It represents the sum of the amplitude values ​​of all non-prime harmonics. The calculation method is shown in formula (14). The amplitude ratio of prime harmonics to non-prime harmonics varies in different styles of vocals, which is beneficial for style recognition.

[0060]

[0061] Harmonic amplitude ratio R harm_spect H harm H represents the sum of the amplitude values ​​of all harmonics. spect The sum of the amplitude values ​​of the entire spectrum is represented by the formula (15). The human voice with stronger harmonic components has a richer timbre and more harmonic details, while the human voice with weaker harmonic components sounds more monotonous.

[0062]

[0063] The amplitude ratio R of lower harmonics to higher harmonics low_high The lower harmonics refer to the sum of the amplitude values ​​H of the first twenty harmonics. low Higher harmonics refer to the sum of the amplitude values ​​H of harmonics of order 20 and above. high The calculation method is shown in formula (16): when the amplitude of the lower harmonics is higher, the human voice sounds deeper and the amplitude of the higher harmonics is higher, the human voice sounds brighter and clearer.

[0064]

[0065] Harmonic variation rate R dec The harmonic variation rate refers to the rate of change of the average amplitude of the first three harmonics and the average amplitude of the fourth to sixth harmonics with respect to frequency. The calculation method is shown in formula (17). The harmonic variation rate can reflect how fast the harmonic amplitude value changes as the frequency increases.

[0066]

[0067] Harmonic peak-to-valley ratio R peak_valley Here, the average peak-to-valley ratio of the first six harmonics is used. The peak-to-valley ratio is the ratio of the average peak value of the harmonic to the average valley value of the left and right harmonics. The amplitude of the i-th harmonic is denoted as H.i The amplitudes of the left and right troughs are denoted as H. li and H ri The calculation method is shown in formula (18): the harmonic peak-to-valley ratio can reflect the smoothness and amplitude reduction of the harmonic peaks at both ends of different styles of vocals. The harmonic peak-to-valley ratios of different styles of vocals vary.

[0068]

[0069] S402. Combine the time-domain refined features extracted in step S302 and the frequency-domain refined features extracted in step S401 into a time-frequency refined feature vector V. fine That is

[0070] V fine =[f std ,f med ,f mean ,f dist ,R fir_sec ,R odd_even ,R prim_none ,R harm_spect ,R low_high ,R dec ,R peak_valley ].

[0071] Furthermore, step S5 is as follows:

[0072] S501. Extract traditional features, including the spectral centroid C. spect Mel cepstral coefficients C MFCC Then, the traditional features and the refined time-frequency feature vector V are combined. fine Combined into an eigenvector V, that is, V = [V fine C spect C MFCC ];

[0073] S502. Extract feature vectors V from all human voices in the dataset and train a support vector machine model.

[0074] Furthermore, in step S6, the feature vector V' of the human voice signal to be tested is first extracted, and the feature vector V' is input into the trained support vector machine model to complete the style recognition of the human voice signal to be tested, achieving a high recognition accuracy.

[0075] The present invention has the following advantages and effects compared with the prior art:

[0076] 1) This invention identifies human voice style based on time-frequency refined analysis, providing a reference standard and identification method for the screening of voice styles for practitioners in film, television, variety shows, broadcasting and other industries, alleviating the problem of limited research in this field, and laying the groundwork for future research in this field.

[0077] 2) This invention reduces the error of fundamental frequency estimation. Existing fundamental frequency estimation methods have the problem of large errors. This invention adopts a fundamental frequency estimation algorithm that combines time-domain autocorrelation and narrowband spectral energy. By using sampling rate transformation and iterative frequency estimation, the maximum spectral peak frequency component is highlighted, while other components are weakened, thus reducing spectral leakage. This makes the frequency estimate as close as possible to the true frequency, allowing the fundamental frequency estimation algorithm to still have good performance under noisy conditions.

[0078] 3) This invention solves the problem of difficult harmonic localization. Existing methods rarely extract and analyze harmonics. This invention uses the estimated fundamental frequency to mark harmonics in the FFT spectrum. To address the problem that the harmonic spacing changes continuously with the frequency, an adaptive method for changing the harmonic spacing is adopted. The average of the previous two harmonic spacings is used to predict the current harmonic spacing. Then, harmonics are searched within a certain range near the predicted value and marked sequentially until the position of the next harmonic exceeds the spectrum range.

[0079] 4) This invention proposes a vocal style recognition method based on refined time-frequency analysis. Existing methods only utilize traditional feature parameters such as MFCC for recognition, resulting in inaccurate feature descriptions and low recognition accuracy. This invention, through refined analysis in both the time and frequency domains, extracts refined time-frequency features in both domains, effectively enhancing the feature expression of vocal style and exhibiting higher robustness and accuracy. Attached Figure Description

[0080] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0081] Figure 1 This is a flowchart of a voice style recognition method based on time-frequency refined analysis disclosed in an embodiment of the present invention;

[0082] Figure 2 This is a flowchart of the fundamental frequency estimation in an embodiment of the present invention;

[0083] Figure 3 This is a flowchart of the adaptive harmonic marking in an example of the present invention. Detailed Implementation

[0084] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0085] Example 1

[0086] This embodiment discloses a method for recognizing vocal styles based on refined time-frequency analysis. The method involves collecting vocal signals, labeling them, and preprocessing them. After fundamental frequency estimation and harmonic labeling, refined features of the vocal signals in the time and frequency domains are extracted. These refined time-frequency features and traditional features are then used to train a support vector machine model to achieve vocal style recognition. Figure 1 As shown, the specific process is as follows:

[0087] S1, Collect human voice signal S voice And label them, build a human voice style dataset; and analyze S voice Preprocessing is performed to obtain the preprocessed human voice signal S. pvoice ;

[0088] S101, Collecting human voice signals S voice The system categorizes individuals and women into seven types of tags: cute and sweet, young and fashionable, mature and intellectual, soft-spoken, steady and magnetic, dignified and profound, and passionate and energetic. This categorization method can provide a good reference standard and basis for the selection of professionals in broadcasting, film and television and other industries.

[0089] S102. Based on the frequency range of the human voice, process the collected human voice signal S voice After filtering with an FIR high-pass filter to remove low-frequency noise, the human voice signal S is obtained. pvoice The coefficients of the high-pass filter are generally set between 0.9 and 1. In this example, they are set to 0.9375, which improves the quality of the human voice signal and reduces noise interference.

[0090] S2, the preprocessed human voice signal S pvoice The fundamental frequency f is obtained by estimating the short-time vowel portion. The flowchart for fundamental frequency estimation is as follows: Figure 2 As shown, the human voice signal S is then... pvoice The spectrum is obtained by performing a Fast Fourier Transform. Adaptive harmonic labeling is then applied to the spectrum using the fundamental frequency f. The flowchart for adaptive harmonic labeling is shown below. Figure 3 As shown;

[0091] S201, the preprocessed human voice signal S pvoiceThe vowel portion of a signal is determined using the maximum value of its time-domain amplitude; first, the maximum value of the signal amplitude is found, denoted as t. max , see t max The subsequent signal is S pvoice The vowel part is then used for subsequent analysis and processing;

[0092] S202. The fundamental frequency f0 is roughly estimated by performing short-time autocorrelation in the time domain on the above vowel part through frame division.

[0093] Signal processing algorithms deal with stationary signals, so analyzing an entire segment of human voice is meaningless. Since human voice signals can be considered stable over a short period, it's important to analyze the signal S... pvoice To divide the frame, the vibration period of human voice is approximately 5-10ms. A single frame contains multiple periods, resulting in a frame length typically between 20-50ms. Half of this frame length is then used as the frame shift. In this example, the frame length is set to 26ms, and the frame shift is 13ms. The first frame, S, is then extracted. pvoice (m) Perform short-time autocorrelation and record the start position t of the frame. start The frame length is N, a is the number of abscissa points of the autocorrelation function, and the short-time autocorrelation function in the time domain is calculated by formula (1). The autocorrelation function has a maximum value at a = 0. Then, find the index of the maximum value among the peak values ​​other than zero, denoted as maxb. The signal sampling rate is Fs, so the fundamental frequency can be roughly estimated. This is then used to calculate the multiple relationship between the frequency represented by the maximum spectral peak and the fundamental frequency.

[0094] S203. Further refine the fundamental frequency estimation using the narrowband spectral energy method and iterative method to reduce the fundamental frequency estimation error;

[0095] First, the frequency represented by the maximum spectral peak is estimated using the narrowband spectral energy method. For S202, the frequency represented by the peak at position t is calculated. start Furthermore, the spectrum X is obtained by performing a Fast Fourier Transform on a frame of signal with a frame length of N. k k is the subscript of the spectrum, |X k | represents the spectral amplitude corresponding to the subscript k, and k0 is the subscript corresponding to the maximum spectral peak. For human voice signals, the narrow bandwidth is generally taken from three points, that is, one point to the left and one point to the right of the maximum spectral peak. The narrow bandwidth range is [k0-1, k0+1]. NDD is the number of points in the FFT. The spectral interval represents the frequency difference between two points in the spectrum. The frequency expression based on narrowband spectral energy estimation is shown in formula (2). The frequency f represented by the maximum spectral peak estimated by the narrowband spectral energy method is... max The coarse estimate of the fundamental frequency f0 calculated in step S202 is used to calculate the multiple relationship between the maximum spectral peak frequency and the fundamental frequency. The round() function is used for rounding to the nearest integer.

[0096] Use the maximum spectral peak frequency f in the spectrum max Based on the multiple relationship t between the coarse estimate of the fundamental frequency f0 and the fundamental frequency, the fundamental frequency is estimated again. At this point, x is 1;

[0097] Then, an iterative method is applied. It is determined whether the fundamental frequency f1 estimated in the first iteration of the narrowband spectral energy method and the fundamental frequency f0 roughly estimated by autocorrelation satisfy the condition that the error is less than one-thousandth, i.e., |f1-f0| < 0.001f1. If this condition is met, the iteration is terminated, and the fundamental frequency f = f1 is determined. If not, the sampling rate is transformed to adjust the signal S. pvoice The vowel portion is resampled, where the resampled frequency is the current fundamental frequency f. x The number of iterations x is increased by 1, and the fundamental frequency f is estimated again using the narrowband spectral energy method. x This continues until the error between the current fundamental frequency and the previous fundamental frequency is less than one-thousandth or the number of iterations is greater than 50, i.e., |f x -f x-1 |<0.001f x The iteration ends only when ∪x > 50, and the fundamental frequency f = f is determined. x Iteration can reduce fundamental frequency error and make the estimated fundamental frequency closer to the true frequency.

[0098] S204. Using an adaptive method to change the harmonic spacing, a fast Fourier transform is performed in S203 to obtain the spectrum X. k In the middle, mark the amplitude values ​​and subscripts of all harmonics;

[0099] For the first three harmonics, i.e., when the harmonic order i <= 3, the fundamental frequency f is used as the interval between adjacent harmonics to mark the FFT spectrum, and the amplitude value and index of the peak point with the largest amplitude value in a certain range nearby are used as the amplitude value H of that harmonic. i and subscript HI i The left boundary l i and right boundary r i The values ​​of HI are shown in formulas (3) and (4), respectively. i-1 This indicates the position of the previous harmonic in the FFT spectrum. When i = 1, it means that the first harmonic is currently being marked. HI0 is set to 0 to calculate the range of l1 and r1.

[0100] To address the issue of harmonic spacing constantly changing with increasing frequency, an adaptive method for adjusting the harmonic spacing is adopted. Starting from the fourth harmonic (i>3), the harmonic spacing is dynamically changed. The average of the previous two harmonic spacings is used as the predicted value for the current harmonic spacing. Furthermore, the amplitude and index of the peak point with the largest frequency spectrum within a certain range are identified as the amplitude value H of that harmonic. i and subscript HI i The left boundary li and right boundary r i The values ​​are shown in formulas (5) and (6), respectively.

[0101] When no peak value is found within this range, the right boundary range will be expanded to continue searching for harmonics and marking them. Each time, the range will be expanded by one point. However, the expansion of the right boundary range is not unlimited and must also satisfy formula (7):

[0102] If no harmonic is found after expanding the right boundary range, then let HI i =2HI i-1 -HI i-2 H i =0 and then proceed to the next harmonic marker;

[0103] Finally, the amplitude sequence of harmonics [H1, H2, ..., H] is obtained. i and the index sequence [HI1,HI2,…,HI] i ].

[0104] S3, for human voice signal S pvoice A refined analysis is performed in the time domain, and refined time-domain features are extracted respectively;

[0105] S301, Regarding human voice signal S pvoice The fundamental frequency sequence is obtained by performing a single-cycle cross-correlation operation in the time domain.

[0106] First, the fundamental frequency f is estimated based on S203, and the time-domain start position t of the short-time vowel used for frequency estimation is determined. start The period length at this position is determined as follows: For each point, first calculate the period to the right of that position, then pass through that interval. The maximum cross-correlation value between the right and adjacent intervals determines the length L of the next period, from which the fundamental frequency of the next period can be obtained. By calculating sequentially, the period lengths and fundamental frequencies of all the cycles on the right side can be obtained. Then, the period lengths and fundamental frequencies of all the cycles on the left side can be calculated in the same way, and finally the entire signal S can be obtained. pvoice The fundamental frequency sequence is denoted as [f t1 ,f t2 ,…,f tm ], where f t1 f t2 ... f tm These represent the fundamental frequencies of the 1st, 2nd, ..., mth periods, respectively; arranging this fundamental frequency sequence in ascending order yields: f (1) ≤f (2) ≤…≤f (m) .

[0107] S302. Calculate and extract refined time-domain features, including the standard deviation f of the single-period fundamental frequency sequence. std Median f of a single-period fundamental frequency sequence med Mean f of a single-period fundamental frequency sequence mean Distance f between the median and mean of a single-period fundamental frequency sequence dist The calculation methods are shown in formulas (8), (9), (10), and (11); the time-domain refinement features reflect the variation characteristics of the fundamental frequency of the entire signal.

[0108] S4. Perform frequency domain refinement analysis on the harmonic-marked spectrum, extracting refined frequency domain features. Then combine the features extracted in steps S3 and S4 into a time-frequency refined feature vector V. fine ;

[0109] S401. Perform frequency domain refinement analysis on the spectrum diagram after harmonic marking in step S204, and extract the refined frequency domain features, including the amplitude ratio R of the first harmonic to the second harmonic. fir_sec The amplitude ratio R of odd-order harmonics to even-order harmonics odd_even The amplitude ratio R of prime harmonics to non-prime harmonics prim_none Harmonic amplitude ratio R harm_spect The amplitude ratio R of lower harmonics to higher harmonics low_high Harmonic variation rate R dec Harmonic peak-to-valley ratio R peak_valley The calculation methods are shown in formulas (12), (13), (14), (15), (16), (17), and (18). The frequency domain refinement features mainly reflect the harmonic structure and characteristics of human voices, and the harmonic performance of human voices varies with different styles.

[0110] S402. Combine the time-domain refined features extracted in step S302 and the frequency-domain refined features extracted in S401 into a time-frequency refined feature vector V. fine That is

[0111] V fine =[f std ,f med ,f mean ,f dist ,R fir_sec ,R odd_even ,R prim_none ,R harm_spect ,R low_high ,R dec ,R peak_valley ].

[0112] S5. Combine traditional features and time-frequency refined features to form a feature vector V and train a support vector machine model;

[0113] S501. Extract the centroid C of the traditional feature spectrum. spect Mel cepstral coefficients C MFCC Then, the traditional features and the refined time-frequency feature vector V are combined. fine Combined into a feature vector V, that is

[0114] V = [V fine C spect C MFCC ];

[0115] S502. Extract feature vectors V from all human voices in the dataset and train a support vector machine model.

[0116] S6. Use the trained support vector machine model to identify the style of the human voice signal to be tested. First, extract the feature vector V' of the human voice signal to be tested, and input the feature vector V' into the support vector machine model trained in step S502 to complete the style recognition of the human voice signal to be tested. The accuracy of each style and the accuracy of the entire dataset are shown in Table 1 below:

[0117] Table 1. Accuracy of Voice Style Recognition

[0118]

[0119]

[0120] As shown in the table above, the recognition accuracy of the entire dataset is as high as 98.8%, and the recognition accuracy of each style category has achieved good results.

[0121] Example 2

[0122] This embodiment further discloses a method for recognizing vocal styles based on time-frequency refined analysis. It involves acquiring and preprocessing vocal signals, extracting refined features of the vocal signals in the time and frequency domains after fundamental frequency estimation and harmonic labeling, and then inputting these refined features and traditional features into a trained support vector machine model to achieve vocal style recognition. The method includes the following steps:

[0123] S1. Referring to the corresponding steps in Example 1, except that the labeling operation is omitted, the human voice signal S to be tested is collected. voice The preprocessed voice signal S is obtained by further processing. pvoice ;

[0124] S2. Referring to the corresponding steps in Example 1, process the preprocessed human voice signal S pvoice The fundamental frequency f is obtained by estimating the fundamental frequency from the short-time vowel portion, and then the human voice signal S is analyzed. pvoice A fast Fourier transform is performed to obtain a spectrum, and adaptive harmonic marking is applied to the spectrum using the fundamental frequency f.

[0125] S3. Referring to the corresponding steps in Example 1, process the human voice signal S... pvoice A refined analysis is performed in the time domain, and refined time-domain features are extracted respectively; wherein, the refined time-domain features include the standard deviation f of the single-cycle fundamental frequency sequence. std Median f of a single-period fundamental frequency sequence med Mean f of a single-period fundamental frequency sequence mean Distance f between the median and mean of a single-period fundamental frequency sequence dist ;

[0126] S4. Referring to the corresponding steps in Example 1, perform frequency domain refinement analysis on the harmonic-marked spectrum diagram, and extract the frequency domain refinement features respectively; wherein, the frequency domain refinement features include the amplitude ratio R of the first harmonic to the second harmonic. fir_sec The amplitude ratio R of odd-order harmonics to even-order harmonics odd_even The amplitude ratio R of prime harmonics to non-prime harmonics prim_none Harmonic amplitude ratio R harm_spect The amplitude ratio R of lower harmonics to higher harmonics low_high Harmonic variation rate R dec Harmonic peak-to-valley ratio R peak_valley The features extracted in steps S3 and S4 are then combined to form a refined time-frequency feature vector V. fine ;

[0127] S5. Referring to the corresponding steps in Example 1, and combining the traditional characteristic spectrum centroid C... spect Mel cepstral coefficients C MFCC and time-frequency refined feature vector V fine The feature vector V' is input into the trained support vector machine model to obtain the style of the human voice signal to be tested;

[0128] The characteristics of the human voice signal to be tested and their corresponding characteristic values ​​are shown in Table 2 below, where C MFCC It has 13 dimensions, denoted by C. MFCC1～13 The voice style recognition result obtained by the support vector machine model is "stable and magnetic".

[0129] Table 2. Characteristics and corresponding feature values ​​of the human voice signal to be tested in Example 2

[0130] feature <![CDATA[f std ]]> <![CDATA[F mean ]]> <![CDATA[f med ]]> <![CDATA[f dist ]]> <![CDATA[R fir_sec ]]> Eigenvalues 18.149 136.625 130.813 5.812 0.925 feature <![CDATA[R odd_even ]]> <![CDATA[R prim_none ]]> <![CDATA[R harm_spect ]]> <![CDATA[R low_high ]]> <![CDATA[R dec ]]> Eigenvalues 1.300 1.096 0.295 10.082 0.019 feature <![CDATA[R peak_valley ]]> <![CDATA[C spect ]]> <![CDATA[C MFCC1 ]]> <![CDATA[C MFCC2 ]]> <![CDATA[C MFCC3 ]]> Eigenvalues 9.937 775.986 27.509 -2.589 -1.105 feature <![CDATA[C MFCC4 ]]> <![CDATA[C MFCC5 ]]> <![CDATA[C MFCC6 ]]> <![CDATA[C MFCC7 ]]> <![CDATA[C MFCC8 ]]> Eigenvalues -105.358 -0.996 10.613 -18.637 9.299 feature <![CDATA[C MFCC9 ]]> <![CDATA[C MFCC10 ]]> <![CDATA[C MFCC11 ]]> <![CDATA[C MFCC12 ]]> <![CDATA[C MFCC13 ]]> Eigenvalues -21.839 -37.007 -20.156 -5.103 -26.068

[0131] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for recognizing vocal styles based on refined time-frequency analysis, characterized in that, The human voice style recognition method includes: Step S1: Acquire human voice signal S voice And label them, build a human voice style dataset, and analyze S voice Preprocessing is performed to obtain the preprocessed human voice signal S. pvoice ; Step S2: Process the preprocessed human voice signal S pvoice The fundamental frequency f is obtained by estimating the fundamental frequency from the short-time vowel portion, and then the human voice signal S is analyzed. pvoice A fast Fourier transform is performed to obtain a spectrum, and adaptive harmonic marking is applied to the spectrum using the fundamental frequency f. Step S3: Process the human voice signal S pvoice A refined analysis is performed in the time domain to extract refined time-domain features, including the standard deviation f of the single-cycle fundamental frequency sequence. std Median f of a single-period fundamental frequency sequence med Mean f of a single-period fundamental frequency sequence mean Distance f between the median and mean of a single-period fundamental frequency sequence dist ; Step S4: Perform frequency domain refinement analysis on the harmonic-marked spectrum and extract refined frequency domain features, wherein the refined frequency domain features include the amplitude ratio R of the first harmonic to the second harmonic. fir_sec The amplitude ratio R of odd-order harmonics to even-order harmonics odd_even The amplitude ratio R of prime harmonics to non-prime harmonics prim_none Harmonic amplitude ratio R harm_spect The amplitude ratio R of lower harmonics to higher harmonics low_high Harmonic variation rate R dec Harmonic peak-to-valley ratio R peak_valley Then, the time-domain refined features and the frequency-domain refined features are combined into a time-frequency refined feature vector V. fine ; Step S5: Combine traditional features with time-frequency refined feature vector V fine The feature vector V is used to train a support vector machine model, wherein the traditional features include the spectral centroid C. spect Mel cepstral coefficients C MFCC ; Step S6: Use the trained support vector machine model to identify the style of the human voice signal under test.

2. The method for voice style recognition based on time-frequency refined analysis according to claim 1, characterized in that, The process of step S1 is as follows: S101, Collecting human voice signals S voice The tags are categorized into seven types: cute and sweet, young and fashionable, mature and intellectual, soft-spoken, steady and magnetic, grand and profound, and passionate and powerful. S102, regarding the collected human voice signal S voice After filtering with an FIR high-pass filter to remove low-frequency noise, the pre-processed human voice signal S is obtained. pvoice .

3. The method for voice style recognition based on time-frequency refined analysis according to claim 1, characterized in that, The process of step S2 is as follows: S201, For the pre-processed human voice signal S pvoice The vowel portion of a signal is determined using the maximum amplitude of the signal. S202, For the pre-processed human voice signal S pvoice The vowel part is subjected to short-time autocorrelation in the time domain by frame segmentation, and the fundamental frequency is roughly estimated to be f0. S203. The fundamental frequency is precisely estimated using the narrowband spectral energy method and iterative method. First, the signal obtained in step S202 through short-time autocorrelation in the time domain is subjected to a fast Fourier transform to obtain its spectrum. Then, the frequency represented by the maximum spectral peak is estimated using the narrowband spectral energy method. Based on the coarse estimate of the fundamental frequency f0, the relationship between the frequency represented by the maximum spectral peak and the multiples of the fundamental frequency is calculated. ,in The function is used for rounding to the nearest integer. Then, based on the aforementioned multiple relationship t, the fundamental frequency of the x-th iteration is estimated again. At this point, x is 1; Then, using an iterative method, the fundamental frequency estimated in the first iteration by the narrowband spectral energy method is determined. The coarse estimate of the fundamental frequency obtained from the autocorrelation coarse estimation Does it meet the requirement that the error is less than one-thousandth? If the condition is met, the iteration ends and the fundamental frequency is determined. If the conditions are not met, then the human voice signal S is adjusted by sampling rate transformation. pvoice The vowel portion is resampled, where the resampling frequency is the fundamental frequency of the x-th iteration. Multiples of 2 to the power of 2, number of iterations Add 1, and continue to estimate the fundamental frequency of the x-th iteration using the narrowband spectral energy method. This continues until the error between the fundamental frequency of the x-th iteration and the fundamental frequency of the (x-1)-th iteration is less than one-thousandth or the number of iterations... Greater than 50, that is Only after the iteration ends and the fundamental frequency is determined... ; S204. Using an adaptive method to change the harmonic spacing, in step S203, the amplitude values ​​and indices of all harmonics are marked in the spectrum obtained by fast Fourier transform. When the first three harmonics, i.e., the harmonic number i <= 3, the fundamental frequency f is used as the adjacent harmonic spacing to mark the FFT spectrum, and the amplitude value and indices of the peak point of the spectrum with the largest value are found in the nearby specified range. An adaptive harmonic spacing adjustment method is adopted. Starting from the i=4th harmonic (i>3), the harmonic spacing is dynamically changed. The average of the previous two harmonic spacings is used as the predicted value of the current harmonic spacing. The amplitude and index of the peak point with the largest frequency spectrum within a certain range are used as the amplitude and index of the current harmonic. If no peak is found within this range, the right boundary range is expanded to continue searching for and marking harmonics. Each time, the range is expanded by one point, finally obtaining the harmonic amplitude sequence. and subscript sequence ,in, , … These represent the amplitudes of the 1st, 2nd, ..., ith harmonics, respectively. , … These represent the subscripts of the 1st, 2nd, ..., ith harmonics, respectively.

4. The method for voice style recognition based on time-frequency refined analysis according to claim 3, characterized in that, The process of step S3 is as follows: S301, for human voice signal S pvoice Perform a single-cycle cross-correlation operation in the time domain to calculate the entire signal S. pvoice fundamental frequency sequence ,in, , … These represent the fundamental frequencies of the 1st, 2nd, ..., mth periods, respectively. S302. Calculate and extract time-domain refined features, wherein the time-domain refined features include the standard deviation f of the single-period fundamental frequency sequence. std Median f of a single-period fundamental frequency sequence med Mean f of a single-period fundamental frequency sequence mean Distance f between the median and mean of a single-period fundamental frequency sequence dist The calculation method is as follows: ， ， ， 。 5. The method for voice style recognition based on time-frequency refined analysis according to claim 4, characterized in that, The process of step S4 is as follows: S401. Perform frequency domain refinement analysis on the spectrum diagram after harmonic marking in step S204, and extract the frequency domain refinement features, including the amplitude ratio R of the first harmonic to the second harmonic. fir_sec The amplitude ratio R of odd-order harmonics to even-order harmonics odd_even The amplitude ratio R of prime harmonics to non-prime harmonics prim_none Harmonic amplitude ratio R harm_spect The amplitude ratio R of lower harmonics to higher harmonics low_high Harmonic variation rate R dec Harmonic peak-to-valley ratio R peak_valley The calculation method is as follows: ， ， ， ， ， ， ， ， In the formula, This represents the sum of the amplitude values ​​of all odd-order harmonics. This represents the sum of the amplitude values ​​of all even-order harmonics. This represents the sum of the amplitude values ​​of all prime harmonics. This represents the sum of the amplitude values ​​of all non-prime harmonics. This represents the sum of the amplitude values ​​of all harmonics. It represents the sum of the amplitude values ​​of the entire spectrum; lower harmonics refer to the sum of the amplitude values ​​of the first twenty harmonics. Higher harmonics refer to the sum of the amplitude values ​​of harmonics of order 20 and above. , The signal sampling rate, For the number of points in the Fast Fourier Transform, express The amplitude of the subharmonic. and These represent the amplitudes of the left and right troughs, respectively. S402. Combine the time-domain refined features extracted in step S302 and the frequency-domain refined features extracted in S401 into a time-frequency refined feature vector V. fine That is V fine =[f std ,f med ,f mean ,f dist ,R fir_sec ,R odd_even ,R prim_none ,R harm_spect ,R low_high ,R dec ,R peak_valley ]。 6. The method for voice style recognition based on time-frequency refined analysis according to claim 5, characterized in that, The process of step S5 is as follows: S501. Extract traditional features, including the spectral centroid C. spect Mel cepstral coefficients C MFCC Then, the traditional features and the refined time-frequency feature vector V are combined. fine Combined into an eigenvector V; that is, V = [V fine C spect C MFCC ]; S502. Extract feature vectors V from all human voices in the dataset and train a support vector machine model. After the model is trained, input the feature vectors and obtain the output classification through the support vector machine model.

7. The method for voice style recognition based on time-frequency refined analysis according to claim 1, characterized in that, In step S6, the feature vector V' of the human voice signal to be tested is first extracted, and the feature vector V' is input into the trained support vector machine model to complete the style recognition of the human voice signal to be tested.

Images