Automatic recognition method of speaker individual attribute based on speech time-frequency fine analysis

Abstract

The application discloses a speaker individual attribute automatic recognition method based on speech time-frequency fine analysis, realizes accurate recognition of speaker individual attributes (gender and age) by fine analysis of time domain and frequency domain characteristics of speech and combination of a classification model based on residual self-attention multilayer perception machine, and comprises the following steps: pre-processing of collected speech signals of speakers; based on the proposed time-frequency fine analysis feature extraction method, calculation of time-frequency characteristics of each frame of the speech signals, then calculation of statistical values such as mean and standard deviation of all frames after combination of the same characteristics as time-frequency fine characteristics; feature weight screening of the time-frequency fine characteristics by variance analysis, and input into the constructed classification model to recognize individual attributes of the speakers. The proposed time-frequency fine analysis can enhance feature expression and improve accuracy, has advantages of easy realization, strong robustness and the like, and can be applied to user portrait recognition, intelligent customer service and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of speaker recognition. Specifically, it relates to a method for automatically identifying individual attributes of a speaker based on fine time-frequency analysis of speech. BACKGROUND

[0002] In recent years, with the rapid development of speaker recognition technology, the automatic identification of individual attributes (gender and age) of a speaker has become a popular research direction and is widely used in customer service, advertisement placement, human-computer interaction, and case investigation. However, the features used in existing gender and age identification are mostly Mel-frequency cepstral coefficients (MFCC) created based on sound formation mechanism and human auditory mechanism. Although MFCC contains a lot of information, the relationship between the speech features of a speaker and individual attributes is complex, and the identification effect is not good by using only MFCC and other features.

[0003] In early studies, the age identification categories were basically broad categories such as children, young adults, middle-aged adults, and old adults. Such broad categories have fewer and relatively simple categories. However, in some practical applications, the degree of classification of age groups is required to be high, such as in case investigation, the degree of subdivision of age affects the speed of locking suspects within a certain range. Traditional classifier models such as decision trees and support vector machines can be used to predict gender and age groups. When the human voice dataset is divided into broad categories, the classification accuracy of such models is good, but when the age groups are subdivided, the performance of the models starts to decline. SUMMARY

[0004] The purpose of the present application is to solve the problem of low recognition accuracy for subdivided age groups in the automatic identification of individual attributes (gender and age) of a speaker, and to provide a method for automatically identifying individual attributes of a speaker based on fine time-frequency analysis of speech. The method preprocesses the collected speech signals of a speaker to reduce interference, uses a fine time-frequency analysis feature extraction method to enhance the expression of features related to gender and age, constructs a classification model based on residual self-attention multilayer perceptron, and inputs the features after variance analysis for feature weight screening into the constructed residual self-attention multilayer perceptron to automatically identify the gender and age of a speaker, effectively improving the recognition accuracy.

[0005] The purpose of the present application can be achieved by adopting the following technical solutions:

[0006] A method for automatically identifying individual attributes of a speaker based on fine time-frequency analysis of speech, the method comprising the following steps:

[0007] S1, pre-processing the collected speech signal of the speaker, including sampling quantization, pre-emphasis, de-noising, frame windowing and endpoint detection, to obtain a pre-processed speech signal; its beneficial effect is that it can reduce the interference of noise and improve the quality of speech, facilitating subsequent analysis;

[0008] S2, using time-frequency refinement analysis, respectively in time domain and frequency domain on the pre-processed speech signal for fine feature extraction; calculate the time-frequency feature parameters of each frame of speech signal, and then calculate the mean, standard deviation, median, maximum and minimum of the combined frames of the same feature as the time-frequency fine feature parameters of the speech signal; the time-frequency fine feature parameters of the speech signal represent the difference of individual attribute features of the speaker; the time-frequency fine feature parameters refer to two types of time domain and frequency domain, which are combined together as time-frequency fine feature parameters; its beneficial effect is that through the joint time domain and frequency domain refinement analysis, more features related to gender and age can be extracted, and the expression of features related to gender and age is enhanced, with higher robustness and accuracy;

[0009] S3, using variance analysis to select feature weights of time-frequency fine feature parameters, and retaining a number of feature parameters with high ranking weights; its beneficial effect is that the features related to gender and age can be expressed as much as possible, and the features are fully and comprehensively expressed; feature selection can retain the most relevant features, thereby improving system robustness and reducing computational complexity;

[0010] S4, constructing a classification model based on residual self-attention multilayer perceptron (Residual Self-Attention MLP, RSA-MLP); the classification model is based on multilayer perceptron, and residual connection and multi-head self-attention module are introduced; and the time-frequency fine feature parameters of the speech signals of multiple speakers are extracted from the pre-established vocal data set by steps S1-S3, and input into the RSA-MLP model for training; the RSA-MLP model combines the characteristics of residual connection, multi-head self-attention mechanism and multilayer perceptron, and can effectively capture long-range dependencies and has strong expression ability, and the performance is better than that of traditional classifiers, which can accurately identify the individual attributes of the speaker;

[0011] S5, real-time collection of speech signals of the speaker to be identified, extraction of time-frequency fine feature parameters by steps S1-S3, input of the screened time-frequency fine feature parameters H' into the classification model based on residual self-attention multilayer perceptron which has been trained, to realize automatic identification of the individual attributes of the speaker, including gender identification of 2 types and age identification of 8 types; its beneficial effect is that the residual and self-attention structure of the model can improve the accuracy of the model, as well as the reliability and robustness of the algorithm.

[0012] Furthermore, the preprocessing operation on the original speech signal in step S1 includes:

[0013] S101, The collected speaker's speech signal Sg is quantized through sampling. o Converted to digital voice signal Sg d It enables more convenient storage, transmission, processing, and analysis;

[0014] S102, Sg is enhanced by pre-weighting. d The high-frequency portion is reduced to mitigate signal attenuation and distortion, resulting in the pre-emphasized speech signal Sg. d_1 ;

[0015] S103, then use spectral subtraction to analyze Sg. d 1 After removing noise interference, the denoised speech signal Sg is obtained. d_2 ;

[0016] S104, then Sg d_2 The signal is converted into a short-time stationary signal by frame segmentation and windowing, resulting in the frame-segmented and windowed speech signal Sg. d_3 ;

[0017] S105. Using the double threshold method to analyze Sg d_3 Perform endpoint detection and extract Sg d_3 The effective portion of the signal is used to obtain the speech signal Sg. d_4 Endpoint detection can reduce interference from invalid speech, improve the quality of speech signals, and facilitate analysis.

[0018] Furthermore, in step S2, the preprocessed speech signal Sg d_4 The operations for extracting time-frequency features through frame-by-frame fine analysis in the time and frequency domains include:

[0019] S201, process the preprocessed speech signal Sg d_4 A detailed analysis is performed in the time domain, and time-domain feature parameters are extracted; for different time-domain feature parameters, Sg is calculated. d_4 The temporal feature parameters of each frame of the signal are extracted and then merged. The extracted temporal feature parameters include the fundamental frequency, short-time energy, energy entropy, and short-time zero-crossing rate. The specific steps are as follows:

[0020] S20101, Extracting Sg d_4 The fundamental frequency of each frame of signal is concatenated, and the fundamental frequencies of all frames are concatenated and marked as f0;

[0021] S20102, Extracting Sg d_4 The short-time energy of each frame of signal is spliced ​​together and marked as f1.

[0022] S20103, extracting Sg d_4 the energy entropy of each frame signal, the specific steps are to calculate the entropy value according to the distribution of the short-time energy of each frame signal obtained in S20102, and the calculation formula is:

[0023]

[0024] wherein E_e represents the energy entropy of each frame signal, N_l represents the number of discrete levels of energy values, pow j is the probability of the jth level, which can be obtained by classifying all sample values; then the energy entropies of all frames are spliced and marked as f2;

[0025] S20104, extracting Sg d_4 the short-time zero-crossing rate of each frame signal, and the short-time zero-crossing rates of all frames are spliced and marked as f3;

[0026] S202, fine analysis in frequency domain to extract frequency domain feature parameters; for different frequency domain feature parameters, calculate the different frequency domain feature parameters of each frame signal Sg d_4 , and respectively combine the frequency domain feature parameters of all frames, wherein the extracted frequency domain feature parameters include spectral contrast, spectral envelope, spectral feature, MFCC feature and pitch information feature (Chroma); the specific steps are as follows:

[0027] S20201, performing short-time Fourier transform on Sg d_4 to obtain the frequency spectrum corresponding to each frame signal, wherein the frequency spectrum of the kth frame of speech signal on the freqth frequency component is represented as X k (freq);

[0028] S20202, fine extraction of spectral contrast; calculate the spectral contrast of each frame signal Sg d_4 , the specific steps are to square the X k (freq) obtained above to obtain the power spectrum X k ′(freq); then convert the power spectrum into a mel spectrum through a mel filter bank; divide the mel spectrum into M subbands, and sort the energy of each frequency component in each subband from small to large, then arrange the energy corresponding to the 95% position as the peak energy of the subband, and arrange the energy corresponding to the 5% position as the valley energy of the subband; calculate the difference between the peak energy and the valley energy of each subband as the energy of the subband; calculate the spectral contrast F contrast of a frame signal by dividing the sum of the energy values of all subbands by the sum of the energy values of the entire mel spectrum; the calculation formula is:

[0029]

[0030] where M represents the number of subbands, b_e m_max , b_e m_min represent the peak energy and valley energy of the mth subband respectively, Mel_F represents the number of frequencies of the mel-frequency spectrum, mel_e mel_f represents the energy value at the mel_fth frequency in the mel-frequency spectrum, F contrast represents the spectral contrast of a frame of signals; the spectral contrasts of all frames are spliced and marked as f4;

[0031] S20203, fine extraction of spectral envelope; the median filter is used for smoothing processing on the spectrum Xx(freq) of the kth frame of speech signals to obtain the spectral envelope, and then the spectral envelopes of all frames are spliced and marked as f5;

[0032] S20204, fine extraction of spectral features; the spectral features mainly include spectral centroid (SC), spectral spread (SS), spectral entropy (SE), spectral flux (SF) and spectral rolloff (SR), wherein the above feature parameters of the kth frame of signals are represented by {SC k , SS k , SE k , SF k , SR k};

[0033] SC k represents the spectral centroid of the kth frame of signals, reflecting the frequency point where the energy is concentrated in the spectrum X k (freq) of each frame of signals; SS k represents the spectral spread of the kth frame of signals, reflecting the distribution width around the spectral centroid in X k (freq); SE k represents the spectral entropy of the kth frame of signals, reflecting the uniformity of the distribution of X k (freq); SF k represents the spectral flux of the kth frame of signals, describing the change of the adjacent frame spectra (X k (freq), X k+1 (freq)); SR k represents the spectral rolloff of the kth frame of signals, reflecting the maximum frequency point corresponding to a certain proportion value ratio of the cumulative energy reaching the total energy in X k (freq), and ratio represents the corresponding proportion value, generally taking 85%; the calculation formulas are respectively:

[0034]

[0035]

[0036]

[0037]

[0038]

[0039] where X k (freq) represents the power spectrum value of the kth frame signal corresponding to the freqth frequency component, freq' represents the frequency value corresponding to the freqth frequency component, and freq_num represents the total number of frequency components of the kth frame signal, i.e., X k (freq) is the frequency dimension value of X

[0040] Sg d_4 is calculated. k {SC k , SS k , SE k , SF k , SR k} of each frame is calculated, and then {SC k , SS k , SE k , SF k , SR d_4} of all frames are spliced respectively and marked as {f6, f7,..., f10}.

[0041] S20205, 13-order MFCC feature parameters of each frame are calculated, marked as MFCC 1～13 , and MFCC 1～13 of all frames are spliced respectively and marked as {f11, f13,..., f23}.

[0042] S20206, tonal information features (Chroma) of Sg d_4 are extracted; the spectrum of each frame signal in Sg d_4 , i.e., X k (freq), is divided into 12 frequency bands, and the sum of the energy of each frequency band is calculated to form a 12-dimensional frequency domain feature parameter, marked as Cr 1～12 , and the standard deviation of Cr 1～12 is also taken as a frequency domain feature parameter, marked as Cr std ; Cr1- 12 ~ Cr std are spliced to form a new 13-dimensional frequency domain feature parameter Cr 1～13 ; and Cr1～13 Concatenate them separately and label them as {f24, f26, ..., f36};

[0043] S203. Combine the time-domain and frequency-domain feature parameters into time-frequency feature parameters, that is, merge the above time-frequency feature parameters into {f0, f1, f2, ..., f36}, and then calculate the mean, standard deviation, median, maximum value, and minimum value of {f0, f1, f2, ..., f36} respectively, as the time-frequency fine feature parameters of the current time-frequency feature; the specific steps are as follows:

[0044] S20301. Calculate the mean, median, standard deviation, 5th percentile, 95th percentile, and relative root mean square deviation of the fundamental frequency f0, labeled as {f01, f02, ..., f06}, as a new parameter H1, i.e., H1 = {f01, f02, ..., f06}. The formula for calculating the relative root mean square deviation of the fundamental frequency f06 is:

[0045]

[0046] Among them, F i Let be the fundamental frequency of the i-th frame. The mean of the fundamental frequency of all frames is Num, and the total number of frames is Num.

[0047] S20302. For each parameter in the remaining time-frequency characteristic parameters {f1, f2, ..., f36} after removing the fundamental frequency f0, calculate its mean, median, standard deviation, maximum, and minimum values. For each parameter, we combine its five statistical values ​​into a new parameter. The new parameter f1′ obtained by combining the five statistical values ​​of parameter f1 is {f11, f12, f13, f14, f15}, and the same applies to other parameters. Then, we combine all the new parameters obtained above to obtain H2, H2 = {f1′, f2′, ..., f36′}, that is, {f11, ..., f15, ..., f361, ..., f365}.

[0048] S20303. Concatenate H1 and H2 to form a 186-dimensional time-frequency fine feature parameter H, H = {H1, H2} = {f01, f02, ..., f06, f11, ..., f15, f21, ..., f361, ..., f365}.

[0049] Furthermore, in step S3, since the new time-frequency feature parameter H after splicing has a high dimension, there may be redundancy. Variance analysis is used to filter the feature weights of H, and the count feature parameters with the highest weight ranking are retained and marked as H′; where count represents the number of feature parameters to be retained, and the value is set in advance.

[0050] Furthermore, in step S4, an RSA-MLP classification model is constructed. This model, based on a multilayer perceptron, introduces residual connections and multi-head self-attention. Speech signals from multiple speakers are selected from the human voice dataset, and time-frequency feature parameters are extracted via steps S1-S3. These parameters are then input into the RSA-MLP classification model to complete training. The construction process of the RSA-MLP classification model is as follows:

[0051] S401. First, a fully connected layer is used to perform a non-linear transformation on the features; the output dimension of the fully connected layer is N x 512, where N is the number of input speech audio files in each training batch, and the activation function is a linear rectified function.

[0052] S402, then through one batch normalization layer and L multi-head self-attention modules, the output of the fully connected layer in S401 is residually connected to the output of the last multi-head self-attention module among the L multi-head self-attention modules. Here, L is specified by the user in advance and defaults to 4; the size of the batch normalization layer is N x 512; and the size of a single multi-head self-attention module is N x 512.

[0053] S403. Then, the output of S402 after residual connection is passed through one batch normalization layer, one feedforward module, and one random deactivation layer. The output of the last multi-head self-attention module in S402 is residually connected with the output of the random deactivation layer. The residual connection improves the nonlinearity of the model. The size of the batch normalization layer is N x 512, and the deactivation probability of the random deactivation layer is 0.1.

[0054] S404. The output after residual connection in S403 is passed through p multilayer perceptron modules for further feature extraction, where P is specified by the user in advance and defaults to 6. In each multilayer perceptron module, the layers sequentially pass through an N x 1024 dimensional fully connected layer with a linear rectified activation function, an N x 1024 dimensional batch normalization layer, a random deactivation layer with a deactivation probability of 0.1, another N x 1024 dimensional fully connected layer with a linear rectified activation function, and another N x 1024 dimensional batch normalization layer. The output of the second fully connected layer is residually connected to the output of the last batch normalization layer to alleviate the gradient vanishing problem caused by excessive network depth.

[0055] S405. The features output from the multilayer perceptron module are normalized using an N x 1024-dimensional batch normalization layer, and finally the prediction results are output using a fully connected layer; where the output dimension is the number of categories to be predicted, and the activation function is a normalized exponential function.

[0056] Further, in step S4, the speech signals of multiple speakers are selected from the human voice dataset, and the time-frequency feature parameters are extracted through steps S1-S3. These parameters are then input into the RSA-MLP model for training. The output of the RSA-MLP model is compared with the real labels, and the error is calculated using the sparse classification cross-entropy loss function. The error is then backpropagated to optimize the RSA-MLP classification model, and finally, training weights are generated. These training weights are used for subsequent recognition tasks.

[0057] Furthermore, in step S5, the speech signal of the speaker to be identified is acquired in real time, and the time-frequency feature parameters are extracted through steps S1-S3. The filtered feature parameters H′ are input into the training-completed classification model based on residual self-attention multilayer perceptron to realize the automatic identification of the speaker's individual attributes. Among them, the gender category is divided into male and female, and the age category is divided into 8 categories, namely: teenagers are no more than 19 years old, 20 years old are 20-29 years old, 30 years old are 30-39 years old, 40 years old are 40-49 years old, 50 years old are 50-59 years old, 60 years old are 60-69 years old, 70 years old are 70-79 years old, and 80 years old are 80-89 years old.

[0058] The present invention has the following advantages and effects compared with the prior art:

[0059] 1) This invention proposes a feature extraction method based on refined time-frequency analysis of speech. Since the relationship between a speaker's speech features and individual attributes is complex, existing methods only utilize traditional feature parameters such as MFCC to identify individual speaker attributes, resulting in inaccurate feature descriptions and low recognition accuracy. This invention, through joint refined analysis in the time and frequency domains, extracts refined time-frequency features in both domains, effectively enhancing the representation of gender and age-related features, and exhibiting higher robustness and accuracy.

[0060] 2) This invention extracts fine time-frequency features from speech signals and then uses analysis of variance to perform feature weighting, removing features that contribute little to the results and retaining the top-ranked feature parameters. Compared with existing technologies, this invention utilizes features that can express gender and age as much as possible. By selecting features, it retains the most relevant features, reduces the impact of redundant features, thereby improving recognition robustness and reducing computational complexity.

[0061] 3) This invention constructs a classification model based on a residual self-attention multilayer perceptron. This model combines the characteristics of residual connections, multi-head self-attention mechanisms, and multilayer perceptrons. Residual connections enhance the model's nonlinearity and alleviate the gradient vanishing problem caused by excessively deep networks. Furthermore, the multi-head self-attention mechanism effectively captures long-range dependencies and possesses strong expressive power, outperforming traditional classifiers and accurately identifying individual speaker attributes. This invention addresses the problem of low accuracy in age group segmentation when using traditional classifier models such as logistic regression and support vector machines for automatic speaker attribute identification, thus improving the system's robustness and accuracy. Attached Figure Description

[0062] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0063] Figure 1 This is a flowchart of the automatic speaker attribute recognition method based on fine time-frequency analysis of speech disclosed in Embodiment 1 of the present invention.

[0064] Figure 2 This is a flowchart of the speech signal preprocessing process in Embodiment 1 of the present invention;

[0065] Figure 3 This is a flowchart of the spectral contrast feature parameter extraction process in Embodiment 1 of the present invention;

[0066] Figure 4 This is a structural diagram of the classification model based on residual self-attention multilayer perceptron constructed in Embodiment 1 of the present invention;

[0067] Figure 5 This is a graph showing the change in training accuracy for the gender recognition task in Embodiment 1 of the present invention;

[0068] Figure 6 This is a loss variation curve for the gender recognition task in Embodiment 1 of the present invention;

[0069] Figure 7 This is a graph showing the change in training accuracy for the age recognition task in Embodiment 2 of the present invention;

[0070] Figure 8 This is a graph showing the loss variation for the age recognition task in Embodiment 2 of the present invention. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] Example 1

[0073] This embodiment discloses an automatic speaker attribute recognition method based on fine time-frequency analysis of speech. The method employs fine time-frequency analysis to extract refined features from the preprocessed speech signal in both the time and frequency domains, calculating feature parameters for each frame. Then, the mean, standard deviation, median, maximum, and minimum values ​​of all frames sharing the same feature are calculated as the fine time-frequency feature parameters of the speech signal. After feature weighting using analysis of variance, the fine time-frequency feature parameters are input into a constructed classification model based on a residual self-attention multilayer perceptron to automatically identify the speaker's gender. Figure 1 As shown, the specific process is as follows:

[0074] S1. Preprocess the acquired speaker's speech signal, including sampling quantization, pre-emphasis, noise reduction, frame segmentation and windowing, and endpoint detection, to obtain the preprocessed speech signal; such as Figure 2 As shown, preprocessing of the original speech includes:

[0075] S101, The collected speaker's speech signal Sg is quantized through sampling. o Converted to digital voice signal Sg d Sampling in this example involves using Sg o Sampling was performed at a sampling rate of 16k, and Sg was obtained after sampling. o This ensures that the voice signal is not distorted while reducing the amount of data computation; in this example, quantization is performed using 16 bits to quantize Sg. o The amplitude value of ′ is mapped to one of a series of discrete digital values, which is then quantized to obtain Sg. d It enables more convenient storage, transmission, processing, and analysis;

[0076] S102, Pre-weighting is achieved by adjusting Sg d Adding a high-pass filter mitigates the impact of the fact that the energy of high-frequency components in a speech signal is generally smaller than that of low-frequency components, thus reducing signal attenuation and distortion, resulting in the pre-emphasized speech signal Sg. d_1 The coefficients of the high-pass filter are generally set between 0.9 and 1; in this example, they are set to 0.9375.

[0077] S103. Since it is difficult to guarantee a completely quiet and interference-free recording environment, background noise is inevitable. Therefore, a frequency-domain-based denoising method, spectral subtraction, is used to remove Sg. d_1 To reduce noise interference and improve the quality of the speech signal, we obtain the denoised speech signal Sg. a_2 ;

[0078] S104, The long Sg is divided into frames and windowed. d_2 The time-varying speech signal is divided into shorter frames, transforming it into a short-time stationary signal, resulting in the framed and windowed speech signal Sg. d_3 The frame length is typically set to 20ms-30ms, and the frame shift is typically set to 30%-50% of the frame length. In this example, the frame length is set to 25ms and the frame shift is set to 10ms, which yields good results. The window length and frame length are kept consistent. A Hamming window is used here, and the window function is applied to each frame to reduce signal leakage between frames, making the speech signal easier to process in terms of time and frequency, which facilitates subsequent detailed time-frequency analysis.

[0079] S105. Finally, the double threshold method is used to test Sg. d_3 Perform endpoint detection and extract Sg d_3 The effective portion of the signal is used to obtain the speech signal Sg. d_4 Sg d_4 It is also a preprocessed speech signal; endpoint detection can reduce interference from invalid signals and improve the quality of the speech signal for subsequent analysis and processing.

[0080] S2. Using a time-frequency refined analysis method, the preprocessed speech signal Sg is analyzed in both the time and frequency domains. d_4 Refined feature extraction is performed; the time-frequency feature parameters of each frame of the speech signal are calculated, and then the mean, standard deviation, median, maximum, and minimum values ​​of all frames with the same feature are calculated as new time-frequency feature parameters of the speech signal; the new time-frequency feature parameters of the speech represent the differences in individual attribute characteristics of the speaker; the time-frequency feature parameters refer to both time-domain and frequency-domain feature parameters, which are combined when merged to form the time-frequency feature parameters; the time-domain fine analysis extracts the fundamental frequency, short-time energy, energy entropy, and short-time zero-crossing rate, while the frequency-domain fine analysis extracts frequency-domain feature parameters including spectral contrast, spectral envelope, spectral features, MFCC features, and pitch information features (Chroma). The combined time-domain and frequency-domain fine analysis enhances the expression of gender-related features, as detailed below:

[0081] S201, Sg d_4 A detailed analysis is performed in the time domain, and time-domain feature parameters are extracted; for different time-domain feature parameters, Sg is calculated. d_4The temporal feature parameters of each frame of the signal are extracted and then merged. The extracted temporal feature parameters include the fundamental frequency, short-time energy, energy entropy, and short-time zero-crossing rate. The specific steps are as follows:

[0082] S20101, Extracting Sg d_4 The fundamental frequency of each frame of signal is concatenated, and the fundamental frequencies of all frames are concatenated and marked as f0;

[0083] S20102, Extracting Sg d_4 The short-time energy of each frame of signal is spliced ​​together and marked as f1.

[0084] S20103, Extracting Sg d_4 The energy entropy of each frame of the signal is calculated by determining the entropy value based on the short-time energy distribution of each frame obtained in S20102. The calculation formula is as follows:

[0085]

[0086] Where E_e represents the energy entropy of each frame of signal, N_l is the number of discretization levels of the energy value, and pow j The probability of the j-th level can be obtained by classifying all sample values ​​into levels; then the energy entropy of all frames is concatenated and labeled as f2;

[0087] S20104, Extracting Sg d_4 The short-time zero-crossing rate of each frame of signal is concatenated and marked as f3.

[0088] S202. Extract frequency domain feature parameters through fine-grained analysis in the frequency domain; calculate the speech signal Sg for different frequency domain feature parameters. d_4 The different frequency domain feature parameters of each frame of the signal are extracted, and the frequency domain feature parameters of all frames are merged. The extracted frequency domain feature parameters include spectral contrast, spectral envelope, spectral features, MFCC features, and tone information features (Chroma). The specific steps are as follows:

[0089] S20201, Sg d_4 Perform a short-time Fourier transform to obtain the spectrum corresponding to each frame of the signal, where the spectrum of the k-th frame of the speech signal corresponding to the freq-th frequency component is represented as Xx(freq);

[0090] S20202, Refined extraction of spectral contrast; Calculation of Sg d_4 The spectral contrast of each frame of the signal is calculated by taking the X obtained above. k The power spectrum X is obtained by squaring (freq). k(freq); The power spectrum is then converted to a Mel spectrum using a Mel filter bank; The Mel spectrum is divided into M sub-bands (M is set to 6 in this example), and the energy of each frequency component in each sub-band is sorted in ascending order. The energy at the 95th percentile is defined as the peak energy of the sub-band, and the energy at the 5th percentile is defined as the valley energy of the sub-band; The difference between the peak energy and valley energy of each sub-band is calculated as the energy of that sub-band; The spectral contrast F of a frame of signal is obtained by dividing the sum of the energy values ​​of all sub-bands by the sum of the energy values ​​of the entire Mel spectrum. contrast The calculation formula is:

[0091]

[0092] Where M represents the number of subbands, b _ e m_max b_e m_min Let denot m_m represent the peak energy and valley energy of the m-th subband, respectively. Let Mel_F represent the number of frequencies in the Mel spectrum, and mel_e represent the peak energy and valley energy of the m-th subband, respectively. mel_f F represents the energy value at the mel_f-th frequency in the Mel spectrum. contrast This represents the spectral contrast of a single frame of signal; the spectral contrasts of all frames are concatenated and labeled as f4.

[0093] S20203, Refined extraction of the spectral envelope; for the spectral X of the k-th frame of the speech signal k (freq) is smoothed using median filtering to obtain the spectral envelope, and then the spectral envelopes of all frames are concatenated and labeled as f5;

[0094] S20204. Refined extraction of spectral features; spectral features mainly include spectral centroid (SC), spectral spread (SS), spectral entropy (SE), spectral flux (SF), and spectral rolloff (SR), where the above feature parameters of the k-th frame signal are respectively represented by {SC}. k SS k SE k SF k SR k}express;

[0095] Among them, SC k This represents the spectral centroid of the k-th frame of the signal, reflecting the spectrum of each frame, i.e., X. k The frequency points where energy is concentrated in (freq); SS k This represents the spectral diffusion of the signal in the k-th frame, reflecting X. kThe distribution width around the centroid of the spectrum in (freq); SE k This represents the spectral entropy of the signal in the k-th frame, reflecting X. k (freq) The uniformity of the distribution; SF k Represents the spectral flux of the signal in the k-th frame, describing the spectrum of adjacent frames (X). k (freq), X k+1 The changes in (freq)); SR k This represents the spectral roll-off of the signal in the k-th frame, reflecting X. k The maximum frequency point corresponding to a certain percentage (ratio) of accumulated energy in (freq) of the total energy is specified. The ratio represents the corresponding percentage, typically taken as 85%, and in this example, it is also 0.85. The calculation formulas are as follows:

[0096]

[0097]

[0098]

[0099]

[0100]

[0101] Where X k '(freq) represents the power spectral density value of the freq-th frequency component of the k-th frame signal, freq' represents the frequency value of the freq-th frequency component, and freq_num represents the total number of frequency components of the k-th frame signal, i.e., X k The frequency dimension value of '(freq);

[0102] Calculate Sg d_4 {SC} of each frame k SS k SE k SF k SR k}, and then all frames {SC k SS k SE k SF k SR k Concatenate them separately and label them as {f6, f7, ..., f10};

[0103] S20205, Calculate Sg d_4 The 13th-order MFCC feature parameters of each frame, labeled as MFCC 1～13 And put all frames' MFCC 1～13Concatenate them separately and label them as {f11, f13, ..., f23};

[0104] S20206, Extracting Sg d_4 Chroma information features; Sg d_4 The spectrum of each frame of the signal, i.e., X k (freq) is divided into 12 frequency bands. For each frequency band, the sum of its energy is calculated, forming a 12-dimensional frequency domain characteristic parameter, denoted as Cr. i～12 and Cr 1～12 The standard deviation is also taken as a frequency domain characteristic parameter, denoted as Cr. std , Cr 1～12 and Cr std The new frequency domain feature parameter Cr is spliced ​​together to form a 13-dimensional feature. 1～13 ; Cr of all frames 1～13 Concatenate them separately and label them as {f24, f26, ..., f36};

[0105] S203. Combine the time-domain and frequency-domain feature parameters into time-frequency feature parameters, that is, merge the above time-frequency feature parameters into {f0, f1, f2, ..., f36}, and then calculate the mean, standard deviation, median, maximum value, and minimum value of {f0, f1, f2, ..., f36} respectively, as the time-frequency fine feature parameters of the current time-frequency feature; the specific steps are as follows:

[0106] S20301. Calculate the mean, median, standard deviation, 5th percentile, 95th percentile, and relative root mean square deviation of the fundamental frequency f0, labeled as {f01, f02, ..., f06}, as a new parameter H1, H1 = {f01, f02, ..., f06}, where the formula for calculating the relative root mean square deviation of the fundamental frequency f06 is:

[0107]

[0108] Among them, F i Let be the fundamental frequency of the i-th frame. The mean of the fundamental frequency of all frames is Num, and the total number of frames is Num.

[0109] S20302. For each parameter in the remaining time-frequency characteristic parameters {f1, f2, ..., f36} after removing the fundamental frequency f0, calculate its mean, median, standard deviation, maximum, and minimum values. For each parameter, we combine its five statistical values ​​into a new parameter. The new parameter f1′ obtained by combining the five statistical values ​​of parameter f1 is {f11, f12, f13, f14, f15}, and the same applies to other parameters. Then, we combine all the new parameters obtained above to obtain H2, H2 = {f1′, f2′, ..., f36′}, that is, {f11, ..., f15, ..., f361, ..., f365}.

[0110] S20303. Concatenate H1 and H2 to form a 186-dimensional time-frequency fine feature parameter H, i.e., H = {H1, H2} = {f01, f02, ..., f06, f11, ..., f15, f21, ..., f361, ..., f365}.

[0111] S3. Since the spliced ​​time-frequency fine feature parameter H has a high dimension, there may be redundancy. Variance analysis is used to filter the feature weights of H and retain the top count feature parameters with the highest weights. In this example, count is set to 88 and the feature parameters are labeled as H′. By retaining the most relevant features, the robustness of the system is improved and the computational complexity is reduced.

[0112] S4. Construct a classification model based on Residual Self-Attention MLP (RSA-MLP). This model is based on MLP, incorporating residual connections and multi-head self-attention modules. Speech signals from multiple speakers are selected from the human voice dataset, and time-frequency feature parameters are extracted through steps S1-S3. These parameters are then input into the RSA-MLP classification model for training. The RSA-MLP classification model combines the characteristics of residual connections, multi-head self-attention mechanisms, and MLP to perform nonlinear transformations on the features, effectively capturing long-range dependencies and possessing strong expressive power. Its performance surpasses traditional classifiers, accurately identifying individual speaker attributes. The structure of the RSA-MLP classification model is as follows: Figure 4 As shown, the specific steps are as follows:

[0113] S401. First, a fully connected (Dense) layer is used to perform a non-linear transformation on the features; where the output dimension of the fully connected layer is N x 512, where N is the number of input speech audios in each training batch, and the activation function is the Rectified Linear Unit (ReLU) function;

[0114] S402. Then, the layer passes through one batch normalization (BatchNorm) layer and L multi-head self-attention (Multi-Head Self-Attention) modules. The output of the fully connected layer in S401 is then residually connected to the output of the last of the L multi-head self-attention modules. Here, L is specified by the user beforehand and defaults to 4; in this example, it is also set to 4. The size of the batch normalization layer is N x 512, and the size of a single multi-head self-attention module is N x 512.

[0115] S403. Then, the output from S402 after residual connection is passed through a batch normalization layer, a feed-forward neural (FFN) module, and a dropout layer. The output of the last of the L multi-head self-attention modules in S402 is residually connected with the output of the dropout layer. The residual connection improves the nonlinearity of the model. The size of the batch normalization layer is N x 512, and the dropout probability of the dropout layer is 0.1.

[0116] S404. The output after residual connection in S403 is passed through P multilayer perceptron (MLP) modules for further feature extraction, where P is specified by the user in advance and defaults to 6, and is also set to 6 in this example. In the multilayer perceptron module, the layers sequentially pass through a fully connected layer of N x 1024 dimensions with a linear rectified activation function, an N x 1024-dimensional batch normalization layer, a random deactivation layer with a deactivation probability of 0.1, another fully connected layer of N x 1024 dimensions with a linear rectified activation function, and an N x 1024-dimensional batch normalization layer. The output of the second fully connected layer is residually connected to the output of the last batch normalization layer to alleviate the gradient vanishing problem caused by the network being too deep.

[0117] S405. The features output from the multilayer perceptron module are normalized using an N x 1024-dimensional batch normalization layer, and finally the prediction results are output using a fully connected layer; where the output dimension is the number of categories to be predicted, and the activation function is the normalization exponent (Softmax) function.

[0118] S406. In each training batch, N audio samples are selected from the human voice dataset. These N audio samples are then processed using the feature extraction methods in S1, S2, and S3 to extract time-frequency features, resulting in N count-dimensional features. These features are then input into the RSA-MLP classification model for training. The output of the RSA-MLP classification model is compared with the true labels. The sparse classification cross-entropy loss function is used to calculate the error, and the error is backpropagated to optimize the model. Finally, a training weight file is generated for subsequent recognition tasks. In this example, the number of input audio samples N in each training batch is set to 256. This invention uses the Common Voice database of the Mozilla speech database and a database established from self-test data to train the RSA-MLP classification model. The training accuracy variation curve is shown below. Figure 5 As shown, with the increase in the number of training epochs, the training accuracy continuously increases and eventually tends to stabilize; the corresponding training loss change curve is shown in Figure 1. Figure 6 As shown, the loss decreases continuously with the increase in the number of training cycles, which is the opposite of the change curve of training accuracy.

[0119] S5. Real-time acquisition of the speaker's speech signal to be identified. Time-frequency feature parameters are extracted through steps S1-S3. The filtered feature parameters H′ are input into a training-ready classification model based on a residual self-attention multilayer perceptron to automatically identify the speaker's individual attributes. The default length of the acquired speech signal is 3 seconds. This primarily involves gender recognition, categorized as male or female. The accuracy for each category and the final overall accuracy are shown in Table 1 below.

[0120] Table 1. Accuracy of Speaker Individual Attribute (Gender) Recognition

[0121] Class Precision Recall F1-score Male 99％ 99％ 99％ Female 99％ 99％ 99％ Accuracy 99％ 99％ 99％ Macro Avg 99％ 99％ 99％ Weighted Avg 99％ 99％ 99％

[0122] As shown in the table above, the F1-score is the harmonic mean of precision and recall, used to comprehensively measure the classification performance of the classifier; the macro average is the average of precision and recall calculated for each class; the weighted average is the weighted average of precision and recall calculated for each class, where the weight of each class is determined based on the number of samples in that class; the achieved accuracy for each class and the final accuracy are both as high as 99%.

[0123] Example 2

[0124] This embodiment further discloses an automatic speaker attribute recognition method based on refined time-frequency analysis of speech. It employs a refined time-frequency analysis method to extract refined time-frequency features from the preprocessed speech signal in both the time and frequency domains, calculating the time-frequency feature parameters for each frame of the speech signal. Then, the mean, standard deviation, median, maximum, and minimum values ​​of all frames merging the same feature are calculated as the refined time-frequency feature parameters of the speech signal. After using variance analysis to filter the feature weights of the refined time-frequency feature parameters, they are input into the constructed classification model based on a residual self-attention multilayer perceptron to automatically identify the speaker's age. The method includes the following steps:

[0125] S1. Referring to the corresponding steps in Example 1, preprocess the collected speaker's speech signal to obtain the preprocessed speech signal.

[0126] S2. Referring to the corresponding steps in Example 1, using the time-frequency refined analysis method, analyze Sg in both the time and frequency domains. d_4 Refined feature extraction is performed; the time-frequency feature parameters of each frame of the speech signal are calculated, and then the mean, standard deviation, median, maximum and minimum values ​​of all frames with the same feature are calculated as the refined time-frequency feature parameters of the speech signal; the refined time-frequency feature parameters of the speech represent the differences in individual attribute characteristics of the speaker; the refined time-frequency feature parameters refer to two types of feature parameters, time domain and frequency domain, which are combined when merged to form the refined time-frequency feature parameters; the refined time-domain analysis extracts the fundamental frequency, short-time energy, energy entropy and short-time zero-crossing rate, and the refined frequency-domain analysis extracts frequency domain feature parameters including spectral contrast, spectral envelope, spectral features, MFCC features and pitch information features (Chroma); through the combined refined analysis of the time domain and frequency domain, the expression of gender-related features is enhanced.

[0127] S3. Referring to the corresponding steps in Example 1, use variance analysis to filter the feature weights of the new time-frequency feature parameters, and retain the count of the feature parameters with the highest weight ranking. In this example, count is set to 88, and the feature parameters are labeled as H′. By retaining the most relevant features, the robustness of the system is improved and the computational complexity is reduced.

[0128] S4. Referring to the corresponding steps in Example 1, a classification model based on Residual Self-Attention MLP (RSA-MLP) is constructed. This model is based on MLP, introduces residual connections and multi-head self-attention modules, and selects speech signals from multiple speakers in the human voice dataset. Time-frequency feature parameters are extracted through steps S1-S3 and input into the RSA-MLP classification model for training. The RSA-MLP model combines the characteristics of residual connections, multi-head self-attention mechanisms, and MLP to perform nonlinear transformations on features, effectively capturing long-range dependencies and possessing strong expressive power. Its performance is superior to traditional classifiers, accurately identifying individual speaker attributes. In this example, the number of input speech audios N in each training batch is set to 128. This invention uses the Common Voice database of the Mozilla speech database and a database established from self-test data to train the RSA-MLP classification model. The training accuracy variation curve is shown below. Figure 7 As shown, the training accuracy increases with the number of training epochs and then gradually stabilizes; the corresponding training loss curve is shown in Figure 1. Figure 8 As shown, the loss decreases continuously with the increase of the number of training cycles, which is the opposite of the change curve of training accuracy.

[0129] S5. Referring to the corresponding steps in Example 1, the speech signal of the speaker to be identified is acquired in real time. Time-frequency feature parameters are extracted through steps S1-S3. The filtered feature parameters H′ are input into the trained classification model based on a residual self-attention multilayer perceptron to automatically identify the speaker's individual attributes. The default length of the acquired speech signal is 3 seconds. The main focus is on age recognition across eight categories: teenagers (no older than 19), 20-29, 30-39, 40-49, 50-59, 60-69, 70-79, and 80-89. The accuracy rates for each category and the final overall accuracy are shown in Table 2 below.

[0130] Table 2. Speaker Individual Attribute (Age) Recognition Accuracy Table

[0131] Class Precision Recall F1-score Adolescents (<= 19 years) 87％ 87％ 87％ Twenty (20-29) 83％ 90％ 86％ Thirty (30-39) 95％ 87％ 91％ Forty (40-49) 93％ 94％ 93％ Fifty (50-59) 96％ 93％ 94％ Sixty (60-69) 98％ 93％ 95％ Seventy (70-79) 100％ 100％ 100％ Eighty (80-89) 98％ 100％ 99％ Accuracy 92％ 92％ 92％ Macro Avg 94％ 93％ 93％ Weighted Avg 92％ 92％ 92％

[0132] As shown in the table above, the F1-score is the harmonic mean of precision and recall, used to comprehensively measure the classification performance of the classifier; the macro average is the average of precision and recall for each class; the weighted average is the weighted average of precision and recall for each class, where the weight of each class is determined by the number of samples in that class; in single-class recognition, the 70-year-old and 80-year-old age groups have higher recognition accuracy than other age groups, and the final recognition accuracy for all age groups is 92%.

[0133] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for automatic speaker individual attribute recognition based on fine time-frequency analysis of speech, characterized in that, The automatic speaker attribute recognition method includes the following steps: S1. Preprocess the collected speaker's speech signal, including sampling quantization, pre-emphasis, noise reduction, frame windowing and endpoint detection, to obtain the preprocessed speech signal; S2. Employing refined time-frequency analysis, refined feature extraction is performed on the preprocessed speech signal in both the time and frequency domains. The time-frequency feature parameters of each frame of the speech signal are calculated, and then the mean, standard deviation, median, maximum, and minimum values ​​of all frames with the same feature are calculated as the refined time-frequency feature parameters of the speech signal. The refined time-frequency feature parameters of the speech signal characterize the differences in individual speaker attribute features. S3. Use analysis of variance to filter the feature weights of the time-frequency fine feature parameters and retain the feature parameters with the highest weight ranking. S4. Construct a classification model based on residual self-attention multilayer perceptron. This classification model is based on multilayer perceptron and introduces residual connection and multi-head self-attention module. Select speech signals of multiple speakers from the pre-established human voice dataset and extract time-frequency fine feature parameters through steps S1-S3. Input the data into the classification model based on residual self-attention multilayer perceptron to complete the training. S5. Real-time acquisition of the speech signal of the speaker to be identified, extraction of time-frequency fine feature parameters through steps S1-S3, inputting the filtered time-frequency fine feature parameters into the training-completed classification model based on residual self-attention multilayer perceptron, to realize automatic identification of individual attributes of the speaker.

2. The method for automatic speaker individual attribute recognition based on fine time-frequency analysis of speech according to claim 1, characterized in that, In step S1, the collected speaker's speech signal Sg is quantized through sampling. o Converted into digital voice signal Sg d ;Sg d Pre-weighting yields Sg d_1 Using spectral subtraction to extract Sg d_1 Denoising yields Sg d_2 ;Sg d_2 Sg is obtained by performing frame segmentation and windowing. d_3 Endpoint detection and Sg extraction using the dual-threshold method d_3 The effective portion is used to obtain the preprocessed speech signal Sg. d_4 .

3. The method for automatic speaker individual attribute recognition based on fine time-frequency analysis of speech according to claim 1, characterized in that, The process of step S2 is as follows: S201, process the preprocessed speech signal Sg d_4 A detailed analysis is performed in the time domain, and time-domain feature parameters are extracted; for different time-domain feature parameters, Sg is calculated. d_4 The temporal feature parameters of each frame of the signal are extracted and then merged. The extracted temporal feature parameters include the fundamental frequency, short-time energy, energy entropy, and short-time zero-crossing rate. The specific steps are as follows: S20101, Extracting Sg d_4 The fundamental frequency of each frame of signal is concatenated, and the fundamental frequencies of all frames are concatenated and marked as f0; S20102, Extracting Sg d_4 The short-time energy of each frame of signal is spliced ​​together and marked as f1. S20103, Extracting Sg d_4 The energy entropy of each frame of signal is calculated by calculating the entropy value based on the short-time energy distribution of each frame of signal obtained in S20102, and then splicing the energy entropy of all frames together and marking it as f2. S20104, Extracting Sg d_4 The short-time zero-crossing rate of each frame of signal is concatenated and marked as f3. S202. Extract frequency domain feature parameters through fine-grained frequency domain analysis; Calculate the speech signal Sg for different frequency domain feature parameters. d_4 The different frequency domain feature parameters of each frame of the signal are extracted, and the frequency domain feature parameters of all frames are merged. The extracted frequency domain feature parameters include spectral contrast, spectral envelope, spectral features, MFCC features, and tone information features. The specific steps are as follows: S20201, Sg d_4 Perform a short-time Fourier transform to obtain the spectrum corresponding to each frame of the signal, where the spectrum of the k-th frame of the speech signal corresponding to the freq-th frequency component is represented as X. k (freq); S20202, Refined extraction of spectral contrast; Calculation of Sg d_4 The spectral contrast of each frame of the signal is calculated by taking the X obtained above. k The power spectrum X is obtained by squaring (freq). k '(freq); The power spectrum is then converted to a Mel spectrum using a Mel filter bank; The Mel spectrum is divided into M sub-bands, and the energy of each frequency component in each sub-band is sorted from smallest to largest. The energy corresponding to the 95th percentile is defined as the peak energy of each sub-band, and the energy corresponding to the 5th percentile is defined as the valley energy of each sub-band; The difference between the peak energy and valley energy of each sub-band is calculated as the sub-band energy; The spectral contrast F of a frame of signal is obtained by dividing the sum of the energy values ​​of all sub-bands by the sum of the energy values ​​of the entire Mel spectrum. contrast The spectral contrast of all frames is stitched together and marked as f4; S20203, Refined extraction of the spectral envelope; for the spectral X of the k-th frame of the speech signal k (freq) is smoothed using median filtering to obtain the spectral envelope, and then the spectral envelopes of all frames are concatenated and labeled as f5; S20204, Refined extraction of spectral features; spectral features include spectral centroid SC, spectral diffusion SS, spectral entropy SE, spectral flux SF, and spectral roll-off SR, wherein the above feature parameters of the k-th frame of the speech signal are respectively represented by {SC}. k ,SS k SE k ,SF k ,SR k } represents; calculate Sg d_4 {SC} of each frame k ,SS k SE k ,SF k ,SR k Then, the feature parameters of all frames are concatenated and labeled as {f6,f7,…,f10}. S20205, Calculate Sg d_4 The 13th-order MFCC feature parameters of each frame, labeled as MFCC 1～13 And put all frames' MFCC 1～13 Perform concatenation separately and label them as {f11,f13,…,f23}; S20206, Extracting Sg d_4 The tonal information features of Sg; d_4 The spectrum of each frame of the signal, i.e., X k (freq) is divided into 12 frequency bands. For each frequency band, the sum of its energy is calculated, forming a 12-dimensional characteristic parameter, denoted as Cr. 1～12 and for Cr 1～12 We take the standard deviation as the new parameter, and the new parameter is Cr. std , Cr 1～12 and Cr std The new 13-dimensional feature parameter Cr is spliced ​​together. 1～13 ; Cr of all frames 1～13 Concatenate them separately and label them as {f24,f26,…,f36}; S203. Combine the time-domain and frequency-domain feature parameters into time-frequency feature parameters, that is, merge the above time-frequency feature parameters into {f0,f1,f2,…,f36}, and then calculate the mean, standard deviation, median, maximum value and minimum value of {f0,f1,f2,…,f36} as the time-frequency fine feature parameters of the current time-frequency feature. The specific steps are as follows: S20301. Calculate the mean, median, standard deviation, 5th percentile, 95th percentile, and relative root mean square deviation of f0, and label them as {f01,f02,…,f06}, respectively, as the new parameter H1, i.e., H1={f01,f02,…,f06}; S20302. For each of the remaining time-frequency characteristic parameters {f1, f2, ..., f36} after removing the fundamental frequency f0, calculate its mean, median, standard deviation, maximum, and minimum statistical values; for each parameter, combine its five statistical values ​​into a new parameter; the new parameter f1 obtained by combining the five statistical values ​​of parameter f1 is... ′ ={f11,f12,f13,f14,f15}, and the other parameters are similar; then combine all the new parameters obtained above to get H2, H2 = {f1 ′ f2′,…,f36′}, that is, {f11,…,f15,…,f361,…,f365}; S20303. Concatenate H1 and H2 to form a 186-dimensional time-frequency fine feature parameter H, H = {H1, H2} = {f01, f02, ..., f06, f11, ..., f15, ..., f361, ..., f365}.

4. The method for automatic speaker individual attribute recognition based on fine time-frequency analysis of speech according to claim 1, characterized in that, In step S3, variance analysis is used to perform feature weight screening on the time-frequency fine feature parameter H, and the top count feature parameters with the highest weight ranking are retained and marked as H′; where count represents the number of feature parameters to be retained, and the value is set in advance.

5. The method for automatic speaker individual attribute recognition based on fine time-frequency analysis of speech according to claim 1, characterized in that, The construction process of the residual self-attention multilayer perceptron classification model is as follows: S401. First, a fully connected layer is used to perform a non-linear transformation on the features; the output dimension of the fully connected layer is N x 512, where N is the number of input speech audio files in each training batch, and the activation function is a linear rectified function. S402, then through one batch normalization layer and L multi-head self-attention modules, and the output of the fully connected layer in S401 is residually connected with the output of the last multi-head self-attention module among the L multi-head self-attention modules, where L is specified by the user in advance and is 4 by default, the size of the batch normalization layer is N x 512, and the size of a single multi-head self-attention module is N x 512; S403. Then, the output after residual connection in S402 is passed through a batch normalization layer, a feedforward module, and a random deactivation layer. The output of the last multi-head self-attention module among the L multi-head self-attention modules in S402 is residually connected with the output of the random deactivation layer. The size of the batch normalization layer is N x 512, and the deactivation probability of the random deactivation layer is 0.

1. S404. The output after residual connection in S403 is processed through P multilayer perceptron modules for feature extraction, where P is specified by the user in advance and defaults to 6. In each multilayer perceptron module, the output is processed sequentially through an N x 1024 dimensional fully connected layer with a linear rectified activation function, an N x 1024 dimensional batch normalization layer, a random deactivation layer with a deactivation probability of 0.1, an N x 1024 dimensional fully connected layer with a linear rectified activation function, and an N x 1024 dimensional batch normalization layer. The output of the second fully connected layer is then residually connected to the output of the last batch normalization layer. S405. The features output from P multilayer perceptron modules are normalized using an N x 1024-dimensional batch normalization layer, and finally the prediction results are output using a fully connected layer; where the output dimension is the number of categories to be predicted, and the activation function is a normalized exponential function.

6. The method for automatic speaker individual attribute recognition based on fine time-frequency analysis of speech according to claim 1, characterized in that, In step S4, speech signals from multiple speakers are selected from the human voice dataset. Time-frequency fine feature parameters are extracted through steps S1-S3 and input into the residual self-attention multilayer perceptron classification model for training. The output of the classification model is compared with the real labels. The error is calculated using the sparse classification cross-entropy loss function, and the error is backpropagated to optimize the classification model. Finally, training weights are generated, which are used for subsequent recognition tasks.

7. The method for automatic speaker individual attribute recognition based on fine time-frequency analysis of speech according to claim 1, characterized in that, In step S5, which automatically identifies the speaker's individual attributes, the individual attributes include gender and age. Gender is categorized into male and female, and age is categorized into eight types: teenagers (no older than 19), 20-29, 30-39, 40-49, 50-59, 60-69, 70-79, and 80-89.

Images