Audio gender recognition method based on LSTM architecture neural network

Abstract

The invention discloses an audio gender recognition method based on an LSTM architecture neural network, and the method comprises the following steps: 1, carrying out the audio preprocessing, and carrying out the sampling rate conversion, noise elimination and voice segment positioning; 2, feature extraction: extracting features from the audio signal, and taking the features as the input of an LSTM network; 3, constructing and training an LSTM (Long Short Term Memory) model; and 4, gender classification: after model training is completed, performing gender classification on a new audio signal by using the constructed model. The invention aims to overcome the limitation of the prior art and improve the accuracy and adaptability of gender recognition, especially in noise environment and cross-age recognition tasks. By introducing an advanced deep learning technology and an innovative feature engineering strategy, the speaker audio gender recognition method is devoted to promote the development of a speaker audio gender recognition technology, so that the speaker audio gender recognition technology can be applied in a wider range of scenes.

Description

Technical Field

[0001] The present invention relates to the field of recognition methods, and in particular to a method for audio gender recognition based on an LSTM architecture neural network. Background Art

[0002] In the past few decades, speech signal processing technology has made significant progress, especially in speaker recognition. Early gender recognition methods mainly rely on simple acoustic features, such as fundamental frequency (i.e. the pitch of the voice), energy distribution, formant frequency, etc. Although these methods can achieve a certain degree of accuracy under ideal conditions, they face many challenges in real-world applications, including: (1) Impact of environmental noise: Background noise, echo, and reverberation in the real environment will seriously interfere with the extraction of acoustic features, resulting in a decrease in recognition rate.

[0003] (2) Limitations due to individual differences: Traditional methods often assume that gender-related acoustic features follow a similar distribution pattern among all speakers. In fact, individual physiological characteristics, accents, speaking speed, etc. will affect the performance of these features, resulting in recognition uncertainty.

[0004] (3) Adaptability across age groups: The acoustic characteristics of children and the elderly are significantly different from those of adults, making it difficult for traditional methods based on fixed thresholds to accurately identify gender across age groups.

[0005] (4) Challenges of voice changing technology: With the development of digital audio editing technology, voice changing has become easier and easier, which brings additional difficulties to gender recognition based on static features. Summary of the invention

[0006] The technical problem to be solved by the present invention is: how to solve the problems of existing recognition methods, such as large influence of environmental noise, many restrictions on individual differences and low adaptability across age groups, and provide a method for audio gender recognition based on LSTM architecture neural network.

[0007] The present invention solves the above technical problems through the following technical solutions, and the present invention comprises the following steps: Step 1: Audio preprocessing, including sampling rate conversion, noise removal and speech segment location.

[0008] Step 2: Feature extraction: extract features from the audio signal and use them as input to the LSTM network; Step 3: LSTM model construction and training; Step 4: Gender classification. After the model training is completed, the constructed model is used to classify the gender of the new audio signal.

[0009] Furthermore, the specific content of the sampling rate conversion is: converting the audio file to a uniform sampling rate; The specific content of noise cancellation is: removing background noise in audio; The specific content of speech segment positioning is: identifying and extracting speech segments in the audio and removing silent or invalid parts.

[0010] Furthermore, the specific process of feature extraction is as follows: Convert the original audio signal into a form that the neural network can process. For gender recognition tasks, commonly used features include: Mel-Spectrogram: The audio signal is analyzed by a Mel-spectrum analyzer to obtain a feature map in the time-frequency domain. Gammatone Cepstral Coefficients (GTCC): reflects the spectral shape of the audio signal and is suitable for the analysis of speech signals; Pitch: The basic frequency of the sound, which plays an important role in gender identification; Harmonic Ratio: Describes the relative strength of the harmonic components in the sound.

[0011] Furthermore, the specific process of constructing and training the LSTM model is as follows: Data preparation, model building, compiling models, training models, model evaluation and testing, model optimization, deployment and application.

[0012] Furthermore, the specific process of the data standard is as follows: Collect audio data: First, the data preprocessed in the audio preprocessing stage is required; Label data: Gender the collected audio data, that is, determine whether each audio segment belongs to a male or female; The specific process of model construction is as follows: Construct the input layer: The input of the model is the feature sequence of the audio signal; Building LSTM layers: After the input layer, one or more LSTM layers can be added. LSTM layers are able to learn long-term dependencies in audio sequences. Constructing a fully connected layer: The output of the LSTM layer is passed to a fully connected layer for further processing and mapping to the decision space of gender classification. The fully connected layer can contain one or more hidden layers and use activation functions such as ReLU. Construct the output layer: The output layer uses the softmax activation function to output the probabilities of the two categories. The softmax function converts the output of the fully connected layer into a probability distribution so that the sum of the probabilities of all categories is 1.

[0013] Furthermore, the specific process of compiling the model is as follows: After the model is built, compile the model to specify the optimizer, loss function, and evaluation metric; Optimizer: used to adjust network parameters to minimize the loss function; Loss function: Since this is a binary classification problem, the cross entropy loss function is usually used; Evaluation metrics: Use accuracy as an indicator to evaluate model performance; The specific process of training the model is as follows: Train the LSTM network using gender-labeled audio data; During the training process, the model calculates the predicted value through forward propagation and adjusts the network parameters through backpropagation to minimize the loss function; Use validation sets to monitor the training process and avoid overfitting; Adjust the learning rate, batch size, and training round parameters as needed; The specific process of model evaluation and testing is as follows: Use an independent test set to evaluate the performance of the model, Calculate the accuracy, recall and precision evaluation indicators of the model; Get an intuitive understanding of the model’s performance through visualization tools such as confusion matrices; The specific process of model optimization Ruixian: Optimize the model according to the evaluation results, including adjusting the network structure, adding regularization terms, and using data enhancement; Use different feature extraction methods or fuse multiple features to improve the recognition accuracy of the model.

[0014] The specific process of deployment and application is as follows: Deploy the trained model to actual applications and process the model in real time or batches according to actual needs to identify gender information in audio.

[0015] Compared with the prior art, the present invention has the following advantages: The method of gender recognition based on audio with LSTM neural network architecture can capture long-term dependencies and long-term memory capabilities: LSTM can effectively capture long-term dependencies in audio signals by introducing memory units and gating mechanisms (including forget gates, input gates, and output gates). This is particularly important for gender recognition, because speech features (such as pitch, timbre, etc.) may persist throughout the audio sequence and affect the final recognition result.

[0016] Compared with traditional RNN (recurrent neural network), LSTM solves the problems of gradient disappearance and gradient explosion through its unique structure, allowing the model to be trained more stably and capture longer dependencies.

[0017] Improve recognition accuracy, feature extraction and selection: LSTM can automatically extract features useful for gender recognition from audio data, and select and filter these features through the gating mechanism, thereby improving recognition accuracy. Strong noise resistance: The gating mechanism of LSTM makes the model robust to noise and outliers in the input data, which can reduce the impact of noise on recognition results to a certain extent.

[0018] Better flexibility and adaptability, multi-layer structure: LSTM can stack multiple LSTM layers to form a deeper network structure. This multi-layer structure enables the model to learn more complex feature representations and improve the model's representation and learning capabilities.

[0019] Adapt to different scenarios: The LSTM model can adapt to audio gender recognition tasks in different scenarios, including audio data in different languages, different accents, and different noise environments.

[0020] Better efficiency and scalability: The gating mechanism in LSTM enables the calculations between each time step to be performed in parallel, improving the efficiency of training and reasoning.

[0021] Strong scalability: With the development of hardware technology and the improvement of computing power, the LSTM model can further expand its scale and processing power to cope with larger-scale data and more complex tasks; Improve the accuracy and adaptability of gender recognition, especially in noisy environments and across age groups. By introducing advanced deep learning techniques and innovative feature engineering strategies, this paper is committed to promoting the development of speaker audio gender recognition technology so that it can be applied in a wider range of scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is the overall structure diagram of the present invention; Figure 2 It is a gender model representation diagram of the present invention; Figure 3 These are the MFCC, delta-MFCC, and delta-delta-MFCC spectra of the present invention. DETAILED DESCRIPTION

[0023] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given, but the protection scope of the present invention is not limited to the following embodiment.

[0024] like Figure 1As shown, this embodiment provides a technical solution: a method for audio gender recognition based on LSTM architecture neural network, and a method for audio gender recognition based on LSTM (long short-term memory) architecture neural network, which mainly involves audio preprocessing, feature extraction, LSTM model construction and training, and gender classification. The following is a detailed explanation of these steps: Audio Preprocessing Audio preprocessing is the first step in the recognition process, which mainly includes the following aspects: Sampling rate conversion: Convert audio files to a uniform sampling rate for subsequent processing.

[0025] Noise Removal: Remove background noise from audio and improve the signal-to-noise ratio.

[0026] Speech segment location: Identify and extract speech segments in audio and remove silent or invalid parts.

[0027] Feature extraction Feature extraction is to convert the original audio signal into a form that the neural network can process. For gender recognition tasks, commonly used features include: Mel-Spectrogram: The audio signal is analyzed by a Mel-spectrum analyzer to obtain a feature map in the time-frequency domain.

[0028] Gammatone Cepstral Coefficients (GTCC): reflects the spectral shape of the audio signal and is suitable for the analysis of speech signals.

[0029] Pitch: The basic frequency of sound, which plays an important role in gender identification.

[0030] Harmonic Ratio: Describes the relative strength of the harmonic components in the sound.

[0031] These features can be extracted from the audio signal and used as input to the LSTM network.

[0032] LSTM model construction and training LSTM network is a special recurrent neural network (RNN) that can capture long-term dependencies in audio signals. LSTM-based gender recognition models usually include the following parts: Input layer: Receives a sequence of features extracted from the audio.

[0033] LSTM layer: It can be a unidirectional or bidirectional LSTM layer, which is used to learn the temporal dependencies in the audio sequence. The bidirectional LSTM layer can consider the previous and next information of the sequence at the same time to improve the recognition accuracy.

[0034] Dense Layer: Converts the output of the LSTM layer into the probability of gender classification.

[0035] Output layer: Uses a softmax activation function to output the probabilities of two classes (usually male and female).

[0036] During the training process, a large amount of gender-labeled audio data is needed to train the LSTM network. By minimizing the loss function (such as cross entropy loss), the network parameters are continuously adjusted to improve the recognition accuracy of the model.

[0037] Gender classification After the model training is completed, it can be used to classify the gender of new audio signals. After preprocessing and feature extraction, the audio signal to be identified is input into the trained LSTM model. The model will output the probabilities of the two categories, and the category with the higher probability will be taken as the final recognition result.

[0038] Precautions Selection of datasets: Gender recognition tasks require a large amount of gender-labeled audio data to train the model. The quality and quantity of the dataset directly affect the recognition performance of the model.

[0039] Adjustment of model parameters: The parameters of the LSTM network (such as the number of hidden layers, number of units, learning rate, etc.) need to be adjusted according to the specific task to achieve the best recognition effect.

[0040] Post-processing: In practical applications, the output of the model can be post-processed as needed, such as threshold setting, decision fusion, etc., to improve the robustness and accuracy of recognition.

[0041] In general, the audio gender recognition method based on LSTM neural network is an effective technical means. It extracts useful features from audio signals and classifies gender through the learning ability of deep learning models. With the continuous development of technology, the recognition accuracy of this method is also constantly improving.

[0042] Data preparation The audio preprocessing part and feature extraction was done using librosa. Starting with the raw audio, we loaded it using librosa and converted the signal to mono with a sampling rate of 44.1K.

[0043] (In this project, we use the Mozilla Common Speech Dataset) Initial audio example:

[0044] After uploading the audio, we did some pre-processing on it, removing background noise and silent segments from the audio.

[0045] This is done in two steps: Apply a threshold method to the entire audio (silence <= 18dB) Applies threshold to leading and trailing parts of audio signal (silence <= 10dB).

[0046] Audio after (1):

[0047] Audio after (2):

[0048] The orange is the harmonic, melodic part of the audio, and the green is the percussive part of the audio. We decided to keep both.

[0049] Audio feature extraction (2-1) We choose to extract 41 features from each audio file (13 MFCCs, 13 delta-MFCCs, 13 delta-delta-MFCCs, 1 pitch estimation, 1 amplitude vector) (2-2) Use librosa to extract pitch and amplitude from the audio samples, then extract the maximum of these values, and then apply a smoothing function (Hann function) on the pitch using a window of 10 milliseconds.

[0050] The pitch of the audio sample:

[0051] The amplitude of the audio sample

[0052] (3) Gender classification: For gender classification, we used a model with 2 LSTM layers with 100 units each, 30% dropout applied to the first layer and 20% dropout applied to the second layer, and an output layer with 2 units using softmax as the activation function.

[0053] We used the categorical cross entropy function as the loss function and trained the model for 10 epochs using the ADAM optimizer to optimize the network weights. After 10 epochs, we achieved a loss score of 0.19, an accuracy of 0.93, and a precision of 0.94 on the validation set.

[0054] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0055] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0056] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A method for audio gender recognition based on LSTM neural network, characterized in that: The following steps are involved: Step 1: Audio preprocessing, including sampling rate conversion, noise elimination and speech segment location; Step 2: Feature extraction: extract features from the audio signal and use them as input to the LSTM network; Step 3: LSTM model construction and training; Step 4: Gender classification. After the model training is completed, the constructed model is used to classify the gender of the new audio signal.

2. The method for audio gender recognition based on LSTM neural network according to claim 1, characterized in that: The specific content of sampling rate conversion is: converting audio files to a uniform sampling rate; The specific content of noise cancellation is: removing background noise in audio; The specific content of speech segment positioning is: identifying and extracting speech segments in the audio and removing silent or invalid parts.

3. The method for audio gender recognition based on LSTM neural network according to claim 1, characterized in that: The specific process of feature extraction is as follows: Convert the original audio signal into a form that the neural network can process. For gender recognition tasks, commonly used features include: Mel spectrum: The audio signal is analyzed by a Mel spectrum analyzer to obtain a feature map in the time-frequency domain; Gammatone cepstral coefficient: reflects the spectral shape of the audio signal and is suitable for the analysis of speech signals; Pitch: The basic frequency of a sound, which plays an important role in gender identification; Harmonic Ratio: Describes the relative strength of the harmonic components in a sound.

4. The method for audio gender recognition based on LSTM neural network according to claim 1, characterized in that: The specific process of LSTM model construction and training is as follows: Data preparation, model building, compiling models, training models, model evaluation and testing, model optimization, deployment and application.

5. The method for audio gender recognition based on LSTM neural network according to claim 5 is characterized in that: The specific process of the data standard is as follows: Collect audio data: First, the data preprocessed in the audio preprocessing stage is required; Label data: Gender the collected audio data, that is, determine whether each audio segment belongs to a male or female; The specific process of model construction is as follows: Construct the input layer: The input of the model is the feature sequence of the audio signal; Building LSTM layers: After the input layer, one or more LSTM layers can be added. LSTM layers are able to learn long-term dependencies in audio sequences. Constructing a fully connected layer: The output of the LSTM layer is passed to a fully connected layer for further processing and mapping to the decision space of gender classification. The fully connected layer can contain one or more hidden layers and use activation functions such as ReLU. Construct the output layer: The output layer uses the softmax activation function to output the probabilities of the two categories. The softmax function converts the output of the fully connected layer into a probability distribution so that the sum of the probabilities of all categories is 1.

6. The method for audio gender recognition based on LSTM neural network according to claim 5, characterized in that: The specific process of compiling the model is as follows: After the model is built, compile the model to specify the optimizer, loss function, and evaluation metric; Optimizer: used to adjust network parameters to minimize the loss function; Loss function: Since this is a binary classification problem, the cross entropy loss function is usually used; Evaluation metrics: Use accuracy as an indicator to evaluate model performance; The specific process of training the model is as follows: Train the LSTM network using gender-labeled audio data; During the training process, the model calculates the predicted value through forward propagation and adjusts the network parameters through backpropagation to minimize the loss function; Use validation sets to monitor the training process and avoid overfitting; Adjust the learning rate, batch size, and training round parameters as needed; The specific process of model evaluation and testing is as follows: Use an independent test set to evaluate the performance of the model, Calculate the accuracy, recall and precision evaluation indicators of the model; Get an intuitive understanding of the model’s performance through visualization tools such as confusion matrices; The specific process of model optimization Ruixian: Optimize the model according to the evaluation results, including adjusting the network structure, adding regularization terms, and using data enhancement; Use different feature extraction methods or fuse multiple features to improve the recognition accuracy of the model; The specific process of deployment and application is as follows: Deploy the trained model to actual applications and process the model in real time or batches according to actual needs to identify gender information in audio.

Images