A speaker recognition method and system based on gender and language

Abstract

The present invention discloses a speaker recognition method and system based on gender and language, belonging to the field of speaker recognition. It includes: obtaining speech data to be recognized, specifically an audio file containing effective speaker audio; obtaining low-noise speech audio by denoising the audio file; extracting the denoised speech audio by SMAC feature to obtain a speech spectrum feature map; inputting the speech spectrum feature map into the ResNet model to obtain a speech feature vector; inputting the speech feature vector into a multi-objective learning model to identify the speaker identity, speaker gender and language information used by the speaker; and obtaining the speaker recognition result corresponding to the speech data to be recognized by weighted fusion of the three recognition task results. The present invention comprehensively utilizes the gender information and language information contained in the speech, effectively improves the robustness of speech recognition, especially in the case of changes in the speaker's voice, the recognition accuracy is high.

Description

Technical Field

[0001] The present invention relates to the field of speaker recognition, and in particular to a gender- and language-based speaker recognition method and system. Background Art

[0002] With the continuous development of artificial intelligence, more and more intelligent identity recognition technologies are being applied in life, including face recognition, fingerprint recognition, and voiceprint recognition, which has emerged in recent years. Voiceprint recognition, also known as speaker recognition, analyzes the content of an audio clip to identify which speaker the audio clip belongs to. Speakers can be used for identity authentication and have attracted widespread attention due to their convenient features.

[0003] In the prior art, most methods for speaker recognition focus on a single factor, namely the recognition of the speaker itself. This method requires the speaker to maintain a similar way of speaking during the two stages of voiceprint registration and voiceprint recognition. When the speaker uses different tones, the recognition accuracy will decrease. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method and system for speech recognition based on gender and language, combining the gender information and language information contained in the voice content to perform speaker recognition, and solve the technical problem that the accuracy of the single-factor recognition method decreases when the language tone changes.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A first object of the present invention is to provide a method for speaker recognition based on gender and language, the method comprising:

[0007] Acquire speech data to be recognized, wherein the speech data is an audio file in wav format containing valid speaker audio;

[0008] The audio file is processed by noise reduction to obtain low-noise speech audio;

[0009] Extract the speech audio through SMAC features to obtain a speech spectrum feature map;

[0010] Input the speech spectrum feature map into the ResNet model to obtain the speech feature vector;

[0011] Input the speech feature vector into the multi-objective learning model to identify the speaker's identity, gender, and language used by the speaker;

[0012] By weighted fusion of the identified speaker identity, speaker gender and language information used by the speaker, the speaker recognition result corresponding to the speech data to be recognized is obtained.

[0013] Furthermore, the multi-objective learning model includes three recognition tasks: speaker identity recognition, speaker gender recognition, and language information recognition used by the speaker, and is composed of N shared layers, three hidden layers, and one fusion layer;

[0014] The N layers of shared layers are connected in sequence, and during the training process, the parameters of the shared layers are affected by the recognition results of the three tasks; the inputs of the three hidden layers are respectively connected to the outputs of the Nth shared layer, and the outputs of the three hidden layers are the recognition results of the speaker identity, the speaker gender, and the language information used by the speaker, respectively. During the training process, the parameters of the hidden layers are only affected by the corresponding recognition tasks;

[0015] The fusion layer is used to fuse the output results of the three recognition tasks. The output result of each recognition task is provided with a trainable weight parameter. The fusion layer uses the weighted results of the three recognition tasks as the final recognition result.

[0016] A second object of the present invention is to provide a gender- and language-based speaker recognition system for implementing the above-mentioned speaker recognition method; the system comprises:

[0017] A sound collection module is used to collect the speaker's voice audio data;

[0018] The audio filtering module is used to filter the collected voice audio data and eliminate noise;

[0019] A speaker recognition module is used to perform speaker recognition on the voice audio data after filtering;

[0020] The recognition result display module is used to visualize the recognition results.

[0021] The beneficial effect of the present invention is that the present invention comprehensively utilizes the gender information and language information contained in the speech, effectively improving the robustness of speech recognition, especially in the case of speaker voice changes, the recognition accuracy is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A flowchart of a gender- and language-based speaker recognition method and system provided by the present invention.

[0023] Figure 2 A schematic diagram of the structure of a speaker recognition framework in an example provided by the present invention.

[0024] Figure 3 A schematic diagram of a speaker recognition system based on speech and text provided by the present invention. DETAILED DESCRIPTION

[0025] The technical framework of the invention is described below in conjunction with the accompanying drawings.

[0026] In the prior art, most methods for speaker recognition focus on a single factor, namely the recognition of the speaker itself. This method requires the speaker to maintain a similar way of speaking during the two stages of voiceprint registration and voiceprint recognition. When the speaker uses different tones, the recognition accuracy will decrease.

[0027] In order to solve the technical problem that most of the existing technologies are based on a single factor, namely the speaker itself, resulting in low robustness of speaker recognition, the embodiments of the present invention provide a gender and language-based speaker recognition method and system.

[0028] The technical solutions provided by various embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0029] A speaker recognition method based on speech and text, as shown in the attached Figure 1 As shown, the method includes:

[0030] Step S101, obtaining data of speech to be recognized.

[0031] The voice data is an audio file in wav format containing effective speaker audio.

[0032] Step S102, the audio file is subjected to noise reduction processing to obtain low-noise speech audio, and the noise-reduced speech audio is subjected to spectrum conversion to obtain a speech spectrum feature graph;

[0033] Step S103, inputting the speech spectrum feature map into the ResNet model to obtain a speech feature vector;

[0034] Step S104, inputting the speech feature vector into a multi-objective learning model to identify the speaker's identity, the speaker's gender, and the language information used by the speaker;

[0035] Regarding step S104, in one example, as shown in the attached Figure 2 As shown, the model framework of the multi-objective learning model includes multiple recognition tasks: speaker recognition (main task), gender recognition and language recognition, and improves the accuracy of speaker recognition by introducing multiple auxiliary recognition factors. In addition, the framework contains a set of shared layers, where the parameters in the shared layers are shared by multiple recognition tasks. In the training process, each recognition task can optimize the parameters of the shared layers under the model. The framework contains multiple task-specific hidden layers. The hidden layers are unique to each recognition task. In the training process, only the results of the corresponding recognition tasks can affect the parameters of the hidden layers.

[0036] Step S105, the speaker recognition result of the speech to be recognized is obtained by weighted fusion of the speaker identity, speaker gender and language information obtained by recognition. In this embodiment, it is achieved by introducing a fusion layer after the three hidden layers of the above-mentioned multi-objective learning model. The fusion layer is used to fuse the output results of the three recognition tasks. The output result of each recognition task is provided with a trainable weight parameter. The fusion layer uses the weighted results of the three recognition tasks as the final recognition result.

[0037] In the process of fusing the three recognition results, the optimal weight coefficient combination is selected. The weight coefficient combination is to test the accuracy of different weight coefficient combinations during model training, and select the group with the highest accuracy as the final weight coefficient. The weight coefficient is affected by the recognition resolution of the recognition task and the relative relationship between the secondary task and the main task.

[0038] In a specific implementation of the present invention, the speech spectrum feature graph is composed of SMAC features of speech, and the SMAC feature extraction method is:

[0039] Process the speech audio through the filter:

[0040]

[0041] X q (ω, t) = X(ω, t)H q (ω)

[0042] q=1,2,...,Q

[0043] Where t represents the time at the tth frame, ω is the independent variable of the spectrum, and X(ω, t) represents the information intensity when the frequency is ω at the tth frame; H q (ω) represents the qth filter, α represents the parameter that controls the bandwidth of the filter, ωq is the center frequency of the qth filter, Q is the number of filters, and X q (ω, t) represents the filtering result of the qth filter.

[0044] Calculate the 0th order central moment and 1st order central moment of the filtering result:

[0045]

[0046] Among them, m represents the order of the center distance, M m (q, t) represents the m-th order central moment of the filtering result;

[0047] The ratio of the 1st order central moment to the 0th order central moment is used as the speech spectrum feature:

[0048]

[0049] Among them, R1 (q, t) represents the qth speech spectrum feature, and Q speech spectrum features constitute a speech spectrum feature graph.

[0050] Different from the general speaker recognition method, the present invention is optimized for speaker recognition with variable pitch, using SMAC features instead of ordinary MFCC features. The feature has the characteristics of robust pitch and can meet the speaker's identity authentication in the scenario of pitch change. A speaker recognition system based on speech and text, as shown in the attached Figure 3 As shown, the system comprises:

[0051] A sound collection module collects audio data of the speaker;

[0052] Audio filtering module, which filters the collected audio and eliminates noise;

[0053] The speaker recognition module performs speaker recognition on the filtered audio. This module includes:

[0054] The audio spectrum conversion module is used to perform spectrum analysis on the input audio, and extract SMAC features from the input original audio to obtain a speech spectrum feature map;

[0055] The spectrum feature extraction module is used to extract the feature vector of the speech spectrum feature map. ; Input the speech spectrum feature map and extract the feature vector through the deep network;

[0056] The multi-objective learning model module includes three recognition tasks: speaker identity recognition, speaker gender recognition and speaker language information recognition, and is composed of N shared layers, three hidden layers and one fusion layer; the N shared layers are connected in sequence, and during the training process, the parameters of the shared layers are affected by the recognition results of the three tasks; the inputs of the three hidden layers are respectively connected to the outputs of the Nth shared layer, and the outputs of the three hidden layers are the speaker identity, speaker gender and speaker language information recognition results, respectively. During the training process, the parameters of the hidden layers are only affected by the corresponding recognition tasks; the fusion layer is used to fuse the output results of the three recognition tasks, and the output result of each recognition task is provided with a trainable weight parameter, and the fusion layer uses the weighted results of the three recognition tasks as the final recognition result.

[0057] The recognition result display module visualizes the recognition results. This module includes:

[0058] The voice prompt module plays the recognized result through voice. If the speaker is not in the registrant list, an alarm sound is played;

[0059] The text display module displays the recognized speaker information in text. If there is a problem during the recognition process, an error will be displayed on this module.

[0060] It should be understood by those skilled in the art that the embodiments of the present invention described above and shown in the accompanying drawings are only examples and do not limit the present invention. The purpose of the present invention has been fully and effectively achieved. The functional and structural principles of the present invention have been demonstrated and explained in the embodiments, and the embodiments of the present invention may be deformed or modified in any way without departing from the principles.

Claims

1. A method for speaker recognition based on gender and language, characterized in that: include: Acquire speech data to be recognized, wherein the speech data is an audio file containing audio of a valid speaker; The audio file is processed by noise reduction to obtain low-noise speech audio; The denoised speech audio is extracted through SMAC features to obtain a speech spectrum feature map; Input the speech spectrum feature map into the ResNet model to obtain the speech feature vector; Input the speech feature vector into the multi-objective learning model to identify the speaker identity, speaker gender and language information used by the speaker; the multi-objective learning model includes three recognition tasks: speaker identity recognition, speaker gender recognition and speaker language information recognition, and is composed of N shared layers and three hidden layers; the N shared layers are connected in sequence, and during the training process, the parameters of the shared layers are affected by the recognition results of the three tasks; the inputs of the three hidden layers are respectively connected to the outputs of the Nth shared layer, and the outputs of the three hidden layers are the recognition results of the speaker identity, speaker gender and language information used by the speaker, and during the training process, the parameters of the hidden layers are only affected by the corresponding recognition tasks; By weighted fusion of the identified speaker identity, speaker gender and language information used by the speaker, the speaker recognition result corresponding to the speech data to be recognized is obtained.

2. The method for speaker recognition based on gender and language according to claim 1, characterized in that: The SMAC feature extraction method is: Process the speech audio through the filter: X q (ω,t)=X(ω,t)H q (oh) q=1,2,…,Q Where t represents the tth moment, ω is the independent variable of the spectrum, and X(ω,t) represents the information intensity at different frequencies at the tth moment; H q (ω) represents the qth filter, α represents the parameter that controls the bandwidth of the filter, and ω q is the center frequency of the qth filter, Q is the number of filters; X q (ω,t) represents the filtering result of the qth filter; Calculate the 0th order central moment and 1st order central moment of the filtering result: M m (q,t)=∫0 π |X q (ω,t)| 2 (ω-ω q ) m dω,m=0,1 Among them, m represents the order of the center distance, M m (q, t) represents the m-th order central moment of the filtering result; The ratio of the 1st order central moment to the 0th order central moment is used as the speech spectrum feature: Among them, R 1 (q, t) represents the qth speech spectrum feature, and Q speech spectrum features constitute a speech spectrum feature graph.

3. The method for speaker recognition based on gender and language according to claim 1, characterized in that: The multi-objective learning model also includes a fusion layer, which is used to fuse the output results of the three recognition tasks. The output result of each recognition task is provided with a trainable weight parameter. The fusion layer uses the weighted results of the three recognition tasks as the final recognition result.

4. A speaker recognition system based on gender and language, characterized in that: For implementing the speaker recognition method according to claim 1, the speaker recognition system comprises: A sound collection module is used to collect the speaker's voice audio data; The audio filtering module is used to filter the collected voice audio data and eliminate noise; A speaker recognition module is used to perform speaker recognition on the voice audio data after filtering; The recognition result display module is used to visualize the recognition results.

5. A gender and language based speaker recognition system according to claim 4, characterized in that: The speaker recognition module comprises: The audio spectrum conversion module is used to extract SMAC features from the speech audio and convert it into a speech spectrum feature map; A spectrum feature extraction module is used to extract feature vectors of speech spectrum feature graphs; The multi-objective learning model module includes three recognition tasks: speaker identity recognition, speaker gender recognition and speaker language information recognition, and is composed of N shared layers, three hidden layers and one fusion layer; the N shared layers are connected in sequence, and during the training process, the parameters of the shared layers are affected by the recognition results of the three tasks; the inputs of the three hidden layers are respectively connected to the outputs of the Nth shared layer, and the outputs of the three hidden layers are the speaker identity, speaker gender and speaker language information recognition results, respectively. During the training process, the parameters of the hidden layers are only affected by the corresponding recognition tasks; the fusion layer is used to fuse the output results of the three recognition tasks, and the output result of each recognition task is provided with a trainable weight parameter, and the fusion layer uses the weighted results of the three recognition tasks as the final recognition result.

6. A gender and language based speaker recognition system according to claim 4, characterized in that: The audio filter module adopts a low-frequency cutting method to cut off all noise signals below the threshold.

7. A gender and language based speaker recognition system according to claim 4, characterized in that: The recognition result display module includes: The voice prompt module plays the recognized result through voice. If the speaker is not in the registrant list, an alarm sound is played; The text display module displays the recognized speaker information in text. If there is a problem during the recognition process, an error will be displayed on this module.

Images