A multi-dialect speech recognition method, system, device and medium
By extracting speech features and label features, using the Transformer model for multi-dial speech recognition, and converting the recognition results into Mandarin text, it solves the shortcomings of traditional speech recognition systems in dialect recognition and achieves higher recognition accuracy and robustness , enhances the naturalness and personalization of human-computer interaction.
Patent Information
- Application Number
- CN202311817602.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-27
- Publication Date
- 2025-01-03
- Estimated Expiration
- 2043-12-27
AI Technical Summary
Traditional speech recognition systems have poor results in processing dialect recognition, resulting in recognition problems for dialect users in applications such as smart assistants and voice navigation.
The multi-dial speech recognition method is adopted to obtain dialect data, extract speech features and label features, and use the Transformer model to identify them, and convert the recognition results into Mandarin text to improve the recognition accuracy and robustness.
Accurate recognition of multiple dialects is achieved, the accuracy and robustness of recognition are improved, misunderstandings and communication barriers are reduced when dialect users interact with machines, and a more natural and personalized human-computer interaction method is provided.
Smart Images
Figure CN118038851B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of speech recognition technology, and more specifically to a multi-dialect speech recognition method, system, device and medium. Background Art
[0002] In order to meet the needs of practical applications, it is of great significance to develop a speech recognition system that can accurately recognize multiple dialects, which can improve people's quality of life and work efficiency. Through this system, not only can the misunderstandings and communication barriers between dialect users and machines be effectively reduced, but also more personalized and customized services can be provided. In human-computer interaction, dialect voice interaction with robots can provide a more natural and friendly way of communication. This way of interaction makes users feel more comfortable and at ease, and it is easier to express their intentions and needs, increasing the interactive experience between users and robots. In addition, the use of dialects can promote cultural exchanges and protect regional culture, making the communication between people and machines more diverse.
[0003] However, traditional speech recognition systems can usually only process standard languages, and have poor recognition effects on dialects. This has led to certain problems for dialect users in applications that use speech recognition technology, such as smart assistants and voice navigation.
[0004] Therefore, how to provide a multi-dialect speech recognition method and system is a problem that those skilled in the art need to solve urgently. Summary of the invention
[0005] In view of this, the present invention provides a multi-dialect speech recognition method, system, device and medium, which can effectively solve the problems of traditional speech recognition systems in processing dialect recognition and improve recognition accuracy and robustness.
[0006] In order to achieve the above object, the present invention adopts the following technical solution:
[0007] In a first aspect, the present invention provides a multi-dialect speech recognition method, comprising:
[0008] Get dialect data;
[0009] Extracting speech feature representation of dialect data and dialect label feature representation;
[0010] The speech feature representation and the dialect label feature representation are used as the input of the multi-dialect speech recognition model, and the dialect speech recognition text corresponding to each dialect is output;
[0011] Convert dialect speech recognition text into Mandarin text.
[0012] Preferably, the specific steps of acquiring the speech feature representation include:
[0013] Preprocess the dialect data;
[0014] Performing frame division and windowing processing on the preprocessed dialect speech signal;
[0015] Perform fast Fourier transform on the windowed frame to obtain a frequency domain signal;
[0016] The frequency domain signal is filtered by a Mel filter bank, and the output energy value is logarithmically operated to obtain a logarithmic energy value sequence;
[0017] Perform discrete cosine transform on the logarithmic energy value sequence to obtain the Mel frequency cepstrum coefficients;
[0018] The first energy feature in the Mel frequency cepstral coefficient is removed to obtain the speech feature representation.
[0019] Preferably, the dialect label feature representation of the dialect data is extracted by a word embedding method.
[0020] Preferably, the multi-dialect speech recognition model is a transformer model;
[0021] The model training phase includes:
[0022] The encoder is responsible for receiving the speech feature representation and the dialect label feature representation, and performing feature extraction and abstract representation through the self-attention mechanism;
[0023] The decoder is responsible for generating the corresponding output;
[0024] The output layer outputs the dialect speech recognition text corresponding to each dialect through the softmax unit.
[0025] Preferably, converting the dialect speech recognition text into Mandarin text specifically includes:
[0026] Comparing the dialect speech recognition text with a preset first threshold;
[0027] If it is less than or equal to the first threshold, converting the dialect speech recognition text into Mandarin text according to a preset rule;
[0028] If it is greater than the first threshold, the dialect speech recognition text is converted into Mandarin text through a machine translation model.
[0029] In a second aspect, the present invention provides a multi-dialect speech recognition system, comprising:
[0030] Data acquisition module: used to obtain dialect data;
[0031] Feature extraction module: used to extract the speech feature representation of dialect data and the dialect label feature representation;
[0032] Multi-dialect speech recognition module: used to take speech feature representation and dialect label feature representation as input of the multi-dialect speech recognition model, and output dialect speech recognition text corresponding to each dialect;
[0033] Language conversion module: converts dialect speech recognition text into Mandarin text.
[0034] Preferably, the feature extraction module obtains the speech feature representation in the following specific processing steps:
[0035] Preprocess the dialect data;
[0036] Performing frame division and windowing processing on the preprocessed dialect speech signal;
[0037] Perform fast Fourier transform on the windowed frame to obtain a frequency domain signal;
[0038] The frequency domain signal is filtered by a Mel filter bank, and the output energy value is logarithmically operated to obtain a logarithmic energy value sequence;
[0039] Perform discrete cosine transform on the logarithmic energy value sequence to obtain the Mel frequency cepstrum coefficients;
[0040] The first energy feature in the Mel frequency cepstral coefficient is removed to obtain the speech feature representation.
[0041] Preferably, the feature extraction module extracts dialect label feature representation of the dialect data through a word embedding method.
[0042] Preferably, the multi-dialect speech recognition model is a transformer model;
[0043] The model training phase includes:
[0044] The encoder is responsible for receiving the speech feature representation and the dialect label feature representation, and performing feature extraction and abstract representation through the self-attention mechanism;
[0045] The decoder is responsible for generating the corresponding output;
[0046] The output layer outputs the dialect speech recognition text corresponding to each dialect through the softmax unit.
[0047] Preferably, the specific processing process of the language conversion module is:
[0048] Comparing the dialect speech recognition text with a preset first threshold;
[0049] If it is less than or equal to the first threshold, converting the dialect speech recognition text into Mandarin text according to a preset rule;
[0050] If it is greater than the first threshold, the dialect speech recognition text is converted into Mandarin text through a machine translation model.
[0051] In a third aspect, the present invention provides a computer device, comprising:
[0052] Memory for storing computer programs;
[0053] A processor is used to implement the steps of the multi-dialect speech recognition method when executing the computer program.
[0054] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the multi-dialect speech recognition method are implemented.
[0055] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a multi-dialect speech recognition method, system, device and medium. By using deep learning-based speech recognition technology combined with multi-dialect speech data sets for training and optimization, accurate recognition of multiple dialects can be achieved, thereby improving recognition accuracy and robustness. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0057] Figure 1 A flow chart of a multi-dialect speech recognition method provided by the present invention.
[0058] Figure 2 A structural block diagram of a multi-dialect speech recognition system provided by the present invention. DETAILED DESCRIPTION
[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0060] The embodiment of the present invention discloses a multi-dialect speech recognition method, such as Figure 1 As shown, including:
[0061] Get dialect data;
[0062] Extracting speech feature representation of dialect data and dialect label feature representation;
[0063] The speech feature representation and the dialect label feature representation are used as the input of the multi-dialect speech recognition model, and the dialect speech recognition text corresponding to each dialect is output;
[0064] Convert dialect speech recognition text into Mandarin text.
[0065] Among them, the following factors should be considered when obtaining dialect data:
[0066] Define the dialect type: First determine the type of dialect you want to collect, as there may be multiple dialects in different regions. Clarifying the dialect type will help determine the scope and objectives of data collection.
[0067] Location selection: Select representative locations to collect dialect speech data. Consider places where dialects are used frequently and where dialects are densely distributed, such as villages, markets, or traditional dialect areas with a large population of dialect speakers.
[0068] Sample selection: Look for diverse samples in the selected locations, including people of different ages, genders, and social backgrounds. This helps collect diverse dialect speech data and improves the generalization ability of dialect recognition.
[0069] Data Recording Device: Choose an appropriate recording device for data recording, which can be a professional voice recording device or a high-quality smartphone. Make sure the recording device has good sound quality and can capture the subtle differences in dialect.
[0070] Data storage and organization: The collected dialect data will be stored and organized. Use data storage equipment and attach corresponding information such as age, gender, dialect type, etc. to each sample.
[0071] Data annotation: Annotate the collected dialect speech data, that is, pair the speech samples with the corresponding text or dialect vocabulary. The accuracy of annotation is very important for training and evaluating the dialect recognition system.
[0072] Data verification and quality control: Check the accuracy and quality of the data and exclude samples that do not meet the requirements. Data verification and quality control can be performed with the assistance of professionals to ensure the validity and usability of the data.
[0073] In this embodiment, the specific steps of acquiring the speech feature representation include:
[0074] Preprocessing: Preprocess the original dialect speech signal, including removing silent segments, noise reduction, and standardization.
[0075] Framing: Split the preprocessed dialect speech signal into frames with a fixed time window length. Usually each frame lasts for 20-40 milliseconds, and there is a certain overlap between adjacent frames.
[0076] Windowing: In order to reduce the impact of discontinuity and spectrum leakage caused by frame edges, a windowing function is applied to each frame. Commonly used window functions include Hanning window and Hamming window.
[0077] Fast Fourier Transform (FFT): Perform a fast Fourier transform on each windowed frame to convert the time domain signal into a frequency domain signal.
[0078] Mel filter bank: Use the Mel filter bank to filter the FFT result. The function of the Mel filter bank is to simulate the human auditory system's perception of audio signals and better adapt to the nonlinear characteristics of human hearing perception of sound.
[0079] Logarithmic operation: Take the logarithm of the energy value output by the Mel filter bank to better adapt to the human ear's perception of sound intensity.
[0080] Cepstral coefficient extraction: Apply discrete cosine transform (DCT) to the logarithmic energy value sequence to obtain Mel-frequency cepstral coefficients. Usually, only the first few cepstral coefficients are retained as feature vectors, which usually contain most of the useful acoustic information.
[0081] Removing energy features: Usually, the first feature in the cepstral coefficients is the energy feature, which does not have much effect on the speech recognition task. Therefore, it is possible to choose to remove this energy feature in the feature vector.
[0082] In this embodiment, for dialect label feature extraction, a word embedding method can be used to represent the dialect label as a low-dimensional vector, and combined with the speech features of the dialect, so as to obtain a more comprehensive and richer feature representation. Common methods include using pre-trained word vector models (such as Word2Vec, GloVe) to obtain word vector representations of dialect labels, or deep learning-based models such as BERT to perform semantic encoding of dialect labels.
[0083] By introducing the word embedding method, we can model from the semantic perspective of dialect labels, better describe dialect characteristics, extract dialect-related semantic information, and help enhance the representation ability of dialect label features and improve the accuracy and robustness of dialect recognition and classification.
[0084] In this embodiment, the multi-dialect speech recognition model is a multi-dialect model based on model parameter sharing, which is a method of modeling a multi-dialect speech model using a Transformer model. This method adopts a multi-task learning framework and utilizes the correlation and commonality between multiple dialect data by sharing the parameters of a deep neural network.
[0085] In this method, the encoder is responsible for receiving the speech feature representation and the dialect label feature representation, and extracting and abstracting the features through the self-attention mechanism, while the decoder is responsible for generating the corresponding output.
[0086] In the Transformer model, the encoder consists of multiple identical encoder layers, each of which consists of a multi-head self-attention mechanism and a feedforward neural network. The self-attention mechanism can help the model capture contextual information in the input sequence, while the feedforward neural network is responsible for further processing feature representation. There are residual connections and layer normalization operations between each layer of the encoder to alleviate gradient disappearance and accelerate the training process.
[0087] When processing the input sequence, the Transformer model also introduces a position encoder to model the position information in the input sequence. The position encoder represents the position information by learning a learnable vector, which is added to the speech feature representation and the dialect label feature representation to obtain the overall input representation.
[0088] When using the Transformer model for dialect sharing, the parameters of the encoder and decoder can be shared between different dialects, thereby reducing the number of model parameters and improving model efficiency. During model optimization, the shared parameters are updated between multiple dialects to make the model better adapt to the speech characteristics and dialect labels of different dialects.
[0089] Specifically, during training, the shared hidden layer parameters are updated based on all dialect data to capture the commonalities and shared feature representations between multiple dialects. Each independent output layer is updated based on the data of its own dialect to enable it to better distinguish and predict the acoustic features of the corresponding dialect.
[0090] The interval-based softmax structure is used to better handle the imbalance problem in multi-dialect acoustic modeling and ensure that the recognition of each dialect has a reasonable weight distribution and adjustment. This structure can help improve the effect and generalization ability of multi-dialect acoustic models.
[0091] This multi-dialect acoustic modeling technology based on model parameter sharing can make full use of the information of multi-dialect data, improve the performance and generalization ability of the model, and adapt to the acoustic characteristics and speech characteristics of different dialects. In practical applications, during the training process, the use of data enhancement techniques, such as speech rate changes and noise addition, can improve the robustness of the model, or build a language model suitable for the dialect, which can further optimize the model effect.
[0092] In this embodiment, the specific conversion method of converting the dialect speech recognition text into Mandarin text includes:
[0093] Comparing the dialect speech recognition text with a preset first threshold, wherein the first threshold is known;
[0094] If the dialect speech recognition text is less than or equal to the first threshold, it means that the dialect is similar to Mandarin pronunciation, such as Northeastern dialect. In this case, rule replacement can be used to write a series of rules for dialect vocabulary and syntactic structure to replace dialect vocabulary and syntactic structure with the corresponding vocabulary and syntactic structure of Mandarin. This requires the design and writing of rules based on the differences and rules between dialects and Mandarin.
[0095] If the dialect speech recognition text is greater than the first threshold, it means that the dialect is quite different from Mandarin pronunciation, such as Cantonese, and a machine translation model is used to translate the dialect speech recognition text into Mandarin text using a machine translation method. You can use a trained dialect to Mandarin translation model, or train a new translation model based on the differences between the dialect and Mandarin.
[0096] The present invention provides a multi-dialect speech recognition system. Figure 2 As shown, including:
[0097] Data acquisition module: used to obtain dialect data;
[0098] Feature extraction module: used to extract the speech feature representation of dialect data and the dialect label feature representation;
[0099] Multi-dialect speech recognition module: used to take speech feature representation and dialect label feature representation as input of the multi-dialect speech recognition model, and output dialect speech recognition text corresponding to each dialect;
[0100] Language conversion module: converts dialect speech recognition text into Mandarin text.
[0101] Among them, the following factors should be considered when obtaining dialect data:
[0102] Define the dialect type: First determine the type of dialect you want to collect, as there may be multiple dialects in different regions. Clarifying the dialect type will help determine the scope and objectives of data collection.
[0103] Location selection: Select representative locations to collect dialect speech data. Consider places where dialects are used frequently and where dialects are densely distributed, such as villages, markets, or traditional dialect areas with a large population of dialect speakers.
[0104] Sample selection: Look for diverse samples in the selected locations, including people of different ages, genders, and social backgrounds. This helps collect diverse dialect speech data and improves the generalization ability of dialect recognition.
[0105] Data Recording Device: Choose an appropriate recording device for data recording, which can be a professional voice recording device or a high-quality smartphone. Make sure the recording device has good sound quality and can capture the subtle differences in dialect.
[0106] Data storage and organization: The collected dialect data will be stored and organized. Use data storage equipment and attach corresponding information such as age, gender, dialect type, etc. to each sample.
[0107] Data annotation: Annotate the collected dialect speech data, that is, pair the speech samples with the corresponding text or dialect vocabulary. The accuracy of annotation is very important for training and evaluating the dialect recognition system.
[0108] Data verification and quality control: Check the accuracy and quality of the data and exclude samples that do not meet the requirements. Data verification and quality control can be performed with the assistance of professionals to ensure the validity and usability of the data.
[0109] In this embodiment, the feature extraction module obtains the speech feature representation in the following specific steps:
[0110] Preprocessing: Preprocess the original dialect speech signal, including removing silent segments, noise reduction, and standardization.
[0111] Framing: Split the preprocessed dialect speech signal into frames with a fixed time window length. Usually each frame lasts for 20-40 milliseconds, and there is a certain overlap between adjacent frames.
[0112] Windowing: In order to reduce the impact of discontinuity and spectrum leakage caused by frame edges, a windowing function is applied to each frame. Commonly used window functions include Hanning window and Hamming window.
[0113] Fast Fourier Transform (FFT): Perform a fast Fourier transform on each windowed frame to convert the time domain signal into a frequency domain signal.
[0114] Mel filter bank: Use the Mel filter bank to filter the FFT result. The function of the Mel filter bank is to simulate the human auditory system's perception of audio signals and better adapt to the nonlinear characteristics of human hearing perception of sound.
[0115] Logarithmic operation: Take the logarithm of the energy value output by the Mel filter bank to better adapt to the human ear's perception of sound intensity.
[0116] Cepstral coefficient extraction: Apply discrete cosine transform (DCT) to the logarithmic energy value sequence to obtain Mel-frequency cepstral coefficients. Usually, only the first few cepstral coefficients are retained as feature vectors, which usually contain most of the useful acoustic information.
[0117] Removing energy features: Usually, the first feature in the cepstral coefficients is the energy feature, which does not have much effect on the speech recognition task. Therefore, it is possible to choose to remove this energy feature in the feature vector.
[0118] In this embodiment, for dialect label feature extraction, a word embedding method can be used to represent the dialect label as a low-dimensional vector, and combined with the speech features of the dialect, so as to obtain a more comprehensive and richer feature representation. Common methods include using pre-trained word vector models (such as Word2Vec, GloVe) to obtain word vector representations of dialect labels, or deep learning-based models such as BERT to perform semantic encoding of dialect labels.
[0119] By introducing the word embedding method, we can model from the semantic perspective of dialect labels, better describe dialect characteristics, extract dialect-related semantic information, and help enhance the representation ability of dialect label features and improve the accuracy and robustness of dialect recognition and classification.
[0120] In this embodiment, the multi-dialect speech recognition model is a multi-dialect model based on model parameter sharing, which is a method of modeling a multi-dialect speech model using a Transformer model. This method adopts a multi-task learning framework and utilizes the correlation and commonality between multiple dialect data by sharing the parameters of a deep neural network.
[0121] In this method, the encoder is responsible for receiving the speech feature representation and the dialect label feature representation, and extracting and abstracting the features through the self-attention mechanism, while the decoder is responsible for generating the corresponding output.
[0122] In the Transformer model, the encoder consists of multiple identical encoder layers, each of which consists of a multi-head self-attention mechanism and a feedforward neural network. The self-attention mechanism can help the model capture contextual information in the input sequence, while the feedforward neural network is responsible for further processing feature representation. There are residual connections and layer normalization operations between each layer of the encoder to alleviate gradient disappearance and accelerate the training process.
[0123] When processing the input sequence, the Transformer model also introduces a position encoder to model the position information in the input sequence. The position encoder represents the position information by learning a learnable vector, which is added to the speech feature representation and the dialect label feature representation to obtain the overall input representation.
[0124] When using the Transformer model for dialect sharing, the parameters of the encoder and decoder can be shared between different dialects, thereby reducing the number of model parameters and improving model efficiency. During model optimization, the shared parameters are updated between multiple dialects to make the model better adapt to the speech characteristics and dialect labels of different dialects.
[0125] Specifically, during training, the shared hidden layer parameters are updated based on all dialect data to capture the commonalities and shared feature representations between multiple dialects. Each independent output layer is updated based on the data of its own dialect to enable it to better distinguish and predict the acoustic features of the corresponding dialect.
[0126] The interval-based softmax structure is used to better handle the imbalance problem in multi-dialect acoustic modeling and ensure that the recognition of each dialect has a reasonable weight distribution and adjustment. This structure can help improve the effect and generalization ability of multi-dialect acoustic models.
[0127] This multi-dialect acoustic modeling technology based on model parameter sharing can make full use of the information of multi-dialect data, improve the performance and generalization ability of the model, and adapt to the acoustic characteristics and speech characteristics of different dialects. In practical applications, the effect of the model can be further optimized by reasonably designing the network structure and training strategy.
[0128] In this embodiment, the specific processing process of the language conversion module is as follows:
[0129] Comparing the dialect speech recognition text with a preset first threshold, wherein the first threshold is known;
[0130] If the dialect speech recognition text is less than or equal to the first threshold, it means that the dialect is similar to Mandarin pronunciation, such as Northeastern dialect. In this case, rule replacement can be used to write a series of rules for dialect vocabulary and syntactic structure to replace dialect vocabulary and syntactic structure with the corresponding vocabulary and syntactic structure of Mandarin. This requires the design and writing of rules based on the differences and rules between dialects and Mandarin.
[0131] If the dialect speech recognition text is greater than the first threshold, it means that the dialect is quite different from Mandarin pronunciation, such as Cantonese, and a machine translation model is used to translate the dialect speech recognition text into Mandarin text using a machine translation method. You can use a trained dialect to Mandarin translation model, or train a new translation model based on the differences between the dialect and Mandarin.
[0132] The present invention provides a computer device, comprising:
[0133] Memory for storing computer programs;
[0134] A processor is used to implement the steps of the above-mentioned multi-dialect speech recognition method when executing a computer program.
[0135] For an introduction to a computer device provided by the present invention, please refer to the above method embodiment, and the present invention will not be described in detail here.
[0136] The computer readable storage medium in the present invention stores a computer program, and when the computer program is executed by a processor, the steps of the multi-dialect speech recognition method described above are implemented. The storage medium includes: a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.
[0137] For an introduction to the computer-readable storage medium provided by the present invention, please refer to the above method embodiment, and the present invention will not be repeated here.
[0138] The present invention can be applied to the following specific scenarios:
[0139] Education: In the field of education, many regions have their own unique dialect systems. The present invention can be used in school education in dialect areas to help students better understand and use standard Mandarin. By converting students' answers expressed in dialect into Mandarin, teachers can more accurately assess students' language proficiency and provide corresponding guidance and counseling.
[0140] Speech translation and cross-cultural communication: In cross-regional and cross-cultural communication, dialects may become a communication barrier. The present invention can be applied to the field of speech translation to convert dialects into Mandarin or other target languages, making it easier for people of different languages to communicate and understand each other. For example, in scenarios such as business meetings, travel guides, and social media, converting dialects into Mandarin in real time through smart devices can greatly promote cross-cultural communication.
[0141] Protection and inheritance of dialect culture: As an important part of regional culture, dialect carries unique history and tradition. The present invention can be applied to the protection and inheritance of dialect culture. By converting dialect speech into Mandarin, it is more convenient to organize, collect and preserve dialect materials, helping researchers and protectors to better record and inherit dialect culture.
[0142] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0143] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A multi-dialect speech recognition method, characterized in that: include: Get dialect data; Extracting speech feature representation of dialect data and dialect label feature representation; The speech feature representation and the dialect label feature representation are used as the input of the multi-dialect speech recognition model, and the dialect speech recognition text corresponding to each dialect is output; Convert dialect speech recognition text into Mandarin text; Extract dialect label feature representation of dialect data through word embedding method; The multi-dialect speech recognition model is a transformer model; The model training phase includes: The encoder is responsible for receiving the speech feature representation and the dialect label feature representation, and performing feature extraction and abstract representation through the self-attention mechanism; The decoder is responsible for generating the corresponding output; The output layer outputs the dialect speech recognition text corresponding to each dialect through the softmax unit; Converting dialect speech recognition text into Mandarin text specifically includes: Comparing the dialect speech recognition text with a preset first threshold; If it is less than or equal to the first threshold, converting the dialect speech recognition text into Mandarin text according to a preset rule; If it is greater than the first threshold, the dialect speech recognition text is converted into Mandarin text through a machine translation model.
2. A multi-dialect speech recognition method according to claim 1, characterized in that: The specific steps of obtaining speech feature representation include: Preprocess the dialect data; Performing frame division and windowing processing on the preprocessed dialect speech signal; Perform fast Fourier transform on the windowed frame to obtain a frequency domain signal; The frequency domain signal is filtered by a Mel filter bank, and the output energy value is logarithmically operated to obtain a logarithmic energy value sequence; Perform discrete cosine transform on the logarithmic energy value sequence to obtain the Mel frequency cepstrum coefficients; The first energy feature in the Mel frequency cepstral coefficient is removed to obtain the speech feature representation.
3. A multi-dialect speech recognition system, characterized in that: include: Data acquisition module: used to obtain dialect data; Feature extraction module: used to extract the speech feature representation of dialect data and the dialect label feature representation; Multi-dialect speech recognition module: used to take speech feature representation and dialect label feature representation as input of the multi-dialect speech recognition model, and output dialect speech recognition text corresponding to each dialect; Language conversion module: converts dialect speech recognition text into Mandarin text; The feature extraction module extracts dialect label feature representation of dialect data through word embedding method; The multi-dialect speech recognition model is a transformer model; The model training phase includes: The encoder is responsible for receiving the speech feature representation and the dialect label feature representation, and performing feature extraction and abstract representation through the self-attention mechanism; The decoder is responsible for generating the corresponding output; The output layer outputs the dialect speech recognition text corresponding to each dialect through the softmax unit; The specific processing process of the language conversion module is as follows: Comparing the dialect speech recognition text with a preset first threshold; If it is less than or equal to the first threshold, converting the dialect speech recognition text into Mandarin text according to a preset rule; If it is greater than the first threshold, the dialect speech recognition text is converted into Mandarin text through a machine translation model.
4. A multi-dialect speech recognition system according to claim 3, characterized in that: The specific processing process of the feature extraction module to obtain speech feature representation is as follows: Preprocess the dialect data; Performing frame division and windowing processing on the preprocessed dialect speech signal; Perform fast Fourier transform on the windowed frame to obtain a frequency domain signal; The frequency domain signal is filtered by a Mel filter bank, and the output energy value is logarithmically operated to obtain a logarithmic energy value sequence; Perform discrete cosine transform on the logarithmic energy value sequence to obtain the Mel frequency cepstrum coefficients; The first energy feature in the Mel frequency cepstral coefficient is removed to obtain the speech feature representation.
5. A computer device, characterized in that: include: Memory for storing computer programs; A processor is used to implement the steps of a multi-dialect speech recognition method as claimed in claim 1 or 2 when executing the computer program.
6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the multi-dialect speech recognition method as claimed in claim 1 or 2 are implemented.