Long speech dialect recognition method and device based on deep learning algorithm

Abstract

The invention relates to the technical field of speech recognition, in particular to a long speech dialect recognition method and device based on a deep learning algorithm, and aims to realize dialect recognition and conversion of long speech data through an adaptive segmentation strategy and a pre-trained dialect conversion model. The method comprises the following steps: adaptively segmenting input long voice data, calculating continuity parameters and zero cross points of voice signals through Fourier transform, and determining the segment length; performing high-pass filtering, wavelet transformation and translation denoising processing on the segmented data, and extracting feature vectors of the typical dialect voice data; and calculating a dialect label probability through an encoder and an attention mechanism, selecting a dialect type, and performing dialect conversion processing according to a pre-trained dialect conversion model. The device adopts the technologies of feature extraction, feature fusion, probability linear discriminant analysis and the like to output a dialect conversion result. The method can effectively improve the dialect recognition precision and conversion effect of the long voice data.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to a method and device for long speech dialect recognition based on a deep learning algorithm. Background Art

[0002] Traditional speech recognition systems often face great challenges when processing speech in different dialects, mainly reflected in the complex dialect differences in speech data. There are often significant differences in pronunciation habits, intonations, and speaking speeds in different regions and cultural backgrounds, which makes it difficult for a unified speech recognition system to accurately recognize and process across dialects. Dialect speech recognition methods in the prior art usually rely on the extraction and matching of speech features and classification through traditional machine learning methods. However, due to the complexity and diversity of dialects, traditional models often fail to capture the complex features of dialect speech. In addition, when processing long speech data, existing systems often adopt a fixed-length segmentation strategy. This approach shows great limitations when processing dialect speech with a large number of detailed changes, and is prone to information loss or noise interference.

[0003] For example, a Chinese patent with publication number CN118116366A discloses a speech recognition system for different dialects, including a dialect type recognition module, a speech recognition module and a post-processing module; the dialect type recognition module is used to identify the dialect type of the input speech signal and add a dialect type label to the speech signal; the speech recognition module is used to recognize the speech signal of the corresponding dialect and output the recognized language information text; the post-processing module is used to correct the erroneous text in the language information text and output the final recognition result. This invention can recognize continuously input speech signals of different dialects without manually switching the dialect recognition algorithm or model in the middle, and further improve the recognition accuracy of the system through the post-processing module.

[0004] The above patents all have the problem raised by this background technology: when processing long speech data, a fixed-length segmentation strategy is often adopted. This method shows great limitations when processing dialect speech containing a large number of detail changes, and it is easy to cause information loss or noise interference. In order to solve the above problems, this application designs a long speech dialect recognition method and device based on a deep learning algorithm. Summary of the invention

[0005] The technical problem to be solved by the present invention is to address the deficiencies of the prior art and provide a method and device for long speech dialect recognition based on a deep learning algorithm. The method comprises: adaptively segmenting the input long speech data, and calculating the continuity parameters and zero crossing points of the speech signal through Fourier transform to determine the segment length; performing high-pass filtering, wavelet transform and translation denoising on the segmented data to extract the feature vector of typical dialect speech data; calculating the dialect label probability through an encoder and an attention mechanism, selecting the dialect type, and performing dialect conversion processing according to a pre-trained dialect conversion model. The device uses technologies such as feature extraction, feature fusion, and probabilistic linear discriminant analysis to output the dialect conversion result. The present invention can effectively improve the dialect recognition accuracy and conversion effect of long speech data.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A long speech dialect recognition method based on a deep learning algorithm, the method comprising:

[0008] Segment the input long speech data according to the adaptive segmentation strategy to obtain segmented data;

[0009] Preprocessing the segmented data, extracting typical dialect speech data, encoding vectors of the typical dialect speech data, and calculating the dialect label probability corresponding to the encoded vector;

[0010] The dialect type corresponding to the highest dialect label probability is used as the dialect type of the long speech data, and a pre-trained dialect conversion model is selected according to the dialect type;

[0011] The segmented data is processed by a dialect conversion model, and a dialect conversion result is output.

[0012] The adaptive segmentation strategy includes:

[0013] Converting the long speech data into a Fourier spectrum by short-time Fourier transform, and calculating the continuity parameter of the Fourier spectrum;

[0014] Performing spectrum analysis on the Fourier spectrum, counting the number of breakpoints of the long voice data, judging whether the information of subsequent frames belongs to a voice switching point at the breakpoint of the long voice data according to the delay window, and marking the breakpoint as a zero crossing point if it is a voice switching point;

[0015] The segment length of the long speech data is calculated according to the continuity parameter and the zero crossing point, and the segment data is output according to the segment length. The calculation formula of the segment length is:

[0016]

[0017] Among them, l gIndicates the segment length, L p represents the preset segment length, T represents the zero crossing rate of the zero crossing point, F represents the fundamental frequency characteristic of the Fourier spectrum, E represents the short-time maximum energy of the Fourier spectrum, σ k represents the standard deviation of the energy value of the Fourier spectrum, δ represents the range of variation of the segment length, γ represents the parameter for adjusting the rate of change of the segment length, C represents the continuity parameter, and θ represents the continuity threshold.

[0018] The preprocessing of the segmented data comprises:

[0019] Pre-emphasize the segmented data through a high-pass digital filter;

[0020] The pre-emphasized segmented data is translated to change the position of the discontinuous point, and the translated signal is denoised according to the wavelet transform, and the signal is reversely translated to obtain the denoised signal;

[0021] Decomposing the denoised signal according to variational mode decomposition, calculating speech decomposition feature variables, and calculating feature vectors of segmented data according to the speech decomposition feature variables;

[0022] The segmented data corresponding to the largest feature vector is taken as typical dialect speech data.

[0023] The step of encoding the vector of the typical dialect speech data and calculating the dialect label probability corresponding to the encoded vector includes:

[0024] Performing feature extraction on the typical dialect speech data, determining the variance of the maximized feature of the typical dialect speech data according to the extraction result, and calculating the encoding vector;

[0025] The encoding vector is passed through a gated recurrent unit to generate a hidden vector h 1 , another gated recurrent unit calculates in the reverse direction to generate the hidden vector h 2 , concatenate the two hidden vectors and obtain the dialect label through the fully connected layer;

[0026] Calculating the dialect label attention weight through the attention mechanism, reconstructing the dialect label according to the attention weight, and decoding the reconstructed dialect label through the decoder;

[0027] The conditional probability and the posterior probability of each label in the dialect label are calculated, and the conditional probability and the posterior probability are combined through Gaussian distribution as the dialect label probability.

[0028] The processing of the segmented data by using the dialect conversion model includes:

[0029] A feature extraction layer, for spatially representing the target voiceprint information and channel information of the segmented data through full difference space modeling, extracting voiceprint vector features according to a channel compensation algorithm, and extracting musical characteristics of the segmented data according to an acoustic mapping function pre-trained by the dialect conversion model;

[0030] A feature fusion layer, used for fusing the voiceprint vector feature and the musicality feature through cascade fusion to calculate a global fusion feature;

[0031] The dialect recognition layer is used to model the global fusion feature through the probabilistic linear discriminant analysis of the dialect conversion model, obtain the matching speech of the global fusion feature, score the matching speech and the segmented data by log-likelihood ratio, and output the matching speech with the highest score as the dialect conversion result.

[0032] A long speech dialect recognition device based on a deep learning algorithm, the device comprising:

[0033] Adaptive segmentation module, segmentation data processing module, dialect recognition module and dialect conversion module;

[0034] The adaptive segmentation module is configured with an adaptive segmentation strategy, and the adaptive segmentation strategy is used to segment the long voice data into multiple segments of different lengths;

[0035] The segmented data processing module is configured with a data processing strategy, and the data processing strategy is used to pre-process the segmented long voice data and extract typical dialect voice data;

[0036] The dialect recognition module is used to determine the dialect type of the long speech data through a neural network according to the typical dialect speech data;

[0037] The dialect conversion module is used to select a pre-trained dialect conversion model according to the dialect recognition result, and perform dialect conversion processing on the segmented data.

[0038] The adaptive segmentation module comprises:

[0039] A continuity parameter calculation unit, used for converting long speech data into frequency domain by short-time Fourier transform, extracting speech spectrum information, and calculating continuity parameters of the speech spectrum information;

[0040] The breakpoint detection unit is used to analyze the Fourier spectrum, count the breakpoints in the long voice data, and determine whether it is a voice switching point through the delay window technology;

[0041] A segment length calculation unit, used for dynamically calculating the length of each segment according to the continuity parameter, the zero crossing point and the preset segment length;

[0042] The adaptive segmentation strategy includes continuity detection logic, breakpoint detection logic and segmentation logic;

[0043] The continuity detection logic is used to convert the long speech data into a Fourier spectrum by short-time Fourier transform, and calculate the continuity parameter of the Fourier spectrum;

[0044] The breakpoint detection logic is used to perform spectrum analysis on the Fourier spectrum, count the number of breakpoints of the long voice data, and determine whether the information of the subsequent frame belongs to a voice switching point at the breakpoint of the long voice data according to the delay window, and if it is a voice switching point, mark the breakpoint as a zero crossing point;

[0045] The segmentation logic is used to calculate the segmentation length of the long speech data according to the continuity parameter and the zero crossing point, and output the segmentation data according to the segmentation length;

[0046] The continuity detection logic is configured in the continuity parameter calculation unit, the breakpoint detection logic is configured in the breakpoint detection unit, and the segmentation logic is configured in the segmentation length calculation unit.

[0047] The segmented data processing module comprises:

[0048] A pre-emphasis unit is used to perform pre-emphasis processing on the segmented data through a high-pass filter to enhance the high-frequency part of the speech signal;

[0049] A denoising unit is used to denoise the signal by wavelet transform technology and perform reverse translation to restore the original speech;

[0050] A modal decomposition unit, used to decompose the denoised signal by variational modal decomposition, extract each mode in the speech signal, and generate feature variables;

[0051] The data processing strategy includes preprocessing logic and typical speech extraction logic;

[0052] The preprocessing logic is used to perform pre-emphasis processing and denoising processing on the dialect speech;

[0053] The typical speech extraction logic is used to extract different frequency components in the speech signal through variational mode decomposition, calculate the feature vector of the segmented data by decomposing the feature variables, and use the segmented data corresponding to the maximum value of the feature vector as the typical dialect speech data;

[0054] The pre-processing logic is configured in the pre-emphasis unit and the denoising unit, and the typical speech extraction logic is configured in the modal decomposition unit.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] The present invention uses an adaptive segmentation strategy to segment long speech, ensuring that the segment length can be dynamically adjusted in different speech environments. The adaptive segmentation algorithm combines multiple technical means such as short-time Fourier transform, spectrum analysis, zero crossing points and continuity parameters to better capture the changing characteristics of speech signals. This refined segmentation method improves the system's recognition effect on long speech dialects, especially when processing long speech data containing complex speech switching points and discontinuities. The dynamic adjustment of the segment length makes the extraction of dialect speech signals more stable and accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0058] Figure 1 This is a flow chart of a method for long speech dialect recognition based on a deep learning algorithm according to Embodiment 1 of the present invention;

[0059] Figure 2 This is a schematic diagram of the speech segmentation process of Embodiment 1 of the present invention;

[0060] Figure 3 This is a schematic diagram of signal segmentation in Example 1 of the present invention;

[0061] Figure 4 A denoised signal curve and signal waveform characteristics of a certain section in Example 1 of the present invention;

[0062] Figure 5 This is a diagram of a dialect recognition network structure according to Embodiment 1 of the present invention;

[0063] Figure 6 This is a schematic diagram of the dialect conversion network structure of Example 1 of the present invention;

[0064] Figure 7 This is a module schematic diagram of a long speech dialect recognition device based on a deep learning algorithm according to Example 2 of the present invention. DETAILED DESCRIPTION

[0065] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0066] Embodiment 1:

[0067] See also Figure 1 , an embodiment provided by the present invention: a long speech dialect recognition method based on a deep learning algorithm, the specific steps of the method are as follows:

[0068] S1: Adaptively segment the input long speech signal to generate multiple segmented speech segments;

[0069] In this step, long speech data is received as input, and the long speech data is greater than 30 seconds, and the speech data is intelligently segmented using an adaptive segmentation strategy. Adaptive segmentation dynamically segments long speech based on the continuity characteristics and zero crossing points of the speech signal to generate multiple speech segments. During the segmentation process, the relatively stable speech part will extend the segment length, while the part with drastic changes in the speech signal will shorten the segment length to capture important speech transition points. In this way, the integrity and continuity of the speech can be maintained to the maximum extent while reducing redundant data.

[0070] S2: Perform data preprocessing on the segmented speech segments, extract dialect features, and screen out representative dialect speech data;

[0071] In this step, each generated speech segment is further preprocessed, including denoising, speech enhancement, normalization and other steps to ensure that the speech data has good quality and consistency. At this time, the focus will be on dealing with background noise, speech rate changes, inconsistent intonation and other issues in the speech, making the segmented data more suitable for subsequent dialect feature extraction. Preprocessing can also include enhancement of specific frequency bands to improve the ability of dialect recognition to perceive subtle differences.

[0072] S3: vectorize and encode the extracted typical dialect speech data, and calculate the probability distribution of each dialect label based on the encoding vector;

[0073] In this step, the encoded speech feature vector is input into a pre-trained dialect recognition model, and the model generates a probability distribution of dialect labels based on the feature vector. This model may be a deep learning model trained on a large-scale multi-dialect dataset, which can evaluate the matching degree of various dialect labels based on the input feature vector. The device sorts the calculated probabilities of each dialect label and selects the label with the highest probability as the candidate dialect type.

[0074] S4: taking the item with the highest dialect label probability as the recognized dialect type of the long speech data;

[0075] In this step, according to the dialect label probability distribution, the dialect label with the highest probability is selected as the preliminary dialect type judgment of the current long speech data. The device will further optimize this judgment result by combining the historical speech segment information extracted in the previous segmentation process to ensure that the dialect features extracted from the long speech data can be consistently recognized. Finally, the overall dialect type of the long speech data is determined to provide a basis for the subsequent dialect conversion step.

[0076] S5: Selecting a corresponding pre-trained dialect conversion model according to the identified dialect type;

[0077] S6: Use the selected dialect conversion model to process the segmented speech fragments and output the converted dialect result.

[0078] See also Figure 2 , a schematic diagram of the speech segmentation process of an embodiment of the present invention, firstly, a short-time Fourier transform (STFT) is performed on the input long speech data. By converting the speech signal in the time domain into the Fourier spectrum in the frequency domain, the spectral characteristics of the speech signal and its dynamic changes over time can be better captured. Based on the changes in the Fourier spectrum, the continuity parameter of the Fourier spectrum is calculated to judge the stability and changes of the speech signal. In addition, the breakpoints in the long speech data (such as places where the speech pauses or changes) are detected through spectral analysis, and the delay window technology is further used to identify whether the breakpoint is a speech switching point. For the breakpoints identified as switching points, they are marked as zero crossing points. Finally, the segment length is calculated in combination with the continuity parameter and the zero crossing point to generate adaptively segmented speech segments. The specific steps of S1 are as follows:

[0079] S1.1: Convert the long speech data into Fourier spectrum through short-time Fourier transform, divide the signal into small time segments through window function, and perform Fourier transform on each window to obtain its spectral characteristics;

[0080] S1.2: After acquiring the Fourier spectrum, analyze the changes of the spectrum between adjacent frames and calculate the continuity parameter of the Fourier spectrum. The continuity parameter is used to characterize the smoothness of the change of the speech signal in the time domain to detect the speech mutation point;

[0081] S1.3: Perform spectrum analysis on the Fourier spectrum and count the number of breakpoints in the long speech data. A breakpoint is defined as a point where the Fourier spectrum energy suddenly weakens or changes in a certain frequency band. The identification of breakpoints depends on the frequency change of the speech signal. When the Fourier spectrum energy of a certain time frame decreases by more than a preset threshold, it is marked as a breakpoint.

[0082] S1.4: Based on the delay window technology, at the breakpoint of the long voice data, determine whether the subsequent frame information belongs to the voice switching point. If it is a voice switching point, mark the breakpoint as a zero crossing point;

[0083] S1.5: Calculate the segment length of the long speech data according to the continuity parameter and the zero crossing point, and output the segment data according to the segment length;

[0084] The basic frequency characteristics of the speech signal are measured by calculating the ratio of the zero-crossing rate to the fundamental frequency characteristics at the zero-crossing point. When the zero-crossing points are frequent, it means that the signal may be switching quickly and the segment length needs to be shortened.

[0085] The energy fluctuation degree of the frequency band is measured by calculating the ratio of the maximum energy of the Fourier spectrum to the energy standard deviation. If the energy fluctuation of the Fourier spectrum is large, the segment length needs to be shortened in order to capture the signal changes more finely.

[0086] The segment length is smoothly adjusted through the continuity parameter. When the continuity parameter is close to the continuity threshold, it means that the segment length changes smoothly and there is no need to adjust the segment length in a large range. When the continuity parameter is significantly greater than the continuity threshold, the segment length will increase because the speech signal continuity is strong at this time. When the continuity parameter is significantly less than the continuity threshold, the segment length will decrease because the speech signal continuity is strong at this time.

[0087] The calculation formula for segment length is:

[0088]

[0089] Among them, L g Indicates the segment length, L p represents the preset segment length, T represents the zero crossing rate of the zero crossing point, F represents the fundamental frequency characteristic of the Fourier spectrum, E represents the short-time maximum energy of the Fourier spectrum, σ k represents the standard deviation of the energy value of the Fourier spectrum, δ represents the range of variation of the segment length, γ represents the parameter for adjusting the rate of change of the segment length, C represents the continuity parameter, and θ represents the continuity threshold.

[0090] See also Figure 3 , a schematic diagram of signal segmentation according to an embodiment of the present invention;

[0091] Signal waveform (black solid line): shows the overall fluctuation of the speech signal, representing the change of the original amplitude of the long speech data over time.

[0092] Breakpoints (marked with black dotted lines): Breakpoint1 and Breakpoint3 are represented as breakpoints, which are places where the signal energy suddenly changes or decreases, indicating that the speech signal may pause or switch. 1 and 3 in the figure mark two breakpoints, indicating that the signal at these places has changed significantly.

[0093] Zero-crossing points (marked with black dotted lines): Zero-Crossing2 and Zero-Crossing4 are represented as zero-crossing points, indicating the switching points of the signal from positive to negative or from negative to positive. 2 and 4 mark the switching positions of these signals, which can help further determine the continuity of the speech signal.

[0094] The specific steps of S2 are as follows:

[0095] S2.1: When speaking, the high-frequency components of speech will undergo a decay process. Although the human ear can clearly receive the high-frequency information, the machine equipment is not particularly sensitive to the high-frequency components. Therefore, it is necessary to pre-emphasize the dialect speech so that the energy of the high-frequency components of the speech signal can be increased, that is, to highlight the resonance peak of the high-frequency components, and pre-emphasize the segmented data through a high-pass digital filter;

[0096] S2.2: In the real environment, noise is everywhere. In order to suppress the interference of noise on the original dialect speech, the speech is denoised by wavelet transform. Wavelet transform does not rely on the prior experience of noise frequency and dialect frequency, and can effectively remove noise interference. The pre-emphasized segmented data is translated to change the position of the discontinuous point. According to the wavelet transform, the translated signal is denoised, and the signal is reversely translated to obtain the denoised signal.

[0097] S2.3: Decompose the denoised signal according to variational mode decomposition, and calculate the speech decomposition feature variable. Through variational mode decomposition, different frequency components in the speech signal can be extracted to analyze the structural characteristics of the speech in more detail. The calculation formula of the speech decomposition feature variable is:

[0098] X ppy =max[H p (y)]-min[H p (y)]X ppx =max[H p (x)]-min[H p (x)]

[0099]

[0100] Among them, X ppy represents the peak-to-peak value of the normal component of the denoised signal, X ppx represents the peak-to-peak value of the tangential component of the denoised signal, max(·) represents the maximum value function, min(·) represents the minimum value function, K y represents the normal component signal gradient of the denoised signal, K x represents the signal gradient of the tangential component of the denoised signal, H p (y) represents the normal component signal of the denoised signal, H p (x) represents the tangential component signal of the denoised signal, M represents the abnormal point of the denoised signal, H p(·) (y) represents the abnormal point signal of the normal component of the denoised signal, H p(·) (x) represents the abnormal point signal of the tangential component of the denoised signal, M+1 represents the sampling point of the denoised signal, and L y(·) Indicates the abnormal point length of the normal component of the denoised signal, L x(·)Indicates the length of abnormal points of the tangential component of the denoised signal;

[0101] See also Figure 4 , a certain section of the denoised signal curve and signal waveform characteristics of an embodiment of the present invention, the normal gradient Ky is a derived feature in the denoised signal, representing the signal change of the speech signal in the direction perpendicular to the speech detection direction, reflecting the audio changes in the speech signal caused by accent, intonation or vocal cord vibration. The larger the peak-to-peak value, the more drastic the amplitude change in the speech signal, which can be used to identify the strong and weak parts of the speech, help extract the emotional characteristics of the speech or distinguish the pitch change. The tangential gradient Kx actually represents the amplitude change of the speech signal along the tangent (usually the time axis). This feature can describe the rapid changes in the speech signal, such as at the speech switching point, pause, or the transition between syllables in the speech. The area with a large peak-to-peak value is usually the transition part of the speech or the high-frequency component of the speech signal. The peak-to-peak value X of the normal component signal of the denoised signal is ppy , and the peak-to-peak value of the tangential component of the denoised signal X ppx It is used to describe the rate of change of the amplitude of the speech signal on the time axis. It can reflect fast switching and high-frequency components in speech, such as syllable transition points in continuous speech. A large tangential gradient means that there are significant changes in the speech signal, such as a sudden pronunciation after a pause or a rapid tone switching during pronunciation.

[0102] The feature vector of the segmented data is calculated according to the speech decomposition feature variable. The feature vector is composed of the energy, frequency and phase features of each mode. By constructing the feature vector, the complex speech signal can be represented as a key feature, which is convenient for subsequent pattern recognition and classification operations. The calculation formula of the feature vector of the segmented data is:

[0103]

[0104] Among them, P FV represents the feature vector of the segmented data, ln represents the natural logarithm, f represents the average frequency of the segmented data, FL(·) represents the Fourier transform function, x represents the intermediate frame of the segmented data, M x-n Indicates the Mel cepstral coefficient features of the first n frames of the middle frame of the segmented data, M x+n Represents the Mel cepstral coefficient features of the next n frames of the middle frame of the segmented data, μ 1 Represents the peak-to-peak variation coefficient of the segmented data signal, μ 2 represents the gradient change coefficient of the segmented data signal, and a represents a constant greater than 0;

[0105] S2.4: The segmented data corresponding to the maximum value of the feature vector is used as typical dialect speech data. The segment corresponding to the maximum feature vector usually contains the most important dialect features.

[0106] S3 includes a dialect identification network, see Figure 5 , the network structure diagram of dialect recognition in an embodiment of the present invention, involving the convolution and encoder modules of the input layer, the fully connected layer and attention mechanism of the vector processing layer, the decoder module and the Bayesian calculation module in the output layer. The design of the entire network is based on a deep learning framework, extracting local features of speech signals through the convolution layer, capturing global temporal features of speech through a bidirectional gated recurrent unit (GRU), combining the attention mechanism to perform weighted processing of dialect labels, and finally implementing classification of dialect labels through Bayesian calculation. The dialect recognition network includes:

[0107] The input layer is used to input typical dialect speech data into the network and extract local and global features from the speech signal;

[0108] The input layer includes a head convolution module and an encoder module;

[0109] The head convolution module consists of a convolution layer, a batch normalization layer, a maximum pooling layer and an activation layer, which are used to extract spatial features and temporal local features in speech signals; the composition structure is a convolution layer with a convolution kernel of 7X7. The convolution operation locally perceives the input speech features, which helps to capture the spectral information of the speech signal. The 7x7 convolution kernel can capture larger local features and is suitable for processing complex dialect speech data. A batch normalization layer is used to accelerate the training process and prevent the gradient from disappearing. The normalization process ensures that the data output by the convolution layer remains stable and that the data distribution of different dialects is similar. A maximum pooling layer with a size of 3x3 reduces data redundancy through downsampling operations while retaining the most significant features. Pooling can effectively reduce the size of the feature map and improve computational efficiency. And an activation layer with an activation function of RELU can improve the nonlinear representation ability and accelerate the convergence speed of the network. According to the input layer, the variance of the maximized features of typical dialect speech data is determined through convolution and pooling operations, and the encoding vector is calculated;

[0110] The encoder module includes a gated recurrent unit A and a gated recurrent unit B. The gated recurrent unit A is used to perform forward propagation on the encoding vector to calculate the hidden vector h 1 , used to capture the forward temporal dependency of the speech signal, the gated recurrent unit B is used to back-propagate the encoding vector and calculate the hidden vector h 2 , used to capture the reverse temporal dependency of speech signals. Through the bidirectional GRU structure, it can fully capture the bidirectional temporal features in speech signals. Especially in dialect speech recognition, not only the contextual information of speech is crucial, but the reverse dependency relationship can also provide strong support;

[0111] A vector processing layer, wherein the vector processing layer includes a fully connected module, an attention mechanism module, and a decoder module;

[0112] The fully connected module includes a fully connected layer, which concatenates two hidden vectors to obtain a dialect label. The concatenation operation can integrate the forward and backward information of the speech signal and improve the temporal understanding ability of the dialect speech data. The fully connected layer is used to map the concatenated high-dimensional vector to the dialect label space, providing a basis for subsequent label classification.

[0113] The attention mechanism module is used to calculate the attention weight of the dialect label, and reconstruct the dialect label according to the attention weight. The attention mechanism is used to assign different weights to the dialect label according to different parts of the speech signal, and dynamically calculate the attention weight of each dialect label. The goal of the attention mechanism is to capture the significant features in the speech signal, assign a higher weight to the most relevant part, and re-weight the encoding vector of the dialect label according to the calculated attention weight to generate a reconstructed dialect label vector. The attention mechanism can enhance the network's attention to the key parts of the speech signal and effectively improve the accuracy of classification. In particular, in long speech segments, the attention mechanism can dynamically select the most representative speech features;

[0114] The decoder module is used to decode the reconstructed dialect label to obtain the input vector of the output layer. The decoder can further refine the dialect label encoding vector weighted by the attention mechanism to generate a more accurate classification result.

[0115] The output layer includes a Bayesian calculation module, which is used to calculate the conditional probability and posterior probability of each label in the input vector of the output layer according to the Bayesian theorem, and combine the conditional probability and posterior probability through Gaussian distribution as the dialect label probability. The probability corresponding to each dialect label obeys the multivariate Gaussian distribution, and the probability of the label is calculated through maximum likelihood estimation. The Bayesian calculation module combines the prior knowledge and the probability distribution of the observed data, and can accurately calculate the probability of each dialect label through the Gaussian distribution model.

[0116] In the recognition stage, each long speech data segment is processed to generate a feature vector and input it into the dialect recognition network. The probability corresponding to each dialect label is obtained through the Bayesian calculation module of the network. For example, assuming there are three dialects: dialect A, dialect B and dialect C, the dialect label probabilities of all input segmented data are weighted averaged. If the calculated probability of dialect A is the highest, the long speech data is marked as dialect A, and then a specific dialect conversion model is selected based on dialect A. The dialect conversion model is used to convert the dialect into Mandarin for output.

[0117] The S6 specifically includes a dialect conversion network, see Figure 6 , a schematic diagram of the dialect conversion network structure of an embodiment of the present invention, wherein the segmented speech data is processed by the dialect conversion model, and includes three main functional modules: a feature extraction layer, a feature fusion layer, and a dialect recognition layer. These modules work together to ultimately achieve dialect conversion and recognition. Through full difference space modeling, channel compensation algorithm, cascade fusion and probabilistic linear discriminant analysis (PLDA), the voiceprint information and rhythm features in the dialect speech can be extracted and processed, and the result of the dialect conversion is ultimately generated. The dialect conversion network includes:

[0118] The feature extraction layer is responsible for extracting voiceprint features and rhythm features from the input segmented speech data. This layer models and represents the target voiceprint information and channel information of the speech through the full difference space modeling method;

[0119] The full difference space modeling is to represent the speaker features and channel features in the speech as a low-dimensional vector in the same space. Specifically, the feature extraction layer will represent the voiceprint and channel information of the input speech signal as a low-dimensional representation that integrates multiple features through full difference space modeling;

[0120] Target voiceprint information: refers to the speaker's voice characteristics, such as tone, speaking speed, pronunciation habits, etc. This information is directly related to the speaker's individual characteristics.

[0121] Channel information: indicates the interference of voice signals during transmission, such as noise introduced by external factors such as different recording devices, recording environments, and network jitter.

[0122] The full difference space model represents the target voiceprint information and channel information as a voiceprint vector feature by uniformly modeling these information;

[0123] According to the pre-trained acoustic mapping function of the dialect conversion model, the musical features in the segmented speech data are extracted. The musical features include the pitch, duration, intensity, rhythmic pattern, etc. of the speech. The acoustic mapping function can quantify these features of the speech so that the conversion model can adapt to the pronunciation and rhythm of the source dialect;

[0124] The feature fusion layer cascades and fuses the voiceprint vector features and musical features generated by the feature extraction layer, and calculates the global fusion features. Cascade fusion is a technology that connects multiple features in series in a dimension. By combining features from different sources, the model can simultaneously learn the associations between these features;

[0125] First, the extracted voiceprint vector is processed for dimensionality reduction to ensure that the voiceprint information will not cause dimensional expansion when it is fused with the musical features. Then, the voiceprint vector features and musical features are fused in a cascade form to calculate the global fusion features, which contain the comprehensive representation of the voiceprint information and musical information. These features can not only describe the speaker characteristics in the segmented data, but also reflect the pronunciation pattern and rhythm characteristics of the speech. During the fusion process, the model can also weight the features of different time steps through the self-attention mechanism to enhance the representation ability of the key parts of the dialect. The cascade fusion ensures the unified representation of the voiceprint and musical features, and can effectively capture the multi-dimensional information in the speech signal. The global fusion features contain both the timing information and frequency information of the speech, providing rich feature data for subsequent dialect identification;

[0126] The dialect recognition layer is used to perform dialect modeling based on the fused global features and output the final dialect conversion result through the log-likelihood ratio scoring mechanism;

[0127] The global fusion features are modeled by probabilistic linear discriminant analysis (PLDA). PLDA is a discriminative modeling method commonly used in speech recognition, which can map speech features into a discriminant space. The core idea of ​​PLDA is to ensure that the distributions of different dialects are as separated as possible by maximizing the intra-class variance and minimizing the inter-class variance;

[0128] The model uses the Log-Likelihood Ratio (LLR) to score the dialects. The model compares the PLDA models of each dialect and outputs the dialect with the highest score as the conversion result.

[0129] Embodiment 2:

[0130] See also Figure 7 The present invention provides an embodiment: a long speech dialect recognition device based on a deep learning algorithm, the device comprising:

[0131] Adaptive segmentation module, segmentation data processing module, dialect recognition module and dialect conversion module;

[0132] The adaptive segmentation module is configured with an adaptive segmentation strategy, and the adaptive segmentation strategy is used to segment the long voice data into multiple segments of different lengths;

[0133] The segmented data processing module is configured with a data processing strategy, and the data processing strategy is used to pre-process the segmented long voice data and extract typical dialect voice data;

[0134] The dialect recognition module is used to determine the dialect type of the long speech data through a neural network according to the typical dialect speech data;

[0135] The dialect conversion module is used to select a pre-trained dialect conversion model according to the dialect recognition result, and perform dialect conversion processing on the segmented data.

[0136] The adaptive segmentation module comprises:

[0137] A continuity parameter calculation unit, used for converting long speech data into frequency domain by short-time Fourier transform, extracting speech spectrum information, and calculating continuity parameters of the speech spectrum information;

[0138] The breakpoint detection unit is used to analyze the Fourier spectrum, count the breakpoints in the long voice data, and determine whether it is a voice switching point through the delay window technology;

[0139] A segment length calculation unit, used for dynamically calculating the length of each segment according to the continuity parameter, the zero crossing point and the preset segment length;

[0140] The adaptive segmentation strategy includes continuity detection logic, breakpoint detection logic and segmentation logic;

[0141] The continuity detection logic is used to convert the long speech data into a Fourier spectrum by short-time Fourier transform, and calculate the continuity parameter of the Fourier spectrum;

[0142] The breakpoint detection logic is used to perform spectrum analysis on the Fourier spectrum, count the number of breakpoints of the long voice data, and determine whether the information of the subsequent frame belongs to a voice switching point at the breakpoint of the long voice data according to the delay window, and if it is a voice switching point, mark the breakpoint as a zero crossing point;

[0143] The segmentation logic is used to calculate the segmentation length of the long speech data according to the continuity parameter and the zero crossing point, and output the segmentation data according to the segmentation length;

[0144] The continuity detection logic is configured in the continuity parameter calculation unit, the breakpoint detection logic is configured in the breakpoint detection unit, and the segmentation logic is configured in the segmentation length calculation unit.

[0145] The segmented data processing module comprises:

[0146] A pre-emphasis unit is used to perform pre-emphasis processing on the segmented data through a high-pass filter to enhance the high-frequency part of the speech signal;

[0147] A denoising unit is used to denoise the signal by wavelet transform technology and perform reverse translation to restore the original speech;

[0148] A modal decomposition unit, used to decompose the denoised signal by variational modal decomposition, extract each mode in the speech signal, and generate feature variables;

[0149] The data processing strategy includes preprocessing logic and typical speech extraction logic;

[0150] The preprocessing logic is used to perform pre-emphasis processing and denoising processing on the dialect speech;

[0151] The typical speech extraction logic is used to extract different frequency components in the speech signal through variational mode decomposition, calculate the feature vector of the segmented data by decomposing the feature variables, and use the segmented data corresponding to the maximum value of the feature vector as the typical dialect speech data;

[0152] The pre-processing logic is configured in the pre-emphasis unit and the denoising unit, and the typical speech extraction logic is configured in the modal decomposition unit.

[0153] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A long speech dialect recognition method based on deep learning algorithm, characterized in that: The method comprises: Segmenting the input long speech data according to the adaptive segmentation strategy to obtain segmented data; Preprocessing the segmented data, extracting typical dialect speech data, encoding vectors of the typical dialect speech data, and calculating the dialect label probability corresponding to the encoded vector; The dialect type corresponding to the highest dialect label probability is used as the dialect type of the long speech data, and a pre-trained dialect conversion model is selected according to the dialect type; The segmented data is processed by a dialect conversion model, and a dialect conversion result is output.

2. According to the method for long speech dialect recognition based on deep learning algorithm as described in claim 1, it is characterized in that: The adaptive segmentation strategy includes: Converting the long speech data into a Fourier spectrum by short-time Fourier transform, and calculating the continuity parameter of the Fourier spectrum; Performing spectrum analysis on the Fourier spectrum, counting the number of breakpoints of the long voice data, judging whether the information of subsequent frames belongs to a voice switching point at the breakpoint of the long voice data according to the delay window, and marking the breakpoint as a zero crossing point if it is a voice switching point; The segment length of the long speech data is calculated according to the continuity parameter and the zero crossing point, and the segment data is output according to the segment length. The calculation formula of the segment length is: Among them, L g Indicates the segment length, L p represents the preset segment length, T represents the zero crossing rate of the zero crossing point, F represents the fundamental frequency characteristic of the Fourier spectrum, E represents the short-time maximum energy of the Fourier spectrum, σ k represents the standard deviation of the energy value of the Fourier spectrum, δ represents the range of variation of the segment length, γ represents the parameter for adjusting the rate of change of the segment length, C represents the continuity parameter, and θ represents the continuity threshold.

3. According to the method for long speech dialect recognition based on deep learning algorithm as claimed in claim 1, it is characterized in that: The preprocessing of the segmented data comprises: Pre-emphasize the segmented data through a high-pass digital filter; The pre-emphasized segmented data is translated to change the position of the discontinuous point, and the translated signal is denoised according to the wavelet transform, and the signal is reversely translated to obtain the denoised signal; Decomposing the denoised signal according to variational mode decomposition, calculating speech decomposition feature variables, and calculating feature vectors of segmented data according to the speech decomposition feature variables; The segmented data corresponding to the largest feature vector is taken as typical dialect speech data.

4. According to claim 3, a long speech dialect recognition method based on deep learning algorithm is characterized in that: The step of encoding the vector of the typical dialect speech data and calculating the dialect label probability corresponding to the encoded vector includes: Performing feature extraction on the typical dialect speech data, determining the variance of the maximized feature of the typical dialect speech data according to the extraction result, and calculating the encoding vector; The encoding vector is passed through a gated recurrent unit to generate a hidden vector h1, and another gated recurrent unit is used to calculate in the opposite direction to generate a hidden vector h2. The two hidden vectors are concatenated, and a dialect label is obtained through a fully connected layer; Calculating the dialect label attention weight through the attention mechanism, reconstructing the dialect label according to the attention weight, and decoding the reconstructed dialect label through the decoder; The conditional probability and the posterior probability of each label in the dialect label are calculated, and the conditional probability and the posterior probability are combined through Gaussian distribution as the dialect label probability.

5. According to the method for long speech dialect recognition based on deep learning algorithm as claimed in claim 4, it is characterized in that: The processing of the segmented data by using the dialect conversion model includes: A feature extraction layer, for spatially representing the target voiceprint information and channel information of the segmented data through full difference space modeling, extracting voiceprint vector features according to a channel compensation algorithm, and extracting musical characteristics of the segmented data according to an acoustic mapping function pre-trained by the dialect conversion model; A feature fusion layer, used for fusing the voiceprint vector feature and the musicality feature through cascade fusion to calculate a global fusion feature; The dialect recognition layer is used to model the global fusion feature through the probabilistic linear discriminant analysis of the dialect conversion model, obtain the matching speech of the global fusion feature, score the matching speech and the segmented data by log-likelihood ratio, and output the matching speech with the highest score as the dialect conversion result.

6. A long speech dialect recognition device based on a deep learning algorithm, used to implement a long speech dialect recognition method based on a deep learning algorithm as described in any one of claims 1 to 5, characterized in that: The device comprises: Adaptive segmentation module, segmentation data processing module, dialect recognition module and dialect conversion module; The adaptive segmentation module is configured with an adaptive segmentation strategy, and the adaptive segmentation strategy is used to segment the long voice data and divide the long voice data into a plurality of segments of different lengths; The segmented data processing module is configured with a data processing strategy, and the data processing strategy is used to pre-process the segmented long voice data and extract typical dialect voice data; The dialect recognition module is used to determine the dialect type of the long speech data through a neural network according to the typical dialect speech data; The dialect conversion module is used to select a pre-trained dialect conversion model according to the dialect recognition result, and perform dialect conversion processing on the segmented data.

7. The long speech dialect recognition device based on deep learning algorithm according to claim 6, characterized in that: The adaptive segmentation module comprises: A continuity parameter calculation unit, used for converting long speech data into frequency domain by short-time Fourier transform, extracting speech spectrum information, and calculating continuity parameters of the speech spectrum information; The breakpoint detection unit is used to analyze the Fourier spectrum, count the breakpoints in the long voice data, and determine whether it is a voice switching point through the delay window technology; A segment length calculation unit, used for dynamically calculating the length of each segment according to the continuity parameter, the zero crossing point and the preset segment length; The adaptive segmentation strategy includes continuity detection logic, breakpoint detection logic and segmentation logic; The continuity detection logic is used to convert the long speech data into a Fourier spectrum by short-time Fourier transform, and calculate the continuity parameter of the Fourier spectrum; The breakpoint detection logic is used to perform spectrum analysis on the Fourier spectrum, count the number of breakpoints of the long voice data, and determine whether the information of the subsequent frame belongs to a voice switching point at the breakpoint of the long voice data according to the delay window, and if it is a voice switching point, mark the breakpoint as a zero crossing point; The segmentation logic is used to calculate the segmentation length of the long speech data according to the continuity parameter and the zero crossing point, and output the segmentation data according to the segmentation length; The continuity detection logic is configured in the continuity parameter calculation unit, the breakpoint detection logic is configured in the breakpoint detection unit, and the segmentation logic is configured in the segmentation length calculation unit.

8. The long speech dialect recognition device based on deep learning algorithm according to claim 6, characterized in that: The segmented data processing module comprises: A pre-emphasis unit is used to perform pre-emphasis processing on the segmented data through a high-pass filter to enhance the high-frequency part of the speech signal; A denoising unit is used to denoise the signal by wavelet transform technology and perform reverse translation to restore the original speech; A modal decomposition unit, used to decompose the denoised signal by variational modal decomposition, extract each mode in the speech signal, and generate feature variables; The data processing strategy includes preprocessing logic and typical speech extraction logic; The preprocessing logic is used to perform pre-emphasis processing and denoising processing on the dialect speech; The typical speech extraction logic is used to extract different frequency components in the speech signal through variational mode decomposition, calculate the feature vector of the segmented data by decomposing the feature variables, and use the segmented data corresponding to the maximum value of the feature vector as the typical dialect speech data; The pre-processing logic is configured in the pre-emphasis unit and the denoising unit, and the typical speech extraction logic is configured in the modal decomposition unit.

Images