An AI-based speech processing method
By performing fine segmentation and multi-dimensional feature extraction on the speech signal, combined with multi-channel BP neural network processing, the problem of low speech recognition accuracy in existing technologies has been solved, achieving higher recognition accuracy and noise resistance.
Patent Information
- Application Number
- CN202511200071.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-26
- Publication Date
- 2025-11-14
- Estimated Expiration
- 2045-08-26
AI Technical Summary
Existing speech recognition technologies are coarse in segmenting speech signals during the feature extraction and matching stages, failing to finely distinguish between stable and disturbed segments, and lacking attention to the feature transfer patterns between adjacent speech segments, resulting in low recognition accuracy. They are particularly susceptible to noise interference in complex environments or short speech segments.
The speech signal is divided into multiple short time frames, and stationary and perturbation segments are marked. The fundamental frequency shift ratio, harmonic centroid shift ratio, spectral bandwidth shift ratio, and power spectrum shift ratio are extracted to construct a shift ratio feature matrix. The feature matching vector is processed by a multi-channel BP neural network to achieve speaker identification.
By analyzing the combined features of stable and disturbed segments, the migration patterns between speech segments are fully explored, improving the accuracy and noise resistance of speech recognition and enhancing the discriminative ability of features.
Smart Images
Figure CN120748414B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of speech recognition technology, and more specifically to a speech processing method based on artificial intelligence. Background Technology
[0002] In the field of speech processing, speaker recognition technology, as a core support for scenarios such as human-computer interaction, identity authentication, and information security, has always been a research hotspot. Its core objective is to accurately identify the speaker's identity by analyzing the personalized acoustic features contained in the speech signal.
[0003] In existing technologies, the feature extraction stage of speaker recognition methods largely revolves around the basic acoustic properties of speech signals. For example, fundamental frequency features are used to capture pitch information, Mel-frequency cepstral coefficients (MFCCs) extract spectral envelope features by simulating human auditory characteristics, and linear prediction coefficients (LPCs) describe the linear prediction characteristics of speech signals based on vocal tract models. These feature extractions are based on the static properties of continuous frames. For dynamic feature processing, common methods include performing difference operations on static features to obtain temporal variation information, or modeling the probability distribution of feature sequences using Hidden Markov Models (HMMs) or Gaussian Mixture Models (GMMs) to reflect the dynamic evolution of speech.
[0004] However, existing technologies have significant limitations: First, most methods segment speech signals in a coarse manner, failing to finely divide the inherent differences between stable segments (such as vowels) and disturbed segments (such as consonants) commonly found in speech, resulting in feature extraction lacking specificity for the characteristics of different speech segments. Second, feature analysis often focuses on the acoustic properties within a single speech segment, paying insufficient attention to the feature transfer patterns between adjacent speech segments (such as stable and disturbed segments, or disturbed and stable segments). These transfer patterns are closely related to the speaker's pronunciation habits, vocal cords, and vocal tract physical characteristics, and are important personalized distinguishing information. Furthermore, in the feature matching stage, existing methods often rely on the similarity measurement of single-type features, exhibiting weak collaborative analysis capabilities for features in multi-combination scenarios. They struggle to fully utilize the differentiated information contained in different speech segment combinations, and in complex environments or short speech segments, they are susceptible to noise interference or insufficient feature information, leading to low recognition accuracy. Summary of the Invention
[0005] In view of the above-mentioned shortcomings in the prior art, the present invention provides an artificial intelligence-based speech processing method that solves the problem of low speech recognition accuracy in the prior art.
[0006] To achieve the above-mentioned objectives, the technical solution adopted by this invention is: a speech processing method based on artificial intelligence, comprising the following steps:
[0007] The speech signal is divided into multiple short time frames, and the short time frames are marked based on the zero-crossing rate to obtain the stationary segment and the perturbation segment;
[0008] Extract the combinations of stationary segments-stationary segments, stationary segments-disturbance segments, disturbance segments-stationary segments, and disturbance segments-disturbance segments;
[0009] For each combination, extract the fundamental frequency shift ratio, harmonic centroid shift ratio, spectral bandwidth shift ratio, and power spectrum shift ratio, and construct the shift ratio feature matrix for the corresponding combination;
[0010] For each combination, calculate the similarity coefficient between each column of the feature and the stored feature in the transfer ratio feature matrix, and take the average of the similarity coefficients of each column to obtain the feature matching coefficients corresponding to each column, thus forming a feature matching vector.
[0011] A multi-channel BP neural network is used to process four combinations of transfer ratio feature matrices and feature matching vectors to obtain speaker identification results.
[0012] Furthermore, the process of obtaining the stationary segment and the perturbation segment includes:
[0013] The speech signal is divided into multiple short time frames;
[0014] The zero-crossing rate of each short frame is calculated, and the average of the zero-crossing rates of all short frames is taken.
[0015] Short-time frames with a zero-crossing rate greater than the mean are marked as perturbation segments;
[0016] Short frames with a zero-crossing rate less than or equal to the mean are marked as stationary segments.
[0017] Furthermore, the process of constructing the transfer ratio feature matrix for the corresponding combination includes:
[0018] Perform a Fourier transform on each signal segment in each combination to obtain the spectrum of each signal segment in the combination;
[0019] In a combination, based on the spectrum of each signal segment, the fundamental frequency shift ratio, harmonic centroid shift ratio, spectral bandwidth shift ratio, and power spectral shift ratio of each combination are obtained;
[0020] The fundamental frequency shift ratios of the same combination at each time point are used to construct a fundamental frequency shift ratio vector;
[0021] The harmonic centroid shift ratios of the same combination at each time point are used to construct a harmonic centroid shift ratio vector.
[0022] Construct a spectral bandwidth-to-mobility vector from the spectral bandwidth-to-mobility ratios of the same combination at each time point;
[0023] The power spectral mobility ratios of the same combination at each time point are used to construct a power spectral mobility ratio vector.
[0024] The fundamental frequency mobility ratio vector, harmonic centroid mobility ratio vector, spectral bandwidth mobility ratio vector, and power spectrum mobility ratio vector are used as row vectors to form the mobility ratio characteristic matrix of this combination.
[0025] Furthermore, the process of obtaining the fundamental frequency shift ratio includes: in a combination, taking the frequency corresponding to the largest amplitude value in the spectrum of each signal segment as the fundamental frequency, and taking the ratio of the fundamental frequency of the later signal segment to the fundamental frequency of the earlier signal segment as the fundamental frequency shift ratio;
[0026] The process of obtaining the harmonic centroid shift ratio includes: in a combination, extracting each harmonic based on the fundamental frequency in the spectrum of each signal segment, calculating the harmonic centroid of each harmonic, and taking the ratio of the harmonic centroid of the later signal segment to the harmonic centroid of the earlier signal segment as the harmonic centroid shift ratio.
[0027] The process of obtaining the spectral bandwidth shift ratio includes: in a combination, taking the ratio of the spectral bandwidth of the later signal to the spectral bandwidth of the earlier signal as the spectral bandwidth shift ratio;
[0028] The process of obtaining the power spectral mobility ratio includes: in a combination, taking the ratio of the power spectrum of the later signal to the power spectrum of the earlier signal as the power spectral mobility ratio.
[0029] Furthermore, the process of constructing the feature matching vector includes:
[0030] The first similarity coefficient is obtained based on the difference between the baseband shift ratio and the storage baseband shift ratio for each combination;
[0031] The second similarity coefficient is obtained based on the difference between the harmonic centroid shift ratio of each combination and the stored harmonic centroid shift ratio.
[0032] The third similarity coefficient is obtained based on the difference between the spectral bandwidth transfer ratio and the storage spectral bandwidth transfer ratio of each combination.
[0033] The fourth similarity coefficient is obtained based on the difference between the power spectral mobility ratio of each combination and the stored power spectral mobility ratio.
[0034] The average of the first, second, third, and fourth similarity coefficients corresponding to each column of the transfer ratio feature matrix for each combination is taken to obtain the feature matching coefficient for each column.
[0035] The feature matching coefficients of each column are used to construct a feature matching vector.
[0036] Furthermore, the process of obtaining the first similarity coefficient includes: subtracting the fundamental frequency shift ratio from the storage fundamental frequency shift ratio, taking the absolute value of the subtraction result, normalizing the value after taking the absolute value to obtain the difference coefficient, and subtracting the difference coefficient from 1 to obtain the first similarity coefficient;
[0037] The process of obtaining the second similarity coefficient includes: subtracting the harmonic centroid shift ratio from the stored harmonic centroid shift ratio, taking the absolute value of the subtraction result, normalizing the value after taking the absolute value to obtain the difference coefficient, and subtracting the difference coefficient from 1 to obtain the second similarity coefficient.
[0038] The process of obtaining the third similarity coefficient includes: subtracting the spectral bandwidth mobility ratio from the storage spectral bandwidth mobility ratio, taking the absolute value of the subtraction result, normalizing the value after taking the absolute value to obtain the difference coefficient, and subtracting the difference coefficient from 1 to obtain the third similarity coefficient.
[0039] The process of obtaining the fourth similarity coefficient includes: subtracting the power spectrum mobility ratio from the storage power spectrum mobility ratio, taking the absolute value of the subtraction result, normalizing the absolute value to obtain the difference coefficient, and subtracting the difference coefficient from 1 to obtain the fourth similarity coefficient.
[0040] Furthermore, the multi-channel BP neural network includes: 4 convolutional feature enhancement channels, a Concat layer, and a BP neural network;
[0041] The first input of each convolutional feature enhancement channel is used to input a combined transfer ratio feature matrix, and the second input is used to input the same combined feature matching vector.
[0042] The input of the Concat layer is connected to the output of the four convolutional feature enhancement channels, and its output is connected to the input of the BP neural network.
[0043] The output of the BP neural network is used as the output of the multi-channel BP neural network.
[0044] Furthermore, each convolutional feature enhancement channel includes: a first convolutional layer, a second convolutional layer, and a multiplier;
[0045] The input of the first convolutional layer serves as the first input of the convolutional feature enhancement channel, and its output is connected to the input of the second convolutional layer.
[0046] The first input of the multiplier is connected to the output of the second convolutional layer, its second input serves as the second input of the convolutional feature enhancement channel, and its output serves as the output of the convolutional feature enhancement channel.
[0047] Furthermore, the kernel size of the first convolutional layer is 1×4, and the kernel size of the second convolutional layer is 1×1. The first convolutional layer is used to convolve each column of the 4×N transfer ratio feature matrix to obtain 1×N convolutional compressed features. The second convolutional layer is used to extract features from the 1×N convolutional compressed features to obtain 1×N mapping features. At the multiplier, the 1×N mapping features are multiplied element-wise with the 1×N feature matching vector to obtain 1×N convolutional enhanced features, where N is the length.
[0048] The beneficial effects of this invention are as follows:
[0049] 1. This invention performs feature analysis using four combinations of stable and disturbed segments, fully exploring the feature transfer patterns between adjacent speech segments of different types. These transfer patterns are directly related to the speaker's pronunciation habits, vocal cords, and vocal tract physical characteristics, effectively compensating for the shortcomings of existing technologies in paying insufficient attention to this type of information.
[0050] 2. The fundamental frequency shift ratio, harmonic centroid shift ratio, spectral bandwidth shift ratio, and power spectrum shift ratio extracted by this invention quantify the migration characteristics of features in different combinations from multiple dimensions. Compared with traditional static features, they can better reflect the unique way a speaker speaks in the continuous pronunciation process and improve the discriminative ability of features.
[0051] 3. This invention performs similarity calculation on the transfer ratio feature matrix and stored features of each combination, and obtains the feature matching vector through mean processing. It combines a multi-channel BP neural network to perform collaborative processing on the feature matrix and matching vector of the four combinations, making full use of the differentiated information contained in different speech segment combinations, thereby improving the accuracy of speech recognition. Attached Figure Description
[0052] Figure 1 A flowchart of an artificial intelligence-based speech processing method;
[0053] Figure 2 A schematic diagram of a combination of steady segments;
[0054] Figure 3 This is a schematic diagram of the combination of a stable segment and a disturbance segment;
[0055] Figure 4 This is a schematic diagram of the combination of disturbance segment and stationary segment;
[0056] Figure 5 This is a schematic diagram of a combination of disturbance segments;
[0057] Figure 6 This is a schematic diagram of the structure of a multi-channel BP neural network. Detailed Implementation
[0058] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.
[0059] like Figure 1As shown, an artificial intelligence-based speech processing method includes the following steps:
[0060] The speech signal is divided into multiple short time frames, and the short time frames are marked based on the zero-crossing rate to obtain the stationary segment and the perturbation segment;
[0061] Extract the combinations of stationary segments-stationary segments, stationary segments-disturbance segments, disturbance segments-stationary segments, and disturbance segments-disturbance segments;
[0062] For each combination, extract the fundamental frequency shift ratio, harmonic centroid shift ratio, spectral bandwidth shift ratio, and power spectrum shift ratio, and construct the shift ratio feature matrix for the corresponding combination;
[0063] For each combination, calculate the similarity coefficient between each column of the feature and the stored feature in the transfer ratio feature matrix, and take the average of the similarity coefficients of each column to obtain the feature matching coefficients corresponding to each column, thus forming a feature matching vector.
[0064] A multi-channel BP neural network is used to process four combinations of transfer ratio feature matrices and feature matching vectors to obtain speaker identification results.
[0065] In this embodiment, the process of obtaining the stationary segment and the perturbation segment includes:
[0066] The speech signal is divided into multiple short time frames;
[0067] The zero-crossing rate of each short frame is calculated, and the average of the zero-crossing rates of all short frames is taken.
[0068] Short-time frames with a zero-crossing rate greater than the mean are marked as perturbation segments;
[0069] Short frames with a zero-crossing rate less than or equal to the mean are marked as stationary segments.
[0070] In this embodiment, the length of the short-time frame ranges from 20 to 30 milliseconds, and adjacent frames do not overlap.
[0071] Zero-crossing rate refers to the number of times a speech signal waveform crosses the zero level (from positive to negative or from negative to positive) per unit time. Its value is directly related to the proportion of high-frequency components in the signal: For stable speech segments such as vowels, the vocal cords vibrate regularly during pronunciation, the signal is dominated by low-frequency harmonic components, the waveform is smooth, and the number of times it crosses the zero level is small, resulting in a low zero-crossing rate; For disturbed speech segments such as consonants (such as plosives "p" and "t", fricatives "s" and "sh"), the airflow generates turbulence or plosives in the vocal tract during pronunciation, the signal contains a large number of high-frequency noise components, the waveform fluctuates violently, and the number of times it crosses the zero level is large, resulting in a high zero-crossing rate.
[0072] The two short-time frames in the combination are adjacent.
[0073] Short frames with a zero-crossing rate higher than the mean have significant high-frequency perturbation characteristics, which are consistent with the acoustic properties of "perturbation segments" such as consonants; short frames with a zero-crossing rate lower than or equal to the mean have more prominent low-frequency stability characteristics, which are consistent with the acoustic properties of "stable segments" such as vowels.
[0074] like Figure 2 As shown, both the preceding and following segments are smooth segments with a low zero-crossing rate (such as the transition from vowel "a" to vowel "i", as in the word "love" ài, where "a" becomes "i". For example... Figure 3 As shown, the first segment is a stable segment (such as the vowel "u"), and the second segment is a perturbation segment with a high zero-crossing rate (such as the voiceless consonant "sh"). Figure 4 As shown, the first segment is a disturbed segment (such as the voiceless consonant "p"), and the second segment is a stable segment (such as the vowel "a"). Figure 5 As shown, both the first and second segments are perturbation segments with a high zero-crossing rate (such as the voiceless consonant "s" followed by the voiceless consonant "h").
[0075] In this embodiment, the process of constructing the transfer ratio feature matrix for the corresponding combination includes:
[0076] Perform a Fourier transform on each signal segment in each combination to obtain the spectrum of each signal segment in the combination;
[0077] In a combination, based on the spectrum of each signal segment, the fundamental frequency shift ratio, harmonic centroid shift ratio, spectral bandwidth shift ratio, and power spectral shift ratio of each combination are obtained;
[0078] The fundamental frequency shift ratios of the same combination at each time point are used to construct a fundamental frequency shift ratio vector;
[0079] The harmonic centroid shift ratios of the same combination at each time point are used to construct a harmonic centroid shift ratio vector.
[0080] Construct a spectral bandwidth-to-mobility vector from the spectral bandwidth-to-mobility ratios of the same combination at each time point;
[0081] The power spectral mobility ratios of the same combination at each time point are used to construct a power spectral mobility ratio vector.
[0082] The fundamental frequency mobility ratio vector, harmonic centroid mobility ratio vector, spectral bandwidth mobility ratio vector, and power spectrum mobility ratio vector are used as row vectors to form the mobility ratio characteristic matrix of this combination.
[0083] The process of constructing the transfer ratio characteristic matrix of a stationary segment-stationary segment combination is illustrated by taking a stationary segment-stationary segment combination as an example: Fourier transform is performed on each signal segment in the stationary segment-stationary segment combination to obtain the spectrum of each signal segment in the combination.
[0084] In the combination of stationary segments, the fundamental frequency shift ratio, harmonic centroid shift ratio, spectral bandwidth shift ratio, and power spectral shift ratio of the combination are obtained based on the spectrum of each signal segment.
[0085] The fundamental frequency shift ratios of the stationary segment-stationary segment combinations at each time point are used to construct the fundamental frequency shift ratio vector;
[0086] The harmonic centroid shift ratio vector is formed by the harmonic centroid shift ratio of the combination of stationary segments at each time point.
[0087] The spectral bandwidth mobility ratio is formed by combining the stationary segments at each time point.
[0088] The power spectral mobility ratios of the stationary segments combined at each time point are used to construct the power spectral mobility ratio vector.
[0089] By constructing row vectors from the fundamental frequency mobility ratio vector, harmonic centroid mobility ratio vector, spectral bandwidth mobility ratio vector, and power spectrum mobility ratio vector, a mobility ratio characteristic matrix combining stationary segments and stationary segments is formed.
[0090] The process of obtaining the combination of stationary segment-perturbation segment, perturbation segment-stationary segment, and perturbation segment-perturbation segment is similar to obtaining the transfer ratio feature matrix of the combination of stationary segment-stationary segment.
[0091] In this embodiment, the process of obtaining the fundamental frequency shift ratio includes: in a combination, taking the frequency corresponding to the largest amplitude value in the spectrum of each signal segment as the fundamental frequency, and taking the ratio of the fundamental frequency of the later signal segment to the fundamental frequency of the earlier signal segment as the fundamental frequency shift ratio.
[0092] Taking the combination of stationary segments as an example, the process of obtaining the fundamental frequency shift ratio is explained as follows: In the combination of stationary segments, the frequency corresponding to the largest amplitude value in the spectrum of each signal segment is taken as the fundamental frequency, and the ratio of the fundamental frequency of the later stationary segment to the fundamental frequency of the earlier stationary segment is taken as the fundamental frequency shift ratio of the combination of stationary segments.
[0093] The process of obtaining the harmonic centroid shift ratio includes: in a combination, extracting each harmonic based on the fundamental frequency in the spectrum of each signal segment, calculating the harmonic centroid of each harmonic, and taking the ratio of the harmonic centroid of the later signal segment to the harmonic centroid of the earlier signal segment as the harmonic centroid shift ratio.
[0094] In this embodiment, the harmonic centroid is calculated using the existing spectral frequency centroid formula.
[0095] Taking the combination of stationary segments as an example, the process of obtaining the harmonic centroid shift ratio is explained as follows: In the combination of stationary segments, each harmonic is extracted according to the fundamental frequency in the spectrum of each signal segment, the centroid of each harmonic is calculated, and the ratio of the centroid of the harmonic in the later stationary segment to the centroid of the harmonic in the earlier stationary segment is taken as the harmonic centroid shift ratio.
[0096] The process of obtaining the spectral bandwidth shift ratio includes: in a combination, taking the ratio of the spectral bandwidth of the later signal to the spectral bandwidth of the earlier signal as the spectral bandwidth shift ratio.
[0097] Taking the combination of stationary segments as an example, the process of obtaining the spectral bandwidth transfer ratio is explained: In the combination of stationary segments, the ratio of the spectral bandwidth of the later stationary segment to the spectral bandwidth of the earlier stationary segment is taken as the spectral bandwidth transfer ratio.
[0098] The process of obtaining the power spectral mobility ratio includes: in a combination, taking the ratio of the power spectrum of the later signal to the power spectrum of the earlier signal as the power spectral mobility ratio.
[0099] Taking a combination of stationary segments as an example, the process of obtaining the power spectral mobility ratio is explained as follows: In a combination of stationary segments, the ratio of the power spectrum of the later stationary segment to the power spectrum of the earlier stationary segment is taken as the spectral bandwidth mobility ratio.
[0100] The processes for obtaining the fundamental frequency shift ratio, harmonic centroid shift ratio, spectral bandwidth shift ratio, and power spectral shift ratio are the same for the combination of stable segment-disturbance segment, the combination of disturbance segment-stable segment, the combination of disturbance segment-disturbance segment, and the combination of stable segment-stable segment.
[0101] The fundamental frequency is directly related to the speaker's vocal cord vibration frequency, and its transfer ratio can accurately reflect the individualized changes in vocal cord vibration during the transition between different speech segments. The harmonic centroid transfer ratio targets the harmonic components of the fundamental frequency, capturing the migration pattern of harmonic energy distribution through the ratio of harmonic centroids. Harmonic characteristics are closely related to the speaker's vocal tract resonance characteristics, reflecting individual differences in vocal tract structure. The spectral bandwidth transfer ratio focuses on the relative change in spectral width. The spectral bandwidth reflects the distribution range of frequency components in the speech signal, and its transfer ratio can characterize the degree of spectral broadening or contraction between the preceding and following segments, further enriching the dynamic information of spectral features. The power spectrum transfer ratio focuses on the relative change in signal energy distribution. The power spectrum is related to the intensity and volume of the speech signal, and its transfer ratio can reflect the individualized changes in energy between the preceding and following segments. These four types of transfer ratios extract feature migration information for different speech segment combinations from four dimensions: vocal cord vibration, vocal tract resonance, spectral distribution range, and energy distribution.
[0102] The transfer ratios in this invention are calculated by the ratio of the parameters of the later signal segment to the parameters of the earlier signal segment. This quantification method of relative change can effectively capture the transition patterns of the two signal segments in specific acoustic features. Compared with the static parameters of a single signal segment, the transfer ratio can better highlight the individual speech patterns of different speech segment combinations (such as from a stable segment to a stable segment, from a stable segment to a disturbed segment, from a disturbed segment to a stable segment, and from a disturbed segment to a disturbed segment). These dynamic patterns contain the speaker's unique pronunciation habits and language rhythm, providing more discriminative features for distinguishing different speakers.
[0103] In this embodiment, the process of constructing the feature matching vector includes:
[0104] The first similarity coefficient is obtained based on the difference between the baseband shift ratio and the storage baseband shift ratio for each combination;
[0105] The second similarity coefficient is obtained based on the difference between the harmonic centroid shift ratio of each combination and the stored harmonic centroid shift ratio.
[0106] The third similarity coefficient is obtained based on the difference between the spectral bandwidth transfer ratio and the storage spectral bandwidth transfer ratio of each combination.
[0107] The fourth similarity coefficient is obtained based on the difference between the power spectral mobility ratio of each combination and the stored power spectral mobility ratio.
[0108] The average of the first, second, third, and fourth similarity coefficients corresponding to each column of the transfer ratio feature matrix for each combination is taken to obtain the feature matching coefficient for each column.
[0109] The feature matching coefficients of each column are used to construct a feature matching vector.
[0110] The storage baseband shift ratio includes: the storage baseband shift ratio of the combination of stable segment-stable segment, the storage baseband shift ratio of the combination of stable segment-disturbance segment, the storage baseband shift ratio of the combination of disturbance segment-stable segment, and the storage baseband shift ratio of the combination of disturbance segment-disturbance segment. These storage baseband shift ratios are derived from the baseband shift ratios of four combinations of pre-recorded user voice signals.
[0111] The stored harmonic centroid shift ratios include: the stored harmonic centroid shift ratios of the combination of stationary segments and stationary segments, the stored harmonic centroid shift ratios of the combination of stationary segments and disturbance segments, the stored harmonic centroid shift ratios of the combination of disturbance segments and stationary segments, and the stored harmonic centroid shift ratios of the combination of disturbance segments and disturbance segments. These stored harmonic centroid shift ratios are derived from the harmonic centroid shift ratios of four combinations of pre-recorded user voice signals.
[0112] The storage spectral bandwidth shift ratio includes: the storage spectral bandwidth shift ratio of the combination of stationary segment-stationary segment, the storage spectral bandwidth shift ratio of the combination of stationary segment-disturbance segment, the storage spectral bandwidth shift ratio of the combination of disturbance segment-stationary segment, and the storage spectral bandwidth shift ratio of the combination of disturbance segment-disturbance segment. These storage spectral bandwidth shift ratios are derived from the spectral bandwidth shift ratios of the four combinations of pre-recorded user voice signals.
[0113] The storage power spectral shift ratio includes: the storage power spectral shift ratio of the combination of stationary segments and stationary segments, the storage power spectral shift ratio of the combination of stationary segments and disturbance segments, the storage power spectral shift ratio of the combination of disturbance segments and stationary segments, and the storage power spectral shift ratio of the combination of disturbance segments and disturbance segments. These storage power spectral shift ratios are derived from the power spectral shift ratios of four combinations of pre-recorded user voice signals.
[0114] In this embodiment, the process of obtaining similarity coefficients includes: subtracting the fundamental frequency shift ratio, harmonic centroid shift ratio, spectral bandwidth shift ratio, and power spectrum shift ratio from the corresponding stored shift ratios, taking the absolute value of the difference and normalizing it to obtain the difference coefficient, and then subtracting the difference coefficient from 1 to obtain the first, second, third, and fourth similarity coefficients, respectively.
[0115] In this embodiment, the process of obtaining the first similarity coefficient includes: subtracting the fundamental frequency shift ratio from the storage fundamental frequency shift ratio, taking the absolute value of the subtraction result, normalizing the value after taking the absolute value to obtain the difference coefficient, and subtracting the difference coefficient from 1 to obtain the first similarity coefficient.
[0116] The process of obtaining the second similarity coefficient includes: subtracting the harmonic centroid shift ratio from the stored harmonic centroid shift ratio, taking the absolute value of the subtraction result, normalizing the value after taking the absolute value to obtain the difference coefficient, and subtracting the difference coefficient from 1 to obtain the second similarity coefficient.
[0117] The process of obtaining the third similarity coefficient includes: subtracting the spectral bandwidth mobility ratio from the storage spectral bandwidth mobility ratio, taking the absolute value of the subtraction result, normalizing the value after taking the absolute value to obtain the difference coefficient, and subtracting the difference coefficient from 1 to obtain the third similarity coefficient.
[0118] The process of obtaining the fourth similarity coefficient includes: subtracting the power spectrum mobility ratio from the storage power spectrum mobility ratio, taking the absolute value of the subtraction result, normalizing the absolute value to obtain the difference coefficient, and subtracting the difference coefficient from 1 to obtain the fourth similarity coefficient.
[0119] In this embodiment, the formula for calculating the similarity coefficient is: Where θ is the similarity coefficient and V is the absolute value. This invention calculates the similarity coefficient for each combination based on the four transfer ratios and takes the average to obtain the feature matching coefficient for that combination. A larger feature matching coefficient indicates a closer resemblance to the stored human speech, thus improving the recognition accuracy of the multi-channel BP neural network.
[0120] like Figure 6 As shown, the multi-channel BP neural network includes: 4 convolutional feature enhancement channels, a Concat layer, and a BP neural network;
[0121] The first input of each convolutional feature enhancement channel is used to input a combined transfer ratio feature matrix, and the second input is used to input the same combined feature matching vector.
[0122] The input of the Concat layer is connected to the output of the four convolutional feature enhancement channels, and its output is connected to the input of the BP neural network.
[0123] The output of the BP neural network is used as the output of the multi-channel BP neural network.
[0124] exist Figure 6 In the above, the first combination is a steady segment-steady segment combination, the second combination is a steady segment-disturbance segment combination, the third combination is a disturbance segment-steady segment combination, and the fourth combination is a disturbance segment-disturbance segment combination.
[0125] This invention uses each convolutional feature enhancement channel to process a combined transfer ratio feature matrix and feature matching vector to enhance the features. Then, the features are concatenated through a Concat layer, and a BP neural network is used to identify the features based on the concatenated features.
[0126] In this embodiment, each convolutional feature enhancement channel includes: a first convolutional layer, a second convolutional layer, and a multiplier;
[0127] The input of the first convolutional layer serves as the first input of the convolutional feature enhancement channel, and its output is connected to the input of the second convolutional layer.
[0128] The first input of the multiplier is connected to the output of the second convolutional layer, its second input serves as the second input of the convolutional feature enhancement channel, and its output serves as the output of the convolutional feature enhancement channel.
[0129] In this embodiment, the kernel size of the first convolutional layer is 1×4, and the kernel size of the second convolutional layer is 1×1. The first convolutional layer is used to convolve each column of the 4×N transfer ratio feature matrix to obtain 1×N convolutional compressed features. The second convolutional layer is used to extract features from the 1×N convolutional compressed features to obtain 1×N mapping features. At the multiplier, the 1×N mapping features are multiplied element-wise with the 1×N feature matching vector to obtain 1×N convolutional enhanced features, where N is the length.
[0130] This invention employs a first convolutional layer to process a 4×N mobility ratio feature matrix, compressing the fundamental frequency mobility ratio, harmonic centroid mobility ratio, spectral bandwidth mobility ratio, and power spectral mobility ratio in the same column, converting them into 1×N convolutionally compressed features. These are then mapped using a second convolutional layer. A multiplier is used to element-wise multiply the mapped features with the feature matching vector. Locations in the mapped features with high matching degrees (large matching coefficients) with the stored features are made more significant, while locations with low matching degrees (small matching coefficients) are suppressed, increasing the difference between different features. This weighting method allows the network to automatically focus on key information consistent with the target speaker's features, weakening irrelevant or significantly different interfering information.
[0131] In this embodiment, the speaker's identity can be set to be valid if the output of the BP neural network is greater than or equal to a threshold, and invalid if it is less than the threshold, and the identity verification fails.
[0132] This invention utilizes four combinations of stable and disturbed segments for feature analysis, fully exploring the feature transfer patterns between adjacent speech segments of different types. These transfer patterns are directly related to the speaker's pronunciation habits, vocal cords, and vocal tract physical characteristics, effectively compensating for the shortcomings of existing technologies in focusing on this type of information.
[0133] The fundamental frequency shift ratio, harmonic centroid shift ratio, spectral bandwidth shift ratio, and power spectrum shift ratio extracted by this invention quantify the migration characteristics of features in different combinations from multiple dimensions. Compared with traditional static features, they can better reflect the unique way a speaker speaks in the continuous pronunciation process and improve the discriminative ability of features.
[0134] This invention calculates the similarity between the transfer ratio feature matrix and the stored features for each combination, and obtains the feature matching vector through mean processing. It then combines a multi-channel BP neural network to collaboratively process the feature matrices and matching vectors of the four combinations, making full use of the differentiated information contained in different speech segment combinations and improving the accuracy of speech recognition.
[0135] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A speech processing method based on artificial intelligence, characterized in that, Includes the following steps: The speech signal is divided into multiple short time frames, and the short time frames are marked based on the zero-crossing rate to obtain the stationary segment and the perturbation segment; Extract the combinations of stationary segments-stationary segments, stationary segments-disturbance segments, disturbance segments-stationary segments, and disturbance segments-disturbance segments; For each combination, extract the fundamental frequency shift ratio, harmonic centroid shift ratio, spectral bandwidth shift ratio, and power spectrum shift ratio, and construct the shift ratio feature matrix for the corresponding combination; The process of obtaining the fundamental frequency shift ratio includes: in a combination, taking the frequency corresponding to the largest amplitude value in the spectrum of each signal segment as the fundamental frequency, and taking the ratio of the fundamental frequency of the later signal segment to the fundamental frequency of the earlier signal segment as the fundamental frequency shift ratio; The process of obtaining the harmonic centroid shift ratio includes: in a combination, extracting each harmonic based on the fundamental frequency in the spectrum of each signal segment, calculating the harmonic centroid of each harmonic, and taking the ratio of the harmonic centroid of the later signal segment to the harmonic centroid of the earlier signal segment as the harmonic centroid shift ratio. The process of obtaining the spectral bandwidth shift ratio includes: in a combination, taking the ratio of the spectral bandwidth of the later signal to the spectral bandwidth of the earlier signal as the spectral bandwidth shift ratio; The process of obtaining the power spectral mobility ratio includes: in a combination, taking the ratio of the power spectrum of the later signal to the power spectrum of the earlier signal as the power spectral mobility ratio; For each combination, calculate the similarity coefficient between each column of the feature and the stored feature in the transfer ratio feature matrix, and take the average of the similarity coefficients of each column to obtain the feature matching coefficients corresponding to each column, thus forming a feature matching vector. A multi-channel BP neural network is used to process four combinations of transfer ratio feature matrices and feature matching vectors to obtain speaker identification results.
2. The speech processing method based on artificial intelligence according to claim 1, characterized in that, The process of obtaining the stationary and perturbation segments includes: The speech signal is divided into multiple short time frames; The zero-crossing rate of each short frame is calculated, and the average of the zero-crossing rates of all short frames is taken. Short-time frames with a zero-crossing rate greater than the mean are marked as perturbation segments; Short frames with a zero-crossing rate less than or equal to the mean are marked as stationary segments.
3. The artificial intelligence-based speech processing method according to claim 1, characterized in that, The process of constructing the transfer ratio feature matrix for the corresponding combination includes: Perform a Fourier transform on each signal segment in each combination to obtain the spectrum of each signal segment in the combination; In a combination, based on the spectrum of each signal segment, the fundamental frequency shift ratio, harmonic centroid shift ratio, spectral bandwidth shift ratio, and power spectral shift ratio of each combination are obtained; The fundamental frequency shift ratios of the same combination at each time point are used to construct a fundamental frequency shift ratio vector; The harmonic centroid shift ratios of the same combination at each time point are used to construct a harmonic centroid shift ratio vector. Construct a spectral bandwidth-to-mobility vector from the spectral bandwidth-to-mobility ratios of the same combination at each time point; The power spectral mobility ratios of the same combination at each time point are used to construct a power spectral mobility ratio vector. The fundamental frequency mobility ratio vector, harmonic centroid mobility ratio vector, spectral bandwidth mobility ratio vector, and power spectrum mobility ratio vector are used as row vectors to form the mobility ratio characteristic matrix of this combination.
4. The speech processing method based on artificial intelligence according to claim 1, characterized in that, The process of constructing a feature matching vector includes: The first similarity coefficient is obtained based on the difference between the baseband shift ratio and the storage baseband shift ratio for each combination; The second similarity coefficient is obtained based on the difference between the harmonic centroid shift ratio of each combination and the stored harmonic centroid shift ratio. The third similarity coefficient is obtained based on the difference between the spectral bandwidth transfer ratio and the storage spectral bandwidth transfer ratio of each combination. The fourth similarity coefficient is obtained based on the difference between the power spectral mobility ratio of each combination and the stored power spectral mobility ratio. The average of the first, second, third, and fourth similarity coefficients corresponding to each column of the transfer ratio feature matrix for each combination is taken to obtain the feature matching coefficient for each column. The feature matching coefficients of each column are used to construct a feature matching vector.
5. The artificial intelligence-based speech processing method according to claim 1 or 4, characterized in that, The process of obtaining the first similarity coefficient includes: subtracting the fundamental frequency shift ratio from the storage fundamental frequency shift ratio, taking the absolute value of the subtraction result, normalizing the value after taking the absolute value to obtain the difference coefficient, and subtracting the difference coefficient from 1 to obtain the first similarity coefficient. The process of obtaining the second similarity coefficient includes: subtracting the harmonic centroid shift ratio from the stored harmonic centroid shift ratio, taking the absolute value of the subtraction result, normalizing the value after taking the absolute value to obtain the difference coefficient, and subtracting the difference coefficient from 1 to obtain the second similarity coefficient. The process of obtaining the third similarity coefficient includes: subtracting the spectral bandwidth mobility ratio from the storage spectral bandwidth mobility ratio, taking the absolute value of the subtraction result, normalizing the value after taking the absolute value to obtain the difference coefficient, and subtracting the difference coefficient from 1 to obtain the third similarity coefficient. The process of obtaining the fourth similarity coefficient includes: subtracting the power spectrum mobility ratio from the storage power spectrum mobility ratio, taking the absolute value of the subtraction result, normalizing the absolute value to obtain the difference coefficient, and subtracting the difference coefficient from 1 to obtain the fourth similarity coefficient.
6. The artificial intelligence-based speech processing method according to claim 1, characterized in that, The multi-channel BP neural network consists of: 4 convolutional feature enhancement channels, a Concat layer, and a BP neural network; The first input of each convolutional feature enhancement channel is used to input a combined transfer ratio feature matrix, and the second input is used to input the same combined feature matching vector. The input of the Concat layer is connected to the output of the four convolutional feature enhancement channels, and its output is connected to the input of the BP neural network. The output of the BP neural network is used as the output of the multi-channel BP neural network.
7. The artificial intelligence-based speech processing method according to claim 6, characterized in that, Each convolutional feature enhancement channel includes: a first convolutional layer, a second convolutional layer, and a multiplier; The input of the first convolutional layer serves as the first input of the convolutional feature enhancement channel, and its output is connected to the input of the second convolutional layer. The first input of the multiplier is connected to the output of the second convolutional layer, its second input serves as the second input of the convolutional feature enhancement channel, and its output serves as the output of the convolutional feature enhancement channel.
8. The artificial intelligence-based speech processing method according to claim 7, characterized in that, The kernel size of the first convolutional layer is 1×4, and the kernel size of the second convolutional layer is 1×1. The first convolutional layer is used to convolve each column of the 4×N transfer ratio feature matrix to obtain 1×N convolutional compressed features. The second convolutional layer is used to extract features from the 1×N convolutional compressed features to obtain 1×N mapping features. At the multiplier, the 1×N mapping features are multiplied element-wise with the 1×N feature matching vector to obtain 1×N convolutional enhanced features, where N is the length.
Citation Information
Patent Citations
Human voice detection method and device
CN114242074A
Intelligent platform for appearance design
CN115168333A