Voice analysis method and device based on AI

Abstract

The invention relates to the technical field of artificial intelligence speech analysis, and discloses an AI-based speech analysis method and device, and the method comprises the steps: collecting the speech data of a user, separating the pronunciation and noise in the speech, and generating a first speech feature set; extracting voice features to form a second feature vector; performing dimension adjustment on the second feature vector to obtain a third feature vector; calculating an initial weight, and obtaining a first weight set; combining with historical voice data updating weights to form a second weight set; optimizing the weight and the path cost to generate a third weight set; a first analysis result is obtained by combining the analysis instruction intention and pitch and rhythm pause; the analysis logic is updated, the phoneme frequency and the voice energy mean value are fused, and a second analysis result is output; and finally generating a voice analysis report containing each characteristic index. According to the method, the weight distribution of the voice feature vector can be dynamically optimized based on the language habit of the user, and the logic is analyzed in combination with the instruction updated in real time.

Description

Technical Field

[0001] This invention relates to the field of speech analysis technology, and in particular to an AI-based speech analysis method and apparatus. Background Technology

[0002] In the field of artificial intelligence, speech analysis plays a crucial role in intelligent interaction, identity authentication, and personalized services. By capturing an individual's pronunciation, intonation, and expression habits, speech analysis can provide users with a precise interactive experience and is widely used in areas such as intelligent assistants, customer service systems, and medical auxiliary diagnosis.

[0003] In existing technologies, speech recognition often employs neural network models to automatically learn pronunciation rules from large amounts of speech data, thereby recognizing user-input speech. However, current speech analysis methods have significant limitations when handling complex speech scenarios. Existing solutions often struggle to adapt to changes in users' language habits in different contexts. For example, in noisy environments or during emotional fluctuations, speech feature extraction is easily interfered with, leading to unstable analysis results. Furthermore, traditional methods lack dynamic adaptation to users' long-term language habits when parsing instructions, making it difficult to accurately capture subtle differences in personalized expressions. This results in decreased accuracy in instruction parsing when faced with diverse speech inputs.

[0004] In summary, existing speech recognition technologies struggle to adapt to changes in users' language habits across different contexts, and the extraction of speech features is easily subject to interference, resulting in inaccurate speech analysis reports. Summary of the Invention

[0005] This invention provides an AI-based speech analysis method and apparatus to adapt to changes in users' language habits in different contexts and improve the accuracy of speech analysis reports.

[0006] Firstly, in order to solve the above-mentioned technical problems, the present invention provides an AI-based speech analysis method, comprising:

[0007] Acquire user voice data in the target scenario, separate the voice signal and background noise in the voice data to obtain a first voice feature set;

[0008] The second speech feature vector is obtained by extracting pronunciation clarity, intonation fluctuation amplitude, speech rate change rate and pitch fundamental frequency from the first speech feature set.

[0009] The contribution score is calculated based on the second speech feature vector, and the key features are filtered by combining the contribution score and the recursive feature elimination algorithm to obtain the third speech feature vector.

[0010] The weighted k-nearest neighbor algorithm is used to calculate the initial weights of the pronunciation clarity, intonation fluctuation amplitude, and speech rate change rate of the third speech feature vector, and the first weight allocation set is obtained.

[0011] Obtain the user's historical voice data in emotional fluctuation scenarios, and update the weights of pronunciation clarity and intonation fluctuation amplitude in combination with the first weight allocation set to obtain the second weight allocation set;

[0012] The intonation time series in the third speech feature vector is obtained. If the frequency domain feature value of the intonation time series exceeds the preset intonation change threshold, the intonation time series is aligned to obtain an aligned time series. The second weight allocation set is adjusted in combination with the aligned time series to obtain a third weight allocation set.

[0013] Combining the third weight allocation set and the third speech feature vector, a hidden Markov model is used to parse the instruction intent in the user's speech to obtain the first instruction parsing result;

[0014] If the tone time series matching degree between the first instruction parsing result and the historical instruction parsing result is lower than the preset matching threshold, the instruction parsing logic is updated through a Bayesian network to obtain the second instruction parsing result.

[0015] The second instruction parsing result is combined to generate a speech analysis report that includes instruction intent, pronunciation clarity, intonation fluctuation range, speech rate change rate, and rhythmic pause interval.

[0016] In one optional implementation, the step of acquiring the user's voice data in the target scene, separating the voice signal and background noise in the voice data to obtain a first voice feature set includes:

[0017] Collect voice data, and digitize the voice data to obtain a first voice dataset;

[0018] The time-domain signal of the first speech dataset is converted into a frequency-domain signal using Fourier transform. If there is high-frequency interference in the frequency-domain signal with a frequency higher than a preset frequency threshold, the high-frequency interference is removed by a low-pass filter to obtain the second speech dataset.

[0019] The speech signal and background noise of the second speech dataset are separated by the least mean square error algorithm to obtain the first speech feature set.

[0020] In one optional implementation, the step of extracting articulation, intonation fluctuation amplitude, speech rate change rate, and pitch fundamental frequency from the first speech feature set to obtain a second speech feature vector includes:

[0021] Extract a first feature vector set from the first speech feature set, which includes pronunciation clarity, intonation fluctuation amplitude, speech rate change, and pitch fundamental frequency;

[0022] The covariance matrix of the first eigenvector set is calculated using the principal component analysis algorithm. The eigenvalues ​​and eigenvectors corresponding to the principal components are obtained. The principal components with eigenvalues ​​higher than the preset contribution threshold are selected to obtain the second eigenvector set.

[0023] Linear discriminant analysis is used to classify the second feature vector set to determine whether the feature vectors meet the preset speech quality threshold requirements. If they do not meet the requirements, further denoising processing is performed on the second feature vector set. If they do meet the requirements, the second speech feature vector is obtained.

[0024] In one optional implementation, the step of calculating a contribution score based on the second speech feature vector, and then combining the contribution score with a recursive feature elimination algorithm to perform key feature filtering to obtain a third speech feature vector, includes:

[0025] The signal-to-noise ratio and harmonic noise ratio of the second speech feature vector are calculated using a recursive feature elimination algorithm to generate an initial contribution score and obtain a preliminary feature set.

[0026] The fundamental frequency variance and the rise and fall slope are extracted from the feature set initially screened, and the covariance matrix of the fundamental frequency variance and the rise and fall slope is calculated to obtain a quantitative representation of the pitch fluctuation amplitude.

[0027] If the quantitative representation of the amplitude of the tone fluctuation is greater than the preset amplitude threshold, the consistency of speech rate and rhythm is adjusted by weighting through a linear regression algorithm to generate a comprehensive contribution score and obtain a simplified feature set.

[0028] The principal component analysis algorithm is used to eliminate features from the feature vectors of the simplified feature set. Features with a comprehensive contribution score higher than a preset contribution threshold are selected to obtain the third speech feature vector.

[0029] In one optional implementation, the initial weights for the articulation clarity, intonation fluctuation amplitude, and speech rate change rate of the third speech feature vector are calculated using a weighted k-nearest neighbor algorithm to obtain a first weight allocation set, including:

[0030] The k-nearest neighbor algorithm is used to calculate the Euclidean distance between the feature vectors in the third speech feature vector, determine the initial weight of each feature vector, and obtain the initial weight allocation set.

[0031] The weight coefficients of the initial weight allocation set are adjusted by combining speech clarity, intonation fluctuation amplitude, and speech rate change rate to obtain a second initial weight allocation set;

[0032] If the weight of a feature in the second initial weight allocation set is lower than the preset weight threshold, the Euclidean distance is recalculated and the weights are reassigned through iterative optimization to obtain the first weight allocation set.

[0033] In one optional implementation, the step of acquiring the user's historical voice data under emotional fluctuation scenarios, and updating the weights of pronunciation clarity and intonation fluctuation amplitude in conjunction with the first weight allocation set to obtain a second weight allocation set includes:

[0034] Extract speech data features from historical speech data, denoise and segment them, classify them according to emotional fluctuation scenarios, obtain features of pronunciation clarity and intonation fluctuation amplitude, and obtain a preprocessed speech data feature set;

[0035] The weights of pronunciation clarity and intonation fluctuation in the preprocessed speech data feature set are initialized by combining the first weight allocation set. If the weight of the feature value exceeds the preset weight threshold, the weight is iteratively optimized by the gradient boosting algorithm to obtain the historical weight allocation set.

[0036] By combining the historical weight allocation set and the speech signal quality assessment, the speech data features are weighted. If the weighted feature values ​​meet the preset emotion recognition threshold, a classification label for the corresponding emotion fluctuation scenario is generated, and the classified speech data features are obtained.

[0037] The first weight allocation set is dynamically updated based on the characteristics of the classified speech data to obtain the second weight allocation set.

[0038] In one optional implementation, the step of obtaining the intonation time series in the third speech feature vector, if the frequency domain feature value of the intonation time series exceeds a preset intonation change threshold, involves aligning the intonation time series to obtain an aligned time series, and then adjusting the second weight allocation set in conjunction with the aligned time series to obtain a third weight allocation set, including:

[0039] The intonation time series is extracted from the third speech feature vector. If the feature value of the intonation time series exceeds the intonation change threshold, the intonation time series is aligned to obtain the aligned time series.

[0040] Based on the aligned time series, the weights of pronunciation clarity and intonation fluctuation in the second weight allocation set are adjusted to generate a third weight allocation set.

[0041] In one optional implementation, the step of combining the third weight allocation set and the third speech feature vector, and using a hidden Markov model to parse the instruction intent in the user's speech to obtain a first instruction parsing result, includes:

[0042] The state transition probability of the third speech feature vector is calculated using a hidden Markov model to obtain a temporal state sequence.

[0043] If the probability distribution of the time-series state sequence is greater than a preset distribution threshold, then the time-series state sequence is decoded in conjunction with the third weight allocation set to obtain an instruction candidate set.

[0044] Calculate the probability value of each candidate instruction in the instruction candidate set, sort them to obtain the instruction with the highest probability, and output the parsing result of the first instruction.

[0045] In one optional implementation, if the tone time series matching degree between the first instruction parsing result and the historical instruction parsing result is lower than a preset matching threshold, then updating the instruction parsing logic through a Bayesian network to obtain a second instruction parsing result includes:

[0046] The intonation time series, phoneme frequency distribution and speech energy mean are extracted from the first instruction parsing result to obtain a speech feature set;

[0047] If the matching degree between the intonation time series of the speech feature set and the intonation time series of historical instruction data is lower than a preset matching threshold, then the feature difference vector is determined by comparing the phoneme frequency distribution and the average speech energy.

[0048] Combining the aforementioned feature difference vectors with Bayesian network updates of instruction parsing logic, optimized instruction parsing logic is obtained;

[0049] The speech feature set is processed according to the optimized instruction parsing logic to generate a second instruction parsing result.

[0050] Secondly, the present invention provides an AI-based voice analysis device, comprising:

[0051] The data extraction module is used to acquire the user's voice data in the target scenario, separate the voice signal and background noise in the voice data, and obtain a first voice feature set;

[0052] The feature extraction module is used to extract pronunciation clarity, intonation fluctuation amplitude, speech rate change rate and pitch fundamental frequency from the first speech feature set to obtain the second speech feature vector;

[0053] The feature elimination module is used to calculate a contribution score based on the second speech feature vector, and to perform key feature filtering by combining the contribution score with a recursive feature elimination algorithm to obtain a third speech feature vector.

[0054] The weight allocation module is used to calculate the initial weights of the pronunciation clarity, intonation fluctuation amplitude and speech rate change rate of the third speech feature vector using the weighted k-nearest neighbor algorithm, and obtain the first weight allocation set;

[0055] The weight update module is used to obtain the user's historical voice data in emotional fluctuation scenarios, and update the weights of pronunciation clarity and intonation fluctuation amplitude in combination with the first weight allocation set to obtain the second weight allocation set;

[0056] The weight optimization module is used to obtain the intonation time series in the third speech feature vector. If the frequency domain feature value of the intonation time series exceeds the preset intonation change threshold, the intonation time series is aligned to obtain an aligned time series. The second weight allocation set is then adjusted in combination with the aligned time series to obtain a third weight allocation set.

[0057] The instruction parsing module is used to combine the third weight allocation set and the third speech feature vector, and use a hidden Markov model to parse the instruction intent in the user's speech to obtain the first instruction parsing result.

[0058] The instruction optimization module is used to update the instruction parsing logic through a Bayesian network to obtain the second instruction parsing result if the tone time series matching degree between the first instruction parsing result and the historical instruction parsing result is lower than a preset matching threshold.

[0059] The report output module is used to generate a speech analysis report that includes the instruction intent, pronunciation clarity, intonation fluctuation range, speech rate change rate, and rhythmic pause interval, based on the second instruction parsing result.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] (1) This invention separates the speech signal from the background noise in the speech data obtained in the target scene, and extracts features such as pronunciation clarity, pitch fluctuation amplitude, speech rate change rate and pitch fundamental frequency layer by layer. It can extract the effective speech features of users more accurately in noisy environments, thereby improving the accuracy and robustness of speech feature extraction.

[0062] (2) This invention selects key features by combining contribution scoring and recursive feature elimination algorithm, and uses weighted k-nearest neighbor algorithm to adaptively allocate weights for pronunciation clarity, intonation fluctuation amplitude and speech rate change rate. At the same time, it dynamically adjusts the weights by combining user historical speech data and intonation time series features, which can effectively improve the rationality and adaptability of feature weight allocation, thereby enhancing the system’s generalization ability in different users and different contexts.

[0063] (3) This invention combines Hidden Markov Model and Bayesian Network to parse and update the command intent in the user's speech, ensuring that self-learning and self-optimization can still be performed when the command intonation matching degree is insufficient, thereby improving the accuracy and stability of command parsing; and the final generated speech analysis report includes command intent and multi-dimensional speech feature indicators, providing more comprehensive data support for subsequent human-computer interaction and user status evaluation. Attached Figure Description

[0064] Figure 1 This is a schematic diagram of the AI-based speech analysis method provided in the first embodiment of the present invention;

[0065] Figure 2 This is a schematic diagram of the structure of the AI-based voice analysis device provided in the second embodiment of the present invention. Detailed Implementation

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] Reference Figure 1 The first embodiment of the present invention provides an AI-based speech analysis method, including the following steps:

[0068] S11, acquire the user's voice data in the target scenario, separate the voice signal and background noise in the voice data, and obtain the first voice feature set;

[0069] S12, extract pronunciation clarity, intonation fluctuation amplitude, speech rate change rate and pitch fundamental frequency from the first speech feature set to obtain the second speech feature vector;

[0070] S13, calculate the contribution score based on the second speech feature vector, and perform key feature screening by combining the contribution score and the recursive feature elimination algorithm to obtain the third speech feature vector;

[0071] S14, the weighted k-nearest neighbor algorithm is used to calculate the initial weights of the pronunciation clarity, intonation fluctuation amplitude and speech rate change rate of the third speech feature vector to obtain the first weight allocation set;

[0072] S15, acquire the user's historical voice data in emotional fluctuation scenarios, and update the weights of pronunciation clarity and intonation fluctuation amplitude in combination with the first weight allocation set to obtain the second weight allocation set;

[0073] S16, obtain the intonation time sequence in the third speech feature vector. If the frequency domain feature value of the intonation time sequence exceeds the preset intonation change threshold, align the intonation time sequence to obtain an aligned time sequence, and adjust the second weight allocation set in combination with the aligned time sequence to obtain a third weight allocation set.

[0074] S17, Combining the third weight allocation set and the third speech feature vector, a hidden Markov model is used to parse the instruction intent in the user's speech to obtain the first instruction parsing result;

[0075] S18, if the tone time series matching degree between the first instruction parsing result and the historical instruction parsing result is lower than the preset matching threshold, then the instruction parsing logic is updated through a Bayesian network to obtain the second instruction parsing result;

[0076] S19, combine the second instruction parsing results to generate a speech analysis report including instruction intent, pronunciation clarity, intonation fluctuation range, speech rate change rate, and rhythmic pause interval.

[0077] In step S11, the user's voice data in the target scene is acquired, and the voice signal and background noise in the voice data are separated to obtain a first voice feature set, including:

[0078] Collect voice data, and digitize the voice data to obtain a first voice dataset;

[0079] The time-domain signal of the first speech dataset is converted into a frequency-domain signal using Fourier transform. High-frequency interference with frequencies higher than a preset frequency threshold is removed by a low-pass filter, and low-frequency interference with frequencies lower than a preset frequency threshold is removed by a high-pass filter, thus obtaining the second speech dataset.

[0080] The speech signal and background noise of the second speech dataset are separated by the least mean square error algorithm to obtain the first speech feature set.

[0081] It should be noted that the target scenario refers to the user's voice acquisition environment, including quiet scenarios such as indoor offices, homes, and conference rooms—places with low background noise; and noisy scenarios such as subways, buses, shopping malls, and restaurants—environments with high background noise. The digital processing refers to converting continuous analog signals into discrete digital signal sequences using time-domain sampling methods. The resulting digital signals are time-domain signals, which need to be Fourier transformed to obtain frequency-domain signals to remove high-frequency noise.

[0082] It should also be noted that the core speech frequency band is typically between 300Hz and 4kHz. Therefore, the frequency threshold of the high-pass filter can be set to 200–300 Hz to filter out low-frequency noise; the frequency threshold of the low-pass filter can be set to 3400–4000 Hz to filter out high-frequency interference. The threshold can be adjusted within this range according to the application scenario, and this invention is not limited thereto. For example, in a shopping mall scenario, high-frequency air conditioning noise may be mixed in. After filtering, a cleaner second speech dataset can be obtained, retaining the main speech frequency band and improving the accuracy of subsequent processing. The low-pass filter removes high-frequency interference by setting a cutoff frequency (e.g., 8kHz). The high-pass filter is used to remove low-frequency interference. Assuming there is 50Hz power supply noise and 100Hz mechanical vibration noise in the environment, by designing a high-pass filter with a cutoff frequency of 200Hz, these low-frequency interferences can be effectively filtered out, retaining the core speech frequency band (typically between 300Hz and 4kHz).

[0083] Specifically, the minimum mean square error algorithm dynamically adjusts the filter coefficients W (weight vector) by minimizing the mean square error E between the target speech and the estimated speech, gradually enhancing the speech signal and suppressing continuous noise such as crowd noise, thereby separating the speech signal from the background noise.

[0084] In step S12, pronunciation clarity, intonation fluctuation amplitude, speech rate change rate, and pitch fundamental frequency are extracted from the first speech feature set to obtain a second speech feature vector, including:

[0085] Extract a first feature vector set from the first speech feature set, which includes pronunciation clarity, intonation fluctuation amplitude, speech rate change, and pitch fundamental frequency;

[0086] The covariance matrix of the first eigenvector set is calculated using the principal component analysis algorithm. The eigenvalues ​​and eigenvectors corresponding to the principal components are obtained. The principal components with eigenvalues ​​higher than the preset contribution threshold are selected to obtain the second eigenvector set.

[0087] Linear discriminant analysis is used to classify the second feature vector set to determine whether the feature vectors meet the preset speech quality threshold requirements. If they do not meet the requirements, further denoising processing is performed on the second feature vector set. If they do meet the requirements, the second speech feature vector is obtained.

[0088] It should be noted that speech intelligibility can be obtained by calculating the proportion of high-frequency energy, which is the percentage of the speech signal's energy in the high-frequency range within the total energy of the speech signal. Since speech intelligibility primarily depends on consonant information, and consonants are typically concentrated in the high-frequency range, and consonant frequencies are usually greater than 2kHz, we define speech signals greater than 2kHz as high-frequency energy. A higher proportion of high-frequency energy results in higher speech intelligibility. Pitch fluctuation amplitude is obtained by calculating the fundamental frequency over time, represented as a pitch time series. Speech rate variation is obtained by detecting energy abrupt changes between frames, represented as the variance of the energy in each frame; a larger variance indicates a faster speech rate. The fundamental frequency of pitch is extracted using the autocorrelation method. Given a frame of speech signal x(n), its autocorrelation function is defined as: ;in Indicates frame length; Indicates a delay; This represents the similarity between the signal and its own delayed value. It is calculated for each frame of the speech signal. Because it only represents energy level, not period. Human fundamental frequency is typically 50–400 Hz, and the sampling rate... Typically, 16kHz is used (which can cover 0–8 kHz, sufficient to preserve speech characteristics), therefore the corresponding delay range of the baseband is: , ,in It can be set to 400 Hz. It can be set to 50Hz. Within the range, find the maximum peak point of the autocorrelation function. , this That is, the fundamental frequency period Then the fundamental frequency of the pitch is The above features are summarized to obtain the first set of feature vectors.

[0089] It should also be noted that, when calculating the covariance matrix of the first set of eigenvectors, the eigenvalues ​​are sorted by size, and the principal components whose contribution rates are greater than a preset contribution threshold are selected. The preset contribution threshold can be between 85% and 95%, chosen according to the accuracy requirements of the scenario; 85% can be selected for low accuracy requirements, and 95% for high accuracy requirements. This invention does not impose any limitation. For example, through eigenvalue decomposition, after sorting, the principal components with a cumulative contribution rate of 85% are selected to generate the second eigenvector.

[0090] It should also be noted that the speech quality is obtained by classifying the second feature vector set using linear discriminant analysis (LDA), and is represented by a clarity score. Assuming a preset speech quality threshold of 80 points (out of 100), LDA classifies the feature vectors into "high quality" and "low quality" categories by maximizing inter-class variance and minimizing intra-class variance. The feature vectors of the low-quality category require further denoising processing using a filter, and are then merged with the feature vectors of the high-quality category to obtain the second speech feature vector.

[0091] In step S13, a contribution score is calculated based on the second speech feature vector. The contribution score is then combined with a recursive feature elimination algorithm to perform key feature filtering, resulting in a third speech feature vector, including:

[0092] The signal-to-noise ratio and harmonic noise ratio of the second speech feature vector are calculated using a recursive feature elimination algorithm to generate an initial contribution score and obtain a preliminary feature set.

[0093] The fundamental frequency variance and the rise and fall slope are extracted from the feature set initially screened, and the covariance matrix of the fundamental frequency variance and the rise and fall slope is calculated to obtain a quantitative representation of the pitch fluctuation amplitude.

[0094] If the quantitative representation of the amplitude of the tone fluctuation is greater than the preset amplitude threshold, the consistency of speech rate and rhythm is adjusted by weighting through a linear regression algorithm to generate a comprehensive contribution score and obtain a simplified feature set.

[0095] The principal component analysis algorithm is used to eliminate features from the feature vectors of the simplified feature set. Features with a comprehensive contribution score higher than a preset contribution threshold are selected to obtain the third speech feature vector.

[0096] It should be noted that signal-to-noise ratio (SNR) measures the ratio of speech components to noise components in a speech signal, while harmonic noise ratio (HNR) measures the ratio of periodic (harmonic) to non-periodic (noise) components in the speech signal. An initial contribution score for each feature is obtained by weighting the SNR and HNR with preset weights. Features with initial contribution scores below a preset contribution threshold are iteratively eliminated to obtain a preliminarily selected feature set. For example, when calculating SNR, the energy ratio of the speech signal to the background noise is analyzed, and a value of 20dB indicates that the speech signal strength far exceeds the noise. HNR is calculated by the ratio of harmonic components to noise components. Assuming that the harmonic energy is concentrated in the 300Hz-2kHz range and the noise is mainly high-frequency random interference, a HNR of 15dB is obtained. These indicators are weighted to generate an initial contribution score. For example, with an SNR weight of 0.6 and a HNR weight of 0.4, the overall score is 18, which is higher than the preset threshold of 15, so the feature is retained.

[0097] It should also be noted that the fundamental frequency of pitch is extracted from the initially screened feature set, and the variance of the fundamental frequency is calculated, i.e., the fundamental frequency variance. The fundamental frequency variance reflects the stability of pitch. For example, if the fundamental frequency fluctuates between 200Hz and 250Hz, a variance of 25Hz² indicates that the pitch is relatively stable. The rise and fall slope quantifies the speed of pitch change. For example, if the fundamental frequency rises by 50Hz per second, it indicates that the pitch is gradually rising. The covariance matrix of the fundamental frequency variance and the rise and fall slope is calculated. The matrix element Cij represents the correlation between the fundamental frequency variance and the rise and fall slope, i.e., a quantitative representation of the amplitude of pitch fluctuation.

[0098] It should also be noted that if the amplitude of intonation fluctuation exceeds a preset threshold, the contribution score needs to be adjusted in conjunction with the consistency of speech rate and tempo, using the variance of energy per frame to reflect the consistency of speech rate and tempo. Specifically, the variance of energy per frame is used as the independent variable, and the amplitude of intonation fluctuation is used as the dependent variable; linear regression is used to fit the data to obtain weight coefficients; these coefficients are then applied to the contribution score to obtain a comprehensive contribution score. Finally, features with comprehensive contribution scores higher than a preset contribution threshold are selected to obtain the third speech feature vector.

[0099] In step S14, the weighted k-nearest neighbor algorithm is used to calculate the initial weights of the pronunciation clarity, intonation fluctuation amplitude, and speech rate change rate of the third speech feature vector, resulting in a first weight allocation set, including:

[0100] The k-nearest neighbor algorithm is used to calculate the Euclidean distance between the feature vectors in the third speech feature vector, determine the initial weight of each feature vector, and obtain the initial weight allocation set.

[0101] The weight coefficients of the initial weight allocation set are adjusted by combining speech clarity, intonation fluctuation amplitude, and speech rate change rate to obtain a second initial weight allocation set;

[0102] If the weight of a feature in the second initial weight allocation set is lower than the preset weight threshold, the Euclidean distance is recalculated and the weights are reassigned through iterative optimization to obtain the first weight allocation set.

[0103] For example, there are 100 speech samples, each a 12-dimensional vector. The k-value in the k-nearest neighbor algorithm is set to 5. The similarity between samples is measured by calculating the Euclidean distance between each sample and its five nearest neighbors. Features with smaller distances indicate higher similarity and can be considered to have more concentrated feature contributions; features with larger distances may be more unique or contain more information. Initial weights are determined based on the distances, with smaller distances resulting in lower weights. The initial weights reflect the contribution of spectral features to speech classification; for example, spectral energy has a weight of 0.4, pitch has a weight of 0.3, and syllable count has a weight of 0.2.

[0104] It should also be noted that speech intelligibility, intonation fluctuation, and speech rate variation contribute differently to the final speech quality. For example, features that are far apart may contribute little to intelligibility, thus requiring a lower weight. For instance, speech intelligibility is measured by the energy proportion of the high-frequency portion of the spectrum; a high-frequency energy proportion exceeding 60% indicates high intelligibility, meaning a high contribution, and thus requires a higher weight. Intonation fluctuation is calculated using the standard deviation of the pitch curve; for example, a standard deviation between 50-100Hz indicates a stable pitch, indicating a high contribution, and thus requires a higher weight. Speech rate variation is estimated by the change in the number of syllables over time; for example, a change in the number of syllables per second of less than 1 indicates a stable speech rate, meaning a high contribution, and thus requires a higher weight.

[0105] It should also be noted that if any weights after adjustment fall below a preset weight threshold, the weights need to be reallocated. Specifically, the weight allocation can be updated by recalculating the Euclidean distance by changing the k value, increasing the k value by a preset step size. For example, if the preset step size is set to 2, then the k value is adjusted to 7, and the weight allocation is updated again. In other embodiments, the preset step size can be set to 0.5, 1, 1.5, etc., and this invention does not limit this. This process is repeated until no weights fall below the preset weight threshold, preventing a single feature from being ignored.

[0106] In step S15, historical voice data of the user in emotional fluctuation scenarios is obtained, and the weights of pronunciation clarity and intonation fluctuation amplitude are updated in combination with the first weight allocation set to obtain the second weight allocation set, including:

[0107] Extract speech data features from historical speech data, denoise and segment them, classify them according to emotional fluctuation scenarios, obtain features of pronunciation clarity and intonation fluctuation amplitude, and obtain a preprocessed speech data feature set;

[0108] The weights of pronunciation clarity and intonation fluctuation in the preprocessed speech data feature set are initialized by combining the first weight allocation set. If the weight of the feature value exceeds the preset weight threshold, the weight is iteratively optimized by the gradient boosting algorithm to obtain the historical weight allocation set.

[0109] By combining the historical weight allocation set and the speech signal quality assessment, the speech data features are weighted. If the weighted feature values ​​meet the preset emotion recognition threshold, a classification label for the corresponding emotion fluctuation scenario is generated, and the classified speech data features are obtained.

[0110] The first weight allocation set is dynamically updated based on the characteristics of the classified speech data to obtain the second weight allocation set.

[0111] It should be noted that the denoising and segmentation refer to removing high-frequency and low-frequency interference from the speech data through filters, and then performing frame segmentation to facilitate subsequent extraction of speech features.

[0112] It should also be noted that the classification based on emotional fluctuation scenarios refers to judging the emotion of the speech data based on the amplitude of pitch fluctuations. For example, a pitch standard deviation of 50Hz for a certain speech segment indicates a large pitch fluctuation, which may correspond to an excited emotion; while a standard deviation of 20Hz may correspond to a calm emotion.

[0113] After initializing the weights of the preprocessed speech data feature set, if any feature value's weight exceeds a preset weight threshold, the weights need to be iteratively optimized using a gradient boosting algorithm. Specifically, the objective function is defined as follows: ,in Represents true emotion recognition. This represents the prediction result obtained based on the current feature weights. The loss function is typically chosen as the mean squared error function. In gradient boosting, the gradient of the current model with respect to the loss function is calculated in each round, yielding the residual vector. ,in The number of rounds. A weak learner (such as a regression tree) is used to fit the residual vector. The correction direction is obtained. Finally, the weights are updated based on the learning rate and the correction direction, and the above operation is repeated until no weight exceeds the preset weight threshold.

[0114] It should also be noted that after adjusting the weights, the weighted feature values ​​are determined to correspond to a specific emotion type. For example, if the weighted feature values ​​of a certain speech segment indicate "anger," a corresponding label is generated. After obtaining the classified speech data features, the weights are further adjusted using the gradient boosting algorithm described above based on the corresponding emotion label. Specifically, for example, if the emotion label of a certain speech segment is detected as "anger," the intonation weight is dynamically adjusted from 0.35 to 0.45 to more accurately reflect the trend of emotion changes.

[0115] In step S16, the intonation time series in the third speech feature vector is obtained. If the frequency domain feature value of the intonation time series exceeds a preset intonation change threshold, the intonation time series is aligned to obtain an aligned time series. The second weight allocation set is then adjusted based on the aligned time series to obtain a third weight allocation set, including:

[0116] The intonation time series is extracted from the third speech feature vector. If the feature value of the intonation time series exceeds the intonation change threshold, the intonation time series is aligned to obtain the aligned time series.

[0117] Based on the aligned time series, the weights of pronunciation clarity and intonation fluctuation in the second weight allocation set are adjusted to generate a third weight allocation set.

[0118] It should be noted that the intonation time series reflects the amplitude of intonation changes. If the amplitude of intonation changes exceeds the preset intonation change threshold, an intonation fluctuation detection result is generated, and the intonation time series is marked as "high fluctuation," requiring sequence alignment. For example, if the intonation change threshold is set to a dominant frequency energy of 10Hz, after transformation, if the dominant frequency energy of a certain intonation time series is concentrated between 2-5Hz, it indicates a relatively gentle intonation change, possibly corresponding to a calm emotion; if the dominant frequency energy is distributed between 8-12Hz, it may indicate an excited emotion.

[0119] It should also be noted that dynamic time warping is used to achieve sequence alignment. Specifically, the Euclidean distance between each element of the two time series is calculated to construct a distance matrix; by calculating the minimum cumulative distance matrix, the minimum cumulative distance and alignment path are obtained. The minimum cumulative distance is the alignment error. For example, considering the intonation curves of two speech segments, one spoken quickly and the other slowly, dynamic time warping can stretch or compress the sequences to align key peaks and troughs, thereby eliminating the impact of speech rate differences. Through sequence alignment, differences in speech rate and rhythm are eliminated, and then the weights of articulation and intonation fluctuation amplitude are dynamically adjusted based on the alignment results to generate a more accurate weight set.

[0120] After alignment, the energy concentration of the intonation time series is calculated to determine if there are obvious energy peaks, and the weight of pronunciation clarity is adjusted accordingly. Specifically, the sequence is divided into frames of 20-30 ms each. The frame energy of each frame is calculated, and then the total energy of the entire sequence is calculated. Finally, the ratio of the frame energy corresponding to the highest energy frame to the total energy is calculated; this ratio is the energy concentration. The closer the energy concentration is to 1, the more concentrated the energy and the clearer the pronunciation. If the energy concentration is greater than 0.6, the pronunciation clarity weight increases by 0.05; if the energy concentration is less than 0.3, the pronunciation clarity weight decreases by 0.05; if the energy concentration is between 0.3 and 0.6, the pronunciation clarity weight remains unchanged.

[0121] After alignment, the extreme value difference of the intonation time series is calculated, and the extreme value difference is normalized with a normalization magnitude of . ,in These represent the maximum and minimum amplitude values ​​in the intonation time series. The normalization magnitude is the mean. If the normalization magnitude is greater than 0.3, the weight of intonation fluctuation magnitude increases by 0.05; if the normalization magnitude is less than 0.15, the weight of intonation fluctuation magnitude decreases by 0.05; if the normalization magnitude is between 0.15 and 0.3, the weight of intonation fluctuation magnitude remains unchanged.

[0122] For example, the intonation time series of two speech segments are 5 seconds and 6 seconds respectively. After alignment using dynamic time warping, the alignment error is 0.2 seconds, and the path optimization cost is 0.15. In the second weight set, the weight for pronunciation clarity is 0.65, and the weight for intonation fluctuation amplitude is 0.35. Based on the increased intonation fluctuation amplitude detected in the aligned intonation time series, the weights are adjusted to pronunciation clarity 0.6 and intonation fluctuation amplitude 0.4.

[0123] In step S17, combining the third weight allocation set and the third speech feature vector, a hidden Markov model is used to parse the instruction intent in the user's speech to obtain the first instruction parsing result, including:

[0124] The state transition probability of the third speech feature vector is calculated using a hidden Markov model to obtain a temporal state sequence.

[0125] If the probability distribution of the time-series state sequence is greater than a preset distribution threshold, then the time-series state sequence is decoded in conjunction with the third weight allocation set to obtain an instruction candidate set.

[0126] Calculate the probability value of each candidate instruction in the instruction candidate set, sort them to obtain the instruction with the highest probability, and output the parsing result of the first instruction.

[0127] It should be noted that when using a Hidden Markov Model to calculate state transition probabilities, the speech signal can be modeled into multiple states, such as "instruction words," "parameter words," and "pauses." The transition probability from "instruction words" to "parameter words" is calculated by training the model, resulting in a temporal state sequence. If the probability distribution of the temporal state sequence is greater than a preset distribution threshold, then decoding is performed using the third weight allocation set. For example, if the user instruction is "turn on the living room light" or "turn off the bedroom light," different weights can be assigned to "turn on" and "turn off" using the third weight allocation set, such as a weight of 0.7 for "turn on" and 0.3 for "turn off," generating a set of instruction candidates, such as "turn on the living room light" and "turn on the bedroom light."

[0128] It should also be noted that the probability value of each candidate instruction is calculated using the maximum a posteriori probability method, and the instruction with the highest probability is selected as the first instruction parsing result. For example, a user often says "turn on the living room light" at night, and its posterior probability is 0.85, higher than the 0.6 of "turn on the bedroom light". The final output first instruction parsing result is "turn on the living room light". The first instruction parsing result will include the feature analysis results of the user's voice input, including the pitch time series, phoneme frequency distribution, and mean speech energy.

[0129] In step S18, if the tone time series matching degree between the first instruction parsing result and the historical instruction parsing result is lower than a preset matching threshold, the instruction parsing logic is updated through a Bayesian network to obtain a second instruction parsing result, including:

[0130] The intonation time series, phoneme frequency distribution and speech energy mean are extracted from the first instruction parsing result to obtain a speech feature set;

[0131] If the matching degree between the intonation time series of the speech feature set and the intonation time series of historical instruction data is lower than a preset matching threshold, then the feature difference vector is determined by comparing the phoneme frequency distribution and the average speech energy.

[0132] Combining the aforementioned feature difference vectors with Bayesian network updates of instruction parsing logic, optimized instruction parsing logic is obtained;

[0133] The speech feature set is processed according to the optimized instruction parsing logic to generate a second instruction parsing result.

[0134] It should be noted that the intonation time series reflects the pattern of intonation changes over time (tone, pitch rise and fall trends); the phoneme frequency distribution statistics show the probability and distribution of each phoneme during pronunciation; and the mean speech energy reflects the loudness or intensity of speech, used to distinguish between unstressed and stressed syllables. The intonation time series matching degree is obtained by calculating the minimum alignment distance; the closer it is to 1 after normalization, the more similar the intonations. The phoneme frequency distribution matching degree can be obtained by calculating the cosine similarity; the closer the cosine similarity is to 1, the higher the matching degree. The mean speech energy is a scalar and can be directly measured by the difference. The matching degrees of these three items are weighted and fused with preset weights to obtain the comprehensive matching degree. If the comprehensive matching degree is lower than the preset matching threshold, it is necessary to determine the feature difference vector to optimize the instruction.

[0135] Specifically, the generation of the feature difference vector relies on a quantitative comparison of phoneme frequency distribution and average speech energy. For example, the system might find that the frequency of the "t" phoneme in the current speech is 10% higher than in historical data, while the average energy is 5 dB higher. These differences are integrated into a multidimensional vector, namely the feature difference vector.

[0136] The feature difference vector is input into a Bayesian network, which integrates the conditional probability relationships between different features to infer the most likely true instruction. The training data for the Bayesian network consists of a large number of speech instruction samples (audio + correctly labeled instructions). Each sample corresponds to: pitch time series, phoneme frequency distribution, average speech energy, first instruction parsing result, and the actual labeled true instruction. The state is defined as follows: Input feature nodes: pitch difference (low / medium / high), phoneme difference (low / medium / high), energy difference (low / medium / high); latent intermediate nodes: pronunciation clarity (clear / vague), speech emotional state (stable / excited / fatigued), environmental noise influence (low / high); output nodes: directly modeled as the "most likely true instruction category" (turn on lights / turn off lights / turn on TV, etc.). The prior probability and conditional probability table (CPT) are derived from historical data. For example, prior probabilities: P(intonation difference = high) = 0.2, P(phoneme difference = low) = 0.5, P(energy difference = medium) = 0.3; CPT: P(correct instruction | intonation difference = high, phoneme difference = low, energy difference = low) = 0.85, P(correct instruction | intonation difference = high, phoneme difference = high, energy difference = high) = 0.15, P(correct instruction | intonation difference = low, phoneme difference = low, energy difference = medium) = 0.90.

[0137] Updating the parsing logic means adjusting the system's decision path during parsing, such as increasing sensitivity to differences in phoneme distribution and reducing the weight of the average energy. The instructions are optimized based on the instruction parsing logic obtained from the Bayesian network to obtain the parsing result of the second instruction.

[0138] In step S19, a speech analysis report is generated by combining the second instruction parsing results, which includes instruction intent, pronunciation clarity, intonation fluctuation amplitude, speech rate change rate, and rhythmic pause interval.

[0139] It should be noted that a weighted average algorithm is used to calculate the prosodic pause interval by combining pronunciation clarity, intonation fluctuation amplitude, speech rate variation rate, and pause duration distribution. For example, assigning a weight of 0.4 to pronunciation clarity, 0.3 to intonation fluctuation amplitude, 0.2 to speech rate variation rate, and 0.1 to pause distribution, the calculated prosodic pause interval is 0.25 seconds. The speech analysis report is obtained by summarizing the instruction intent, pronunciation clarity, intonation fluctuation amplitude, speech rate variation rate, and prosodic pause interval.

[0140] In summary, this invention separates the speech signal from the background noise in the speech data acquired in the target scene, and extracts features such as pronunciation clarity, pitch fluctuation amplitude, speech rate change rate, and pitch fundamental frequency layer by layer. This enables more accurate extraction of effective speech features from users in noisy environments, thereby improving the accuracy and robustness of speech feature extraction. Furthermore, this invention combines contribution scoring and recursive feature elimination algorithms for key feature selection, and utilizes a weighted k-nearest neighbor algorithm to adaptively allocate weights for pronunciation clarity, pitch fluctuation amplitude, and speech rate change rate. Simultaneously, it dynamically adjusts these weights by incorporating users' historical speech data and pitch time-series features. This effectively improves the rationality and adaptability of feature weight allocation, thereby enhancing the system's generalization ability across different users and contexts. This invention combines Hidden Markov Models and Bayesian Networks to parse and update the command intent in user speech, ensuring self-learning and self-optimization even when the command intonation matching is insufficient, thereby improving the accuracy and stability of command parsing. Furthermore, the final generated speech analysis report includes command intent and multi-dimensional speech feature indicators, providing more comprehensive data support for subsequent human-computer interaction and user status assessment.

[0141] Reference Figure 2 The second embodiment of the present invention provides an AI-based voice analysis device, comprising:

[0142] The data extraction module is used to acquire the user's voice data in the target scenario, separate the voice signal and background noise in the voice data, and obtain a first voice feature set;

[0143] The feature extraction module is used to extract pronunciation clarity, intonation fluctuation amplitude, speech rate change rate and pitch fundamental frequency from the first speech feature set to obtain the second speech feature vector;

[0144] The feature elimination module is used to calculate a contribution score based on the second speech feature vector, and to perform key feature filtering by combining the contribution score with a recursive feature elimination algorithm to obtain a third speech feature vector.

[0145] The weight allocation module is used to calculate the initial weights of the pronunciation clarity, intonation fluctuation amplitude and speech rate change rate of the third speech feature vector using the weighted k-nearest neighbor algorithm, and obtain the first weight allocation set;

[0146] The weight update module is used to obtain the user's historical voice data in emotional fluctuation scenarios, and update the weights of pronunciation clarity and intonation fluctuation amplitude in combination with the first weight allocation set to obtain the second weight allocation set;

[0147] The weight optimization module is used to obtain the intonation time series in the third speech feature vector. If the frequency domain feature value of the intonation time series exceeds the preset intonation change threshold, the intonation time series is aligned to obtain an aligned time series. The second weight allocation set is then adjusted in combination with the aligned time series to obtain a third weight allocation set.

[0148] The instruction parsing module is used to combine the third weight allocation set and the third speech feature vector, and use a hidden Markov model to parse the instruction intent in the user's speech to obtain the first instruction parsing result.

[0149] The instruction optimization module is used to update the instruction parsing logic through a Bayesian network to obtain the second instruction parsing result if the tone time series matching degree between the first instruction parsing result and the historical instruction parsing result is lower than a preset matching threshold.

[0150] The report output module is used to generate a speech analysis report that includes the instruction intent, pronunciation clarity, intonation fluctuation range, speech rate change rate, and rhythmic pause interval, based on the second instruction parsing result.

[0151] It should be noted that the AI-based speech analysis device provided in this embodiment of the invention is used to execute all the process steps of the AI-based speech analysis method in the above embodiment. The working principles and beneficial effects of the two are one-to-one, so they will not be described again.

[0152] This invention also provides an electronic device. The electronic device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a weight update program. When the processor executes the computer program, it implements the steps described in the various AI-based speech analysis method embodiments above, for example... Figure 1 The step S11 shown. Alternatively, when the processor executes the computer program, it implements the functions of each module / unit in the above-described device embodiments, such as the data extraction module.

[0153] For example, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.

[0154] The electronic device may be a desktop computer, laptop, handheld computer, or smart tablet, etc. The electronic device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above components are merely examples of electronic devices and do not constitute a limitation on the electronic device. It may include more or fewer components than described above, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0155] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the electronic device, connecting all parts of the electronic device via various interfaces and lines.

[0156] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0157] Wherein, if the modules / units integrated in the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0158] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0159] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. An AI-based speech analysis method, characterized in that, include: Acquire user voice data in the target scenario, separate the voice signal and background noise in the voice data to obtain a first voice feature set; The second speech feature vector is obtained by extracting pronunciation clarity, intonation fluctuation amplitude, speech rate change rate and pitch fundamental frequency from the first speech feature set. The contribution score is calculated based on the second speech feature vector, and the key features are filtered by combining the contribution score and the recursive feature elimination algorithm to obtain the third speech feature vector. The weighted k-nearest neighbor algorithm is used to calculate the initial weights of the pronunciation clarity, intonation fluctuation amplitude, and speech rate change rate of the third speech feature vector, and the first weight allocation set is obtained. Obtain the user's historical voice data in emotional fluctuation scenarios, and update the weights of pronunciation clarity and intonation fluctuation amplitude in combination with the first weight allocation set to obtain the second weight allocation set; The intonation time series in the third speech feature vector is obtained. If the frequency domain feature value of the intonation time series exceeds the preset intonation change threshold, the intonation time series is aligned to obtain an aligned time series. The second weight allocation set is adjusted in combination with the aligned time series to obtain a third weight allocation set. Combining the third weight allocation set and the third speech feature vector, a hidden Markov model is used to parse the instruction intent in the user's speech to obtain the first instruction parsing result; If the tone time series matching degree between the first instruction parsing result and the historical instruction parsing result is lower than the preset matching threshold, the instruction parsing logic is updated through a Bayesian network to obtain the second instruction parsing result. The second instruction parsing result is combined to generate a speech analysis report that includes instruction intent, pronunciation clarity, intonation fluctuation range, speech rate change rate, and rhythmic pause interval.

2. The AI-based speech analysis method according to claim 1, characterized in that, The step of acquiring user voice data in a target scenario, separating the voice signal and background noise in the voice data, and obtaining a first voice feature set includes: Collect voice data, and digitize the voice data to obtain a first voice dataset; The time-domain signal of the first speech dataset is converted into a frequency-domain signal using Fourier transform. High-frequency interference with frequencies higher than a preset frequency threshold is removed by a low-pass filter, and low-frequency interference with frequencies lower than a preset frequency threshold is removed by a high-pass filter, thus obtaining the second speech dataset. The speech signal and background noise of the second speech dataset are separated by the least mean square error algorithm to obtain the first speech feature set.

3. The AI-based speech analysis method according to claim 1, characterized in that, The second speech feature vector is obtained by extracting pronunciation clarity, intonation fluctuation amplitude, speech rate change rate, and pitch fundamental frequency from the first speech feature set, including: Extract a first feature vector set from the first speech feature set, which includes pronunciation clarity, intonation fluctuation amplitude, speech rate change, and pitch fundamental frequency; The covariance matrix of the first eigenvector set is calculated using the principal component analysis algorithm. The eigenvalues ​​and eigenvectors corresponding to the principal components are obtained. The principal components with eigenvalues ​​higher than the preset contribution threshold are selected to obtain the second eigenvector set. Linear discriminant analysis is used to classify the second feature vector set to determine whether the feature vectors meet the preset speech quality threshold requirements. If they do not meet the requirements, further denoising processing is performed on the second feature vector set. If they do meet the requirements, the second speech feature vector is obtained.

4. The AI-based speech analysis method according to claim 1, characterized in that, The third speech feature vector is obtained by calculating a contribution score based on the second speech feature vector, and then combining the contribution score with a recursive feature elimination algorithm to perform key feature filtering, including: The signal-to-noise ratio and harmonic noise ratio of the second speech feature vector are calculated using a recursive feature elimination algorithm to generate an initial contribution score and obtain a preliminary feature set. The fundamental frequency variance and the rise and fall slope are extracted from the feature set initially screened, and the covariance matrix of the fundamental frequency variance and the rise and fall slope is calculated to obtain a quantitative representation of the pitch fluctuation amplitude. If the quantitative representation of the amplitude of the tone fluctuation is greater than the preset amplitude threshold, the consistency of speech rate and rhythm is adjusted by weighting through a linear regression algorithm to generate a comprehensive contribution score and obtain a simplified feature set. The principal component analysis algorithm is used to eliminate features from the feature vectors of the simplified feature set. Features with a comprehensive contribution score higher than a preset contribution threshold are selected to obtain the third speech feature vector.

5. The AI-based speech analysis method according to claim 1, characterized in that, The weighted k-nearest neighbor algorithm is used to calculate the initial weights of the pronunciation clarity, intonation fluctuation amplitude, and speech rate change rate of the third speech feature vector, resulting in a first weight allocation set, including: The k-nearest neighbor algorithm is used to calculate the Euclidean distance between the feature vectors in the third speech feature vector, determine the initial weight of each feature vector, and obtain the initial weight allocation set. The weight coefficients of the initial weight allocation set are adjusted by combining speech clarity, intonation fluctuation amplitude, and speech rate change rate to obtain a second initial weight allocation set; If the weight of a feature in the second initial weight allocation set is lower than the preset weight threshold, the Euclidean distance is recalculated and the weights are reassigned through iterative optimization to obtain the first weight allocation set.

6. The AI-based speech analysis method according to claim 1, characterized in that, The process involves acquiring historical voice data of users in emotionally fluctuating scenarios, updating the weights of pronunciation clarity and intonation fluctuation amplitude using the first weight allocation set, and obtaining a second weight allocation set, including: Extract speech data features from historical speech data, denoise and segment them, classify them according to emotional fluctuation scenarios, obtain features of pronunciation clarity and intonation fluctuation amplitude, and obtain a preprocessed speech data feature set; The weights of pronunciation clarity and intonation fluctuation in the preprocessed speech data feature set are initialized by combining the first weight allocation set. If the weight of the feature value exceeds the preset weight threshold, the weight is iteratively optimized by the gradient boosting algorithm to obtain the historical weight allocation set. By combining the historical weight allocation set and the speech signal quality assessment, the speech data features are weighted. If the weighted feature values ​​meet the preset emotion recognition threshold, a classification label for the corresponding emotion fluctuation scenario is generated, and the classified speech data features are obtained. The first weight allocation set is dynamically updated based on the characteristics of the classified speech data to obtain the second weight allocation set.

7. The AI-based speech analysis method according to claim 1, characterized in that, The step involves obtaining the intonation time series from the third speech feature vector. If the frequency domain feature value of the intonation time series exceeds a preset intonation change threshold, the intonation time series is aligned to obtain an aligned time series. The aligned time series is then used to adjust the second weight allocation set to obtain a third weight allocation set, which includes: The intonation time series is extracted from the third speech feature vector. If the feature value of the intonation time series exceeds the intonation change threshold, the intonation time series is aligned to obtain the aligned time series. Based on the aligned time series, the weights of pronunciation clarity and intonation fluctuation in the second weight allocation set are adjusted to generate a third weight allocation set.

8. The AI-based speech analysis method according to claim 1, characterized in that, The process of combining the third weight allocation set and the third speech feature vector, and using a Hidden Markov Model to parse the instruction intent in the user's speech to obtain the first instruction parsing result includes: The state transition probability of the third speech feature vector is calculated using a hidden Markov model to obtain a temporal state sequence. If the probability distribution of the time-series state sequence is greater than a preset distribution threshold, then the time-series state sequence is decoded in conjunction with the third weight allocation set to obtain an instruction candidate set. Calculate the probability value of each candidate instruction in the instruction candidate set, sort them to obtain the instruction with the highest probability, and output the parsing result of the first instruction.

9. The AI-based speech analysis method according to claim 1, characterized in that, If the tone time series matching degree between the first instruction parsing result and the historical instruction parsing result is lower than a preset matching threshold, then the instruction parsing logic is updated through a Bayesian network to obtain a second instruction parsing result, including: The intonation time series, phoneme frequency distribution and speech energy mean are extracted from the first instruction parsing result to obtain a speech feature set; If the matching degree between the intonation time series of the speech feature set and the intonation time series of historical instruction data is lower than a preset matching threshold, then the feature difference vector is determined by comparing the phoneme frequency distribution and the average speech energy. Combining the aforementioned feature difference vectors with Bayesian network updates of instruction parsing logic, optimized instruction parsing logic is obtained; The speech feature set is processed according to the optimized instruction parsing logic to generate a second instruction parsing result.

10. An AI-based voice analysis device, characterized in that, include: The data extraction module is used to acquire the user's voice data in the target scenario, separate the voice signal and background noise in the voice data, and obtain a first voice feature set; The feature extraction module is used to extract pronunciation clarity, intonation fluctuation amplitude, speech rate change rate and pitch fundamental frequency from the first speech feature set to obtain the second speech feature vector; The feature elimination module is used to calculate a contribution score based on the second speech feature vector, and to perform key feature filtering by combining the contribution score with a recursive feature elimination algorithm to obtain a third speech feature vector. The weight allocation module is used to calculate the initial weights of the pronunciation clarity, intonation fluctuation amplitude and speech rate change rate of the third speech feature vector using the weighted k-nearest neighbor algorithm, and obtain the first weight allocation set; The weight update module is used to obtain the user's historical voice data in emotional fluctuation scenarios, and update the weights of pronunciation clarity and intonation fluctuation amplitude in combination with the first weight allocation set to obtain the second weight allocation set; The weight optimization module is used to obtain the intonation time series in the third speech feature vector. If the frequency domain feature value of the intonation time series exceeds the preset intonation change threshold, the intonation time series is aligned to obtain an aligned time series. The second weight allocation set is then adjusted in combination with the aligned time series to obtain a third weight allocation set. The instruction parsing module is used to combine the third weight allocation set and the third speech feature vector, and use a hidden Markov model to parse the instruction intent in the user's speech to obtain the first instruction parsing result. The instruction optimization module is used to update the instruction parsing logic through a Bayesian network to obtain the second instruction parsing result if the tone time series matching degree between the first instruction parsing result and the historical instruction parsing result is lower than a preset matching threshold. The report output module is used to generate a speech analysis report that includes the instruction intent, pronunciation clarity, intonation fluctuation range, speech rate change rate, and rhythmic pause interval, based on the second instruction parsing result.

Images