Incremental ASR algorithm suitable for PSTN call
By using an incremental ASR algorithm to process PSTN calls in real time, the response delay and dynamic adaptation problems of traditional ASR systems are solved, achieving fast and accurate speech recognition and improving the user experience.
Patent Information
- Application Number
- CN202511149673.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-18
- Publication Date
- 2025-11-11
AI Technical Summary
Traditional ASR systems suffer from response delays and an inability to adapt to the dynamic nature of spoken communication during PSTN calls, resulting in poor user experience and poor interactivity.
An incremental ASR algorithm is adopted, which performs speech recognition in real time through data preprocessing, feature extraction and normalization, LSTM model processing, partial result output judgment and output buffer management, thereby reducing latency and improving accuracy.
It significantly reduces speech recognition latency, improves response speed and the reliability of recognition results, adapts to the dynamic nature of spoken communication, and enhances the user experience.
Smart Images
Figure CN120932636A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of public switched telephone network (PSTN) calls, and specifically relates to an incremental ASR algorithm adapted to PSTN calls. Background Technology
[0002] In traditional Automatic Speech Recognition (ASR) systems, when processing calls from the Public Switched Telephone Network (PSTN), speech recognition processing typically begins only after the user has finished speaking. While this approach ensures accuracy, it suffers from significant response latency. Users must wait for the system to process the entire speech segment before receiving feedback, making the interaction feel unnatural and disjointed, resulting in a poor user experience.
[0003] Furthermore, traditional ASR systems cannot adapt to the dynamic nature of spoken communication, such as thinking, pausing, repeating, and correcting while speaking. This results in delays and incompatibility with spoken communication, further reducing the system's real-time performance and interactivity. Consequently, traditional ASR systems perform poorly in real-time interactive scenarios that require rapid responses, failing to meet users' needs for efficient and natural voice interaction.
[0004] Therefore, the above problems urgently need to be solved. Summary of the Invention
[0005] Purpose of the invention: In order to overcome the above shortcomings, the purpose of this invention is to provide an incremental ASR algorithm adapted to PSTN calls. It adopts an incremental processing method to perform speech recognition in real time while the user is speaking, which significantly reduces latency and improves response speed.
[0006] Technical solution: To achieve the above objectives, this invention provides an incremental ASR algorithm adapted to PSTN calls, comprising:
[0007] S1): Data preprocessing;
[0008] S2): Feature extraction and normalization: The current frame is normalized to obtain the input of the current frame; the normalized features can better reflect the characteristics of the speech signal and improve the distinguishability of the features.
[0009] S3): Concatenate the input of the current frame with the historical frames in the buffer into a single input block and input it into the LSTM model to calculate the hidden state and output probability distribution of the current frame. By concatenating the current frame with the historical frames, the LSTM model can utilize contextual information to better understand the features of the current frame.
[0010] S4): Update the hidden state and cell state of all LSTM layers, update the candidate path set of the decoder, and cache the current frame features as historical frames for use in subsequent frames; the hidden state and cell state of the LSTM model can be dynamically updated to adapt to changes in the speech signal.
[0011] S5): Partial result output judgment. If the probability of a word in the current recognition path exceeds the set threshold and the subsequent frames do not change, then output to the output buffer; otherwise, jump to execute S7.
[0012] S501): Let the current frame be frame t, and the decoder maintains a set of candidate optimal paths p. t ={p1, p2, ... p i ., p k}, where each path P i Represents a possible identification sequence; for path P i Define the last word of frame t as Its conditional probability is:
[0013]
[0014] Where, x 1:j For the audio features of the first t frames, P(w) j |x 1:j ) represents the audio features x in the first j frames. 1：j In the case of the j-th word w being identified j The probability of;
[0015] S502): If there exists a word w such that P(w|x) 1：t )>θ, where θ is the confidence threshold; if the word w still exists in the candidate optimal path in the subsequent frames of the first t frames, it will be marked as "confirmed" and output to the output buffer; when the probability of a word in the identification path exceeds the set threshold, the result can be output in real time to reduce the user's waiting time; at the same time, by checking whether the subsequent frames have changed, the erroneous output caused by misjudgment can be reduced, realizing double verification and ensuring the reliability of the output result;
[0016] S6): The output buffer organizes the output content and outputs the final text, ending the step; the output buffer can detect and merge duplicate or redundant segments, reducing redundant output and improving text readability;
[0017] S7): Determine if a new frame exists. If a new frame exists, jump to S1; otherwise, the step ends. S7 can dynamically adjust the recognition result based on the input of a new frame to adapt to changes in the speech signal.
[0018] Furthermore, S2 includes S201): windowing and STFT, applying a Hamming window function w[n] to each frame of audio signal x[n] to obtain the windowed signal x. w [n] = x[n]·w[n], and then calculate its short-time Fourier transform (STFT) to obtain the STFT result X(k), as shown in the following formula:
[0019]
[0020] Where N is the frame length (number of sampling points), k represents a frequency channel (or frequency point) in the frequency domain, and j is the imaginary unit; windowing reduces the discontinuity of frame edges and reduces spectral leakage; STFT converts the speech frame from the time domain to the frequency domain to obtain the frequency distribution information of each frame.
[0021] Furthermore, S2 also includes S202): mapping the STFT result to a Mel scale using a triangular filter bank H. m (k) Calculate the energy E for each channel m The formula is as follows:
[0022]
[0023] Where m is the index of a specific filter in the Mel-scale filter bank; the filter bank simulates human auditory perception with high resolution in the low-frequency region and low resolution in the high-frequency region, compressing hundreds of frequency points into dozens of filter channels, and has better tolerance to noise and speaker differences.
[0024] Furthermore, S2 also includes S203): performing a logarithmic operation on the output of the filter bank, and then performing a discrete cosine transform (DCT) to obtain the Mel frequency cepstral coefficients (MFCC);
[0025]
[0026] Where M is the total number of filters in the Mel filter bank used; N coeff To preserve the dimensions of the MFCC coefficients, 13 or 26 dimensions are typically chosen. Performing logarithmic operations on the filter bank can enhance the dynamic range compression capability of the features, making the feature distribution closer to a Gaussian distribution, which is beneficial for subsequent modeling. Perform DCT transformation on the logarithmic filter bank energy, extract the first few coefficients as MFCC features, and decorrelate the highly correlated parts of the filter bank energy to retain the most important low-frequency coefficients and remove high-frequency noise.
[0027] Furthermore, S2 also includes S204): calculating the first-order difference (Delta) and the second-order difference (Delta-Delta) of the MFCC coefficients to obtain the Delta characteristic Δc. tDelta-Delta features, Delta feature Δc t The formula is as follows:
[0028]
[0029] Among them, C t This represents the original feature vector at time t; T is the size of the time window considered when calculating the difference, usually taking the value of 1 or 2; τ is the MFCC coefficient of the current frame;
[0030] The Delta-Delta feature is obtained by differentiating the Delta feature; the first-order difference (Delta) and the second-order difference (Delta-Delta) are calculated for the MFCC coefficients to capture dynamic changes in speech, such as pronunciation speed and pitch changes, thereby improving the reliability of the recognition system and enhancing the model's ability to model the dynamic characteristics of speech.
[0031] Furthermore, S2 also includes S205): concatenating the MFCC coefficients, Delta features, and Delta-Delta features to obtain the input of the current frame. Concatenating the static MFCC, Delta MFCC, and Delta-Delta MFCC into a fixed-dimensional feature vector can construct a complete speech feature representation, including static spectrum, dynamic spectrum, and energy information.
[0032] Furthermore, in step S6, the output buffer detects whether there are duplicate or redundant segments and performs merging and deduplication processing; when the word "confirmed" is detected, it is removed from the buffer and sent to form text output. By detecting and merging duplicate segments, the same content can be output multiple times, improving overall efficiency; at the same time, the deduplicated text is more concise and clear, allowing users to obtain key information more quickly, thus improving the user experience.
[0033] Furthermore, the S1 data preprocessing includes signal adjustment and frame segmentation. Frame segmentation can capture dynamic changes in the speech signal, such as intonation and speech rate, and can better understand the dynamic characteristics of the speech signal; frame segmentation can better adapt to sudden changes in the speech signal, such as pauses and stresses, thereby improving the robustness of this method.
[0034] Furthermore, the historical frames in S3 include features from the past 5 frames. Incorporating historical frames into the feature calculation of the current frame can better capture the dynamic information and temporal correlation of the speech signal, providing rich contextual information, thereby improving the accuracy and robustness of recognition.
[0035] Furthermore, the confidence threshold in S502 ranges from 0.7 to 0.9, which can balance accuracy and recall in different application scenarios, reduce false positives and false negatives, provide a certain degree of flexibility and adjustability, and optimize the user experience.
[0036] As can be seen from the above technical solution, the present invention has the following beneficial effects:
[0037] 1. The present invention provides an incremental ASR algorithm adapted to PSTN calls, which adopts an incremental processing method to perform speech recognition in real time while the user is speaking, significantly reducing latency and improving response speed;
[0038] 2. This invention provides an incremental ASR algorithm adapted to PSTN calls. Through a series of feature processing methods, it extracts features and achieves effective support for PSTN calls. This not only improves the speed of speech recognition but also ensures the accuracy and reliability of the recognition results. Attached Figure Description
[0039] Figure 1 This is a schematic diagram illustrating the steps of an incremental ASR algorithm adapted to PSTN calls as described in this invention.
[0040] Figure 2 This is a flowchart of an incremental ASR algorithm adapted to PSTN calls, as described in this invention. Detailed Implementation
[0041] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.
[0042] Example
[0043] In this embodiment, as Figure 1 and Figure 2 This invention discloses an incremental ASR algorithm adapted to PSTN calls, comprising:
[0044] S1): Data preprocessing;
[0045] S2): Feature extraction and normalization: Standardize the current frame to obtain the input of the current frame;
[0046] S3): Concatenate the input of the current frame with the historical frames in the buffer into an input block, and input it into the LSTM model to calculate the hidden state and output probability distribution of the current frame;
[0047] S4): Update the hidden states and cell states of all LSTM layers, update the candidate path set of the decoder, and cache the current frame features as historical frames for subsequent frames;
[0048] S5): Determine the output of partial results. If the probability of a certain word in the current recognition path exceeds the set threshold and does not change in subsequent frames, output it to the output buffer; otherwise, jump to execute S7;
[0049] S501): Let the current frame be the t-th frame, and the decoder maintains a set of candidate optimal paths p t ={p1, p2,..p i ., p k}, where each path P i represents a possible recognition sequence; for path P i , define the end word at the t-th frame as Its conditional probability is:
[0050]
[0051] where, x 1：j is the audio feature of the first t frames, and P(w j |x 1:j ) represents the probability of the j-th word w 1:j recognized given the audio feature x j of the first j frames;
[0052] S502): If there exists a word w such that P(w|x 1:t )>θ, where θ is the confidence threshold; and in several subsequent frames of the first t frames, the word w still exists in the candidate optimal path, then mark it as "confirmed" and output it to the output buffer;
[0053] S6): Organize the output content in the output buffer and output the final text, and the step ends;
[0054] S7): Determine whether there is a new frame. If there is a new frame, jump to S1; otherwise, the step ends.
[0055] Specifically, assume there is a piece of speech. After feature extraction and model inference, the following recognition results for several frames are obtained during the judgment of partial result output:
[0056] Frame number Optimal path End word Confidence level t＝1 ["you"] "you" 0.92 t＝2 ["hello"] "good" 0.85 t＝3 ["how are you"] "right" 0.78 t＝4 ["how are you"] "right" 0.8 (unchanged) t＝5 ["how are you"] "right" 0.8 (still unchanged)
[0057] At frame t = 3, the word "ma" is recognized, and its confidence is 0.78;
[0058] At frames t = 4 and t = 5, the word "ma" still appears in the optimal path and is not replaced;
[0059] If the set confidence threshold θ = 0.75, then at frame t = 3, the confidence of the word "ma" has exceeded the threshold. Therefore, at frame t = 5, since the word remains stable in subsequent frames, it can be marked as "confirmed" and output to the output buffer.
[0060] In this embodiment, S2 includes: S201): Windowing and STFT. Apply the Hamming window function w[n] to each frame of the audio signal x[n] to obtain the windowed signal x w [n] = x[n] · w[n], and then calculate its short-time Fourier transform (STFT) to obtain the STFT result X(k). The formula is as follows:
[0061]
[0062] where N is the length of the frame (number of sampling points), k represents a frequency channel (or frequency point) in the frequency domain, and j is the imaginary unit.
[0063] Specifically, if the speech sampling rate is 16 kHz and the frame length is 25 ms, then N = 16000 × 0.025 = 400 sampling points. The resolution and computational complexity of the Fourier transform are determined by N; the range of a frequency channel (or frequency point) k in the frequency domain is as follows: k = 0, 1, 2,..., N - 1; the imaginary unit j is used to represent the phase information of the signal.
[0064] In this embodiment, S2 further includes S202): Map the STFT result to the Mel scale and use the triangular filter bank H m (k) to calculate the energy E of each channel m , and the formula is as follows:
[0065]
[0066] where m is the index of a specific filter in the Mel scale filter bank.
[0067] Specifically, the value range of m is adjusted according to the number of filters. If 40 filters are set, then the value range of m is from 1 to 40.
[0068] In this embodiment, S2 further includes S203): After performing a logarithmic operation on the output of the filter bank, perform a discrete cosine transform (DCT) to obtain the Mel-frequency cepstral coefficients (MFCC);
[0069]
[0070] where M is the total number of filters in the Mel filter bank used; N coeffTo preserve the dimensions of the MFCC coefficients, 13 or 26 dimensions are typically chosen.
[0071] Specifically, each filter covers a specific frequency range, and these filters are distributed according to the Mel scale.
[0072] In this embodiment, S2 further includes S204): calculating the first-order difference (Delta) and the second-order difference (Delta-Delta) of the MFCC coefficients to obtain the Delta characteristic Δc. t Delta-Delta features, Delta feature Δc t The formula is as follows:
[0073]
[0074] Among them, C t This represents the original feature vector at time t; T is the size of the time window considered when calculating the difference, usually taking the value of 1 or 2; τ is the MFCC coefficient of the current frame;
[0075] The Delta-Delta feature is obtained by differentiating the Delta feature.
[0076] Specifically, the time window size T considered when calculating the difference is usually 1 or 2, taking into account 1 to 2 frames before and after the current frame to calculate the difference; τ is the MFCC coefficient of the current frame, which is the time offset relative to the current frame, ranging from -T to +T, and is used for weighted summation to calculate the difference between the previous and next frames and the current frame.
[0077] In this embodiment, S2 further includes S205): concatenating the MFCC coefficients, Delta features, and Delta-Delta features to obtain the input of the current frame.
[0078] Specifically, Delta features capture the rate of change of features between adjacent frames, providing information about how the speech signal changes over time and distinguishing different pronunciation details such as rapidly changing consonants or slowly changing vowels; Delta-Delta features further capture the changes in the rate of change of features, providing deeper dynamic information.
[0079] In this embodiment, in step S6, the output buffer detects whether there are duplicate or redundant segments and performs merging and deduplication processing; when the word "confirmed" is detected, it is removed from the buffer and sent to form text output.
[0080] Specifically, hash table matching can be used as a preferred method for duplicate detection. The hash table can be used to quickly find whether a word or phrase in the current window has appeared in the previous text. If it has appeared, it is marked as a duplicate segment.
[0081] In this embodiment, the S1 data preprocessing includes signal adjustment and frame segmentation.
[0082] Specifically, signal adjustment includes processing operations such as gain control and noise suppression on the received PSTN call audio signal to ensure the quality of the input audio; frame processing specifically involves dividing the continuous audio stream into multiple small segments (frames) according to a fixed-length time window to facilitate subsequent processing.
[0083] In this embodiment, the historical frames in S3 include features from the past 5 frames.
[0084] Specifically, the number of historical frames can be increased or decreased to expand features or improve running speed.
[0085] In this embodiment, the confidence threshold in S502 ranges from 0.7 to 0.9.
[0086] Specifically, in demanding scenarios such as medical or legal matters, the threshold can be set to 0.9; in general customer service scenarios, the threshold can be set to 0.7.
[0087] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements can be made without departing from the principle of the present invention, and these improvements should also be considered within the scope of protection of the present invention.
Claims
1. An incremental ASR algorithm adapted to PSTN calls, characterized in that: include: S1): Data preprocessing; S2): Feature extraction and normalization: Standardize the current frame to obtain the input of the current frame; S3): Concatenate the input of the current frame with the historical frames in the buffer into an input block, and input it into the LSTM model to calculate the hidden state and output probability distribution of the current frame; S4): Update the hidden state and cell state of all LSTM layers, update the candidate path set of the decoder, and cache the current frame features as historical frames for use in subsequent frames; S5): Partial result output judgment. If the probability of a word in the current recognition path exceeds the set threshold and the subsequent frames do not change, then output to the output buffer; otherwise, jump to execute S7. S501): Let the current frame be frame t, and the decoder maintains a set of candidate optimal paths p. t ={p1, p2, ... p i ., p k }, where each path P i Represents a possible identification sequence; for path P i Define the last word of frame t as Its conditional probability is: Where, x 1：j For the audio features of the first t frames, P(w) j |x 1:j ) represents the audio features x in the first j frames. 1：j In the case of the j-th word w being identified j The probability of; S502): If there exists a word w such that P(w|x) 1：t If the word w still exists in the candidate optimal path in the subsequent frames of the first t frames, it will be marked as "confirmed" and output to the output buffer. S6): The output buffer organizes the output content and outputs the final text. The step ends here. S7): Determine if a new frame exists. If a new frame exists, jump to S1; otherwise, the step ends.
2. The incremental ASR algorithm adapted to PSTN calls according to claim 1, characterized in that: S2 includes: S201): Windowing and STFT: Apply the Hamming window function w[n] to each frame of audio signal x[n] to obtain the windowed signal x. w [n] = x[n]·w[n], and then calculate its short-time Fourier transform (STFT) to obtain the STFT result X(k), as shown in the following formula: Where N is the frame length (number of sampling points), k represents a frequency channel (or frequency point) in the frequency domain, and j is the imaginary unit; S202): Map the STFT results to the Mel scale using a triangular filter bank H. m (k) Calculate the energy E for each channel m The formula is as follows: Where m is the index of a specific filter in the Mel-scale filter bank; S203): After performing a logarithmic operation on the output of the filter bank, a discrete cosine transform (DCT) is then performed to obtain the Mel frequency cepstral coefficients (MFCC). Where M is the total number of filters in the Mel filter bank used; N coeff To preserve the dimensions of the MFCC coefficients, 13 or 26 dimensions are typically chosen. S204): Calculate the first-order difference (Delta) and second-order difference (Delta-Delta) of the MFCC coefficients to obtain the Delta characteristic Δc. t Delta-Delta features, Delta feature Δc t The formula is as follows: Among them, C t This represents the original feature vector at time t; T is the size of the time window considered when calculating the difference, usually taking the value of 1 or 2; τ is the MFCC coefficient of the current frame; The Delta-Delta feature is obtained by differentiating the Delta feature.
3. The incremental ASR algorithm adapted to PSTN calls according to claim 2, characterized in that: S2 also includes S205): concatenating the MFCC coefficients, Delta features, and Delta-Delta features to obtain the input of the current frame.
4. The incremental ASR algorithm adapted to PSTN calls according to claim 1, characterized in that: In step S6, the output buffer detects whether there are duplicate or redundant segments and performs merging and deduplication processing; when the word "confirmed" is detected, it is removed from the buffer and sent to form text output.
5. The incremental ASR algorithm adapted to PSTN calls according to claim 1, characterized in that: The S1 data preprocessing includes signal adjustment and frame segmentation.
6. The incremental ASR algorithm adapted to PSTN calls according to claim 1, characterized in that: The historical frames in S3 include features from the past 5 frames.
7. The incremental ASR algorithm adapted to PSTN calls according to claim 1, characterized in that: The confidence threshold in S502 has a value range of 0.7 to 0.9.