System and method for optimizing granularity between double-layer sequences in voice keyword recognition
Through a two-layer sequence granularity optimization system, the problems of modeling unit selection, dialect adaptability and environmental robustness in speech recognition technology are solved, and efficient speech recognition is achieved, which is suitable for scenarios with high real-time requirements such as security.
Patent Information
- Application Number
- CN202510914654.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-03
- Publication Date
- 2025-10-17
AI Technical Summary
Existing speech recognition technology has shortcomings in modeling unit selection, dialect adaptability and environmental robustness, resulting in limited recognition performance. In particular, confusion errors occur frequently when processing professional terms or new words, the dialect recognition accuracy is low, and background noise and channel interference have a significant impact.
A two-layer sequence granularity optimization system is adopted to process speech segments in different channels through coarse-grained and fine-grained models to generate filter group features. The coarse-grained and fine-grained sequence results are generated through the text encoding module, and the decoding module is combined to perform task classification and finally output readable text.
It improves the robustness and accuracy of speech recognition, adapts to complex environments, reduces dependence on large-scale annotated data, and improves recognition effects, especially in dialect and personalized pronunciation scenarios.
Smart Images
Figure CN120808787A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of speech recognition, more particularly, it relates to a system and method for granularity optimization between double-layer sequences in speech keyword recognition. BACKGROUND
[0002] Although speech recognition technology has made significant progress, it still faces many technical challenges in practical applications. Current systems are mainly limited by three key problems: modeling unit selection, dialect adaptability and environmental robustness, which seriously restrict the further improvement of recognition performance.
[0003] In terms of modeling units, existing systems cannot dynamically adjust the recognition granularity. Fixed use of phoneme units will cause confusion errors caused by pronunciation variation, while the use of word units will encounter the dilemma of insufficient vocabulary coverage. Especially when dealing with professional terms or new words, this contradiction is more prominent. In terms of dialect recognition, due to the serious lack of dialect samples in training data, the model is difficult to capture the pronunciation characteristics unique to the dialect, resulting in a significant decrease in recognition accuracy. In terms of environmental adaptability, background noise and channel interference can greatly reduce system performance, and existing noise reduction methods still perform poorly in complex acoustic environments.
[0004] Therefore, a system and method for granularity optimization between double-layer sequences in speech keyword recognition are provided to solve the above problems. SUMMARY
[0005] In view of the deficiencies of the prior art, the purpose of the present application is to provide a system and method for granularity optimization between double-layer sequences in speech keyword recognition to improve overall robustness and recognition accuracy.
[0006] To achieve the above purpose, the present application provides the following technical scheme: a system for granularity optimization between double-layer sequences in speech keyword recognition, comprising a collection module for collecting a speech segment spoken by a user; a language processing module for dividing the speech segment into two channels according to a coarse-grained model and a fine-grained model, aligning the coarse-grained and fine-grained in the two channels according to the number of frames, and generating coarse-grained filter group features and fine-grained filter group features; a text encoding module for encoding the coarse-grained filter group features and fine-grained filter group features in the corresponding channels according to the number of frames to generate coarse-grained sequence results and fine-grained sequence results; and a decoding output module for performing task classification on the coarse-grained sequence results, fine-grained sequence results, coarse-grained filter group features and fine-grained filter group features contained in the two channels, respectively, and outputting readable double-layer sequence text after summarizing.
[0007] The application is further provided as follows: any channel of the language processing module comprises a preprocessing unit, a frame dividing unit, a windowing unit, an algorithm unit, a power calculation unit, an alignment unit and a mel filtering unit; the preprocessing unit is used to lift high-frequency components of the voice segment to generate emphasized data; the frame dividing unit is used to segment according to the emphasized data and a grain model to generate multiple groups of segmented data; the windowing unit is used to perform smoothing processing on the segmented data to obtain framed data; the algorithm unit is used to generate frequency domain distribution data according to the framed data; the power calculation unit is used to calculate single-point average power according to the frequency distribution data; the alignment unit is used to correspond the single-point average power with the frame number to obtain alignment data; and the mel filtering unit is used to perform mel filtering according to the alignment data to obtain filtered group features.
[0008] The application is further provided as follows: any channel of the text encoding module comprises a projection unit and a time sequence feature unit; the projection unit is used to obtain projection data according to the filtered group features; and the time sequence feature unit is used to generate a grain sequence result according to the projection data and the frame number.
[0009] The application is further provided as follows: the frame number is obtained by counting the time of the voice segment.
[0010] By adopting the above technical solution, a plurality of language recognition frameworks can be compatible by means of module design, and an end-to-end low delay is realized by a pipeline design of feature extraction, encoding, decoding and optimization, so as to meet the requirements of high real-time scenes such as security.
[0011] A grain optimization method between double-layer sequences in voice keyword recognition, using a voice keyword recognition double-layer sequence grain optimization system as described above, comprises the following steps: S1, collecting a voice segment said by a user; S2, dividing the voice segment into two channels according to a coarse-grain model and a fine-grain model, aligning the coarse grain and the fine grain in the two channels according to the frame number, and generating coarse-grain filtered group features and fine-grain filtered group features; S3, encoding the coarse-grain filtered group features and the fine-grain filtered group features in the corresponding channels according to the frame number to generate coarse-grain sequence results and fine-grain sequence results; S4, comparing the coarse-grain sequence results and the fine-grain sequence results contained in the two channels according to a comparison function, filling the coarse-grain filtered group features corresponding to the coarse-grain sequence results with high recognition rate into the fine-grain filtered group features, and outputting readable double-layer sequence text after summarizing.
[0012] S2 is more specifically as follows: S21, lifting high-frequency components of the voice segment to generate emphasized data; S22, duplicate the weighted data, and the two weighted data enter the coarse-grained channel and the fine-grained channel respectively, wherein the weighted data in the coarse-grained channel and the weighted data in the fine-grained channel perform steps S23-S28 respectively; S23, segment according to the weighted data and the grain model to generate segmented data; S24, frame and smooth processing is performed on both ends of the segmented data to obtain framed data; S25, generate frequency domain distribution data according to the framed data; S26, calculate single-point average power according to the frequency distribution data; S27, correspond the single-point average power with the frame number to obtain alignment data; S28, perform Mel filtering according to the alignment data to obtain filter bank characteristics.
[0013] The application is further provided that: the frequency domain distribution data is calculated by a fast Fourier transform algorithm.
[0014] The application is further provided that: the grain sequence result is calculated by a probability model or a hidden Markov model or a Gaussian mixture model or an attention neural network or a recurrent neural network algorithm or a convolutional neural network algorithm or a hybrid architecture algorithm.
[0015] The application is further provided that: the task classification according to the grain sequence result and the filter bank characteristics is classified by a self-recurrent algorithm or a non-self-recurrent algorithm or a high-efficiency convolutional neural network algorithm.
[0016] By adopting the above technical solution, the problem of low coverage or high confusion rate of the traditional single-grain model in a low-resource scene is solved, thereby improving the recognition accuracy; dynamic superimposed noise is performed in the system training stage to simulate the recognition learning of a complex environment, and the overall robustness is improved.
[0017] In summary, the present application includes at least one of the following beneficial technical effects: 1. The module design is compatible with multiple language recognition frameworks, and the system realizes end-to-end low delay through the pipeline design of feature extraction, coding, decoding and optimization, and meets the real-time requirement of high scenes such as security.
[0018] 2. The problem of low coverage or high confusion rate of the traditional single-grain model in a low-resource scene is solved, thereby improving the recognition accuracy; dynamic superimposed noise is performed in the system training stage to simulate the recognition learning of a complex environment, and the overall robustness is improved. BRIEF DESCRIPTION OF DRAWINGS
[0019] Fig. 1 It is a relationship diagram of a grain optimization system between double layers in a voice keyword recognition in the application. Fig. 2 A flowchart of steps of the granularity optimization method S1-S5 between double-layer sequences in a speech keyword recognition in the application is shown in the figure. Fig. 3 A flowchart of specific steps of the granularity optimization method S2 between double-layer sequences in a speech keyword recognition in the application is shown in the figure. DETAILED DESCRIPTION
[0020] It should be noted that the embodiments and features in the embodiments in the application can be combined with each other without conflict. The application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0021] It should be noted that, unless otherwise specified, all technical and scientific terms used in the application have the same meaning as generally understood by those skilled in the art to which the application belongs.
[0022] Please refer to Figs. 1-3 The application provides the following technical solutions: Embodiment one, please refer to Fig. 1 A granularity optimization system between double-layer sequences in a speech keyword recognition, comprising a text collection module, a language processing module, a text encoding module and a decoding output module. The collection module is used to collect a speech segment spoken by a user. The language processing module is used to divide the speech segment into two channels according to a coarse-grained model and a fine-grained model, align the coarse-grained and fine-grained in the two channels according to the frame number, and generate coarse-grained filter group features and fine-grained filter group features, the frame number being obtained by counting the time of the speech segment. Any channel of the language processing module comprises a preprocessing unit, a framing unit, a windowing unit, an algorithm unit, a power calculation unit, an alignment unit and a mel filter unit. The preprocessing unit is used to boost the high-frequency components of the speech segment to generate emphasis data. The framing unit is used to segment according to the emphasis data and the grained model to generate multiple sets of segmented data. The windowing unit is used to smooth the segmented data to obtain framed data. The algorithm unit is used to generate frequency domain distribution data according to the framed data. The power calculation unit is used to calculate the single-point average power according to the frequency distribution data. The alignment unit is used to one-to-one correspond the single-point average power with the frame number to obtain alignment data. The mel filter unit is used to perform mel filtering according to the alignment data to obtain filter group features.
[0023] The text encoding module is configured to encode the coarse-grained filter group features and the fine-grained filter group features corresponding to the channels according to the frame number to generate coarse-grained sequence results and fine-grained sequence results. Any channel of the text encoding module comprises a projection unit and a time sequence feature unit. The projection unit is configured to derive projection data from the filter group features. The time sequence feature unit is configured to generate a granularity sequence result according to the projection data and the frame number. The text encoding module mainly uses an encoder to realize the above functions.
[0024] The decoding output module is configured to respectively perform task classification on the coarse-grained sequence results, the fine-grained sequence results, the coarse-grained filter group features and the fine-grained filter group features contained in the two groups of channels, and output readable double-layer sequence texts after summarizing. The decoding output module mainly uses a decoder to realize the above functions.
[0025] Through the above system, a speech recognition framework compatible with multiple languages can be designed by modules, and through the pipeline design of feature extraction, encoding, decoding and optimization, end-to-end low latency is realized to meet the real-time requirements of security and other scenarios.
[0026] Embodiment two, refer to Figs. 2-3 A granularity optimization method between double-layer sequences in speech keyword recognition, using the above-mentioned granularity optimization system between double-layer sequences in speech keyword recognition, comprising the following steps: S1, collecting a voice segment said by a user; S2, dividing the voice segment into two channels according to a coarse-grained model and a fine-grained model, aligning the coarse-grained and fine-grained according to the frame number in the two channels, and generating coarse-grained filter group features and fine-grained filter group features; The more specific steps of S2 are as follows: S21, enhancing the high-frequency components of the voice segment to generate emphasized data; S22, copying one copy of the emphasized data, and the two copies of the emphasized data enter the coarse-grained channel and the fine-grained channel respectively, wherein the emphasized data in the coarse-grained channel and the emphasized data in the fine-grained channel perform steps S23-S28 respectively; S23, segmenting the emphasized data according to the grain model to generate segmented data; S24, performing frame adding and smoothing processing on both ends of the segmented data to obtain frame-added data; S25, generating frequency domain distribution data according to the frame-added data; wherein the frequency domain distribution data is calculated by a fast Fourier transform algorithm.
[0027] S26, calculating the single-point average power according to the frequency distribution data; S27, the single-point average power is corresponded to the frame number to obtain alignment data; S28, the alignment data is subjected to Mel filtering to obtain filtered group features.
[0028] S3, the coarse-grained sequence results and the fine-grained sequence results are generated by encoding the coarse-grained filtered group features and the fine-grained filtered group features of the corresponding channels according to the frame number; wherein the sequence results are calculated by a probability model or a hidden Markov model or a Gaussian mixture model or an attention neural network or a recurrent neural network algorithm or a convolutional neural network algorithm or a hybrid architecture algorithm.
[0029] S4, the coarse-grained sequence results and the fine-grained sequence results contained in the two groups of channels and a comparison function are compared, the coarse-grained filtered group features corresponding to the coarse-grained sequence results with high recognition rate are filled into the fine-grained filtered group features, and readable double-layer sequence texts are output after summarization, wherein the task classification according to the sequence results and the filtered group features is classified by a self-recurrent algorithm or a non-self-recurrent algorithm or a high-efficiency convolutional neural network algorithm. That is, the coarse-grained sequence results P(xy) and the fine-grained sequence results P(x,y) can be compared and the better one is selected. Finally, an optimized granularity set is formed, that is, ; Through the prediction and screening of coarse and fine granularities, the problem of low coverage or high confusion rate of traditional single-granularity models in low-resource scenarios is solved, thereby improving the recognition accuracy; through the double-layer granularity optimization strategy, the dependence on large-scale labeled data is reduced, which is especially suitable for dialect, personalized pronunciation and other data-scarce scenarios; dynamic superimposed noise is added in the system training stage to simulate the recognition learning in complex environments and improve the overall robustness.
[0030] Obviously, the above-described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.
Claims
1. A granularity optimization system for two-layer sequences in speech keyword recognition, characterized by: include: A collection module, used to collect voice clips spoken by users; The language processing module is used to divide the speech segment into two channels according to the coarse-grained model and the fine-grained model, align the coarse-grained and fine-grained features in the two channels according to the number of frames, and generate coarse-grained filter group features and fine-grained filter group features; A text encoding module, configured to encode the coarse-grained filter group features and the fine-grained filter group features of the corresponding channel according to the number of frames to generate a coarse-grained sequence result and a fine-grained sequence result; and The decoding output module is used to perform task classification according to the coarse-grained sequence results, fine-grained sequence results, coarse-grained filter group features and fine-grained filter group features contained in the two groups of channels, and output readable two-layer sequence text after summarization.
2. The system for optimizing the granularity between two-layer sequences in speech keyword recognition according to claim 1, characterized in that: Any channel of the language processing module includes a pre-processing unit, a framing unit, a windowing unit, an algorithm unit, a power calculation unit, an alignment unit and a Mel filter unit; The pre-processing unit is used to enhance the high-frequency components of the speech segment to generate weighted data; The frame segmentation unit is used to segment the weighted data and the particle model to generate multiple groups of segmented data; The windowing unit is used to perform smoothing on the segmented data to obtain frame data; The algorithm unit is used to generate frequency domain distribution data according to the added frame data; The power calculation unit is used to calculate the single-point average power according to the frequency distribution data; The alignment unit is used to match the single-point average power with the frame number one by one to obtain alignment data; The Mel filter unit is used to perform Mel filtering according to the aligned data to obtain filter group features.
3. The system for optimizing the granularity between two-layer sequences in speech keyword recognition according to claim 2, characterized in that: Any channel of the text encoding module includes a projection unit and a temporal feature unit; The projection unit is used to obtain projection data according to the filter group characteristics; The time series feature unit is used to generate a granularity sequence result according to the projection data and the number of frames.
4. The system for optimizing granularity between two-layer sequences in speech keyword recognition according to claim 3, characterized in that: The number of frames is obtained by counting the time of the speech segment.
5. A method for optimizing the granularity between two-layer sequences in speech keyword recognition, using the system for optimizing the granularity between two-layer sequences in speech keyword recognition as claimed in claim 4, characterized in that: The following steps are involved: S1. Collect the voice clips spoken by the user; S2. Divide the speech segment into two channels according to the coarse-grained model and the fine-grained model, align the coarse-grained and fine-grained features in the two channels according to the number of frames, and generate coarse-grained filter group features and fine-grained filter group features; S3. Encode the coarse-grained filter group features and the fine-grained filter group features of the corresponding channel according to the number of frames to generate a coarse-grained sequence result and a fine-grained sequence result; S4. Compare the coarse-grained sequence results, fine-grained sequence results and comparison functions contained in the two groups of channels, fill the coarse-grained filter group features corresponding to the coarse-grained sequence results with high recognition rate into the fine-grained filter group features, and output a readable double-layer sequence text after summarizing.
6. The method for optimizing granularity between two-layer sequences in speech keyword recognition according to claim 5, characterized in that: The more specific steps of S2 are: S21, enhancing the high-frequency components of the speech segment to generate weighted data; S22, copying the weighted data, and the two copies of weighted data enter the coarse-grained channel and the fine-grained channel respectively, wherein the weighted data in the coarse-grained channel and the weighted data in the fine-grained channel respectively execute steps S23-S28; S23, segmenting according to the weighted data and the particle model to generate segmented data; S24, performing frame addition and smoothing processing on both ends of the segmented data to obtain framed data; S25, generating frequency domain distribution data according to the added frame data; S26. Calculate the single-point average power based on the frequency distribution data; S27, matching the single-point average power with the frame number one by one to obtain alignment data; S28. Perform Mel filtering on the aligned data to obtain filter group features.
7. The method for optimizing granularity between two-layer sequences in speech keyword recognition according to claim 6, characterized in that: The frequency domain distribution data is calculated by a fast Fourier transform algorithm.
8. The method for optimizing granularity between two-layer sequences in speech keyword recognition according to claim 5, characterized in that: The granularity sequence result is calculated by a probability model, a hidden Markov model, a Gaussian mixture model, an attention neural network, a recurrent neural network algorithm, a convolutional neural network algorithm, or a hybrid architecture algorithm.
9. The method for optimizing granularity between two-layer sequences in speech keyword recognition according to claim 5, characterized in that: The task classification according to the granularity sequence results and the filter group characteristics is performed by an autoregressive algorithm, a non-autoregressive algorithm, or an efficient convolutional neural network algorithm.