Self-defined voice wake-up word detection method based on finite-state machine

Abstract

The invention discloses a self-defined voice wake-up word detection method based on a finite-state machine. The method comprises the following steps: training a Transform network model by using an open-source Chinese voice data set on the Internet; performing framing windowing processing on the input voice signal, and extracting corresponding features; the extracted features are input into a Transform network model, and a frame probability tensor is generated; configuring target wake-up word content, obtaining an initial and final pinyin sequence corresponding to the target wake-up word, and constructing a finite-state machine decoding network; processing the frame probability tensor to generate a new frame probability tensor; inputting the new frame probability tensor into a finite-state machine decoding network, operating a Viterbi algorithm, and searching an optimal phoneme path; and calculating an average value of all phoneme state scores, comparing the average value with a preset threshold value, and activating the wake-up word when the average value exceeds the preset threshold value. According to the method, the Transform model and the finite-state machine technology are combined, self-defined voice wake-up word detection is achieved, and the method has the advantages of being flexible in configuration, low in power consumption operation, high in precision and high in robustness.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to a custom speech wake-up word detection method based on a finite state machine. Background Art

[0002] With the development of artificial intelligence technology, speech recognition technology has become an important part of smart devices. As the entrance to the speech recognition system, the voice wake-up function can start the device through a specific wake-up word, avoiding resource waste and privacy issues caused by continuous monitoring. This function is widely used in smart speakers, car systems, mobile devices and other fields, realizing natural human-computer interaction. However, existing voice wake-up technology still faces challenges in terms of diversified needs, performance optimization and scalability.

[0003] In the prior art, common voice wake-up methods mainly rely on deep learning models to classify voice signals. These methods classify the features of each frame of voice through a neural network model to determine whether it meets the specific pattern of the target wake-up word. Voice features are usually extracted using MFCC, FilterBank, or logarithmic Mel spectrum, which can effectively capture the spectral characteristics of voice signals. However, most prior arts use a single model that is directly trained on a specific wake-up word, that is, the acoustic model must be retrained every time a new wake-up word needs to be supported. This method not only consumes a lot of time and computing resources, but also lacks flexibility for users.

[0004] In addition, existing voice wake-up systems are often limited by resource consumption and operating efficiency. In view of the actual needs of low-power devices, complex neural network models may lead to excessive computing and storage burdens. Some methods reduce resource consumption by reducing the size of the model, but at the same time sacrifice the performance and robustness of the model. In noisy environments, far-field pickup or multi-language scenarios, these methods have high false wake-up rates and missed wake-up rates, which are difficult to meet the actual needs of users.

[0005] In order to improve the accuracy of wake-up word detection, existing technologies usually directly model speech through end-to-end training. However, the model training cost of this method is high and lacks flexibility, especially in scenarios that support multiple wake-up words, where models need to be built or retrained for each wake-up word. Support for target wake-up words is limited to model parameters during training, making it difficult for these systems to adapt to rapidly changing user needs. For example, when a user wants to add a new wake-up word or modify an existing wake-up word, the system needs to re-label the data, train the model, and deploy a new version, which is not only time-consuming and complex, but also limits user autonomy.

[0006] In the prior art, there are also some voice wake-up methods based on template matching or keyword detection. These methods attempt to make a wake-up judgment by comparing the matching degree between the feature template of the target wake-up word and the real-time voice signal. However, the accuracy of the template matching method is usually low, especially under the influence of diverse user pronunciations and complex environmental noise, its adaptability is obviously insufficient. In addition, the template matching method is difficult to expand, especially in scenarios that support multiple wake-up words or personalized wake-up words. As the number of wake-up words increases, the system performance will drop significantly.

[0007] Speech processing methods based on finite state machines have gradually attracted attention in recent years. This method can effectively describe the structure of the wake-up word by modeling the phoneme sequence of the speech signal, and decode the target phoneme path through state transition. However, traditional finite state machine methods are mostly used in simple rule matching scenarios, lack effective integration with modern deep learning technology, and are difficult to deal with uncertainties in complex speech signals. In addition, existing decoding methods based on finite state machines usually rely heavily on computing resources and are difficult to adapt to the needs of low-power devices.

[0008] In summary, the existing voice wake-up technology has the following defects: First, the existing system usually needs to train a model for each wake-up word separately, resulting in poor flexibility and difficulty in quickly adapting to user needs; second, the traditional method has limited performance in multiple wake-up word detection scenarios and cannot efficiently support the decoding and detection of multiple wake-up words; third, on resource-constrained embedded devices, the existing technology is difficult to achieve low-power operation while ensuring performance; fourth, for changeable actual usage environments, such as noisy environments, far-field voice, etc., the existing methods have high false wake-up rates and missed wake-up rates, and lack robustness. Summary of the invention

[0009] One object of the present invention is to propose a custom voice wake-up word detection method based on a finite state machine. The present invention combines the advantages of the Transformer network model and the finite state machine decoding. There is no need to retrain the neural network model for each wake-up word. Support for multiple wake-up words can be completed by only configuring the wake-up word content, thereby significantly improving the flexibility and scalability of the system. In addition, the introduction of the finite state machine enables the system to efficiently decode the phoneme path of the target wake-up word, optimizes the computing resource requirements of the detection process, and reduces the false wake-up rate and missed wake-up rate. Through efficient feature extraction and model reasoning, the method can realize robust voice wake-up function in complex scenarios, meeting the requirements of practical applications for low power consumption, high precision and flexible configuration.

[0010] A method for detecting a custom voice wake-up word based on a finite state machine according to an embodiment of the present invention includes the following steps:

[0011] S1. Use the open source Chinese speech dataset on the Internet to train the Transformer network model;

[0012] S2, perform frame and window processing on the input speech signal, use 512-point window length and 256-point frame shift parameters to frame the speech signal, and extract corresponding features;

[0013] S3, input the extracted features into the trained Transformer network model, infer the features of each frame, generate the pinyin category probability distribution of each frame of speech, and form a frame probability tensor;

[0014] S4. Based on the target wake-up word and the probability distribution of pinyin categories, configure the target wake-up word content and obtain the pinyin sequence of consonants and vowels corresponding to the target wake-up word through dictionary mapping, and build a finite state machine decoding network according to the pinyin sequence of consonants and vowels;

[0015] S5. Process the frame probability tensor, retain the phoneme category scores related to the target wake-up word in the frame probability tensor, and replace the silent symbol <sil>Keep it as a separate category and combine the scores of the remaining categories into <gbg>Score, generate a new frame probability tensor;

[0016] S6. Input the new frame probability tensor into the finite state machine decoding network, run the Viterbi algorithm, search for the optimal phoneme path, and determine whether the optimal phoneme path completes the phoneme sequence corresponding to the wake-up word;

[0017] S7. According to the searched optimal phoneme path, record the maximum score of each phoneme state and calculate the average of all phoneme state scores as the wake-up word score, and compare the wake-up word score with the preset threshold value. When the wake-up word score exceeds the preset threshold value, activate the wake-up word, otherwise return to the initial state to continue monitoring.

[0018] Optionally, the S1 specifically includes:

[0019] S11, collecting an open source Chinese speech data set from the Internet, wherein the open source Chinese speech data set includes speech signals and corresponding transcribed texts, and normalizing the speech data through data preprocessing, wherein the normalization includes unifying the sampling rate, filtering invalid audio, and format conversion, and standardizing all speech files to a PCM format with a 16k sampling rate, a single channel, and 16-bit quantization;

[0020] S12, mapping Chinese characters in the transcribed text into initials and finals combinations word by word through a dictionary, generating corresponding initials and finals pinyin sequences, wherein the initials and finals pinyin sequences include 23 initials and 24 finals, and do not consider tones and erhua phenomena;

[0021] S13, define the output categories of the Transformer network as 48 categories, including 23 initials, 24 finals, and 1 mute symbol <sil>, the mute symbol is used to mark silent frame segments;

[0022] S14. Design the training objectives of the Transformer network model, take the feature vector of each frame of speech signal as input, use the cross entropy loss function to calculate the training loss, adjust the network parameters to minimize the loss function, and implement the training of the Transformer network model.

[0023] Optionally, the S2 specifically includes:

[0024] S21, divide the input speech signal into frames, the length of each frame is N sampling points, the length of the overlapping part between adjacent frames is 0, and the frame length and frame shift are set to 512 and 256;

[0025] S22, applying a Hamming window function to each frame of speech signal:

[0026]

[0027] x w (n) = x(n)·w(n);

[0028] Where w(n) represents the Hamming window function, N represents the frame length, n represents a constant, x(n) represents the frame speech signal, and x w (n) represents the windowed signal;

[0029] S23, perform short-time Fourier transform on the windowed signal to convert the time domain signal into a frequency domain signal:

[0030]

[0031] Where X(k) represents the frequency domain signal, k represents the index of the frequency component, and j represents the imaginary unit;

[0032] S24. Calculate the power spectral density of the spectrum:

[0033]

[0034] Among them, P(k) represents the power spectral density, which represents the power of each frequency component;

[0035] S25. Process the power spectrum density through the Mel filter bank to map the linear frequency domain to the Mel frequency domain:

[0036]

[0037] Among them, S m represents the energy of the mth Mel band, H m (k) represents the weight matrix of the filter bank, and m represents a constant;

[0038] S26, taking the logarithm of the energy of each Mel frequency band to obtain a logarithmic Mel spectrum;

[0039] S27. Apply discrete cosine transform to the logarithmic Mel spectrum to extract the Mel frequency cepstrum coefficients:

[0040]

[0041] Where C(n) represents the nth cepstral coefficient;

[0042] S28. Combining the features of each frame to form a feature matrix, wherein the features include Mel frequency cepstral coefficients and logarithmic Mel spectrum.

[0043] Optionally, the S3 specifically includes:

[0044] S31, inputting the extracted feature matrix into the trained Transformer network model, the input layer of the Transformer network model receives the feature matrix, and extracting local time-frequency features through the first layer of convolution operation;

[0045] S32, performing a pooling operation on the output of the convolutional layer, wherein the pooling method is maximum pooling;

[0046] S33, applying an activation function to the output after pooling, wherein the activation function adopts a ReLU function;

[0047] S34, after multi-layer convolution, activation and pooling operations, high-level time-frequency features are extracted to generate intermediate feature representations;

[0048] S35, input the intermediate feature representation into the fully connected layer for classification reasoning, output the probability value of each frame of speech belonging to 48 pinyin categories, and generate the pinyin category probability distribution:

[0049]

[0050] Among them, P(y=c|x) represents the probability that the input feature belongs to category c, z c represents the network output value corresponding to category c, K represents the total number of categories, z k represents the network output value corresponding to category k, exp represents the exponential function, and y represents the input feature;

[0051] S36. Combining the probability values ​​of all time frames of the pinyin category probability distribution into a frame probability tensor, the shape of the frame probability tensor is (T, 48), T represents the number of speech frames, and 48 represents the number of pinyin categories.

[0052] Optionally, the S4 specifically includes:

[0053] S41, configuring the target wake-up word content, and mapping the Chinese characters of the target wake-up word to corresponding consonant and vowel pinyin sequences through a dictionary;

[0054] S42. Construct the initial state set of the finite state machine, define the mapping of the initial state, and allow the use of the mute symbol <sil>and unrelated phoneme symbols <gbg>Self-jump, which means keeping the current state when the phonemes or silence segments are not the target wake-up words;

[0055] S43, creating a state node set in sequence according to the pinyin sequence of the target wake-up word, constructing a state transition path from the initial state to the wake-up completion state, and in each state node, allowing a jump to the next state according to the corresponding pinyin phoneme:

[0056]

[0057] Among them, q i and q i+1 Represent two adjacent states, p i represents the score of the i-th phoneme, i.e. the probability value;

[0058] S44: Setting a self-jump path for each state node, wherein the self-jump path absorbs the same phoneme in consecutive frames or processes a short silent symbol. <sil>:

[0059]

[0060] Among them, q i Indicates the current state, p i Indicates the score of the corresponding phoneme;

[0061] S45. Optimize all paths in the finite state machine, merge identical or repeated paths, merge state nodes of the finite state machine with multiple wake-up words, and avoid redundant nodes and paths.

[0062] S46: Generate a final finite state machine decoding network, wherein the finite state machine decoding network contains paths of all wake-up word phonemes.

[0063] Optionally, the S5 specifically includes:

[0064] S51, extracting the phoneme categories related to the target wake-up word in the frame probability tensor as separate scores to form a target phoneme score set, wherein the scores represent the possibility that the current frame belongs to each category;

[0065] S52. Silence symbol in the frame probability tensor <sil>The scores of are retained separately to generate a set of silent symbol scores;

[0066] S53, merge the scores of all phoneme categories in the frame probability tensor that are not related to the wake-up word into <gbg>Category score;

[0067] S54, the target phoneme score set, the silent symbol score set and <gbg>The class scores are combined into a new frame probability tensor, and the new frame probability tensor is arranged in time series order.

[0068] Optionally, the S6 specifically includes:

[0069] S61, inputting the new frame probability tensor into the finite state machine decoding network, and defining the state transition weight as the corresponding score in the frame probability tensor;

[0070] S62, initializing the starting state of the Viterbi algorithm to the initial state of the finite state machine, and defining the initial path score of each state to be negative infinity, and initializing the starting state score to 0;

[0071] S63, traverse each time frame in turn, and calculate the path score for each state:

[0072] V t (q j )=max qi∈Q [V t-1 (q i )+logp ij ];

[0073] Among them, V t (q j ) represents the state q in time frame t j The path score, V t-1 (q i ) represents the state q in time frame t-1 i The path score, p ij Indicates that from state q i Transfer to state q j The score, Q represents the node set of the finite state machine decoding network, and max represents the maximum value;

[0074] S64, record the best precursor state set for path transfer:

[0075] Backpointer(q j ,t)=argmax qi∈Q [V T-1 (q i )+log p ij ];

[0076] Among them, Backpointer(q j ,t) represents the state q in time frame t j The best front-wheel drive state, q f Indicates the target end state of the finite state machine, V T-1 (q i ) represents the state q of the time frame T-1 i Path score;

[0077] S65. Iterate and update the state path scores of all time frames until the last time frame is calculated, and record the maximum path score of the target state:

[0078] V T (q f )=max qi∈Q [V T-1 (q i )+logp if ];

[0079] Among them, V T (q f ) represents the state q of time frame T f The path score, q f Indicates the target end state of the finite state machine, V T-1 (q i ) represents the state q of the time frame T-1 i Path score;

[0080] S66. Backtrack the predecessor state set, construct a phoneme sequence through the recorded optimal phoneme path, and determine whether the path covers the phoneme sequence of the target wake-up word. If the path covers the phoneme sequence of the target wake-up word, output the phoneme sequence and path score of the path.

[0081] Optionally, the S7 specifically includes:

[0082] S71, according to the searched optimal phoneme path, obtaining a maximum score set of each phoneme state in the optimal phoneme path;

[0083] S72, normalizing the state score of each phoneme, and performing weighted average on the normalized maximum score set of the phonemes to calculate the path score of the wake-up word;

[0084] S73, smoothing the path score of the wake-up word, and calculating the moving average of the score using a sliding window mechanism;

[0085] S74: Compare the final score of the wake-up word with the preset threshold value. When the final score of the wake-up word exceeds the preset threshold value, activate the wake-up word; otherwise, return to the initial state and continue monitoring.

[0086] The beneficial effects of the present invention are:

[0087] First, by deeply combining the Transformer network model with the finite state machine, the present invention does not need to train a neural network model for each wake-up word separately. Instead, the new wake-up word can be adapted by simply configuring the wake-up word content. This design significantly improves the flexibility of the system, and users can quickly add, delete, or modify wake-up words without retraining the model, thereby meeting the needs for dynamic expansion and customization in practical applications.

[0088] Secondly, the present invention improves the detection accuracy and robustness of the system in complex environments by optimizing the feature extraction, neural network reasoning and finite state machine decoding of speech signals. In the feature extraction step, features such as Mel frequency cepstral coefficients and logarithmic Mel spectrum are used to fully capture the spectral characteristics of the speech signal, providing high-quality input for subsequent neural network reasoning. The Transformer network model infers each frame of speech through the self-attention mechanism, effectively mining the temporal relationship and global features in the speech signal, and generating a high-precision phoneme probability distribution. This design not only improves the system's ability to suppress false wake-up in multi-voice interference scenarios, but also ensures low-power operation of the system on embedded devices through a lightweight model architecture.

[0089] Again, the introduction of the finite state machine is an important innovation of the present invention. It maps the pinyin sequence of the target wake-up word's initials and finals into a state transition path, and effectively distinguishes the target phoneme sequence from other interfering phonemes through dynamic decoding. The finite state machine can absorb silent symbols and irrelevant phoneme symbols to achieve fault-tolerant processing of irrelevant segments in the speech signal, further reducing the probability of false wake-up. Combined with the Viterbi algorithm, the system can efficiently search for the optimal phoneme path and calculate the score of the wake-up word. The score is calculated based on the average of the maximum value of the phoneme state, and the comparison mechanism with the threshold value effectively avoids the phenomenon of missed wake-up. At the same time, the present invention uses the deterministic and minimization algorithms of the finite state machine to optimize and merge multiple wake-up word paths, greatly reducing the redundant paths in the decoding process and improving decoding efficiency and resource utilization.

[0090] In addition, the present invention shows excellent robustness in complex scenarios, including noisy environments, far-field speech, and multi-speaker scenarios. By combining the frame probability output of the neural network with the decoding rules of the finite state machine, the system can adapt to the diversity of speech signals and ensure high-precision wake-up detection results. At the same time, by introducing a sliding window and dynamic smoothing mechanism, the wake-up word score is post-processed, further improving the stability of wake-up detection.

[0091] In summary, the present invention realizes flexible wake-up word configuration, efficient multi-wake-up word detection, low-power operation, and high-precision and robust wake-up detection functions, which solves the problems of poor flexibility, high computational complexity, high false wake-up rate and insufficient environmental adaptability in the prior art, and provides an efficient and practical voice wake-up solution for intelligent voice interaction devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0093] Figure 1 A flowchart of a custom voice wake-up word detection method based on a finite state machine proposed by the present invention;

[0094] Figure 2 This is a processing schematic diagram of speech signal framing, windowing and feature extraction of a custom speech wake-up word detection method based on a finite state machine proposed in the present invention. DETAILED DESCRIPTION

[0095] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0096] refer to Figure 1 and Figure 2 , a custom voice wake-up word detection method based on a finite state machine, comprising the following steps:

[0097] S1. Use the open source Chinese speech dataset on the Internet to train the Transformer network model;

[0098] S2, perform frame and window processing on the input speech signal, use 512-point window length and 256-point frame shift parameters to frame the speech signal, and extract corresponding features;

[0099] S3, input the extracted features into the trained Transformer network model, infer the features of each frame, generate the pinyin category probability distribution of each frame of speech, and form a frame probability tensor;

[0100] S4. Based on the target wake-up word and the probability distribution of pinyin categories, configure the target wake-up word content and obtain the pinyin sequence of consonants and vowels corresponding to the target wake-up word through dictionary mapping, and build a finite state machine decoding network according to the pinyin sequence of consonants and vowels;

[0101] S5. Process the frame probability tensor, retain the phoneme category scores related to the target wake-up word in the frame probability tensor, and replace the silent symbol <sil>Keep it as a separate category and combine the scores of the remaining categories into <gbg>Score, generate a new frame probability tensor;

[0102] S6. Input the new frame probability tensor into the finite state machine decoding network, run the Viterbi algorithm, search for the optimal phoneme path, and determine whether the optimal phoneme path completes the phoneme sequence corresponding to the wake-up word;

[0103] S7. According to the searched optimal phoneme path, record the maximum score of each phoneme state and calculate the average of all phoneme state scores as the wake-up word score, and compare the wake-up word score with the preset threshold value. When the wake-up word score exceeds the preset threshold value, activate the wake-up word, otherwise return to the initial state to continue monitoring.

[0104] In this implementation manner, the S1 specifically includes:

[0105] S11, collecting an open source Chinese speech data set from the Internet, wherein the open source Chinese speech data set includes speech signals and corresponding transcribed texts, and normalizing the speech data through data preprocessing, wherein the normalization includes unifying the sampling rate, filtering invalid audio, and format conversion, and standardizing all speech files to a PCM format with a 16k sampling rate, a single channel, and 16-bit quantization;

[0106] S12, mapping Chinese characters in the transcribed text into initials and finals combinations word by word through a dictionary, generating corresponding initials and finals pinyin sequences, wherein the initials and finals pinyin sequences include 23 initials and 24 finals, and do not consider tones and erhua phenomena;

[0107] S13, define the output categories of the Transformer network as 48 categories, including 23 initials, 24 finals, and 1 mute symbol <sil>, the mute symbol is used to mark silent frame segments;

[0108] S14. Design the training objectives of the Transformer network model, take the feature vector of each frame of speech signal as input, use the cross entropy loss function to calculate the training loss, adjust the network parameters to minimize the loss function, and implement the training of the Transformer network model.

[0109] In this implementation, S2 specifically includes:

[0110] S21, divide the input speech signal into frames, the length of each frame is N sampling points, the length of the overlapping part between adjacent frames is 0, and the frame length and frame shift are set to 512 and 256;

[0111] S22, applying a Hamming window function to each frame of speech signal:

[0112]

[0113] x w (n) = x(n)·w(n);

[0114] Where w(n) represents the Hamming window function, N represents the frame length, n represents a constant, x(n) represents the frame speech signal, and x w (n) represents the windowed signal;

[0115] S23, perform short-time Fourier transform on the windowed signal to convert the time domain signal into a frequency domain signal:

[0116]

[0117] Where X(k) represents the frequency domain signal, k represents the index of the frequency component, and j represents the imaginary unit;

[0118] S24. Calculate the power spectral density of the spectrum:

[0119]

[0120] Among them, P(k) represents the power spectral density, which represents the power of each frequency component;

[0121] S25. Process the power spectrum density through the Mel filter bank to map the linear frequency domain to the Mel frequency domain:

[0122]

[0123] Among them, S m represents the energy of the mth Mel band, H m (k) represents the weight matrix of the filter bank, and m represents a constant;

[0124] S26, taking the logarithm of the energy of each Mel frequency band to obtain a logarithmic Mel spectrum;

[0125] S27. Apply discrete cosine transform to the logarithmic Mel spectrum to extract the Mel frequency cepstrum coefficients:

[0126]

[0127] Where C(n) represents the nth cepstral coefficient;

[0128] S28. Combining the features of each frame to form a feature matrix, wherein the features include Mel frequency cepstral coefficients and logarithmic Mel spectrum.

[0129] In this implementation, S3 specifically includes:

[0130] S31, inputting the extracted feature matrix into the trained Transformer network model, the input layer of the Transformer network model receives the feature matrix, and extracting local time-frequency features through the first layer of convolution operation;

[0131] S32, performing a pooling operation on the output of the convolutional layer, wherein the pooling method is maximum pooling;

[0132] S33, applying an activation function to the output after pooling, wherein the activation function adopts a ReLU function;

[0133] S34, after multi-layer convolution, activation and pooling operations, high-level time-frequency features are extracted to generate intermediate feature representations;

[0134] S35, input the intermediate feature representation into the fully connected layer for classification reasoning, output the probability value of each frame of speech belonging to 48 pinyin categories, and generate the pinyin category probability distribution:

[0135]

[0136] Among them, P(y=c|x) represents the probability that the input feature belongs to category c, z c represents the network output value corresponding to category c, K represents the total number of categories, z k represents the network output value corresponding to category k, exp represents the exponential function, and y represents the input feature;

[0137] S36. Combining the probability values ​​of all time frames of the pinyin category probability distribution into a frame probability tensor, the shape of the frame probability tensor is (T, 48), T represents the number of speech frames, and 48 represents the number of pinyin categories.

[0138] In this implementation, S4 specifically includes:

[0139] S41, configuring the target wake-up word content, and mapping the Chinese characters of the target wake-up word to corresponding consonant and vowel pinyin sequences through a dictionary;

[0140] S42. Construct the initial state set of the finite state machine, define the mapping of the initial state, and allow the use of the mute symbol <sil>and unrelated phoneme symbols <gbg>Self-jump, which means keeping the current state when the phonemes or silence segments are not the target wake-up words;

[0141] S43, creating a state node set in sequence according to the pinyin sequence of the target wake-up word, constructing a state transition path from the initial state to the wake-up completion state, and in each state node, allowing a jump to the next state according to the corresponding pinyin phoneme:

[0142]

[0143] Among them, q i and q i+1 Represent two adjacent states, p i represents the score of the i-th phoneme, i.e. the probability value;

[0144] S44: Setting a self-jump path for each state node, wherein the self-jump path absorbs the same phoneme in consecutive frames or processes a short silent symbol. <sil>:

[0145]

[0146] Among them, q i Indicates the current state, p i Indicates the score of the corresponding phoneme;

[0147] S45. Optimize all paths in the finite state machine, merge identical or repeated paths, merge state nodes of the finite state machine with multiple wake-up words, and avoid redundant nodes and paths.

[0148] S46: Generate a final finite state machine decoding network, wherein the finite state machine decoding network contains paths of all wake-up word phonemes.

[0149] In this implementation manner, S5 specifically includes:

[0150] S51, extracting the phoneme categories related to the target wake-up word in the frame probability tensor as separate scores to form a target phoneme score set, wherein the scores represent the possibility that the current frame belongs to each category;

[0151] S52. Silence symbol in the frame probability tensor <sil>The scores of are retained separately to generate a set of silent symbol scores;

[0152] S53, merge the scores of all phoneme categories in the frame probability tensor that are not related to the wake-up word into <gbg>Category score;

[0153] S54, the target phoneme score set, the silent symbol score set and <gbg>The class scores are combined into a new frame probability tensor, and the new frame probability tensor is arranged in time series order.

[0154] In this implementation manner, S6 specifically includes:

[0155] S61, inputting the new frame probability tensor into the finite state machine decoding network, and defining the state transition weight as the corresponding score in the frame probability tensor;

[0156] S62, initializing the starting state of the Viterbi algorithm to the initial state of the finite state machine, and defining the initial path score of each state to be negative infinity, and initializing the starting state score to 0;

[0157] S63, traverse each time frame in turn, and calculate the path score for each state:

[0158]

[0159] Among them, V t (q j ) represents the state q in time frame t j The path score, V t-1 (q i ) represents the state q in time frame t-1 i The path score, p ij Indicates that from state q i Transfer to state q j The score, Q represents the node set of the finite state machine decoding network, and max represents the maximum value;

[0160] S64, record the best precursor state set for path transfer:

[0161]

[0162] Among them, Backpointer(q j ,t) represents the state q in time frame t j The best front-wheel drive state, q f Indicates the target end state of the finite state machine, V T-1 (q i ) represents the state q of the time frame T-1 i Path score;

[0163] S65. Iterate and update the state path scores of all time frames until the last time frame is calculated, and record the maximum path score of the target state:

[0164]

[0165] Among them, V T (q f ) represents the state q of time frame T f The path score, q f Indicates the target end state of the finite state machine, V T-1 (q i ) represents the state q of the time frame T-1 i Path score;

[0166] S66. Backtrack the predecessor state set, construct a phoneme sequence through the recorded optimal phoneme path, and determine whether the path covers the phoneme sequence of the target wake-up word. If the path covers the phoneme sequence of the target wake-up word, output the phoneme sequence and path score of the path.

[0167] In this implementation manner, the S7 specifically includes:

[0168] S71, according to the searched optimal phoneme path, obtaining a maximum score set of each phoneme state in the optimal phoneme path;

[0169] S72, normalizing the state score of each phoneme, and performing weighted average on the normalized maximum score set of the phonemes to calculate the path score of the wake-up word;

[0170] S73, smoothing the path score of the wake-up word, and calculating the moving average of the score using a sliding window mechanism;

[0171] S74: Compare the final score of the wake-up word with the preset threshold value. When the final score of the wake-up word exceeds the preset threshold value, activate the wake-up word; otherwise, return to the initial state and continue monitoring.

[0172] Embodiment 1:

[0173] This embodiment provides a specific application scenario of a custom voice wake-up word detection method based on a finite state machine. The scenario simulates a user using the voice wake-up function of a smart speaker in a home environment, and the target wake-up words are "Hello Xiaodu", "Tmall Genie" and "iFLYTEK". The home environment contains a variety of complex audio interference sources, such as background music, household appliance noise, multi-person conversations, etc. During the test, the user needs to wake up the smart speaker by saying the target wake-up word. After detecting the wake-up word, the speaker lights up the prompt light and enters the standby state. This test aims to evaluate the flexibility, low false wake-up rate, high robustness and multi-wake-up word support capability of the method proposed in the present invention in actual scenarios.

[0174] In the test, the Transformer network model was trained using an open source Chinese speech dataset, and the sampling rate of the input speech was unified to 16kHz. First, the target wake-up words "Hello Xiaodu", "Tmall Genie" and "iFLYTEK" were mapped into a sequence of initials and finals through a dictionary, and a decoding network was constructed for each wake-up word through a finite state machine. All paths shared the initial state, and a total of 1,000 test audio data were collected, including 500 voice data of three target wake-up words, 300 non-target voices (such as ordinary conversations unrelated to the target wake-up words), and 200 random voices with background noise. The audio length ranged from 1 second to 5 seconds. The test was conducted in three typical environments, including a quiet environment (no obvious noise in the room), a light noise environment (background music or TV sound), and a heavy noise environment (multi-person conversations, kitchen operation sounds). The test equipment is an embedded smart speaker equipped with a single-channel microphone. During the test, each audio is divided into frames, windows and features are extracted to generate a frame probability tensor, which is then input into the trained Transformer network model to output the phonetic category probability distribution for each frame. Based on the finite state machine decoding network, the Viterbi algorithm is used to calculate the phoneme path score and determine whether to activate the wake-up word.

[0175] Table 1 Experimental data comparison table

[0176] Test Dimensions Quiet environment Light noise environment Heavy noise environment Total target wake-up times 200 200 100 Number of successful wake-ups 198 192 89 Wake-up rate 99％ 96％ 89％ Total number of non-target speech 100 100 100 False wake-up times 1 3 6 False wake-up rate 1％ 3％ 6％ Average detection delay (ms) 120 145 185 Average power consumption (W) 0.5 0.6 0.8

[0177] It can be seen from the above experimental data comparison table that the voice wake-up method of the present invention exhibits excellent performance and robustness in different environments. In a quiet environment, the wake-up rate is as high as 99.0%, the false wake-up rate is only 1.0%, and the average detection delay is 120 milliseconds. It can quickly respond to the user's wake-up command, showing extremely high accuracy and sensitivity. This shows that the present invention can complete the voice wake-up task well in an interference-free scenario, providing users with a fast and efficient experience, and is particularly suitable for quiet home or office environments. In this environment, the finite state machine combined with the Transformer network's modeling and decoding capabilities for the wake-up word phoneme sequence are fully utilized, and the target phoneme path can be accurately identified and the wake-up word can be quickly activated.

[0178] In a light noise environment (such as background music or TV sound), the performance of the present invention is still satisfactory, the wake-up rate remains at 96.0%, the false wake-up rate slightly increases to 3.0%, and the average detection delay is 145 milliseconds, showing good anti-interference ability. This result shows that the neural network model and finite state machine decoding algorithm of the present invention can effectively handle the interference of background noise, especially the silent symbol. <sil>and irrelevant phonemes <gbg>The absorption mechanism plays a key role. Although the clarity of the voice signal is affected to a certain extent by the ambient noise, the system can still accurately distinguish the target wake-up word through high-quality feature extraction and a robust path matching algorithm. This performance makes the present invention suitable for daily use in ordinary household scenarios, for example, it can still reliably wake up the device when the user plays background music.

[0179] In more complex heavy noise environments (such as multi-person conversations, kitchen noise, etc.), the present invention shows strong robustness, with a wake-up rate of 89.0%, a false wake-up rate of 6.0%, and an average detection delay of 185 milliseconds. Although the complex noise environment causes great interference to the voice signal, the present invention can still effectively complete the wake-up task. In particular, with the help of the path optimization and deterministic algorithm of the finite state machine, the system effectively filters irrelevant phonemes in the noise. The Viterbi algorithm further enhances the search capability for the optimal phoneme path, ensuring that the target wake-up word still has high detectability in complex environments. Although the performance has declined, it is still within an acceptable range, indicating that the present invention can adapt to more complex scenarios and meet the needs of users in noisy environments.

[0180] In terms of power consumption, the average power consumption of the present invention in different noise environments is maintained at a low level, which is 0.5W, 0.6W and 0.8W respectively. Even in the most complex heavy noise environment, it can still operate under low power consumption conditions and adapt to resource-constrained embedded devices. This low power consumption feature is due to the efficient design of the lightweight Transformer network model and the finite state machine decoding network of the present invention. The parameter volume of the model is only 1.2M, and a single decoding takes only 80 milliseconds, which enables the system to maintain high performance and significantly reduce power consumption in real-time operation. This feature makes the present invention particularly suitable for embedded devices with strict power consumption requirements, such as smart speakers and car voice assistants, while also meeting the user's demand for long-term standby operation of the device.

[0181] In short, the experimental data fully verifies the effectiveness and practicality of the present invention in voice wake-up detection. The present invention not only has excellent wake-up rate and false wake-up rate in different environments, but also shows advantages in multiple wake-up word support and low power consumption operation. This design with high precision, flexibility and robustness provides a reliable solution for intelligent voice interaction systems, which can meet the application needs of multiple scenarios such as home and car.

[0182] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.< / gbg> < / sil> < / gbg> < / gbg> < / sil> < / sil> < / gbg> < / sil> < / sil> < / gbg> < / sil> < / gbg> < / gbg> < / sil> < / sil> < / gbg> < / sil> < / sil> < / gbg> < / sil>

Claims

1. A custom voice wake-up word detection method based on a finite state machine, characterized in that: The steps include: S1. Use the open source Chinese speech dataset on the Internet to train the Transformer network model; S2, perform frame and window processing on the input speech signal, use 512-point window length and 256-point frame shift parameters to frame the speech signal, and extract corresponding features; S3, input the extracted features into the trained Transformer network model, infer the features of each frame, generate the pinyin category probability distribution of each frame of speech, and form a frame probability tensor; S4. Based on the target wake-up word and the probability distribution of pinyin categories, configure the target wake-up word content and obtain the pinyin sequence of consonants and vowels corresponding to the target wake-up word through dictionary mapping, and build a finite state machine decoding network according to the pinyin sequence of consonants and vowels; S5. Process the frame probability tensor, retain the phoneme category scores related to the target wake-up word in the frame probability tensor, and replace the silent symbol <sil>Keep it as a separate category and combine the scores of the remaining categories into <gbg> Score, generate a new frame probability tensor;< / gbg> < / sil> S6. Input the new frame probability tensor into the finite state machine decoding network, run the Viterbi algorithm, search for the optimal phoneme path, and determine whether the optimal phoneme path completes the phoneme sequence corresponding to the wake-up word; S7. According to the searched optimal phoneme path, record the maximum score of each phoneme state and calculate the average of all phoneme state scores as the wake-up word score, and compare the wake-up word score with the preset threshold value. When the wake-up word score exceeds the preset threshold value, activate the wake-up word, otherwise return to the initial state to continue monitoring.

2. According to the method for detecting a custom voice wake-up word based on a finite state machine according to claim 1, it is characterized in that: The S1 specifically includes: S11, collecting an open source Chinese speech data set from the Internet, wherein the open source Chinese speech data set includes speech signals and corresponding transcribed texts, and normalizing the speech data through data preprocessing, wherein the normalization includes unifying the sampling rate, filtering invalid audio, and format conversion, and standardizing all speech files to a PCM format with a 16k sampling rate, a single channel, and 16-bit quantization; S12, mapping Chinese characters in the transcribed text into initials and finals combinations word by word through a dictionary, generating corresponding initials and finals pinyin sequences, wherein the initials and finals pinyin sequences include 23 initials and 24 finals, and do not consider tones and erhua phenomena; S13, define the output categories of the Transformer network as 48 categories, including 23 initials, 24 finals, and 1 mute symbol <sil> , the mute symbol is used to mark silent frame segments;< / sil> S14. Design the training objectives of the Transformer network model, take the feature vector of each frame of speech signal as input, use the cross entropy loss function to calculate the training loss, adjust the network parameters to minimize the loss function, and implement the training of the Transformer network model.

3. According to the method for detecting a custom voice wake-up word based on a finite state machine according to claim 1, it is characterized in that: The S2 specifically includes: S21, divide the input speech signal into frames, the length of each frame is N sampling points, the length of the overlapping part between adjacent frames is 0, and the frame length and frame shift are set to 512 and 256; S22, applying a Hamming window function to each frame of speech signal: x w (n)=x(n)·w(n); Where w(n) represents the Hamming window function, N represents the frame length, n represents a constant, x(n) represents the frame speech signal, and x w (n) represents the windowed signal; S23, perform short-time Fourier transform on the windowed signal to convert the time domain signal into a frequency domain signal: Where X(k) represents the frequency domain signal, k represents the index of the frequency component, and j represents the imaginary unit; S24. Calculate the power spectral density of the spectrum: Among them, P(k) represents the power spectral density, which represents the power of each frequency component; S25. Process the power spectrum density through the Mel filter bank to map the linear frequency domain to the Mel frequency domain: Among them, S m represents the energy of the mth Mel band, H m (k) represents the weight matrix of the filter bank, and m represents a constant; S26, taking the logarithm of the energy of each Mel frequency band to obtain a logarithmic Mel spectrum; S27. Apply discrete cosine transform to the logarithmic Mel spectrum to extract the Mel frequency cepstrum coefficients: Where C(n) represents the nth cepstral coefficient; S28. Combining the features of each frame to form a feature matrix, wherein the features include Mel frequency cepstral coefficients and logarithmic Mel spectrum.

4. According to the method for detecting a custom voice wake-up word based on a finite state machine according to claim 1, it is characterized in that: The S3 specifically includes: S31, inputting the extracted feature matrix into the trained Transformer network model, the input layer of the Transformer network model receives the feature matrix, and extracting local time-frequency features through the first layer of convolution operation; S32, performing a pooling operation on the output of the convolutional layer, wherein the pooling method is maximum pooling; S33, applying an activation function to the output after pooling, wherein the activation function adopts a ReLU function; S34, after multi-layer convolution, activation and pooling operations, high-level time-frequency features are extracted to generate intermediate feature representations; S35, input the intermediate feature representation into the fully connected layer for classification reasoning, output the probability value of each frame of speech belonging to 48 pinyin categories, and generate the pinyin category probability distribution: Among them, P(y=c|x) represents the probability that the input feature belongs to category c, z c represents the network output value corresponding to category c, K represents the total number of categories, z k represents the network output value corresponding to category k, exp represents the exponential function, and y represents the input feature; S36. Combining the probability values ​​of all time frames of the pinyin category probability distribution into a frame probability tensor, the shape of the frame probability tensor is (T, 48), T represents the number of speech frames, and 48 represents the number of pinyin categories.

5. According to the method for detecting a custom voice wake-up word based on a finite state machine according to claim 1, it is characterized in that: The S4 specifically includes: S41, configuring the target wake-up word content, and mapping the Chinese characters of the target wake-up word to corresponding consonant and vowel pinyin sequences through a dictionary; S42. Construct the initial state set of the finite state machine, define the mapping of the initial state, and allow the use of the mute symbol <sil>and unrelated phoneme symbols <gbg> Self-jump, which means keeping the current state when the phonemes or silence segments are not the target wake-up words;< / gbg> < / sil> S43, creating a state node set in sequence according to the pinyin sequence of the target wake-up word, constructing a state transition path from the initial state to the wake-up completion state, and in each state node, allowing a jump to the next state according to the corresponding pinyin phoneme: Among them, q i and q i+1 Represent two adjacent states, p i represents the score of the i-th phoneme, i.e. the probability value; S44: Setting a self-jump path for each state node, wherein the self-jump path absorbs the same phoneme in consecutive frames or processes a short silent symbol. <sil> ：< / sil> Among them, q i Indicates the current state, p i Indicates the score of the corresponding phoneme; S45. Optimize all paths in the finite state machine, merge identical or repeated paths, merge state nodes of the finite state machine with multiple wake-up words, and avoid redundant nodes and paths. S46: Generate a final finite state machine decoding network, wherein the finite state machine decoding network contains paths of all wake-up word phonemes.

6. According to the method for detecting a custom voice wake-up word based on a finite state machine according to claim 1, it is characterized in that: The S5 specifically includes: S51, extracting the phoneme categories related to the target wake-up word in the frame probability tensor as separate scores to form a target phoneme score set, wherein the scores represent the possibility that the current frame belongs to each category; S52. Silence symbol in the frame probability tensor <sil> The scores of are retained separately to generate a set of silent symbol scores;< / sil> S53, merge the scores of all phoneme categories in the frame probability tensor that are not related to the wake-up word into <gbg> Category score;< / gbg> S54, the target phoneme score set, the silent symbol score set and <gbg> The class scores are combined into a new frame probability tensor, and the new frame probability tensor is arranged in time series order.< / gbg> 7. According to the method for detecting a custom voice wake-up word based on a finite state machine according to claim 1, it is characterized in that: The S6 specifically includes: S61, inputting the new frame probability tensor into the finite state machine decoding network, and defining the state transition weight as the corresponding score in the frame probability tensor; S62, initializing the starting state of the Viterbi algorithm to the initial state of the finite state machine, and defining the initial path score of each state to be negative infinity, and initializing the starting state score to 0; S63, traverse each time frame in turn, and calculate the path score for each state: V t (q j )=max qi∈Q [V t-1 (q i )+logp ij ]; Among them, V t (q j ) represents the state q in time frame t j The path score, V t-1 (q i ) represents the state q in time frame t-1 i The path score, p ij Indicates that from state q i Transfer to state q j The score, Q represents the node set of the finite state machine decoding network, and max represents the maximum value; S64, record the best precursor state set for path transfer: Among them, Backpointer(q j ,t) represents the state q in time frame t j The best front-wheel drive state, q f Indicates the target end state of the finite state machine, V T-1 (q i ) represents the state q of the time frame T-1 i Path score; S65. Iterate and update the state path scores of all time frames until the last time frame is calculated, and record the maximum path score of the target state: Among them, V T (q f ) represents the state q of time frame T f The path score, q f Indicates the target end state of the finite state machine, V T-1 (q i ) represents the state q of the time frame T-1 i Path score; S66. Backtrack the predecessor state set, construct a phoneme sequence through the recorded optimal phoneme path, and determine whether the path covers the phoneme sequence of the target wake-up word. If the path covers the phoneme sequence of the target wake-up word, output the phoneme sequence and path score of the path.

8. According to the method for detecting a custom voice wake-up word based on a finite state machine according to claim 1, it is characterized in that: The S7 specifically includes: S71, according to the searched optimal phoneme path, obtaining a maximum score set of each phoneme state in the optimal phoneme path; S72, normalizing the state score of each phoneme, and performing weighted average on the normalized maximum score set of the phonemes to calculate the path score of the wake-up word; S73, smoothing the path score of the wake-up word, and calculating the moving average of the score using a sliding window mechanism; S74: Compare the final score of the wake-up word with the preset threshold value. When the final score of the wake-up word exceeds the preset threshold value, activate the wake-up word; otherwise, return to the initial state and continue monitoring.

Images