Device wake-up method, apparatus, and computer-readable storage medium

Abstract

The present disclosure relates to a device wake-up method, device and computer readable storage medium, and relates to the field of device control. The method comprises: obtaining a plurality of phoneme frames according to received voice information, each phoneme frame having a preset time length; inputting the plurality of phoneme frames into a preset acoustic model to obtain a first phoneme sequence, the first phoneme sequence comprising a plurality of phonemes corresponding to the plurality of phoneme frames and a determination probability of each phoneme; and determining a wake-up result of the voice information according to the first phoneme sequence and the determination probability of the plurality of phonemes in the first phoneme sequence, in a case where invalid phonemes and / or missing phonemes in the first phoneme sequence meet a preset condition. The present disclosure can realize recognition of phonemes in voice information without setting a complex decoder, and can simplify phoneme-level voice wake-up technology.

Description

Technical Field

[0001] This disclosure relates to the field of device control, and more particularly to a device wake-up method, apparatus, and computer-readable storage medium. Background Technology

[0002] With the development of intelligent voice technology, more and more intelligent voice devices have emerged. Voice wake-up is the entry point for intelligent voice devices to interact with users. It refers to the first step in starting voice interaction when an intelligent voice device (such as a mobile phone, home appliance, or in-vehicle system) detects a specific voice command while in sleep or locked state.

[0003] Voice wake-up technology can be divided into keyword recognition and voiceprint recognition, depending on the target of recognition. Keyword recognition refers to identifying specific voice commands, or wake-up words, within a continuous stream of speech. Voiceprint recognition, also known as speaker recognition, is a technology that extracts the speaker's voice features to verify the speaker's identity. Keyword recognition technology for voice wake-up can be further subdivided into end-to-end voice wake-up, word-level voice wake-up, and phoneme-level voice wake-up.

[0004] In related technologies, for phoneme-level voice wake-up technology, it is usually necessary to first determine the phoneme corresponding to each frame of the speech, and then to use a decoder for further judgment. The implementation of this decoder is relatively complex and has certain hardware requirements. Summary of the Invention

[0005] To overcome the problems existing in the related technologies, this disclosure provides a device wake-up method, apparatus and computer-readable storage medium.

[0006] According to a first aspect of the present disclosure, a device wake-up method is provided, comprising:

[0007] Multiple phoneme frames are obtained based on the received voice information, and the duration of each phoneme frame is a preset duration;

[0008] The multiple phoneme frames are input into a preset acoustic model to obtain a first phoneme sequence, which includes multiple phonemes corresponding to the multiple phoneme frames and the determination probability of each phoneme.

[0009] If invalid phonemes and / or missing phonemes in the first phoneme sequence meet preset conditions, the wake-up result of the voice information is determined based on the first phoneme sequence and the determination probability of multiple phonemes in the first phoneme sequence.

[0010] Optionally, the step of inputting the plurality of phoneme frames into a preset acoustic model to obtain a first phoneme sequence, the first phoneme sequence including a plurality of phonemes corresponding to the plurality of phoneme frames, and a determination probability for each of the phonemes, includes:

[0011] The first phoneme sequence is input frame by frame into the acoustic model to obtain multiple phoneme sequences output by the acoustic model. The multiple phoneme sequences correspond one-to-one with the multiple phoneme frames. Each phoneme sequence contains at least one phoneme and the determination probability of the at least one phoneme.

[0012] For each of the plurality of phoneme frames, the phoneme with the highest determination probability in the phoneme sequence corresponding to the phoneme frame is determined as the phoneme corresponding to the phoneme frame.

[0013] The multiple phonemes corresponding to the obtained multiple phoneme frames are used as the first phoneme sequence.

[0014] Optionally, for each of the plurality of phoneme frames, determining the phoneme with the highest determination probability in the phoneme sequence corresponding to the phoneme frame as the phoneme corresponding to the phoneme frame includes:

[0015] The determination probability of each phoneme in the phoneme sequence corresponding to the phoneme frame is normalized to obtain the normalized determination probability of each phoneme.

[0016] The phoneme with the highest normalized probability in the phoneme sequence is determined as the phoneme corresponding to the phoneme frame.

[0017] Optionally, the invalid phonemes include isolated phonemes and additional phonemes, and the method further includes:

[0018] Obtain the second phoneme sequence corresponding to the preset standard wake-up voice information;

[0019] Based on the second phoneme sequence, determine whether the isolated phoneme, the additional phoneme, and / or the missing phoneme exist in the first phoneme sequence; wherein, any phoneme that is unrelated to the preceding adjacent phoneme and / or the following adjacent phoneme is the isolated phoneme; any phoneme that is unrelated to the preceding adjacent phoneme and / or the following adjacent phoneme and is not an isolated phoneme is the additional phoneme; any phoneme that exists in the second phoneme sequence that is related to the preceding adjacent phoneme and / or the following adjacent phoneme and does not exist in the first phoneme sequence is the missing phoneme;

[0020] If it is determined that the isolated phoneme, the additional phoneme, and / or the missing phoneme exist in the first phoneme sequence, it is determined whether the isolated phoneme, the additional phoneme, and / or the missing phoneme satisfy the preset condition.

[0021] Optionally, the preset conditions include:

[0022] The number of isolated phonemes is less than a first quantity threshold, the number of additional phonemes is less than a second quantity threshold, and the number of missing phonemes is less than a third quantity threshold.

[0023] Optionally, when it is determined that invalid phonemes and / or missing phonemes in the first phoneme sequence meet preset conditions, determining the wake-up result of the voice information based on the first phoneme sequence and the determination probabilities of multiple phonemes in the first phoneme sequence includes:

[0024] Remove the phonemes whose determination probability is less than the probability threshold from the multiple phonemes in the first phoneme sequence to obtain the third phoneme sequence;

[0025] Obtain the sum of the determination probabilities of multiple phonemes in the third phoneme sequence;

[0026] If the sum of the determined probabilities is less than the total probability threshold, the voice information wake-up is determined to have failed.

[0027] If the sum of the determined probabilities is greater than or equal to the total probability threshold, the voice information is determined to be successfully activated.

[0028] Optionally, the step of inputting the plurality of phoneme frames frame by frame into the acoustic model to obtain the plurality of phoneme sequences output by the acoustic model includes:

[0029] Extract the feature vector of each phoneme frame from the plurality of phoneme frames;

[0030] The feature vectors of multiple phoneme frames of the first phoneme sequence are input into the acoustic model in a frame-by-frame manner to obtain the multiple phoneme sequences output by the acoustic model.

[0031] According to a second aspect of the present disclosure, a device wake-up apparatus is provided, comprising:

[0032] The acquisition module is configured to acquire multiple phoneme frames based on the received voice information, with each phoneme frame having a preset duration.

[0033] The recognition module is configured to input the plurality of phoneme frames into an acoustic model to obtain a first phoneme sequence, the first phoneme sequence including a plurality of phonemes corresponding to the plurality of phoneme frames, and a determination probability of each phoneme;

[0034] The processing module is configured to determine the wake-up result of the voice information based on the first phoneme sequence and the determination probability of multiple phonemes in the first phoneme sequence, when it is determined that invalid phonemes and / or missing phonemes in the first phoneme sequence meet preset conditions.

[0035] Optionally, the identification module includes: an identification submodule and a determination submodule;

[0036] The recognition submodule is configured to input the plurality of phoneme frames frame by frame into the acoustic model to obtain a plurality of phoneme sequences output by the acoustic model. The plurality of phoneme sequences correspond one-to-one with the plurality of phoneme frames. Each phoneme sequence contains at least one phoneme and the determination probability of the at least one phoneme.

[0037] The determining submodule is configured to, for each of the plurality of phoneme frames, determine the phoneme with the highest determination probability in the phoneme sequence corresponding to the phoneme frame as the phoneme corresponding to the phoneme frame; and use the plurality of phonemes corresponding to the plurality of phoneme frames as the first phoneme sequence.

[0038] Optionally, the determining submodule is configured as follows:

[0039] The determination probability of each phoneme in the phoneme sequence corresponding to the phoneme frame is normalized to obtain the normalized determination probability of each phoneme.

[0040] The phoneme with the highest normalized probability in the phoneme sequence is determined as the phoneme corresponding to the phoneme frame.

[0041] Optionally, the invalid phonemes include isolated phonemes and additional phonemes, and the device further includes: a preprocessing module configured to:

[0042] Obtain the second phoneme sequence corresponding to the preset standard wake-up voice information;

[0043] Based on the second phoneme sequence, determine whether the isolated phoneme, the additional phoneme, and / or the missing phoneme exist in the first phoneme sequence; wherein, any phoneme that is unrelated to the preceding adjacent phoneme and / or the following adjacent phoneme is the isolated phoneme; any phoneme that is unrelated to the preceding adjacent phoneme and / or the following adjacent phoneme and is not an isolated phoneme is the additional phoneme; any phoneme that exists in the second phoneme sequence that is related to the preceding adjacent phoneme and / or the following adjacent phoneme and does not exist in the first phoneme sequence is the missing phoneme;

[0044] If it is determined that the isolated phoneme, the additional phoneme, and / or the missing phoneme exist in the first phoneme sequence, it is determined whether the isolated phoneme, the additional phoneme, and / or the missing phoneme satisfy the preset condition.

[0045] Optionally, the preset conditions include:

[0046] The number of isolated phonemes is less than a first quantity threshold, the number of additional phonemes is less than a second quantity threshold, and the number of missing phonemes is less than a third quantity threshold.

[0047] Optionally, the processing module is configured as follows:

[0048] Remove the phonemes whose determination probability is less than the probability threshold from the multiple phonemes in the first phoneme sequence to obtain the third phoneme sequence;

[0049] Obtain the sum of the determination probabilities of multiple phonemes in the third phoneme sequence;

[0050] If the sum of the determined probabilities is less than the total probability threshold, the voice information wake-up is determined to have failed.

[0051] If the sum of the determined probabilities is greater than or equal to the total probability threshold, the voice information is determined to be successfully activated.

[0052] Optionally, the identification submodule is configured as follows:

[0053] Extract the feature vector of each phoneme frame from the plurality of phoneme frames;

[0054] The feature vectors of multiple phoneme frames of the first phoneme sequence are input into the acoustic model in a frame-by-frame manner to obtain the multiple phoneme sequences output by the acoustic model.

[0055] According to a third aspect of the present disclosure, a device wake-up device is provided, comprising:

[0056] processor;

[0057] Memory used to store processor-executable instructions;

[0058] The processor is configured to implement the steps of any one of the methods in the first aspect when executing the executable instructions.

[0059] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, having stored thereon computer program instructions, wherein the program instructions, when executed by a processor, implement the steps of the method described in any one of the first aspects.

[0060] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0061] In the above technical solution, multiple phoneme frames are obtained based on the received voice information. Each phoneme frame has a preset duration. These multiple phoneme frames are then input into a preset acoustic model to obtain a first phoneme sequence. This first phoneme sequence includes multiple phonemes corresponding to the multiple phoneme frames, as well as the determination probability of each phoneme. Then, if invalid phonemes and / or missing phonemes in the first phoneme sequence meet preset conditions, the wake-up result of the voice information is determined based on the first phoneme sequence and the determination probabilities of the multiple phonemes in the first phoneme sequence. This implementation method enables phoneme recognition in voice information without the need for a complex decoder, avoiding the hardware requirements of a complex decoder and thus simplifying phoneme-level voice wake-up technology.

[0062] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0063] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0064] Figure 1 This is a flowchart illustrating a device wake-up method according to an exemplary embodiment.

[0065] Figure 2 This is a flowchart illustrating a phoneme recognition method according to an exemplary embodiment.

[0066] Figure 3 This is a flowchart illustrating another phoneme recognition method according to an exemplary embodiment.

[0067] Figure 4 This is a block diagram illustrating a device wake-up device according to an exemplary embodiment.

[0068] Figure 5 This is a block diagram illustrating a device wake-up device according to an exemplary embodiment. Detailed Implementation

[0069] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0070] In related technologies, phoneme-level voice wake-up technology typically involves two stages: a training stage and a recognition stage. The training stage involves training acoustic and language models using speech and text databases, respectively. The recognition stage involves using the received speech input to obtain speech recognition results from these acoustic and language models, and then decoding the recognition results to obtain the corresponding text.

[0071] The decoder typically employs a WFST (Weighted Finite State Transducers) decoder. Before using WFST for decoding, a static decoding graph HCLG (HMMContext-dependency Lexicon Grammar) needs to be constructed. This refers to the acoustic model (HMM), context-dependency phonemes, lexicon, and language model (Grammar), where HMM stands for Hidden Markov Model. The dictionary, or pronunciation dictionary, maps phonemes to words and connects the acoustic and language models. The speech model then uses the phonemes output from the acoustic model, combined with the pronunciation dictionary, to generate the text sequence.

[0072] The main idea of ​​the WFST-based speech recognition decoding described above is to represent the three different levels of models—acoustic model, language model, and pronunciation dictionary—in the form of finite state converters, and then integrate these three different levels of finite state converters into a single weighted finite state converter to form a decoding network / search network.

[0073] However, the aforementioned WFST decoding method involves a complex and slow HCLG (Hypertext Collection Graph) construction process, making it more suitable for speech recognition scenarios requiring a text database than for speech wake-up scenarios that require recognition of fixed, limited phonemes but fast response. Furthermore, the HCLG used in WFST decoding occupies a large amount of space and memory during the decoding process, placing high demands on hardware and making direct deployment on mobile devices difficult. Therefore, to address these issues, this disclosure provides a device wake-up method applicable to any electronic device that supports voice wake-up, such as smart speakers, smartphones, tablets, smart TVs, smart wearable devices, PDAs (Personal Digital Assistants), portable computers, and other mobile terminals, as well as desktop computers and other fixed terminals. The device wake-up method is described below.

[0074] Figure 1 This is a flowchart illustrating a device wake-up method according to an exemplary embodiment, such as... Figure 1 As shown, the device wake-up method can be used in the aforementioned electronic device and includes the following steps.

[0075] In step S11, multiple phoneme frames are obtained based on the received voice information, and the duration of each phoneme frame is a preset duration.

[0076] A phoneme is the smallest unit of speech defined based on the natural attributes of speech. It is analyzed according to the articulation actions within a syllable; one action constitutes one phoneme. Phonemes are divided into two main categories: vowels and consonants. Electronic devices can collect voice information emitted by users through microphones. This voice information can be a continuous stream of speech information with a certain duration. The speech information can be divided into multiple frames according to a preset duration, with each frame corresponding to one phoneme. Therefore, one frame can be understood as containing one phoneme, hence the term "phoneme frame." For example, the preset duration can be 25 milliseconds, thus dividing the received voice information into multiple phoneme frames based on its duration.

[0077] In step S12, the multiple phoneme frames are input into a preset acoustic model to obtain a first phoneme sequence, which includes multiple phonemes corresponding to the multiple phoneme frames and the determination probability of each phoneme.

[0078] The acoustic model can be a trained acoustic model; for example, it can be the HMM model described above. By inputting multiple phoneme frames from the first phoneme sequence into the acoustic model, the phoneme corresponding to each phoneme frame and the probability of identifying each phoneme can be obtained. This probability of identification characterizes the likelihood that the phoneme frame is identified as having the corresponding phoneme. For example, if the acoustic model outputs that phoneme frame 1 corresponds to phoneme A with a probability of 0.78, it can be understood that there is a 78% probability that the phoneme corresponding to phoneme frame 1 is phoneme A.

[0079] In step S13, if it is determined that the invalid phonemes and / or missing phonemes in the first phoneme sequence meet the preset conditions, the wake-up result of the voice information is determined according to the first phoneme sequence and the determination probability of multiple phonemes in the first phoneme sequence.

[0080] It is understandable that the presence of invalid and / or missing phonemes in a phoneme sequence will affect the speech recognition results. The more invalid and / or missing phonemes there are, the greater the impact. Therefore, before further decoding the first phoneme sequence, it is necessary to determine whether there are invalid and / or missing phonemes in the first phoneme sequence, and how many of them exist.

[0081] For example, certain rules can be pre-set based on whether invalid phonemes and / or missing phonemes are allowed, and the number of invalid phonemes and / or missing phonemes allowed, as preset conditions. Then, if the invalid phoneme and / or missing phoneme meets the preset conditions, step S13 is executed. In step S13, decoding of the first phoneme sequence can be achieved by comparing the determination probabilities of multiple phonemes in the first phoneme sequence with preset probability thresholds, retaining phonemes that reach the probability thresholds, removing phonemes that are below the probability thresholds, and then determining whether the sum of the determination probabilities of the remaining phonemes meets certain conditions. If it does, it can be determined that the voice information wake-up is successful, and the step of waking up the electronic device can be executed.

[0082] Figure 2 This is a flowchart illustrating a phoneme recognition method according to an exemplary embodiment, such as... Figure 2 As shown, step S12, which involves inputting the multiple phoneme frames into a preset acoustic model to obtain a first phoneme sequence, includes multiple phonemes corresponding to the multiple phoneme frames and the determination probability of each phoneme. This step may include the following steps:

[0083] Step S121: Input the multiple phoneme frames into the acoustic model frame by frame to obtain the multiple phoneme sequences output by the acoustic model.

[0084] In one implementation, features can be extracted from multiple phoneme frames of the speech information, and then the features extracted from each phoneme frame can be streamed into the acoustic model. This can, for example, include the following steps:

[0085] First, the feature vector of each phoneme frame in the multiple phoneme frames of the speech information is extracted. The method for extracting the feature vector is not limited in this disclosure.

[0086] Secondly, the feature vectors of multiple phoneme frames of the first phoneme sequence are input into the acoustic model in a frame-by-frame manner to obtain multiple phoneme sequences output by the acoustic model.

[0087] The acoustic model outputs multiple phoneme sequences that correspond one-to-one with multiple input phoneme frames. Each phoneme sequence contains at least one phoneme and the probability of determining the at least one phoneme.

[0088] Step S122: For each of the multiple phoneme frames, the phoneme with the highest probability in the phoneme sequence corresponding to the phoneme frame is determined as the phoneme corresponding to the phoneme frame.

[0089] Step S123: The multiple phonemes corresponding to the obtained multiple phoneme frames are used as the first phoneme sequence.

[0090] For example, assume that the wake word of electronic device A is "Xiao Ming Tong Xue", and its corresponding phoneme sequence is "x", "iao3", "m", "ing2", "t", "ong2", "x", "ve2", where the number in the phoneme represents the Pinyin tone. To wake up the electronic device A, the user needs to pronounce the wake word according to the phonemes. What the user utters is usually a piece of continuous voice information. As described above, the voice information is divided into a plurality of phoneme frames. In the process of inputting the phoneme frames into the acoustic model one by one, the acoustic model will output the phoneme sequence corresponding to each phoneme frame, where each phoneme sequence includes at least one phoneme, and the determination probability of the at least one phoneme. The above wake word is only exemplary, and can be set to any other wake word according to actual needs, or the wake word of the electronic device can be set by the user.

[0091] For example, the multiple phoneme frames included in the voice information are respectively phoneme frame 1, phoneme frame 2, and phoneme frame 3. After inputting phoneme frames 1 to 4 into the acoustic model frame by frame, assume that the phoneme sequence corresponding to phoneme frame 1 is: "x", "t", with determination probabilities of 0.85 and 0.15 respectively; the phoneme sequence corresponding to phoneme frame 2 is: "x", "iao3", "t", with determination probabilities of 0.1, 0.85, and 0.05 respectively; the phoneme sequence corresponding to phoneme frame 3 is: "m", "in2", "ing2", with determination probabilities of 0.07, 0.15, and 0.78 respectively. Then the phoneme with the highest determination probability is selected from the phoneme sequence corresponding to each phoneme frame, so it can be concluded that the phoneme corresponding to phoneme frame 1 is "x", the phoneme corresponding to phoneme frame 2 is "iao3", and the phoneme corresponding to phoneme frame 3 is "ing2". The obtained first phoneme sequence is thus: "x", "iao3", "ing2".

[0092] Further, Figure 3 is a flowchart illustrating another phoneme recognition method according to an exemplary embodiment, as Figure 3 shown in the figure, the step of determining, for each of the plurality of phoneme frames, the phoneme with the highest determination probability in the phoneme sequence corresponding to the phoneme frame as the phoneme corresponding to the phoneme frame as described in step S122 may include the following steps:

[0093] Step S1221: Perform normalization processing on the determination probability of each phoneme in the phoneme sequence corresponding to the phoneme frame, to obtain the normalized determination probability of each phoneme.

[0094] Step S1222: Determine the phoneme with the highest normalized determination probability in the phoneme sequence as the phoneme corresponding to the phoneme frame.

[0095] During the phoneme identification process using the aforementioned acoustic model, there may be unidentified phonemes and quiet segments. Unidentified phonemes can be labeled "unk," and quiet segments can be labeled "sil." Because of the presence of "unk" and "sil," the following situation may occur: a phoneme frame may contain "unk" and / or "sil" in its corresponding phoneme sequence, and its identification probability is higher than that of other phonemes in the sequence. Therefore, to avoid interference from "unk" and "sil," the identification probability of each phoneme in the phoneme sequence corresponding to the phoneme frame can be normalized first. During this normalization process, "unk" and / or "sil" can be ignored.

[0096] For example, suppose the phoneme sequence corresponding to a certain phoneme frame is "m", "in2", "ing2", "unk", with determination probabilities of 0.07, 0.15, 0.38, and 0.4 respectively. Before normalization, the determination probability of "unk" is 0.4, which is the maximum value in the phoneme sequence. If normalization is not performed, the phoneme corresponding to this phoneme frame will be determined as "unk".

[0097] The normalization method is as follows: divide the probability of phoneme i by the sum of the probabilities of all other phonemes in the phoneme sequence except "unk" and "sil". The quotient is the normalized probability of phoneme i, where phoneme i is any phoneme in the phoneme sequence except "unk" and "sil".

[0098] Based on the above method, it can be determined that:

[0099] The normalized probability of decision for “m” is 0.07 ÷ (0.07 + 0.15 + 0.38) = 0.12

[0100] The normalized probability of determining "in2" is 0.15 ÷ (0.07 + 0.15 + 0.38) = 0.25

[0101] The normalized probability of "ing2" is 0.38 ÷ (0.07 + 0.15 + 0.38) = 0.63

[0102] Therefore, after normalization and processing, "ing2" has the highest probability of being determined, and it can be determined that the phoneme corresponding to this phoneme frame is "ing2".

[0103] Optionally, the invalid phonemes mentioned in step S13 above may include isolated phonemes and additional phonemes. An isolated phoneme is any phoneme that is unrelated to its preceding and / or following adjacent phonemes. An additional phoneme is any phoneme that is unrelated to its preceding and / or following adjacent phonemes and is not isolated. A missing phoneme is any phoneme present in the second phoneme sequence that is related to its preceding and / or following adjacent phonemes and does not exist in the first phoneme sequence. For any phoneme, its preceding adjacent phoneme refers to the preceding phoneme in the sequence in which it is located, and its following adjacent phoneme refers to the following phoneme in the sequence in which it is located.

[0104] Before step S13 above, the method may further include: determining whether there are invalid phonemes and / or missing phonemes in the first phoneme sequence, which may include the following steps:

[0105] First, obtain the second phoneme sequence corresponding to the preset standard wake-up voice information.

[0106] Secondly, based on the second phoneme sequence, the isolated phonemes, extra phonemes, and / or missing phonemes in the first phoneme sequence are determined, that is, whether one or more of isolated phonemes, extra phonemes, and missing phonemes exist in the first phoneme sequence.

[0107] Furthermore, if it is determined that the isolated phoneme, the additional phoneme, and / or the missing phoneme exist in the first phoneme sequence, it is determined whether the isolated phoneme, the additional phoneme, and / or the missing phoneme satisfy the preset condition.

[0108] The preset conditions in step S13 above include: the number of isolated phonemes is less than a first threshold, the number of extra phonemes is less than a second threshold, and the number of missing phonemes is less than a third threshold. In other words, for any phoneme sequence, the number of isolated phonemes, extra phonemes, and missing phonemes in that sequence cannot exceed a certain number.

[0109] For example, taking the aforementioned "Xiao Ming classmate" as an example, the second phoneme sequence may be "x", "iao3", "m", "ing2", "t", "ong2", "x", "ve2". This sequence serves as the phoneme sequence corresponding to the standard wake-up voice information, and by referring to this sequence, it can be determined whether there are isolated phonemes, extra phonemes and missing phonemes in the first phoneme sequence. For example, if the first phoneme sequence is "x", "iao3", "t", "m", "ing2", "t", "ong2", "x", "ve2", then according to the second phoneme sequence, it can be known that the previous adjacent phoneme of the phoneme "t" is "ing2", the next adjacent phoneme is "ong2", and the phoneme related to the phoneme "t" is "ong2", which together form the pronunciation of the Chinese character "tong", whereby it can be determined that the third phoneme "t" in the first phoneme sequence belongs to an isolated phoneme.

[0110] If the first phoneme sequence is "x", "iao3", "t", "t", "m", "ing2", "t", "ong2", "x", "ve2", then according to the same judgment principle as described above, it can be known that the third phoneme and the fourth phoneme are two consecutive phonemes "t", which are not related to the previous adjacent phoneme "iao3" and the next adjacent phoneme "m", but do not belong to isolated phonemes, so it can be determined that the two consecutive phonemes "t" which are the third phoneme and the fourth phoneme in the first phoneme sequence belong to extra phonemes.

[0111] If the first phoneme sequence is "x", "iao3", "m", "ing2", "t", "ong2", "ve2", then according to the second phoneme sequence, "x" should exist between "ong2" and "ve2", so it can be determined that there is a missing phoneme "x" in the first phoneme sequence.

[0112] After determining the isolated phoneme, extra phoneme and / or missing phoneme existing in the first phoneme sequence, the isolated phoneme, extra phoneme and / or missing phoneme may be marked.

[0113] The above-mentioned first quantity threshold, second quantity threshold and third quantity threshold may be the same or different, for example, all may be set to 2, and may be set according to actual requirements, which is not limited in the present disclosure.

[0114] Optionally, the step S13 of determining the wake-up result of the voice information according to the first phoneme sequence and the determination probabilities of a plurality of phonemes in the first phoneme sequence in the case where it is determined that the invalid phoneme and / or missing phoneme in the first phoneme sequence meets a preset condition includes the following steps:

[0115] removing phonemes whose determination probability is less than the probability threshold from the plurality of phonemes in the first phoneme sequence to obtain a third phoneme sequence;

[0116] Obtain the sum of the determination probabilities of multiple phonemes in the third phoneme sequence;

[0117] If the sum of the probabilities is less than the total probability threshold, the voice message is deemed to have failed to wake up.

[0118] If the sum of these probabilities is greater than or equal to the total probability threshold, the voice message is determined to be successfully activated.

[0119] For example, suppose the speech information includes 10 phoneme frames, labeled as phoneme frames 1-10. After step 102, the first phoneme sequence obtained from phoneme frames 1-10 contains 10 phonemes: “x”, “iao3”, “m”, “m”, “ing2”, “t”, “ong2”, “x”, “ve2”, and “t”. The probability of each phoneme is 0.78, 0.85, 0.96, 0.52, 0.88, 0.72, 0.65, 0.91, 0.89, and 0.45, respectively. With the probability threshold set to 0.6, it can be determined that the probability of the fourth phoneme “m” and the tenth phoneme “t” is less than the probability threshold. Therefore, these two phonemes are removed, and the remaining phonemes are retained. Thus, the third phoneme sequence obtained is: “x”, “iao3”, “m”, “ing2”, “t”, “ong2”, “x”, and “ve2”, a total of 8 phonemes.

[0120] For example, the total probability threshold can be set to 5, and the sum of the determination probabilities of each phoneme in the third phoneme sequence = 0.78 + 0.85 + 0.96 + 0.88 + 0.72 + 0.65 + 0.91 + 0.89 = 6.64 > 5. Therefore, it can be determined that the voice information wake-up was successful, and then the operation of waking up the electronic device can be performed.

[0121] The probability thresholds and total probability thresholds mentioned above are just examples, and their values ​​can be set according to actual needs.

[0122] Additionally, it's worth mentioning that it's also possible to determine whether wake-up was successful by directly using the sum of the probabilities of all phonemes, without judging the probability of individual phonemes. Alternatively, one can skip judging whether the sum of the probabilities exceeds a total probability threshold and instead judge only the probability of individual phonemes. For example, a pass rate threshold for a phoneme sequence can be preset, such as 70%. A pass rate exceeding this threshold is considered a successful wake-up. Since 8 out of the 10 phonemes in the first phoneme sequence mentioned above exceed this probability threshold, its pass rate is 80%, and the speech information can be judged as successfully woken up.

[0123] Using the above method, there is no need to construct a complex decoding diagram. Therefore, the decoder constructed based on the rules shown in step S13 is simple to implement and has low requirements for both storage space and running content. This allows for accurate phoneme recognition while reducing hardware requirements.

[0124] In the above technical solution, multiple phoneme frames are obtained based on the received voice information, each phoneme frame having a preset duration. These multiple phoneme frames are then input into a preset acoustic model to obtain a first phoneme sequence. The first phoneme sequence includes multiple phonemes corresponding to the multiple phoneme frames, and the determination probability of each phoneme. Then, if invalid phonemes and / or missing phonemes in the first phoneme sequence meet preset conditions, the wake-up result of the voice information is determined based on the first phoneme sequence and the determination probabilities of the multiple phonemes in the first phoneme sequence. This implementation method enables phoneme recognition in voice information without setting up a complex decoder, avoiding the hardware requirements of a complex decoder and thus simplifying phoneme-level voice wake-up technology.

[0125] Figure 4 This is a block diagram illustrating a device wake-up device according to an exemplary embodiment. (Refer to...) Figure 4 The device 400 includes an acquisition module 410, an identification module 420, and a processing module 430.

[0126] The acquisition module 410 is configured to acquire multiple phoneme frames based on the received voice information, and the duration of each phoneme frame is a preset duration.

[0127] The recognition module 420 is configured to input the multiple phoneme frames into an acoustic model to obtain a first phoneme sequence, which includes multiple phonemes corresponding to the multiple phoneme frames and the determination probability of each phoneme.

[0128] The processing module 430 is configured to determine the wake-up result of the voice information based on the first phoneme sequence and the determination probability of multiple phonemes in the first phoneme sequence when it is determined that invalid phonemes and / or missing phonemes in the first phoneme sequence meet preset conditions.

[0129] Optionally, the identification module 420 includes: an identification submodule and a determination submodule;

[0130] The recognition submodule is configured to input the multiple phoneme frames of the first phoneme sequence into the acoustic model frame by frame to obtain multiple phoneme sequences output by the acoustic model. The multiple phoneme sequences correspond one-to-one with the multiple phoneme frames. Each phoneme sequence contains at least one phoneme and the determination probability of the at least one phoneme.

[0131] The determining submodule is configured to, for each of the multiple phoneme frames, determine the phoneme with the highest determination probability in the phoneme sequence corresponding to the phoneme frame as the phoneme corresponding to the phoneme frame; and use the multiple phonemes corresponding to the multiple phoneme frames as the first phoneme sequence.

[0132] Optionally, this determining submodule is configured as follows:

[0133] The determination probability of each phoneme in the phoneme sequence corresponding to the phoneme frame is normalized to obtain the normalized determination probability of each phoneme.

[0134] The phoneme with the highest normalized probability in the phoneme sequence is identified as the phoneme corresponding to the phoneme frame.

[0135] Optionally, the invalid phoneme includes isolated phonemes and additional phonemes, and the device wake-up device further includes: a preprocessing module configured to:

[0136] Obtain the second phoneme sequence corresponding to the preset standard wake-up voice information;

[0137] Based on the second phoneme sequence, the isolated phoneme, the extra phoneme, and / or the missing phoneme in the first phoneme sequence are determined; wherein, any phoneme that is unrelated to the preceding adjacent phoneme and / or the following adjacent phoneme is the isolated phoneme; any phoneme that is unrelated to the preceding adjacent phoneme and / or the following adjacent phoneme and is not an isolated phoneme is the extra phoneme; any phoneme that exists in the second phoneme sequence that is related to the preceding adjacent phoneme and / or the following adjacent phoneme and does not exist in the first phoneme sequence is the missing phoneme;

[0138] If it is determined that the isolated phoneme, the additional phoneme, and / or the missing phoneme exist in the first phoneme sequence, it is determined whether the isolated phoneme, the additional phoneme, and / or the missing phoneme satisfy the preset condition.

[0139] Optionally, the preset conditions include:

[0140] The number of isolated phonemes is less than a first quantity threshold, the number of additional phonemes is less than a second quantity threshold, and the number of missing phonemes is less than a third quantity threshold.

[0141] Optionally, the processing module is configured as follows:

[0142] In the first phoneme sequence, phonemes whose probability of determination is less than the probability threshold are removed to obtain the third phoneme sequence.

[0143] Obtain the sum of the determination probabilities of multiple phonemes in the third phoneme sequence;

[0144] If the sum of these probabilities is less than the total probability threshold, the voice message is deemed to have failed to wake up.

[0145] If the sum of the probabilities is greater than or equal to the total probability threshold, the voice message is determined to be successfully activated.

[0146] Optionally, the identification submodule is configured as follows:

[0147] Extract the feature vector of each phoneme frame from the multiple phoneme frames;

[0148] The feature vectors of multiple phoneme frames of the first phoneme sequence are input into the acoustic model frame by frame to obtain the multiple phoneme sequences output by the acoustic model.

[0149] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0150] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the device wake-up method provided in this disclosure.

[0151] Figure 5 This is a block diagram illustrating a device wake-up device 500 according to an exemplary embodiment. For example, device 500 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0152] Reference Figure 5 The device 500 may include one or more of the following components: a processing component 502, a memory 504, a power component 506, a multimedia component 508, an audio component 510, an input / output (I / O) interface 512, a sensor component 514, and a communication component 516.

[0153] Processing component 502 typically controls the overall operation of device 500, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 502 may include one or more processors 520 to execute instructions to complete all or part of the steps of the device wake-up method described above. Furthermore, processing component 502 may include one or more modules to facilitate interaction between processing component 502 and other components. For example, processing component 502 may include a multimedia module to facilitate interaction between multimedia component 508 and processing component 502.

[0154] Memory 504 is configured to store various types of data to support the operation of device 500. Examples of such data include instructions for any application or method operating on device 500, contact data, phonebook data, messages, pictures, videos, etc. Memory 504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0155] The power supply component 506 provides power to the various components of the device 500. The power supply component 506 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 500.

[0156] Multimedia component 508 includes a screen that provides an output interface between the device 500 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 508 includes a front-facing camera and / or a rear-facing camera. When the device 500 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0157] Audio component 510 is configured to output and / or input audio signals. For example, audio component 510 includes a microphone (MIC) configured to receive external audio signals when device 500 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 504 or transmitted via communication component 516. In some embodiments, audio component 510 also includes a speaker for outputting audio signals.

[0158] I / O interface 512 provides an interface between processing component 502 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0159] Sensor assembly 514 includes one or more sensors for providing status assessments of various aspects of device 500. For example, sensor assembly 514 may detect the on / off state of device 500, the relative positioning of components such as the display and keypad of device 500, changes in the position of device 500 or a component of device 500, the presence or absence of user contact with device 500, the orientation or acceleration / deceleration of device 500, and temperature changes of device 500. Sensor assembly 514 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 514 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 514 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0160] Communication component 516 is configured to facilitate wired or wireless communication between device 500 and other devices. Device 500 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 516 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 516 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0161] In an exemplary embodiment, the device 500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the device wake-up method described above.

[0162] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 504 including instructions, which can be executed by the processor 520 of the device 500 to complete the device wake-up method described above. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0163] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the device wake-up method described above when executed by the programmable device.

[0164] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0165] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A device wake-up method, characterized in that, include: Multiple phoneme frames are obtained based on the received voice information, and the duration of each phoneme frame is a preset duration; The multiple phoneme frames are input into a preset acoustic model to obtain a first phoneme sequence, which includes multiple phonemes corresponding to the multiple phoneme frames and the determination probability of each phoneme. If the invalid phonemes and / or missing phonemes in the first phoneme sequence meet the preset conditions, the wake-up result of the voice information is determined according to the first phoneme sequence and the determination probability of multiple phonemes in the first phoneme sequence. The step of determining the wake-up result of the voice information based on the first phoneme sequence and the determination probability of multiple phonemes in the first phoneme sequence when the invalid phonemes and / or missing phonemes in the first phoneme sequence meet the preset conditions includes: Remove the phonemes whose determination probability is less than the probability threshold from the multiple phonemes in the first phoneme sequence to obtain the third phoneme sequence; Obtain the sum of the determination probabilities of multiple phonemes in the third phoneme sequence; If the sum of the determined probabilities is less than the total probability threshold, the voice information wake-up is determined to have failed. If the sum of the determined probabilities is greater than or equal to the total probability threshold, it is determined that the voice information wake-up was successful; The invalid phonemes include isolated phonemes and extra phonemes, and the method further includes: Obtain the second phoneme sequence corresponding to the preset standard wake-up voice information; Based on the second phoneme sequence, determine whether the isolated phoneme, the additional phoneme, and / or the missing phoneme exist in the first phoneme sequence; wherein, any phoneme that is unrelated to the preceding adjacent phoneme and / or the following adjacent phoneme is the isolated phoneme; any phoneme that is unrelated to the preceding adjacent phoneme and / or the following adjacent phoneme and is not an isolated phoneme is the additional phoneme; any phoneme that exists in the second phoneme sequence that is related to the preceding adjacent phoneme and / or the following adjacent phoneme and does not exist in the first phoneme sequence is the missing phoneme; If it is determined that the isolated phoneme, the additional phoneme, and / or the missing phoneme exist in the first phoneme sequence, it is determined whether the isolated phoneme, the additional phoneme, and / or the missing phoneme satisfy the preset condition.

2. The method according to claim 1, characterized in that, The step of inputting the plurality of phoneme frames into a preset acoustic model to obtain a first phoneme sequence, the first phoneme sequence including a plurality of phonemes corresponding to the plurality of phoneme frames, and a determination probability for each of the phonemes, includes: The multiple phoneme frames are input into the acoustic model frame by frame to obtain multiple phoneme sequences output by the acoustic model. The multiple phoneme sequences correspond one-to-one with the multiple phoneme frames. Each phoneme sequence contains at least one phoneme and the determination probability of the at least one phoneme. For each of the plurality of phoneme frames, the phoneme with the highest determination probability in the phoneme sequence corresponding to the phoneme frame is determined as the phoneme corresponding to the phoneme frame. The multiple phonemes corresponding to the obtained multiple phoneme frames are used as the first phoneme sequence.

3. The method according to claim 2, characterized in that, For each of the plurality of phoneme frames, determining the phoneme with the highest probability in the phoneme sequence corresponding to the phoneme frame as the phoneme corresponding to the phoneme frame includes: The determination probability of each phoneme in the phoneme sequence corresponding to the phoneme frame is normalized to obtain the normalized determination probability of each phoneme. The phoneme with the highest normalized probability in the phoneme sequence is determined as the phoneme corresponding to the phoneme frame.

4. The method according to claim 1, characterized in that, The preset conditions include: The number of isolated phonemes is less than a first quantity threshold, the number of additional phonemes is less than a second quantity threshold, and the number of missing phonemes is less than a third quantity threshold.

5. The method according to claim 2, characterized in that, The step of inputting the multiple phoneme frames frame by frame into the acoustic model to obtain the multiple phoneme sequences output by the acoustic model includes: Extract the feature vector of each phoneme frame from the plurality of phoneme frames; The feature vectors of the multiple phoneme frames are input into the acoustic model frame by frame to obtain the multiple phoneme sequences output by the acoustic model.

6. A device wake-up device, characterized in that, include: The acquisition module is configured to acquire multiple phoneme frames based on the received voice information, with each phoneme frame having a preset duration. The recognition module is configured to input the plurality of phoneme frames into an acoustic model to obtain a first phoneme sequence, the first phoneme sequence including a plurality of phonemes corresponding to the plurality of phoneme frames, and a determination probability of each phoneme; The processing module is configured to determine the wake-up result of the voice information based on the first phoneme sequence and the determination probability of multiple phonemes in the first phoneme sequence when it is determined that invalid phonemes and / or missing phonemes in the first phoneme sequence meet preset conditions. The processing module is configured as follows: Remove the phonemes whose determination probability is less than the probability threshold from the multiple phonemes in the first phoneme sequence to obtain the third phoneme sequence; Obtain the sum of the determination probabilities of multiple phonemes in the third phoneme sequence; If the sum of the determined probabilities is less than the total probability threshold, the voice information wake-up is determined to have failed. If the sum of the determined probabilities is greater than or equal to the total probability threshold, it is determined that the voice information wake-up was successful; The invalid phonemes include isolated phonemes and extra phonemes, and the device further includes a preprocessing module configured to: Obtain the second phoneme sequence corresponding to the preset standard wake-up voice information; Based on the second phoneme sequence, determine whether the isolated phoneme, the additional phoneme, and / or the missing phoneme exist in the first phoneme sequence; wherein, any phoneme that is unrelated to the preceding adjacent phoneme and / or the following adjacent phoneme is the isolated phoneme; any phoneme that is unrelated to the preceding adjacent phoneme and / or the following adjacent phoneme and is not an isolated phoneme is the additional phoneme; any phoneme that exists in the second phoneme sequence that is related to the preceding adjacent phoneme and / or the following adjacent phoneme and does not exist in the first phoneme sequence is the missing phoneme; If it is determined that the isolated phoneme, the additional phoneme, and / or the missing phoneme exist in the first phoneme sequence, it is determined whether the isolated phoneme, the additional phoneme, and / or the missing phoneme satisfy the preset condition.

7. The apparatus according to claim 6, characterized in that, The identification module includes: an identification submodule and a determination submodule; The recognition submodule is configured to input the plurality of phoneme frames into the acoustic model frame by frame to obtain a plurality of phoneme sequences output by the acoustic model. The plurality of phoneme sequences correspond one-to-one with the plurality of phoneme frames. Each phoneme sequence contains at least one phoneme and the determination probability of the at least one phoneme. The determining submodule is configured to, for each of the plurality of phoneme frames, determine the phoneme with the highest determination probability in the phoneme sequence corresponding to the phoneme frame as the phoneme corresponding to the phoneme frame; and use the plurality of phonemes corresponding to the plurality of phoneme frames as the first phoneme sequence.

8. The apparatus according to claim 7, characterized in that, The determining submodule is configured as follows: The determination probability of each phoneme in the phoneme sequence corresponding to the phoneme frame is normalized to obtain the normalized determination probability of each phoneme. The phoneme with the highest normalized probability in the phoneme sequence is determined as the phoneme corresponding to the phoneme frame.

9. The apparatus according to claim 6, characterized in that, The preset conditions include: The number of isolated phonemes is less than a first quantity threshold, the number of additional phonemes is less than a second quantity threshold, and the number of missing phonemes is less than a third quantity threshold.

10. The apparatus according to claim 7, characterized in that, The identification submodule is configured as follows: Extract the feature vector of each phoneme frame from the plurality of phoneme frames; The feature vectors of the multiple phoneme frames are input into the acoustic model frame by frame to obtain the multiple phoneme sequences output by the acoustic model.

11. A device wake-up device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the steps of the method according to any one of claims 1-5 when executing the executable instructions.

12. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they implement the steps of the method according to any one of claims 1-5.

Images