A keyword detection method, system, device, medium and product
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- VERISILICON MICROELECTRONICS (NANJING) CO LTD
- Filing Date
- 2026-05-15
- Publication Date
- 2026-07-14
Smart Images
Figure CN122392496A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of speech recognition technology, and in particular to a keyword detection method, system, device, medium and product. Background Technology
[0002] In voice interaction systems, Keyword Spotting (KWS) or Wake-up Word recognition is a key entry point technology, aiming to accurately and reliably detect predefined keyword phrases in a continuous audio stream. Traditional deep learning-based KWS systems are typically trained on general speech datasets of a large number of anonymized users, with the optimization goal of achieving high recall rates across the widest possible range of pronunciations and scenarios.
[0003] In existing technologies, incorporating speaker identity information as an additional supervisory signal into the model to reduce false wake-ups caused by non-specific user speech is a common keyword detection method. However, its optimization dimension is limited to speaker identity; its core discrimination is "who spoke," rather than addressing the variability of pronunciation itself (such as accent, speech rate, and mispronunciation). Furthermore, the model's performance is highly dependent on training with a large amount of high-quality audio data from specific speakers. This significantly increases the data cost, computational cost, and engineering complexity of model training. Summary of the Invention
[0004] The purpose of this application is to provide a keyword detection method, system, device, medium, and product to improve the above-mentioned problems in the prior art.
[0005] For the purposes mentioned above, this application provides the following technical solution: The first aspect of this application provides a keyword detection method, including: Construct a keyword mapping relationship library, which includes the mapping relationship between each keyword and its pronunciation variant sequences, wherein each keyword corresponds to multiple pronunciation variant sequences; For the speech signal input in the current time frame, obtain the acoustic feature vector corresponding to the speech signal. Based on the acoustic feature vector and the historical token sequence, predict the current token and concatenate the current token with the historical token sequence to obtain the current token sequence. The number of candidate paths is determined based on the current token sequence. Based on the continuously input speech signals in each time frame, the current token sequence is expanded in parallel on each candidate path. During each expansion step, the expanded token sequence of each candidate path is matched with each pronunciation variant sequence in real time. If the match is successful, the keyword corresponding to the matched pronunciation variant sequence is output. If the match fails, the score of each candidate path is adjusted according to the matching result. The expanded token sequence corresponding to the candidate path with the highest score is selected for prediction of the speech signal input in the next time frame.
[0006] Furthermore, adjusting the scores of each candidate path based on the matching results includes: If the extended token sequence of the candidate path exactly matches the prefix of the first pronunciation variant sequence, a first reward score is added to the score of the candidate path.
[0007] Furthermore, if the extended token sequence of the candidate path matches the prefix portion of the first pronunciation variant sequence, a second reward score is added to the score of the candidate path, the second reward score being less than the first reward score.
[0008] Furthermore, the method also includes: After the parallel candidate path expansion of the current Token sequence is completed based on the voice signal input at each time frame, a second number of candidate paths and their corresponding candidate Token sequences are obtained. The expanded Token sequences corresponding to the first number of candidate paths with the highest scores are selected from the second number of candidate paths as the candidate Token sequence set. Each candidate token sequence in the candidate token sequence set is matched with each pronunciation variant sequence. If any subsequence of any candidate token sequence completely matches the second pronunciation variant sequence in the keyword mapping relationship library, or if the similarity between any subsequence of any candidate token sequence and the second pronunciation variant sequence exceeds a preset threshold, then it is determined that the match is successful, and the keyword corresponding to the second pronunciation variant sequence is output as the speech recognition result.
[0009] Furthermore, if any of the candidate token sequences simultaneously matches multiple pronunciation variant sequences in the keyword mapping database, then: The keyword corresponding to the pronunciation variant sequence with the highest weight among the multiple pronunciation variant sequences is output as the recognition result of the speech signal.
[0010] Furthermore, the method for setting the weights of the pronunciation variant sequences includes: The keyword is converted into a third pronunciation variant sequence according to a preset phoneme conversion rule. A mapping relationship between the third pronunciation variant sequence and the keyword is constructed. The third pronunciation variant sequence has a first weight. Obtain the fourth pronunciation variant sequence that appears more frequently than a preset value in the historical recognition results of the keyword, construct the mapping relationship between the fourth pronunciation variant sequence and the keyword, and the fourth pronunciation variant sequence has a second weight, which is greater than the first weight.
[0011] Furthermore, low-confidence identification data and rejection identification data of the keywords are collected as the historical identification results.
[0012] Furthermore, the method for setting the weight of the pronunciation variant sequence also includes: obtaining a fifth pronunciation variant sequence based on user registration, constructing a mapping relationship between the fifth pronunciation variant sequence and the keyword, and determining the weight of the fifth pronunciation variant sequence based on user settings.
[0013] A second aspect of this application provides a keyword detection system, the system being used to implement the keyword detection method described in the first aspect of this application, the system comprising: A pronunciation variant registration module is used to construct a keyword mapping relationship library, which includes the mapping relationship between each keyword and its pronunciation variant sequences, and each keyword corresponds to multiple pronunciation variant sequences; The speech recognition module is used to obtain the acoustic feature vector corresponding to the speech signal input in the current time frame, predict the current token based on the acoustic feature vector and the historical token sequence, and concatenate the current token with the historical token sequence to obtain the current token sequence. The dynamic matching module is used to determine the number of candidate paths based on the current token sequence. Based on the continuously input speech signals in each time frame, the current token sequence is expanded in parallel along each candidate path. In each expansion step, the expanded token sequence of each candidate path is matched with each pronunciation variant sequence in real time. If the match is successful, the keyword corresponding to the matched pronunciation variant sequence is output. If the match fails, the score of each candidate path is adjusted according to the matching result. The expanded token sequence corresponding to the candidate path with the highest score is selected for prediction of the speech signal input in the next time frame.
[0014] Furthermore, the dynamic matching module is also used for: If the extended token sequence of the candidate path exactly matches the prefix of the first pronunciation variant sequence, then a first reward score is added to the score of the candidate path; If the extended token sequence of the candidate path matches the prefix portion of the first pronunciation variant sequence, a second reward score is added to the score of the candidate path, the second reward score being less than the first reward score.
[0015] A third aspect of this application provides an electronic device, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is used for the instructions to implement a keyword detection method as described in the first aspect of this application.
[0016] The fourth aspect of this application provides a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the keyword detection method described in the first aspect of this application.
[0017] The fifth aspect of this application provides a computer program product comprising a computer program that, when executed by a processor, implements the steps of a keyword detection method as described in the first aspect of this application.
[0018] The keyword detection method described above, provided in this application, can achieve at least the following technical effects: This application deeply embeds the registration mechanism into the real-time decoding process of automatic speech recognition by pre-registering a keyword mapping relationship library. During the speech recognition process, it combines the pronunciation variants of each pre-registered keyword for dynamic matching, so that each keyword no longer relies on a single standard pronunciation token sequence for recognition, but can match multiple pre-registered pronunciation variant token sequences, thereby significantly improving the adaptability to pronunciation diversity and increasing the recall rate. Attached Figure Description
[0019] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0020] Figure 1 This is a schematic flowchart of a keyword detection method provided in an embodiment of this application; Figure 2 This is a schematic diagram of a keyword detection system provided in an embodiment of this application; Figure 3 A schematic diagram of a computer device provided for an embodiment of this application; Reference numerals: 200, a keyword detection system; 201, a pronunciation variant registration module; 202, a speech recognition module; 203, a dynamic matching module; 301, a memory; 302, a processor. Detailed Implementation
[0021] The technical solutions of this application will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0022] This application deeply embeds the registration mechanism into the real-time decoding process of automatic speech recognition by pre-registering token sequences of multiple pronunciation variations, thereby expanding the recognition range of keywords and improving the system's adaptability to pronunciation diversity. Figure 1 As shown, the keyword detection method provided in this embodiment specifically includes the following steps: Step S100: Construct a keyword mapping relationship library, which includes the mapping relationship between each keyword and its pronunciation variant sequences, and each keyword corresponds to multiple pronunciation variant sequences; Specifically, in this embodiment, a registration mechanism is first employed to pre-register a mapping relationship library between a keyword and multiple possible pronunciation variant sequences. Logically, the keyword mapping relationship library is a set of key-value pairs, where the Key is the target keyword (e.g., "pause music"), and the Value is all the legal pronunciation variant sequences corresponding to that keyword. Each pronunciation variant sequence corresponds to a Token sequence. Each pronunciation variant sequence is generated based on preset phoneme conversion rules, historical data mining, or user-customized generation. Each pronunciation variant sequence corresponds to an initial weight, which is set according to the different methods of obtaining the pronunciation variant sequence.
[0023] Step S200: For the speech signal input in the current time frame, obtain the acoustic feature vector corresponding to the speech signal, predict the current token based on the acoustic feature vector and the historical token sequence, and concatenate the current token with the historical token sequence to obtain the current token sequence; Specifically, in this embodiment, the current audio input is mapped to corresponding text for output based on an ASR (Automatic Speech Recognition) model. An ASR model generally includes an encoder and a decoder. The encoder receives the raw audio spectrum or waveform, processes it through a multi-layer neural network, and transforms it into a series of higher-level, more compact acoustic feature representations. The decoder generates the final text sequence. In the ASR model, a token refers to the smallest recognition unit corresponding to the encoded speech signal, such as a phoneme, character, or word unit. The speech signal input in the current time frame, after preprocessing (including framing, windowing, and Fbank feature extraction), is fed into the encoder, which outputs a series of high-level, context-dependent acoustic feature vectors. The decoder, based on the acoustic feature vectors output by the encoder and the token sequences generated by its own historical predictions, predicts the current input speech signal and concatenates the predicted current token sequence with the historical token sequences to obtain the current token sequence. The current token sequence contains each candidate path and its corresponding extended token sequence, where the initial number of candidate paths is determined based on the historical token sequences.
[0024] Optionally, this embodiment uses Zipformer as the encoder, which has efficient timing modeling capabilities.
[0025] Step S300: Determine the number of candidate paths based on the current Token sequence. Based on the continuously input speech signals in each time frame, expand the current Token sequence in parallel on each candidate path. During each expansion step, match the expanded Token sequence of each candidate path with each pronunciation variant sequence in real time. If the match is successful, output the keyword corresponding to the matched pronunciation variant sequence. If the match fails, adjust the score of each candidate path according to the matching result, and select the expanded Token sequence corresponding to the candidate path with the highest score to predict the speech signal input in the next time frame. Specifically, in this preferred embodiment, the decoder is based on the Beam Search algorithm to find the token sequence that best matches the currently input speech signal. The Beam Search algorithm is used to maintain a beam of size k during the decoding process, i.e., k candidate paths with the highest scores, where k is a preset beam width, i.e., the first quantity mentioned above. For the speech signal input in the current time frame, the decoder combines the historical token sequence including k historical candidate paths retained in the previous frame, and predicts the k most likely candidate tokens at the current moment for each historical candidate path, thereby obtaining an expanded k×k candidate path. The expanded token sequence corresponding to each candidate path consists of the corresponding historical candidate path plus one currently predicted candidate token.
[0026] Further, based on the extended token sequence generated for each candidate path, a probability score for the next predicted token is generated. Then, the k tokens with the highest scores are extended into the current token sequence. Unlike traditional decoding, this embodiment performs a rapid match between the extended token sequence corresponding to each candidate path and all pronunciation variant sequences in a pre-built keyword mapping database during each step of the Beam path extension. If the match is successful, the keyword corresponding to the matched pronunciation variant sequence is output; if the match fails, the scores of each candidate path are adjusted according to the matching results, and the extended token sequences corresponding to the k candidate paths with the highest overall scores are selected and retained for predicting the speech signal input in the next time frame. This process is repeated. For the initial frame, the number of candidate paths in the historical token sequence is 1. After extension, k candidate paths are obtained and retained, and each subsequent frame is extended in the same way as the k historical paths.
[0027] Furthermore, in steps S100~S300, upon receiving the voice signal input in the next time frame, the following steps are also included: Obtain the acoustic feature vector corresponding to the input speech signal in the next time frame; based on the acoustic feature vector and the k candidate paths retained in the previous time frame, predict the k most likely candidate tokens at the current time for each candidate path; append k candidate tokens to each candidate path to generate k new extended token sequences for each historical path, resulting in a total of k×k new extended token sequences. The above k×k new extended token sequences are matched with each of the pronunciation variant sequences: if an extended token sequence completely matches a pronunciation variant sequence, the keyword corresponding to the pronunciation variant sequence is output; if the match fails, the score of the extended token sequence corresponding to each candidate path is updated according to the matching result; from the k×k extended token sequences, the k extended token sequences with the highest scores are retained as candidate paths for the current time frame and used for the extension of the next time frame.
[0028] This embodiment continuously recognizes continuously input speech signals by iteratively executing the above steps. Flexible and efficient keyword detection is achieved by matching multiple registered variants in real time during the Beam Search decoding process.
[0029] Further, in step S300, adjusting the scores of each candidate path according to the matching results includes: if the extended token sequence of the candidate path completely matches the prefix of the first pronunciation variant sequence, then a first reward score is added to the score of the candidate path; if the extended token sequence of the candidate path partially matches the prefix of the first pronunciation variant sequence, then a second reward score is added to the score of the candidate path, wherein the second reward score is less than the first reward score.
[0030] Specifically, the matching of the extended token sequence of each candidate path with each pronunciation variant sequence includes two strategies: exact matching and fuzzy matching. This embodiment guides the decoding process by dynamically matching with a registered keyword mapping database, allowing paths that might be pruned due to low probability in traditional decoding but match the registered variants to be retained, thereby significantly increasing the probability of the target keyword being successfully decoded.
[0031] Furthermore, the two strategies, exact matching and fuzzy matching, include: Exact matching occurs when the extended token sequence of a candidate path in the Beam exactly matches the prefix of a first pronunciation variant sequence. The candidate path is preserved and given a first reward score (Boost Score) to encourage the decoder to continue searching in the direction of the keyword corresponding to that pronunciation variant sequence.
[0032] To handle more complex pronunciation distortions, this embodiment introduces fuzzy matching based on EditDistance or CTC-like algorithms. Allowing a small number (optionally 1-2) of tokens to be inserted, deleted, or replaced between the extended token sequence and the prefix of a first pronunciation variant sequence is considered a successful match; that is, the extended token sequence partially matches the prefix of the aforementioned first pronunciation variant sequence. This candidate path is retained and assigned a second reward score. The second reward score assigned based on fuzzy matching is lower than the first reward score for exact matching.
[0033] Optionally, the matching methods in this embodiment also include attention weighted matching and graph matching.
[0034] Furthermore, steps S100~S300 also include: Step S400: After the parallel candidate path expansion of the current Token sequence is completed based on the voice signal input in each time frame, a second number of candidate paths and their corresponding extended Token sequences are obtained. The extended Token sequences corresponding to the first number of candidate paths with the highest scores are selected from the second number of candidate paths as a set of candidate Token sequences. Step S500: Match each candidate token sequence in the candidate token sequence set with each pronunciation variant sequence, including: if any sub-sequence of any candidate token sequence completely matches the second pronunciation variant sequence in the keyword mapping relationship library, or if the similarity between any sub-sequence of any candidate token sequence and the second pronunciation variant sequence exceeds a preset threshold, then it is determined that the match is successful, and the keyword corresponding to the second pronunciation variant sequence is output as the speech recognition result; if the candidate token sequence completely matches multiple pronunciation variant sequences in the keyword mapping relationship library at the same time, then the keyword corresponding to the pronunciation variant sequence with the highest weight among the multiple matched pronunciation variant sequences is output as the recognition result of the speech signal.
[0035] Specifically, once Beam Search of all input speech signals is complete, or once a candidate path has been expanded to generate a complete set of candidate token sequences, the system compares all k final extended token sequences with all pronunciation variant sequences in the keyword mapping database. Based on the keyword mapping database, further result identification and decision-making are performed on each candidate token sequence. If a match is successful (including exact or fuzzy matching), the corresponding keyword is output. By setting multiple matching rules, the recall rate of keyword detection can be improved while ensuring the accuracy of the recognition results.
[0036] Further, in step S500, if any subsequence of a candidate token sequence completely matches any second pronunciation variant sequence corresponding to a keyword in the keyword mapping relation library, then the keyword corresponding to the second pronunciation variant sequence is determined to be a correct recognition result, and the keyword is output as the final speech recognition result; or, a preset similarity threshold is used. If the similarity between any candidate token sequence and any second pronunciation variant sequence exceeds the preset threshold, then the keyword corresponding to the second pronunciation variant sequence is also determined to be a correct recognition result, and the keyword is output as the final speech recognition result. In step S500, if any candidate token sequence completely matches multiple pronunciation variant sequences in the keyword mapping relation library, then the keyword corresponding to the pronunciation variant sequence with the highest weight among the matched multiple pronunciation variant sequences is output as the final speech recognition result; the final speech recognition result is the decoding result of the decoder.
[0037] In most cases, a decoded candidate token sequence will explicitly match a pronunciation variant sequence of a single keyword; a decoded candidate token sequence that happens to be a pronunciation variant sequence of two or more different keywords in the keyword mapping database is an edge case.
[0038] For example, suppose the system registers the following two keywords and their pronunciation variant sequences: Keyword A: "PlayMusic", variant 1: ["play", "music"], Keyword B: "Plain Music", variant 1: ["play", "music"]. "Plain" is registered as a variant because it sounds extremely similar to "Play" in some accents. When a user utters an ambiguous pronunciation, the ASR decoder outputs the candidate token sequence ["play", "music"]. This candidate token sequence matches a variant of both keyword A and keyword B.
[0039] Further, in step S500, the scores of each candidate token sequence output by the encoder, the weights of the matched pronunciation variant sequences themselves, and the deductions made during the result recognition and decision-making process of the candidate token sequences are combined to obtain a comprehensive score for each candidate token sequence. This comprehensive score is compared with an adjustable threshold; only recognition results exceeding this threshold are ultimately output and trigger subsequent actions. This mechanism ensures that while improving recall, the false acceptance rate is kept within an acceptable range.
[0040] Further, in step S100, each pronunciation variant sequence corresponds to an initial weight, which is set according to the different methods of obtaining the pronunciation variant sequence. The method for generating the pronunciation variant sequence includes: Phonetic rule generation: The keywords are phoneme-converted according to preset phoneme conversion rules to obtain a third pronunciation variant sequence. A mapping relationship is constructed between the third pronunciation variant sequence and the keywords. The third pronunciation variant sequence has a first weight. For example: for the vowel / ɔ... (e.g., "au" in "pause"), the rule allows it to be converted to / ʊ / (as in "o" in "post") or / æ / (as in "a" in "pass"). Based on preset factor transformation rules, it can automatically generate candidate variant sequences including ["post", "music"], ["pass", "music"], etc. for the keyword "pause music".
[0041] Historical data mining: Low-confidence recognition data and rejected recognition data from the historical recognition results of the keyword are obtained, and a fourth pronunciation variant sequence with a frequency greater than a preset value is acquired. A mapping relationship is constructed between the fourth pronunciation variant sequence and the keyword. The fourth pronunciation variant sequence has a second weight, which is greater than the first weight. For example, if the system detects multiple instances of misidentifying "pause music" as "paws music," it can automatically propose or directly add ["paws", "music"] as a registered variant of "pause music."
[0042] User-initiated customization: A fifth pronunciation variant sequence is obtained based on user registration. A mapping relationship is constructed between the fifth pronunciation variant sequence and the keyword. The weight of the fifth pronunciation variant sequence is determined based on user settings. In scenarios requiring extremely high accuracy (such as industrial instructions), administrators can explicitly register common misreadings or dialectal variants for key instructions via text or voice, achieving fine-grained control.
[0043] Furthermore, the pronunciation variant sequence can also be generated using speech synthesis + ASR reverse generation; or automatically clustered based on the user's historical speech data.
[0044] For example, this embodiment will illustrate the above specific implementation method in conjunction with the following intelligent vehicle system scenario, including the following steps: S1. Keyword mapping relation database construction: For the instruction "turn on the air conditioner", in addition to the standard sequence ["open", "air conditioner"], the system automatically registers common pronunciation variations ["open", "air conditioner"] (accents), etc.
[0045] S2. Speech Recognition: The user says "Turn on the air conditioner." Traditional ASR might output "turn on" or other meaningless words, causing the instruction to fail. In this embodiment, after generating "turn on," the decoder finds through matching that it is a prefix of a variant of "turn on the air conditioner," thus guiding the subsequent generation of "turn on the air conditioner." Ultimately, it successfully matches the variant ["turn on", "air conditioner"], thereby correctly executing the "turn on the air conditioner" instruction.
[0046] S3. Optimization: The system records this successful match and can appropriately increase the weight of this variant in the future, making it easier to trigger. More specifically, an advanced feature of this embodiment is that it not only passively uses pre-registered pronunciation variants but also continuously optimizes itself based on actual usage data after deployment. Its core is the introduction of a dynamic weight self-learning mechanism. This mechanism can automatically evaluate the effectiveness of each pronunciation variant based on the user's actual interaction and adjust its influence in the matching process, thereby achieving a "gets smarter with use" effect.
[0047] First, the system continuously monitors and records two types of key data in the background: 1. Successful match records: Whenever a pronunciation variant is successfully matched and ultimately triggers the correct keyword function (e.g., successfully triggering "pause music" through variants ["post", "music"]), the system records this event. 2. Implicit user confirmation: After successful recognition, if the user does not cancel the operation within a reasonable time window (e.g., 3-5 seconds), or the operation is successfully executed (e.g., the music does pause), it is considered a valid positive feedback. Then, the system maintains a dynamic weight W for each pronunciation variant (the initial value can be set by the administrator or determined based on the source). The weight is updated according to the following rules: When a variant successfully matches and receives positive feedback, its weight is positively reinforced; if the system recognizes a keyword and executes an action, but the user immediately cancels the action (e.g., the system executes "pause music" after the user says it, but the user says "continue playing" within 1 second), this may mean that the recognition was a false trigger. The system can trace back to the variant that caused the false trigger and penalize its weight. The adjusted weights will directly affect the subsequent recognition process: In Beam Search, variants with higher weights will have higher boost scores, making the decoder more inclined to generate these validated sequences during the search.
[0048] Based on the same inventive concept, this invention also provides a keyword detection system 200 in specific embodiments, as described in the following examples. Since the principle by which a keyword detection system 200 solves the problem is similar to that of a keyword detection method, the implementation of a keyword detection system 200 can refer to the implementation of a keyword detection method, and repeated details will not be elaborated further. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.
[0049] Figure 2 This is a structural block diagram of a keyword detection system 200 according to an embodiment of this application, such as... Figure 2 As shown, it includes: The pronunciation variant registration module 201 is used to construct a keyword mapping relationship library, which includes the mapping relationship between each keyword and its pronunciation variant sequence, and each keyword corresponds to multiple pronunciation variant sequences; The speech recognition module 202 is used to obtain the acoustic feature vector corresponding to the speech signal input in the current time frame, predict the current token based on the acoustic feature vector and the historical token sequence, and concatenate the current token with the historical token sequence to obtain the current token sequence. The dynamic matching module 203 is used to determine the number of candidate paths based on the current token sequence, and expand the current token sequence in parallel on each candidate path based on the continuously input speech signals in each time frame. In each step of the expansion process, the expanded token sequence of each candidate path is matched with each pronunciation variant sequence in real time: if the match is successful, the keyword corresponding to the matched pronunciation variant sequence is output; if the match fails, the score of each candidate path is adjusted according to the matching result, and the expanded token sequence corresponding to the candidate path with the highest score is selected for prediction of the speech signal input in the next time frame.
[0050] Furthermore, the dynamic matching module 203 is also used for adjusting the score of the candidate path according to the matching result, including: If the extended token sequence of the candidate path exactly matches the prefix of the first pronunciation variant sequence, a first reward score is added to the score of the candidate path.
[0051] Furthermore, the dynamic matching module 203 is also configured to: if the extended token sequence of the candidate path matches the prefix portion of the first pronunciation variant sequence, then add a second reward score to the score of the candidate path, wherein the second reward score is less than the first reward score.
[0052] Furthermore, the dynamic matching module 203 is also used for: After parallel candidate path expansion of the current Token sequence based on the voice signal input at each time frame, a second number of candidate paths and their corresponding extended Token sequences are obtained. The extended Token sequences corresponding to the first number of candidate paths with the highest scores are selected from the second number of candidate paths as a set of candidate Token sequences. Each candidate token sequence in the candidate token sequence set is matched with each pronunciation variant sequence. If any subsequence of any candidate token sequence completely matches the second pronunciation variant sequence in the keyword mapping relationship library, or if the similarity between any subsequence of any candidate token sequence and the second pronunciation variant sequence exceeds a preset threshold, then it is determined that the match is successful, and the keyword corresponding to the second pronunciation variant sequence is output as the speech recognition result.
[0053] Furthermore, the dynamic matching module 203 is also configured to: if any of the candidate token sequences simultaneously and completely match multiple pronunciation variant sequences in the keyword mapping relation library, then: The keyword corresponding to the pronunciation variant sequence with the highest weight among the multiple pronunciation variant sequences is output as the recognition result of the speech signal.
[0054] Furthermore, the pronunciation variant registration module 201 is also used for: the method for setting the weights of the pronunciation variant sequences includes: The keyword is converted into a third pronunciation variant sequence according to a preset phoneme conversion rule. A mapping relationship between the third pronunciation variant sequence and the keyword is constructed. The third pronunciation variant sequence has a first weight. Obtain the fourth pronunciation variant sequence that appears more frequently than a preset value in the historical recognition results of the keyword, construct the mapping relationship between the fourth pronunciation variant sequence and the keyword, and the fourth pronunciation variant sequence has a second weight, which is greater than the first weight.
[0055] Furthermore, the pronunciation variant registration module 201 is also used to: collect low-confidence recognition data and rejection recognition data of the keyword as the historical recognition results.
[0056] Furthermore, the pronunciation variant registration module 201 is also used for: the method for setting the weight of the pronunciation variant sequence further includes: obtaining a fifth pronunciation variant sequence based on user registration, constructing a mapping relationship between the fifth pronunciation variant sequence and the keyword, and determining the weight of the fifth pronunciation variant sequence based on user settings.
[0057] In this embodiment, a computer device is also provided, such as... Figure 3 As shown, it includes a memory 301, a processor 302, and a computer program stored in the memory 301 and executable on the processor 302. When the processor 302 executes the computer program, it implements any of the above-mentioned keyword detection methods.
[0058] Specifically, the computer device can be a computer terminal, a server, or a similar computing device.
[0059] In this embodiment, a storage medium is provided, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the keyword detection method described above in this embodiment.
[0060] In this embodiment, a computer program product is provided, which includes a computer program. When the computer program is executed by a processor, it implements the steps of the keyword detection method described above in this embodiment.
[0061] In this embodiment, a computer-readable storage medium is provided, which stores a computer program that executes any of the keyword detection methods described above.
[0062] Specifically, computer-readable storage media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer-readable storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, and optical disc read-only memory (CD-ROM). ROM, digital multifunction optical disc (DVD) or other optical storage, magnetic cassette tape, magnetic magnetic disk storage or other magnetic storage devices or any other non-transfer medium, may be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include transient media such as modulated data signals and carrier waves.
[0063] The embodiments of the present invention achieve the following technical effects: 1. This application deeply embeds the registration mechanism into the real-time decoding process of automatic speech recognition by pre-registering a keyword mapping relationship library. During the speech recognition process, it combines the pronunciation variants of each pre-registered keyword for dynamic matching, so that each keyword no longer relies on a single standard pronunciation token sequence for recognition, but can match multiple pre-registered pronunciation variant token sequences, thereby significantly improving the adaptability to pronunciation diversity and improving the recall rate. 2. The keyword mapping relation library provided in this application supports the dynamic registration of new pronunciation variant sequences, thereby adapting to new users, new accents, and new contexts; 3. This mechanism can be embedded into existing E2E ASR (End-to-End Automatic Speech Recognition) systems without retraining the model; only the matching vocabulary needs to be expanded. 4. This application can control the false positive rate while improving the recall rate through a dynamic score adjustment mechanism and variant screening strategy; 5. This application has multiple application scenarios. It can be used not only for keyword detection, but also for voice command understanding and query expansion in voice search. It is suitable for keyword recognition in multilingual and mixed language scenarios, and can also be used in real-time voice interaction scenarios such as smart homes, in-vehicle systems, and voice assistants.
[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.
Claims
1. A keyword detection method, characterized in that, include: Construct a keyword mapping relationship library, which includes the mapping relationship between each keyword and its pronunciation variant sequences, wherein each keyword corresponds to multiple pronunciation variant sequences; For the speech signal input in the current time frame, obtain the acoustic feature vector corresponding to the speech signal. Based on the acoustic feature vector and the historical token sequence, predict the current token and concatenate the current token with the historical token sequence to obtain the current token sequence. The number of candidate paths is determined based on the current token sequence. Based on the continuously input speech signals in each time frame, the current token sequence is expanded in parallel on each candidate path. During each expansion step, the expanded token sequence of each candidate path is matched with each pronunciation variant sequence in real time. If the match is successful, the keyword corresponding to the matched pronunciation variant sequence is output. If the match fails, the score of each candidate path is adjusted according to the matching result. The expanded token sequence corresponding to the candidate path with the highest score is selected for prediction of the speech signal input in the next time frame.
2. The keyword detection method according to claim 1, characterized in that, The step of adjusting the scores of each candidate path based on the matching results includes: If the extended token sequence of the candidate path exactly matches the prefix of the first pronunciation variant sequence, a first reward score is added to the score of the candidate path.
3. The keyword detection method according to claim 2, characterized in that, If the extended token sequence of the candidate path matches the prefix portion of the first pronunciation variant sequence, a second reward score is added to the score of the candidate path, the second reward score being less than the first reward score.
4. The keyword detection method according to claim 1, characterized in that, Also includes: After the parallel candidate path expansion of the current Token sequence is completed based on the voice signal input at each time frame, a second number of candidate paths and their corresponding extended Token sequences are obtained. The extended Token sequences corresponding to the first number of candidate paths with the highest scores are selected from the second number of candidate paths as the candidate Token sequence set. Each candidate token sequence in the candidate token sequence set is matched with each of the pronunciation variant sequences. If any subsequence of any candidate token sequence completely matches the second pronunciation variant sequence in the keyword mapping relationship library, or if the similarity between any subsequence of any candidate token sequence and the second pronunciation variant sequence exceeds a preset threshold, then it is determined that the match is successful, and the keyword corresponding to the second pronunciation variant sequence is output as the speech recognition result.
5. The keyword detection method according to claim 4, characterized in that, If any of the candidate token sequences matches exactly with multiple pronunciation variant sequences in the keyword mapping database, then: The keyword corresponding to the pronunciation variant sequence with the highest weight among the multiple pronunciation variant sequences is output as the recognition result of the speech signal.
6. The keyword detection method according to claim 5, characterized in that, The method for setting the weights of the pronunciation variant sequences includes: The keyword is converted into a third pronunciation variant sequence according to a preset phoneme conversion rule. A mapping relationship between the third pronunciation variant sequence and the keyword is constructed. The third pronunciation variant sequence has a first weight. Obtain the fourth pronunciation variant sequence that appears more frequently than a preset value in the historical recognition results of the keyword, construct the mapping relationship between the fourth pronunciation variant sequence and the keyword, and the fourth pronunciation variant sequence has a second weight, which is greater than the first weight.
7. The keyword detection method according to claim 6, characterized in that, The low-confidence identification data and rejection identification data of the keywords are collected as the historical identification results.
8. The keyword detection method according to claim 6, characterized in that, The method for setting the weight of the pronunciation variant sequence further includes: obtaining a fifth pronunciation variant sequence based on user registration, constructing a mapping relationship between the fifth pronunciation variant sequence and the keyword, and determining the weight of the fifth pronunciation variant sequence based on user settings.
9. A keyword detection system, characterized in that, include: A pronunciation variant registration module is used to construct a keyword mapping relationship library, which includes the mapping relationship between each keyword and its pronunciation variant sequences, and each keyword corresponds to multiple pronunciation variant sequences; The speech recognition module is used to obtain the acoustic feature vector corresponding to the speech signal input in the current time frame, predict the current token based on the acoustic feature vector and the historical token sequence, and concatenate the current token with the historical token sequence to obtain the current token sequence. The dynamic matching module is used to determine the number of candidate paths based on the current token sequence. Based on the continuously input speech signals in each time frame, the current token sequence is expanded in parallel on each candidate path. In each step of the expansion process, the expanded token sequence of each candidate path is matched with each pronunciation variant sequence in real time: if the match is successful, the keyword corresponding to the matched pronunciation variant sequence is output; if the match fails, the score of each candidate path is adjusted according to the matching result, and the expanded token sequence corresponding to the candidate path with the highest score is selected for prediction of the speech signal input in the next time frame.
10. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is used for the instructions to implement a keyword detection method as described in any one of claims 1 to 8.
11. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of a keyword detection method as described in any one of claims 1 to 8.
12. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of a keyword detection method as described in any one of claims 1 to 8.