Speech recognition method and device, equipment and medium

Abstract

The invention discloses a speech recognition method and device, equipment and a medium, and belongs to the technical field of speech recognition. The method comprises the following steps: acquiring to-be-recognized voice features; recognizing the voice features based on a preset command word recognition network to obtain a first recognition result, and judging whether command words exist in the voice features or not; if the command word exists, judging whether the command word is an error-prone word or not; if the command word is an error-prone word, recognizing the voice feature based on an error-prone word recognition network to obtain a second recognition result; judging whether the first recognition result is consistent with the second recognition result or not; and if yes, executing an action corresponding to the voice feature. According to the method, through a dual-network cascade verification mechanism, the misrecognition rate of words such as short command words which are easily misrecognized is reduced, meanwhile, the response efficiency is improved through rapid processing of non-error-prone words, and the contradiction between the voice recognition accuracy and resource consumption in embedded equipment is solved.

Description

Technical Field

[0001] This application relates to the field of speech recognition technology, and in particular to a speech recognition method, apparatus, device, and medium. Background Technology

[0002] In smart homes and smart terminals, voice recognition technology has become one of the core methods of human-computer interaction. The accurate recognition of wake words and command words directly impacts the user experience. Currently, command word recognition is widely used in the voice controls of smart devices, where users trigger corresponding actions by uttering specific command words (such as "turn on the lights" or "I'm home"). However, these command words are typically short (only two or three characters) and colloquial, making them prone to misrecognition in practical applications. For example, the device may respond incorrectly without a user's instruction, affecting the reliability of the interaction.

[0003] Currently, most speech recognition algorithms for embedded devices employ the CTC (Connectionist Temporal Classification) decoding algorithm. Its advantages include no need for sequence alignment, fast inference speed, and low memory consumption, making it suitable for deployment on low-resource devices. However, the core objective of the CTC algorithm is to maximize the probability of the correct path, which presents a significant drawback when processing short command words: due to the limited feature representation space of short command words, the algorithm struggles to effectively distinguish similar speech signals, leading to a higher false recognition rate. For example, when speech fragments similar to command words appear in ambient noise or background sound, a recognition system using the CTC algorithm may misinterpret them as valid instructions, potentially causing malfunctions in the device.

[0004] Furthermore, existing single-network model recognition schemes struggle to balance feature extraction capabilities with resource consumption: using large-scale models to improve recognition accuracy increases the computational burden on embedded devices; simplifying models to suit low-resource devices may further reduce the robustness of short command word recognition. Therefore, how to reduce the false recognition rate of short command words and colloquial command words while ensuring efficient resource utilization of embedded devices has become a pressing technical problem in the field of speech recognition. Summary of the Invention

[0005] This invention provides a speech recognition method, apparatus, device, and medium, aiming to solve the technical problem of high probability of speech misrecognition in the prior art.

[0006] To achieve the above-mentioned objective, the first aspect of the present invention provides a speech recognition method, the method comprising: Obtain the speech features to be recognized; The speech features are identified based on a preset command word recognition network to obtain a first recognition result, and it is determined whether there are command words in the speech features. If a command word exists, determine whether the command word is a misspellable word; If the command word is a misspellable word, then the speech features are identified based on the misspellable word recognition network to obtain a second recognition result; Determine whether the first identification result and the second identification result are consistent; If they match, then the action corresponding to the spoken feature is executed.

[0007] Furthermore, the acquisition of the speech features to be recognized includes: Acquire audio signals; Based on the feature extraction part of the preset speech recognition network, the features of the speech signal are extracted to obtain the speech features.

[0008] Furthermore, the feature extraction parts of the command word recognition network and the error-prone word recognition network are the same as the feature extraction parts of the speech recognition network, and reuse the weights of the feature extraction parts of the speech recognition network; The number of parameters in the error-prone word recognition network is smaller than that in the command word recognition network; and the number of parameters in the command word recognition network is smaller than that in the speech recognition network.

[0009] Furthermore, the training method for the command word recognition network includes: The general corpus and the command corpus are mixed according to a preset ratio to obtain the command word training set; An initial command word recognition network is trained based on the command word training set to obtain the command word recognition network, wherein the feature extraction part of the initial command word recognition network is frozen and the weights of the feature extraction part of the speech recognition network are reused.

[0010] Furthermore, the step of mixing the general corpus and the command corpus according to a preset ratio to obtain the command word training set includes: The general corpus and the command corpus are respectively enhanced using a preset enhancement method; The enhanced general corpus and the command word corpus are mixed according to a preset ratio to obtain the command word training set.

[0011] Furthermore, the training method for the speech recognition network includes: The general corpus of a preset number of words is enhanced according to the enhancement method to obtain a general training set. The initial speech recognition network is trained based on the general training set to obtain the speech recognition network.

[0012] Furthermore, the training method for the error-prone word recognition network includes: The general corpus and the corpus of easily confused words are mixed in a preset ratio to obtain the training set of easily confused words; wherein, the easily confused words are a subset of the command words, including command words with fewer than a preset number of words, important command words and / or command words that are easily identified as being wrong after testing; An initial error-prone word recognition network is trained based on the error-prone word training set to obtain the error-prone word recognition network. The feature extraction part of the initial error-prone word recognition network is frozen, and the weights of the feature extraction part of the speech recognition network are reused.

[0013] A second aspect of the present invention provides a speech recognition method apparatus, comprising: Acquisition unit, used to acquire the speech features to be recognized; The first recognition unit is used to recognize the speech features based on a preset command word recognition network, obtain a first recognition result, and determine whether there are command words in the speech features; The first judgment unit is used to determine whether the command word is a mispronounceable word if a command word exists. The second recognition unit is used to recognize the speech features based on the error-prone word recognition network if the command word is an error-prone word, and obtain a second recognition result. The second judgment unit is used to determine whether the first recognition result and the second recognition result are consistent; An execution unit is configured to execute the action corresponding to the speech feature if the first recognition result and the second recognition result are consistent.

[0014] A third aspect of the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the speech recognition method described in any of the preceding claims.

[0015] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the speech recognition method described in any of the preceding claims.

[0016] Beneficial effects: The speech recognition method, apparatus, device, and medium of this invention employ a cascaded architecture of "initial screening by a command word recognition network + secondary verification by a misidentification word recognition network" to perform dual judgment on high-risk misidentification words, executing an action only when the recognition results of the two networks are consistent. Compared with existing single CTC algorithms, this mechanism utilizes the low probability overlap of independent decision-making by the two networks to reduce the misidentification rate of easily misidentified words such as short command words. By forcing the recognition results of the two networks to be consistent before executing an action, a "logical AND" security verification mechanism is formed. This mechanism uses two independently trained networks to cross-validate the judgment results of the same speech feature, overcoming the misjudgment defect of a single network when the feature representation of short command words is ambiguous. The command word recognition network and the misidentification word recognition network can be lightweight designs, maintaining low computational overhead while implementing secondary verification. In existing technologies, single networks based on CTC algorithms struggle to balance the recognition accuracy and resource consumption of short command words: either the model simplification leads to a high misidentification rate, or the complex model increases the burden on the device. This invention, through its innovative architecture of "hierarchical recognition + secondary verification," reduces the false recognition rate by an order of magnitude compared to existing technologies without significantly increasing computing resources. At the same time, it maintains high response efficiency through a rapid processing mechanism for non-error-prone words, fundamentally solving the contradiction between "accuracy and resource consumption" in embedded speech recognition. Attached Figure Description

[0017] Figure 1 A flowchart illustrating a speech recognition method according to an embodiment of the invention; Figure 2 This is a schematic block diagram of the structure of a voice recognition device according to an embodiment of the invention; Figure 3 This is a schematic block diagram of a computer device according to an embodiment of the invention.

[0018] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0020] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of features, integers, steps, operations, elements, modules, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, modules, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any modules and all combinations of one or more associated listed items.

[0021] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0022] Reference Figure 1 This invention provides a speech recognition method, including the following steps S1-S6: S1: Obtain the speech features to be recognized.

[0023] The aforementioned speech features are characteristic parameters extracted from speech signals that can be used for recognition, such as Mel-frequency cepstral coefficients (MFCCs) and spectral features, used to characterize the acoustic properties of speech. Speech signals from the environment are acquired via a microphone, preprocessed (e.g., noise reduction, framing), and then input into a feature extraction module (as described in the feature extraction part of the speech recognition network below) to obtain speech features. When a user says "turn on the light," the microphone acquires the speech signal, and the spectral features are extracted using a pre-defined feature extractor (e.g., the first three convolutional layers of a Conformer model), resulting in an 80-dimensional MFCC feature sequence. Extracting robust speech features provides a foundation for subsequent recognition and reduces the interference of environmental noise on the recognition process.

[0024] S2: The speech features are identified based on a preset command word recognition network to obtain a first recognition result, and it is determined whether there are command words in the speech features.

[0025] The aforementioned command word recognition network is a neural network model used to recognize preset command words. It has a small number of parameters (e.g., 300K) and focuses on recognizing specific command words. Speech features are input into the command word recognition network, and the recognition result is obtained through forward propagation. The network then determines whether the result belongs to the preset command word set (e.g., "turn on the light" or "turn off the light"). For example, the command word recognition network decodes the speech features of "turn on the light," outputting the first recognition result as "turn on the light," thus determining that a command word exists. This dedicated network focuses on command word recognition, improving recognition speed and specificity while reducing redundant computation in general-purpose models.

[0026] S3: If a command word exists, determine whether the command word is a misspellable word.

[0027] The aforementioned error-prone words are a subset of command words selected due to their short length, high importance, or high misidentification rate (e.g., "turn on the lights" or "come back"). Based on preset rules (less than 3 characters, user interaction entry point, high misidentification rate during testing), it is determined whether the current command word is an error-prone word. For example, "turn on the lights" is a two-character command word, belonging to the preset error-prone word set, and is therefore identified as an error-prone word. Accurately locating high-risk misidentification words provides a basis for secondary verification and avoids redundant judgments on all command words.

[0028] If the command word is not a misspellable word, the corresponding action is executed directly based on the first recognition result, ignoring the output of the misspellable word recognition network. For example, if a user says "turn on the air conditioner," the command word recognition network outputs "turn on the air conditioner" as the first recognition result. If this command word is determined to be not a misspellable word (because it has ≥3 characters and the test false recognition rate is low), then the command to turn on the air conditioner is sent directly without calling the misspellable word recognition network, or the output of the misspellable word recognition network is ignored. Skipping the secondary recognition process for non-misspellable words reduces unnecessary computational consumption (such as saving 10ms inference time), improves system response efficiency, and avoids redundant processing of non-high-risk words by the lightweight misspellable word recognition network.

[0029] S4: If the command word is a misspellable word, then the speech features are identified based on the misspellable word recognition network to obtain a second recognition result.

[0030] The aforementioned error-prone word recognition network is a dedicated network with a smaller parameter count (e.g., 100K), trained only on error-prone words for secondary validation. The speech features corresponding to the error-prone words are then input into the error-prone word recognition network again to obtain a second recognition result. For example, the error-prone word recognition network processes the speech features of "turn on the lights" again, outputting the second recognition result as "turn on the lights." This secondary recognition mechanism, through independent judgment by two networks, significantly reduces the probability of misrecognition by a single network.

[0031] S5: Determine whether the first recognition result and the second recognition result are consistent.

[0032] S6: If they match, then execute the action corresponding to the speech feature.

[0033] The system compares the first and second recognition results. If they match, the device is triggered to perform the corresponding action (such as turning on the lights). For example, if both recognitions are "turn on the lights," a command to turn on the lights is sent to the smart light. Actions are only executed when both networks simultaneously confirm the change, significantly reducing the probability of false triggers and improving interaction reliability.

[0034] In this embodiment, a two-stage mechanism of "initial screening by a command word recognition network + secondary verification by a misspellable word recognition network" solves the problem of misidentification of short command words and other misspellable words. Specifically, the command word recognition network focuses on identifying specific command words, while the misspellable word recognition network strengthens verification for high-risk words. The combination of these two mechanisms forms a cascaded judgment, avoiding the misjudgment defects of a single network. The dedicated network structure is adapted to embedded devices, reducing computational resource consumption while ensuring accuracy.

[0035] In one embodiment, obtaining the speech features to be identified includes: S11: Acquire audio signals.

[0036] Speech signals from the environment, such as user commands to turn off the lights, are acquired using microphone arrays and other sound pickup devices. The sampling rate is 16kHz, and the quantization precision is 16bit. High-fidelity acquisition ensures the quality of the original signal, providing reliable input for feature extraction.

[0037] S12: Based on the feature extraction part of the preset speech recognition network, extract the features of the speech signal to obtain the speech features.

[0038] The feature extraction part of the aforementioned speech recognition network refers to the first few convolutional layers (e.g., the first three layers) of a general recognition network (such as Conformer), used to downsample the speech signal and extract basic acoustic features. The acquired speech signal is input into the feature extraction part of the general recognition network (the weights have been pre-trained on a large-scale general corpus) for feature extraction. For example, the first three convolutional layers of the general recognition network (10M parameters) process the speech signal for "turn off the lights," outputting a feature vector containing both time and frequency domain information. The pre-trained feature extractor is reused, leveraging its robust feature representation capabilities learned on the general corpus to improve the noise resistance and accuracy of subsequent recognition.

[0039] In this embodiment, the speech signal is processed by the feature extraction part of a general recognition network, converting the original audio into a feature vector suitable for recognition. The feature extractor is trained on a large-scale general corpus, possessing strong generalization ability and adapting to speech signal processing in different environments. Compared with randomly initialized networks, pre-trained feature extractors have improved feature representation capabilities and reduce the impact of environmental noise on features; reusing feature extractors can avoid repeated training, reducing model training time and computational resource consumption.

[0040] In one embodiment, the feature extraction parts of the command word recognition network and the error-prone word recognition network are the same as the feature extraction part of the speech recognition network, and the weights of the feature extraction part of the speech recognition network are reused.

[0041] Reusing the weights of the feature extraction part of the speech recognition network means that the command word recognition network and the error-prone word recognition network directly use the feature extraction layer weights trained by the general recognition network, without needing to retrain. Specifically, the first three convolutional layers of the command word recognition network and the error-prone word recognition network are completely identical to the general recognition network, and the weights are directly loaded from the training results of the general network. For example, if the first three convolutional layers of the general recognition network have a weight size of 8M, the command word recognition network directly reuses these weights, training only the subsequent classification layers (with 300K parameters). This avoids redundant training of the feature extraction layers, significantly reducing training time and the number of model parameters, while ensuring the consistency of feature extraction capabilities.

[0042] The number of parameters in the error-prone word recognition network is smaller than that in the command word recognition network; and the number of parameters in the command word recognition network is smaller than that in the speech recognition network.

[0043] In this embodiment, the general recognition network (10M) > command word recognition network (300K) > error-prone word recognition network (100K), and the design adapts to low-resource devices through a decreasing number of parameters. For example, the error-prone word recognition network only contains a feature extraction layer (reusing the general network) and a lightweight classification layer, with a total of 100K parameters, which can be deployed on smart speaker chips with limited memory. The lightweight design allows the network to be deployed in embedded devices, reducing inference time (e.g., reducing the time for recognizing a single command from 50ms to 15ms) and reducing power consumption.

[0044] In this embodiment, an architecture of "feature extraction layer reuse + parameter reduction" is designed to achieve lightweight network deployment while ensuring feature extraction capabilities. Reusing pre-trained weights ensures feature consistency, while decreasing the number of parameters accommodates the resource constraints of different devices. Weight reuse makes the feature extraction capabilities of the command word recognition network and the error-prone word recognition network consistent with the general network, improving recognition accuracy. The lightweight design allows the model to be deployed on embedded devices with less than 1MB of memory, adapting to low-resource scenarios such as smart homes.

[0045] In one embodiment, the training method for the command word recognition network includes: The general corpus and the command corpus are mixed according to a preset ratio to obtain the command word training set.

[0046] The general corpus refers to a large amount of everyday speech data (such as thousands of hours of dialogue and broadcast data), used to train general feature extraction capabilities. The command word corpus refers to specific command word speech data (such as "turn on the lights" or "play music") synthesized through TTS (Text-to-Speech) or collected manually. The general corpus and command word corpus are mixed in an 8:2 ratio to form a balanced training set, preventing the model from overfitting to command words. For example, the general corpus might consist of 1000 hours of everyday dialogue, and the command word corpus might consist of 200 hours of command word speech synthesized through TTS, resulting in a 1200-hour training set. This mixed training set allows the model to maintain its general feature extraction capabilities while focusing on command word recognition, improving generalization.

[0047] An initial command word recognition network is trained based on the command word training set to obtain the command word recognition network, wherein the feature extraction part of the initial command word recognition network is frozen and the weights of the feature extraction part of the speech recognition network are reused.

[0048] The initial command word recognition network loads and freezes the weights of the feature extraction layer from the general recognition network, training only the parameters of the subsequent classification layer. For example, during training, the weights of the feature extraction layer remain fixed, while the classification layer optimizes its parameters through backpropagation, converging after 100 iterations. Freezing the pre-trained feature extraction layer avoids catastrophic forgetting and ensures that the model still possesses strong robust feature representation capabilities during command word recognition.

[0049] In this embodiment, a strategy of "mixed corpus training + feature extraction layer freezing" enables the command word recognition network to learn command word recognition capabilities specifically while reusing general features. The mixing ratio and freezing strategy balance generality and specialization. Compared with training using command word corpora alone, mixed training significantly reduces the WER of command word recognition; freezing the feature extraction layer reduces the number of training iterations, improving training efficiency.

[0050] In one embodiment, the step of mixing the general corpus and the command word corpus according to a preset ratio to obtain the command word training set includes: The general corpus and the command corpus are respectively enhanced using a preset enhancement method.

[0051] Data augmentation methods refer to techniques that artificially expand training data, including adding noise, reverberation, speed modulation, pitch shifting, and spectral enhancement, to simulate speech changes in real-world scenarios. The same augmentation strategies are applied to both general and imperative corpora to ensure consistent adaptability of the feature extractor across different data types. For example, 5dB of office background noise is added to a segment of everyday conversation in the general corpus, and the speed is adjusted by ±3%; the imperative phrase "turn on the lights" is synthesized via TTS and then reverberated for 60ms to simulate an echo effect in a home environment. Data augmentation allows the training data to cover more realistic scenarios, improving the model's robustness to noise, reverberation, and other interferences, and preventing overfitting.

[0052] The enhanced general corpus and the command word corpus are mixed according to a preset ratio to obtain the command word training set.

[0053] The enhanced general corpus and command word corpus are mixed in a preset ratio (e.g., 8:2) to form a balanced training set, ensuring that the model learns both general features and focuses on command word recognition. For example, the enhanced general corpus is 800 hours long (including noise reduction, speed adjustment, etc.), and the command word corpus is 200 hours long (including reverberation, pitch shifting, etc.), resulting in a 1000-hour training set for training the command word recognition network. Mixing the enhanced corpus exposes the model to diverse speech features during training, improving its recognition accuracy in complex environments and reducing the false recognition rate compared to unenhanced data.

[0054] In this embodiment, a "first augment, then mix" strategy ensures that the general corpus and the command word corpus are trained under the same augmentation conditions, avoiding performance degradation of the feature extractor due to differences in data distribution. Data augmentation simulates speech interference in real-world scenarios, while the mixing ratio balances the model's generality and specificity. Unified data augmentation ensures the consistency of the feature extractor across different corpora, avoiding performance fluctuations caused by differences in training conditions. The augmented mixed training set improves the accuracy of the command word recognition network in noisy environments, significantly reducing misidentification caused by environmental factors.

[0055] In one embodiment, the training method for the above-mentioned speech recognition network includes: The general corpus of a preset number of words is enhanced according to the enhancement method to obtain a general training set.

[0056] A general training set was formed by selecting 1000 hours of general corpus and processing it using the same enhancement strategies as the command word training set (such as adding 10dB white noise, 50ms reverberation, ±5% speed variation, etc.).

[0057] The core purpose of ensuring consistency between the enhancement method and the command word training set is to guarantee the "feature consistency" of the feature extractor. For example, if the general training set does not add reverberation, while the command word training set does, the feature extractor of the general network may not be able to learn the speech representation in a reverberant environment. When this feature extractor is reused in the command word recognition network, it will exhibit recognition bias when faced with command word corpora containing reverberation. For instance, both the general training set and the command word training set add 60ms of room reverberation to the speech. During the training phase of the general network, the feature extractor learns the spectral features in a reverberant environment. When the command word recognition network reuses this feature extractor, it does not need to readapt to reverberation interference and can directly perform effective feature extraction on the command word corpora, avoiding the performance degradation caused by inconsistent enhancement methods (e.g., WER increasing from 3% to 8%). By enhancing a preset amount of general corpus according to the enhancement method, the consistency of feature transfer can be guaranteed, ensuring that the anti-interference features (such as noise and reverberation robustness) learned by the general network can be directly transferred to the command word recognition network, avoiding adaptation problems caused by differences in feature distribution; there is no need to re-optimize the feature extractor for the command word recognition network, reducing the number of training iterations; the unified enhancement strategy enables the feature extractor to maintain stable performance in different scenarios, and compared with inconsistent enhancement methods, the false recognition rate of the command word recognition network is significantly reduced.

[0058] The initial speech recognition network is trained based on the general training set to obtain the speech recognition network.

[0059] The initial network is trained using an enhanced general training set, with parameters except for the feature extraction part frozen until the word error rate (WER) on the validation set is below 5% or the loss function converges. For example, the initial network uses a Conformer architecture, with the first three convolutional layers acting as feature extractors and subsequent layers as general recognition modules. During training, the parameters of the entire network are optimized through backpropagation. After 200 iterations, the WER on the validation set drops to 4.2%, at which point training stops. Training on a large-scale enhanced corpus gives the feature extractor of the general network strong generalization ability, providing high-quality feature input for command word recognition networks and error-prone word recognition networks. Compared to training without enhancement, the feature representation capability is improved by approximately 30%.

[0060] In this embodiment, the design logic of "enhancing method consistency" is a key prerequisite for achieving "feature extractor reuse." By forcing consistency in enhancement strategies between the general training set and the command word training set, the stability of the feature extractor during cross-task transfer is ensured, fundamentally avoiding performance fluctuations caused by differences in data processing. Through enhanced processing and large-scale training of the general corpus, the feature extractor of the speech recognition network can capture rich acoustic features of speech, adapting to different noise and reverberation environments. The consistency between the enhancement method and the subsequent command word recognition network ensures the compatibility of the feature extractor in different tasks. The enhanced general training set significantly improves the anti-interference ability of the speech recognition network's feature extractor in complex environments; the pre-trained feature extractor provides a high-quality feature foundation for the subsequent command word recognition network and error-prone word recognition network, reducing performance losses in transfer learning.

[0061] In one embodiment, the training method for the above-mentioned error-prone word recognition network includes: The general corpus and the corpus of error-prone words are mixed in a preset ratio to obtain the error-prone word training set; wherein the error-prone words are a subset of the command words, including command words with fewer than a preset number of words, important command words, and / or command words that are easily identified as being incorrect after testing.

[0062] The aforementioned error-prone words are high-risk misidentification words selected from command words, meeting one of the following conditions: fewer than 3 characters (e.g., two-character command words), core user interaction words (e.g., wake-up words), or words with a misidentification rate >10% in command word recognition network tests. For example, 10 error-prone words (e.g., "turn on the lights," "I'm back," "hello") are selected from 60 command words, and the general corpus and error-prone word corpus are mixed in a 7:3 ratio to form a dedicated training set. In a specific example, the error-prone word selection process is as follows: the command word recognition network is tested, and it is found that the misidentification rate of "turn on the lights" is 15% (higher than the threshold of 10%), "I'm back" is a two-character command word, and "hello" is a wake-up word, all of which are included in the error-prone word set. 500 hours of general corpus and 200 hours of error-prone word corpus (synthesized through TTS or collected manually) are mixed in a 7:3 ratio to obtain a 700-hour training set. Focusing on a training set of high-risk words allows the model to specifically optimize its ability to identify error-prone words, avoiding applying equal effort to all command words and improving training efficiency.

[0063] An initial error-prone word recognition network is trained based on the error-prone word training set to obtain the error-prone word recognition network. The feature extraction part of the initial error-prone word recognition network is frozen, and the weights of the feature extraction part of the speech recognition network are reused.

[0064] The initial error-prone word recognition network loads and freezes the weights of the feature extraction layer of the general network, training only the subsequent lightweight classification layer (with <100K parameters) until the error-prone word recognition WER on the validation set is <2%. For example, the error-prone word recognition network architecture is: reusing the first three convolutional layers of the general network (with frozen weights), followed by a 2-layer fully connected classification layer, with a total of 100K parameters. After 50 rounds of iterative training using a mixed training set, the false recognition rate for "turn on the light" on the validation set drops to 1.8%, at which point training stops. Freezing the pre-trained feature extractor ensures that the model inherits the strong feature representation capabilities of the general network, the lightweight classification layer design reduces the number of parameters, allowing the network to be deployed on low-resource devices, and the specialized training for error-prone words improves the recognition accuracy.

[0065] In this embodiment, a strategy of "precise screening of common misspelled words + lightweight network-specific training" is employed. Based on reusing a general feature extractor, reinforcement learning is performed on high-risk words. The hybrid training set balances generality and specificity, the freezing strategy avoids degradation of feature extraction capabilities, and the lightweight structure is adaptable to embedded devices. Targeted training significantly reduces the false recognition rate of common misspelled words compared to command word recognition networks alone, and the probability of false triggering is significantly reduced after cascading the two networks. The lightweight design (100K parameters) allows the common misspelled word recognition network to be deployed on devices with less than 512KB of memory, and its power consumption is lower than that of general models, meeting the low-resource requirements of scenarios such as smart homes.

[0066] Reference Figure 2 The present invention also provides a speech recognition method apparatus for executing any of the above embodiments, comprising: Acquisition unit 10 is used to acquire the speech features to be recognized; The first recognition unit 20 is used to recognize the speech features based on a preset command word recognition network, obtain a first recognition result, and determine whether there is a command word in the speech features; The first judgment unit 30 is used to determine whether the command word is a mispronounced word if a command word exists. The second recognition unit 40 is used to recognize the speech features based on the error-prone word recognition network if the command word is an error-prone word, and obtain a second recognition result. The second judgment unit 50 is used to determine whether the first recognition result and the second recognition result are consistent; The execution unit 60 is configured to execute the action corresponding to the speech feature if the first recognition result and the second recognition result are consistent.

[0067] Reference Figure 3 The present invention also provides a computer device, the internal structure of which can be as follows: Figure 3As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor in this computer device is designed to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores operating devices, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device is used to store network models, etc. The network interface of the computer device is used for communication with external terminals via a network connection. Furthermore, the computer device may also be equipped with an input device and a display screen, etc. When the computer program is executed by the processor, it implements the speech recognition method of any of the above embodiments. Those skilled in the art will understand that... Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer equipment on which the present application is applied.

[0068] One embodiment of this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the speech recognition method of any of the above embodiments. It is understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0069] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in this application and in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0070] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0071] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A speech recognition method, characterized in that, The method includes: Obtain the speech features to be recognized; The speech features are identified based on a preset command word recognition network to obtain a first recognition result, and it is determined whether there are command words in the speech features. If a command word exists, determine whether the command word is a misspellable word; If the command word is a misspellable word, then the speech features are identified based on the misspellable word recognition network to obtain a second recognition result; Determine whether the first identification result and the second identification result are consistent; If they match, then the action corresponding to the spoken feature is executed.

2. The speech recognition method according to claim 1, characterized in that, The acquisition of the speech features to be identified includes: Acquire audio signals; Based on the feature extraction part of the preset speech recognition network, the features of the speech signal are extracted to obtain the speech features.

3. The speech recognition method according to claim 2, characterized in that, The feature extraction parts of the command word recognition network and the error-prone word recognition network are the same as the feature extraction parts of the speech recognition network, and reuse the weights of the feature extraction parts of the speech recognition network. The number of parameters in the error-prone word recognition network is smaller than that in the command word recognition network; and the number of parameters in the command word recognition network is smaller than that in the speech recognition network.

4. The speech recognition method according to claim 3, characterized in that, The training method for the command word recognition network includes: The general corpus and the command corpus are mixed according to a preset ratio to obtain the command word training set; An initial command word recognition network is trained based on the command word training set to obtain the command word recognition network, wherein the feature extraction part of the initial command word recognition network is frozen and the weights of the feature extraction part of the speech recognition network are reused.

5. The speech recognition method according to claim 4, characterized in that, The process of mixing general corpora and command word corpora according to a preset ratio to obtain a command word training set includes: The general corpus and the command corpus are respectively enhanced using a preset enhancement method; The enhanced general corpus and the command word corpus are mixed according to a preset ratio to obtain the command word training set.

6. The speech recognition method according to claim 5, characterized in that, The training method for the speech recognition network includes: The general corpus of a preset number of words is enhanced according to the enhancement method to obtain a general training set. The initial speech recognition network is trained based on the general training set to obtain the speech recognition network.

7. The speech recognition method according to claim 4, characterized in that, The training method for the error-prone word recognition network includes: The general corpus and the corpus of easily confused words are mixed in a preset ratio to obtain the training set of easily confused words; wherein, the easily confused words are a subset of the command words, including command words with fewer than a preset number of words, important command words and / or command words that are easily identified as being wrong after testing; An initial error-prone word recognition network is trained based on the error-prone word training set to obtain the error-prone word recognition network. The feature extraction part of the initial error-prone word recognition network is frozen, and the weights of the feature extraction part of the speech recognition network are reused.

8. A speech recognition method apparatus, characterized in that, include: Acquisition unit, used to acquire the speech features to be recognized; The first recognition unit is used to recognize the speech features based on a preset command word recognition network, obtain a first recognition result, and determine whether there are command words in the speech features; The first judgment unit is used to determine whether the command word is a mispronounceable word if a command word exists. The second recognition unit is used to recognize the speech features based on the error-prone word recognition network if the command word is an error-prone word, and obtain a second recognition result. The second judgment unit is used to determine whether the first recognition result and the second recognition result are consistent; An execution unit is configured to execute the action corresponding to the speech feature if the first recognition result and the second recognition result are consistent.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the speech recognition method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the speech recognition method as described in any one of claims 1 to 7.

Images