Offline speech recognition vocabulary expansion methods, systems, devices, and storage media

Abstract

This application relates to the field of speech recognition technology, and in particular to an offline speech recognition vocabulary expansion method, system, device, and storage medium. The method includes: performing offline semantic parsing based on baseline speech commands in a vocabulary library to obtain semantic structure data; performing multimodal vocabulary generalization processing based on the semantic structure data to generate semantically equivalent variant vocabulary; injecting the variant vocabulary into the vocabulary library of the offline speech recognition system; and recognizing and executing commands through the vocabulary library when a user inputs a speech command. This application intelligently generates a large number of variant vocabulary entries that conform to daily communication habits, significantly improving the accuracy and flexibility of recognition.

Description

Technical Field

[0001] This application relates to the field of speech decoding, and in particular to an offline speech recognition term expansion method, system, device and storage medium. Background Technology

[0002] With the rapid development of voice interaction applications such as smart homes and in-vehicle systems, users tend to control devices using natural language commands. However, the ways in which the same intention is expressed are highly diverse (e.g., "turn on the air conditioner" may be expressed as "start the cooling system" or "turn on the air conditioner for me"). To address this, existing technologies typically employ offline speech recognition solutions based on predefined rules or thesauruses, generalizing terms by manually configuring limited synonyms (e.g., mapping "open" to "start") or using simple template matching. However, this method heavily relies on human experience, struggles to cover the rich expressions of natural language, and requires manual updates to the thesaurus for every new expression, resulting in high maintenance costs. This leads to a significant drop in recognition accuracy for offline speech recognition systems when faced with diverse expression scenarios, limiting the user experience.

[0003] Therefore, how to achieve automated, high-coverage term generalization in an offline environment and improve the accuracy of speech recognition is an urgent problem to be solved. Summary of the Invention

[0004] The main objective of this application is to provide an offline speech recognition term expansion method, system, device, and storage medium, aiming to solve the technical problem of how to achieve automated, high-coverage term generalization in an offline environment and improve the accuracy of speech recognition.

[0005] To achieve the aforementioned objectives, this application proposes an offline speech recognition vocabulary expansion method, comprising:

[0006] Offline semantic parsing is performed based on baseline speech commands in the dictionary to obtain semantic structure data;

[0007] Based on the semantic structure data, multimodal term generalization processing is performed to generate semantically equivalent variant terms;

[0008] Inject the variant terms into the dictionary of the offline speech recognition system;

[0009] When a user inputs a voice command, the command is recognized and executed using the dictionary.

[0010] Further, prior to the step of injecting the variant term into the lexicon of the offline speech recognition system, the following steps are included:

[0011] Based on the lexicon, an accent feature library containing pronunciation variation rules and dialect word usage features is constructed;

[0012] The standard words corresponding to the baseline speech command are mapped by pronunciation similarity through the accent feature library to generate variant words that conform to regional pronunciation habits;

[0013] Based on the dialect word features in the accent feature library, regional word replacement is performed on the variant entries, and the final accent-adapted variant entries are output.

[0014] Furthermore, the step of performing offline semantic parsing based on benchmark speech commands in the lexicon to obtain semantic structure data includes:

[0015] Semantic analysis of baseline speech commands in the lexicon is performed based on an offline NLP model;

[0016] Based on the semantic analysis results, the baseline speech command is part-of-speech segmented to form the smallest semantic units with different parts of speech;

[0017] Part-of-speech tagging is performed on each smallest semantic unit word to obtain semantic structure data.

[0018] Furthermore, the step of performing multimodal term generalization processing based on the semantic structure data to generate semantically equivalent variant terms includes:

[0019] Based on a natural language processing model, synonym replacement is performed on phrases of different parts of speech contained in semantic structure data to generate generalized vocabulary entries;

[0020] Based on the semantic structure of the generalized term and the reference term corresponding to the reference speech instruction, language modification elements are inserted at specified positions in the semantic structure of the generalized term and the reference term to obtain semantically equivalent variant terms.

[0021] Further, after the step of inserting language modifier elements at specified positions in the semantic structures of the generalized term and the reference term corresponding to the reference speech instruction, the process includes:

[0022] Based on a preset syntax rule library, sentence transformation is performed on the generalized terms and the benchmark terms corresponding to the benchmark speech commands;

[0023] By adding time, location, or conditional adverbs, the generalized terms and the reference terms corresponding to the reference speech commands are compounded and expanded.

[0024] The order of sentence components is adjusted based on a pre-defined syntactic rule base.

[0025] Further, the step of injecting the variant terms into the lexicon of the offline speech recognition system includes:

[0026] The variant terms are formatted and encoded, duplicate terms in the dictionary are eliminated, and the processed variant terms are incrementally loaded into the dictionary index of the offline speech recognition engine.

[0027] Furthermore, the step of recognizing and executing user-input voice commands using the dictionary includes:

[0028] A lightweight noise reduction model is used to filter out environmental noise in real time and extract pure human voice signals from voice commands.

[0029] The start and end positions of voice commands are determined based on a time-series analysis model, and the acoustic features of the voice commands are extracted.

[0030] The acoustic features are matched against all entries in the dictionary, and the control commands corresponding to the matched entries are executed.

[0031] A second aspect of this application also proposes an offline speech recognition vocabulary expansion system, comprising:

[0032] The parsing module is used to perform offline semantic parsing based on benchmark speech commands in the lexicon to obtain semantic structure data;

[0033] The generalization processing module is used to perform multimodal term generalization processing based on the semantic structure data to generate semantically equivalent variant terms;

[0034] The injection lexicon module is used to inject the variant entries into the lexicon of the offline speech recognition system;

[0035] The instruction execution module is used to recognize and execute user-input voice commands using the dictionary.

[0036] A third aspect of this application also includes a computer device comprising a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of any of the methods described above.

[0037] The fourth aspect of this application also includes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0038] Beneficial effects:

[0039] This application obtains semantic structure data through offline semantic parsing based on baseline speech commands in a lexicon. Then, it performs multimodal term generalization processing to generate semantically equivalent variant terms. Finally, these variant terms are injected into the lexicon of an offline speech recognition system. This method significantly expands the speech recognition system's ability to understand users' colloquial expressions. Traditional systems are limited by fixed preset term lists and perform poorly when faced with diverse, natural language commands. This invention, however, can intelligently generate a large number of variant terms that conform to daily communication habits, significantly improving the accuracy and flexibility of recognition. Secondly, since all processing is completed locally without relying on a network connection, it ensures efficient operation even in offline environments. This greatly reduces the workload of manual term maintenance, lowers costs, and improves the system's maintainability and adaptability, making voice interaction more natural and fluent, and greatly improving the user experience. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating an embodiment of the offline speech recognition term expansion method of this application;

[0041] Figure 2 This is a schematic block diagram of the structure of an offline speech recognition vocabulary expansion system according to an embodiment of this application;

[0042] Figure 3 This is a schematic block diagram of the structure of a computer device according to an embodiment of this application.

[0043] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0045] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of features, integers, steps, operations, elements, modules, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, modules, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and or” as used herein includes all or any modules and all combinations of one or more associated listed items.

[0046] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0047] Reference Figure 1 This invention provides an offline speech recognition vocabulary expansion method, including steps S1-S4, specifically:

[0048] S1. Perform offline semantic parsing based on the baseline speech commands in the dictionary to obtain semantic structure data;

[0049] S2. Based on the semantic structure data, perform multimodal term generalization processing to generate semantically equivalent variant terms;

[0050] S3. Inject the variant terms into the dictionary of the offline speech recognition system;

[0051] S4. When a user inputs a voice command, the command is recognized and executed using the dictionary.

[0052] As described in step S1, the system first receives one or more baseline voice commands as input. These commands come from a pre-defined command vocabulary, such as "turn on the air conditioner." Next, a lightweight natural language processing (NLP) model optimized for offline environments is used to perform deep semantic understanding on the core commands. This process mainly includes several stages: intent recognition, key information extraction and segmentation, part-of-speech tagging, and semantic role labeling.

[0053] In the intent recognition phase, the system accurately determines that the user's core intent is "to turn on the device," meaning that "turn on the air conditioner" represents the user's desire to turn on a certain device. Next, in the key information extraction and segmentation, "turn on the air conditioner" is broken down into two smallest semantic units: "turn on" (verb / action) and "air conditioner" (noun / object). Further, through part-of-speech tagging, these two units are labeled with their respective parts of speech, such as verb and noun. This step provides the necessary structured information for subsequent generalization generation. Finally, through semantic role tagging, the system identifies the agent, patient, and other semantic roles in the instruction. For example, in this case, "air conditioner" is the patient, and "turn on" is the action performed on that patient.

[0054] The data processing involved throughout the process includes, but is not limited to, text preprocessing (such as stop word removal), word segmentation, and part-of-speech tagging. Furthermore, to adapt to different application scenarios, various implementation methods can be adopted. For example, for certain specific domains (such as smart home control), a predefined domain knowledge base can be used to enhance the accuracy of intent recognition; while in broader everyday communication scenarios, more complex semantic analysis algorithms may be needed to capture subtle semantic differences.

[0055] The advantage of this step lies in its ability to provide a precise foundation for semantic understanding and to operate efficiently even completely offline, without relying on a network connection. This is particularly important for resource-constrained embedded devices. For example, suppose there is a smart speaker; if a user says "turn on the air conditioner," even without a network connection, the speaker can accurately understand the user's intent through the above steps and prepare for the next step of word generalization. This capability significantly improves the flexibility and robustness of voice interaction systems, enabling the system to accurately understand and respond to user needs even in the face of diverse spoken expressions. Simultaneously, this lays a solid foundation for multimodal word generalization in subsequent steps, ensuring that variant words richly and accurately reflect the core meaning of the original instruction.

[0056] Step S2 performs multimodal term generalization processing based on the semantic structure data obtained from step S1, generating semantically equivalent but diverse variant terms. This step is a key step in improving the flexibility and robustness of the speech recognition system.

[0057] First, the system uses a natural language processing (NLP) model to perform synonymous or near-synonymous replacements on different词性词组 in the semantic structure data, generating preliminary generalized entries. For example, in the instruction "turn on the air conditioner", "turn on" can be replaced with "activate", "start", and "air conditioner" may be replaced with "air conditioner" or "air conditioner". This replacement is not limited to simple word substitution, but also includes more complex semantic associations such as hypernyms and hyponyms, ensuring that the generated variant entries can cover as wide a range of spoken language expression habits as possible.

[0058] Next, the system intelligently adds various natural language modifiers to these preliminarily generated generalized entries. This includes but is not limited to polite expressions (such as "please", "trouble"), imperative auxiliaries (such as "help me", "for me"), modal particles (such as "ah", "ne"), and degree adverbs (such as "slightly", "immediately"). In this way, the system can further generalize "turn on the air conditioner" into various forms such as "please help me turn on the air conditioner" and "trouble turn on the air conditioner for a moment". This not only increases the diversity of the entries but also makes the generated variants more in line with users' daily communication habits.

[0059] In addition, the system also presets a series of syntactic rules and transformation patterns based on the target language (such as Chinese) to achieve batch generation of generalized entries. For example, the system can transform active sentences and passive sentences (such as "turn on the air conditioner" into "let the air conditioner be turned on"), or add adverbial components such as time, place, and condition (such as "turn on the air conditioner now", "turn on the air conditioner in the living room"), and adjust the order of sentence components to generate variant entries with different structures but the same semantics. This step greatly enriches the system's vocabulary library and improves its ability to handle diverse expressions.

[0060] For users with strong accents, the present invention also particularly introduces a specially optimized NLP model. This model is built based on specific regional or accent characteristics and can map standard Mandarin entries to accent words with similar pronunciations but different glyphs. For example, in some regional accents where "n" and "l" are indistinguishable, the system can adapt "air conditioner" to a more suitable expression according to this. At the same time, according to the dialect word usage habits, the system can also perform regional word replacement, such as replacing "air conditioner" with "refrigerator" commonly used in some regions.

[0061] Step S2 not only significantly expands the vocabulary of the speech recognition system but also enhances its adaptability and user experience. Whether faced with diverse spoken expressions or regional accents, the system can accurately understand and respond to user commands, achieving truly intelligent and personalized service. For example, when a user says "Turn on the air conditioner," even with slight accent differences or informal expressions, the system can correctly recognize and execute the corresponding control command.

[0062] In step S3, the semantically equivalent variant entries generated in step S2 are injected into the vocabulary of the offline speech recognition system to ensure that the system can recognize and respond to diverse user expressions. This process involves several key steps and technical details, which not only improves the system's recognition accuracy and adaptability but also enhances the user experience.

[0063] In terms of specific implementation methods, the first step is to standardize the format and convert the encoding of the generated variant entries. Since entries from different sources may have format differences (such as punctuation, capitalization rules, etc.), the system will standardize these entries to ensure consistency and compatibility. For example, although the expressions "turn on the air conditioner" and "turn on the air conditioner" have the same semantics, their formats and structures may differ, so corresponding adjustments are needed to ensure they can be processed consistently in the dictionary.

[0064] The next step is deduplication, which eliminates redundant entries in the dictionary, avoiding unnecessary storage overhead and potential misidentification risks. A unique identifier for each entry is calculated using a hash algorithm and compared with existing entries in the dictionary, allowing for rapid detection and removal of duplicates. For example, when generating a variant of "turn on the air conditioner," if the entry already exists in the dictionary, it doesn't need to be added again, thus keeping the dictionary concise and efficient.

[0065] After the above preprocessing is completed, the variant words will be incrementally loaded into the offline speech recognition engine's dictionary index. The advantage of this approach is that it does not replace existing dictionary content, but rather adds it incrementally as a supplement. This reduces the performance burden of a full update while ensuring the dictionary is always up-to-date. For example, suppose the existing dictionary contains "turn on the air conditioner," and the newly generated variants include "start the air conditioner," "please turn on the air conditioner," etc. These new words will be directly added to the dictionary without requiring a complete rebuild of the dictionary.

[0066] This step offers significant advantages. First, by adding a large number of semantically equivalent but diverse variant entries, the system can better understand and respond to users' colloquial expressions, significantly improving recognition accuracy. Second, for accents or dialects with local characteristics, the system enhances its adaptability to users in different regions by introducing a specially optimized NLP model to generate entries that conform to specific pronunciation features. For example, in some regions where n / l are not distinguished, the system can automatically generate variant entries suitable for local users based on this pronunciation feature, such as adapting "air conditioner" to a more accurate pronunciation.

[0067] Assuming the baseline command is "turn on the air conditioner," after multimodal generalization processing in step S2, the system generates various variants such as "turn on the air conditioner," "please start the air conditioner," and "please turn on the air conditioner." In step S3, these variants are incrementally loaded into the dictionary after format unification, encoding conversion, and deduplication. When a user issues the command "please start the air conditioner," even if this expression differs from the original "turn on the air conditioner," the system can still accurately recognize the user's intent and execute the corresponding operation because the relevant variant entries have been added to the dictionary. This demonstrates how the present invention, through an intelligent dictionary expansion method, enables the offline speech recognition system to have stronger generalization capabilities and higher flexibility, thereby better serving a wide range of users.

[0068] As described in S4 above, firstly, when receiving voice commands from the user, the system uses a lightweight noise reduction model to filter out environmental noise in real time, ensuring that a pure human voice signal is extracted. For example, in a noisy home environment, a user might say, "Please turn on the air conditioner," while there might be television sound or other noise interference in the background. By using a noise reduction model such as RNNoise, these interfering factors can be effectively removed, preserving clear voice information.

[0069] Next, the start and end points of the voice command are determined based on a time-series analysis model (such as a lightweight bidirectional LSTM model), and the acoustic features of the voice command are extracted from it. The key to this stage is to accurately capture the complete voice command and avoid recognition errors caused by truncation. For example, if the sentence "Turn on the air conditioner" is mistakenly truncated as "Turn on the air conditioner," the user's intention cannot be correctly understood. Therefore, accurate determination of the start and end points is crucial to improving recognition accuracy.

[0070] Subsequently, the system matches the extracted acoustic features with all entries in the dictionary. Since a large number of semantically equivalent variant entries have been injected into the dictionary in step S3, the system can now recognize various expressions. For example, in addition to the standard "turn on the air conditioner," the dictionary includes variations such as "turn on the air conditioner" and "please start the air conditioner." When the user's actual voice command successfully matches one of these variations, the system executes the corresponding control command.

[0071] Suppose a user issues a voice command in a relatively noisy environment: "Please turn on the air conditioner." First, the system uses a noise reduction model to filter out background television sound and other environmental noise, extracting a clear speech signal. Then, a bidirectional LSTM model is used to determine the start and end points of the voice command, ensuring that the command "Please turn on the air conditioner" is fully captured. Next, the system matches the extracted acoustic features with entries in the vocabulary library. Since the vocabulary library contains various variant entries generated in step S2, such as "turn on the air conditioner" and "start the air conditioner," the system can accurately identify the user's intent and execute the corresponding air conditioner-turning operation. This example fully demonstrates how this invention achieves broader and more flexible voice command recognition capabilities through intelligent vocabulary expansion and optimization, thereby providing a more natural and convenient user experience.

[0072] By combining noise reduction processing, precise temporal analysis, and a rich vocabulary database, the system can accurately recognize users' voice commands in complex environments. Even if users have different expressions or accents, it can achieve high-precision matching. The lightweight model and algorithm design ensures efficient operation on resource-constrained embedded devices while reducing reliance on cloud resources and enhancing privacy protection.

[0073] In one embodiment, prior to the step of injecting the variant term into the lexicon of the offline speech recognition system, the following steps are included:

[0074] S20. Based on the lexicon, construct an accent feature library that includes pronunciation variation rules and dialect word usage features;

[0075] S21. The standard words corresponding to the reference speech command are mapped by pronunciation similarity through the accent feature library to generate variant words that conform to the regional pronunciation habits;

[0076] S22. Based on the dialect word features in the accent feature library, perform regional word replacement on the variant entries and output the final accent-adapted variant entries.

[0077] In this embodiment, precise accent adaptation is achieved through a multi-level dialect model. The system constructs a modular accent feature library, which can include regional sub-libraries such as the dialect model for Region A and the dialect model for Region B. Each sub-library of the accent feature library contains two types of core data: 1. Pronunciation variation rules: For example, the rule of mixing "zh / ch / sh" for flat and retroflex sounds in the Region A model, and the rule of confusing "n / l" in the Region B model; 2. Dialect vocabulary mapping table: For example, the Region B model maps "electric fan" to "fan", and the Region C model maps "turn off the light" to "turn off the bright". Then, through the phoneme alignment algorithm, pronunciation similarity mapping is achieved. For example, for the standard entry "turn on the air conditioner", the Region A model generates "da kai kong tiao" (flattening), and the Region D model generates "da hoi hong tiu" (replacement of initial and final consonants). Finally, semantic consistency verification is performed. For example, for "turn off the light", after replacing it with "turn off the bright" through the Region C model, the semantic equivalence with the standard instruction is verified; the generated results are screened to eliminate variants that do not conform to the grammar rules (such as "have you turned on the air conditioner" may be erroneously expanded to "has the air conditioner been turned on, right, right" in some dialects).

[0078] In summary, this embodiment constructs an accent feature library specifically for specific regions or accent characteristics based on the existing word library. When processing the standard entries input by the user, the system uses this accent feature library for pronunciation similarity mapping, converting the entries in standard Mandarin into habitual expressions closer to the local accent. To further optimize the matching effect, the system also replaces dialect words according to the accent feature library to ensure that the generated entries are more in line with the actual speaking habits of the users. Through this method, not only can the regional adaptability of the system be significantly improved, but also the user experience can be enhanced, making the voice interaction more natural and fluent. No matter how the user's accent changes, as long as their instructions can be recognized by the system and matched to the corresponding generalized entries, the corresponding control operations can be successfully executed. This design not only improves the regional adaptability and robustness of the system, but also greatly enhances the user experience, making the voice interaction more natural and fluent. In addition, by continuously accumulating and optimizing the accent feature library, the system can gradually improve its support ability for various complex accents and dialects, thus better serving the vast user group.

[0079] In one embodiment, the step of performing offline semantic parsing on the benchmark voice instruction in the word library to obtain semantic structure data includes:

[0080] S30. Perform semantic analysis on the benchmark voice instruction in the word library based on an offline NLP model;

[0081] S31. Perform词性切分 on the benchmark voice instruction based on the semantic analysis result to form the smallest semantic units of different词性;

[0082] S32. Perform part-of-speech tagging on each smallest semantic unit word to obtain semantic structure data.

[0083] In this embodiment, the system employs an efficient offline Natural Language Processing (NLP) method to parse benchmark speech commands and extract their semantic structure. First, the system uses a lightweight NLP model optimized for offline environments to perform deep semantic analysis on benchmark speech commands in the lexicon. This step involves more than just simple text analysis; it also includes complex processes such as intent recognition, key information extraction, and segmentation. For example, for the command "turn on the air conditioner," the system can accurately determine that the intent is to turn on a device and decompose the command into "turn on" (verb / action) and "air conditioner" (noun / object). Next, based on the semantic analysis results, the system performs part-of-speech segmentation on the command, breaking it down into the smallest semantic units and tagging each unit with its part of speech. This not only facilitates subsequent generalization processing but also provides the system with accurate semantic structure data. For example, "turn on" is tagged as a verb, and "air conditioner" is tagged as a noun. This tagging information allows the system to understand the user's intent and respond accordingly even when completely offline. Furthermore, this semantic structure-based data processing method greatly improves the system's flexibility and robustness, enabling it to better handle diverse forms of spoken expression.

[0084] In one embodiment, the step of performing multimodal term generalization processing based on the semantic structure data to generate semantically equivalent variant terms includes:

[0085] S40. Based on a natural language processing model, synonym replacement is performed on phrases of different parts of speech contained in semantic structure data to generate generalized vocabulary entries;

[0086] S41. Based on the semantic structure of the generalized term and the reference term corresponding to the reference speech instruction, insert language modification elements at specified positions in the semantic structure of the generalized term and the reference term to obtain semantically equivalent variant terms.

[0087] This embodiment details how to utilize natural language processing (NLP) technology to expand existing semantic structure data, thereby generating more diverse variant entries. First, the system applies a pre-trained NLP model to perform synonym or near-synonym replacements on different parts-of-speech phrases in the semantic structure data. For example, in "turn on the air conditioner," "turn on" can be replaced with "open" or "start," while "air conditioner" might become "cooling machine" or "air conditioner." This replacement is not limited to simple word substitution but also includes complex semantic associations such as hypernyms and hyponyms. Next, based on these initially generated generalized entries, the system intelligently adds various natural language modifiers to the middle, beginning, and end of the corresponding entries according to their structure. This includes polite expressions (such as "please"), imperative particles (such as "help me"), modal particles (such as "ah"), and degree adverbs (such as "slightly"). For example, "turn on the air conditioner" can be expanded to "please help me turn on the air conditioner" or "please turn on the air conditioner." In this way, the system not only increases the diversity of entries but also makes the generated variants more in line with users' daily communication habits. This step greatly enriches the system's vocabulary and improves its ability to handle diverse expressions, thereby achieving a more flexible and natural voice interaction experience.

[0088] In one embodiment, after the step of inserting language modifier elements at specified positions in the semantic structures of the generalized term and the reference term corresponding to the reference speech instruction, the method includes:

[0089] S50. Based on a preset syntax rule library, perform sentence conversion on the generalized terms and the benchmark terms corresponding to the benchmark speech commands;

[0090] S51. By adding time, location, or conditional adverbial components, the generalized term and the reference term corresponding to the reference speech instruction are compounded and expanded;

[0091] S52. Adjust the order of sentence components according to the preset syntax rule library.

[0092] In this embodiment, firstly, the system uses a pre-defined syntactic rule base to perform sentence transformation on existing generalized and baseline terms. For example, "turn on the air conditioner" can be transformed into the active sentence "turn on the air conditioner" or the passive sentence "let the air conditioner be turned on". Furthermore, the system can also perform compound expansion by adding time, location, or conditional adverbs, such as "turn on the air conditioner now" or "turn on the air conditioner in the living room". This expansion not only increases the diversity of terms but also allows the system to better understand the user's contextual information. Finally, the system adjusts the order of sentence components according to the syntactic rule base, for example, changing "turn on the air conditioner" to "the air conditioner was turned on". These operations work together to enable the system to generate a large number of variant terms with different structures but consistent semantics, greatly improving the system's vocabulary coverage and recognition accuracy. At the same time, this also enhances the system's ability to understand diverse user expressions, making voice interaction more natural and fluent.

[0093] In one embodiment, the step of injecting the variant word into the lexicon of the offline speech recognition system includes:

[0094] S60. Standardize the format and convert the encoding of the variant entries, and eliminate duplicate entries in the thesaurus;

[0095] S61. Load the processed variant word increments into the word index of the offline speech recognition engine.

[0096] In this embodiment, firstly, the system performs format standardization and encoding conversion on all newly generated variant entries to ensure they are consistent with the format of entries in the existing dictionary, avoiding recognition errors caused by format differences. For example, some entries may contain special characters or non-standard punctuation marks, which need to be standardized. Next, to avoid redundant entries in the dictionary occupying unnecessary storage space and reducing retrieval efficiency, the system uses a hash algorithm to calculate a unique identifier for each entry and compares it with existing entries in the dictionary to quickly detect and remove duplicates. For example, when generating a variant of "turn on the air conditioner," if the entry already exists in the dictionary, it does not need to be added again. After the above processing is completed, the variant entries are updated to the dictionary index of the offline speech recognition engine in an incremental manner. This means that the new entries do not replace the existing dictionary content, but are added incrementally as a supplement. This method not only reduces the performance overhead of full updates, but also ensures that the dictionary is always up-to-date and ready to respond to diverse user commands. In this way, the system can efficiently manage the dictionary, ensuring its streamlined and efficient operation.

[0097] In one embodiment, the step of recognizing and executing a user-input voice command using the dictionary includes:

[0098] S70 uses a lightweight noise reduction model to filter out environmental noise in real time and extract pure human voice signals from voice commands.

[0099] S71. Determine the start and end positions of the voice command based on the time sequence analysis model, and extract the acoustic features of the voice command.

[0100] s72. Match the acoustic features with all entries in the dictionary and execute the control commands corresponding to the matched entries.

[0101] In this embodiment, the system employs a lightweight noise reduction model (such as RNNoise) to filter out environmental noise in real time, ensuring the extraction of pure human voice signals. This is crucial for accurately capturing user voice commands in noisy environments. For example, in a home environment with television sound or other background noise, the system can effectively remove these interfering factors and retain clear voice information. Next, the start and end positions of the voice command are determined based on a temporal analysis model (such as a bidirectional LSTM model), and the acoustic features of the voice command are extracted from it. The key to this step is to accurately capture the complete voice command and avoid recognition errors caused by truncation. For example, if the sentence "Turn on the air conditioner" is mistakenly truncated as "Turn on the air conditioner," the user's intention cannot be correctly understood. Therefore, accurate determination of the start and end points is very important for improving recognition accuracy. Finally, the system matches the extracted acoustic features with all entries in the vocabulary. Since a large number of semantically equivalent variant entries have been injected into the vocabulary in the previous steps, the system can now recognize various different expressions. For example, in addition to the standard "Turn on the air conditioner," the vocabulary also includes variations such as "Turn on the air conditioner" and "Please start the air conditioner." Once the user's actual voice command matches one of the variants, the system executes the corresponding control command. This method not only improves recognition accuracy but also enhances the system's adaptability and user experience, making voice interaction more natural and fluid.

[0102] Reference Figure 2 This is a block diagram of an offline speech recognition vocabulary expansion system according to an embodiment of this application. The system includes:

[0103] The parsing module 100 is used to perform offline semantic parsing based on the benchmark speech commands in the lexicon to obtain semantic structure data;

[0104] The generalization processing module 200 is used to perform multimodal term generalization processing based on the semantic structure data to generate semantically equivalent variant terms;

[0105] The injection lexicon module 300 is used to inject the variant entries into the lexicon of the offline speech recognition system.

[0106] The instruction execution module 400 is used to recognize and execute instructions through the dictionary when a user inputs a voice instruction.

[0107] In one embodiment, the system further includes an accent adaptation module, which includes:

[0108] The feature library construction unit is used to build an accent feature library containing pronunciation variation rules and dialect word features based on the lexicon;

[0109] The pronunciation mapping unit is used to map the standard words corresponding to the reference speech command through the accent feature library to generate variant words that conform to the regional pronunciation habits.

[0110] The dialect replacement unit is used to perform regional word replacement on the variant entries based on the dialect word features in the accent feature library, and output the final accent-adapted variant entries.

[0111] In one embodiment, the semantic parsing module 100 includes:

[0112] The semantic analysis unit is used to perform semantic analysis on the baseline speech commands in the lexicon based on the offline NLP model;

[0113] Part-of-speech segmentation unit is used to segment the baseline speech command into parts of speech based on semantic analysis results, forming the smallest semantic units with different parts of speech;

[0114] Part-of-speech tagging units are used to tag each smallest semantic unit (vocabulary) with part-of-speech tags to obtain semantic structure data.

[0115] In one embodiment, the generalization processing module 200 includes:

[0116] The synonym substitution unit is used to perform synonym substitution on phrases of different parts of speech contained in semantic structure data based on natural language processing models, and generate generalized entries.

[0117] The modification insertion unit is used to insert language modification elements at specified positions in the semantic structures of the generalized term and the reference term corresponding to the reference speech instruction, based on the semantic structure of the generalized term and the reference term, to obtain semantically equivalent variant terms.

[0118] In one embodiment, the generalization processing module 200 further includes:

[0119] The sentence conversion unit is used to perform sentence conversion on the generalized terms and the reference terms corresponding to the reference speech commands based on a preset syntax rule library;

[0120] The composite expansion unit is used to composite expand the generalized term and the reference term corresponding to the reference speech instruction by adding time, location or conditional adverbial components;

[0121] A word order adjustment unit is used to adjust the order of sentence components according to a preset syntactic rule base. In one embodiment, the injected lexicon module 300 includes:

[0122] The conversion and deduplication processing unit is used to unify the format and convert the encoding of the variant entries, and eliminate duplicate entries in the thesaurus;

[0123] The incremental loading unit is used to incrementally load the processed variant words into the dictionary index of the offline speech recognition engine.

[0124] In one embodiment, the instruction execution module 400 includes:

[0125] The noise reduction unit is used to filter out environmental noise in real time through a lightweight noise reduction model and extract pure human voice signals from voice commands.

[0126] The timing analysis unit is used to determine the start and end positions of voice commands based on the timing analysis model and to extract the acoustic features of the voice commands.

[0127] The matching execution unit is used to match acoustic features with all entries contained in the dictionary and execute the control instructions corresponding to the matched entries.

[0128] Reference Figure 3 This application also provides a computer device, which may be a server, and its internal structure may be as follows: Figure 3As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor is designed to provide computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores usage data during the offline speech recognition term expansion method. The network interface is used to communicate with external terminals via a network connection. Furthermore, the computer device may also include an input device and a display screen. When the computer program is executed by the processor to implement the offline speech recognition term expansion method, it includes the following steps: performing offline semantic parsing based on baseline speech commands in the dictionary to obtain semantic structure data; performing multimodal term generalization processing based on the semantic structure data to generate semantically equivalent variant terms; injecting the variant terms into the dictionary of the offline speech recognition system; and recognizing and executing commands through the dictionary when user input voice commands are recognized. Those skilled in the art will understand that... Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer equipment on which the present application is applied.

[0129] One embodiment of this application also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements an offline speech recognition vocabulary expansion method, including the following steps: performing offline semantic parsing based on baseline speech commands in a vocabulary library to obtain semantic structure data; performing multimodal vocabulary generalization processing based on the semantic structure data to generate semantically equivalent variant vocabulary; injecting the variant vocabulary into the vocabulary library of an offline speech recognition system; and recognizing and executing commands through the vocabulary library when a user inputs a speech command. It is understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0130] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media provided in this application and in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0131] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0132] The above description is only a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An offline speech recognition vocabulary expansion method, characterized in that, include: It receives one or more baseline voice command inputs from a lexicon, and performs deep semantic understanding on the baseline voice commands using a lightweight natural language processing model optimized for offline environments. It sequentially performs intent recognition, key information extraction and segmentation, part-of-speech tagging and semantic role tagging. After text preprocessing and word segmentation, it obtains semantic structure data containing the smallest semantic unit part-of-speech and semantic role tagging. The text preprocessing is to remove stop words. Based on the semantic structure data, multimodal term generalization processing is performed to generate semantically equivalent variant terms; Inject the variant terms into the dictionary of the offline speech recognition system; When a user inputs a voice command, the command is recognized and executed using the aforementioned dictionary. Before the step of injecting the variant words into the dictionary of the offline speech recognition system, the following steps are included: Based on the lexicon, an accent feature library is constructed, which includes modular regional sub-libraries. Each regional sub-library contains a pronunciation variation rule library and a dialect vocabulary mapping table. The pronunciation variation rule library includes regional pronunciation variation rules, and the dialect vocabulary mapping table records the correspondence between dialect-specific vocabulary and standard vocabulary. The standard words corresponding to the baseline speech command are mapped to the accent feature library by phoneme alignment algorithm to generate variant words that conform to the regional pronunciation habits. Based on the dialect vocabulary mapping table in the accent feature database, the variant entries are replaced with regional words, and the replaced variant entries are subjected to semantic consistency verification and grammatical error screening. Variants with semantic inequivalence or grammatical errors are eliminated, and the final accent-adapted variant entries are output.

2. The offline speech recognition vocabulary expansion method according to claim 1, characterized in that, The step of performing offline semantic parsing based on benchmark speech commands in the lexicon to obtain semantic structure data includes: Semantic analysis of baseline speech commands in the lexicon is performed based on an offline NLP model; Based on the semantic analysis results, the baseline speech command is part-of-speech segmented to form the smallest semantic units with different parts of speech; Part-of-speech tagging is performed on each smallest semantic unit word to obtain semantic structure data.

3. The offline speech recognition vocabulary expansion method according to claim 1, characterized in that, The step of performing multimodal term generalization processing based on the semantic structure data to generate semantically equivalent variant terms includes: Based on a natural language processing model, synonym replacement is performed on phrases of different parts of speech contained in semantic structure data to generate generalized vocabulary entries; Based on the semantic structure of the generalized term and the reference term corresponding to the reference speech instruction, language modification elements are inserted at specified positions in the semantic structure of the generalized term and the reference term to obtain semantically equivalent variant terms.

4. The offline speech recognition vocabulary expansion method according to claim 3, characterized in that, After the step of inserting language modifier elements at specified positions in the semantic structures of the generalized term and the reference term corresponding to the reference speech instruction, the following steps are included: Based on a preset syntax rule library, sentence transformation is performed on the generalized terms and the benchmark terms corresponding to the benchmark speech commands; By adding time, location, or conditional adverbs, the generalized terms and the reference terms corresponding to the reference speech commands are compounded and expanded. The order of sentence components is adjusted based on a pre-defined syntactic rule base.

5. The offline speech recognition vocabulary expansion method according to claim 1, characterized in that, The step of injecting the variant words into the dictionary of the offline speech recognition system includes: The variant terms are formatted and encoded, duplicate terms in the dictionary are eliminated, and the processed variant terms are incrementally loaded into the dictionary index of the offline speech recognition engine.

6. The offline speech recognition vocabulary expansion method according to claim 1, characterized in that, The step of recognizing and executing user-input voice commands using the dictionary includes: A lightweight noise reduction model is used to filter out environmental noise in real time and extract pure human voice signals from voice commands. The start and end positions of voice commands are determined based on a time-series analysis model, and the acoustic features of the voice commands are extracted. The acoustic features are matched against all entries in the dictionary, and the control commands corresponding to the matched entries are executed.

7. An offline speech recognition vocabulary expansion system, used to implement the method described in any one of claims 1-6, characterized in that, include: The parsing module is used to perform offline semantic parsing based on benchmark speech commands in the lexicon to obtain semantic structure data; The generalization processing module is used to perform multimodal term generalization processing based on the semantic structure data to generate semantically equivalent variant terms; The injection lexicon module is used to inject the variant entries into the lexicon of the offline speech recognition system; The instruction execution module is used to recognize and execute user-input voice commands using the dictionary.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images