Engineering context-based offline speech recognition dynamic hotword injection method

Abstract

The present application relates to the technical field of speech recognition, in particular to an offline speech recognition dynamic hotword injection method based on engineering context. The method comprises: in response to a voice input trigger instruction, obtaining a working directory path of a current development environment and extracting engineering context features; performing engineering word segmentation processing on the features to generate a candidate hotword set; sorting and intercepting according to a preset weight rule to construct a dynamic prompt word sequence; injecting the sequence into a decoding stage of a local offline speech recognition model to guide the model to recognize audio data to be recognized; and outputting a recognition result to a downstream application module. The present application dynamically generates hotwords by real-time sensing of software engineering state and injects guidance in the decoding stage, without model fine-tuning or cloud service, thereby improving the recognition accuracy of engineering-specific vocabulary and realizing local offline processing to protect data privacy.

Description

Technical Field

[0001] This invention relates to the field of speech recognition technology, and in particular to an offline speech recognition dynamic hot word injection method based on engineering context. Background Technology

[0002] Speech recognition technology has been widely applied in human-computer interaction scenarios. In software development environments, developers hope to improve work efficiency by inputting code editing instructions, command-line operation instructions, or development assistance instructions via voice. However, existing speech recognition solutions still have the following technical problems in the specialized scenario of software engineering.

[0003] 1. General offline speech recognition models have low accuracy in recognizing engineering-specific vocabulary.

[0004] Current mainstream offline speech recognition models are typically trained on general domain corpora such as news and everyday conversations, lacking targeted modeling for software engineering-specific vocabulary. Frequently used terms in development scenarios, such as framework names like "SpringBoot," technical abbreviations like "JWT" ​​and "API," project codenames, branch names like "bugfix / jwt-authentication," file names like "auth_middleware.py," function names, and variable names, are often not in the vocabulary of general models or have similar pronunciations, leading to errors in word segmentation and homophone substitution. For example, "JWT" ​​might be recognized as "jay double u" or similar-sounding common words, and "auth" might be recognized as "office" or "off." These errors make the speech recognition results unsuitable for direct use in code editing, command execution, or development tool invocation.

[0005] 2. Cloud-based customized keyword solutions pose privacy and compliance risks.

[0006] To improve the accuracy of speech recognition in specialized scenarios, some speech recognition services allow users to upload custom word lists or hot word lists, which are then dynamically biased using cloud-based language models. However, this approach requires uploading sensitive information such as audio streams, project names, code terminology, and project context to external servers. For local development environments, corporate intranet environments, or confidential R&D scenarios involving trade secrets or personal privacy, external data transmission poses a risk of information leakage and may violate internal security protocols or relevant laws and regulations. Therefore, many development scenarios require speech recognition capabilities to operate entirely offline locally, making reliance on cloud-based hot word customization services unacceptable.

[0007] 3. Model fine-tuning methods are costly and difficult to dynamically adapt.

[0008] Another method to improve domain recognition accuracy is to fine-tune the offline speech recognition model, which involves additionally training the model using domain-annotated corpora. However, this approach requires a large amount of high-quality speech-text pairing corpora from the engineering domain, as well as iterative model training, validation, and deployment, resulting in high implementation costs. Furthermore, software engineering scenarios are highly dynamic: developers frequently switch branches, create new files, and modify function names, and specialized terms related to the current task, such as the current branch name and filename, may only be highly relevant for a few minutes or hours. Static fine-tuning models cannot be updated in real time with the engineering status and struggle to adapt to the linguistic features of the current development task.

[0009] 4. Existing local thesaurus schemes lack dynamic correlation with the current project status.

[0010] Some offline recognition solutions allow users to pre-configure static vocabularies or domain-specific dictionaries to improve the recognition probability of specific words. However, the hot words in these solutions are fixed and cannot be automatically adjusted based on dynamic contexts such as the developer's current project, the current version control branch, or files edited within a preset time window prior to the current point in time. Therefore, static vocabularies have limited support for task-related and temporarily generated proprietary terms such as newly created branch names and temporary variable names, and still struggle to meet the needs of real-time changing development scenarios.

[0011] 5. Limitations of existing context-based keyword technology

[0012] Furthermore, several technologies have proposed context-based hotword activation schemes. For example, patent CN116052661B discloses a "context-based hotword" method that activates or deactivates predefined hotwords such as "next song" or "stop" based on the context of the application running on the computing device, such as a music player, device location, time, or motion status. When the user speaks these hotwords, the corresponding operation is executed directly. This method can reduce the number of times the user repeatedly speaks the wake word, improving the convenience of interaction in general consumer electronics scenarios.

[0013] However, in software engineering applications, the above solution has the following limitations: the contextual information it relies on comes from device status, location, and movement, rather than version control information or file system dynamics specific to software engineering; and the hot words it processes are limited, predefined functional command words, rather than dynamically generated, unpredictable engineering identifiers such as branch names, file names, and function names. Therefore, this solution cannot solve the problem of low accuracy in recognizing proprietary terms in software engineering scenarios.

[0014] In summary, the existing technology still has the following problems: in a local offline environment, it is difficult to dynamically generate recognition hot words based on the current software engineering status, and it is impossible to effectively guide the offline speech recognition decoding process without modifying the underlying model parameters, so as to improve the recognition accuracy of proprietary words in engineering scenarios. Summary of the Invention

[0015] To address the aforementioned technical problems, the technical solution adopted by this invention is as follows:

[0016] This invention provides a method for dynamic hot word injection in offline speech recognition based on engineering context, the method comprising the following steps:

[0017] In response to a voice input trigger command, the working directory path of the current development environment is obtained, and project context features are extracted based on the working directory path; the project context features include the branch name or tag name parsed from the version control system, and / or the file name currently being edited or within a preset time window before the current time point parsed from the file system.

[0018] The extracted engineering context features are processed by engineering-based word segmentation to generate dynamic candidate hot word units; the dynamic candidate hot word units are fused with the static candidate hot word units in the static domain vocabulary set to generate a candidate hot word set; the engineering-based word segmentation process includes: according to the naming convention of engineering identifiers, the composite units in the engineering context features are segmented into the smallest units that constitute the composite units.

[0019] The lexical units in the candidate hot word set are sorted according to a preset weighting rule, and at least a portion of the hot words are selected from the sorted candidate hot word set to construct a dynamic prompt word sequence according to the prompt length limit of the target offline speech recognition model.

[0020] The dynamic prompt word sequence is used as a priori context and injected into the decoding stage of the locally running target offline speech recognition model to guide the target offline speech recognition model to recognize the audio data to be recognized.

[0021] The system acquires the audio data to be recognized input by the user, outputs the text recognition result through an offline speech recognition model that has been injected with the dynamic prompt word sequence, and passes the text recognition result to the downstream application module.

[0022] This invention dynamically generates a sequence of hot words highly relevant to the current development task by real-time sensing of the current software engineering status, including version control branches and file editing within a preset time window prior to the current point in time. This sequence is then injected as prior context during the offline speech recognition decoding stage. This improves the recognition accuracy of project-specific terms, such as "jwt" in the branch name "jwt-authentication" and "auth" in the filename "auth_middleware.py," without requiring retraining or fine-tuning the underlying speech recognition model and maintaining the original model weights. Furthermore, the entire process is completed on a local computing device, without transmitting any audio or project information to external servers, thus balancing data privacy and compliance requirements.

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A flowchart of an offline speech recognition dynamic hot word injection method based on engineering context provided in an embodiment of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0028] It should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. A process can be terminated when its operation is complete, but it may also have additional steps not included in the figures. A process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0029] First Embodiment

[0030] Existing offline speech recognition technology has at least the following technical shortcomings in software engineering applications:

[0031] 1. Insufficient accuracy in recognizing engineering-specific terms.

[0032] General offline speech recognition models are trained on general domain corpora and lack targeted modeling for software engineering-specific vocabulary (such as framework names, technical abbreviations, project codes, branch names, file names, function names, variable names, etc.). This makes them prone to misidentification, splitting errors, or homophone substitutions, resulting in recognition results that are difficult to use directly for code editing or command execution.

[0033] 2. Static vocabulary cannot adapt to real-time changes in project status.

[0034] Existing local thesaurus or dictionary solutions use fixed or pre-defined sets of hot words, which cannot automatically adjust based on the dynamic project context, such as the developer's current project, version control branch, or files edited within a preset time window prior to the current time. This results in limited support for temporary proprietary vocabulary related to the current task. The preset time window refers to a time period (e.g., 5 minutes, 10 minutes, or 30 minutes) configured by the system or set by the user, used to limit the time range of "editing within a preset time window prior to the current time." This time window length can be dynamically adjusted according to the development scenario. 3. Model fine-tuning methods suffer from high adaptation costs and delayed updates.

[0035] Improving recognition accuracy through domain fine-tuning requires preparing a large amount of labeled data and conducting model training and iterative deployment, which is costly. At the same time, the software engineering context is highly dynamic, and static fine-tuning models are difficult to adapt to real-time changes in branch names, file names, and other engineering identifiers.

[0036] 4. Cloud-based hot keyword customization solutions pose a risk of data leakage.

[0037] The hot word customization solution that relies on cloud interfaces requires uploading sensitive information such as audio streams, project names, and code terms to external servers, which poses risks of information leakage and compliance in local development environments, corporate intranets, or confidential R&D scenarios.

[0038] To address the aforementioned shortcomings, this application provides a dynamic hotword injection method for offline speech recognition based on engineering context. This method aims to solve the following technical problem: how to dynamically generate recognition hotwords based on the current software engineering state in a local offline environment, and effectively guide the offline speech recognition decoding process without retraining or fine-tuning the underlying speech recognition model (i.e., keeping the original model weights unchanged), thereby improving the recognition accuracy of proprietary vocabulary in software engineering scenarios, while also considering local deployment and data privacy requirements.

[0039] To facilitate the description of the technical solution of this application, the relevant terms are defined as follows:

[0040] "Software engineering scenario" specifically refers to professional activity scenarios related to computer software development, maintenance, debugging, and deployment. In this scenario, users (usually developers) issue commands or inquiries to computing devices via voice input, related to code editing, command-line operations, version control management, build execution, debugging instructions, or development assistance queries. The software engineering scenario differs from general daily life or office scenarios in that its prominent feature is the frequent inclusion of engineering identifiers with specific technical meanings in voice commands, such as: framework names, technical abbreviations, project codes, version control branch names, tag names, file names, directory names, function names, class names, variable names, configuration item names, etc. These engineering identifiers are often not within the vocabulary of general speech recognition models, or their combinations and pronunciation characteristics differ significantly from everyday language.

[0041] "Hot words" or "engineering context hot words" refer to the smallest units of words automatically generated through engineering word segmentation based on the version control status and / or file system status of the current software engineering scenario. These units are used as prior context in the offline speech recognition decoding stage to guide the offline speech recognition model to accurately identify specific engineering identifiers.

[0042] like Figure 1 As shown in the figure, the offline speech recognition dynamic hot word injection method based on engineering context provided in this embodiment includes the following steps:

[0043] S100, in response to a voice input trigger command, obtains the working directory path of the current development environment, and extracts project context features based on the working directory path. Specifically, this includes the following sub-steps:

[0044] S101, voice triggering and environmental awareness.

[0045] In response to user-initiated voice input commands, such as pressing a specified shortcut key, clicking the voice input button in the IDE, or speaking a preset wake word, the system retrieves the working directory path corresponding to the currently active terminal, command line window, or integrated development environment (IDE).

[0046] If the directory is not the project root directory and does not contain version control metadata (e.g., a .git folder), it automatically recursively searches upwards to locate the nearest parent directory containing version control metadata as the project root directory, which is then used for subsequent context feature extraction. A context-aware process associated with this project root directory is then established.

[0047] Specifically, during the execution of this sensing process, the following three functional units are initialized:

[0048] File system monitoring unit: Used to monitor file changes in the current working directory and its subdirectories, including file creation, deletion, renaming, content modification, and other events, as well as record the most recent access time and modification time of the files.

[0049] Version control parsing unit: Used to read the status information of the version control system (such as Git) in the current project directory, including the current branch name, tag name, commit hash, working directory identifier, etc.

[0050] Context information extraction unit: used to extract engineering context features from the raw information obtained by the above units and convert them into structured data that can be used for subsequent word segmentation processing.

[0051] Through the collaboration of the three units mentioned above, the current development environment can be perceived in real time before or during speech recognition, providing an accurate contextual basis for dynamically generating hot words.

[0052] S102, Engineering Context Feature Extraction.

[0053] Extract project context features related to the current development task, based on the current working directory. These project context features include at least one or more of the following categories:

[0054] (1) Version control context features

[0055] The version control parsing unit reads the version control status information in the current project directory and parses at least one of the following: the current branch name (e.g., bugfix / jwt-authentication), the tag name (e.g., v2.0.1), the commit identifier (e.g., a3f2c9e), or the working directory identifier. Preliminary cleaning of the branch name or identifier name, such as removing the `origin / ` prefix, can be performed for subsequent word segmentation processing.

[0056] (2) File system context features

[0057] The file system listening unit acquires at least one of the following information: the project root directory name, such as nano-claude-code; the name of the currently active file in the editor, such as auth_middleware.py; the names of recently accessed files, such as files opened in the past 5 minutes; and the names of recently modified files, such as files in the most recent Git modification log. From this information, directory terms, file noun terms, and extension-related terms (such as core terms after removing extensions) are extracted.

[0058] (3) Extended project context features

[0059] To further enrich the sources of hot words, one or more of the following additional features can be extracted:

[0060] The module name or dependency library name in project configuration files such as package.json and requirements.txt;

[0061] The task name in a build script such as Makefile, build.gradle, or a test script such as test_*.py;

[0062] The file name in the editor's active tab;

[0063] Task lists or to-do items, such as TODO.md, or keywords in the IDE's built-in task list.

[0064] Furthermore, code symbol information can be extracted from the currently active code file through abstract syntax tree analysis. Specifically, based on the programming language type of the currently active code file, such as Python, Java, or JavaScript, the corresponding compiler front-end or parser, such as Python's ast module, JavaCompiler for Java, or esprima for JavaScript, is invoked to parse the source code into an abstract syntax tree. Then, the syntax tree nodes are traversed to extract at least one symbol name from the following: class declaration, function declaration, method declaration, variable declaration, interface declaration, or enum declaration. The extracted symbol names serve as one of the project context features and are used for subsequent word segmentation and hot word generation.

[0065] In a scenario where developers are fixing defects in the authentication module, the current active window's project working directory is named nano-claude-code, the current Git branch is named bugfix / jwt-authentication, and the files modified within the preset time window prior to the current time include auth_middleware.py. Extended contexts such as requirements.txt contain PyJWT dependencies, and the Makefile contains the test-auth task. This information is structured and stored for engineering-level word segmentation processing in step S200.

[0066] S200, perform engineering-style word segmentation on the extracted engineering context features to generate dynamic candidate hot word units; fuse the dynamic candidate hot word units with the static candidate hot word units in the static domain vocabulary set to generate a candidate hot word set. The engineering-style word segmentation includes: segmenting the composite units in the engineering context features into the smallest units constituting the composite units according to the naming convention of the engineering identifier.

[0067] Specifically, the naming conventions for the project identifiers include one or more of the following:

[0068] Hyphen segmentation: Identify the hyphen "-" as a word boundary, for example, split jwt-authentication into jwt and authentication;

[0069] Underscore segmentation: Identify underscores "_" as word boundaries, for example, split auth_middleware into auth and middleware;

[0070] CamelCase naming splitting: Identifies uppercase letters as boundaries and splits camelCase names into independent words. For example, AuthMiddleware is split into Auth and Middleware, and parseUserInput is split into parse, User and Input.

[0071] Number-letter boundary splitting: Identify the boundaries between numbers and letters, for example, splitting v2 into v and 2;

[0072] File extension stripping: For file names, first strip the extensions such as .py, .java, .js, and then perform the above splitting on the core part of the file name.

[0073] In practice, the above word segmentation rules can be executed by combining string traversal with character type judgment. For an input compound word string, the following steps are performed sequentially:

[0074] Extension stripping: If the string ends with a file extension, first identify the position of the last dot, strip the extension part, and keep only the core part of the file name for subsequent word segmentation.

[0075] Boundary recognition and segmentation: A single traversal is performed on the string after extension stripping, determining the type of each character, including lowercase letters, uppercase letters, numbers, hyphens, underscores, etc. A lexical boundary is marked when the types of adjacent characters change in the following ways:

[0076] Jump from a hyphen or underscore to any letter or number;

[0077] Jump from lowercase to uppercase (camel case boundary).

[0078] Jump from letters to numbers, or from numbers to letters (number-letter boundary).

[0079] At the end of a continuous sequence of uppercase letters, such as between "HTTP" and "Server" in "HTTPServer".

[0080] Lexical generation: Based on the boundary positions of the markers, the string after extension stripping is segmented to extract individual lexical units. All extracted lexical units are converted to lowercase (or retain the original case, depending on the model requirements), and empty strings and invalid lexical units with a length less than a preset threshold, such as 1 character, are removed to obtain dynamic candidate hot word lexical units.

[0081] The above rules can be pre-compiled into a set of regular expressions or implemented as a deterministic finite state automaton to improve word segmentation efficiency. For compound terms containing multiple naming conventions, such as bugfix / jwt-authentication, they can be first split into multiple segments according to path separators (such as / ), and then the above word segmentation rules can be applied to each segment independently. In this embodiment, a static candidate term set is formed by loading a preset vocabulary from a locally pre-built static vocabulary. The preset vocabulary refers to common terms in the software engineering field that are pre-stored in the local vocabulary. The static vocabulary includes the following four categories:

[0082] Programming Glossary: ​​Contains commonly used programming keywords, such as python, grep, and refactor;

[0083] Framework Terminology: Includes the names of development frameworks, such as Django, Flask, and Spring;

[0084] Tool Terminology: Includes development tool commands such as git, make, and docker;

[0085] Business Area Terminology: Contains professional terms defined according to the actual business of the project.

[0086] The preset words loaded from the four types of vocabulary lists mentioned above are deduplicated and normalized (e.g., uniformly converted to lowercase) to form a static candidate hot word lexical set. The lexical units in this static candidate hot word lexical set are used to supplement the contextual lexical units dynamically extracted in the previous steps, ensuring that the offline speech recognition model can cover commonly used engineering vocabulary and avoiding recognition errors caused by omissions of general terms due to dynamic extraction. The dynamic candidate hot word lexical units obtained from dynamic word segmentation are merged with the static candidate hot word lexical units loaded from the static vocabulary lists. Since each source lexical unit has undergone denoising, normalization, and basic filtering during the generation process, only global deduplication is needed after merging to generate the candidate hot word set. This candidate hot word set consists of dynamically extracted version control lexical units, file system lexical units, and static vocabulary lexical units, which are used in step S300 to construct the dynamic prompt word sequence.

[0087] For branch names or tag names parsed from the version control system, such as bugfix / jwt-authentication or v2.0.1, the following word segmentation optimization operations based on acoustic features can be performed to further improve word segmentation accuracy:

[0088] S201, obtain the speech sample corresponding to the branch name or tag name.

[0089] The voice samples are obtained in the following way:

[0090] Historical voice input samples: If the user has spoken the branch name or tag name in the current session or in a previous session, the corresponding recording segment is retrieved from the local cache or logs as a historical voice input sample. This historical voice input sample can realistically reflect the user's pronunciation habits and speaking speed.

[0091] Synthesized speech samples: If no historical speech input samples exist, a speech sample with the given name is synthesized using a Text-to-Speech (TTS) engine. Specifically, based on the string content of the branch name or tag name, a local or offline TTS module, such as eSpeak, Festival, or a deep learning-based TTS model, is invoked to generate the corresponding audio data. The speech rate and pitch of the synthesized speech can be set to default values.

[0092] If multiple samples exist, historical speech input samples that are closer to the actual pronunciation can be used first, or features can be extracted from multiple samples separately and then fused.

[0093] In this application, "user" refers to a developer or other operator involved in software development activities who uses this method in a software engineering context. The user issues commands or queries related to code editing, command-line operations, version control management, build execution, debugging instructions, or development assistance queries to the computing device via voice input.

[0094] S202, extract the acoustic features of the acquired speech samples, and identify silence segments, pitch transition points, or energy transition points in the acoustic features as potential word boundaries. The specific steps are as follows:

[0095] Feature extraction: Segment the speech sample into fixed-length frames, for example, each frame is 25 milliseconds long, with a frame shift of 10 milliseconds. Extract one or more of the following acoustic features from each frame:

[0096] Short-term energy: reflects the intensity changes of the speech signal;

[0097] Fundamental Frequency (F0): Reflects changes in pitch;

[0098] Mel frequency cepstral coefficients (MFCC) or other spectral features.

[0099] Boundary detection: Based on the time series of the above features, the following acoustic events are identified, and the location of each acoustic event (represented by frame number or time point) is marked as a latent vocabulary boundary:

[0100] Silent segment boundary: Calculate the short-time energy of each frame. When the short-time energy of multiple consecutive frames is lower than a first preset threshold (e.g., 0.05 times the maximum energy), and the duration of the consecutive segment exceeds a second preset threshold (e.g., 100 milliseconds), the time points corresponding to the start and end frames of the consecutive segment are marked as potential vocabulary boundaries. Silent segments typically correspond to pauses between words.

[0101] Pitch transition point: Calculate the fundamental frequency difference between adjacent frames. When the absolute value of this fundamental frequency difference exceeds a third preset threshold, such as 5 semitones, the position between the adjacent frames is marked as a potential word boundary. Pitch transitions typically correspond to changes in tone or accent positions.

[0102] Energy jump point: Calculate the short-term energy change rate between adjacent frames. When the change rate exceeds a fourth preset threshold, for example, when the energy ratio between adjacent frames is greater than 3 times, the position between those adjacent frames is marked as a potential lexical boundary. Energy jumps typically correspond to transitions from consonants to vowels or phoneme boundaries.

[0103] It should be noted that in continuous audio signals, a boundary time point refers to a specific moment on the audio timeline that is determined to be a word boundary, such as 1.25 seconds. This time point corresponds to the position between two characters in a compound word string, such as jwt|authentication. S203, the potential word boundaries, i.e., a set of time points, are fused with the naming convention of the engineering identifier to determine the final word segmentation position. The fusion rules are as follows:

[0104] Mapping from time points to character indices: A forced alignment technique is used to map the time point corresponding to each latent word boundary to the character index position of the branch name or label name string, determining which two adjacent characters the time point falls between. Specifically, phoneme alignment techniques, such as forced alignment based on a Hidden Markov Model, are used to determine the time range corresponding to each character in the speech sample, thereby determining which two adjacent characters each time point falls between. For example, if time point t falls between the end of the character "jwt" and the beginning of the character "authentication", then this time point is mapped to the index position between the 3rd and 4th characters in the string.

[0105] Calculate the split confidence score: For any two adjacent characters in the string, i.e., the potential split position, calculate the fusion score.

[0106] Naming convention confidence: If the position conforms to the naming convention rules, such as the presence of hyphens, underscores, or uppercase / lowercase boundaries, or alphanumeric boundaries, then a first preset value, such as 0.9, is assigned; otherwise, a second preset value, such as 0.1, is assigned.

[0107] Acoustic boundary confidence: If a potential word boundary is detected near the location, such as a boundary marker within ±50 milliseconds of the corresponding time point, a comprehensive score is given based on the intensity of the boundary type, and the comprehensive score is normalized to between 0 and 1; otherwise, a third preset value, such as 0.1, is assigned.

[0108] Fusion score = naming convention confidence × α + acoustic boundary confidence × β, where α and β are preset weights, for example α=0.6, β=0.4, which can be adjusted according to the actual effect.

[0109] Determine the word segmentation position: The position where the fusion score exceeds the fifth preset threshold, such as 0.7, is determined as the final word segmentation position. Segmentation is performed at the corresponding position to obtain the final word segmentation result.

[0110] By integrating the naming conventions of engineering identifiers with real-world acoustic features, the segmentation boundaries of compound words can be determined more accurately. For example, in the branch name bugfix / jwt-authentication, the naming convention suggests segmentation at the hyphen; while acoustic boundary detection may further verify that the user's actual pronunciation includes a slight pause or pitch change between jwt and authentication, thus confirming the correctness of the segmentation position. This step is particularly suitable for handling segmentation ambiguities in non-standard names, such as combinations of the pure uppercase abbreviation sequence JWT and ordinary words.

[0111] It should be noted that this step is an optional optimization and can be enabled or disabled depending on the actual application scenario. If this step is not enabled, word segmentation will only be performed based on the naming convention of the project identifier.

[0112] Furthermore, the candidate hot word set is not limited to the sources mentioned above. In other embodiments, in addition to version control information and file system information, additional engineering context features can be extracted from one or more of the following sources to generate the candidate hot word set, thereby adapting to more complex software engineering scenarios:

[0113] The module name or dependency library name in the project configuration file such as package.json;

[0114] The name of the task in the build script (such as Makefile) or test script;

[0115] The file name in the editor's active tab;

[0116] Task lists or to-do items, such as keywords in TODO.md;

[0117] The abstract syntax tree (API) analyzes the class names, function names, variable names, interface names, or enumeration names extracted from the currently active code file.

[0118] The function name, variable name, module name, or interface name obtained from the code completion service in the local development environment.

[0119] S300: The lexical units in the candidate hot word set are sorted according to a preset weighting rule, and at least a portion of the hot words are selected from the sorted candidate hot word set to construct a dynamic prompt word sequence based on the prompt length limit of the target offline speech recognition model. Specifically, this includes the following steps:

[0120] S301. According to the preset weighting rules, calculate the weight score for each word element in the candidate hot word set, and sort them from high to low to obtain the sorted candidate hot word set.

[0121] The preset weighting rule is determined based on one or more of the following factors:

[0122] Lexical source priority: Lexical sources have different basic weights. Specifically, lexical sources parsed from version control systems, such as branch name fragments and tag name fragments, have the first priority basic weight; lexical sources parsed from file systems, such as filename core words and directory names, have the second priority basic weight; and lexical sources loaded from static vocabularies have the third priority basic weight. The weight value corresponding to this factor is called the source priority weight.

[0123] The first priority has a higher weight than the second priority, and the second priority has a higher weight than the third priority. For example, the JWT parsed from the branch name bugfix / jwt-authentication has a higher weight than the auth parsed from the file name auth_middleware.py.

[0124] The most recent modification timestamp of the file corresponding to a term: For terms parsed from the file system, obtain the most recent modification timestamp of the file corresponding to it. The closer the time indicated by this timestamp is to the current time, the higher the weight of the corresponding term. This factor reflects the timeliness of the term, the activity level of the file, and the frequency of recent edits. For example, a term extracted from a file edited within the last 5 minutes has a higher weight than a term extracted from a file edited 1 hour ago. For terms without corresponding files, such as version control terms and static terms, a default timestamp benchmark can be set, such as using the current time or the most recent commit time of the branch. The weight value corresponding to this factor is called the timeliness weight, or time decay weight.

[0125] The number of times a word unit was successfully matched or its confidence score during historical recognition: This records the number of times each word unit was successfully matched in previous speech recognition processes, reflecting the recent hit frequency; or the confidence score at each match, reflecting the historical recognition success rate. The more successful matches or the higher the average confidence score, the higher the weight of the corresponding word unit. Historical data can be maintained through a hot word caching mechanism (see other implementation methods below). The weight value corresponding to this factor is called the historical matching weight.

[0126] When determining weights based on multiple factors simultaneously, linear weighting or multiplicative methods can be used for fusion. For example, the final weight score of a word / character can be calculated as follows: α × source priority weight + β × timeliness weight + γ × historical matching weight, where α, β, and γ are preset coefficients that can be adjusted according to the actual scenario, such as α = 0.5, β = 0.3, and γ = 0.2. The weight values ​​of each factor can be pre-normalized to the same numerical range, such as between 0 and 1, before fusion to ensure additivity.

[0127] It should be noted that the above weight fusion method is only an example, and this method is not limited to a specific fusion function. In actual implementation, a single factor can also be used, such as sorting based solely on the priority of word source, or a combination of factors can be dynamically selected according to the application scenario.

[0128] S302, based on the prompt length limit of the target offline speech recognition model, select at least a portion of the hot words from the sorted candidate hot word set to construct a dynamic prompt word sequence.

[0129] Different offline speech recognition models have varying capabilities in supporting prior cues: some models allow input of fixed-length cues, such as a maximum of 128 characters or 20 lexical units, while others allow input of a list of candidate lexical units as a decoding bias. This method determines the upper limit L of the cues length supported by the selected model based on its interface specification, with the upper limit expressed in terms of the number of lexical units or characters.

[0130] Select at least a subset of hot words from the sorted candidate hot word set to construct a dynamic prompt word sequence. Selection strategies include, but are not limited to, the following:

[0131] Top-K selection: Select the top K words with the highest weight scores, where K is determined by the prompt length limit L. For example, when L represents the number of words, K=L.

[0132] Threshold selection: Select all words with weight scores exceeding a preset weight threshold, and select them sequentially from highest to lowest weight until the length limit is reached. The preset weight threshold ranges from 0 to 1, for example, 0.7.

[0133] Proportion selection: Select the top p% of words, for example, the top 20%, where p is an adjustable hyperparameter.

[0134] The selected hot words are combined into a dynamic prompt word sequence either in descending order of weight or while retaining the original string order. This sequence can be a space-separated string, such as "jwt auth middleware bugfix", or a list of terms, such as ["jwt", "auth", "middleware", "bugfix"]. The specific format is determined according to the input requirements of the target offline speech recognition model.

[0135] It should be noted that this step only involves the construction of prompts during the inference phase and does not involve any model training or parameter update process. The target offline speech recognition model can be any local speech recognition model engine that supports prior prompts or candidate word constraints, such as an end-to-end model using a Transformer or Conformer architecture. The dynamic prompt word sequence will be injected into the decoding phase in subsequent step S400 to guide the recognition process.

[0136] S400, the dynamic prompt word sequence is used as a priori context and injected into the decoding stage of the locally running target offline speech recognition model to guide the target offline speech recognition model to recognize the audio data to be recognized.

[0137] The process of injecting the dynamic prompt word sequence into the decoding stage of the locally running target offline speech recognition model includes at least one of the following methods:

[0138] Method 1: Initial Prompt Text Input

[0139] For end-to-end models that support prefix prompts, such as those based on Transformer or Conformer architectures, the dynamic prompt word sequence is used as the initial input text for the decoder. Specifically, before the model begins decoding, a dynamic prompt word sequence, such as "jwt auth middleware," is provided as a prefix to the decoder. This allows the model to obtain contextual information when generating the first output token, thus making it more likely to output results containing these hot words in subsequent decoding.

[0140] Method 2: Candidate lexical bias and decoding path weighting

[0141] For decoders that support custom bias vocabularies, such as those based on WFST or CTC, perform the following:

[0142] Phoneme / Subword Mapping: Mapping each lexical unit in the dynamic prompt word sequence to a corresponding phoneme sequence or subword unit. For example, the lexical unit "git" can be mapped to a corresponding phoneme sequence or subword tag.

[0143] Word Graph Path Bias: In the word graph of the decoder of the target offline speech recognition model, a positive bias weight is set for the decoding path corresponding to the phoneme sequence or sub-word unit. The positive bias weight means that when a path matching a word in the dynamic prompt word sequence is encountered in the word graph search of multiple candidate paths, the cumulative score of that path is multiplied by a bias factor greater than 1, such as 1.2, or a fixed bias value, such as 0.5, is added to the score of that path.

[0144] The magnitude of the positive bias weight is dynamically adjusted according to the priority of the corresponding term in the preset weighting rules. Terms with higher priority, such as jwt parsed from a version control branch name, have a larger bias weight; terms with lower priority, such as the static term refactor, have a smaller bias weight. Specifically, the bias factor can be set to 1 + k × normalized priority weight, where k is a preset coefficient, for example, k = 0.5, used to control the bias strength and can be adjusted according to the actual scenario.

[0145] Through the above bias operation, the recognition path containing lexical units in the dynamic prompt word sequence obtains a higher probability score, thereby winning in the decoding competition and improving the recognition accuracy of relevant engineering-specific words.

[0146] Method 3: Reordering after decoding

[0147] For models that support N-best list output, after the decoder generates multiple candidate recognition results, the candidate recognition results are re-scored based on the dynamic prompt word sequence. The specific steps are as follows:

[0148] Obtain the original decoding score: For the N candidate recognition results (N-best list) output by the decoder, each candidate recognition result Ri corresponds to an original decoding score Score_original(Ri), which is usually the log probability or confidence score.

[0149] Calculate the weighted score for hot words: For each candidate recognition result Ri, count the dynamic hot words it contains, and calculate the weighted score Score_hotword(Ri) according to the preset weighting rules. Specifically, Score_hotword(Ri) is equal to the sum (or weighted sum) of the priority weights of each dynamic hot word appearing in the candidate recognition result. For example, if the candidate recognition result contains jwt (weight 0.9) and middleware (weight 0.7), then Score_hotword = 0.9 + 0.7 = 1.6.

[0150] Calculate the final score: Add the hot word weighted score to the original decoding score or use other fusion methods, such as weighted summation, to obtain the final score Score_final(Ri) for each candidate result:

[0151] Score_final(Ri) = Score_original(Ri) + λ × Score_hotword(Ri), where λ is a preset coefficient, for example, λ = 0.5, used to control the strength of the hot word weighting.

[0152] Reordering and Output: Reorder all candidate recognition results from highest to lowest according to the final scores of the N candidate recognition results, and select the candidate recognition result with the highest score after reordering as the final output recognition result.

[0153] Through the above reordering, the candidate recognition results containing high-priority dynamic hot words obtain higher final scores, thus being more likely to be selected as the final output, effectively improving the recognition accuracy of engineering-specific terms.

[0154] It should be noted that the above injection methods can be selected based on the specific interface of the target offline speech recognition model. For example, method one can be used for models that support prefix prompts; method two can be used for models that only support bias vocabularies; and method three can be used for closed models that only provide N-best outputs. In practice, multiple methods can also be combined to further enhance the guidance effect.

[0155] It should be noted that this step embodies the core improvement of this invention that distinguishes it from the prior art. The prior art usually improves the accuracy of proprietary vocabulary recognition in the following three ways: (1) relying on cloud hot word services, which poses a risk of data leakage; (2) fine-tuning the model, which requires a large amount of labeled data and is subject to update delays; (3) using static word lists, which cannot adapt to dynamic engineering contexts. All of the above methods have their own limitations.

[0156] Unlike the methods described above, this step achieves recognition guidance by injecting dynamic prompt word sequences during the decoding stage, offering the following technical advantages:

[0157] (1) No need to modify model parameters. This method only provides prior context to the decoder during the inference stage and does not involve any model training, fine-tuning or parameter update process. This means that: (a) no domain-annotated corpus needs to be prepared, resulting in low implementation cost; (b) no need to re-adapt when updating the model, resulting in good model independence; (c) it can run on resource-constrained local devices without the need for training hardware such as GPUs.

[0158] (2) Achieve dynamic adaptation to project status. Since the dynamic prompt word sequence is derived from the version control status and file system status extracted in real time from S100-S300, such as the current Git branch name bugfix / jwt-authentication, when the development environment changes, such as switching branches or editing new files, the prompt word sequence will be automatically updated and re-injected. This allows this method to respond to dynamic changes in the project context in real time, which is something that static vocabulary or fine-tuning models cannot do.

[0159] (3) Improve recognition accuracy from the decoding source. Traditional vocabulary schemes perform post-processing replacement or error correction after recognition, while this method directly integrates hot word information into the decoding process: by setting positive bias weights or using them as initial prompts, the decoding path containing hot words obtains a higher probability score from the beginning. This "source-guided" approach is more effective than post-processing because post-processing cannot correct misrecognitions that have already occurred at the acoustic level. For example, if "jwt" is recognized as "jay double u", post-processing is unlikely to recover it.

[0160] (4) Balancing privacy and performance. The entire process is executed offline locally, with audio data and engineering information remaining on the user's device, meeting the security requirements of enterprise intranets and confidential development scenarios. Furthermore, since no network communication is required, the recognition latency is low and unaffected by network conditions.

[0161] In a preferred embodiment, to avoid the misidentification of ordinary words as engineering hot words due to excessive dynamic hot word injection, or the inability to correctly identify engineering-specific words due to insufficient injection, this method further introduces an adaptive hot word bias algorithm based on acoustic confusion probability and engineering context relevance.

[0162] Specifically, for any hot word lexical h in the candidate hot word set, the hot word lexical h is first converted into the corresponding phoneme sequence or subword sequence P(h), denoted as:

[0163] P(h) = {p1, p2, ..., p} r ,……}

[0164] Where, p r Let f(h) represent the r-th phoneme or sub-word unit of the hot word lexicon h, where r ranges from 1 to f(h), and f(h) is the total number of phonemes or sub-word units of the hot word lexicon h. Based on the confusion records generated during the historical recognition process by the local offline speech recognition model, a phoneme-level or sub-word-level confusion matrix C is established, where C(a,b) represents the probability that the real phoneme or sub-word a is recognized as b. The confusion matrix can be obtained by statistically analyzing the alignment results between the user's historical confirmation results, the recognition results, and the manually corrected text, or by initializing it using a locally preset general confusion model.

[0165] For a hot word lexical h, its acoustic confusion metric Ra(h) is calculated. In one implementation, the acoustic confusion metric can be expressed as: Ra(h) = ΣC(p) r ,b(r))

[0166] Where Σ is the summation symbol. b(r) is the summation symbol for p. r Candidate phonemes or sub-words that are acoustically easily confused. A hot word has a high acoustic confusion metric if its pronunciation is highly similar to that of a common word, English abbreviation, command word, or other engineering identifier. For example, "auth" is easily recognized as "off," and "JWT" ​​is easily recognized as "jay double u" or can be broken down into multiple common syllables, thus having a high acoustic confusion metric.

[0167] Simultaneously, the engineering context relevance Rc(h) of the hot word lexical h is calculated. This engineering context relevance is determined based on at least one or more of the following factors:

[0168] Hot word source types include current Git branch name, currently active file name, recently modified file, AST symbol name, dependency library name, or static project dictionary;

[0169] Hot words correspond to the time freshness of engineering objects, such as the interval between the most recent modification time of a file and the current time.

[0170] The structural location of hot words in the current project, such as whether they appear in the current active file, current function, current class, or current test task;

[0171] The number of times a hot word was confirmed or corrected by the user as the correct result in the history of voice input;

[0172] The degree of match between hot words and the current downstream application intent, such as whether the current voice entry is in Git command mode, code editing mode, test execution mode, or document query mode.

[0173] In one implementation, project context relevance can be expressed as:

[0174] Rc(h)=λ1·Ss(h)+λ2·St(h)+λ3·Sa(h)+λ4·Sn(h)+λs·Si(h).

[0175] Where Ss(h) represents the source priority score of hot word lexical h, St(h) represents the time decay score of hot word lexical h, Sa(h) represents the activity code position score of hot word lexical h, Sn(h) represents the historical confirmation score of hot word lexical h, Si(h) represents the downstream intent matching score of hot word lexical h, and λ1, λ2, λ3, λ4 and λs are the corresponding weight coefficients.

[0176] The source priority score is determined based on the source type of the hot word element. Specifically, if the hot word element originates from a branch name or tag name parsed by a version control system, its source priority score is the highest, for example, set to 1.0; if it originates from a file name currently being edited or edited within a preset time window parsed by a file system, its source priority score is the next highest, for example, set to 0.7; if it originates from a preset vocabulary loaded from a static vocabulary list, its source priority score is relatively low, for example, set to 0.4; if it originates from other extended sources such as abstract syntax tree analysis or code completion services, its score can be dynamically determined between 0.5 and 0.8 depending on the specific source type.

[0177] The time decay score is calculated based on the difference between the most recent modification time of the file corresponding to the hot word / element and the current time. Specifically, the exponential decay function St(h) = exp(-Δt(h) / t0) is used to calculate the time decay score, where Δt(h) is the time difference between the most recent modification time of the file corresponding to the hot word / element h and the current time, and t0 is a preset time constant, such as 5 minutes. The smaller Δt is, the newer the file is, and the closer the score is to 1.0; the larger Δt is, the older the file is, and the closer the score is to 0. For hot words / elements without corresponding files, such as version control words and static words, their time decay score is set to 1.0.

[0178] The activity code location score is determined based on whether the hot term appears in the code context of the current activity. If the hot term is the class name, function name, or variable name where the cursor is located, its activity code location score is the highest, for example, set to 1.0; if the hot term is another symbol name extracted from the abstract syntax tree analysis of the current activity file, the score is next, for example, set to 0.8; if the hot term is a symbol name in other files of the current project, the score is relatively low, for example, set to 0.4; if the hot term does not appear in any code symbol, the score is 0.

[0179] The historical confirmation score is determined based on statistics from records in the hot word cache table. For each proper word recorded in the hot word cache table, the number of times it was successfully recognized during the historical recognition process is counted, and the contribution of each successful recognition is accumulated after time decay. The increment of a single successful recognition can be set to 0.1, and the time decay factor λ t Using λ t =exp(-Δt cache / τ cache ), where Δt cache For this successful identification, the interval from the current time is τ. cache This is the historical decay constant, for example, set to 60 seconds. The maximum sum of all contributions is capped at 1.0. If there are false trigger records for hot words, each false trigger will penalize the historical confirmation score, for example, multiplying the current score by 0.7.

[0180] The downstream intent matching score is determined based on the degree of matching between the system's current operation mode and the hot word lexicon h. The operation mode can be identified based on the context of the current voice entry point, such as Git command mode, code editing mode, test execution mode, or document query mode. If the hot word lexicon highly matches the current operation mode, for example, if the hot word is "checkout" or "branch" in Git command mode, the downstream intent matching score is set to 1.0; if it partially matches or is uncertain, it can be set to 0.5; if it does not match the current operation mode, it is set to 0.2 as the base score.

[0181] It should be noted that the specific numerical values ​​and function forms of the above scores are only examples. In actual implementation, they can be adjusted according to the application scenario and model characteristics. This embodiment does not limit them.

[0182] Furthermore, the dynamic injection intensity B(h) of the hot word h is calculated jointly based on the acoustic confusion metric and the engineering context relevance:

[0183] B(h)=B0·σ(α·Rc(h)+β·Ra(h)-γ·F(h)).

[0184] Where B0 is the maximum bias strength, σ is the Sigmoid function, α, β, and γ are adjustment parameters, and F(h) is the false trigger penalty term for the hot word lexical h. The false trigger penalty term characterizes the number of times or probability that the hot word was incorrectly promoted to the final recognition result in historical recognition. When the hot word is highly relevant to the current engineering context and has a high risk of acoustic confusion, the bias strength of the hot word is increased; when the hot word exists in the vocabulary but has low relevance to the current engineering task, or has a high historical false trigger probability, the bias strength of the hot word is decreased.

[0185] During the decoding stage, if a candidate path y of the decoder contains a hot word lexical h, the original decoding score of that candidate path is corrected as follows:

[0186] Score′(y)=Score(y)+ΣB(h)·M(h,y).

[0187] Where Score(y) is the original path score output by the target offline speech recognition model, and Score′(y) is the decoding score of the candidate path y after correction; M(h,y) represents the matching function between the hot word unit h and the candidate path y. If the candidate path completely contains the hot word unit h, then M(h,y)=1; if the candidate path only contains part of the phonemes, part of the subwords, or an approximate expression with an edit distance less than a preset threshold of the hot word h, then M(h,y) takes a continuous value between 0 and 1; if the candidate path does not contain the hot word, then M(h,y)=0.

[0188] Furthermore, the bias strength can be dynamically adjusted based on acoustic evidence during the decoding process. Specifically, when the decoder generates an intermediate candidate prefix, the phoneme prefix matching degree At(h) between that prefix and the hot word lexicon h is calculated. If the acoustic output corresponding to the current audio frame sequence is highly consistent with the phoneme prefix of the hot word lexicon h, then B(h) is increased; if the acoustic output deviates significantly from the phoneme prefix of the hot word h, then B(h) is decreased. This process can be expressed as: B t (h)=B(h)·At(h).

[0189] Among them, B t(h) represents the real-time bias strength at the t-th decoding moment. In this way, hot word injection is no longer a static addition of points to all hot word paths, but rather changes dynamically with the audio acoustic evidence, avoiding interference from irrelevant hot words on the normal recognition results.

[0190] The technical advantages of the aforementioned adaptive hot word bias algorithm are as follows: First, it can provide stronger guidance for the words most easily misidentified by general offline models in engineering-specific terms; second, it can dynamically distinguish between highly relevant and low-relevance hot words based on the current engineering context, avoiding misidentification caused by static hot word lists; third, it can adjust the bias intensity by combining real-time acoustic evidence during the decoding process, so that the hot word enhancement is consistent with the actual audio pronunciation, taking into account both the accuracy of engineering-specific term recognition and the stability of ordinary speech recognition.

[0191] In another implementation, considering that the target offline speech recognition model has a limitation on the length of the dynamic prompt word sequence, the hot word selection process can be modeled as a profit maximization problem with length constraints.

[0192] Suppose that the candidate hot word set contains multiple candidate hot words. Each candidate hot word has a corresponding contextual benefit value, acoustic confusion benefit value, historical error correction benefit value, and length consumption.

[0193] The context benefit value measures the relevance of the hot term to the current software engineering context. In one implementation, the context benefit value can be directly taken as the aforementioned engineering context relevance.

[0194] The acoustic confusion gain is used to measure the risk of a hot word being misidentified by a general speech recognition model. In one implementation, the acoustic confusion gain can be directly taken as the aforementioned acoustic confusion metric.

[0195] The historical error correction benefit value is used to measure the frequency or importance of the hot word being corrected by the user in the historical identification process. In one implementation, the historical error correction benefit value can be directly taken as the aforementioned historical confirmation score.

[0196] Length consumption represents the length resources occupied by the hot word in the prompt word sequence. In one implementation, length consumption is the number of characters in the hot word string. In other implementations, it can also be the number of sub-word units or phoneme units, depending on the actual input format of the target offline speech recognition model.

[0197] The overall benefit of each candidate hot word is obtained by weighted summation of the above three benefit values, and the weight coefficients can be preset according to the actual application scenario.

[0198] Given a fixed upper limit on the prompt length, the objective of constructing a dynamic prompt word sequence is to maximize the sum of the overall benefits of the selected hot words, provided that the total length consumption of the selected hot words does not exceed the upper limit. In other words, it involves selecting the combination of hot words with the highest overall benefit within a limited prompt length, rather than simply selecting the top few hot words with the highest weight. This optimization problem can be solved using dynamic programming, greedy approximation, or grouping constraint selection algorithms.

[0199] Furthermore, to avoid multiple hot words expressing the same or highly similar engineering meanings, a redundancy penalty mechanism can be introduced. If two hot words originate from the same engineering object, such as bugfix, jwt, and authentication split from the same branch name bugfix / jwt-authentication, or if they are highly similar in character composition, phoneme sequence, or semantics, then a certain amount of redundancy benefit value will be deducted when they are simultaneously selected into the prompt word sequence.

[0200] In one implementation, the redundancy between two hot words is determined by multiplying their normalized similarity by a preset redundancy penalty coefficient, where the product is the redundancy benefit value. The normalized similarity ranges from 0 to 1, and the preset redundancy penalty coefficient is used to control the overall strength of the redundancy penalty, for example, it can be set to 0.3.

[0201] The normalized similarity can be calculated using character edit distance, which determines the similarity based on the degree of character difference between two hot words; the smaller the difference, the higher the similarity. Alternatively, it can be calculated using phoneme sequence similarity, which involves converting the two hot words into phoneme sequences and then calculating the matching degree between the sequences. If the two hot words originate from the same project object, the normalized similarity is directly set to 1.

[0202] If two hot words originate from different engineering objects and have significant character differences, the normalized similarity will approach 0, and the redundant penalty term can be ignored.

[0203] After introducing redundancy penalties, the optimization objective is adjusted to maximize the net value of the sum of the overall benefits of the selected hot words minus the sum of the pairwise redundancy benefits of all simultaneously selected hot words, while meeting the prompt length limit. This approach prioritizes hot words with high contextual relevance, high acoustic confusion risk, high historical error correction value, and low redundancy when prompt length is limited. It avoids multiple similar word units split from the same project object crowding out the limited prompt length, thereby improving the information density of the dynamic prompt word sequence.

[0204] S400 improves the recognition accuracy of proprietary words in software engineering scenarios by using the dynamic hot word injection decoding stage technology, without modifying model parameters, relying on the cloud, or increasing training costs. It also achieves dynamic real-time synchronization with the engineering status, solving the problem that existing technologies struggle to balance accuracy, dynamic adaptability, cost, and privacy protection.

[0205] S500: Obtain the audio data to be recognized input by the user, output the text recognition result through the offline speech recognition model with the dynamic prompt word sequence injected, and pass the text recognition result to the downstream application module.

[0206] The system uses a microphone array on the local computing device or an external microphone to capture the user's voice input in real time, obtaining a real-time audio stream as the audio data to be recognized. Audio acquisition parameters, such as a sampling rate of 16kHz, a bit depth of 16bit, and mono, are set according to the requirements of the target offline speech recognition model. Optionally, pre-recorded audio files, such as stored voice command files, can also be read as audio input.

[0207] The acquired audio data can be preprocessed, including but not limited to: removing ambient noise, performing Voice Activity Detection (VAD) to identify the start and end positions of valid speech segments, and segmenting the audio into duration segments suitable for model input.

[0208] The preprocessed audio data is input into a local offline speech recognition model that has already been injected with dynamic cue word sequences during the decoding stage. This local offline speech recognition model has received the dynamic cue word sequences as prior context in step S400; therefore, during audio recognition, the decoding process prioritizes recognition paths that match the lexical units in the dynamic cue word sequences. Specifically, the model performs the following operations:

[0209] Acoustic feature extraction: The input audio data is segmented and windowed to extract acoustic feature sequences such as MFCC or Filter Bank features. This feature sequence is a numerical representation of the audio signal, adapted to the input format of the speech recognition model.

[0210] Encoding representation: The acoustic feature sequence is input into the encoder of an offline speech recognition model, such as a Conformer or Transformer encoder, to calculate the corresponding acoustic representation, which is the encoder output feature sequence. This acoustic representation captures the pronunciation information in the audio signal.

[0211] Decoding Output: The acoustic representation is input into the decoder, along with a dynamic cue word sequence injected into the S400, which may be used as an initial cue, bias weight, or reordering criterion. The decoder generates the text recognition result. During decoding, words in the dynamic cue word sequence are assigned higher probability scores, guiding the model to output recognition results containing these words.

[0212] Finally, the model outputs one or more candidate recognition results (an N-best list), typically selecting the one with the highest score as the final text recognition result. The output text can be a sequence of words, characters, or subwords, which are concatenated into a readable string as needed.

[0213] In this embodiment, the downstream application module can be one or more of the following:

[0214] Code editor: Inserts the recognition result at the current cursor position for code completion, code generation, or comment addition. For example, after recognizing "git checkout jwt-branch", the editor automatically executes the command or inserts the text.

[0215] Command-line terminal: The recognition results are submitted as commands to the terminal for execution, or populated into the command-line buffer for user confirmation. For example, if "make test-auth" is recognized, the terminal will automatically execute the test script.

[0216] Command execution module: Performs semantic parsing on the recognition results, identifies the operation intent such as "refactor", "add log", "run test" and parameters such as function name validate_token, and calls the corresponding development tool API to execute the operation.

[0217] Voice Assistant / Question-Answer Module: Uses the recognition results as a query to retrieve development documentation or perform information queries such as "jwt token expiration time", and returns relevant answers.

[0218] Integrated Development Environment (IDE) Plugin: Passes the identification results to the command processing center of the IDE plugin, triggering functions such as code jump, error location, and refactoring suggestions.

[0219] Furthermore, if the confidence score of the recognition result is lower than the preset confidence threshold, the user can be asked to repeat or confirm. Specifically, when the confidence score of the recognition result is less than the preset confidence threshold, such as 0.6, a voice prompt "Please say it again" or an interface pop-up window can be used to ask "Did you say '...'?", and corrections or re-recognition can be made based on user feedback. This feedback can also be used to update the historical recognition success rate in the hot word cache (see historical matching weights in S300).

[0220] For example, in an authentication module repair scenario, the user speaks the voice command: "refactor the JWTmiddleware to fix the auth bug." The collected real-time audio is input into an offline speech recognition model that has been injected with a dynamic prompt word sequence. The model outputs the text recognition result: "refactor the JWT middleware to fix the auth bug." This result is passed to a code editor plugin, which recognizes the keywords "refactor," "JWTmiddleware," and "auth bug," automatically locates the JWT-related functions in the auth_middleware.py file and performs the preset refactoring operation, or inserts the recognized text into the editor for further editing by the user.

[0221] The method of passing the data to downstream modules can be determined based on the type of the downstream module, including but not limited to:

[0222] For integrated development environment (IDE) plugins: the recognition result string is sent to the plugin's event handling center through the API interface provided by the plugin, and the plugin decides whether to insert it into the editor, execute a command, or jump symbol.

[0223] For command-line terminals: the recognition results are passed to the terminal process as standard input or command-line arguments, or filled into the current command-line buffer by simulating keyboard input.

[0224] For code editors: The recognition results can be inserted into the current cursor position or the selected text can be replaced through the editor's extended interfaces such as VS Code's executeCommand and Sublime Text's run_command.

[0225] For standalone command execution modules: the identification results are sent to the command parser via inter-process communication such as named pipes, Unix Domain Sockets, or HTTP local interfaces, and the parser identifies the intent and calls the corresponding development tools.

[0226] After the transmission is complete, the downstream application module performs corresponding operations based on the recognition results, such as code generation, command execution, and document querying. This method does not limit the specific implementation of the downstream module, as long as it can receive the text string and perform the corresponding operations.

[0227] With the implementation of S500, users no longer need to manually input complex engineering terminology; they can complete operations such as code editing and command execution solely through voice, thereby improving development efficiency. Furthermore, since the entire recognition process runs locally offline, the audio data never leaves the user's device, meeting data privacy and compliance requirements.

[0228] Furthermore, based on the above embodiments, this method may also include a real-time dynamic update step:

[0229] S601, monitors in real time changes in version control status and / or file system changes under the working directory path. Wherein:

[0230] The version control status changes include at least one of the following: branch switching, tag change, commit identifier update, and workspace identifier change; the file system changes include at least one of the following: file creation, file deletion, file renaming, and file content modification.

[0231] S602, in response to the detection of the version control status change and / or the file system change, steps S100 to S400 are automatically re-executed, namely: re-extracting project context features, generating a candidate hot word set, constructing a dynamic prompt word sequence, and re-executing the step of injecting the dynamic prompt word sequence into the decoding stage of the target offline speech recognition model running locally based on the updated dynamic prompt word sequence.

[0232] Through the aforementioned real-time monitoring and automatic update mechanism, when developers switch Git branches, modify files, or perform other project changes, the dynamic prompt word sequence can be automatically updated without manual user intervention. This ensures that the speech recognition decoding stage is always guided by the latest hot words of the current project status, thereby maintaining the best recognition accuracy.

[0233] Furthermore, the method provided in this embodiment also includes:

[0234] S701, after generating the dynamic prompt word sequence, for each word element (referred to as "specific word element") in the dynamic prompt word sequence, check whether there is a pre-trained hot word model corresponding to the specific word element in the local hot word model library, and determine the target hot word model to be added to the hot word detector based on the check results.

[0235] If a corresponding pre-trained hot word model exists, then the pre-trained hot word model is activated as the target hot word model and loaded into the hot word detector.

[0236] If no corresponding pre-trained hot word model exists, a temporary hot word model is dynamically generated from a general acoustic model based on the pronunciation or spelling features of the lexical units, serving as the target hot word model. This target hot word model is then loaded into the hot word detector. The method for dynamically generating a temporary hot word model includes converting the string of a specific lexical unit into a phoneme sequence, for example, using a character-to-phoneme converter. Then, the corresponding acoustic feature template is extracted from the general acoustic model to construct a lightweight hot word detection model, which serves as the temporary hot word detection model. The specific operations of the above implementation are well-known in the art and will not be detailed here.

[0237] In this context, a pre-trained hotword model refers to a specialized acoustic model trained in advance using a large number of speech samples to detect whether a specific word motif appears in audio. One pre-trained hotword model corresponds to only one specific word motif. During the inference phase, the input to the pre-trained hotword model is the acoustic features of the audio, such as MFCC or Filter Bank features, and the output is a hotword confidence score between 0 and 1, representing the probability that the current audio segment contains that specific word motif.

[0238] In this embodiment, the pre-trained hot word model can employ a deep neural network binary classifier known in the art. Its training method involves collecting a large number of positive sample speech containing the specific word and negative sample speech not containing the specific word, extracting acoustic features, and then training the model. The specific structure and training process of the above model are common knowledge in the art and will not be elaborated here.

[0239] The general acoustic model refers to a model that is not targeted at specific lexical units or general speech acoustic features, such as an acoustic feature extractor trained on a large amount of general speech data or an acoustic encoder of a general speech recognition model. During the inference phase, the input of the general acoustic model is the acoustic features of the audio, and the output is the corresponding acoustic feature representation, such as a sequence of feature vectors output by an encoder. In this embodiment, the general acoustic model can employ an acoustic encoder of a general speech recognition model known in the art, such as the Conformer encoder, to extract general speech features. Its specific structure is common knowledge in the art and will not be elaborated here.

[0240] It should be noted that after the temporary hot word model is built, it has the same input and output interface as the pre-trained hot word model: input is the acoustic features of the audio, and output is the confidence score of the specific word.

[0241] S702, the hot word detector extracts acoustic features such as MFCC or Filter Bank features from the received audio data, inputs the extracted acoustic features into the loaded target hot word model, including the pre-trained hot word model and the temporarily generated temporary hot word model, and calculates the hot word confidence score for each specific word element.

[0242] In this embodiment, the hot word detector runs in parallel with the target offline speech recognition model to detect hot words in the audio data. The hot word detector is independent of the target offline speech recognition model and adopts a lightweight neural network structure, such as a deep neural network or convolutional neural network with 2-4 layers. Its parameter count is usually less than 1% of that of the target offline speech recognition model. Therefore, the computational overhead is low, and it can process audio streams in real time on resource-constrained local devices without significantly increasing the overall recognition latency or power consumption.

[0243] S703, when the confidence score of a hot word for a specific word exceeds a preset confidence threshold, such as 0.7, the specific word and its confidence score are fed as prior information to the decoder of the target offline speech recognition model.

[0244] After receiving the prior information, the decoder adjusts the priority of the decoding path corresponding to the word: specifically, during the decoding process, a positive bias is added to the recognition path containing the word, for example, the path score is multiplied by 1 + confidence score, thereby increasing the probability that the word is correctly recognized.

[0245] If the confidence scores of all words do not exceed the preset threshold, the hot word detector does not feed any prior information to the decoder, and the decoder performs recognition according to the original model.

[0246] Through the aforementioned dynamic loading and confidence detection mechanism for hot word models, this method can dynamically adapt to any engineering identifier without pre-training all possible hot word models, further improving the versatility and real-time performance of this method.

[0247] Furthermore, after each completion of the dynamic prompt word sequence construction (S300), the following hot word caching and memory enhancement steps can be performed to improve the continuous recognition effect in the multi-turn voice interaction process.

[0248] S801, Create a hot word cache table associated with the current session.

[0249] The hot word cache table is used to record the proper words that meet the preset success conditions in the previous recognition rounds and their corresponding temporary weight offsets.

[0250] The preset success conditions include at least one of the following:

[0251] After the proprietary word is recognized by the target offline speech recognition model, the corresponding hot word confidence score exceeds a preset success threshold, such as 0.8.

[0252] The results of this proper noun recognition are adopted by downstream application modules, for example, if the user does not correct it or the command is successfully executed.

[0253] Each entry in the hot word cache table contains: the proper word string, the temporary weight offset, and the timestamp of the last successful recognition.

[0254] S802, when a proper word is recorded in the hot word cache table, a corresponding temporary weight offset is added to the proper word based on the preset weight rules to improve the sorting priority of the proper word in the current candidate hot word set.

[0255] Specifically, the final ranking weight of this proper noun = base weight + temporary weight offset. The base weight is determined by factors such as source priority, timeliness, and historical match count.

[0256] The initial value of the temporary weight offset can be set based on the number of successful recognitions or the confidence score of the proper word, for example:

[0257] Upon successful initial recognition, the offset is set to +0.1.

[0258] When consecutive successful recognitions occur, the offset can be accumulated, but it cannot exceed the upper limit, such as +0.5.

[0259] The temporary weight offset decreases over time according to a preset decay function. The decay function can be:

[0260] Exponential decay: Offset(t) = Initial offset × exp(-t / T), where t is the time since the last successful identification, and T is a time constant, such as 30 seconds;

[0261] Linear decay: offset (t) = initial offset × (1-t / Tmax), and returns to zero when t≥Tmax;

[0262] Step decay: Decrease by a fixed step size, such as 0.05, every fixed time interval, such as 10 seconds.

[0263] When the temporary weight offset decays to zero or is less than a preset zeroing threshold, such as 0.01, the proper word is removed from the hot word cache table.

[0264] To more accurately reflect recent usage patterns, multiple rounds of statistical analysis can be used to update the weights of specialized terms:

[0265] Count the number of successful recognitions for each proper noun in the most recent M rounds (M can be set to 5 or 10);

[0266] The higher the success rate of a proper word, the greater its temporary weight shift. For example, temporary weight shift = success rate × 0.05.

[0267] Meanwhile, proper nouns that are successfully identified more recently are assigned a higher temporary weight offset, such as time-weighted weighting. For example, using a time decay factor λ=exp(-Δt / τ), where Δt is the time difference between the time of successful identification and the current time, and τ is a preset time constant such as 60 seconds, the actual contribution of this successful identification to the temporary weight offset is λ×single offset increment, for example, the single offset increment is set to 0.1.

[0268] Assume that in the first two rounds of recognition, the user successfully uttered the proper nouns "jwt" and "auth". The hot word cache table records:

[0269] jwt: Offset +0.12, last success time 5 seconds ago;

[0270] auth: offset +0.08, last success time 10 seconds ago.

[0271] In the third round of identification, when sorting the candidate hot word set, the basic weights of jwt and auth are 0.7 and 0.6 respectively. After adding the offset, the final sorting weights are 0.82 and 0.68 respectively, which makes jwt obtain a higher sorting position and thus be selected first in the dynamic prompt word sequence.

[0272] Assuming an initial offset of 0.1, if the jwt is not detected again after 30 seconds, the initial offset decays exponentially to approximately 0.04, and is eventually removed from the cache table when it reaches zero.

[0273] Through the aforementioned hot word caching and dynamic priority adjustment mechanism, the system can remember frequently used engineering-specific terms in the current conversation and continuously prioritize them in subsequent recognition. This solves the recognition stability problem when the same specialized term appears repeatedly in multi-turn dialogues, while avoiding interference from outdated words through time decay, ensuring that the hot word sequence remains synchronized with the current development task.

[0274] Furthermore, in this embodiment, all steps of extracting engineering context features, generating candidate hot word sets, constructing dynamic prompt word sequences, and injecting them into the decoding stage are performed on the local computing device to form a local closed-loop processing that does not rely on external servers; wherein, the audio data, engineering context features, and candidate hot words do not leave the local computing device, that is, the above data are not transmitted to or exposed to any external server.

[0275] In summary, the method provided in this embodiment has the following technical advantages compared to the prior art:

[0276] 1. Improve the accuracy of identifying engineering-specific terms.

[0277] By parsing the current branch name (e.g., bugfix / jwt-authentication) from the version control system and retrieving the file name edited within the current or preset time window (e.g., auth_middleware.py) from the file system, and performing word segmentation based on project naming conventions on these compound words, a candidate hot word set composed of minimal semantic words is generated. After injecting this dynamic prompt word sequence into the decoding stage of the offline speech recognition model, the model applies a positive bias to the recognition path containing the above words during decoding, thereby reducing misidentifications such as recognizing jwt as "jay double u" or auth as "office".

[0278] 2. Achieve real-time dynamic synchronization with the development environment.

[0279] Because the project context features are derived from real-time version control and file system states, and the system can monitor events such as branch switching and file modifications and automatically trigger re-extraction, word segmentation, sorting, and injection processes, the dynamic suggestion word sequence always reflects the relevant vocabulary of the current development task. When a developer switches from branch A to branch B, the proprietary words in the original branch automatically exit the hot word sequence, and the proprietary words in the new branch are automatically added; when editing a new file, the core words in the filename are added to the hot word sequence in real time. This dynamic update mechanism requires no manual configuration or intervention from the user and has higher real-time performance compared to static word lists or periodically updated fine-tuning models.

[0280] 3. Reduce deployment costs and eliminate the risk of data leakage.

[0281] All steps are performed on local computing devices, without relying on external servers, and without requiring the uploading of audio streams, project context information, or candidate hot words. Compared to solutions that require collecting labeled corpora and fine-tuning the model, this method eliminates the need for a training process; compared to solutions that rely on cloud-based hot word APIs, this method avoids the network transmission of project names, code terminology, and audio data, complying with enterprise security guidelines that prevent the leakage of internal data.

[0282] 4. Compatible with multiple offline speech recognition models

[0283] This method adapts to different types of offline speech recognition models through three optional injection methods: initial prompt text input, decoding word graph path bias, and N-best list reordering. It does not require modification of the underlying model's parameters or architecture and has good model independence.

[0284] 5. Maintain recognition continuity across multiple rounds of interaction.

[0285] An optional hot word cache table records proper nouns that meet preset success conditions in previous recognition rounds, along with their temporary weight offsets. These words are then prioritized in subsequent rounds, and their offsets are reduced over time using a decay function until they reach zero and are removed. This mechanism ensures that frequently used proper nouns in the same conversation are consistently and accurately recognized across multiple voice inputs.

[0286] 6. It has greater adaptability to word segmentation of engineering identifiers.

[0287] Beyond basic word segmentation rules, this method also provides optional word segmentation optimization based on acoustic features: extracting silence segments, pitch transition points, or energy transition points as potential word boundaries, and fusing them with engineering naming conventions to determine the final word segmentation position. When processing combinations of pure uppercase abbreviation sequences and ordinary words, this approach can additionally utilize boundary information in the speech signal, further reducing word segmentation errors caused by ambiguous naming conventions.

[0288] 7. Supports dynamically generating temporary hot word models

[0289] For lexical units in the dynamic prompt word sequence that do not have a pre-trained hot word model, this method can generate a temporary hot word model from a general acoustic model based on the pronunciation or spelling features of the lexical unit and load it into the hot word detector. This mechanism allows any engineered lexical unit not in the local hot word model library to still receive hot word detection support without needing to pre-train a model for each lexical unit.

[0290] It should be noted that technical bias has long existed in the field of speech recognition in software engineering. Those skilled in the art generally believe that improving the accuracy of offline speech recognition models in recognizing engineering-specific vocabulary can only be achieved through the following three approaches: (1) collecting a large amount of annotated corpus in the engineering field to fine-tune the model; (2) relying on cloud-based hot word services to achieve dynamic bias; and (3) using a pre-set static vocabulary list for post-processing replacement. These approaches each have drawbacks such as high cost, privacy risks, and inability to dynamically adapt.

[0291] However, due to this technological bias, those skilled in the art have long overlooked another possibility: in a local offline environment, hot words can be dynamically generated by parsing the version control status and file system status in real time, and then directly injected into the decoding stage of the speech recognition model for guidance.

[0292] This invention is proposed to overcome the aforementioned technical biases. Through a complete technical solution from S100 to S500, this invention effectively improves the recognition accuracy of proprietary terms in software engineering scenarios without relying on model fine-tuning, cloud services, or static vocabularies. Its specific mechanism is as follows:

[0293] Dynamically extract project-specific terms: By parsing version control systems such as Git branch names like bugfix / jwt-authentication and file systems such as the currently edited file auth_middleware.py in real time, it can obtain specialized terms that are strongly related to the current development task, ensuring that the hot words always match the project identifier that the user is about to say.

[0294] Engineering-based word segmentation eliminates boundary ambiguity: Based on the naming conventions unique to engineering identifiers, compound words are broken down into the smallest semantic words, such as splitting jwt-authentication into jwt and authentication, to avoid recognition errors caused by misjudgment of word boundaries.

[0295] Multi-factor weighted sorting to select the most relevant hot words: Candidate hot words are sorted according to the priority of word source, timeliness (the more recent the file modification timestamp, the higher the weight) and historical recognition success rate. The most critical hot words are retained within the length limit suggested by the model to improve injection efficiency.

[0296] The decoding stage guides the correction of misidentification at its source: Dynamic prompt word sequences are injected as prior context into the decoding stage, rather than being replaced by post-processing. Through methods such as initial prompt text, path biasing of the decoding word graph, or N-best reordering, recognition paths containing hot words achieve higher probability scores. This method can fundamentally correct acoustic-level homonym misidentification, such as misidentifying "jwt" as "jay double u," while post-processing schemes cannot recover paths excluded by the acoustic model.

[0297] Through the above mechanism, this invention achieves dynamic hot word injection that is synchronized with the engineering status in real time. It does not require model training or cloud services, improves the recognition accuracy of engineering-specific vocabulary, and solves a long-standing but unresolved technical problem.

[0298] Second Embodiment

[0299] This embodiment provides an offline speech recognition dynamic hot word injection device based on engineering context, including:

[0300] The environment awareness unit is configured to respond to a voice input trigger command, obtain the working directory path of the current development environment, and extract project context features based on the working directory path; the project context features include the branch name or tag name parsed from the version control system, and / or the file name currently being edited or within a preset time window before the current time point parsed from the file system.

[0301] The hot word generation unit is configured to perform engineering-oriented word segmentation on the extracted engineering context features to generate dynamic candidate hot word units; and to fuse the dynamic candidate hot word units with static candidate hot word units in the static domain vocabulary set to generate a candidate hot word set; the engineering-oriented word segmentation includes: segmenting the composite units in the engineering context features into the smallest units constituting the composite units according to the naming convention of the engineering identifier;

[0302] The injection unit is configured to inject the dynamic prompt word sequence as a priori context into the decoding stage of the target offline speech recognition model running locally, so as to guide the target offline speech recognition model to recognize the audio data to be recognized.

[0303] The recognition execution unit is configured to acquire the audio data to be recognized input by the user, output the text recognition result through an offline speech recognition model with injected dynamic prompt word sequence, and pass the text recognition result to the downstream application module.

[0304] The specific operation procedures, parameter settings, optional implementation methods, and technical effects achieved by each of the above units are the same as the corresponding steps in the first embodiment. For the sake of simplicity, this embodiment only provides a general description of the functions of each unit. For detailed implementation methods, please refer to the description of the corresponding steps in the first embodiment, which will not be repeated here.

[0305] Furthermore, the various extended implementation methods described in the first embodiment, such as abstract syntax tree analysis, code completion, dynamic loading of hot word models, real-time monitoring and updating, hot word caching mechanism, and local closed-loop privacy protection, are also applicable to the device of this embodiment. That is, the corresponding units in the device can be further configured to implement the above-mentioned extended functions.

[0306] This invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being configured to perform the method described in this invention.

[0307] This invention also provides a computer-readable storage medium storing computer-executable instructions for performing the methods described in this invention.

[0308] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0309] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for dynamic hot word injection in offline speech recognition based on engineering context, characterized in that, The method includes the following steps: In response to a voice input trigger command, the working directory path of the current development environment is obtained, and project context features are extracted based on the working directory path; the project context features include the branch name or tag name parsed from the version control system, and / or the file name currently being edited or within a preset time window before the current time point parsed from the file system. The extracted engineering context features are processed using engineering-based word segmentation to generate dynamic candidate hot word units; The dynamic candidate hot word units are fused with the static candidate hot word units in the static domain vocabulary set to generate a candidate hot word set; The engineering-based word segmentation process includes: segmenting the compound words in the engineering context features into the smallest words that constitute the compound words according to the naming convention of the engineering identifier; The word units in the candidate hot word set are sorted according to the preset weighting rules, and at least a portion of the hot words are selected from the sorted candidate hot word set to construct a dynamic prompt word sequence according to the prompt length limit of the target offline speech recognition model. The dynamic prompt word sequence is used as a priori context and injected into the decoding stage of the target offline speech recognition model running locally, so as to guide the target offline speech recognition model to recognize the audio data to be recognized. The system acquires the audio data to be recognized input by the user, outputs the text recognition result through an offline speech recognition model that has been injected with the dynamic prompt word sequence, and passes the text recognition result to the downstream application module.

2. The method according to claim 1, characterized in that, The preset weighting rule is determined based on at least one of the following factors: The priority of the source of the lexical element, where the lexical elements obtained from the version control system have a higher priority than the lexical elements obtained from the file system; The most recent modification timestamp of the file corresponding to the term is used. The closer the time indicated by the most recent modification timestamp is to the current time, the higher the weight of the corresponding term. The number of times a word was successfully matched or its confidence score during the historical recognition process.

3. The method according to claim 1, characterized in that, Also includes: Real-time monitoring of version control status changes or file system changes under the working directory path; In response to the detection of a change in version control status or file system change, the steps of extracting project context features, generating a candidate hot word set, and constructing a dynamic prompt word sequence are automatically re-executed to update the dynamic prompt word sequence; The step of injecting the dynamic prompt sequence into the decoding stage of the locally running target offline speech recognition model is then re-executed based on the updated dynamic prompt sequence.

4. The method according to claim 1, characterized in that, Also includes: An abstract syntax tree analysis is performed on the current active code file, and at least one of the following is extracted from the current active code file: class name, function name, variable name, interface name, or enumeration name, which is then added as additional hot words to the candidate hot word set.

5. The method according to claim 1, characterized in that, The decoding stage of injecting the dynamic prompt word sequence into the locally running target offline speech recognition model includes: Each word in the dynamic prompt word sequence is mapped to a corresponding phoneme sequence or sub-word unit; In the word graph of the decoder of the target offline speech recognition model, a positive bias weight is set for the decoding path corresponding to the phoneme sequence or sub-word unit to improve the probability score of the recognition result containing the lexical units in the dynamic prompt word sequence. The positive bias weight is dynamically adjusted according to the priority of the lexical units in the preset weighting rules.

6. The method according to claim 1, characterized in that, Also includes: After generating the dynamic prompt word sequence, for each word element in the dynamic prompt word sequence, check whether there is a corresponding pre-trained hot word model in the local hot word model library, and determine the target hot word model to be added to the hot word detector based on the check results; The hot word detector extracts acoustic features from the received audio data and inputs the extracted acoustic features into the loaded target hot word model to calculate the hot word confidence score for each word element. When the confidence score of the hot word exceeds a preset threshold, the word and its confidence score are fed as prior information to the decoder of the target offline speech recognition model to adjust the priority of the decoding path corresponding to the word.

7. The method according to claim 1, characterized in that, The engineered word segmentation process also includes: For the branch name or tag name parsed from the version control system, obtain the speech sample corresponding to the branch name or tag name. The speech sample is a historical speech input sample or a synthesized speech sample. The acoustic features of the acquired speech samples are extracted, and the silence segments, pitch transition points or energy transition points in the acoustic features are identified as potential word boundaries. The final word segmentation position is determined by integrating the potential vocabulary boundaries with the naming conventions of the engineering identifiers.

8. The method according to claim 1, characterized in that, Also includes: Establish a hot word cache table associated with the current session. The hot word cache table is used to record the proper words that meet the preset success conditions in the previous recognition rounds and their corresponding temporary weight offsets. When a proper word is recorded in the hot word cache table, a corresponding temporary weight offset is added to the proper word based on the preset weight rules to improve the sorting priority of the proper word in the current candidate hot word set. The temporary weight offset decreases over time according to a preset decay function until it reaches zero, at which point the proper word is removed from the hot word cache table.

9. The method according to claim 1, characterized in that, Injecting the dynamic prompt word sequence into the decoding stage of the locally running target offline speech recognition model also includes: Each word in the dynamic prompt word sequence is converted into a corresponding phoneme sequence or subword sequence; Based on the confusion records generated during the historical recognition process by the local offline speech recognition model, a phoneme-level or sub-word-level confusion matrix is ​​established. Each element in the confusion matrix represents the statistical probability that a real phoneme or sub-word is recognized as another phoneme or sub-word. The acoustic confusion metric value for each word is calculated based on the confusion matrix. The acoustic confusion metric value is equal to the sum of the confusion probabilities between each phoneme in the word phoneme sequence and its corresponding confused phoneme. The dynamic bias strength of a lexical is calculated by combining its engineering context relevance and acoustic confusion metric, wherein the higher the engineering context relevance and the higher the acoustic confusion metric, the greater the bias strength.

10. The method according to claim 1, characterized in that, Based on the prompt length limit of the target offline speech recognition model, at least a portion of the hot words are selected from the sorted candidate hot word set to construct a dynamic prompt word sequence, including: A comprehensive benefit is calculated for each candidate hot word, which is determined by weighted summation of the context benefit value, acoustic confusion benefit value, and historical error correction benefit value of the candidate hot word; Get the length consumption of each candidate hot word; Under the constraint of not exceeding the upper limit of prompt length, one or more candidate combinations are determined based on the comprehensive benefits and length consumption of each candidate hot word. The total comprehensive benefit of each candidate combination is equal to the sum of the comprehensive benefits of each hot word in the candidate combination. For any two hot words selected simultaneously in any candidate combination, if they come from the same project object or the character similarity exceeds a preset threshold, the redundancy penalty value between the two hot words is calculated, and the redundancy penalty value is deducted from the total comprehensive benefit of the candidate combination to obtain the net benefit of the candidate combination. The candidate combination with the highest net profit is selected as the final dynamic prompt word sequence.

Images