Apparatus And Method For Retraining Of Voice Recognition Based On Speech-to-Text In High Noise Envirronments
The speech recognition learning system addresses high-noise environment challenges by automatically detecting transcription errors and configuring retraining datasets, improving accuracy and reducing manual effort, particularly in air traffic control and military communications.
Patent Information
- Application Number
- KR1020250058791
- Authority / Receiving Office
- KR · KR
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-05-07
- Publication Date
- 2026-11-16
AI Technical Summary
Existing STT-based speech recognition technologies suffer from high recognition error rates in high-noise environments due to severe wireless channel instability and background noise, and existing retraining methods lack intensive training on core data where errors repeatedly occur, making it difficult to improve accuracy and requiring significant manual effort.
A speech recognition learning system that includes a user interface for keyword registration, a transcription engine for converting speech to text, a transcription error detection module for error analysis using CER and Levenshtein distance, and a retraining data configuration module to automatically generate datasets for retraining based on user-defined criteria.
The system improves speech recognition accuracy in high-noise environments by automatically detecting and addressing transcription errors, reducing manual effort, and enabling efficient retraining focused on high-frequency errors, thus enhancing performance in domains like air traffic control and military communications.
Smart Images

Figure PAT00002_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to a Speech-to-Text (STT)-based speech recognition technology that converts speech into text, and more specifically, to an STT-based speech recognition retraining device and method in a high-noise environment that can improve the accuracy of speech recognition by automatically detecting transcription errors according to user-defined criteria and configuring a dataset for retraining. Background Technology
[0002] With the advancement of machine learning and the rising sophistication of speech and natural language processing technologies, STT performance is steadily improving, leading to a surge in demand for speech recognition technology across various fields. For instance, speech recognition technology is being applied in air traffic control radio communications for communication between control towers and pilots; in disaster and emergency response communications that recognize speech utterances to provide guidance on evacuation routes; and in military radio communications for command, reconnaissance, and reporting between commanders and personnel.
[0003] In harsh, high-noise environments such as air traffic control and military wireless communications, severe wireless channel instability and background noise result in high recognition error rates when using existing STT speech recognition models, making it difficult to apply them to real-time operations.
[0004] While the performance of speech recognition can be improved by utilizing the training capabilities of speech recognition models, public institutions requiring security face limitations in accessing publicly available external training data due to the constraint that they cannot use cloud-based speech recognition solutions.
[0005] Existing retraining methods for STT-based speech recognition are performed based on batch and large-scale data, but they lack intensive training on core data where errors repeatedly occur in real-world usage environments.
[0006] When using a language model for a specific domain, continuous retraining is necessary to improve STT performance by reflecting domain-specific linguistic characteristics, unique terms, and important keywords. The process of users directly identifying errors in STT recognition results and constructing training data based on this information requires significant manpower and time, and it is also difficult to guarantee the accuracy of the error identification performed by the users. Furthermore, it is nearly impossible to identify high-frequency error keywords requiring retraining using user-based error identification methods, and the low efficiency of the identification process makes it very difficult for users to directly construct and manage retraining data for speech recognition. The problem to be solved
[0007] The inventors recognized that the accuracy of STT-based speech recognition is significantly affected by the training data of the speech recognition model in high-noise environments. For example, when converting a speech signal into text based on STT technology utilizing a speech recognition model, the accuracy of speech recognition decreases due to the occurrence of transcription errors.
[0008] The objective of the present invention is to improve the accuracy of speech recognition by systematically detecting transcription errors that occur when converting voice communication to text in a high-noise environment, and by automatically constructing a retraining dataset for a speech recognition model based on user-defined words and utilizing it for domain-adaptive retraining of the speech recognition model. means of solving the problem
[0009] The STT-based speech recognition retraining device in a high-noise environment according to the present invention for achieving the above objective is a speech recognition learning system applicable to a user terminal and a control server communicating wirelessly in a high-noise environment, wherein STT-based speech recognition technology is applied. The speech recognition learning system is characterized by comprising: a user interface for inputting a user command to register keywords defined by a user; a user keyword storage unit for storing user keywords input through the user interface; an STT-based transcription engine for transcribing speech data into text using a high-performance speech recognition model; a transcription error detection module for detecting transcription errors in the transcription results generated by the STT-based transcription engine; a retraining data configuration module for automatically generating a dataset required for retraining by extracting speech data based on words where transcription errors occurred and configuring the transcription error text and the correct text related to the speech into a set; and a performance evaluation application module for determining whether to retrain the speech recognition model by analyzing the accuracy of speech recognition based on the average similarity value for the entire transcription result of speech recognition.
[0010] In addition, the above STT-based transcription engine is characterized by using the Whispher Large V3 model as a high-performance speech recognition model and converting speech data into text according to the real-time multi-channel RTP (Real-Time Transport Protocol) protocol.
[0011] In addition, the transcription error detection module is characterized by determining whether there is an error by analyzing keyword similarity according to preset similarity criteria when a word of a text sentence transcribed by the STT-based transcription engine corresponds to a user keyword registered in the user keyword storage unit.
[0012] In addition, the above transcription error detection module is characterized by analyzing keyword similarity using a Character Error Rate (CER) analysis value as a character similarity indicator.
[0013] In addition, the above transcription error detection module is characterized by using Levenshtein distance as an auxiliary indicator as a pronunciation similarity indicator when analyzing keyword similarity.
[0014] In addition, the above-mentioned transcription error detection module is characterized by selectively setting either a manual review in which the user directly configures retraining data or an automatic error confirmation when the similarity judgment result is classified as an error candidate corresponding to the boundary between normal and error based on the CER analysis value.
[0015] In addition, the retraining data configuration module is characterized by extracting speech segments containing error words on a sentence-by-sentence or continuous-utterance basis.
[0016] In addition, the performance evaluation application module is characterized by determining whether to retrain the speech recognition model based on the average value of the overall CER analysis for the transcribed sentence of the speech recognition target.
[0017] A method for retraining STT-based speech recognition in a high-noise environment according to the present invention for achieving the above objective comprises: (a) a step of storing user keywords through a user interface; (b) a step of converting real-time speech data into text using an STT-based transcription engine for a speech recognition model; (c) a step of detecting transcription errors by comparing the transcription result generated through the STT-based transcription engine of step (b) with the registered user keywords; and (d) a step of automatically generating a dataset necessary for retraining by extracting speech data based on the word in which the detected transcription error occurred in step (c), and configuring the transcription error text and the correct text related to the speech into a set.
[0018] In addition, the method is characterized by applying the Whisper Large V3 model as a high-performance speech recognition model and applying RTP as a multi-channel communication protocol in step (b) above to perform conversion.
[0019] In addition, the method is characterized by detecting transcription errors by determining the similarity between the user keyword extracted from the transcription result and the registered user-defined keyword using CER, a character similarity performance indicator, in step (c) above.
[0020] In addition, the method is characterized by using the Levenshtein distance, a pronunciation similarity performance indicator, as an auxiliary indicator when determining similarity in step (c) above.
[0021] In addition, the method is characterized by storing the occurrence time, channel ID, CER analysis value, and comparison target keyword as metadata for the words of the sentence determined to be errors in step (c) above.
[0022] In addition, (e) the need for retraining is determined based on the error rate of the entire transcription result by the STT-based transcription engine, and the user is notified that retraining is necessary if the average value of the CER analysis for the entire transcription result is greater than or equal to a threshold value. Effects of the invention
[0023] Since the present invention performs retraining based on error data that occurs repeatedly in the user environment, the accuracy of speech recognition in specific domains of high-noise environments (air traffic control, military communications, etc.) can be substantially improved.
[0024] The present invention can reduce the burden of manual work and ensure consistency and efficiency of the retraining process by automatically configuring retraining data based on error logs that occur during the operation of STT-based speech recognition technology.
[0025] This invention enables seamless operation in a security network-based on-premise environment, allowing for the safe operation and acquisition of training data in public institutions, defense organizations, and the like where external networks are blocked.
[0026] The present invention enables learning by focusing on high-frequency regions where many errors occur, thereby overcoming the problem of excessive time required when retraining based on the entire data in existing technologies and saving retraining time and resources.
[0027] The present invention enables CER-based performance analysis and automated error tracking, and allows for the quantitative management and continuous improvement of speech recognition performance. Brief explanation of the drawing
[0028] FIG. 1 is an overall configuration block diagram of a user terminal equipped with a voice recognition function for use in a high-noise wireless communication environment and a control server according to an embodiment of the present invention. FIG. 2 is a detailed block diagram of the speech recognition learning system of FIG. 1, FIG. 3 is an overall flowchart illustrating a retraining method for STT-based speech recognition in a high-noise environment according to an embodiment of the present invention, Figure 4 is a detailed flowchart of the user-defined keyword registration step of Figure 3, FIG. 5 is a detailed flowchart of the transcription result analysis and error detection steps of FIG. 3. FIG. 6 is a detailed flowchart of the automatic retraining data configuration step of FIG. 3. Figure 7 is a detailed flowchart of the performance evaluation and retraining linkage step of Figure 3. Specific details for implementing the invention
[0029] The present invention will be explained below by describing embodiments of the present invention with reference to the attached drawings.
[0030] The embodiment can be applied to high-noise wireless communication environments with high error rates in voice recognition, such as air traffic control and military wireless communication fields.
[0031] Referring to FIG. 1, a user can perform wireless communication with a control server (2) using a user terminal (1), and, for example, an air pilot can perform wireless communication with a control server (2) of a control tower using a user terminal (1) to which STT-based speech recognition technology is applied. To this end, the user terminal (1) and the control server (2) may be commonly equipped with a communication module (10) for wireless communication and a speech recognition learning system (100) for implementing a speech recognition function.
[0032] A speech recognition learning system (100) may include a speech signal preprocessing unit (110), a user interface (120), a user keyword storage unit (130), an STT-based transcription engine (140), a transcription error detection module (150), a retraining data configuration module (160), and a performance evaluation application module (170).
[0033] The voice signal preprocessing unit (110) can preprocess the voice signal. Specifically, the voice signal preprocessing unit (110) can perform preprocessing to increase the voice signal recognition rate of the voice recognition model. For example, the voice signal preprocessing unit (110) can preprocess the voice signal using a low-pass filter or a high-pass filter.
[0034] The user interface (120) inputs a user command to register a user-defined word, and the input user-defined word is stored as a user keyword in the user keyword storage unit (130), and at this time, the importance grade and classification information of the registered word may be stored together. In this way, the user can register important words for which transcription accuracy must be ensured as user keywords through the user interface (120).
[0035] The STT-based transcription engine (140) performs a transcription function that converts voice data into text using a high-performance speech recognition model, and the Whispher Large V3 model can be used as the high-performance speech recognition model.
[0036] The STT-based transcription engine (140) converts real-time multi-channel RTP (Real-Time Transport Protocol) voice data into text and generates a transcription result file containing each channel and time information.
[0037] The transcription error detection module (150) searches for user keywords registered in the user keyword storage unit (130) for words of a text sentence transcribed by the STT-based transcription engine (140), and analyzes keyword similarity according to preset similarity criteria. In the embodiment, the similarity criteria for determining whether there is an error in the words of the transcribed text sentence by the transcription error detection module (150) may apply the Character Error Rate (CER) as a character similarity indicator, and additionally, the Levenshtein distance may be applied as an auxiliary indicator as a pronunciation similarity indicator.
[0038] The transcription error detection module (150) determines the possibility of a transcription error based on the result of keyword similarity analysis and logs metadata including the occurrence time, channel ID, CER analysis value, and comparison target keyword of the word in the transcribed text sentence determined to be an error. This metadata of the error word can be used as a standard for future high-frequency error word analysis and retraining data configuration.
[0039] The retraining data configuration module (160) can extract voice data of a voice segment containing a word that has an error and automatically generate a dataset necessary for retraining, including transcribed error text and normal text related to the voice. The retraining dataset can be configured to include three items consisting of input voice, error transcribed text, and correct text.
[0040] The performance evaluation application module (170) can analyze the accuracy of speech recognition based on the average similarity value for the entire transcription result for speech recognition. It can determine whether the recognition model needs to be retrained based on the average value of the overall CER analysis for the transcribed sentences of the speech recognition target. For example, it can determine whether the voice recognition model (Whispher Large V3 model) used in the speech recognition learning system (100) of the user terminal (1) and the control server (2) needs to be retrained by analyzing the recognition accuracy of the voice recognition model. The generated retraining dataset is transmitted to a separate voice recognition model learning server (not shown), and an administrator can manually perform retraining on the voice recognition model.
[0041] Referring to FIG. 3, a retraining method for STT-based speech recognition in a high-noise environment according to an embodiment of the present invention may include a user-defined keyword registration step (200), a transcription result analysis and error detection step (210), a retraining data automatic configuration step (220), and a performance evaluation and retraining linkage step (230).
[0042] Referring to Fig. 4, the user-defined keyword registration step (200) is described.
[0043] User keywords are entered through a user interface (120) implemented via the web or a dedicated program (201). The user can directly register and set key words or expressions for which the accuracy of voice recognition is particularly important as keywords. For example, domain-specific commands of the language model or high-frequency nouns such as 'command post', 'launch', and 'return' can be registered as user keywords.
[0044] User keywords entered according to user commands can be stored in a structured form rather than as a simple string. User keywords can be structured according to information linked to importance grades, classifications, user groups, or work types (202).
[0045] Structuring user-defined keywords [Example 1]
[0046] * Keyword: 'Command Post'
[0047] * Importance Grade: High
[0048] * Category: Command / Operation / Report
[0049] * User group or work type: Fire Department
[0050] Structured user keywords are stored in the keyword list of the user-dedicated database (DB) of the user keyword storage unit (130). The user-dedicated database (DB) can be updated in real time. When analyzing company-wide errors, the company-wide error detection module (150) determines whether there is an error by referring to the user keywords in the keyword list.
[0051] Referring to Fig. 5, the transcription result analysis and error detection step (210) is explained.
[0052] The STT-based transcription engine (140) converts voice data into text using a speech recognition model (Whispher Large V3 model) and generates a transcription result file containing each channel and time information.
[0053] The transcription error detection module (150) collects multi-channel text transcription results generated by the STT-based transcription engine (140). The collected transcription results include channel ID, time information, and text content (211).
[0054] Next, the transcription error detection module (150) searches for the presence of user keywords in the transcribed text sentence and determines that there is a high probability of error if the keyword similarity is low. It may apply the Character Error Rate (CER) as a character similarity indicator and additionally apply the Levenshtein distance as a pronunciation similarity indicator (212).
[0055] CER-based error judgment [Example 2]
[0056] * Normal recognition threshold 0.2 / Error recognition threshold 0.3
[0057] * CER analysis value < 0.2: Recognized as normal
[0058] * 0.2 ≤ CER analysis value < 0.3: Classified as error candidate
[0059] * CER analysis value ≥ 0.3: Error detected (low similarity)
[0060] Next, the transcription error detection module (150) records the occurrence time, channel ID, CER analysis value, comparison target keyword, and log information as metadata for the words of the sentences determined to be errors in the metadata database (DB) (213).
[0061] When the CER analysis value corresponds to the error candidate boundary value (0.2 ≤ CER analysis value < 0.3), either manual review or automatic error confirmation can be selectively set for the classified error candidate (214).
[0062] Referring to Fig. 6, the steps for automatically configuring retraining data are explained.
[0063] The retraining data configuration module (160) automatically extracts a speech segment of a certain length, for example ±3 seconds (sec), based on a sentence or context containing an error word (221). The extraction range of this speech segment can be set by the user. Preferably, considering the problem of interrupting the speech flow when only words are cut out, the extraction is performed in sentence units or continuous speech units.
[0064] Next, the retraining data configuration module (160) configures the correct text for the error word. The correct text can be configured based on user keywords registered by the user or predefined reference sentence data (222). Optionally, an administrator may manually input the correct text or provide automatic recommendation candidates to specify the desired correct text.
[0065] Next, the retraining data configuration module (160) can automatically generate a dataset required for retraining, including transcription error text and normal text related to the corresponding voice. The retraining dataset can be configured as a single set including three items consisting of input voice, error transcription text, and correct text (223).
[0066] Re-learning dataset [Example 3]
[0067] * Input audio: audio.wav
[0068] * Error Transcription Text : wrong.txt
[0069] * Correct answer text : correct.txt
[0070] The performance evaluation application module (170) can determine whether retraining is necessary based on the error rate of the overall transcription result by the STT-based transcription engine (140) and perform a retraining linkage operation. These performance evaluation and retraining linkage steps are explained with reference to FIG. 7.
[0071] First, the performance evaluation application module (170) can calculate the average value of the CER analysis for the entire transcript result and, based on this, quantitatively analyze the recognition accuracy of the voice recognition model. Based on the quantitative analysis information, the trend of CER change and the error distribution of specific channels and time periods can be identified (231).
[0072] If the average value of the CER analysis is 0.2 or higher, the performance evaluation application module (170) recognizes that performance has degraded and notifies the user through the user interface (120) that retraining is required (232).
[0073] In the embodiment, a dataset for retraining can be automatically configured by the retraining data configuration module (160) in response to an event caused by transcription error detection, but automatic fine-tuning of the STT-based transcription engine (140) is not performed unconditionally in response to a transcription error event. This is in consideration of policy design guidelines for ensuring model stability and verification procedures in a secure network environment.
[0074] The performance evaluation application module (170) transmits the retraining dataset generated by the retraining data configuration module (160) to a separate voice recognition model training server and allows for manual retraining to be performed after review by an administrator (233).
[0075] The following describes, in an exemplary manner, a series of processes for automatically configuring speech recognition retraining data according to an embodiment of the present invention.
[0076] Data configuration process for retraining [Example 4]
[0077] situation : It is assumed that the recognition accuracy of the word 'command post' is set as a very important keyword in a 119 situation room or military communication environment. It is assumed that the user registers the keyword in the user keyword storage unit (120) and that the STT-based transcription engine (140) converts the voice into text in real time.
[0078] ① Voice Input and Transcription
[0079] The instruction to "return to the command post immediately" was transmitted from the field radio, but the output of the STT-based transmission engine (140) was "return to the command post immediately."
[0080] ② Error Detection
[0081] The transcription error detection module (150) compares the transcription text with the user-registered keyword 'command post' and determines that a transcription error has occurred if the CER analysis value of the word is 0.25.
[0082] ③ Metadata Recording
[0083] The total error detection module (150) records the time of occurrence of the total error, channel ID, CER analysis value, comparison target keyword, and log information in the metadata database (DB).
[0084] ④ Configuration of retraining dataset
[0085] Audio clips are extracted based on the voice segment of ±3 seconds where the error occurred, and the transcribed error text 'Return to the base immediately' and the correct text 'Return to the command post immediately' are constructed to form a retraining dataset as follows.
[0086] - audio_001.wav
[0087] - audio_001.wrong.txt
[0088] - audio_001.correct.txt
[0089] ⑤ Implementation of linked relearning
[0090] The generated retraining dataset is linked to the retraining pipeline of the STT-based transcription engine (140) and can be used in the fine-tuning process after manual approval.
[0091] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that the present invention can be easily modified into other specific forms without changing the technical concept or essential features of the invention. Industrial applicability
[0092] The present invention has the potential to be used for speech recognition retraining that can improve the accuracy of STT-based speech recognition by automatically detecting transcription errors according to user-defined criteria when converting speech to text and configuring a dataset for retraining. Explanation of the symbols
[0093] 100: Speech Recognition Learning System 110 : Voice signal preprocessing unit 120 : User Interface 130 : User Keyword Storage 140 : STT-based transcription engine 150: Transcription Error Detection Module 160: Retraining Data Configuration Module 170: Performance Evaluation Application Module
Claims
Claim 1 A speech recognition learning system applicable to a user terminal and a control server communicating wirelessly in a high-noise environment, wherein STT-based speech recognition technology is applied, and the speech recognition learning system comprises: a user interface for inputting a user command to register keywords defined by a user; a user keyword storage unit for storing user keywords input through the user interface; an STT-based transcription engine for transcribing speech data into text using a high-performance speech recognition model; a transcription error detection module for detecting transcription errors in the transcription results generated by the STT-based transcription engine; a retraining data configuration module for automatically generating a dataset required for retraining by extracting speech data based on words where transcription errors occurred and configuring the transcription error text and correct text related to the speech into a set; and a performance evaluation application module for determining whether to retrain the speech recognition model by analyzing the accuracy of speech recognition based on the average similarity value for the entire transcription result of speech recognition. Claim 2 A STT-based speech recognition retraining device in a high-noise environment, characterized in that, in claim 1, the STT-based transcription engine uses the Whispher Large V3 model as a high-performance speech recognition model and converts speech data into text according to a real-time multi-channel RTP (Real-Time Transport Protocol) protocol. Claim 3 A STT-based speech recognition retraining device in a high-noise environment, characterized in that, in claim 1, the transcription error detection module determines whether there is an error by analyzing keyword similarity according to a preset similarity criterion when a word of a text sentence transcribed by the STT-based transcription engine corresponds to a user keyword registered in the user keyword storage unit. Claim 4 In paragraph 3, the STT-based speech recognition retraining device in a high-noise environment is characterized in that the transcription error detection module analyzes keyword similarity using a CER (Character Error Rate) analysis value as a character similarity indicator. Claim 5 In claim 4, the above transcription error detection module is characterized by using Levenshtein distance as an auxiliary indicator as a pronunciation similarity indicator during keyword similarity analysis, in an STT-based speech recognition retraining device in a high-noise environment. Claim 6 In paragraph 3, the STT-based speech recognition retraining device in a high-noise environment is characterized by the fact that, when the transcription error detection module classifies a similarity judgment result based on a CER analysis value as an error candidate corresponding to the boundary between normal and error, the user selectively sets either a manual review to configure retraining data or an automatic error confirmation. Claim 7 A STT-based speech recognition retraining device in a high-noise environment, wherein, in claim 1, the retraining data configuration module extracts speech segments containing error words in sentence units or continuous utterance units. Claim 8 A STT-based speech recognition retraining device in a high-noise environment, characterized in that, in claim 1, the performance evaluation application module determines whether to retrain the speech recognition model based on the average value of the overall CER analysis for the transcribed sentence of the speech recognition target. Claim 9 (a) a step of storing user keywords through a user interface; (b) a step of converting real-time voice data into text using an STT-based transcription engine for a voice recognition model; (c) a step of detecting transcription errors by comparing the transcription result generated through the STT-based transcription engine of step (b) with the registered user keywords; (d) a step of extracting voice data based on the word in which the detected transcription error occurred in step (c), and forming a set of transcription error text and correct text related to the voice to automatically generate a dataset necessary for retraining; characterized by comprising an STT-based voice recognition retraining method in a high-noise environment. Claim 10 A method for retraining STT-based speech recognition in a high-noise environment, characterized in that, in step (b) above, the Whisper Large V3 model is applied as a high-performance speech recognition model and RTP is applied as a multi-channel communication protocol for conversion. Claim 11 A method for retraining STT-based speech recognition in a high-noise environment, characterized in that, in claim 9, transcription errors are detected by determining the similarity between user keywords extracted from transcription results and registered user-defined keywords using CER, a character similarity performance indicator, in step (c) above. Claim 12 A method for retraining STT-based speech recognition in a high-noise environment, characterized in that, in claim 11, the Levenshtein distance, which is a pronunciation similarity performance indicator, is used as an auxiliary indicator when determining similarity in step (c) above. Claim 13 A method for retraining STT-based speech recognition in a high-noise environment, characterized in that, in claim 9, the occurrence time, channel ID, CER analysis value, and comparison target keyword are stored as metadata for the words of the sentences determined to be errors in step (c) above. Claim 14 In claim 9, (e) a method for retraining STT-based speech recognition in a high-noise environment, characterized by determining whether retraining is necessary based on the error rate of the entire transcription result by the STT-based transcription engine, and notifying the user that retraining is necessary if the average value of the CER analysis for the entire transcription result is greater than or equal to a reference value.
