An offline real-time voice transcription optimization method and device for a domestic operating system

Abstract

The application discloses an offline real-time voice transcription optimization method and device for a domestic operating system and belongs to the voice transcription field. The application provides an offline real-time voice transcription optimization method for a domestic operating system. The method realizes dynamic adjustment of voice cutting length and output speed through segmentation and recognition of original audio signals, enhances the real-time performance of voice transcription while ensuring transcription accuracy, and improves user experience. After obtaining preliminary transcription text, real-time word correction is performed to further improve the accuracy of transcription results. The transcription text after correction is post-processed to obtain offline real-time voice transcription results, proper punctuation marks are automatically added to the transcription results according to semantics to improve the readability of the text, and the voice transcription process is automatically stopped, so that the transcription process is more in line with the intention and use habit of users. The offline real-time voice transcription optimization method for a domestic operating system provided by the application does not need to rely on the Internet and can stably run in a network-free environment, is suitable for various application scenarios, effectively avoids the risk of data leakage and privacy intrusion, and guarantees the privacy security of users.

Description

Technical Field

[0001] This application relates to the field of speech transcription, and in particular to an offline real-time speech transcription optimization method and apparatus for domestic operating systems. Background Technology

[0002] In today's digital age, real-time speech transcription technology, as an important tool for information exchange, has been widely applied in various scenarios, including meeting recording, real-time caption generation, virtual assistants, and voice input. With the continuous development of artificial intelligence and natural language processing technologies, speech transcription technology has made significant progress. However, as the demands of different scenarios become more complex, this technology still faces many challenges in terms of accuracy, response speed, and applicability.

[0003] First, when processing long periods of continuous voice input, the instability of the voice signal and interference from environmental noise often lead to a decrease in recognition accuracy. Especially in complex acoustic environments, such as noisy background noise or multi-party conversations, the performance of traditional voice recognition systems is unsatisfactory. Current solutions mostly rely on complex voice processing algorithms, which, while improving accuracy, still have shortcomings in terms of computational resource consumption and ease of operation. Furthermore, in scenarios with high real-time requirements, such as real-time conference translation or subtitle generation, slow response times directly impact user experience, while a relentless pursuit of fast response times may lead to a decrease in recognition accuracy.

[0004] Secondly, compatibility issues with multiple languages ​​and regional environments are also a bottleneck in existing speech transcription technologies. For example, current technology struggles to fully meet the needs of users in different regions when converting between traditional and simplified Chinese characters. Furthermore, during real-time speech transcription, the emergence of new words, slang, or slips of the tongue often exceeds the coverage of the system's dictionary and grammatical rules, thus affecting the transcription quality.

[0005] Furthermore, text generated from speech transcription often lacks proper punctuation, which not only affects readability but also reduces its value for subsequent use. Traditional systems, relying on static rules for punctuation addition, struggle to generate text that conforms to logical expression based on context. By introducing context-based automatic punctuation addition technology, not only can text quality be significantly improved, but the user's reading experience can also be enhanced.

[0006] Therefore, effectively improving the accuracy, response speed, and naturalness of text generation in real-time speech transcription technology under complex environments has become a key research direction. This requires combining advanced speech recognition algorithms, multilingual compatible designs, and intelligent text processing technologies to promote the practical application of speech transcription technology in more scenarios. Summary of the Invention

[0007] This application provides an offline real-time speech transcription optimization method and apparatus for domestic operating systems.

[0008] On the one hand, an offline real-time speech transcription optimization method for domestic operating systems is provided, the method comprising:

[0009] The raw audio signal is acquired and segmented and recognized to obtain preliminary transcribed text. The raw audio signal includes a custom long audio signal and a custom short audio signal.

[0010] The preliminary transcribed text is subjected to real-time word correction to obtain the corrected transcribed text;

[0011] The transcribed text after error correction is post-processed to obtain offline real-time speech transcription results. The post-processing includes potential pauses, sentence boundary judgment, and speech transcription process stop judgment.

[0012] In response to the detection that the original audio signal is the custom short audio signal, the custom short audio signal is segmented and identified using a preset audio sampling rate and a preset buffer.

[0013] Optionally, the step of acquiring the original audio signal, performing segmentation and recognition operations to obtain preliminary transcribed text includes:

[0014] Receive the raw audio signal acquired by the acquisition terminal;

[0015] The original audio signal is preprocessed to obtain the enhanced audio signal;

[0016] The enhanced audio signal is segmented and spliced ​​together based on the voice activity detection results to obtain the audio signal to be recognized;

[0017] The audio signal to be identified is subjected to speech recognition and converted into the preliminary transcribed text.

[0018] Optionally, the step of segmenting and splicing the enhanced audio signal according to the speech activity detection result to obtain the audio signal to be recognized includes:

[0019] Based on the speech activity detection results, the enhanced audio signal is segmented into several speech segments, wherein the segmentation is based on natural pauses or a custom time length.

[0020] Adjacent speech segments are spliced ​​together to obtain the audio signal to be identified, wherein the adjacent speech segments have an overlap of a custom length at the splicing point.

[0021] Optionally, the step of performing speech recognition on the audio signal to be recognized and converting it into the preliminary transcribed text includes:

[0022] The overlapping portions of the audio signal to be identified are identified using a sliding window mode;

[0023] The overlapping portions are removed to obtain the preliminary transcribed text.

[0024] Optionally, the step of performing real-time word correction on the initial transcribed text to obtain the corrected transcribed text includes:

[0025] The preliminary transcribed text was checked word by word for grammar and spelling using word segmentation technology;

[0026] Use contextual information to identify words that do not conform to grammar or semantics and mark them as uncertain words;

[0027] Calculate the edit distance of the uncertain words and generate a set of candidate corrected words by combining them with context information;

[0028] The candidate correction words are sorted, and the correction words of the best match are selected and replaced in the preliminary transcribed text to obtain the error-corrected transcribed text.

[0029] Optionally, the post-processing of the corrected transcribed text to obtain offline real-time speech transcription results includes:

[0030] The grammatical structure, themes, and contextual semantics of the corrected transcribed text were analyzed to identify potential pauses and sentence boundaries.

[0031] The system analyzes the response level of the original audio signal in real time during speech input, and identifies significant pauses or emphasis changes, as well as rhythm and pitch changes, in the speech stream corresponding to the original audio signal.

[0032] The identification of significant pauses or emphasis changes, as well as rhythm and tone variations, helps determine potential pauses and sentence boundaries.

[0033] Punctuation suggestions are generated for the post-corrected transcribed text based on potential pauses and sentence boundaries using natural language processing technology.

[0034] Based on the pause length threshold specified by the user, the silence duration in the original audio signal is monitored in real time.

[0035] In response to the silence duration in the original audio signal exceeding the user-defined silence duration, the automatic stop function of the speech transcription process is triggered.

[0036] Optionally, after obtaining the raw audio signal, performing segmentation and recognition operations to obtain the preliminary transcribed text, the method further includes:

[0037] The confidence score C(x) for each speech segment is calculated, and repetition recognition and adjustment are performed on each speech segment. The formula for calculating the confidence score C(x) is as follows: x represents the recognized speech segment, P i (x) represents the confidence probability of the i-th model for x, and N is the total number of evaluation methods;

[0038] In response to the detection of a speech segment whose confidence score C(x) is lower than the target threshold θ, an automatic adjustment processing strategy is executed.

[0039] Optionally, in response to detecting a speech segment with a confidence score C(x) lower than a target threshold θ, the automatic adjustment processing strategy is executed, including:

[0040] For speech segments with a confidence score C(x) below the target threshold θ, increase the audio length L during repeated recognition. repeat , where L repeat =L base +α×(L complex -L base ), L base It is the base processing length, L complex It is the ideal processing length after complexity assessment, and α is an adjustment coefficient with a value between 0 and 1.

[0041] On the other hand, an offline real-time speech transcription optimization device for domestic operating systems is provided, the device comprising:

[0042] The signal processing module is used to acquire the raw audio signal for segmentation and recognition operations to obtain preliminary transcribed text. The raw audio signal includes a custom long audio signal and a custom short audio signal.

[0043] The word correction module is used to perform real-time word correction on the preliminary transcribed text to obtain the corrected transcribed text.

[0044] The post-processing module is used to post-process the transcribed text after error correction to obtain offline real-time speech transcription results. The post-processing includes potential pauses, sentence boundary judgment, and speech transcription process stop judgment.

[0045] The short audio processing module is used to segment and identify the custom short audio signal in response to the detection that the original audio signal is the custom short audio signal, by using a preset audio sampling rate and a preset buffer.

[0046] On the other hand, a computer-readable storage medium is provided, the storage medium storing at least one instruction, the at least one instruction being executed by a processor to implement the offline real-time speech transcription optimization method for a domestic operating system as described above.

[0047] On the other hand, a computer program product is also provided, which stores at least one instruction, which is loaded and executed by the processor to implement the offline real-time speech transcription optimization method for domestic operating systems described above.

[0048] The technical effects brought about by this application are at least as follows.

[0049] This application provides an offline real-time speech transcription optimization method for domestic operating systems. By segmenting and recognizing the original audio signal, it dynamically adjusts the speech trimming length and output speed, enhancing real-time performance and improving user experience while maintaining transcription accuracy. Real-time word correction is performed after obtaining the initial transcribed text, further improving the accuracy of the transcription results. Post-processing of the corrected transcribed text yields the offline real-time speech transcription result. Appropriate punctuation is automatically added to the transcription result based on semantics, improving readability. The method also automatically stops the speech transcription process, making it more aligned with user intent and usage habits. This offline real-time speech transcription optimization method for domestic operating systems does not rely on the internet and can run stably in offline environments. It is suitable for various application scenarios, effectively avoiding the risks of data leakage and privacy intrusion, and protecting user privacy and security.

[0050] In other embodiments, the method further improves the user experience by adjusting the number of speech repetitions and the output speed according to different user audio input densities.

[0051] In other embodiments, a speech transcription mode using a sliding window is also included to perform repetition recognition and real-time text error correction. At the same time, traditional Chinese characters in the transcribed text are automatically recognized and converted to simplified Chinese characters, further improving the accuracy of the transcription results.

[0052] Other embodiments include more accurately determining audio segment boundaries by real-time monitoring of the loudness of the speech input and performing contextual analysis of the speech. This effectively filters background noise, adapts to complex application scenarios, and makes the transcription results more closely resemble the user's actual semantics. Attached Figure Description

[0053] Figure 1 A flowchart illustrating an exemplary embodiment of this application shows an offline real-time speech transcription optimization method for a domestically developed operating system.

[0054] Figure 2 The corresponding Figure 1 A diagram illustrating the interaction process between users and the AI ​​assistant;

[0055] Figure 3 This is a schematic diagram illustrating the structure of each module corresponding to an embodiment of this application;

[0056] Figure 4 The figures show the test results under quiet and noisy environments;

[0057] Figure 5 The graph shows the test results at different speaking speeds. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0059] In this article, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0060] Example 1

[0061] Please refer to Figure 1 It illustrates a flowchart of an offline real-time speech transcription optimization method for a domestic operating system provided in an exemplary embodiment of this application. Figure 2 The corresponding Figure 1 A diagram illustrating the user's interaction process with the AI ​​assistant, the method including:

[0062] Step 101: Obtain the original audio signal for segmentation and recognition to obtain preliminary transcribed text.

[0063] In one possible implementation, step 101 includes the following steps.

[0064] Step S11: Receive the raw audio signal collected by the acquisition terminal.

[0065] The original audio signal includes custom long audio signals and custom short audio signals, such as a 10-second limit, which is not specified.

[0066] Step S12: Preprocess the original audio signal to obtain the enhanced audio signal.

[0067] The preprocessing of the raw audio signal can be as follows: the processing system first receives the raw audio signal input from the acquisition end, such as a microphone, and analyzes the amplitude distribution and energy density of the input raw audio signal through Fast Fourier Transform (FFT) to provide data support for subsequent amplitude adjustment.

[0068] Furthermore, the dynamic range of the input signal is calculated, the peak amplitude is identified, and the overall power level of the signal is evaluated as a reference for determining a uniform amplitude range. Further, based on the calculated dynamic range, the signal amplitude is adjusted using linear scaling or dynamic compression techniques to map the signal amplitude to the standard range.

[0069] The above processing ensures that all audio signal data is within the system's processing range, which helps reduce distortion and enhance the signal-to-noise ratio.

[0070] Step S13: Based on the speech activity detection results, the enhanced audio signal is segmented and then spliced ​​to obtain the audio signal to be recognized.

[0071] Based on the speech activity detection results, the enhanced audio signal is segmented into several speech segments, with the segmentation based on natural pauses or a custom time length.

[0072] The process of voice activity detection is as follows: After receiving the voice signal, the microphone at the acquisition end saves the voice for synchronous detection. The detection process mainly includes performing a Fast Fourier Transform (FFT) on the input voice, analyzing the amplitude distribution and energy density of the input audio signal, calculating the peak amplitude, and assessing the overall power level; adjusting the overall signal amplitude through linear transformation to tune the signal to a certain range.

[0073] It should be noted that the preset long audio segment is generally defined as audio exceeding 1 minute according to actual needs, such as in scenarios like meetings and speeches. Of course, those skilled in the art can define the duration according to actual needs, and there is no limitation on this.

[0074] Adjacent speech segments are spliced ​​together to obtain the audio signal to be recognized. There is a custom-length overlap at the splicing point between adjacent speech segments.

[0075] In one example, for a preset long audio segment, the backend will extract 10 seconds of the preceding audio segment for duplicate detection, so the length of the repeated audio segment is 10 seconds. Of course, those skilled in the art can define the detection duration of the overlapping part according to actual needs, and it is not limited here.

[0076] Step S14: Perform speech recognition on the audio signal to be recognized and convert it into preliminary transcribed text.

[0077] In one possible implementation, the audio signal to be identified is recognized and converted into text, the text is integrated, and the overlapping parts between the two audio text segments are removed. Specifically, the overlapping parts in the audio signal to be identified are identified in a sliding window mode, and then the overlapping parts are removed to obtain the preliminary transcribed text. This operation can enhance the accuracy of audio recognition at the end of each audio segment while better describing the context of the entire audio segment, serving as a basis for adding appropriate punctuation marks.

[0078] Step 102: Perform real-time word correction on the initial transcribed text to obtain the corrected transcribed text.

[0079] In one possible implementation, step 102 includes the following steps.

[0080] Step S21: Use word segmentation technology to perform word-by-word grammar and spelling checks on the preliminary transcribed text.

[0081] Step S22: Use contextual information to identify words that do not conform to grammar or semantics and mark them as uncertain words;

[0082] Step S23: Calculate the edit distance of uncertain words and generate a set of candidate correction words by combining context information.

[0083] In one possible implementation, basic grammar and spelling checks are performed on the initial transcribed text in real time, such as using word segmentation techniques to break the text down into individual words and phrases to facilitate word-by-word error detection. Contextual information is used to identify words that do not conform to grammar or semantics; these words are marked as uncertain words, i.e., potential errors. The edit distance of uncertain words is calculated, and a set of candidate corrective words, which can be in the form of words or phrases, is generated based on the context. Using context to predict candidate corrective words can improve overall semantic consistency.

[0084] Step S24: Sort the candidate correction words, select the correction words of the best match and replace them in the initial transcribed text to obtain the corrected transcribed text.

[0085] Furthermore, considering common error patterns in speech recognition, the candidate correction words are sorted, and the option most likely to match the context is displayed first. High-confidence error correction suggestions are automatically applied and directly replaced in the transcribed text. That is, the correction words of the best match are selected and replaced in the initial transcribed text to obtain the error-corrected transcribed text.

[0086] In addition, based on different regions and individual usage habits, it is also possible to add content that checks for traditional Chinese characters in the text and converts them into simplified Chinese characters.

[0087] Step 103: Post-process the transcribed text after error correction to obtain offline real-time speech transcription results. Post-processing includes potential pauses, sentence boundary judgment, and speech transcription process termination judgment.

[0088] In one possible implementation, step 103 includes the following steps.

[0089] Step S31: Analyze the grammatical structure, themes, and contextual semantics in the transcribed text after error correction to identify potential pauses and sentence boundaries.

[0090] Step S32: Analyze the response level of the original audio signal in real time during speech input, and identify significant pauses or emphasis changes, as well as rhythm and pitch changes in the speech stream corresponding to the original audio signal.

[0091] Step S33: Based on the identified significant pauses or emphasis changes, as well as rhythm and tone changes, help determine potential pauses and sentence boundaries.

[0092] This step is for speech feature analysis. Specifically, after the initial completion of speech transcription recognition, the corrected transcribed text is obtained. Post-processing is then performed to analyze the grammatical structure, topic, and contextual semantics in the corrected transcribed text, and to identify potential pauses and sentence boundaries. Further, the loudness level of the speech input is analyzed in real time to identify significant pauses or emphasis changes in the speech flow, detect speech rhythm and pitch changes, and assist in judging potential pauses and sentence boundaries, that is, to judge potential punctuation positions to prepare for the next step, such as predicting interrogative sentences based on rising pitch.

[0093] Step S34: Using natural language processing, punctuation suggestions are generated for the corrected transcribed text based on potential pauses and sentence boundaries.

[0094] In one possible implementation, natural language processing techniques are used, combined with contextual information, to generate punctuation suggestions for text segmentation points (i.e., potential pauses and sentence boundaries), and to add appropriate punctuation such as periods, commas, question marks, and exclamation marks for common sentence patterns and semantic patterns.

[0095] Step S35: Monitor the silence duration in the original audio signal in real time according to the pause length threshold specified by the user.

[0096] Step S36: In response to the silence duration in the original audio signal exceeding the user-defined silence duration, the automatic stop function of the speech transcription process is triggered.

[0097] In one possible implementation, the system sends a stop signal to the speech transcription process, pausing the transcription and promptly saving the current offline real-time speech transcription results.

[0098] Furthermore, considering the need for timely response to short audio messages, embodiments of this application also provide a method for processing short audio signals.

[0099] Step 104: In response to the detection that the original audio signal is a custom short audio signal, the custom short audio signal is segmented and identified by a preset audio sampling rate and a preset buffer.

[0100] In one example, fast speech recognition is performed on the raw audio signal using a preset audio sampling rate of 16kHz and a preset buffer of 0.5 seconds.

[0101] In response to the detection that the original audio signal is a short audio signal, the original audio signal is quickly recognized for speech using a preset audio sampling rate of 16kHz and a buffer of 0.5 seconds.

[0102] In one possible implementation, when user input is detected, a shorter audio segment is truncated for speech recognition. The original audio signal is then subjected to fast speech recognition, i.e., FFT processing, using a preset 16kHz audio sampling rate and a 0.5-second buffer. The 16kHz audio sampling rate and 0.5-second buffer can be user-defined or system default settings.

[0103] It should be noted that, since the length of the short audio signal is also determined by the input of the acquisition end, in order to improve the response rate of short speech, in one possible implementation, the segmentation and recognition operation of the custom short audio signal mentioned in step 104 also needs to be performed on short speech as in steps 101 to 103, including segmentation, word correction, and real-time output after processing. For example, within 10 seconds, the first 4 seconds will be cut off for rapid recognition and the recognition result will be output. After subsequent splicing, the recognition will be repeated and the output result will be corrected to ensure the immediacy of the user's input to output.

[0104] Therefore, the embodiments of this application ensure that the system can quickly analyze short audio signals, and the recognized text will be output in real time through a circular buffer mechanism to quickly update the user interface. For example, if the initial response time is set to 2 seconds, the user can see the preliminary transcription results within 3 seconds.

[0105] This application provides an offline real-time speech transcription optimization method for domestic operating systems. By segmenting and recognizing the original audio signal, it dynamically adjusts the speech trimming length and output speed, enhancing real-time performance and improving user experience while maintaining transcription accuracy. Real-time word correction is performed after obtaining the initial transcribed text, further improving the accuracy of the transcription results. Post-processing of the corrected transcribed text yields the offline real-time speech transcription result. Appropriate punctuation is automatically added to the transcription result based on semantics, improving readability. The method also automatically stops the speech transcription process, making it more aligned with user intent and usage habits. This offline real-time speech transcription optimization method for domestic operating systems does not rely on the internet and can run stably in offline environments. It is suitable for various application scenarios, effectively avoiding the risks of data leakage and privacy intrusion, and protecting user privacy and security.

[0106] In addition, it also includes adjusting the number of speech repetitions and output speed according to different user audio input densities, further improving the user experience.

[0107] In addition, it includes a speech transcription mode using a sliding window, which performs repetition recognition and corrects text errors in real time. At the same time, it automatically identifies and converts traditional Chinese characters in the transcribed text to simplified Chinese characters, further improving the accuracy of the transcription results.

[0108] Furthermore, it includes more accurately determining the boundary points of audio segments by monitoring the loudness of the voice input in real time and performing contextual analysis of the speech. It effectively filters background noise, adapts to complex application scenarios, and makes the transcription results closer to the user's actual semantics.

[0109] Example 2

[0110] In addition, to further improve the accuracy of speech-to-text transcription, this application embodiment also provides a repetition recognition and adjustment mechanism. The method step 101 corresponding to this mechanism is as follows.

[0111] The confidence score C(x) for each speech segment is calculated, and repetition recognition and adjustment are performed on each speech segment. The formula for calculating the confidence score C(x) is as follows: x represents the recognized speech segment, P i C(x) represents the confidence probability of the i-th model for x, and N is the total number of evaluation methods. An automatic adjustment processing strategy is executed in response to the detection of speech segments with a confidence score C(x) lower than the target threshold θ. In one example, the model uses the open-source speech recognition model Whisper from OpenAI.

[0112] Furthermore, for speech segments with a confidence score C(x) below the target threshold θ, the audio length L during repeated recognition is increased.repeat , where L repeat =L base +α×(L complex -L base ), L base It is the base processing length, L complex This represents the ideal processing length after complexity assessment, and α is an adjustment coefficient ranging from 0 to 1. When a text segment with a confidence level below a certain threshold θ is detected (i.e., C(x) << θ), the processing strategy is automatically adjusted. The value of α is dynamically adjusted based on the complexity evaluation.

[0113] Furthermore, adjust the text output speed. Based on normal speaking speed, set a standard speech rate (v). standard The speech rate is 160 words per minute (WPM), and a buffer time Δt is set for fast-paced speech scenarios. base The latency is set to 200 milliseconds to ensure stable text output in most environments; the adjustment factor γ is set to 0.2 seconds to fine-tune the sensitivity of the delay to changes in speech rate. For low-speed speech scenarios, the base L is set... base The adjustment coefficient β is set to 0.5 seconds to determine the extent to which the recognition length is increased when the speech rate is lower than the standard. The adjustment coefficient is adjusted when the user input speed (v...)... user (greater than v) standard At that time, the buffer time (Δt) output ) decreases to Δt base 70%-80%; when v user Less than v standard At the same time, the recognition length is increased by a calculation formula, while the update time of the text output is slowed down to ensure high accuracy of speech recognition:

[0114] Δt output =Δt base +γ×(v standard -v user );

[0115] L repeat =L base +β×(v standard -v user ).

[0116] The following explanation is provided for cases where the value is below the threshold. If the value is below the threshold, it indicates that the recognition effect of the segment is poor and further recognition is required.

[0117] Low confidence levels can occur due to, but are not limited to, the presence of unfiltered background noise, insufficient speech clarity, and a high word error rate. The primary approach to addressing low-confidence speech segments is to increase the length of repeated audio segments, aiming to improve accuracy by repeatedly recognizing longer audio passages. However, no specific response strategies are implemented for each individual situation.

[0118] The process of determining complexity is explained below.

[0119] In this application, the complexity assessment is mainly reflected in the processing of low-confidence-scoring speech segments and the repetition recognition mechanism, with the specific steps as follows.

[0120] Once the system calculates the confidence score for each speech segment, if the confidence score is lower than the target threshold (indicating significant uncertainty in the transcription results), further processing is required.

[0121] For low-confidence segments, the following two steps are used for optimization: increasing the audio length and adjusting the text output speed.

[0122] Increasing the audio length improves recognition accuracy, especially during repeated recognition. The specific calculation formula is: L repeat =L base +α×(L complex -L base ).

[0123] The text output speed is dynamically adjusted based on the user's input speed. For example, if the user's input speed is higher than the standard value (160 WPM), the buffer time is reduced to speed up the output. If the user's input speed is lower than the standard value, the buffer time is increased to slow down the output speed, thereby improving recognition accuracy.

[0124] By employing a complexity assessment and processing mechanism, low-confidence segments are effectively optimized, improving the overall accuracy and real-time performance of speech transcription while adapting to users' varying input speeds. This mechanism considers both the dynamic complexity of speech input and enhances the system's performance in complex scenarios through adaptive adjustments.

[0125] The following explains the dynamic adjustment of the range of values ​​for α based on complexity assessment.

[0126] In this application, "the range of values ​​for α is dynamically adjusted based on complexity assessment" means that the system will dynamically adjust the range of values ​​for parameter α according to the complexity of the current voice input to adapt to different scenarios, thereby optimizing the audio processing length and recognition results.

[0127] First, it should be noted that the complexity assessment is mainly based on the following indicators.

[0128] Signal quality: background noise intensity, audio clarity, silence interval, etc.

[0129] Recognition accuracy: judged by confidence score.

[0130] Input speech rate: Information density (number of words / audio length) in an audio segment.

[0131] Number of repetitions: The degree of convergence of multiple recognition results for a certain audio segment.

[0132] When the complexity is high (such as low confidence, high noise, or excessively fast speech), the system needs to dynamically adjust the parameter α to extend the audio processing length, enhance the capture of context, and thus improve the recognition accuracy.

[0133] In one example, for simple speech scenarios (such as clear, single-person speech, low noise), α takes a lower value, such as 0.1-0.3; for complex speech scenarios (such as high background noise, multi-person dialogue, unclear accents), α takes a higher value, such as 0.7-0.9. Of course, the value of α is positively correlated with complexity. In this application, the system dynamically calculates α based on indicators such as confidence level and noise level.

[0134] In summary, by dynamically adjusting the range of values ​​for α, the system can flexibly optimize the processing length according to the actual speech environment. For simple scenarios, it can maintain real-time performance and reduce computational resource consumption; for complex scenarios, it can enhance recognition accuracy and ensure result quality. This mechanism ensures that the system can achieve optimal speech transcription results under different levels of complexity.

[0135] Example 3

[0136] It should be noted that segmenting and splicing the enhanced audio signal based on the speech activity detection results is a real-time operation. Within the segmented speech data, each segment is sequential, and the next step is executed in chronological order. Let's illustrate this with an example.

[0137] like Figure 2 As shown, when interacting with the AI ​​assistant, users can choose between keyboard input and voice input. After receiving voice input, the audio is preprocessed.

[0138] The first step is to perform audio preprocessing, including segmentation and recognition operations.

[0139] Set the initial audio length of the speech segment (e.g., 2 seconds). When a 2-second non-silent audio segment is received, it is segmented to obtain a 2-second speech segment. As long as the non-silent speech segments are continuously obtained in real time, the audio is transcribed into text and stored for each speech segment, and simultaneously output to the user interface. The output process is also continuous in real time due to the real-time transcription, that is, it continues to segment the subsequently acquired raw audio signal and concatenate it with the already recognized audio. The initially transcribed audio is also repeatedly recognized to improve accuracy.

[0140] Secondly, there's the content related to duplicate recognition and text integration.

[0141] The system integrates repeatedly recognized text with stored audio, performs semantic analysis on the integrated text, and segments sentence boundaries. It also analyzes the density of user input, using the ratio of transcribed text word count to audio length as an evaluation metric. When this ratio exceeds the normal speech rate range, the system determines that the user's input speed is too fast. This signal is fed back to the text preprocessing and output stages, increasing the length of the speech concatenation and accelerating the text output speed. Conversely, when the ratio falls below the normal speech rate range, the speech concatenation length is appropriately shortened, and the text output speed is reduced. This allows for repeated speech recognition, strengthening the connection with the preceding and following context and improving recognition accuracy.

[0142] Secondly, the initial transcribed text undergoes real-time word and phrase correction and optimization.

[0143] For the integrated duplicate text, real-time error correction is performed based on the context to identify and correct any possible errors; traditional Chinese characters in the transcribed text are identified and converted to simplified Chinese characters in real time; and appropriate punctuation marks are added at the text boundaries based on the semantic and contextual analysis during the text analysis stage to improve the readability of the text.

[0144] Finally, there's post-processing, which includes automatic stopping and saving the results.

[0145] Based on the user-specified pause threshold, the system monitors the silence duration in the speech signal in real time to determine the termination point of the speech input and the pause of the user's intention. If the silence duration exceeds the user-defined duration, the system triggers an automatic stop function. A stop signal is sent to the transcription process to pause transcription and save the current text result promptly.

[0146] Example 4

[0147] For the practical application of this application, the device module is described using the Qizhi-Lingyu AI Assistant application running on the domestic operating system openKylin as an example.

[0148] like Figure 3As shown, a schematic diagram of the structure of each module corresponding to the embodiment of this application is presented. In order to make a more complete system module, functional modules can be added or removed according to the actual situation, and there is no limitation on this. For example, 3 is a possible improved system module structure.

[0149] In one example, the system mainly includes a front-end page module, an audio processing module, and a text analysis and processing module.

[0150] The front-end page module includes a user audio input unit, a text output unit, and a text editing unit; the audio processing module includes an audio noise reduction and filtering unit, an audio trimming and splicing unit, an audio transcription unit, and an audio input monitoring unit; the text analysis and processing module includes a first functional unit and a second functional unit, wherein the first functional unit is used for text error correction and adding punctuation marks, and the second functional unit is used for analyzing the text context and calculating the user information input density.

[0151] The user audio input unit can be used to execute step S11, the audio noise reduction and filtering unit can be used to execute step S12, the audio trimming and splicing unit can be used to execute step S13, the audio transcription unit can be used to execute step S14, and the first functional unit can be used to execute steps S21, S22, S23, S24, S31, S32, S33, S34, S35 and S36.

[0152] The second functional unit can also be used to process short audio signals, such as by calculating the user information input density based on the short audio signal.

[0153] The text editing unit can be used to execute the duplication identification and adjustment mechanism, such as executing the duplication identification and adjustment mechanism on the continuously obtained transcription results, and performing text editing and adjustment and outputting in real time.

[0154] In one possible implementation, the front-end page module detects user audio input and acquires the initial audio signal, which is then processed by the audio processing module for various units. The audio processing module also monitors the silence state of the user input in real time to determine when to stop transcription. After each segment completes audio transcription, it is sent to the text analysis and processing module for further analysis of the text context. The module calculates the user information input density, performs audio trimming and splicing operations, and continues audio transcription on the segmented segments until transcription stops.

[0155] When the density of user information input is too high, the text output speed will also be adjusted to achieve real-time transcription text output. The output results will also be processed by text editing, i.e. word correction.

[0156] Example 6

[0157] To demonstrate the optimization effect of the offline real-time speech transcription optimization method for domestic operating systems provided in this application, the applicant conducted experimental tests.

[0158] The experiments used the AISHELL-4 multi-channel Chinese conference speech dataset from aishelltech. This dataset contains 211 conferences recorded using an eight-channel Mandarin Chinese microphone array, with 4 to 8 speakers per conference, totaling approximately 120 hours of data. This dataset aims to promote research on multi-speaker processing in real-world scenarios, including various important characteristics of actual conference scenarios such as pauses, overlaps, speaker rotation, and noise. Twenty interview and conference recordings in quiet and noisy environments were selected for the test, including low-speed, normal-speed, and high-speed speech. The transcription accuracy was quantified using A-speech, as shown below: Where N correct N represents the number of correct words in the transcribed text. total This represents the total number of words in the transcribed text. Transcription errors include misspellings, traditional Chinese characters, etc.

[0159] The experimental test was evaluated from three aspects: 1) whether there was environmental noise, all selected recordings were at normal speaking speed; 2) different speaking speeds, all selected voice recordings were recorded in a quiet environment; 3) overall optimization effect.

[0160] Figure 4 The graphs show the test results under quiet and noisy environments. Figure 5 The figures show the test results under different speech rates. Experimental results show that in a quiet environment, the average accuracy of the optimized transcription is 91.67%, an improvement of 3.93% compared to before optimization; in a noisy environment, the average accuracy of the optimized transcription is 90.09%, an improvement of 3.73% compared to before optimization. At lower speech rates, the average accuracy after optimization is 92.14%, an improvement of 1.81% compared to before optimization; at normal speech rates, the average accuracy after optimization is 92.07%, an improvement of 3.58% compared to before optimization; and at high speech rates, the average accuracy after optimization is 90.84%, an improvement of 5.04% compared to before optimization. The optimized transcribed text contains no traditional Chinese characters, and all traditional Chinese characters can be correctly converted to simplified Chinese characters. Experimental results show that the method improves the accuracy of text transcription under different environments and speech rates.

[0161] On the other hand, a computer-readable storage medium is provided, the storage medium storing at least one instruction, the at least one instruction being executed by a processor to implement the offline real-time speech transcription optimization method for a domestic operating system as described above.

[0162] On the other hand, a computer program product is also provided, which stores at least one instruction, which is loaded and executed by the processor to implement the offline real-time speech transcription optimization method for domestic operating systems described above.

[0163] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0164] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0165] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An offline real-time speech transcription optimization method for domestic operating systems, characterized in that, The method includes: The raw audio signal is acquired and segmented and recognized to obtain preliminary transcribed text. The raw audio signal includes a custom long audio signal and a custom short audio signal. The preliminary transcribed text is subjected to real-time word correction to obtain the corrected transcribed text; The transcribed text after error correction is post-processed to obtain offline real-time speech transcription results. The post-processing includes potential pauses, sentence boundary judgment, and speech transcription process stop judgment. In response to the detection that the original audio signal is the custom short audio signal, the custom short audio signal is segmented and identified by a preset audio sampling rate and a preset buffer. The process of acquiring the original audio signal, performing segmentation and recognition operations to obtain preliminary transcribed text includes: Receive the raw audio signal acquired by the acquisition terminal; The original audio signal is preprocessed to obtain the enhanced audio signal; The enhanced audio signal is segmented and spliced ​​together based on the voice activity detection results to obtain the audio signal to be recognized; The audio signal to be identified is subjected to speech recognition and converted into the preliminary transcribed text; The step of segmenting and splicing the enhanced audio signal based on the speech activity detection result to obtain the audio signal to be recognized includes: Based on the speech activity detection results, the enhanced audio signal is segmented into several speech segments, wherein the segmentation is based on natural pauses or a custom time length. Adjacent speech segments are spliced ​​together to obtain the audio signal to be identified, wherein the adjacent speech segments have an overlap of a custom length at the splicing point; After obtaining the raw audio signal, performing segmentation and recognition operations to obtain the preliminary transcribed text, the method further includes: Calculate the confidence score for each speech segment. Repeated recognition and adjustment are performed on each segment of speech, wherein the confidence score is... The calculation formula is , Represents the segment of speech being recognized. Indicates the first Each model pair The confidence probability, This refers to the total number of evaluation methods; In response to the detection of confidence score Below the target threshold The audio segments are processed using an automatic adjustment strategy. The response to detecting a confidence score Below the target threshold The audio segments are processed using an automatic adjustment strategy, including: Confidence scoring Below the target threshold Increase the audio length when repeating speech segments. ,in, , It is the base processing length. This is the ideal processing length after complexity assessment. It is an adjustment coefficient with a value between 0 and 1.

2. The method according to claim 1, characterized in that, The step of performing speech recognition on the audio signal to be recognized and converting it into the preliminary transcribed text includes: The overlapping portions of the audio signal to be identified are identified using a sliding window mode; The overlapping portions are removed to obtain the preliminary transcribed text.

3. The method according to claim 1, characterized in that, The step of performing real-time word correction on the initial transcribed text to obtain the corrected transcribed text includes: The preliminary transcribed text was checked word by word for grammar and spelling using word segmentation technology; Use contextual information to identify words that do not conform to grammar or semantics and mark them as uncertain words; Calculate the edit distance of the uncertain words and generate a set of candidate corrected words by combining them with context information; The candidate correction words are sorted, and the correction words of the best match are selected and replaced in the preliminary transcribed text to obtain the error-corrected transcribed text.

4. The method according to claim 1, characterized in that, The post-processing of the corrected transcribed text to obtain offline real-time speech transcription results includes: The grammatical structure, themes, and contextual semantics of the corrected transcribed text were analyzed to identify potential pauses and sentence boundaries. The system analyzes the response level of the original audio signal in real time during speech input, and identifies significant pauses or emphasis changes, as well as rhythm and pitch changes, in the speech stream corresponding to the original audio signal. The identification of significant pauses or emphasis changes, as well as rhythm and tone variations, helps determine potential pauses and sentence boundaries. Punctuation suggestions are generated for the post-corrected transcribed text based on potential pauses and sentence boundaries using natural language processing technology. Based on the pause length threshold specified by the user, the silence duration in the original audio signal is monitored in real time. In response to the silence duration in the original audio signal exceeding the user-defined silence duration, the automatic stop function of the speech transcription process is triggered.

5. An offline real-time speech transcription optimization device for domestic operating systems, characterized in that, The device includes: The signal processing module is used to acquire the raw audio signal for segmentation and recognition operations to obtain preliminary transcribed text. The raw audio signal includes a custom long audio signal and a custom short audio signal. The word correction module is used to perform real-time word correction on the preliminary transcribed text to obtain the corrected transcribed text. The post-processing module is used to post-process the transcribed text after error correction to obtain offline real-time speech transcription results. The post-processing includes potential pauses, sentence boundary judgment, and speech transcription process stop judgment. The short audio processing module is used to segment and identify the custom short audio signal in response to the detection that the original audio signal is the custom short audio signal, by using a preset audio sampling rate and a preset buffer. The signal operation module is further configured to receive the raw audio signal acquired by the acquisition end; preprocess the raw audio signal to obtain an enhanced audio signal; segment and splice the enhanced audio signal according to the speech activity detection result to obtain an audio signal to be recognized; perform speech recognition on the audio signal to be recognized and convert it into the preliminary transcribed text; the step of segmenting and splicing the enhanced audio signal according to the speech activity detection result to obtain the audio signal to be recognized includes: segmenting the enhanced audio signal into several speech segments according to the speech activity detection result, wherein the segmentation is based on natural pauses or a custom time length; splicing adjacent speech segments to obtain the audio signal to be recognized, wherein the adjacent speech segments have an overlapping part of a custom length at the splicing point; The signal manipulation module is also used to calculate the confidence score of each speech segment. Repeated recognition and adjustment are performed on each segment of speech, wherein the confidence score is... The calculation formula is , Represents the segment of speech being recognized. Indicates the first Each model pair The confidence probability, This refers to the total number of evaluation methods; in response to the detection of confidence scores. Below the target threshold The system executes an automatic adjustment processing strategy on the segmented speech; the response to detecting a confidence score... Below the target threshold The audio segments are processed using an automatic adjustment strategy, including: adjusting the confidence score. Below the target threshold Increase the audio length when repeating speech segments. ,in, , It is the base processing length. This is the ideal processing length after complexity assessment. It is an adjustment coefficient with a value between 0 and 1.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the offline real-time speech transcription optimization method for a domestic operating system as described in any one of claims 1 to 4.

Images