Two-process error correction method and device for real-time speech transcription

Abstract

The invention provides a two-process error correction method and device for real-time speech transcription, and relates to the technical field of speech processing, and the method comprises the steps: extracting the Mel spectrum features of each segment, inputting each Mel spectrum feature into a lightweight end-to-end model, and obtaining a preliminary transcription text; splicing the segments according to a preset number to obtain a plurality of long segments, and inputting each long segment into a speech recognition model to obtain a high-precision transcription text; performing text comparison on the preliminary transcription text and the high-precision transcription text according to the confidence degree set of the preliminary transcription text to obtain all error vocabularies in the preliminary transcription text; and performing corresponding error correction processing on each error vocabulary in the preliminary transcription text according to the type of the error vocabulary and the high-precision transcription text to obtain a final transcription text. According to the method, through a two-process transcription error correction mechanism of the preliminary transcription text and the high-precision transcription text, transcription error accumulation is reduced on the premise that the real-time performance is not affected, and the transcription accuracy in a complex scene is improved.

Description

Technical Field

[0001] The present invention relates to the field of speech processing technology, and in particular to a dual-process error correction method and device for real-time speech transcription. Background Art

[0002] Existing real-time speech transcription systems typically process speech streams in a single process, directly performing real-time transcription after receiving speech input. However, such systems have the following problems: Insufficient contextual information: Real-time processing relies only on the current short segment of speech and lacks contextual analysis of subsequent long segments of speech, resulting in incomplete semantic understanding and a high transcription error rate.

[0003] Error accumulation: A single process cannot dynamically correct the generated transcription results. Errors may continue to accumulate with the voice stream, affecting the final accuracy.

[0004] The contradiction between real-time performance and accuracy: To meet real-time requirements, the system needs to respond quickly, but single-process processing cannot achieve high-precision analysis, especially in complex scenarios (such as multiple languages ​​and professional terminology).

[0005] Some existing solutions attempt to improve accuracy through delayed processing or batch correction, but this sacrifices real-time performance and results in a poor user experience. Therefore, a technical solution is urgently needed that can maintain real-time response while improving accuracy through dynamic correction. Summary of the Invention

[0006] In order to solve the above problems, the present invention provides a dual-process error correction method for real-time speech transcription, comprising the steps of: Obtain a voice stream and divide the voice stream into multiple segments; Extract the mel-spectrogram features of each segment and input them into a lightweight end-to-end model to obtain a preliminary transcript. The segments are spliced ​​together according to a preset number to obtain multiple long segments, and each long segment is input into the speech recognition model to obtain high-precision transcription text; Perform timestamp alignment and phoneme sequence alignment on the preliminary transcription and the high-precision transcription. Perform a text comparison on the preliminary transcription and the high-precision transcription based on the confidence set of the preliminary transcription to obtain all incorrect words in the preliminary transcription. Correct the erroneous words in the preliminary transcription text according to the type of the erroneous words and the high-precision transcription text to obtain the final transcription text.

[0007] Optionally, the process of obtaining the confidence set of the preliminary transcription text specifically includes: The preliminary transcription text is segmented to obtain multiple words; the confidence of each word is calculated using a connectionist time classification algorithm, and the confidence of each word is combined into a confidence set.

[0008] Optionally, the segments are spliced ​​together according to a preset number to obtain multiple long segments, specifically including: Get the N segments of the voice stream, each segment has a duration of t seconds, and the total duration is N×t seconds; set the preset number to n, n is less than or equal to N; starting from the first segment, use the overlap retention method to splice the n segments into a long segment, a total of splicing A long paragraph, Indicates that the remainder is rounded up.

[0009] Optionally, perform timestamp alignment and phoneme sequence alignment on the preliminary and high-precision transcripts, including: Get the start and end time of each segment in the speech stream, use the time interval from the start to the end of each segment as the timestamp of each segment, and align the preliminary transcription text and the high-precision transcription text according to the timestamp of each segment; The phoneme sequences of the preliminary transcription text and the high-precision transcription text are obtained, the optimal matching path between the two phoneme sequences is calculated using the dynamic time warping algorithm, and the phoneme sequences of the preliminary transcription text and the high-precision transcription text are aligned according to the optimal matching path.

[0010] Optionally, a text comparison is performed on the preliminary transcription and the high-precision transcription based on the confidence set of the preliminary transcription to obtain all incorrect words in the preliminary transcription, specifically including: S11: Obtain multiple windows and set the length of each window to k words; S12: Covering the preliminary transcription text and the high-precision transcription text through multiple windows, wherein the i-th window covers the i-th to i+k-1-th words of the preliminary transcription text and the high-precision transcription text; S13: selecting the i-th window, taking the words of the preliminary transcription text in the i-th window whose confidence is greater than or equal to the confidence threshold as the credible words of the preliminary transcription text, and taking the words of the preliminary transcription text with a confidence less than the confidence threshold as the uncredible words of the preliminary transcription text; Compare the credible words in the preliminary transcription text in the i-th window with the words in the corresponding position of the high-precision transcription text, and regard the inconsistent credible words as the incorrect words in the preliminary transcription text; Compare the untrustworthy words in the preliminary transcription text in all windows with the words in the corresponding positions in the high-precision transcription text, and treat the inconsistent untrustworthy words as incorrect words in the preliminary transcription text; S14: Repeat step S13 until all windows are traversed to obtain all incorrect words in the preliminary transcription text.

[0011] Optionally, after obtaining all incorrect words in the preliminary transcription text, the original confidence threshold is modified according to the number of incorrect words to obtain a new confidence threshold, specifically including: The ratio of the number of incorrect words to the total number of words in the preliminary transcription text is taken as the correction rate, and the original confidence threshold × (1-correction rate) is taken as the new confidence threshold.

[0012] Optionally, error correction processing is performed on each erroneous word in the preliminary transcription text according to the type of the erroneous word and the high-precision transcription text to obtain a final transcription text, specifically including: If the type of the erroneous word is a hard error, the erroneous word in the preliminary transcription text will be replaced with the word in the corresponding position in the high-precision transcription text; if the type of the erroneous word is a soft error, the erroneous word in the preliminary transcription text will be highlighted; the preliminary transcription text after all the erroneous words are replaced or highlighted will be used as the final transcription text.

[0013] The present invention also provides a dual-process error correction device for real-time speech transcription, which is used to implement the dual-process error correction method for real-time speech transcription. The device includes: A speech segmentation module is used to obtain a speech stream and segment the speech stream into multiple segments; The preliminary transcript acquisition module is used to extract the mel-spectrogram features of each segment and input each mel-spectrogram feature into a lightweight end-to-end model to obtain the preliminary transcript; A high-precision transcript acquisition module is used to splice the segments according to a preset number to obtain multiple long segments, and then input each long segment into the speech recognition model to obtain high-precision transcripts; The error word acquisition module is used to align the timestamps and phoneme sequences of the preliminary transcription text and the high-precision transcription text, and compare the preliminary transcription text and the high-precision transcription text according to the confidence set of the preliminary transcription text to obtain all the error words in the preliminary transcription text; The error correction module is used to perform corresponding error correction processing on each erroneous word in the preliminary transcription text according to the type of the erroneous word and the high-precision transcription text to obtain the final transcription text.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the dual-process error correction method for real-time speech transcription when executing the program.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the dual-process error correction method for real-time speech transcription.

[0016] The present invention has the following beneficial effects: 1. Segment the speech stream into small segments, transcribe the small segments into preliminary transcripts, and ensure the real-time performance of speech transcription. Combine multiple small segments into lengths, and transcribe the lengths into high-precision transcripts, ensuring the accuracy of speech transcription. Use the high-precision transcripts to correct the preliminary transcripts, performing full-window matching detection on words in the preliminary transcripts with a confidence level lower than the confidence threshold to comprehensively detect potential erroneous words. Through the dual-process transcription error correction mechanism of preliminary and high-precision transcripts, the accumulation of transcription errors is reduced without affecting real-time performance, improving transcription accuracy in complex scenarios. 2. Correct the original confidence threshold based on the number of incorrect words. If the number of incorrect words is too large, it means that the current confidence standard of the model may be too loose and the threshold needs to be lowered. Otherwise, the threshold can be appropriately increased. The confidence standard can be continuously adjusted according to the actual correction situation to improve the reliability of subsequent transcription results. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of a method according to an embodiment of the present invention; Figure 2 This is a structural diagram of an embodiment of the present invention; The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0018] The following will be combined with the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0019] Reference Figure 1 The present invention provides a dual-process error correction method for real-time speech transcription, comprising the steps of: Obtain a voice stream and divide the voice stream into multiple segments; In some embodiments, the voice stream is segmented into segments with a fixed duration of 0.5 seconds. Such segmentation granularity can ensure real-time performance while providing relatively independent voice units for subsequent processing.

[0020] Extract the mel-spectrogram features of each segment and input them into a lightweight end-to-end model to obtain a preliminary transcript. In some embodiments, obtaining the preliminary transcription text is a real-time transcription process, ie, an R process.

[0021] In some embodiments, after segmentation, an 80-dimensional Mel-spectrogram feature is generated for each segment, with a frame length of 25ms. Mel-spectrogram features can better simulate the human auditory system's perception of sound frequencies, highlight key information in the speech signal, and provide more effective data features for accurate model recognition.

[0022] WhisperSmall is an example of a lightweight end-to-end model that meets basic real-time requirements while maintaining a certain processing speed. Its relatively simple structure and small parameter size allow it to run efficiently on end devices, reducing computing resource usage and processing latency.

[0023] In some embodiments, the process of obtaining a confidence set of the preliminary transcription text specifically includes: The preliminary transcription text is segmented to obtain multiple words; the confidence of each word is calculated using a connectionist time classification algorithm, and the confidence of each word is combined into a confidence set.

[0024] In some embodiments, the Connectionist Temporal Classification (CTC) algorithm is a sequence learning algorithm specifically designed for sequences where the lengths of the input and output sequences are inconsistent and cannot be pre-aligned. It is widely used in fields that require processing time series data, such as speech recognition and handwriting recognition. Its core lies in solving the labeling problem of unsegmented sequences by introducing a special mechanism and supporting end-to-end training. The characters in the vocabulary are extracted according to the time step, and the connectionist temporal classification output probability of the characters is calculated by the connectionist temporal classification algorithm. The connectionist temporal classification output probabilities of all characters in the vocabulary are added together to obtain the confidence of the vocabulary. The confidence reflects the credibility of the model's recognition results for the vocabulary. The higher the confidence, the more confident the model is in recognizing the vocabulary. The confidence calculation formula is as follows: S(w)= Among them, S(w) is the confidence of word W, L is the length of word, t represents the time step, W t is the t-th character of vocabulary W, X t is the input feature of the t-th character, Indicates that the t-th character of vocabulary W is in the input feature Connectionist temporal classification output probability under .

[0025] The segments are spliced ​​together according to a preset number to obtain multiple long segments, and each long segment is input into the speech recognition model to obtain high-precision transcription text; In some embodiments, the speech recognition model uses a high-precision speech model, such as WhisperLarge. The WhisperLarge model has a more complex network structure and a larger parameter size, enabling it to learn richer speech features and language knowledge, thereby achieving higher accuracy during transcription. It excels particularly when processing long speech segments and complex semantics.

[0026] In some embodiments, the segments are spliced ​​together according to a preset number to obtain multiple long segments, specifically including: Get the N segments of the voice stream, each segment has a duration of t seconds, and the total duration is N×t seconds; set the preset number to n, n is less than or equal to N; starting from the first segment, use the overlap retention method to splice the n segments into a long segment, a total of splicing A long paragraph, Indicates that the remainder is rounded up.

[0027] In some embodiments, for example, the speech is segmented into six short segments, each 0.5 seconds long, for a total of three seconds. These six segments are then concatenated into continuous speech using the overlap-preserve method. This overlap-preserve method avoids loss of speech information during the concatenation process, ensuring the integrity of the speech signal and providing more reliable input for subsequent high-precision transcription.

[0028] High-precision transcripts are generated based on a global contextual analysis of 3-second speech segments. Compared to preliminary transcripts, they better understand the overall semantics of the speech and reduce semantic errors. Furthermore, by comparing the transcript with the preliminary transcript, errors are identified, pinpointing the exact location of errors in the preliminary transcript and providing a clear target for subsequent corrections.

[0029] Perform timestamp alignment and phoneme sequence alignment on the preliminary transcription and the high-precision transcription. Perform a text comparison on the preliminary transcription and the high-precision transcription based on the confidence set of the preliminary transcription to obtain all incorrect words in the preliminary transcription. In some embodiments, the process of correcting the preliminary transcription text through the high-precision transcription text is a correction process, namely, the C process.

[0030] In some embodiments, performing timestamp alignment and phoneme sequence alignment on the preliminary transcription text and the high-precision transcription text specifically includes: Get the start and end time of each segment in the speech stream, use the time interval from the start to the end of each segment as the timestamp of each segment, and align the preliminary transcription text and the high-precision transcription text according to the timestamp of each segment; In some embodiments, each short segment in the preliminary transcript is annotated with precise start and end times, such as [500ms, 1000ms]. The high-precision transcript breaks down the 3-second segment into six virtual subsegments. These virtual subsegments strictly correspond to the timestamps of the preliminary transcript, ensuring temporal consistency between the two transcripts and providing an accurate time reference for subsequent phoneme sequence alignment and text difference detection.

[0031] The phoneme sequences of the preliminary transcription text and the high-precision transcription text are obtained, the optimal matching path between the two phoneme sequences is calculated using the dynamic time warping algorithm, and the phoneme sequences of the preliminary transcription text and the high-precision transcription text are aligned according to the optimal matching path.

[0032] In some embodiments, the dynamic time warping (DTW) algorithm is used to align the phoneme sequences of the two processes. The DTW algorithm finds the most similar phoneme correspondence by calculating the optimal matching path between the two phoneme sequences.

[0033] D(i,j) = min(D(i-1,j),D(i,j-1),D(i-1,j-1))+‖R_phoneme[i]-C_phoneme[j]‖ D(i,j) represents the cumulative alignment distance between the i-th phoneme in the preliminary transcript and the j-th phoneme in the high-precision transcript. i represents the temporal order of the phoneme sequence index in the preliminary transcript corresponding to the speech segment, and j represents the temporal order of the phoneme sequence index in the high-precision transcript corresponding to the virtual subsegment of the long speech segment. ‖‖ represents the Euclidean distance between the two vectors, which is used to calculate the degree of feature difference between the current phoneme pair. R_phoneme[i] represents the Mel-frequency cepstral coefficient feature of the i-th phoneme in the preliminary transcript, and C_phoneme[j] represents the Mel-frequency cepstral coefficient feature of the j-th phoneme in the high-precision transcript. A cumulative alignment distance greater than 0.2 seconds is considered a potential error. This threshold, determined through extensive experimentation and data analysis, ensures detection accuracy while avoiding excessive false positives.

[0034] In some embodiments, a text comparison is performed on the preliminary transcription text and the high-precision transcription text based on the confidence set of the preliminary transcription text to obtain all incorrect words in the preliminary transcription text, specifically including: S11: Obtain multiple windows and set the length of each window to k words; S12: Covering the preliminary transcription text and the high-precision transcription text through multiple windows, wherein the i-th window covers the i-th to i+k-1-th words of the preliminary transcription text and the high-precision transcription text; S13: selecting the i-th window, taking the words of the preliminary transcription text in the i-th window whose confidence is greater than or equal to the confidence threshold as the credible words of the preliminary transcription text, and taking the words of the preliminary transcription text with a confidence less than the confidence threshold as the uncredible words of the preliminary transcription text; Compare the credible words in the preliminary transcription text in the i-th window with the words in the corresponding position of the high-precision transcription text, and regard the inconsistent credible words as the incorrect words in the preliminary transcription text; Compare the untrustworthy words in the preliminary transcription text in all windows with the words in the corresponding positions in the high-precision transcription text, and treat the inconsistent untrustworthy words as incorrect words in the preliminary transcription text; In some embodiments, full-window matching is performed on untrusted words because words with lower confidence are more likely to contain errors, and full-window matching can more comprehensively detect potential errors.

[0035] S14: Repeat step S13 until all windows are traversed to obtain all incorrect words in the preliminary transcription text.

[0036] In some embodiments, after obtaining all incorrect words in the preliminary transcription text, the original confidence threshold is modified according to the number of incorrect words to obtain a new confidence threshold, specifically including: The ratio of the number of incorrect words to the total number of words in the preliminary transcription text is taken as the correction rate, and the original confidence threshold × (1-correction rate) is taken as the new confidence threshold.

[0037] In some embodiments, by modifying the confidence threshold, the confidence level can be continuously adjusted based on the actual corrections, improving the reliability of subsequent transcription results. If the number of incorrect words is too high, it means that the model's current confidence level may be too loose and the threshold needs to be lowered. Otherwise, the threshold can be appropriately increased.

[0038] Correct the erroneous words in the preliminary transcription text according to the type of the erroneous words and the high-precision transcription text to obtain the final transcription text.

[0039] In some embodiments, error correction processing is performed on each erroneous word in the preliminary transcription text according to the type of the erroneous word and the high-precision transcription text to obtain the final transcription text, specifically including: If the type of the erroneous word is a hard error, the erroneous word in the preliminary transcription text will be replaced with the word in the corresponding position in the high-precision transcription text; if the type of the erroneous word is a soft error, the erroneous word in the preliminary transcription text will be highlighted; the preliminary transcription text after all the erroneous words are replaced or highlighted will be used as the final transcription text.

[0040] In some embodiments, errors are categorized as hard errors and soft errors. Hard errors, such as those involving numbers, dates, and units, are crucial to the accuracy of the information and must be corrected upon detection. A dynamic priority vocabulary is constructed to store frequently corrected terms. When transcribing the initial transcript, terms in the vocabulary are prioritized and the matching weight is increased by a factor of 2. This speeds up the identification and correction of commonly incorrect terms, further improving overall transcription accuracy and enhancing system processing efficiency.

[0041] Soft errors are identified by calculating probability differences using the N-gram model. The N-gram model calculates the probability of a word appearing based on the context of the text. When P_C(text) - P_R(text) > 0.3, the incorrect word is highlighted. P_C(text) is the probability of the incorrect word appearing in the high-precision transcript, and P_R(text) is the probability of the incorrect word appearing in the preliminary transcript. The 0.3 threshold, derived from training and analysis of extensive text data, provides a reasonable basis for determining whether soft errors require correction.

[0042] refer to Figure 2 The present invention provides a dual-process error correction device 20 for real-time speech transcription, which is used to implement the dual-process error correction method for real-time speech transcription. The device includes: A speech segmentation module 21 is used to obtain a speech stream and segment the speech stream into multiple segments; A preliminary transcript acquisition module 22 is configured to extract mel-spectrogram features of each segment and input each mel-spectrogram feature into a lightweight end-to-end model to obtain a preliminary transcript; A high-precision transcript acquisition module 23 is configured to concatenate a preset number of segments to obtain multiple long segments, and input each long segment into a speech recognition model to obtain a high-precision transcript; An incorrect vocabulary acquisition module 24 is configured to perform timestamp alignment and phoneme sequence alignment on the preliminary transcription text and the high-precision transcription text, perform text comparison on the preliminary transcription text and the high-precision transcription text based on the confidence set of the preliminary transcription text, and obtain all incorrect vocabulary in the preliminary transcription text; The error correction module 25 is used to perform corresponding error correction processing on each erroneous word in the preliminary transcription text according to the type of the erroneous word and the high-precision transcription text to obtain a final transcription text.

[0043] An embodiment of the present application provides an electronic device, including a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements the dual-process error correction method for real-time speech transcription of any of the above-mentioned schemes.

[0044] Specifically, the processor may include, for example, a general-purpose microprocessor, an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor may also include onboard memory for caching purposes. The processor may be a single processing unit or multiple processing units for executing different actions of the method flow according to the embodiments of the present application.

[0045] Memory, for example, can be any medium capable of containing, storing, conveying, propagating, or transmitting instructions. For example, memory can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, components, or propagation media. Specific examples of memory include: magnetic storage devices, such as magnetic tape or hard disk drives (HDDs); optical storage devices, such as compact discs (CD-ROMs); random access memory (RAM) or flash memory; and / or wired or wireless communication links.

[0046] This application also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the dual-process error correction method for real-time speech transcription according to any of the above-described solutions. The computer-readable medium may be included in the device / apparatus / system described in the above-described embodiments, or it may exist independently and not be incorporated into the device / apparatus / system. The computer-readable medium carries one or more programs, and when executed, implements the method according to the embodiments of this application.

[0047] According to an embodiment of the present application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, radio frequency signals, or any suitable combination thereof.

[0048] Those skilled in the art will understand that the features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present application. In particular, without departing from the spirit and teachings of the present application, the features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present application. Therefore, the scope of the present application should not be limited to the above-mentioned embodiments, but should be determined not only by the attached claims, but also by the equivalents of the attached claims. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A dual-process error correction method for real-time speech transcription, characterized in that: Including steps: Obtain a voice stream and divide the voice stream into multiple segments; Extract the mel-spectrogram features of each segment and input them into a lightweight end-to-end model to obtain a preliminary transcript. The segments are spliced ​​together according to a preset number to obtain multiple long segments, and each long segment is input into the speech recognition model to obtain high-precision transcription text; Perform timestamp alignment and phoneme sequence alignment on the preliminary transcription and the high-precision transcription. Perform a text comparison on the preliminary transcription and the high-precision transcription based on the confidence set of the preliminary transcription to obtain all incorrect words in the preliminary transcription. Correct the erroneous words in the preliminary transcription text according to the type of the erroneous words and the high-precision transcription text to obtain the final transcription text.

2. The dual-process error correction method for real-time speech transcription according to claim 1, characterized in that: The process of obtaining the confidence set of the preliminary transcription text specifically includes: The preliminary transcription text is segmented to obtain multiple words; the confidence of each word is calculated using a connectionist time classification algorithm, and the confidence of each word is combined into a confidence set.

3. The dual-process error correction method for real-time speech transcription according to claim 1, characterized in that: Splice the segments according to the preset number to obtain multiple long segments, including: Get the N segments of the voice stream, each segment has a duration of t seconds, and the total duration is N×t seconds; set the preset number to n, n is less than or equal to N; starting from the first segment, use the overlap retention method to splice the n segments into a long segment, a total of splicing A long paragraph, Indicates that the remainder is rounded up.

4. The dual-process error correction method for real-time speech transcription according to claim 1, wherein: Perform timestamp alignment and phoneme sequence alignment on the preliminary and high-precision transcriptions, including: Get the start and end time of each segment in the speech stream, use the time interval from the start to the end of each segment as the timestamp of each segment, and align the preliminary transcription text and the high-precision transcription text according to the timestamp of each segment; The phoneme sequences of the preliminary transcription text and the high-precision transcription text are obtained, the optimal matching path between the two phoneme sequences is calculated using the dynamic time warping algorithm, and the phoneme sequences of the preliminary transcription text and the high-precision transcription text are aligned according to the optimal matching path.

5. The dual-process error correction method for real-time speech transcription according to claim 2, characterized in that: Based on the confidence set of the preliminary transcription, the preliminary transcription and the high-precision transcription are compared to obtain all incorrect words in the preliminary transcription, including: S11: Obtain multiple windows and set the length of each window to k words; S12: Covering the preliminary transcription text and the high-precision transcription text through multiple windows, wherein the i-th window covers the i-th to i+k-1-th words of the preliminary transcription text and the high-precision transcription text; S13: selecting the i-th window, taking the words of the preliminary transcription text in the i-th window whose confidence is greater than or equal to the confidence threshold as the credible words of the preliminary transcription text, and taking the words of the preliminary transcription text with a confidence less than the confidence threshold as the uncredible words of the preliminary transcription text; Compare the credible words in the preliminary transcription text in the i-th window with the words in the corresponding position of the high-precision transcription text, and regard the inconsistent credible words as the incorrect words in the preliminary transcription text; Compare the untrustworthy words in the preliminary transcription text in all windows with the words in the corresponding positions in the high-precision transcription text, and treat the inconsistent untrustworthy words as incorrect words in the preliminary transcription text; S14: Repeat step S13 until all windows are traversed to obtain all incorrect words in the preliminary transcription text.

6. The dual-process error correction method for real-time speech transcription according to claim 5, characterized in that: After obtaining all the incorrect words in the preliminary transcription text, the original confidence threshold is modified according to the number of incorrect words to obtain a new confidence threshold, which specifically includes: The ratio of the number of incorrect words to the total number of words in the preliminary transcription text is taken as the correction rate, and the original confidence threshold × (1-correction rate) is taken as the new confidence threshold.

7. The dual-process error correction method for real-time speech transcription according to claim 1, characterized in that: Correct each erroneous word in the preliminary transcript based on the type of erroneous word and the high-precision transcript to obtain the final transcript, specifically including: If the type of the erroneous word is a hard error, the erroneous word in the preliminary transcription text will be replaced with the word in the corresponding position in the high-precision transcription text; if the type of the erroneous word is a soft error, the erroneous word in the preliminary transcription text will be highlighted; the preliminary transcription text after all the erroneous words are replaced or highlighted will be used as the final transcription text.

8. A dual-process error correction device for real-time speech transcription, used to implement the dual-process error correction method for real-time speech transcription according to any one of claims 1 to 7, characterized in that: The device comprises: A speech segmentation module is used to obtain a speech stream and segment the speech stream into multiple segments; The preliminary transcript acquisition module is used to extract the mel-spectrogram features of each segment and input each mel-spectrogram feature into a lightweight end-to-end model to obtain the preliminary transcript; A high-precision transcript acquisition module is used to splice the segments according to a preset number to obtain multiple long segments, and then input each long segment into the speech recognition model to obtain high-precision transcripts; The error word acquisition module is used to align the timestamps and phoneme sequences of the preliminary transcription text and the high-precision transcription text, and compare the preliminary transcription text and the high-precision transcription text according to the confidence set of the preliminary transcription text to obtain all the error words in the preliminary transcription text; The error correction module is used to perform corresponding error correction processing on each erroneous word in the preliminary transcription text according to the type of the erroneous word and the high-precision transcription text to obtain the final transcription text.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the dual-process error correction method for real-time speech transcription according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the dual-process error correction method for real-time speech transcription according to any one of claims 1 to 7 is implemented.

Images