Audio transfer method and device, equipment and medium

Abstract

The invention provides an audio transliteration method and device, equipment and a medium, and the method comprises the steps: segmenting a to-be-transliterated complete audio into a plurality of audio clips, and at least one audio clip containing a complete voice clip exists in two adjacent audio clips; performing audio transliteration on each audio clip in parallel to obtain an audio transliteration result of each audio clip; and combining the audio transliteration results of the audio clips to obtain an audio transliteration result of the complete audio. According to the method, the device, the equipment and the medium provided by the invention, the efficiency and the accuracy of audio transliteration are remarkably improved through a segmentation mode and a parallel processing mode of ensuring that at least one audio clip containing the complete voice clip exists in the adjacent audio clips. And the multi-path transfer engine performs parallel processing on the plurality of audio clips, so that the transfer time of the long audio is greatly shortened.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to an audio transcription method, device, equipment and medium. Background Art

[0002] Audio transcription technology plays an important role in the field of modern information processing and is widely used in scenarios such as conference records and voice assistants that require the rapid and accurate conversion of voice content into text. Currently, there are two main solutions for audio transcription technology: one is to use a single voice engine to transcribe the entire audio to maintain the coherence and accuracy of the transcribed content; the other is to divide long audio into multiple segments and use multiple engines for parallel processing to increase the transcription speed, which is particularly suitable for processing large-capacity audio data.

[0003] However, both solutions have their own shortcomings. When a single-channel engine is used to process long audio, transcription takes a long time and is inefficient; while in the multi-channel engine parallel processing solution, audio segmentation may be performed in the middle of a sentence, resulting in inaccurate transcription and incoherent sentences, affecting the final transcription quality and user experience.

[0004] Therefore, how to strike a balance between audio transcription efficiency and transcription quality is an urgent problem that needs to be solved. Summary of the invention

[0005] The present invention provides an audio transcription method, device, equipment and medium, which are used to solve the defects of slow audio transcription speed and low quality in the related art.

[0006] The present invention provides an audio transcription method, comprising the following steps: Dividing the complete audio to be transcribed into multiple audio segments, wherein at least one of two adjacent audio segments contains a complete voice segment; Performing audio transcription on each audio clip in parallel to obtain an audio transcription result of each audio clip; The audio transcription results of each audio segment are merged to obtain the audio transcription result of the complete audio.

[0007] According to an audio transcription method provided by the present invention, there is an overlapping portion between the two adjacent audio segments; The audio transcription results of the audio segments are merged to obtain the audio transcription result of the complete audio, including: Based on the audio transcription results of the two adjacent audio segments, determining the overlap of the speech segments of the two adjacent audio segments in the overlapped portion; Based on the overlap of the voice segments of the two adjacent audio segments in the overlapping part, the audio transcription results of the two adjacent audio segments are merged to obtain the audio transcription result of the complete audio.

[0008] According to an audio transcription method provided by the present invention, the audio transcription results of the two adjacent audio segments are merged based on the overlap of the voice segments of the two adjacent audio segments in the overlapped part, including: When the speech segment overlaps, the endpoint times of the speech segments of the two audio segments overlap, merging the audio transcription results of the two audio segments based on the overlapped endpoint times; When the voice segment overlaps in a situation where the endpoint times of the two audio segments in the overlapping parts do not overlap, the audio transcription results of the two audio segments are merged based on the time difference between the endpoint times of the voice segments in the overlapping parts of the two audio segments.

[0009] According to an audio transcription method provided by the present invention, merging the audio transcription results of the two audio clips based on the overlapping endpoint times includes: In the case where the overlapping endpoint time is the start time of the voice segment, merging the audio transcription result of the first audio segment before the start time and the audio transcription result of the second audio segment after the start time in the two audio segments; When the overlapping endpoint time is the end time of the voice segment, the audio transcription result of the previous audio segment before the end time and the audio transcription result of the next voice segment after the end time are merged.

[0010] According to an audio transcription method provided by the present invention, the audio transcription results of the two audio segments are merged based on the time difference between the endpoint times of the speech segments in the overlapping parts of the two audio segments, including: Determine the end time of the previous audio segment of the two audio segments at the speech segment of the overlapping portion as the previous end time; Determine, from the start times of the latter audio segment of the two audio segments, a start time with the shortest time difference from the previous end time among the start times of the voice segments in the overlapping portion, as the latter start time; The audio transcription result of the previous audio segment before the previous end time and the audio transcription result of the subsequent voice segment after the subsequent end time are merged.

[0011] According to an audio transcription method provided by the present invention, the endpoint time overlap means that the endpoint times of the voice segments of the two audio segments in the overlapping part are the same, or the time difference between the endpoint times of the voice segments of the two audio segments in the overlapping part is less than a time difference threshold.

[0012] According to an audio transcription method provided by the present invention, the complete audio to be transcribed is divided into multiple audio segments, comprising: Determine the number of segments of the complete audio based on at least one of a maximum number of transcription engine paths, a number of available transcription engine paths, and an audio duration of the complete audio; The complete audio is divided into the audio segments of the specified number.

[0013] The present invention also provides an audio transcription device, comprising the following modules: An audio segmentation unit, used for segmenting the complete audio to be transcribed into a plurality of audio segments, wherein at least one of two adjacent audio segments contains a complete voice segment; An audio transcription unit, used to perform audio transcription on each audio segment in parallel to obtain an audio transcription result of each audio segment; The result merging unit is used to merge the audio transcription results of each audio segment to obtain the audio transcription result of the complete audio.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the audio transcription method described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the audio transcription methods described above.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the audio transcription method described above is implemented.

[0017] The audio transcription method, device, equipment and medium provided by the present invention significantly improve the efficiency and accuracy of audio transcription by ensuring that there is at least one audio segment containing a complete voice segment in adjacent audio segments and by a parallel processing method. The multi-channel transcription engine processes multiple audio segments in parallel, which greatly shortens the transcription time of long audio. By ensuring that there is an audio segment containing a complete voice segment in adjacent audio segments, the voice segments are prevented from being cut, thereby ensuring the consistency and accuracy of the transcribed content. The parallel processing method also improves resource utilization efficiency, allowing audio transcription to run efficiently in a multi-tasking environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a flowchart of the audio transcription method provided by the present invention.

[0020] Figure 2 It is a structural schematic diagram of the audio transcription device provided by the present invention.

[0021] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] With the rapid development of speech recognition technology, audio transcription technology has been widely used in application scenarios such as conference records, voice assistants, and intelligent customer service. The core task of audio transcription is to quickly and accurately convert the voice content in the audio into text to meet the needs of various scenarios for text data. At present, there are two main types of mainstream audio transcription technology solutions: one is to use a single voice engine to transcribe the entire audio. This method can maintain high accuracy and content coherence when processing shorter audio; the other is to divide the long audio into multiple segments, and use multiple voice engines to process each segment in parallel to improve the overall transcription speed. This method is particularly important when a large amount of audio data needs to be processed.

[0024] However, both of these technical solutions have significant deficiencies in practical applications. For the single-channel engine transcription method, although it can maintain the integrity of the content, the transcription process often takes too long when processing long audio, resulting in low efficiency and failure to meet users' needs for quick response. Although multi-channel engine parallel processing can significantly increase the transcription speed, during the audio segmentation process, it often segments in the middle of the sentence, resulting in inaccurate transcription content and incoherent sentences, affecting users' understanding and use of the audio transcription results. This decline in accuracy and loss of sentence fluency has become the main challenge facing audio transcription technology.

[0025] In view of the above problems, an embodiment of the present invention provides an audio transcription method. Figure 1 Schematic diagram of the flow of the audio transcription method provided by the present invention. Figure 1 As shown, the method includes: Step 110, dividing the complete audio to be transcribed into a plurality of audio segments, wherein at least one of two adjacent audio segments contains a complete voice segment.

[0026] In the process of audio transcription, audio segmentation is a key step, which directly affects the final audio transcription accuracy and sentence coherence. The embodiment of the present invention uses a segmentation strategy with overlapping segments to effectively address the problem that in the traditional audio segmentation method, the speech is segmented due to inaccurate segmentation points, which affects the accuracy and fluency of transcription. Specifically, when the complete audio is divided into multiple segments, a certain overlapping area is set between two adjacent segments. This overlapping strategy ensures that at least one of the two adjacent audio segments contains a complete speech segment.

[0027] Alternatively, the embodiment of the present invention may perform voice activity detection (VAD) on the complete audio in advance. The voice activity detection can detect voice activity and silence segments in the audio and provide the start and end point information of the voice segment. The endpoints of the voice segment contained in the complete audio are determined by the result of the voice activity detection, and the complete audio is segmented based on this, thereby ensuring that each audio segment obtained by segmentation contains a complete audio segment, and there is no complete voice segment that is segmented, that is, it is ensured that at least one of the two adjacent audio segments contains a complete voice segment.

[0028] It should be noted that the speech segments here refer to those audio segments containing human voices, which are crucial for accurate transcription because they carry the main content of the speech information. Therefore, during the segmentation process, it is necessary to ensure that each audio segment can completely cover one or more speech segments to avoid semantic loss or inaccuracy caused by the segmentation point falling within the speech segment, and to ensure the accuracy and fluency of the audio transcription results.

[0029] Step 120, performing audio transcription on each audio segment in parallel to obtain an audio transcription result of each audio segment.

[0030] After the audio to be transcribed is segmented, each audio segment will be assigned to a different audio transcription engine for parallel transcription. These transcription engines can be speech recognition models based on deep learning. After training and optimization with a large amount of data, they can accurately convert speech signals into text. Through parallel processing, the audio transcription work of each segment can be carried out simultaneously, which greatly improves the overall efficiency of transcription, especially when processing longer audio, the transcription time can be significantly shortened.

[0031] During the transcription process, the audio transcription engine will analyze the characteristics of the speech signal in the audio and use the pre-trained speech recognition model to accurately convert the speech content in the audio into text. In this process, the audio transcription engine will first pre-process the audio clip, including noise suppression, speech enhancement, etc., to improve the accuracy of subsequent recognition. Then, the engine will extract the acoustic features of the audio, such as spectral features, fundamental frequency, etc., which are crucial for speech recognition. The pre-processed audio clip is decoded by the acoustic model, and the acoustic features are converted into possible speech unit sequences (such as phonemes or morphemes). Acoustic models are usually based on deep learning algorithms, such as recurrent neural networks (RNN), long short-term memory networks (LSTM), and transformer models. After obtaining the decoding results of the acoustic model, the audio transcription engine will further use large language models for correction. Large language models are trained based on a large amount of text data and can capture the co-occurrence relationship of words and grammatical rules in natural language, thereby helping to optimize the audio transcription results into text that is more in line with language habits. At the same time, in order to accurately merge the transcription results of different audio clips later, the audio transcription engine will also mark the corresponding timestamp for each recognized text clip. These timestamps reflect the location where the text content appears in the original audio and are an important basis for merging audio transcription results.

[0032] Through the above operations, each audio transcription engine can independently complete the audio transcription task of its assigned audio clip and output the corresponding audio transcription result. Due to the parallel processing method, the transcription work of multiple audio clips can be carried out simultaneously, thereby significantly improving the overall efficiency of transcription.

[0033] Step 130, merging the audio transcription results of each audio segment to obtain the audio transcription result of the complete audio.

[0034] Specifically, the transcripts are arranged and aligned in the natural order of the original audio, based on the timestamps that are added to each audio segment during the transcription process. These timestamps serve as precise guides to ensure that the order of the text segments matches the chronological order of the audio content.

[0035] For the overlapping areas of adjacent audio clips, an intelligent processing method is adopted. By analyzing the voice activity detection data of adjacent audio clips, the overlapping areas can be accurately identified, and the transcribed content of these overlapping areas can be finely screened and deduplicated according to specific algorithm logic. As important information for recording the start and end time of voice activity in audio, voice activity detection data provides a reliable basis for accurately identifying overlapping areas. By comparing the voice activity detection data of adjacent audio clips, the voice clips that exist in both adjacent clips, that is, the overlapping areas, can be quickly locked. Once the overlapping areas are accurately identified, the content can be finely screened and deduplicated, thereby ensuring that the fluency and accuracy of the original sentences are not damaged while removing redundant content.

[0036] After removing redundant content, the merged text can be considered for further optimization. This includes grammar checking, spelling correction, and the proper use of punctuation to ensure the accuracy and readability of the text. Finally, the transcription results of each audio clip can be merged into a complete document.

[0037] The audio transcription method provided by the embodiment of the present invention significantly improves the efficiency and accuracy of audio transcription by ensuring that there is at least one audio segment containing a complete voice segment in adjacent audio segments and by a parallel processing method. The multi-channel transcription engine processes multiple audio segments in parallel, which greatly shortens the transcription time of long audio. By ensuring that there is an audio segment containing a complete voice segment in adjacent audio segments, the voice segments are prevented from being cut, thereby ensuring the continuity and accuracy of the transcribed content. The parallel processing method also improves the efficiency of resource utilization, allowing audio transcription to run efficiently in a multi-tasking environment.

[0038] Based on the above embodiment, there is an overlapping portion between the two adjacent audio clips; Accordingly, in step 130, the audio transcription results of the audio segments are merged to obtain the audio transcription result of the complete audio, including: Based on the audio transcription results of the two adjacent audio segments, determining the overlap of the speech segments of the two adjacent audio segments in the overlapped portion; Based on the overlap of the voice segments of the two adjacent audio segments in the overlapping part, the audio transcription results of the two adjacent audio segments are merged to obtain the audio transcription result of the complete audio.

[0039] Specifically, in two adjacent audio clips, the overlapping voice clips may appear consistent or inconsistent. This overlap can be judged by whether the time periods corresponding to the transcribed texts in the audio transcription results overlap.

[0040] The overlapped parts are consistent, which means that the transcription results of the two audio clips in the overlapped area are exactly the same, that is, the start and end times of the transcribed text are consistent, and the text content is also the same. In this case, the voice clips in the overlapped part are exactly the same in the two audio clips, and there is no difference in either the timestamp or the actual content of the voice. It can be understood that in the overlapped area of ​​the two clips, the transcribed voice content is completely synchronized in time.

[0041] The inconsistency of the overlapping parts means that there are differences in the speech segments of the two audio clips in the overlapping area. This difference may be reflected in the following aspects: first, the start or end time of the transcription results is inconsistent, that is, the timestamps of the two clips in the overlapping part are different; second, although the overlapping area covers the same audio segment, the transcribed content may be different due to noise, voice quality or recognition errors. In this case, the transcription information of the two clips in the overlapping area cannot be completely matched, and there may be a misalignment in time or content. By analyzing the timestamps and text content of the transcription results of adjacent clips, it is possible to effectively determine whether the speech segments of the two clips in the overlapping part are completely consistent or there are inconsistencies in time or content. This judgment provides a basis for the subsequent merging of transcription results, ensuring that the actual situation of the speech segments can be taken into account when processing the overlapping area, thereby generating a coherent and accurate audio transcription result. For the case where the overlapping parts are consistent, these clips can usually be directly merged without worrying about duplication or omission of content. This situation is relatively simple and can be quickly judged by matching timestamps or voice features. For the case where the overlapping parts are inconsistent, the differences in the overlapping parts need to be further analyzed. The key to dealing with such situations is to ensure that valid speech content is not lost during merging. In the overlapping speech segments, if it is found that the transcription results of the two segments are time-misaligned or content-overlapping, these overlapping areas need to be aligned to generate a coherent transcription result. By analyzing and processing the overlapping parts, the problems of speech segment misalignment or content duplication caused by audio segmentation can be effectively avoided. In the end, the merged audio transcription results remain continuous in time and consistent in content, which not only improves the accuracy of the transcription, but also ensures the fluency and coherence of the sentences, meeting the user's requirements for high-quality transcription results.

[0042] Based on the above embodiment, the step of merging the audio transcription results of the two adjacent audio segments based on the overlap of the voice segments of the two adjacent audio segments in the overlapped portion includes: When the speech segment overlaps, the endpoint times of the speech segments of the two audio segments overlap, merging the audio transcription results of the two audio segments based on the overlapped endpoint times; When the voice segment overlaps in a situation where the endpoint times of the two audio segments in the overlapping parts do not overlap, the audio transcription results of the two audio segments are merged based on the time difference between the endpoint times of the voice segments in the overlapping parts of the two audio segments.

[0043] In the overlapping part of two adjacent audio clips, whether the endpoint time of the voice clips overlaps determines the way the transcription results are merged. In order to ensure the accuracy and consistency of the final transcription results, different merging strategies need to be adopted according to the specific circumstances of the overlapping parts. When the voice clips of two audio clips in the overlapping area have the same endpoint time, the overlapping parts can be directly aligned and merged. At this time, it can be confirmed that their starting and ending points on the timeline are consistent based on the timestamps of the two clips. Based on this, the transcription result of the previous audio clip can be retained to the starting position of the overlapping part, and the transcription result of the next audio clip is continued from the end position of the overlapping part. This processing can avoid repeated transcription of the content in the overlapping area while maintaining temporal consistency.

[0044] If the overlapping parts of two segments are inconsistent in time, the time difference needs to be processed. The time difference of the overlapping area can be determined by comparing the timestamps of the two segments. If the time difference is large, appropriate corrections may be required when merging. The end time of the previous segment can be compared with the start time of the next segment based on the timestamp provided by the voice activity detection to ensure a natural and smooth merge. When the time difference is small, a small range of tolerance can be set to allow slight time misalignment, so as to form a natural voice transition through logical splicing and ensure that the transcription result does not appear abrupt.

[0045] By accurately analyzing and merging the overlapping parts, regardless of whether the timing of the two clips is consistent, it is possible to ensure that the final transcription results remain consistent in content and time, avoid duplication or omission of key content, and thus improve the overall quality of the audio transcription.

[0046] Based on the above embodiment, the step of merging the audio transcription results of the two audio clips based on the overlapping endpoint times includes: In the case where the overlapping endpoint time is the start time of the voice segment, merging the audio transcription result of the first audio segment before the start time and the audio transcription result of the second audio segment after the start time in the two audio segments; When the overlapping endpoint time is the end time of the voice segment, the audio transcription result of the previous audio segment before the end time and the audio transcription result of the next voice segment after the end time are merged.

[0047] Specifically, when the audio transcription results of two audio segments are merged based on the overlapping endpoint time, the situations can be divided into two types: when the overlapping endpoint time is the start time of the voice segment, and when the overlapping endpoint time is the end time of the voice segment.

[0048] When the overlapping endpoint time is the start time of the voice segment, assuming that the transcription result of the overlapping voice segment is "Today the weather is nice, I am going for a walk", and the start time of this part is 10 seconds, the transcription results of the previous audio segment and the next audio segment for this part of the voice segment overlap at this starting time point. At this time, the content of the previous segment before 10 seconds can be retained, and the content of the next segment is also continued from 10 seconds. The merged result completely retains "Today the weather is nice, I am going for a walk." This processing ensures the natural flow of the content and avoids semantic interruption.

[0049] When the overlapping endpoint time is the end time of the voice segment, for example, assuming that the transcription result of the overlapping part is still "The weather is nice today, I am going to take a walk", and the end time of this part is 15 seconds, the transcription results of the previous audio segment and the next audio segment for this part of the voice segment overlap at this end time point. At this time, the merge operation will include all the content of the previous segment before the end time, and the content of the next segment after the end time.

[0050] In this merging operation, the overlapping endpoint times ensure the temporal alignment of the two segments while retaining the complete speech content of each segment. This strategy makes the merged text logically coherent and semantically clear, avoiding content duplication or information loss due to time differences.

[0051] It should be noted that the overlapping part here must include at least one complete voice segment, which has been guaranteed when the audio is segmented.

[0052] Based on the above embodiment, merging the audio transcription results of the two audio segments based on the time difference between the endpoint times of the voice segments in the overlapping parts of the two audio segments includes: Determine the end time of the previous audio segment of the two audio segments at the speech segment of the overlapping portion as the previous end time; Determine, from the start times of the latter audio segment of the two audio segments, a start time with the shortest time difference from the previous end time among the start times of the voice segments in the overlapping portion, as the latter start time; The audio transcription result of the previous audio segment before the previous end time and the audio transcription result of the subsequent voice segment after the subsequent end time are merged.

[0053] The overlapping part may contain multiple voice clips, each of which has its own specific voice activity detection time period. If the start and end times of two adjacent audio clips in the overlapping part are not exactly the same, it is necessary to calculate the time difference and find the minimum time difference to determine the best splicing point. This operation ensures that the audio clips are most closely connected on the timeline to avoid semantic breaks or duplications.

[0054] Specifically, in the overlapping parts of adjacent audio clips, the time difference between them is calculated by comparing the end time of the previous audio clip and the start time of the next audio clip. The time point with the smallest time difference is the most suitable splicing point. The selection of this splicing point is very important because it determines how the two audio clips are connected in content to ensure the coherence and accuracy of the transcription results. When merging, it is necessary to follow the following principles: the previous audio clip retains all the content of the previous audio clip before this time point, and the next audio clip retains all the content of the next audio clip after this time point. For example, suppose the transcription result of the overlapping voice segment is "Today's weather is good, I'm going for a walk, and then I went to the park." The voice activity detection segments of the previous audio segment in the overlapping part are segment1: 58:45 - 59:10, segment2: 59:10 - 59:30, segment3: 59:30 - 59:45; the voice activity detection segments of the latter audio segment in the overlapping part are segment4: 59:12 - 59:25, segment5: 59:25 - 59:40, segment6: 59:40 -60:00. In this case, since there is no completely consistent time point, it is necessary to find a match with the smallest time difference. Through calculation, the time difference between the end time of segment1 (59:10) and the start time of segment4 (59:12) is 2 seconds, which is the minimum time difference. Therefore, 59:10 is selected as the end time of the previous audio segment, and 59:12 is selected as the start time of the next audio segment. In the specific merging operation, firstly, all the transcription results of the previous audio segment before segment2, that is, the content between 58:45 and 59:10, are retained; at the same time, the content starting from segment4 in the next audio segment, that is, all the transcription results after 59:12, will also be retained. Finally, the merged transcription result is the transcription result of the complete overlapping voice segments.

[0055] It should be noted that the time difference here needs to be within a certain time range that allows the merging process, and the error range can be dynamically adjusted, for example, 10 milliseconds.

[0056] Based on the above embodiment, the endpoint time coincidence means that the endpoint times of the voice segments of the two audio segments in the coincident part are the same, or the time difference between the endpoint times of the voice segments of the two audio segments in the coincident part is less than the time difference threshold.

[0057] Specifically, endpoint time overlap means that the endpoint times of the voice segments that appear in the overlapping part of the two previous and next audio segments are the same or close, meeting certain time difference threshold requirements. When the endpoint times of the voice segments of the two segments in the overlapping part are exactly the same, it means that the end time of the voice segment of the previous segment is completely aligned with the start time of the voice segment of the next segment. At this time, the contents of the previous and next segments can be directly spliced ​​together without time misalignment or content duplication. This kind of coincidence with consistent endpoint time can ensure the accuracy and natural connection of the transcription results in the actual merging.

[0058] However, during the audio transcription process, the endpoint times of the previous and next segments are not always exactly the same. To deal with this situation, a certain error is allowed in the endpoint time between the two segments, and this error range is the "time difference threshold". When the time difference between the speech segments of two audio segments is less than the threshold, although their time points are not exactly the same, they can be regarded as having coincident endpoint times, and then reasonable merging can be performed. The setting of the threshold makes the merging operation more flexible, avoiding semantic breaks or speech content duplication caused by small time differences.

[0059] For example, if the voice segment of the previous audio segment ends at 59:10, and the voice segment of the next audio segment starts at 59:12, the time difference between them is 2 seconds. If the time difference threshold is set to 10 milliseconds, the time difference between the two segments is much larger than the threshold, and the endpoint times cannot be considered to coincide, and additional adjustments or processing are required when merging. When the time difference is only 8 milliseconds, since the time difference is less than the time difference threshold, it can be considered that the endpoint times of the two segments coincide, and they can be directly spliced ​​when merging, and the transcription continues from the start time of the next segment, avoiding duplication and interruption of content.

[0060] This judgment method based on the time difference threshold makes it possible to not only rely on precise time alignment during the audio segment merging process, but also to set a flexible time error range to ensure that the audio segment merging in actual applications is more natural and smooth.

[0061] Based on the above embodiment, the step of dividing the complete audio to be transcribed into multiple audio segments includes: Determine the number of segments of the complete audio based on at least one of a maximum number of transcription engine paths, a number of available transcription engine paths, and an audio duration of the complete audio; The complete audio is divided into the audio segments of the specified number.

[0062] Specifically, the segmentation of the complete audio is based on three key factors: the maximum number of transcription engine paths, the number of available transcription engine paths, and the audio duration of the complete audio. These parameters are used to determine the number of audio clips, thereby improving transcription efficiency and resource utilization.

[0063] The maximum number of transcription engines refers to the maximum number of transcription engines that can be used to process audio in parallel at the same time. For example, if a maximum of 5 transcription engines can be used to process audio in parallel, then the audio can be divided into a maximum of 5 segments, and each segment can be transcribed in parallel to speed up the overall processing speed.

[0064] The number of available transcription engines refers to the number of engines that can actually be used for the current task. This value may change due to the resource usage of other tasks or the needs of the current task. For example, when only 3 engines are available, the audio will be divided into 3 segments, each segment is assigned to an engine to ensure the improvement of transcription efficiency.

[0065] The duration of the audio is also an important factor in determining the number of segments. For shorter audio, it is unnecessary to split it into too many segments. For example, audio with a duration of less than 3 minutes can be processed by only one engine. For longer audio, segmented processing can significantly shorten the overall transcription time. For example, if the audio duration is 60 minutes and the number of available transcription engines is 3, the audio can be split into 3 segments, each about 20 minutes long, and processed in parallel by 3 engines, thereby reducing the overall processing time.

[0066] After the number of segments is determined, the audio will be logically divided into the corresponding number of segments. These segments will then be sent to the transcription engine for processing. After each engine has processed its corresponding audio segment, the transcription results will be integrated to finally obtain a complete transcription text.

[0067] Based on the above embodiments, the present invention also provides an audio transcription method. The method uses a voice activity detection engine to segment the audio to improve the transcription speed and maintain the transcription accuracy. Different from fixed time segmentation, the method adopts a dynamic segmentation strategy based on actual voice activity, thereby avoiding transcription errors or semantic incoherence caused by segmentation in the middle of the voice.

[0068] Specifically, first determine the actual number of engines for audio transcription, and decide the number of audio segments based on factors such as the maximum number of transcription engines, the number of available engines, and the audio duration of the complete audio. Then, use the voice activity detection engine to detect voice activity in the audio in real time, automatically identify the start and end positions of the voice segments, and extract the voice endpoint information in the audio. The voice activity detection engine generates an end pointer list for each complete speech, ensuring that each segmented audio segment is segmented at the end point of the complete speech, thereby avoiding transcription errors caused by speech fragmentation.

[0069] Different from the conventional timed segmentation method, this solution uses the voice activity detection engine to accurately locate the voice activity in the audio and performs logical segmentation based on the boundaries of the voice activity, rather than simply segmenting according to the length of time. The specific segmentation process is as follows: First, the entire audio to be transcribed is segmented into segments of a certain byte length (such as 3×1024×1024 bytes) and sent to the voice activity detection engine. The engine extracts the voice activity features of the audio and detects and identifies the start and end positions of each complete speech in the audio. During this process, the voice activity detection engine automatically filters out background noise or silent segments to ensure that the boundaries of the audio segments are located between complete speech segments.

[0070] After the voice activity detection engine completes the detection, it generates an endpoint position list, which contains the pointer position at the end of each complete speech. The segmentation logic is based on these pointer values, and segments are performed according to the natural end points of these speech. At the same time, the start and end positions of each segmentation segment are dynamically adjusted according to the actual duration of the audio and the available engine paths.

[0071] After segmentation, each audio segment will be assigned to a different transcription engine for parallel transcription. At this point, the voice activity detection time of each segment needs to be corrected. Since the audio segments are cut from the original audio, the time returned by the voice activity detection engine is local to the original audio, so it is necessary to recalculate the time by adding the voice activity detection start and end time of each segment to its starting position in the original audio. In this way, the transcription time of each audio segment is ensured to be consistent with the timeline of the original audio.

[0072] In the process of merging audio segments, since the audio is segmented by the voice endpoint detection engine, each segment is segmented at the natural end point of the voice, and there is no overlapping part. Therefore, the transcription results of each audio segment can be directly merged in order without time alignment or overlap processing. The audio transcription device provided by the present invention is described below, and the audio transcription device described below and the audio transcription method described above can be referenced to each other.

[0073] Figure 2 is a schematic diagram of the structure of the audio transcription device provided by the present invention, such as Figure 2 As shown, the device comprises: The audio segmentation unit 210 is used to segment the complete audio to be transcribed into multiple audio segments, wherein at least one of two adjacent audio segments contains a complete voice segment.

[0074] The audio transcription unit 220 is used to perform audio transcription on each audio segment in parallel to obtain an audio transcription result of each audio segment.

[0075] The result merging unit 230 is used to merge the audio transcription results of each audio segment to obtain the audio transcription result of the complete audio.

[0076] The device provided by the embodiment of the present invention significantly improves the efficiency and accuracy of audio transcription by ensuring that there is at least one audio segment containing a complete voice segment in adjacent audio segments and by a parallel processing method. The multi-channel transcription engine processes multiple audio segments in parallel, which greatly shortens the transcription time of long audio. By ensuring that there is an audio segment containing a complete voice segment in adjacent audio segments, the voice segments are prevented from being cut, thereby ensuring the continuity and accuracy of the transcribed content. The parallel processing method also improves resource utilization efficiency, allowing audio transcription to run efficiently in a multi-tasking environment.

[0077] Based on any of the above embodiments, the audio segmentation unit is specifically used for: Determine the number of segments of the complete audio based on at least one of a maximum number of transcription engine paths, a number of available transcription engine paths, and an audio duration of the complete audio; The complete audio is divided into the audio segments of the specified number.

[0078] Based on any of the above embodiments, the result merging unit is specifically used for: The audio transcription results of the audio segments are merged to obtain the audio transcription result of the complete audio, including: Based on the audio transcription results of the two adjacent audio segments, determining the overlap of the speech segments of the two adjacent audio segments in the overlapped portion; Based on the overlap of the voice segments of the two adjacent audio segments in the overlapping part, the audio transcription results of the two adjacent audio segments are merged to obtain the audio transcription result of the complete audio.

[0079] When the speech segment overlaps, the endpoint times of the speech segments of the two audio segments overlap, merging the audio transcription results of the two audio segments based on the overlapped endpoint times; When the voice segment overlaps in a situation where the endpoint times of the two audio segments in the overlapping parts do not overlap, the audio transcription results of the two audio segments are merged based on the time difference between the endpoint times of the voice segments in the overlapping parts of the two audio segments.

[0080] In the case where the overlapping endpoint time is the start time of the voice segment, merging the audio transcription result of the first audio segment before the start time and the audio transcription result of the second audio segment after the start time in the two audio segments; When the overlapping endpoint time is the end time of the voice segment, the audio transcription result of the previous audio segment before the end time and the audio transcription result of the next voice segment after the end time are merged.

[0081] Determine the end time of the previous audio segment of the two audio segments at the speech segment of the overlapping portion as the previous end time; Determine, from the start times of the latter audio segment of the two audio segments, a start time with the shortest time difference from the previous end time among the start times of the voice segments in the overlapping portion, as the latter start time; The audio transcription result of the previous audio segment before the previous end time and the audio transcription result of the subsequent voice segment after the subsequent end time are merged.

[0082] The endpoint time coincidence means that the endpoint times of the speech segments of the two audio segments in the coincident part are the same, or the time difference between the endpoint times of the speech segments of the two audio segments in the coincident part is less than a time difference threshold.

[0083] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the audio transcription method, which includes: Divide the complete audio to be transcribed into multiple audio segments, wherein at least one of two adjacent audio segments contains a complete voice segment; Performing audio transcription on each audio clip in parallel to obtain an audio transcription result of each audio clip; The audio transcription results of each audio segment are merged to obtain the audio transcription result of the complete audio.

[0084] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0085] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can perform the audio transcription method provided by the above methods, the method includes: Divide the complete audio to be transcribed into multiple audio segments, wherein at least one of two adjacent audio segments contains a complete voice segment; Performing audio transcription on each audio clip in parallel to obtain an audio transcription result of each audio clip; The audio transcription results of each audio segment are merged to obtain the audio transcription result of the complete audio.

[0086] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the audio transcription method provided by the above methods is implemented, and the method includes: Divide the complete audio to be transcribed into multiple audio segments, wherein at least one of two adjacent audio segments contains a complete voice segment; Performing audio transcription on each audio clip in parallel to obtain an audio transcription result of each audio clip; The audio transcription results of each audio segment are merged to obtain the audio transcription result of the complete audio.

[0087] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0088] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An audio transcription method, characterized in that: include: Divide the complete audio to be transcribed into multiple audio segments, wherein at least one of two adjacent audio segments contains a complete voice segment; Performing audio transcription on each audio clip in parallel to obtain an audio transcription result of each audio clip; The audio transcription results of each audio segment are merged to obtain the audio transcription result of the complete audio.

2. The audio transcription method according to claim 1, characterized in that: There is an overlapping part between the two adjacent audio clips; The audio transcription results of the audio segments are merged to obtain the audio transcription result of the complete audio, including: Based on the audio transcription results of the two adjacent audio segments, determining the overlap of the speech segments of the two adjacent audio segments in the overlapped part; Based on the overlap of the voice segments of the two adjacent audio segments in the overlapping part, the audio transcription results of the two adjacent audio segments are merged to obtain the audio transcription result of the complete audio.

3. The audio transcription method according to claim 2, characterized in that: The step of merging the audio transcription results of the two adjacent audio segments based on the overlap of the voice segments in the overlapped part of the two adjacent audio segments comprises: When the speech segment overlaps, the endpoint times of the speech segments of the two audio segments overlap, merging the audio transcription results of the two audio segments based on the overlapped endpoint times; When the voice segment overlaps in a situation where the endpoint times of the two audio segments in the overlapping parts do not overlap, the audio transcription results of the two audio segments are merged based on the time difference between the endpoint times of the voice segments in the overlapping parts of the two audio segments.

4. The audio transcription method according to claim 3, characterized in that: The step of merging the audio transcription results of the two audio clips based on the coincident endpoint times includes: In the case where the overlapping endpoint time is the start time of the voice segment, merging the audio transcription result of the first audio segment before the start time and the audio transcription result of the second audio segment after the start time in the two audio segments; When the overlapping endpoint time is the end time of the voice segment, the audio transcription result of the previous audio segment before the end time and the audio transcription result of the next voice segment after the end time are merged.

5. The audio transcription method according to claim 3, characterized in that: The step of merging the audio transcription results of the two audio segments based on the time difference between the endpoint times of the voice segments in the overlapping parts of the two audio segments comprises: Determine the end time of the previous audio segment of the two audio segments at the speech segment of the overlapping portion as the previous end time; Determine, from the start times of the latter audio segment of the two audio segments, a start time with the shortest time difference from the previous end time among the start times of the voice segments in the overlapping portion, as the latter start time; The audio transcription result of the previous audio segment before the previous end time and the audio transcription result of the subsequent voice segment after the subsequent end time are merged.

6. The audio transcription method according to claim 3, characterized in that: The endpoint time coincidence means that the endpoint times of the speech segments of the two audio segments in the coincident part are the same, or the time difference between the endpoint times of the speech segments of the two audio segments in the coincident part is less than a time difference threshold.

7. The audio transcription method according to any one of claims 1 to 6, characterized in that: The step of dividing the complete audio to be transcribed into multiple audio segments includes: Determine the number of segments of the complete audio based on at least one of a maximum number of transcription engine paths, a number of available transcription engine paths, and an audio duration of the complete audio; The complete audio is divided into the audio segments of the specified number.

8. An audio transcription device, characterized in that: include: An audio segmentation unit, used for segmenting the complete audio to be transcribed into a plurality of audio segments, wherein at least one of two adjacent audio segments contains a complete voice segment; An audio transcription unit, used to perform audio transcription on each audio segment in parallel to obtain an audio transcription result of each audio segment; The result merging unit is used to merge the audio transcription results of each audio segment to obtain the audio transcription result of the complete audio.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the audio transcription method as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the audio transcription method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the audio transcription method according to any one of claims 1 to 7 is implemented.

Images