Voice annotation method and system based on long and short audios
By using speech endpoint detection and the Transformer deep learning model to segment long audio, combined with manual annotation and consistency verification strategies, the consistency and accuracy issues in long and short audio annotation are solved, achieving efficient and accurate annotation results.
Patent Information
- Application Number
- CN202410389250.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-02
- Publication Date
- 2025-10-14
AI Technical Summary
In existing speech annotation methods based on long and short audio, the consistency and accuracy issues of annotated data have not been effectively solved. Manual annotation is costly and inefficient, while automatic annotation is difficult to achieve the same level of accuracy as manual annotation in complex scenarios.
The speech endpoint detection method is used to segment long audio into short audio containing semantics. Combined with the Transformer deep learning model and manual annotation, the two annotation results are integrated through consistency verification and fusion strategy to generate the final annotation result.
It improves the efficiency and accuracy of annotation, reduces subjective differences, ensures the consistency and reliability of annotation results, and reduces time and cost.
Smart Images
Figure CN120783791A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of intelligent identification, in particular to a speech annotation method and system based on long and short audio. BACKGROUND
[0002] With the continuous progress and popularization of speech recognition technology, speech data has been widely used in various application scenarios, from voice assistants to voice search to smart home control, etc. This trend has made the demand for speech data annotation grow, especially the accurate annotation of long and short speech segments has become particularly important. Long and short speech segment annotation has a wide range of applications in speech recognition, speech synthesis, sentiment analysis, etc. However, traditional manual annotation methods are inefficient and prone to inconsistent annotation, so an efficient and accurate speech annotation method based on long and short audio is needed to address this challenge.
[0003] Currently, speech annotation based on long and short audio mainly relies on manual annotation and automatic annotation. Manual annotation usually involves professional annotators manually listening to audio and annotating it, which has high accuracy but is slow and costly. Automatic annotation uses machine learning and signal processing techniques to attempt to automate speech annotation, which can improve efficiency but still has challenges in handling complex scenarios and distinguishing speech features.
[0004] In existing speech annotation methods based on long and short audio, a major problem is the consistency and accuracy of annotated data. Although manual annotation can provide high accuracy, subjective differences between annotators inevitably occur during large-scale data annotation, affecting the consistency of the annotation results. While automatic annotation algorithms can improve annotation speed, they often struggle to achieve the accuracy of manual annotation when dealing with complex speech scenarios and distinguishing different speakers. Therefore, the consistency and accuracy of annotated data have become a key bottleneck restricting the application of speech annotation methods based on long and short audio. SUMMARY
[0005] The present application provides a speech annotation method and system based on long and short audio, which can integrate the results of two annotations into a final annotation result through consistency testing and fusion strategies to ensure the consistency and reliability of the annotation result. The present application provides the following technical solutions:
[0006] In a first aspect, the present application provides a speech annotation method based on long and short audio, comprising:
[0007] Obtaining audio data containing long audio and preprocessing the audio data;
[0008] The long audio in the audio data is segmented based on the method of speech endpoint detection to generate short audio containing semantics;
[0009] The Transformer-based deep learning model annotates the short audio containing semantics and generates the first annotation result;
[0010] Manually annotate the short audio containing semantics to generate a second annotation result;
[0011] The final annotation result is generated based on the first annotation result and the second annotation result through consistency checking and fusion strategy methods.
[0012] In a specific implementation scheme, obtaining audio data containing long audio and preprocessing the audio data includes:
[0013] Obtain audio data containing long audio through public voice databases or voice resources on the Internet;
[0014] Use noise removal algorithms to remove ambient noise and interference from audio data;
[0015] Convert the collected audio data into a unified format and unify the sampling rate and bit depth.
[0016] In a specific implementation scheme, the method based on speech endpoint detection segments long audio in audio data to generate short audio containing semantics, including:
[0017] Divide the long audio into small time windows and calculate the short-time energy of the signal in each time window. The calculation formula of short-time energy E(n) is as follows:
[0018]
[0019] Where x(k) represents the amplitude of the audio signal at the kth sampling point, N is the window length, and n represents the number of time windows. The zero-crossing rate of the signal in each time window is calculated to detect changes in the speech signal. The calculation formula for the zero-crossing rate Z(n) is as follows:
[0020]
[0021] Where sign(x(k)) represents the value of the function applied to the audio signal at the kth sampling point;
[0022] Set the thresholds of short-time energy and zero-crossing rate to determine the start and end points of the speech signal, slide the window on the long audio signal, and judge whether the window is the start or end point of the speech segment based on the set thresholds.
[0023] In a specific embodiment, the method based on speech endpoint detection segments long audio in audio data to generate short audio containing semantics, further comprising:
[0024] Use the start and end endpoints obtained by endpoint detection to extract the corresponding fragments from the long audio;
[0025] Use endpoint fine-tuning algorithms to detect possible errors or inaccuracies in endpoints;
[0026] For the generated short audio clips, natural language processing technology is used to evaluate the semantic information they contain and filter out clips that do not contain valid semantics;
[0027] Label and annotate the generated short audio clips, including speech transcription text and emotion labels;
[0028] Expand the semantic information of short audio clips through contextual information or speech processing technology.
[0029] In a specific implementation scheme, the Transformer-based deep learning model annotates a short audio message containing semantics, and generating a first annotation result includes:
[0030] Choose a pre-trained Transformer model and customize it to suit the audio annotation task;
[0031] Fuse audio data with relevant text information, providing both audio feature representation and semantic text representation at the model input;
[0032] Using the pre-trained Transformer model for transfer learning, we fine-tune the model layer by layer on the labeled data of the labeling task through gradual unfreezing to gradually adapt to the semantic labeling requirements of the specific task;
[0033] In the model prediction stage, uncertainty estimation technology is introduced to estimate the label uncertainty of the model prediction;
[0034] Based on the label uncertainty information, an adaptive prediction strategy is designed to dynamically adjust the threshold to optimize the labeling results of short audio clips containing semantics.
[0035] In a specific possible implementation scheme, manually annotating the short audio containing semantics to generate the second annotation result includes:
[0036] Provide the short audio files containing semantics to be annotated to the annotators, and provide relevant annotation guidelines and task instructions;
[0037] Annotators manually annotate each short audio clip according to the annotation guidelines and assign corresponding labels or annotation results to each clip.
[0038] In a specific implementation scheme, the method of generating a final annotation result based on the first annotation result and the second annotation result through consistency checking and fusion strategy includes:
[0039] Comparing and analyzing the first annotation result and the second annotation result to detect the consistency and difference of the annotations;
[0040] Conduct in-depth analysis of inconsistent segments and use consistency metrics, such as Kappa coefficient and F1 score, to quantitatively evaluate the degree of consistency between the first and second annotation results. Based on the consistency metric results, determine the credibility and consistency level of the annotation results.
[0041] For segments with consistent annotation results, a voting strategy is used to select the label with the largest number of occurrences as the final annotation result;
[0042] For segments with different annotation results, a difference processing strategy is used to reduce the impact of annotation differences on the final results.
[0043] In a second aspect, the present application provides a speech annotation system based on long and short audio, which adopts the following technical solutions:
[0044] A speech annotation system based on long and short audio, comprising:
[0045] An audio acquisition module is used to acquire audio data containing long audio and pre-process the audio data;
[0046] The audio segmentation module is used to segment long audio in audio data based on the voice endpoint detection method to generate short audio containing semantics;
[0047] An automatic annotation module, which is used to annotate short audio files containing semantics based on the Transformer deep learning model and generate the first annotation results;
[0048] A manual annotation module, used to manually annotate short audio containing semantics and generate a second annotation result;
[0049] The result fusion module is used to generate a final annotation result based on the first annotation result and the second annotation result through consistency verification and fusion strategy.
[0050] In a third aspect, the present application provides an electronic device comprising a processor and a memory; the memory stores a program, which is loaded and executed by the processor to implement a speech annotation method based on long and short audio as described in the first aspect.
[0051] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the storage medium stores a program, and when the program is executed by a processor, it is used to implement a speech annotation method based on long and short audio as described in the first aspect.
[0052] In summary, the beneficial effects of this application include at least:
[0053] (1) By using speech endpoint detection and deep learning models, this method can automatically segment and annotate long audio files, avoiding the time-consuming and labor-intensive process of traditional manual annotation methods. This automated process improves annotation efficiency and reduces the time and cost of annotation.
[0054] (2) Using a deep learning model to annotate short audio clips avoids subjective differences compared to traditional manual annotation, and the annotation results are more accurate and consistent. Through consistency verification and fusion strategies, the consistency and reliability of the annotation results are further ensured, and the quality of the annotated data is improved.
[0055] (3) Instead of relying on fixed time intervals or manual segmentation, the method dynamically generates short audio segments containing semantic meanings based on speech endpoint detection. This flexibility enables the method to adapt to audio of different lengths and contents, and has a wider range of applications.
[0056] Long audio clips are segmented into short audio segments containing semantic information through speech endpoint detection, rather than relying on manual segmentation or fixed time intervals. This ensures that the generated short audio segments more accurately capture semantic units and avoids the incomplete or overlapping semantic information that may result from fixed time intervals in traditional methods. These short audio segments are annotated using a deep learning model, improving the accuracy and efficiency of annotation. Compared with traditional manual annotation, this avoids subjective differences while reducing the time and cost of annotation. Finally, through consistency verification and fusion strategies, the results of the two annotations are integrated into the final annotation result to ensure the consistency and reliability of the annotation results.
[0057] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application and to implement it in accordance with the contents of the specification, the following is a detailed description of the preferred embodiments of the present application in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] Figure 1 It is a flowchart of the speech annotation method based on long and short audio in an embodiment of the present application.
[0059] Figure 2 This is a structural block diagram of a speech annotation system based on long and short audio in an embodiment of the present application.
[0060] Figure 3 It is a block diagram of an electronic device for voice annotation based on long and short audio in an embodiment of the present application. DETAILED DESCRIPTION
[0061] The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.
[0062] Optionally, the present application uses the voice annotation method based on long and short audio provided in each embodiment as an example for use in an electronic device, where the electronic device is a terminal or a server. The terminal can be a mobile phone, a computer, a tablet computer, etc. This embodiment does not limit the type of electronic device.
[0063] Reference Figure 1 , is a flow chart of a method for speech annotation based on long and short audio provided by an embodiment of the present application, which method includes at least the following steps:
[0064] Step S101: Acquire audio data containing long audio and preprocess the audio data.
[0065] Step S102: Segment the long audio in the audio data based on the voice endpoint detection method to generate short audio containing semantics.
[0066] Step S103: Annotate the short audio containing semantics based on the Transformer deep learning model to generate a first annotation result.
[0067] Step S104: Manually annotate the short audio containing semantics to generate a second annotation result.
[0068] Step S105 : Generate a final annotation result based on the first annotation result and the second annotation result through consistency check and fusion strategy.
[0069] In step S101, audio data containing long audio is first obtained through a public voice database or voice resources on the Internet. The audio data covers various voice scenes and speakers as much as possible, thereby ensuring the representativeness of the data to a certain extent. Subsequently, a noise removal algorithm is used, such as a noise reduction filter or spectral subtraction, to remove environmental noise and interference in the audio data, thereby improving the clarity and quality of the voice. The collected audio data is then converted into a unified format, such as WAV, MP3, etc., and the sampling rate and bit depth are unified to ensure consistency during subsequent processing. In this embodiment, pydub in the audio processing library is used for format conversion.
[0070] In step S102, the speech endpoint detection method is used to segment the long audio in the audio data to generate short audio segments containing semantic meaning. Speech endpoint detection aims to identify the start and end positions of the speech signal to separate the speech signal from background noise or silence, thereby achieving effective segmentation. Specifically, the long audio is first divided into small time windows, and the short-time energy of the signal in each time window is calculated. The calculation formula of short-time energy E(n) is as follows:
[0071]
[0072] Here, x(k) represents the amplitude of the audio signal at the kth sampling point, N is the window length, and n represents the number of time windows. Similarly, the zero-crossing rate of the signal in each time window is calculated to detect changes in the speech signal. The formula for calculating the zero-crossing rate Z(n) is as follows:
[0073]
[0074] Here, sign(x(k)) represents the value of the function applied to the audio signal at the kth sampling point. Thresholds for the short-time energy and zero-crossing rate are then set to determine the start and end points of the speech signal. A window is then slid across the long audio signal, and the thresholds used to determine whether the window represents the start or end point of a speech segment are used.
[0075] During implementation, short audio clips are generated after obtaining the start and end endpoints of a long audio clip. Specifically, the start and end endpoints obtained by endpoint detection are used to extract corresponding segments from the long audio clip. The length of the extracted segments can be adjusted based on task requirements and actual conditions. To account for potential errors or inaccuracies in endpoint detection, an endpoint fine-tuning algorithm can be employed to ensure that the extracted segments contain complete semantic content and exclude non-speech components. The generated short audio segments can be evaluated for their semantic information using speech recognition or other natural language processing techniques. This can help filter out segments that lack valid semantic meaning, thereby improving data quality. The generated short audio segments are labeled and annotated, including information such as speech transcription text and sentiment tags. These tags can assist in subsequent data analysis and processing tasks. Based on task requirements and actual conditions, the length of the generated short audio segments can be adjusted to meet the needs of subsequent processing tasks. This can be achieved by increasing or decreasing the segment length. The semantic information of the short audio segments can be expanded using contextual information or speech processing techniques to improve their ability to understand and express the speech content.
[0076] In step S103, a pre-trained Transformer model, such as BERT or GPT series models, is first selected and further customized to adapt to the audio annotation task. The customization process includes adding additional attention mechanisms and self-attention layers to improve the model's understanding and extraction capabilities of semantic information. The audio data is then fused with relevant text information, and audio feature representations (such as Mel spectrograms) and semantic text representations (such as speech transcription text) are provided at the input of the model at the same time. The model can simultaneously focus on audio and text information through a multi-head attention mechanism to better understand and annotate audio clips. A variety of data enhancement techniques are then used, such as sound distortion, speed changes, noise addition, etc., to increase the diversity of data and the generalization ability of the model. In particular, for short audio clips containing semantics, it is necessary to ensure that data enhancement does not lose the semantic information of the clip. Transfer learning is performed using the pre-trained Transformer model, and fine-tuning is performed on the labeled data of the annotation task. Fine-tuning is performed layer by layer in a progressive thawing manner to gradually adapt to the semantic annotation requirements of specific tasks. During the model prediction phase, uncertainty estimation techniques, such as dropout post-sampling or Monte Carlo dropout, are introduced to estimate the uncertainty of the model's predicted labels. This helps improve confidence in the prediction results. Based on this label uncertainty information, adaptive prediction strategies can be designed. Dynamically adjust thresholds or employ different post-processing methods to optimize the labeling results for short, semantically meaningful audio clips.
[0077] In step S104, the short audio clips containing semantics to be annotated are provided to annotators, along with relevant annotation guidelines and task instructions to ensure they understand the annotation criteria and requirements. Following the annotation guidelines, annotators manually annotate each short audio clip, assigning corresponding labels or annotation results to each clip. Annotators should use their expertise and experience to accurately and consistently annotate the semantics of each clip.
[0078] In step S105, the first and second annotation results are first compared and analyzed to check for consistency and discrepancies. Particular attention is paid to inconsistencies between the annotation results, and in-depth analysis is performed on inconsistent segments. Consistency metrics, such as the Kappa coefficient and F1 score, are then used to quantitatively assess the degree of consistency between the first and second annotation results. Based on the consistency metrics, the credibility and consistency of the annotation results are determined. For segments with consistent annotation results, a voting strategy is directly adopted, selecting the label with the highest number of occurrences as the final annotation result. This improves the credibility and stability of the annotation results. For segments with discrepancies in the annotation results, a reasonable discrepancy handling strategy is adopted. Methods such as manual review, re-annotation, and automatic correction based on discrepancies can be considered to reduce the impact of annotation discrepancies on the final result. Finally, the first and second annotation results, after consistency verification and the annotation result fusion strategy, are integrated. Identical portions are retained, while discrepancies are appropriately handled. The final annotation result is generated, assigning a specific label or annotation result to each audio segment. The final annotation result should be the result of comprehensive consideration and processing to ensure the accuracy and consistency of the annotation data. Through the above steps, the first and second annotation results can be comprehensively utilized to generate the final annotation result, thereby making full use of the information from both annotations, reducing annotation errors, and improving the quality and reliability of the annotation data.
[0079] In summary, the present application divides long audio into short audio segments containing semantic information through the voice endpoint detection method, rather than relying on manual segmentation or fixed time intervals for division. This ensures that the generated short audio segments capture semantic units more accurately, avoiding the problem of incomplete or overlapping semantic information that may be caused by fixed division time intervals in traditional methods. These short audio segments are annotated through a deep learning model, thereby improving the accuracy and efficiency of annotation. Compared with traditional manual annotation, this avoids subjective differences and reduces the time and cost of annotation. Finally, through consistency verification and fusion strategies, the results of the two annotations are integrated into the final annotation result to ensure the consistency and reliability of the annotation results.
[0080] Figure 2 This is a block diagram of a speech annotation system based on long and short audio provided by an embodiment of the present application. The device includes at least the following modules:
[0081] The audio acquisition module is used to acquire audio data containing long audio and preprocess the audio data.
[0082] The audio segmentation module is used to segment long audio in audio data based on the voice endpoint detection method to generate short audio containing semantics.
[0083] The automatic annotation module is used to annotate short audio containing semantics based on the Transformer deep learning model and generate the first annotation result.
[0084] The manual annotation module is used to manually annotate short audio containing semantics and generate a second annotation result.
[0085] The result fusion module is used to generate a final annotation result based on the first annotation result and the second annotation result through consistency verification and fusion strategy.
[0086] For relevant details, please refer to the above method embodiment.
[0087] Figure 3 4 is a block diagram of an electronic device provided in one embodiment of the present application. The device includes at least a processor 401 and a memory 402.
[0088] The processor 401 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 401 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 401 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 401 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 401 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.
[0089] The memory 402 can include one or more computer-readable storage media. The computer-readable storage media can be non-transitory. The memory 402 can also include high-speed random access memory and can include non-volatile memory, such as one or more magnetic disk storage devices, optical storage devices, flash memory devices, or other non-volatile solid-state storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 402 stores at least one instruction for execution by the processor 401 to implement the long-short audio-based speech tagging method provided by the method embodiments in the present application.
[0090] In some embodiments, the electronic device can also optionally include a peripheral device interface and at least one peripheral device. The processor 401, the memory 402, and the peripheral device interface can be connected through a bus or a signal line. Each peripheral device can be connected to the peripheral device interface through a bus, a signal line, or a circuit board. Illustratively, the peripheral device includes, but is not limited to, a radio frequency circuit, a touch display screen, an audio circuit, a power supply, and the like.
[0091] Of course, the electronic device can also include fewer or more components, and the present embodiment does not limit this.
[0092] Optionally, the present application also provides a computer-readable storage medium, which stores a program. The program is loaded and executed by the processor to implement the long-short audio-based speech tagging method of the above-mentioned method embodiments.
[0093] Optionally, the present application also provides a computer product, which includes a computer-readable storage medium. The computer-readable storage medium stores a program. The program is loaded and executed by the processor to implement the long-short audio-based speech tagging method of the above-mentioned method embodiments.
[0094] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combinations of the technical features do not contradict each other, they should be considered within the scope of the present application.
[0095] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the patent scope of the present application. It should be noted that, for those skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the patent protection scope of the present application should be subject to the appended claims.
Claims
1. A speech annotation method based on long and short audio, characterized in that: The method comprises: Obtain audio data containing long audio and preprocess the audio data; The long audio in the audio data is segmented based on the method of speech endpoint detection to generate short audio containing semantics; The Transformer-based deep learning model annotates the short audio containing semantics and generates the first annotation result; Manually annotate the short audio containing semantics to generate a second annotation result; The final annotation result is generated based on the first annotation result and the second annotation result through consistency checking and fusion strategy methods.
2. The speech annotation method based on long and short audio according to claim 1, characterized in that: The obtaining of audio data containing long audio and preprocessing the audio data includes: Obtain audio data containing long audio through public voice databases or voice resources on the Internet; Use noise removal algorithms to remove ambient noise and interference from audio data; Convert the collected audio data into a unified format and unify the sampling rate and bit depth.
3. The speech annotation method based on long and short audio according to claim 1, characterized in that: The method based on speech endpoint detection segments long audio in audio data to generate short audio containing semantics, including: Divide the long audio into small time windows and calculate the short-time energy of the signal in each time window. The calculation formula of short-time energy E(n) is as follows: Where x(k) represents the amplitude of the audio signal at the kth sampling point, N is the window length, and n represents the number of time windows. The zero-crossing rate of the signal in each time window is calculated to detect changes in the speech signal. The zero-crossing rate Z(n) is calculated as follows: Where sign(x(k)) represents the value of the function applied to the audio signal at the kth sampling point; Set the thresholds of short-time energy and zero-crossing rate to determine the start and end points of the speech signal, slide the window on the long audio signal, and judge whether the window is the start or end point of the speech segment based on the set thresholds.
4. The speech annotation method based on long and short audio according to claim 3, characterized in that: The method based on speech endpoint detection segments the long audio in the audio data to generate short audio containing semantics, further comprising: Use the start and end endpoints obtained by endpoint detection to extract the corresponding fragments from the long audio; Use endpoint fine-tuning algorithms to detect possible errors or inaccuracies in endpoints; For the generated short audio clips, natural language processing technology is used to evaluate the semantic information they contain and filter out clips that do not contain valid semantics; Label and annotate the generated short audio clips, including speech transcription text and emotion labels; Expand the semantic information of short audio clips through contextual information or speech processing technology.
5. The method for speech annotation based on long and short audio according to claim 1, characterized in that: The Transformer-based deep learning model annotates the short audio containing semantics, and generates a first annotation result including: Choose a pre-trained Transformer model and customize it to suit the audio annotation task; Fuse audio data with relevant text information, providing both audio feature representation and semantic text representation at the model input; Using the pre-trained Transformer model for transfer learning, we fine-tune the model layer by layer on the labeled data of the labeling task through gradual unfreezing to gradually adapt to the semantic labeling requirements of the specific task; In the model prediction stage, uncertainty estimation technology is introduced to estimate the label uncertainty of the model prediction; Based on the label uncertainty information, an adaptive prediction strategy is designed to dynamically adjust the threshold to optimize the labeling results of short audio clips containing semantics.
6. The method for speech annotation based on long and short audio according to claim 1, characterized in that: Manually annotating the short audio containing semantics to generate a second annotation result includes: Provide the short audio files containing semantics to be annotated to the annotators, and provide relevant annotation guidelines and task instructions; Annotators manually annotate each short audio clip according to the annotation guidelines and assign corresponding labels or annotation results to each clip.
7. The method for speech annotation based on long and short audio according to claim 1, characterized in that: The method of generating a final annotation result based on the first annotation result and the second annotation result through consistency checking and fusion strategy includes: Comparing and analyzing the first annotation result and the second annotation result to detect the consistency and difference of the annotations; Conduct in-depth analysis of inconsistent segments and use consistency metrics, such as Kappa coefficient and F1 score, to quantitatively evaluate the degree of consistency between the first and second annotation results. Based on the consistency metric results, determine the credibility and consistency level of the annotation results. For segments with consistent annotation results, a voting strategy is used to select the label with the largest number of occurrences as the final annotation result; For segments with different annotation results, a difference processing strategy is used to reduce the impact of annotation differences on the final results.
8. A speech annotation system based on long and short audio, characterized by: include: An audio acquisition module is used to acquire audio data containing long audio and pre-process the audio data; The audio segmentation module is used to segment long audio in audio data based on the voice endpoint detection method to generate short audio containing semantics; An automatic annotation module, which is used to annotate short audio files containing semantics based on the Transformer deep learning model and generate the first annotation results; A manual annotation module, used to manually annotate short audio containing semantics and generate a second annotation result; The result fusion module is used to generate a final annotation result based on the first annotation result and the second annotation result through consistency verification and fusion strategy.
9. An electronic device, characterized in that: The device includes a processor and a memory; the memory stores a program, and the program is loaded and executed by the processor to implement a speech annotation method based on long and short audio according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that The storage medium stores a program, which, when executed by a processor, is used to implement a speech annotation method based on long and short audio according to any one of claims 1 to 7.