Speaker labeling method and apparatus, electronic device, and storage medium

Abstract

The application provides a speaker labeling method and device, electronic equipment and storage medium, and relates to the technical field of voice processing. The method comprises the following steps: acquiring voice data, and transcribing the voice data to obtain transcribed text; dividing the voice data to obtain at least one voice segment, and extracting the voiceprint features of each voice segment; for each set time period, performing clustering analysis on the voiceprint features in the set time period to obtain a speaker separation result; using the speaker separation result to perform clustering update on a first historical speaker separation result, and labeling the transcribed text based on the updated first historical speaker separation result; wherein the first historical speaker separation result is determined based on the speaker separation results corresponding to at least one set time period before the set time period. The technical scheme provided by the application can label the voice transcribed text with a speaker, and improve the accuracy of speaker labeling.

Description

Technical Field

[0001] This invention relates to the field of speech processing technology, and in particular to a speaker annotation method, apparatus, electronic device, and storage medium. Background Technology

[0002] Voice is the most natural way for people to communicate and an important means of obtaining information. For example, speeches, court hearings, and meetings all involve communication via voice. These situations typically require recording the content of the communication, organizing it into meeting minutes or work reports.

[0003] With the development of artificial intelligence technology, speech-to-text systems are increasingly being used in scenarios such as meeting minutes and court transcripts. By transcribing speech in real time or offline, speech content can be converted into text, improving the efficiency of meeting minutes and court transcripts. In multi-person discussion scenarios, the transcribed text will come from different speakers. If the transcribed text can be labeled with the corresponding speaker, it will help readers read and archive the document. Summary of the Invention

[0004] This invention provides a speaker annotation method, apparatus, electronic device, and storage medium to achieve speaker annotation of speech-to-text, facilitating subsequent text reading and archiving.

[0005] This invention provides a speaker annotation method, comprising:

[0006] Acquire speech data and transcribe the speech data to obtain transcribed text;

[0007] The speech data is segmented to obtain at least one speech segment, and the voiceprint features of each speech segment are extracted.

[0008] For each set time period, cluster analysis is performed on the voiceprint features within the set time period to obtain speaker separation results;

[0009] The first historical speaker separation result is clustered and updated using the speaker separation result, and the transcribed text is labeled with speakers based on the updated first historical speaker separation result; wherein, the first historical speaker separation result is determined based on the speaker separation result corresponding to at least one set time period before the set time period.

[0010] According to a speaker annotation method provided by the present invention, before performing clustering updates on the first historical speaker separation results using the speaker separation results, the method further includes:

[0011] Obtain the first speaker separation result corresponding to the first set time period, wherein the first set time period is the previous set time period of the set time period;

[0012] The first speaker separation result is used to cluster and update the second historical speaker separation result to obtain the first historical speaker separation result; wherein, the second historical speaker separation result is determined based on the speaker separation result corresponding to at least one set time period before the first set time period.

[0013] According to a speaker annotation method provided by the present invention, the step of performing speaker annotation on the transcribed text based on the updated first historical speaker separation result includes:

[0014] Upon detecting a speech end command, based on the final updated first historical speaker separation result, the transcribed text content corresponding to consecutive speech segments belonging to the same speaker is integrated to obtain integrated text content.

[0015] The integrated text content is labeled with the speaker's identification information.

[0016] According to a speaker annotation method provided by the present invention, the step of annotating the integrated text content with speaker identification information includes:

[0017] Obtain the initial identification information of the integrated text content annotation;

[0018] During the process of annotating the integrated text content based on the finally obtained updated first historical speaker separation results, it is detected whether the initial identification information is manually annotated identification information;

[0019] If the initial identifier information of the integrated text content is manually labeled, the integrated text content will be labeled with the manually labeled identifier information.

[0020] According to a speaker annotation method provided by the present invention, after performing cluster analysis on the voiceprint features within the set time period to obtain speaker separation results, the method further includes:

[0021] Based on the speaker separation results, speaker annotation is performed on the transcribed text content corresponding to the set time period to obtain speaker-annotated text.

[0022] The speaker's annotated text is displayed in real time.

[0023] According to a speaker annotation method provided by the present invention, the method further includes:

[0024] Receive modification instructions;

[0025] In response to the modification instruction, the speaker's tagged text is modified.

[0026] According to the speaker annotation method provided by the present invention, the updated first historical speaker separation result includes speaker identification information;

[0027] After performing speaker annotation on the transcribed text based on the updated first historical speaker separation results, the method further includes:

[0028] Based on the speaker's identification information, the transcribed text after the speaker's annotation is displayed; wherein, the display parameters of the identification information of different speakers are different, and / or, the identification information of different speakers is represented by the parameter information of the voice playback progress bar.

[0029] According to a speaker annotation method provided by the present invention, after updating the first historical speaker separation result by clustering using the speaker separation result, the method further includes:

[0030] Save the updated first historical speaker separation result, which includes the speaker's identification information and voiceprint features.

[0031] The present invention also provides a speaker annotation device, comprising:

[0032] The voice acquisition module is used to acquire voice data;

[0033] The transcription module is used to transcribe the speech data to obtain transcribed text;

[0034] The processing module is used to segment the speech data based on speech boundary detection to obtain at least one speech segment, and extract the voiceprint features of each speech segment;

[0035] The analysis module is used to perform cluster analysis on the voiceprint features within each set time period to obtain speaker separation results.

[0036] The annotation module is used to cluster and update the first historical speaker separation result using the speaker separation result, and to annotate the transcribed text with speakers based on the updated first historical speaker separation result; wherein, the first historical speaker separation result is determined based on the speaker separation result corresponding to at least one set time period before the set time period.

[0037] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the speaker annotation methods described above.

[0038] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the speaker annotation method as described above.

[0039] The speaker annotation method, apparatus, electronic device, and storage medium provided by this invention segment the acquired speech data, extract the voiceprint features of each segmented speech segment, perform cluster analysis on the voiceprint features within each set time period to obtain speaker separation results, update the first historical speaker separation results using the speaker separation results, and annotate the transcribed text obtained from the speech data based on the updated first historical speaker separation results. This achieves speaker annotation of the transcribed text, which can distinguish the speakers corresponding to the text content, helping readers understand the text and facilitating subsequent text reading and archiving. Moreover, the first historical speaker separation results are determined based on the speaker separation results corresponding to at least one set time period before the set time period. Thus, the updated first historical speaker separation results comprehensively consider the speech characteristics of the current time period and the time periods before it, improving the accuracy of the speaker separation results. Therefore, using the updated first historical speaker separation results to annotate the transcribed text further improves the accuracy of speaker annotation. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0041] Figure 1 This is a flowchart illustrating the speaker annotation method provided in an embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram illustrating the principle of clustering and updating the first historical speaker separation results in an embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram illustrating the principle of the speaker annotation method provided in this embodiment of the invention;

[0044] Figure 4This is a schematic diagram of the speech transcription interface in an embodiment of the present invention;

[0045] Figure 5 This is a schematic diagram of the speaker annotation device provided in an embodiment of the present invention;

[0046] Figure 6 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0048] It should be noted that the serial numbers assigned to components or described objects in this invention, such as "first" and "second", are only used to distinguish the described objects and do not have any sequential or technical meaning.

[0049] In applications such as meeting content organization and speech transcription, speaker separation technology plays an important role. It can classify and organize the audio data of each frame in the speech data according to different speakers, and merge multiple frames of audio data belonging to the same speaker into a speech segment, so that each speech segment corresponds to a speaker, so that speaker information can be marked on each speech segment later.

[0050] In related technologies, speaker separation in speech-to-text scenarios can include real-time speaker separation and offline speaker separation. Real-time speaker separation can annotate speaker information in real time during meeting recording and return the annotated speaker information along with the transcription result. However, because it can only utilize speech data collected in the current time period, the amount of data analyzed is limited, resulting in low accuracy. Offline speaker separation, on the other hand, separates the speakers from the recorded audio after the meeting ends and then annotates the transcription result with speaker information. It utilizes richer speech data for speaker separation and has higher accuracy than real-time speaker separation, but it is more time-consuming, affecting the timely production of the first draft of the text.

[0051] Based on this, embodiments of the present invention provide a speaker annotation method, which can segment acquired speech data and extract the voiceprint features of each segmented speech segment; for each set time period, cluster analysis is performed on the voiceprint features within the set time period to obtain speaker separation results; the speaker separation results are used to cluster and update the first historical speaker separation results, and speaker annotation is performed on the transcribed text obtained from the speech data based on the updated first historical speaker separation results; wherein, the first historical speaker separation results are determined based on the speaker separation results corresponding to at least one set time period prior to the set time period. By annotating the transcribed text with speakers, readers can better understand the text, facilitating subsequent text reading and archiving. Moreover, the updated first historical speaker separation results comprehensively consider the speech characteristics of the current time period and the time periods before it, which can improve the accuracy of speaker separation results, thereby improving the accuracy of speaker annotation.

[0052] The following is combined Figures 1-4 The speaker annotation method of the present invention is described below. This speaker annotation method can be applied to electronic devices such as servers, mobile phones, computers, and speech-to-text devices, and can also be applied to speaker annotation devices installed in such electronic devices. These speaker annotation devices can be implemented through software, hardware, or a combination of both. For example, a speech-to-text system can run in the electronic device, and system functions for implementing the speaker annotation method provided in the embodiments of the present invention can be integrated into this speech-to-text system.

[0053] Figure 1 An exemplary flowchart of the speaker annotation method provided in an embodiment of the present invention is shown below. Figure 1 As shown, the speaker annotation method may include the following steps 110 to 140.

[0054] Step 110: Acquire speech data and transcribe the speech data to obtain transcribed text.

[0055] Voice data can be real-time voice data captured by a voice acquisition device (such as a microphone) or pre-recorded offline voice data. Electronic devices can acquire real-time voice data from the voice acquisition device or read pre-recorded offline voice data from storage.

[0056] For example, an electronic device can activate a transcription engine, and the voice data collected by the voice acquisition device can be sent to the transcription engine. After receiving the voice data, the transcription engine will transcribe the voice data into text to obtain the transcribed text.

[0057] Step 120: Segment the speech data to obtain at least one speech segment, and extract the voiceprint features of each speech segment.

[0058] Electronic devices can activate a speaker separation engine to segment the acquired speech data, obtain at least one speech segment, and then use a voiceprint recognition algorithm to extract the voiceprint features of each speech segment.

[0059] For example, the speaker segmentation engine can segment the acquired speech data based on Voice Activity Detection (VAD).

[0060] Understandably, electronic devices can simultaneously activate the transcription engine and the speaker separation engine, sending the acquired speech data to the transcription engine and the speaker separation engine respectively. The transcription engine converts the speech data into text, while the speaker separation engine separates the speech data from the speaker.

[0061] Step 130: For each set time period, perform cluster analysis on the voiceprint features within the set time period to obtain the speaker separation results.

[0062] At set time intervals, the electronic device can perform a clustering analysis of voiceprint features using the speaker separation engine. Specifically, for each set time interval, the speaker separation engine can use a clustering algorithm to perform clustering analysis of the voiceprint features within that time interval, such as density-based spatial clustering of applications with noise (DBSCAN). The resulting speaker separation results can include the speaker's identification information and voiceprint feature information.

[0063] Step 140: Use the speaker separation results to cluster and update the first historical speaker separation results, and perform speaker annotation on the transcribed text based on the updated first historical speaker separation results.

[0064] Each time the speaker separation engine obtains a speaker separation result for a set time period, it clusters and merges that result with the first historical speaker separation result preceding that set time period to update the first historical speaker separation result. The first historical speaker separation result is determined based on speaker separation results corresponding to at least one set time period preceding the set time period.

[0065] For example, before updating the first historical speaker separation result using the speaker separation result through clustering, the speaker annotation method provided in this embodiment of the invention may further include a step of determining the first historical speaker separation result. This may include the following steps: obtaining the first speaker separation result corresponding to a first set time period, where the first set time period is the previous set time period; updating the second historical speaker separation result through clustering using the first speaker separation result to obtain the first historical speaker separation result; wherein the second historical speaker separation result is determined based on the speaker separation result corresponding to at least one set time period prior to the first set time period. Based on this, the first historical speaker separation result can be continuously updated based on newly obtained speaker separation results.

[0066] For example, Figure 2 An exemplary diagram illustrates the principle of clustering and updating the first historical speaker separation results. (Refer to...) Figure 2 As shown, assuming the duration of a given time period is T, L1 to L4 represent the speech data corresponding to each time period, C1, C2, C4, and C6 represent the speaker separation results obtained in real time for each time period, and C3, C5, and C7 represent the first historical speaker separation results obtained at each stage. It is understandable that, for speech data, when dividing by time period, the duration of the last segment of speech may be shorter than the duration of the divided time period. Figure 2 In this context, the duration of L4 is less than or equal to T.

[0067] according to Figure 2 In the L-stage, the speaker separation engine can use VAD (Voice over Audio) to segment the speech data acquired within a preset time interval, obtaining at least one speech segment, with each speech segment corresponding to a time segment. Then, the speaker separation engine can use a voiceprint recognition algorithm to extract the voiceprint features of the speech segment corresponding to each time segment. In the C-stage, the speaker separation engine can use clustering algorithms such as DBSCAN to cluster the obtained voiceprint features. Speech segments with similar voiceprint features will be determined to belong to the same speaker, obtaining the speaker separation result. This speaker separation result can be re-clustered and merged with the previously clustered results.

[0068] Specifically, C1 is the speaker separation result obtained after clustering the voiceprint features of the first speech data segment L1 to be processed within the first set time period. C2 is the speaker separation result obtained after clustering the voiceprint features of the second speech data segment L2 to be processed within the second set time period. After obtaining C2, C2 and C1 are clustered and merged to obtain the merged clustering result C3. At this time, the first historical speaker separation result is updated from C1 to C3. Then, speech data segment L3 is obtained in the third set time period, and its corresponding speaker separation result is C4. After obtaining C4, C4 and C3 are clustered and merged to obtain C5, that is, the first historical speaker separation result is updated from C3 to C5. This process continues until the last speech data segment is processed, resulting in the final updated first historical speaker separation result. For example, in... Figure 2 C7 was obtained from this.

[0069] Understandably, the clustering and merging process involves adding new speech data segment voiceprint feature information to the existing clustering and merging results, and then re-clustering all voiceprint features. This process may modify previous clustering results. The first historical speaker separation results will be continuously updated during the clustering and merging process.

[0070] In one example embodiment, the speaker separation engine can update the first historical speaker separation result at each set time interval, and then use the updated first historical speaker separation result to perform speaker annotation on the transcribed text corresponding to the current set time interval, or perform speaker annotation on the transcribed text of the current set time interval and each set time interval before the current set time interval. The transcribed text and the first historical speaker separation result contain their respective corresponding time information, and the correspondence between the speaker and the transcribed text content can be determined based on the correspondence between the time information during speaker annotation.

[0071] For example, in Figure 2 After obtaining C5, C5 can be used to perform speaker annotation on the transcribed text content corresponding to L3, or on the transcribed text content corresponding to L1 to L3.

[0072] In one example embodiment, the speaker separation engine can perform speaker annotation on the transcribed text of the entire recording after the recording ends or after detecting the end marker of the voice data, using the finally obtained updated first historical speaker separation result. Specifically, speaker annotation on the transcribed text based on the updated first historical speaker separation result can include: when a voice end command is detected, integrating the transcribed text content corresponding to consecutive voice segments belonging to the same speaker based on the finally obtained updated first historical speaker separation result to obtain integrated text content; and annotating the integrated text content with speaker identification information. The detection of a voice end command can include: detecting a control command to stop recording, or detecting an end marker of voice data, etc.

[0073] For example, in Figure 2 In the process, after the meeting ends and recording stops, the last speech data segment L4 is obtained. The speaker separation result C6 corresponding to speech data segment L4 is clustered and merged with the previous clustering and merging result C5 to obtain the final updated first historical speaker separation result C7. C7 will represent the speaker separation result of the entire speech data from L1 to L4. C7 can be used to integrate the transcribed text content corresponding to consecutive speech segments belonging to the same speaker. For example, in C7, 1 to 3 seconds correspond to speaker A, 3 to 4 seconds are silent segments, 4 to 7 seconds correspond to speaker A, and 7 to 10 seconds correspond to speaker B. The transcribed text content corresponding to 1 to 7 seconds can be integrated together and labeled with the identification information of speaker A, and the transcribed text content corresponding to 7 to 10 seconds can be integrated together and labeled with the identification information of speaker B.

[0074] The speaker annotation method provided in this invention segments the acquired speech data, extracts the voiceprint features of each segmented speech segment, performs cluster analysis on the voiceprint features within each set time period to obtain speaker separation results, updates the first historical speaker separation results using the speaker separation results, and annotates the transcribed text obtained from the speech data based on the updated first historical speaker separation results. This achieves speaker annotation of the transcribed text, which can distinguish the speakers corresponding to the text content, helping readers understand the text and facilitating subsequent text reading and archiving. Moreover, the first historical speaker separation results are determined based on the speaker separation results corresponding to at least one set time period before the set time period. Thus, the updated first historical speaker separation results comprehensively consider the speech characteristics of the current time period and the time periods before it, improving the accuracy of the speaker separation results. Therefore, using the updated first historical speaker separation results to annotate the transcribed text can improve the accuracy of speaker annotation.

[0075] In addition, combined Figure 2 If t represents the time taken to calculate the speaker separation result, then the time to calculate Cn can be denoted as tn, and the duration of the last speech data segment L4 can be denoted as T4. Therefore, the delay from the end of recording to the generation of the transcribed text annotated with speaker identification information can include:

[0076] ①If (t4+t5)≤T4, then the delay = t6+t7;

[0077] ②If (t4+t5)>T4, then the delay is Delay=t4+t5-T4+t6+t7.

[0078] In this embodiment of the invention, speech transcription and speaker separation can be processed separately by different engines. The updates of speaker separation and the first historical speaker separation results can also be distributed across different kernels. Therefore, the sum of the processing time of the speech data segment processed in the current time period and the processing time of cluster merging usually does not exceed the duration of the speech data segment recorded in the next stage. Thus, the delay is mainly in the case of ① mentioned above. The final delay can be expressed as the sum of the processing time of L4 to obtain C6 and the processing time of cluster merging to obtain C7. That is, the final delay can be expressed as the sum of the processing time of the last speech data segment and the processing time of cluster merging. This will not result in a long delay. While improving the accuracy of speaker separation results, the delay is short, ensuring the timeliness of the manuscript output.

[0079] based on Figure 1 The speaker annotation method corresponding to the embodiment, Figure 3 An exemplary schematic diagram of the speaker annotation method provided in an embodiment of the present invention is shown below. Figure 3 As shown, during meeting recording, electronic devices can collect voice data through voice acquisition devices. The collected voice data is then sent to a transcription engine and a speaker separation engine for speech transcription and speaker separation processing. After the meeting ends and recording stops, the electronic devices can integrate the transcribed text based on time correspondence and the final speaker separation results. Text content belonging to the same speaker is integrated along the timeline, and speaker identification information is annotated onto the integrated text based on the final speaker separation results. During the annotation process, the integrated text content can be traversed to determine whether each piece of integrated text has been manually annotated with speaker identification information. If it has been manually annotated, the integrated text content is based on the manually annotated identification information; otherwise, the speaker separation results obtained from the speaker separation engine are used for annotation.

[0080] Combination Figure 1 and Figure 3In one example embodiment, annotating speaker identification information in integrated text content may include: obtaining initial identification information for integrated text content annotation; during the process of annotating the integrated text content based on the finally obtained updated first historical speaker separation results, detecting whether the initial identification information is manually annotated; and if the initial identification information for integrated text content annotation is manually annotated, then annotating the integrated text content with manually annotated identification information. The initial identification information may be speaker identification information annotated during the recording process based on real-time speaker separation results. Users can modify the real-time annotated speaker identification information during the recording process; correspondingly, the initial identification information may include manually annotated identification information.

[0081] based on Figure 1 In one example embodiment of the speaker annotation method corresponding to the embodiments, after obtaining the annotated transcribed text, the annotated transcribed text can be displayed in a set display format. Specifically, the updated first historical speaker separation result includes speaker identification information. After annotating the transcribed text based on the updated first historical speaker separation result, the method may further include: displaying the annotated transcribed text based on the speaker identification information; wherein the display parameters of the identification information of different speakers are different, and / or, the identification information of different speakers is represented by the parameter information of the voice playback progress bar. The display parameters may include color, etc., and the parameter information of the voice playback progress bar may include playback interval information and color information, etc.

[0082] For example, different speakers' identification information can be displayed in different colors to facilitate locating the speech content of different speakers. Another example is the display interface of the transcribed text after speaker annotation, which can show a speech playback progress bar control. This speech playback bar control can correspond to the content of the transcribed text based on time information, and different speakers' corresponding playback segments have different colors. In this way, the speech content of different speakers can be located from the speech playback progress bar based on different colors.

[0083] based on Figure 1 In one example embodiment of the speaker annotation method corresponding to the embodiments, during the process of clustering analysis of voiceprint features, the electronic device can use the real-time obtained speaker separation results to annotate the corresponding transcribed text content and display the annotated transcribed text content in real time. Specifically, after performing clustering analysis on voiceprint features within a set time period to obtain speaker separation results, the speaker annotation method may further include: annotating the transcribed text content corresponding to the set time period according to the speaker separation results to obtain speaker-annotated text; and displaying the speaker-annotated text in real time.

[0084] For example, users can also modify the speaker separation results. Specifically, the speaker annotation method may further include: receiving a modification instruction; and modifying the speaker annotation text in response to the modification instruction.

[0085] For example, combining Figure 2 During speaker separation, after obtaining speaker separation result C1, C1 can be used to annotate the transcribed text content corresponding to L1 with speaker information, and the annotated transcribed text content is displayed. Then, after obtaining C2, C2 is used to annotate the transcribed text content corresponding to L2 with speaker information, and the annotated transcribed text content is displayed. This process continues, allowing for real-time display of speaker-annotated text for each set time period. During this process, the speaker identification information in the displayed speaker-annotated text can be modified. In the document processing stage after recording, the electronic device can use C7 to re-detect the identification information of all speaker-annotated text. If there is a conflict between the speaker separation result in C7 and the manually annotated speaker information, the manually modified identification information displayed during the real-time process takes precedence.

[0086] Specifically, during the document processing stage after recording, the electronic device can traverse the entire transcribed text. During the traversal, if it detects that the transcribed text content has been manually annotated with the speaker's identification information, then the speaker's identification information in the transcribed text content is marked as manually annotated. If no manual annotation is detected, and the time period corresponding to the transcribed text content in C7 contains only one speaker's identification information, then the transcribed text content is directly annotated with that speaker's identification information according to C7. If no manual annotation is detected, and the time period corresponding to the transcribed text content in C7 contains at least two speaker's identification information, then the transcribed text content can be split based on the identification information of at least two speakers, and the corresponding speaker's identification information can be annotated for each split result. For example, if the transcribed text from 1 to 7 seconds is "How's the weather today? The weather is nice today, perfect for outings.", and the speaker separation results indicate that 1 to 3 seconds is speaker A and 3 to 7 seconds is speaker B, then the transcribed text can be split into "How's the weather today?" and "The weather is nice today, perfect for outings.", with speaker A's information marked for "How's the weather today?" and speaker B's information marked for "The weather is nice today, perfect for outings."

[0087] Based on the speaker annotation methods of the above embodiments, after recording stops, speech transcription is completed, and the electronic device can display the transcribed text in the speech transcription interface. Simultaneously, speaker identification information is annotated on the transcribed text, and the user can also edit the speaker identification information on the displayed transcribed text. Specifically, the speaker annotation method may further include: receiving an identification information modification instruction; and, in response to the identification information modification instruction, modifying the speaker identification information integrated with the text content annotation.

[0088] For example, Figure 4 An exemplary diagram of a speech-to-text interface is shown below. Figure 4 As shown, the speech-to-text interface may include a speaker annotation control 41. When this control 41 is triggered, the electronic device can separate and annotate the speakers in the transcribed text. After completion, the corresponding speaker's identification information is displayed in the content area corresponding to the transcribed text. For example, different speakers' identification information can be displayed in different colors. For example, the speech-to-text interface may also display a speech playback progress bar 42. The speech playback progress bar 42 can use different colored areas to represent the speech content of different speakers. For instance, area E1 in the speech playback progress bar 42 corresponds to speaker A's text content 1, area E2 corresponds to speaker B's text content 2, and area E3 corresponds to speaker C's text content 3, and the display colors of areas E1, E2, and E3 are different.

[0089] The electronic device can receive operation commands directed to the speech-to-text interface. For example, before saving the transcribed text with speaker identification information displayed on the speech-to-text interface, the user can organize the transcribed text to be saved, such as modifying the text content or the speaker identification information. For instance, the speaker identification information displayed on the speech-to-text interface can be in the form of an editable control, which the user can click to modify. For example, if a user finds that the "Speaker A" label displayed in the current window is incorrect while organizing a document, they can click on "Speaker A." After receiving the click operation, the electronic device controls the state of "Speaker A" to be editable, for example, by activating it as an input box. Then, it receives input from the input box, confirms the modified identification information, and then changes "Speaker A" to the input identification information, for example, to "Speaker D."

[0090] For example, the speech-to-text interface can provide functional controls for batch modification of speaker identification information, such as... Figure 4The electronic device responds to a trigger operation on the Batch Speaker Modification Control 43 in the interface by displaying the batch modification control in a preset area of ​​the speech-to-text interface. The batch modification control may include a target identifier information input control 431 and a target identifier information input control 432. The electronic device responds to an input operation on the target identifier information input control 431 and the target identifier information input control 432 by determining the identifier information to be modified and the target identifier information after modification. After determination, all identifier information in the transcribed text that is the same as the identifier information to be modified is modified to the target identifier information, such as changing "Speaker B" to "Leader".

[0091] After clicking save, the electronic device can save the modified transcribed text marked with speaker identification information.

[0092] In real-time conferencing scenarios, recordings may be paused or interrupted, resulting in the loss of speaker information. For example, in a meeting with three speakers, A, B, and C, if they speak in sequence, the speaker identification engine returns speaker identifiers as A corresponding to 1, B to 2, and C to 3. If the meeting is interrupted, the previous speaker information is lost, and the system starts again from 1. If the order before the interruption is followed, and B speaks first, B would be identified as 1, which can cause confusion when reading and transcribing the text.

[0093] Therefore, in one exemplary embodiment of the present invention, the speaker separation engine can, after obtaining a speaker separation result each time, use the speaker separation result to cluster and update the first historical speaker separation result. After the clustering update, the updated first historical speaker separation result is saved, which includes the speaker's identification information and voiceprint features. In this way, if the meeting recording is interrupted, such as during a break or due to equipment failure, when the recording connection is re-established, speaker separation processing can continue based on the previously saved first historical speaker separation result, ensuring that speaker information is not lost.

[0094] For example, electronic devices can also store current meeting information and corresponding speaker separation engine session information. If the meeting is interrupted, the service side can carry the speaker separation engine session information before the disconnection when re-establishing the connection. At this time, the speaker separation engine can continue to perform cluster analysis of speaker separation based on the previously saved first historical speaker separation results. This can avoid the problem of inconsistent speaker information in the same meeting.

[0095] The speaker annotation device provided by the present invention will be described below. The speaker annotation device described below can be referred to in correspondence with the speaker annotation method described above.

[0096] Figure 5 An exemplary schematic diagram of the speaker annotation device provided in an embodiment of the present invention is shown, with reference to... Figure 5 As shown, the speaker annotation device 500 may include: a speech acquisition module 510 for acquiring speech data; a transcription module 520 for transcribing the speech data to obtain transcribed text; a processing module 530 for segmenting the speech data based on speech boundary detection to obtain at least one speech segment, and extracting the voiceprint features of each speech segment; an analysis module 540 for performing cluster analysis on the voiceprint features within each set time period to obtain speaker separation results; and an annotation module 550 for updating the first historical speaker separation results using the speaker separation results, and annotating the transcribed text with speaker information based on the updated first historical speaker separation results; wherein the first historical speaker separation results are determined based on the speaker separation results corresponding to at least one set time period prior to the set time period.

[0097] In one example embodiment, the speaker annotation device 500 may further include: a separation result acquisition module, configured to acquire a first speaker separation result corresponding to a first set time period, wherein the first set time period is the previous set time period; and an update module, configured to use the first speaker separation result to perform clustering update on the second historical speaker separation result to obtain the first historical speaker separation result; wherein the second historical speaker separation result is determined based on the speaker separation result corresponding to at least one set time period before the first set time period.

[0098] In one example embodiment, the annotation module 550 may include: an integration unit, configured to integrate the transcribed text content corresponding to consecutive speech segments belonging to the same speaker based on the finally obtained updated first historical speaker separation result when a speech end instruction is detected, to obtain integrated text content; and an annotation unit, configured to annotate the integrated text content with the speaker's identification information.

[0099] In one example embodiment, the annotation unit may include: an acquisition subunit for acquiring initial identification information for integrated text content annotation; a detection subunit for detecting whether the initial identification information is manually annotated identification information during the process of annotating the integrated text content based on the finally obtained updated first historical speaker separation result; and an annotation subunit for annotating the integrated text content with manually annotated identification information if the initial identification information for integrated text content annotation is manually annotated identification information.

[0100] In one example embodiment, the speaker annotation device 500 may further include a first display module; the annotation module 550 may also be used to perform cluster analysis on the voiceprint features within a set time period to obtain speaker separation results, and then perform speaker annotation on the transcribed text content corresponding to the set time period based on the speaker separation results to obtain speaker-annotated text; the first display module is used to display the speaker-annotated text in real time.

[0101] In one example embodiment, the speaker annotation device 500 may further include: a receiving module for receiving a modification instruction; and a modification module for modifying the speaker annotation text in response to the modification instruction.

[0102] In one example embodiment, the updated first historical speaker separation result includes speaker identification information; the speaker annotation device 500 may further include: a second display module, used to display the transcribed text annotated with speakers based on the speaker identification information after the transcribed text is annotated with speakers based on the updated first historical speaker separation result; wherein the display parameters of the identification information of different speakers are different, and / or the identification information of different speakers is represented by the parameter information of the voice playback progress bar.

[0103] In one example embodiment, the speaker labeling device 500 may further include: a storage module, used to store the updated first historical speaker separation result after clustering and updating the first historical speaker separation result using the speaker separation result; wherein the updated first historical speaker separation result includes the speaker's identification information and voiceprint features.

[0104] Figure 6 An example is a schematic diagram of the structure of an electronic device, such as... Figure 6As shown, the electronic device may include a processor 610, a communication interface 620, a memory 630, and a communication bus 640. The processor 610, communication interface 620, and memory 630 can communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute the speaker annotation methods provided in the above-described method embodiments. This method may include, for example, acquiring speech data and transcribing the speech data to obtain transcribed text; segmenting the speech data to obtain at least one speech segment and extracting the voiceprint features of each speech segment; performing cluster analysis on the voiceprint features within each set time period to obtain speaker separation results; updating the first historical speaker separation results using the speaker separation results; and annotating the transcribed text with speakers based on the updated first historical speaker separation results; wherein the first historical speaker separation results are determined based on speaker separation results corresponding to at least one set time period prior to the set time period.

[0105] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0106] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the speaker annotation method provided in the above-described method embodiments. This method may include, for example, acquiring speech data and transcribing the speech data to obtain transcribed text; segmenting the speech data to obtain at least one speech segment and extracting the voiceprint features of each speech segment; performing cluster analysis on the voiceprint features within each set time period to obtain speaker separation results; updating the first historical speaker separation results using the speaker separation results; and annotating the transcribed text with speakers based on the updated first historical speaker separation results; wherein the first historical speaker separation results are determined based on the speaker separation results corresponding to at least one set time period prior to the set time period.

[0107] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the speaker annotation method provided in the above-described method embodiments. This method may include, for example, acquiring speech data and transcribing the speech data to obtain transcribed text; segmenting the speech data to obtain at least one speech segment and extracting the voiceprint features of each speech segment; performing cluster analysis on the voiceprint features within each set time period to obtain speaker separation results; updating the first historical speaker separation results using the speaker separation results; and performing speaker annotation on the transcribed text based on the updated first historical speaker separation results; wherein the first historical speaker separation results are determined based on the speaker separation results corresponding to at least one set time period prior to the set time period.

[0108] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0109] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A speaker annotation method, characterized in that, include: Acquire speech data and transcribe the speech data to obtain transcribed text; The speech data is segmented to obtain at least one speech segment, and the voiceprint features of each speech segment are extracted. For each set time period, cluster analysis is performed on the voiceprint features within the set time period to obtain speaker separation results; The first historical speaker separation result is clustered and updated using the speaker separation result, and the transcribed text is labeled with speakers based on the updated first historical speaker separation result; wherein, the first historical speaker separation result is determined based on the speaker separation result corresponding to at least one set time period before the set time period.

2. The speaker annotation method according to claim 1, characterized in that, Before using the speaker separation results to cluster and update the first historical speaker separation results, the method further includes: Obtain the first speaker separation result corresponding to the first set time period, wherein the first set time period is the previous set time period of the set time period; The first speaker separation result is used to cluster and update the second historical speaker separation result to obtain the first historical speaker separation result; wherein, the second historical speaker separation result is determined based on the speaker separation result corresponding to at least one set time period before the first set time period.

3. The speaker annotation method according to claim 1, characterized in that, The speaker annotation of the transcribed text based on the updated first historical speaker separation results includes: Upon detecting a speech end command, based on the final updated first historical speaker separation result, the transcribed text content corresponding to consecutive speech segments belonging to the same speaker is integrated to obtain integrated text content. The integrated text content is labeled with the speaker's identification information.

4. The speaker annotation method according to claim 3, characterized in that, The annotation of the speaker's identification information on the integrated text content includes: Obtain the initial identification information of the integrated text content annotation; During the process of annotating the integrated text content based on the finally obtained updated first historical speaker separation results, it is detected whether the initial identification information is manually annotated identification information; If the initial identifier information of the integrated text content is manually labeled, the integrated text content will be labeled with the manually labeled identifier information.

5. The speaker annotation method according to any one of claims 1 to 4, characterized in that, After performing cluster analysis on the voiceprint features within the set time period to obtain speaker separation results, the method further includes: Based on the speaker separation results, speaker annotation is performed on the transcribed text content corresponding to the set time period to obtain speaker-annotated text. The speaker's annotated text is displayed in real time.

6. The speaker annotation method according to claim 5, characterized in that, The method further includes: Receive modification instructions; In response to the modification instruction, the speaker's tagged text is modified.

7. The speaker annotation method according to any one of claims 1 to 4, characterized in that, The updated first historical speaker separation result includes the speaker's identification information; After performing speaker annotation on the transcribed text based on the updated first historical speaker separation results, the method further includes: Based on the speaker's identification information, the transcribed text after the speaker's annotation is displayed; wherein, the display parameters of the identification information of different speakers are different, and / or, the identification information of different speakers is represented by the parameter information of the voice playback progress bar.

8. The speaker annotation method according to any one of claims 1 to 4, characterized in that, After updating the first historical speaker separation results by clustering using the speaker separation results, the method further includes: Save the updated first historical speaker separation result, which includes the speaker's identification information and voiceprint features.

9. A speaker annotation device, characterized in that, include: The voice acquisition module is used to acquire voice data; The transcription module is used to transcribe the speech data to obtain transcribed text; The processing module is used to segment the speech data based on speech boundary detection to obtain at least one speech segment, and extract the voiceprint features of each speech segment; The analysis module is used to perform cluster analysis on the voiceprint features within each set time period to obtain speaker separation results. The annotation module is used to cluster and update the first historical speaker separation result using the speaker separation result, and to annotate the transcribed text with speakers based on the updated first historical speaker separation result; wherein, the first historical speaker separation result is determined based on the speaker separation result corresponding to at least one set time period before the set time period.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the speaker annotation method as described in any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the speaker annotation method as described in any one of claims 1 to 8.

Images