Voice processing method and device

Abstract

The invention discloses a voice processing method and device, and relates to the technical field of voice processing. The specific implementation mode of the method comprises the following steps: separating voice data to obtain non-overlapped voice and overlapped voice; recognizing a plurality of main voiceprints from the non-overlapping voices, and extracting voiceprint features from the overlapping voices; determining a target main voiceprint corresponding to the overlapped voices from the plurality of main voiceprints according to the voiceprint features; and segmenting the non-overlapping voice according to the plurality of main voiceprints to obtain a plurality of first voice segments, separating the overlapping voice according to the target main voiceprint to obtain a plurality of second voice segments, and splicing the plurality of first voice segments and the plurality of second voice segments to obtain a plurality of target voice segments. According to the embodiment of the invention, the problem of low accuracy caused by voiceprint recognition of overlapped voices and non-overlapped voices is solved, the accuracy of voiceprint recognition is improved, and the technical effect of accurately determining speaker information is realized.

Description

Technical Field

[0001] The present invention relates to the field of speech processing technology, and in particular to a speech processing method and device. Background Art

[0002] Currently, in a multi-person conversation scenario, the collected audio information may contain overlapping speech from multiple people speaking at the same time. For overlapping speech, the speaker log service usually uses the following two methods: 1. The speaker closest in time to the overlapping speech segment is determined as the speaker corresponding to the overlapping speech; 2. The collected audio information is directly input into the neural network model, which directly outputs the speaker information through clustering, without considering the overlapping speech separately.

[0003] However, the inventors have found in practice that the above two methods have the following problems: the first method simply determines the speaker information based on the time distance, and has low accuracy; the second method, in order to improve the accuracy of overlapping speech recognition, requires a large amount of overlapping speech training data to train the model, and the overlapping speech training data will also be used for the voiceprint clustering of the model, which will affect the effect of the model on non-overlapping speech clustering. Summary of the invention

[0004] In view of this, the embodiment of the present invention provides a speech processing method and device, which obtains non-overlapping speech and overlapping speech by separating speech data; identifies multiple main voiceprints from non-overlapping speech, and extracts voiceprint features from overlapping speech; determines the target main voiceprint corresponding to the overlapping speech from the multiple main voiceprints according to the voiceprint features; segments the non-overlapping speech according to the multiple main voiceprints to obtain multiple first speech segments, separates the overlapping speech according to the target main voiceprint to obtain multiple second speech segments, and splices the multiple first speech segments and the multiple second speech segments to obtain multiple target speech segments. Thus, by separating the overlapping speech and the non-overlapping speech respectively and performing voiceprint recognition on them respectively, and segmenting or separating the speech data according to the multiple main voiceprints corresponding to the non-overlapping speech and the target main voiceprint corresponding to the overlapping speech, speech segments of multiple speakers speaking in different time periods are obtained, thereby solving the problem of low accuracy caused by performing voiceprint recognition on the overlapping speech and the non-overlapping speech as a whole, improving the accuracy of voiceprint recognition, and achieving the technical effect of accurately determining the speaker information.

[0005] To achieve the above objective, according to one aspect of an embodiment of the present invention, a speech processing method is provided.

[0006] A speech processing method according to an embodiment of the present invention comprises: separating speech data to obtain non-overlapping speech and overlapping speech; identifying multiple main voiceprints from the non-overlapping speech and extracting voiceprint features from the overlapping speech; determining a target main voiceprint corresponding to the overlapping speech from the multiple main voiceprints according to the voiceprint features; segmenting the non-overlapping speech according to the multiple main voiceprints to obtain multiple first speech segments, separating the overlapping speech according to the target main voiceprint to obtain multiple second speech segments, and splicing the multiple first speech segments and the multiple second speech segments to obtain multiple target speech segments.

[0007] Optionally, determining a target main voiceprint corresponding to the overlapping speech from a plurality of the main voiceprints based on the voiceprint feature includes: respectively calculating a first similarity between the voiceprint feature and the plurality of the main voiceprints; for each first similarity corresponding to the main voiceprint, in response to the first similarity being greater than a first threshold, taking the main voiceprint as the target main voiceprint.

[0008] Optionally, the non-overlapping speech is segmented according to the multiple main voiceprints to obtain multiple first speech segments, including: starting from the time starting point of the non-overlapping speech, detecting the main voiceprint corresponding to the non-overlapping speech; in response to detecting that the main voiceprint corresponding to the non-overlapping speech at a target time point is different from that at a previous time point, segmenting the non-overlapping speech at the target time point to obtain multiple first speech segments.

[0009] Optionally, the number of the target main voiceprints is at least two; and the separating the overlapping speech according to the target main voiceprint to obtain multiple second speech segments includes: extracting the speech data corresponding to each target main voiceprint from the overlapping speech; and taking the temporally continuous speech data corresponding to each target main voiceprint as a second speech segment.

[0010] Optionally, the splicing of the multiple first voice segments and the multiple second voice segments to obtain multiple target voice segments includes: determining the start time and the end time corresponding to each second voice segment; determining, from the multiple first voice segments, a first voice segment that is adjacent to the second voice segment in time according to the start time and the end time; in response to the adjacent first voice segment and the second voice segment corresponding to the same main voiceprint, splicing the first voice segment with the second voice segment to obtain a spliced ​​voice segment; and using the other first voice segments excluding the adjacent first voice segments and the spliced ​​voice segment as the multiple target voice segments.

[0011] Optionally, identifying multiple main voiceprints from the non-overlapping speech includes: extracting voiceprint features of the non-overlapping speech; clustering the voiceprint features to obtain multiple voiceprint feature sets; and generating a main voiceprint representing each voiceprint feature set based on each voiceprint feature set.

[0012] Optionally, the method provided by the present invention also includes: dividing the non-overlapping speech into multiple target speech segments according to the multiple main voiceprints; calculating the second similarity between the voiceprint features corresponding to each two adjacent first speech segments; in response to the second similarity being greater than a second threshold, merging two adjacent first speech segments; and determining the corresponding main voiceprint for the merged first speech segment.

[0013] Optionally, the calculating the second similarity between the voiceprint features corresponding to each two adjacent first speech segments includes: determining a connection area between two adjacent first speech segments, the connection area including a first part and a second part; calculating the similarity between the voiceprint features of the first part and the voiceprint features of the second part, and determining the similarity as the second similarity.

[0014] Optionally, the method provided by the present invention also includes: converting the voice data into text data and its corresponding first timestamp; allocating corresponding text data to the multiple target voice segments according to the first timestamp and the second timestamps corresponding to the multiple target voice segments; generating a speaker log according to the second timestamp, the text data corresponding to the multiple target voice segments and the speaker identifiers corresponding to the multiple target voice segments; the speaker identifier is determined based on the primary voiceprints corresponding to the multiple target voice segments respectively.

[0015] Optionally, assigning corresponding text data to the multiple target voice segments according to the first timestamp and the second timestamps corresponding to the multiple target voice segments includes: determining the second timestamp corresponding to each voice segment; and determining the text data corresponding to the first timestamp that is the same as the second timestamp as the text data corresponding to the voice segment.

[0016] Optionally, the method provided by the present invention also includes: for each independent sentence in the text data: in response to the independent sentence corresponding to at least two adjacent voice segments, modifying the independent sentence according to the text meaning of the independent sentence and the boundary of the adjacent voice segments.

[0017] To achieve the above objective, according to another aspect of an embodiment of the present invention, a speech processing device is provided.

[0018] A speech processing device according to an embodiment of the present invention comprises: a speech separation module, which is used to separate speech data to obtain non-overlapping speech and overlapping speech; a voiceprint extraction module, which is used to identify multiple main voiceprints from the non-overlapping speech and extract voiceprint features from the overlapping speech; a speech analysis module, which is used to determine, according to the voiceprint features, a target main voiceprint corresponding to the overlapping speech from the multiple main voiceprints; a speech segmentation module, which is used to segment the non-overlapping speech according to the multiple main voiceprints to obtain multiple first speech segments, separate the overlapping speech according to the target main voiceprint to obtain multiple second speech segments, and splice the multiple first speech segments and the multiple second speech segments to obtain multiple target speech segments.

[0019] To achieve the above objective, according to another aspect of an embodiment of the present invention, an electronic device for speech processing is provided.

[0020] An electronic device for speech processing according to an embodiment of the present invention comprises: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement a speech processing method according to an embodiment of the present invention.

[0021] To achieve the above objective, according to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided.

[0022] A computer-readable storage medium according to an embodiment of the present invention stores a computer program, and when the program is executed by a processor, a speech processing method according to an embodiment of the present invention is implemented.

[0023] One embodiment of the above invention has the following advantages or beneficial effects: by separating the speech data, non-overlapping speech and overlapping speech are obtained; multiple main voiceprints are identified from the non-overlapping speech, and voiceprint features are extracted from the overlapping speech; according to the voiceprint features, the target main voiceprint corresponding to the overlapping speech is determined from the multiple main voiceprints; the non-overlapping speech is segmented according to the multiple main voiceprints to obtain multiple first speech segments, the overlapping speech is separated according to the target main voiceprint to obtain multiple second speech segments, and the multiple first speech segments and the multiple second speech segments are spliced ​​to obtain multiple target speech segments. Thus, by separating the overlapping speech and the non-overlapping speech respectively and performing voiceprint recognition on them respectively, and according to the multiple main voiceprints corresponding to the non-overlapping speech and the target main voiceprint corresponding to the overlapping speech, the speech data is segmented or separated to obtain speech segments of multiple speakers speaking in different time periods, thereby solving the problem of low accuracy caused by voiceprint recognition of the overlapping speech and the non-overlapping speech as a whole, improving the accuracy of voiceprint recognition, and achieving the technical effect of accurately determining the speaker information.

[0024] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0026] Figure 1 is a flow chart of a speech processing method according to an embodiment of the present invention;

[0027] Figure 2 is a schematic diagram of performing voiceprint recognition on non-overlapping speech according to an embodiment of the present invention;

[0028] Figure 3 is a schematic diagram of a result after error correction of a voiceprint recognition result of non-overlapping speech according to an embodiment of the present invention;

[0029] Figure 4 is a schematic diagram of a method for correcting errors in voiceprint recognition results of non-overlapping speech according to an embodiment of the present invention;

[0030] Figure 5 is a schematic diagram of voiceprint recognition of overlapping speech according to an embodiment of the present invention;

[0031] Figure 6 is a schematic diagram of segmenting voice data according to an embodiment of the present invention;

[0032] Figure 7a is a schematic diagram of separating overlapping voice data according to an embodiment of the present invention;

[0033] Figure 7b is a schematic diagram of splicing a first speech segment and a second speech segment according to an embodiment of the present invention;

[0034] Figure 8 is a schematic diagram of performing error correction on text data according to an embodiment of the present invention;

[0035] Fig. 9 2 is a schematic diagram of a structural framework of a speaker log service system according to an embodiment of the present invention;

[0036] Fig.10 is a flow chart of another speech processing method according to an embodiment of the present invention;

[0037] Fig.11 is a schematic diagram of main modules of a speech processing device according to an embodiment of the present invention;

[0038] Fig.12is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0039] Fig.13 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0041] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0042] It should be pointed out that the embodiments of the present invention and the technical features therein may be combined with each other without conflict.

[0043] Figure 1 is a schematic diagram of main steps of a speech processing method according to an embodiment of the present invention.

[0044] like Figure 1 As shown, the speech processing method of the embodiment of the present invention mainly includes steps S101-S104:

[0045] Step S101: Separate the voice data to obtain non-overlapping voice and overlapping voice. The voice processing method of this embodiment is applicable to formal or informal multi-person conversation scenarios. Voice data refers to audio data collected during a multi-person conversation. During a multi-person conversation, the voice data generated by a single person speaking is non-overlapping voice, and the voice data generated by multiple people speaking at the same time is overlapping voice, where multiple people refer to two or more people. Separate the collected voice data, separate the non-overlapping voice of a single person speaking from the overlapping voice of multiple people speaking at the same time, and process the non-overlapping voice and overlapping voice separately. It should be noted that in actual scenarios, especially in formal occasions, it is rare for many people to speak at the same time. Therefore, in the embodiment of the present invention, the overlapping voice data generated by two people speaking at the same time is taken as an example to explain the voice processing method of this embodiment in detail.

[0046] Step S102: Identify multiple main voiceprints from the non-overlapping speech, and extract voiceprint features from the overlapping speech. Extract the voiceprint features of each small segment of the non-overlapping speech, cluster the extracted voiceprint features, and obtain multiple voiceprint feature sets, wherein each voiceprint feature set corresponds to a speaker. According to the multiple voiceprint features included in each voiceprint feature set, calculate the main voiceprint that characterizes the voice characteristics of the speaker. Therefore, according to the multiple voiceprint feature sets, multiple main voiceprints that characterize the voice characteristics of multiple speakers are calculated respectively.

[0047] In order to identify multiple speakers included in non-overlapping speech, in an optional embodiment of the present invention, identifying multiple main voiceprints from the non-overlapping speech includes: extracting voiceprint features of the non-overlapping speech; clustering the voiceprint features to obtain multiple voiceprint feature sets; and generating a main voiceprint representing each voiceprint feature set based on each voiceprint feature set. The voiceprint features extracted from the non-overlapping speech are clustered to obtain multiple voiceprint feature sets 1, 2, 3, 4, and 5. Based on the voiceprint features included in the voiceprint feature sets 1, 2, 3, 4, and 5, multiple main voiceprints A, B, C, D, and E are calculated. The multiple main voiceprints A, B, C, D, and E can represent the voice features of multiple speakers, respectively. Therefore, based on the multiple main voiceprints A, B, C, D, and E, it can be determined that the speakers corresponding to the non-overlapping speech are Xiao Li, Xiao Wang, Xiao Zhao, Xiao Sun, and Xiao Liu, respectively. That is, the voice segments generated by the speeches of Xiao Li, Xiao Wang, Xiao Zhao, Xiao Sun, and Xiao Liu are determined from the non-overlapping speech, such as Figure 2 As shown, the voice clip includes the speech content and the speech time.

[0048] During a multi-person conversation, the sound collection device may have unstable performance, resulting in differences in the sound features collected by the device for the speech of the same speaker, such as the volume of the voice. In addition, even for the same speaker, the sound features are different when speaking. For example, the intensity of the voice may fluctuate due to emotional excitement. In order to prevent the voice of one speaker from being recognized as the voice of two speakers and further improve the accuracy of voice recognition, after the initial recognition of the voice data, further error correction is required. For example, in the preliminary recognition results, there is a situation where two voice segments corresponding to the same main voiceprint are sandwiched between a voice segment corresponding to another main voiceprint (such as B, C, B). At this time, it is necessary to further check it to determine whether the main voiceprint C mixed in the main voiceprint B is a recognition error, that is, to determine whether the voice in the middle of Xiao Wang’s speech is recognized as Xiao Zhao’s voice.

[0049] In view of the above situation, in an optional embodiment of the present invention, the method further includes: dividing the non-overlapping speech into a plurality of first speech segments according to the plurality of the main voiceprints; calculating the second similarity between the voiceprint features corresponding to each two adjacent first speech segments; merging the two adjacent first speech segments in response to the second similarity being greater than the second threshold; and determining the corresponding main voiceprint for the merged first speech segment. The application scenario of this embodiment is that among the obtained plurality of first speech segments, there are two speech segments corresponding to the same main voiceprint, and a speech segment corresponding to another main voiceprint is sandwiched between them, such as B, C, and B. In order to prevent voiceprint recognition errors and identify the main voiceprint B as C, it is necessary to further determine the similarity between the adjacent B and C, or the similarity between C and B, and determine whether the occurrence of speech segments B, C, and B is due to recognition errors according to the similarity. If the similarity between B and C is higher than the second threshold, B and C are merged into B, that is, the three first speech segments B, C, and B are merged into one speech segment B, that is, the main voiceprint corresponding to the merged new speech segment is B.

[0050] Specifically, according to the multiple main voiceprints A, B, C, D and E obtained by clustering the voiceprint features of non-overlapping speech, the non-overlapping speech collected during the multi-person conversation is segmented to obtain multiple first speech segments. Since the multiple first speech segments are segmented based on the main voiceprint, the multiple first speech segments can be represented by the main voiceprint identifiers A, B, C, B, D, E, such as Figure 3 As shown. For adjacent first voice segments, such as B and C, respectively extract their corresponding voiceprint features, and calculate the second similarity between the voiceprint features of the two (B and C). Compare the calculated second similarity with the second threshold value. If the second similarity is greater than the second threshold value, it is determined that the two adjacent first voice segments are voice segments generated by the same speaker, so the two adjacent first voice segments are merged, for example, the two voice segments corresponding to B and C are merged into the voice segment corresponding to B, or merged into the voice segment corresponding to C.

[0051] As to whether to merge the two voice segments B and C into B or into C, it can be determined according to the actual situation. For example, in this embodiment, the speech contents of the three adjacent first voice segments B, C and B can be combined to further determine whether to merge the two voice segments into B. Figure 3 As shown, further checking of the preliminary speech recognition results is achieved, ensuring a high accuracy of speech recognition.

[0052] Furthermore, in actual situations, if a speaker speaks for a long time, his voice features may change to a certain extent during the speech. In order to prevent the change of voice features from affecting voice recognition, when calculating the similarity of adjacent first voice segments of a longer duration, only a part of the voice data of two adjacent first voice segments can be selected to calculate their similarity. For example, the last part of the previous voice segment and the beginning part of the next voice segment are selected to calculate the similarity of the voiceprint features of these two parts, so as to accurately judge the similarity between the two adjacent first voice segments. In an optional embodiment of the present invention, the calculation of the second similarity between the voiceprint features corresponding to each two adjacent first voice segments includes: determining the connection area of ​​two adjacent first voice segments, the connection area including the first part and the second part; calculating the similarity between the voiceprint features of the first part and the voiceprint features of the second part, and determining the similarity as the second similarity. Figure 4 As shown, the connection area is composed of the last part B1 of the voice segment B and the beginning part C1 of the voice segment C, the voiceprint features of the B1 segment and the C1 segment are extracted respectively, and the similarity between the voiceprint features of the two parts is calculated, and it is determined whether the similarity is greater than a second threshold. When the similarity is greater than the second threshold, it is determined that the voice segments B1 and C1 are voice data generated by the same speaker, and then the voice segments B and C are determined to be voice data generated by the same speaker, and the voice segments B and C are merged into one voice segment B or C.

[0053] After determining the main voiceprint corresponding to the non-overlapping speech and the speaker corresponding to the main voiceprint, the overlapping speech data is processed, that is, the voiceprint features of the overlapping speech data are extracted, so as to compare the voiceprint features included in the overlapping speech with the multiple main voiceprints identified from the non-overlapping speech respectively. According to the comparison result, it is determined which two of the multiple main voiceprints are superimposed on the overlapping speech, that is, it is determined which two speakers spoke simultaneously to form the overlapping speech.

[0054] Step S103: According to the voiceprint features, determine the target main voiceprint corresponding to the overlapping speech from the multiple main voiceprints. Specifically, respectively calculate the similarities between the voiceprint features included in the overlapping speech and the multiple main voiceprints to obtain multiple similarities, and determine the main voiceprint corresponding to the similarity greater than a threshold among the multiple similarities as the target main voiceprint included in the overlapping speech.

[0055] In an optional embodiment of the present invention, the method of determining the target main voiceprint corresponding to the overlapping speech from the plurality of main voiceprints according to the voiceprint feature comprises: respectively calculating the first similarities between the voiceprint feature and the plurality of main voiceprints; for each first similarity corresponding to the main voiceprint, in response to the first similarity being greater than a first threshold, taking the main voiceprint as the target main voiceprint. Specifically, the similarity between the voiceprint feature extracted from the overlapping speech and each main voiceprint is calculated to obtain a plurality of similarities, such as calculating the similarity between the voiceprint feature and the main voiceprint A to obtain S1, calculating the similarity between the voiceprint feature and the main voiceprint B to obtain S2, calculating the similarity between the voiceprint feature and the main voiceprint C to obtain S3, calculating the similarity between the voiceprint feature and the main voiceprint D to obtain S4, and calculating the similarity between the voiceprint feature and the main voiceprint E to obtain S5. The plurality of first similarities S1, S2, S3, S4 and S5 are respectively compared with the first threshold, and the main voiceprint included in the overlapping speech, i.e., the target main voiceprint, is determined according to the comparison result. Among the multiple first similarities S1, S2, S3, S4 and S5, if S1 and S5 are greater than the first threshold, it can be determined that the voiceprint features in the overlapping speech have a high similarity with the main voiceprint A and the main voiceprint E. Therefore, it is determined that the overlapping speech includes the main voiceprint A and the main voiceprint E, that is, the overlapping speech is the speech data generated by Xiao Li (main voiceprint A) and Xiao Wang (main voiceprint E) speaking at the same time. Figure 5 .

[0056] There may be other situations in actual scenarios, such as a person named Xiao Yang who did not speak during the single-person speaking stage of a multi-person conversation, but only interrupted someone else (such as Xiao Zhao) to speak, thus forming a segment of overlapping voice data. For the overlapping voice data, by extracting the voiceprint features and calculating its similarity with multiple main voiceprints in non-overlapping voices, in this case, only one target main voiceprint (such as C) included in the overlapping voice can be determined, and the target main voiceprint corresponding to Xiao Yang in the overlapping voice cannot be determined. For this case, the target main voiceprint C can be separated from the overlapping voice, and the voiceprint features of the remaining voice data in the overlapping voice are extracted, and the main voiceprint F representing the voice features of Xiao Yang's speech is generated based on the extracted voiceprint features. However, this situation has a low probability of occurring in actual multi-person conversation scenarios, especially in formal occasions, and the importance of the overlapping voice generated in this situation is low, which is not the focus of the embodiment of the present invention, so the embodiment of the present invention does not discuss it in detail.

[0057] Step S104: segmenting the non-overlapping speech according to the multiple main voiceprints to obtain multiple first speech segments, separating the overlapping speech according to the target main voiceprint to obtain multiple second speech segments, and splicing the multiple first speech segments and the multiple second speech segments to obtain multiple target speech segments.

[0058] After determining the multiple main voiceprints A, B, C, D and E included in the non-overlapping speech and the target main voiceprints A and E corresponding to the overlapping speech, the non-overlapping speech is segmented according to the multiple main voiceprints A, B, C, D and E on the time axis of the collected speech data to obtain multiple first speech segments, such as Figure 6 The C, A, D, B and E fragments shown. In addition, Figure 6 The segments corresponding to "A and E" in are overlapping speech segments.

[0059] In an optional embodiment of the present invention, the segmenting of the non-overlapping speech according to the plurality of primary voiceprints to obtain a plurality of first speech segments includes: starting from the time starting point of the non-overlapping speech, detecting the primary voiceprint corresponding to the non-overlapping speech; in response to detecting that the primary voiceprint corresponding to the non-overlapping speech at a target time point is different from that at a previous time point, segmenting the non-overlapping speech at the target time point to obtain a plurality of first speech segments. Figure 6 For example, a continuous non-overlapping speech data is segmented. At the start time of this speech data, the main voiceprint C is detected at 0 minutes, and voiceprint A is detected at 2 minutes. The speech data is segmented once at 2 minutes, and voiceprint A lasts until the end of the speech, that is, the first speech segments C and A are obtained. For another non-overlapping speech, the same method is used for segmentation to obtain the first speech segments D, B and E.

[0060] Next, the overlapping speech is separated according to the target main voiceprints A and E. In an optional embodiment of the present invention, the number of the target main voiceprints is at least two; the overlapping speech is separated according to the target main voiceprint to obtain multiple second speech segments, including: extracting the speech data corresponding to each target main voiceprint from the overlapping speech; taking the temporally continuous speech data corresponding to each target main voiceprint as a second speech segment. In an actual multi-person conversation scenario, there may be a situation where one person (such as Xiao Li, whose corresponding main voiceprint A) has not finished speaking, and another person (such as Xiao Liu, whose corresponding main voiceprint E) starts to interrupt, which will generate a segment of speech data as shown in Figure 7, wherein the first part is the non-overlapping speech data of Xiao Sun speaking alone, and the second part is the overlapping speech data of Xiao Sun and Xiao Liu speaking at the same time. In this case, it is necessary to separate the overlapping speech data according to the main voiceprints A and E corresponding to Xiao Sun and Xiao Liu respectively, and separate the speech data corresponding to the main voiceprint A and the speech data corresponding to the main voiceprint E, that is, the speech content of Xiao Sun and the speech content of Xiao Liu. Figure 6 The overlapping speech segments corresponding to "A and E" in are separated to obtain Figure 7a The dashed parts shown in the figure represent speech segments A and E, ie, the second speech segment.

[0061] In an optional embodiment of the present invention, the multiple first voice segments and the multiple second voice segments are spliced ​​to obtain multiple target voice segments, including: determining the start time and end time corresponding to each second voice segment; determining, from the multiple first voice segments, a first voice segment that is adjacent in time to the second voice segment based on the start time and the end time; in response to the adjacent first voice segment and the second voice segment corresponding to the same main voiceprint, splicing the first voice segment with the second voice segment to obtain a spliced ​​voice segment; and using the other first voice segments except the adjacent first voice segment and the spliced ​​voice segment as the multiple target voice segments. Specifically, based on the start and end times corresponding to the second voice segment and the first voice segment, respectively, the first voice segment adjacent to the second voice segment is determined, and it is determined whether the two correspond to the same main voiceprint, that is, whether the two correspond to the speech of the same person. If so, the two are spliced ​​to obtain a spliced ​​voice segment. Reference Figure 7b , the voice segment A represented by the solid line is the first voice segment, and the voice segment A represented by the dotted line is the second voice segment. The two are adjacent and correspond to the same main voiceprint A, so the two are merged to obtain the spliced ​​voice data A. In addition, the second voice segment E represented by the dotted line, the main voiceprint corresponding to its adjacent first voice segment is not E, therefore, the second voice segment E and its adjacent first voice segment cannot be spliced, but for the convenience of expression, the second voice segment E is also called the spliced ​​voice segment. The spliced ​​voice segment and the unspliced ​​first voice segment are used as target voice segments, that is, the voice segments corresponding to multiple people speaking in different time periods. The technical effect of the embodiment of the present invention is that when generating a speaker log, the recorded speech content of each speaker is continuous and complete, so that the user experience is better.

[0062] In the speaker log service, in addition to segmenting the voice data corresponding to each speaker to obtain multiple target voice segments, a speaker log can also be generated so that the user can clearly see which speaker spoke in which time period, as well as the order of speakers' speeches, etc. In an optional embodiment of the present invention, the method further includes: converting the voice data into text data and its corresponding first timestamp; assigning corresponding text data to the multiple target voice segments according to the first timestamp and the second timestamp corresponding to the multiple target voice segments; generating a speaker log according to the second timestamp, the text data corresponding to the multiple target voice segments, and the speaker identifiers corresponding to the multiple target voice segments; the speaker identifier is determined according to the primary voiceprints corresponding to the multiple target voice segments. The voice data collected during the multi-person conversation is converted to obtain the corresponding text data and its corresponding first timestamp, and the text data corresponding to the first timestamp corresponding to the second timestamp of each voice segment is determined from the text data. For example, the second timestamp corresponding to voice segment A is 9:00-9:03, and the text data corresponding to 9:00-9:03 is determined from the first timestamps 9:00-10:00 of the text data, and the text data corresponding to 9:00-9:03 is used as the text data corresponding to voice segment A. The speaker log service can generate a speaker log, such as Xiao Li, 9:00-9:03, speech content: "...". The ellipsis part is the text data corresponding to voice segment A.

[0063] In an optional embodiment of the present invention, the method of assigning corresponding text data to the multiple target voice segments according to the first timestamp and the second timestamps corresponding to the multiple target voice segments includes: determining the second timestamp corresponding to each voice segment; and determining the text data corresponding to the first timestamp that is the same as the second timestamp as the text data corresponding to the voice segment. For example, if the second timestamp corresponding to voice segment A is 9:00-9:03, the text data corresponding to 9:00-9:03 is determined from the first timestamps 9:00-10:00 of the text data, and the text data corresponding to 9:00-9:03 is used as the text data corresponding to voice segment A.

[0064] In order to improve the quality of the speaker log service, according to the above embodiment, after the corresponding text data is assigned to each voice segment, the text data is further corrected to improve the quality of the speaker log service. In the process of converting voice data into text data, it is very likely that there will be sentence segmentation errors, missing punctuation marks or punctuation errors. In this case, the text data is assigned to each voice segment according to the timestamp, and there will be a situation where a complete sentence is assigned to two voice segments. In this way, when generating the speaker log, it is impossible to accurately determine which voice segment the sentence belongs to, and which speaker's speech generates the text content. The sentence may be assigned to the previous speaker or the next speaker according to a simple rule, so that the generated speaker log has errors, and the generated log quality is low.

[0065] In view of the above situation, in an optional embodiment of the present invention, the method further comprises: for each independent sentence in the text data: in response to the independent sentence corresponding to at least two adjacent voice segments, modifying the independent sentence according to the text meaning of the independent sentence and the boundary of the adjacent voice segments. Figure 8 As shown, for a complete independent sentence in the text data obtained by voice conversion, "This is why the exception cannot be transmitted across servers", the independent sentence is assigned to voice segments C and D during allocation, which seems to be contradictory. Therefore, it is necessary to correct the independent sentence according to the specific semantics of the text and the dividing line between the two voice segments C and D, such as re-segmenting and supplementing the missing punctuation marks, such as "Why is this? The exception cannot be transmitted across servers." The correspondence between the corrected text data and a voice segment is shown in the figure. Figure 8 As shown. Figure 8 It can be seen that the logical relationship between the corrected text data and the corresponding relationship between the voice fragments or speakers is relatively reasonable. Therefore, the speaker service log generated in this way has high accuracy, improves the generation quality of the speaker log, and provides a better user experience.

[0066] The speech processing method is further illustrated below with a specific embodiment. The execution subject of the speech processing method of this embodiment can be a speaker log service system, which includes a conversion unit for converting speech into text, an overlapping speech detection model for detecting and separating overlapping speech, a first model for non-overlapping speech recognition, a second model for overlapping speech recognition, a speech error correction unit, a speech analysis unit, and a log generation unit for text data distribution, error correction and log generation. The structural framework of the speech processing system is as follows: Fig. 9 As shown. Fig. 9 The speech processing method performed by the speech processing system in the embodiment mainly includes the following steps: Fig.10 As shown:

[0067] Step S1001: using an overlapping speech detection model, voice data is separated to obtain non-overlapping speech and overlapping speech.

[0068] Step S1002: using the first model to identify multiple primary voiceprints from non-overlapping speech, and using the second model to extract voiceprint features from overlapping speech.

[0069] Step S1003: using the speech error correction unit to merge each two adjacent speech segments with high similarity in voiceprint features for the multiple speech segments obtained by segmenting the non-overlapping speech, and determining the corresponding primary voiceprint for the merged speech segments.

[0070] Step S1004: extracting voiceprint features from the overlapping speech using the second model.

[0071] Step S1005: using the speech analysis unit to respectively calculate the first similarities between the voiceprint feature and the plurality of primary voiceprints; and determining the target primary voiceprint corresponding to the overlapping speech according to the first similarities.

[0072] Step S1006: using a speech analysis unit to segment non-overlapping speech according to multiple main voiceprints to obtain a first speech segment, separating overlapping speech according to a target main voiceprint to obtain a second speech segment, and splicing the first speech segment and the second speech segment to obtain multiple target speech segments.

[0073] Step S1007: using a conversion unit to convert the speech data into text data.

[0074] Step S1008: using the log generation unit to distribute the text data to a plurality of target voice segments according to the time axis in the voice data.

[0075] Step S1009: Utilize the log generation unit to perform error correction on the allocated text data.

[0076] Step S1010: using a log generation unit to generate a speaker service log according to the error-corrected text data.

[0077] The speech processing method of the embodiment of the present invention obtains non-overlapping speech and overlapping speech by separating speech data; identifies multiple main voiceprints from non-overlapping speech, and extracts voiceprint features from overlapping speech; determines the target main voiceprint corresponding to the overlapping speech from the multiple main voiceprints according to the voiceprint features; segments the non-overlapping speech according to the multiple main voiceprints to obtain multiple first speech segments, separates the overlapping speech according to the target main voiceprint to obtain multiple second speech segments, and splices the multiple first speech segments and the multiple second speech segments to obtain multiple target speech segments. Thus, by separating the overlapping speech and the non-overlapping speech respectively and performing voiceprint recognition on them respectively, and segmenting or separating the speech data according to the multiple main voiceprints corresponding to the non-overlapping speech and the target main voiceprint corresponding to the overlapping speech, speech segments of multiple speakers speaking in different time periods are obtained, thereby solving the problem of low accuracy caused by performing voiceprint recognition on the overlapping speech and the non-overlapping speech as a whole, improving the accuracy of voiceprint recognition, and achieving the technical effect of accurately determining the speaker information.

[0078] Fig.11 4 is a schematic diagram of main modules of a speech processing device according to an embodiment of the present invention.

[0079] like Fig.11 As shown, the speech processing device 1100 of the embodiment of the present invention includes:

[0080] The speech separation module 1101 is used to separate the speech data to obtain non-overlapping speech and overlapping speech;

[0081] A voiceprint extraction module 1102, configured to identify a plurality of primary voiceprints from the non-overlapping speech and extract voiceprint features from the overlapping speech;

[0082] A speech analysis module 1103 is used to determine a target main voiceprint corresponding to the overlapping speech from the plurality of main voiceprints according to the voiceprint features;

[0083] The speech segmentation module 1104 is used to segment the non-overlapping speech according to the multiple main voiceprints to obtain multiple first speech segments, separate the overlapping speech according to the target main voiceprint to obtain multiple second speech segments, and splice the multiple first speech segments and the multiple second speech segments to obtain multiple target speech segments.

[0084] In an optional embodiment of the present invention, the speech analysis module 1103 is further used to respectively calculate the first similarities between the voiceprint feature and the multiple main voiceprints; for the first similarity corresponding to each main voiceprint, in response to the first similarity being greater than a first threshold, the main voiceprint is used as the target main voiceprint.

[0085] In an optional embodiment of the present invention, the speech segmentation module 1104 is also used to detect the main voiceprint corresponding to the non-overlapping speech starting from the time starting point of the non-overlapping speech; in response to detecting that the main voiceprint corresponding to the non-overlapping speech at the target time point is different from that at the previous time point, the non-overlapping speech is segmented at the target time point to obtain multiple first speech segments.

[0086] In an optional embodiment of the present invention, the speech segmentation module 1104 is further used to extract speech data corresponding to each target main voiceprint from the overlapping speech; and use the temporally continuous speech data corresponding to each target main voiceprint as a second speech segment.

[0087] In an optional embodiment of the present invention, the speech segmentation module 1104 determines the start time and the end time corresponding to each second speech segment; according to the start time and the end time, determines a first speech segment that is temporally adjacent to the second speech segment from the multiple first speech segments; in response to the adjacent first speech segment and the second speech segment corresponding to the same main voiceprint, splices the first speech segment and the second speech segment to obtain a spliced ​​speech segment; and uses the other first speech segments except the adjacent first speech segments and the spliced ​​speech segment as the multiple target speech segments.

[0088] In an optional embodiment of the present invention, the voiceprint extraction module 1102 is further used to extract the voiceprint features of the non-overlapping speech; cluster the voiceprint features to obtain multiple voiceprint feature sets; and generate a main voiceprint representing each voiceprint feature set based on each voiceprint feature set.

[0089] In an optional embodiment of the present invention, the speech analysis module 1103 is further used to divide the non-overlapping speech into multiple target speech segments according to the multiple primary voiceprints; calculate the second similarity between the voiceprint features corresponding to each two adjacent first speech segments; in response to the second similarity being greater than a second threshold, merge the two adjacent first speech segments; and determine the corresponding primary voiceprint for the merged first speech segment.

[0090] In an optional embodiment of the present invention, the speech analysis module 1103 is further used to determine a connection area between two adjacent first speech segments, wherein the connection area includes a first part and a second part; calculate the similarity between the voiceprint features of the first part and the voiceprint features of the second part, and determine the similarity as the second similarity.

[0091] In an optional embodiment of the present invention, the speech segmentation module 1104 is also used to convert the speech data into text data and its corresponding first timestamp; assign corresponding text data to the multiple target speech segments according to the first timestamp and the second timestamps corresponding to the multiple target speech segments; generate a speaker log according to the second timestamp, the text data corresponding to the multiple target speech segments and the speaker identifiers corresponding to the multiple target speech segments; the speaker identifier is determined based on the primary voiceprints corresponding to the multiple target speech segments respectively.

[0092] In an optional embodiment of the present invention, the speech segmentation module 1104 is further used to determine a second timestamp corresponding to each speech segment; and determine the text data corresponding to the first timestamp that is the same as the second timestamp as the text data corresponding to the speech segment.

[0093] In an optional embodiment of the present invention, the speech segmentation module 1104 is also used to, for each independent sentence in the text data: in response to the independent sentence corresponding to at least two adjacent speech segments, modify the independent sentence according to the text meaning of the independent sentence and the boundary between the adjacent speech segments.

[0094] The speech processing device of the embodiment of the present invention obtains non-overlapping speech and overlapping speech by separating speech data; identifies multiple main voiceprints from non-overlapping speech, and extracts voiceprint features from overlapping speech; determines the target main voiceprint corresponding to the overlapping speech from the multiple main voiceprints according to the voiceprint features; segments the non-overlapping speech according to the multiple main voiceprints to obtain multiple first speech segments, separates the overlapping speech according to the target main voiceprint to obtain multiple second speech segments, and splices the multiple first speech segments and the multiple second speech segments to obtain multiple target speech segments. Thus, by separating the overlapping speech and the non-overlapping speech respectively and performing voiceprint recognition on them respectively, and segmenting or separating the speech data according to the multiple main voiceprints corresponding to the non-overlapping speech and the target main voiceprint corresponding to the overlapping speech, speech segments of multiple speakers speaking in different time periods are obtained, thereby solving the problem of low accuracy caused by performing voiceprint recognition on the overlapping speech and the non-overlapping speech as a whole, improving the accuracy of voiceprint recognition, and achieving the technical effect of accurately determining the speaker information.

[0095] Fig.12 An exemplary system architecture 1200 is shown to which the speech processing method or speech processing apparatus according to the embodiment of the present invention can be applied.

[0096] like Fig.12As shown, the system architecture 1200 may include a voice collection terminal 1201, networks 1202 and 1203, a server 1204, and speaker terminals 1205, 1206, and 1207. The networks 1202 and 1203 are used to provide a medium for communication links between the voice collection terminal 1201, the server 1204, and the speaker terminals 1205, 1206, and 1207. The networks 1202 and 1203 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0097] The voice collection terminal 1201 and the speaker terminals 1205, 1206, and 1207 interact with the server 1204 through the networks 1202 and 1203 to receive or send messages, etc. The voice collection terminal 1201 is an audio collection device for collecting voice data at the scene of a multi-person conversation. The server 1204 obtains voice data from the voice collection terminal 1201, analyzes the voice data, obtains analysis results including the speech content of each speaker in the corresponding time period, and sends the analysis results to the speaker terminals 1205, 1206, and 1207.

[0098] The server 1204 may be a server that provides various services, for example, the server 1204 is a background management server that provides support for obtaining voice data from the voice acquisition terminal 1201 and analyzing and processing the voice data. The background management server may analyze and process the obtained voice data, and send the processing results, such as the speech content of each speaker in the corresponding time period, to the speaker terminals 1205, 1206, and 1207.

[0099] It should be noted that the speech processing method provided in the embodiment of the present invention is generally executed by the server 1204 , and accordingly, the speech processing device is generally disposed in the server 1204 .

[0100] It should be understood that Fig.12 The number of voice collection terminals, networks, service terminals and speaker terminals in the embodiment is only for illustration. Any number of voice collection terminals, networks, service terminals and speaker terminals may be provided according to the implementation requirements.

[0101] Reference below Fig.13 , which shows a schematic diagram of the structure of a computer system 1300 of an electronic device suitable for implementing an embodiment of the present invention. Fig.13 The voice collection end shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0102] like Fig.13As shown, the computer system 1300 includes a central processing unit (CPU) 1301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1302 or a program loaded from a storage part 1308 into a random access memory (RAM) 1303. In the RAM 1303, various programs and data required for the operation of the computer system 1300 are also stored. The CPU 1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0103] The following components are connected to the I / O interface 1305: an input section 1306 including a keyboard, a mouse, etc.; an output section 1307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1308 including a hard disk, etc.; and a communication section 1309 including a network interface card such as a LAN card, a modem, etc. The communication section 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the I / O interface 1305 as needed. A removable medium 1311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1310 as needed, so that a computer program read therefrom is installed into the storage section 1308 as needed.

[0104] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1309, and / or installed from the removable medium 1311. When the computer program is executed by the central processing unit (CPU) 1301, the above-mentioned functions defined in the system of the present invention are executed.

[0105] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0106] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0107] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor. For example, they may be described as: a processor includes a speech separation module, a voiceprint extraction module, a speech analysis module, and a speech segmentation module. The names of these modules do not, in some cases, constitute limitations on the modules themselves. For example, the speech segmentation module may also be described as a "module for segmenting the non-overlapping speech".

[0108] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: separating the voice data to obtain non-overlapping voice and overlapping voice; identifying multiple main voiceprints from the non-overlapping voice, and extracting voiceprint features from the overlapping voice; according to the voiceprint features, determining the target main voiceprint corresponding to the overlapping voice from the multiple main voiceprints; segmenting the non-overlapping voice according to the multiple main voiceprints to obtain multiple first voice segments, separating the overlapping voice according to the target main voiceprint to obtain multiple second voice segments, and splicing the multiple first voice segments and the multiple second voice segments to obtain multiple target voice segments.

[0109] According to the technical solution of the embodiment of the present invention, by separating the speech data, non-overlapping speech and overlapping speech are obtained; multiple main voiceprints are identified from the non-overlapping speech, and voiceprint features are extracted from the overlapping speech; according to the voiceprint features, the target main voiceprint corresponding to the overlapping speech is determined from the multiple main voiceprints; the non-overlapping speech is segmented according to the multiple main voiceprints to obtain multiple first speech segments, the overlapping speech is separated according to the target main voiceprint to obtain multiple second speech segments, and the multiple first speech segments and the multiple second speech segments are spliced ​​to obtain multiple target speech segments. Thus, by separating the overlapping speech and the non-overlapping speech respectively and performing voiceprint recognition on them respectively, and according to the multiple main voiceprints corresponding to the non-overlapping speech and the target main voiceprint corresponding to the overlapping speech, the speech data is segmented or separated to obtain speech segments of multiple speakers speaking in different time periods, thereby solving the problem of low accuracy caused by voiceprint recognition of the overlapping speech and the non-overlapping speech as a whole, improving the accuracy of voiceprint recognition, and achieving the technical effect of accurately determining the speaker information.

[0110] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A speech processing method, characterized in that: include: Separating the speech data to obtain non-overlapping speech and overlapping speech; Identifying a plurality of primary voiceprints from the non-overlapping speech, and extracting voiceprint features from the overlapping speech; According to the voiceprint feature, determining a target main voiceprint corresponding to the overlapping speech from the plurality of main voiceprints; The non-overlapping speech is segmented according to the multiple main voiceprints to obtain multiple first speech segments, the overlapping speech is separated according to the target main voiceprint to obtain multiple second speech segments, and the multiple first speech segments and the multiple second speech segments are spliced ​​to obtain multiple target speech segments.

2. The method according to claim 1, characterized in that The step of determining a target main voiceprint corresponding to the overlapping speech from the plurality of main voiceprints according to the voiceprint features includes: respectively calculating first similarities between the voiceprint feature and the plurality of main voiceprints; For each main voiceprint corresponding to a first similarity, in response to the first similarity being greater than a first threshold, the main voiceprint is used as a target main voiceprint.

3. The method according to claim 1, characterized in that The step of segmenting the non-overlapping speech according to the plurality of main voiceprints to obtain a plurality of first speech segments includes: Starting from the time starting point of the non-overlapping speech, detecting the main voiceprint corresponding to the non-overlapping speech; In response to detecting that the main voiceprint corresponding to the non-overlapping speech at the target time point is different from that at the previous time point, at the target time point, the non-overlapping speech is segmented to obtain a plurality of first speech segments.

4. The method according to claim 3, characterized in that The number of the target main voiceprints is at least two; and the step of separating the overlapping voices according to the target main voiceprints to obtain a plurality of second voice segments includes: Extracting speech data corresponding to each target main voiceprint from the overlapping speech; The temporally continuous voice data corresponding to each target main voiceprint is taken as a second voice segment.

5. The method according to claim 4, characterized in that The step of splicing the plurality of first voice segments and the plurality of second voice segments to obtain a plurality of target voice segments includes: Determine the start time and end time corresponding to each second voice segment; Determining, from the plurality of first voice segments, a first voice segment adjacent in time to the second voice segment according to the start time and the end time; In response to the fact that the primary voiceprints corresponding to the adjacent first voice segment and the second voice segment are the same, the first voice segment and the second voice segment are spliced ​​to obtain a spliced ​​voice segment; The other first speech segments excluding the adjacent first speech segments and the spliced ​​speech segment are used as the multiple target speech segments.

6. The method according to claim 1, characterized in that The identifying a plurality of main voiceprints from the non-overlapping speech comprises: Extracting voiceprint features of the non-overlapping speech; Clustering the voiceprint features to obtain multiple voiceprint feature sets; According to each of the voiceprint feature sets, a main voiceprint representing each of the voiceprint feature sets is generated.

7. The method according to claim 3, characterized in that Also includes: Calculating a second similarity between the voiceprint features corresponding to every two adjacent first speech segments; In response to the second similarity being greater than a second threshold, merging two adjacent first speech segments; For the merged first voice segment, determine the corresponding primary voiceprint.

8. The method according to claim 7, characterized in that The calculating the second similarity between the voiceprint features corresponding to each two adjacent first voice segments includes: Determine a connection area between two adjacent first speech segments, where the connection area includes a first part and a second part; The similarity between the voiceprint features of the first part and the voiceprint features of the second part is calculated, and the similarity is determined as the second similarity.

9. The method according to claim 1, characterized in that: Also includes: Converting the speech data into text data and its corresponding first timestamp; Allocating corresponding text data to the multiple target voice segments according to the first timestamp and the second timestamps corresponding to the multiple target voice segments; A speaker log is generated according to the second timestamp, the text data corresponding to the multiple target voice segments, and the speaker identifiers corresponding to the multiple target voice segments; the speaker identifier is determined according to the primary voiceprints corresponding to the multiple target voice segments respectively.

10. The method according to claim 9, characterized in that The allocating corresponding text data to the multiple target voice segments according to the first timestamp and the second timestamps corresponding to the multiple target voice segments includes: Determine a second timestamp corresponding to each voice segment; The text data corresponding to the first timestamp that is the same as the second timestamp is determined as the text data corresponding to the voice segment.

11. The method according to claim 10, characterized in that Also includes: For each independent sentence in the text data: In response to the independent sentence corresponding to at least two adjacent speech segments, the independent sentence is modified according to the textual meaning of the independent sentence and the boundary of the adjacent speech segments.

12. A speech processing device, characterized in that: include: A speech separation module is used to separate speech data to obtain non-overlapping speech and overlapping speech; A voiceprint extraction module, used to identify multiple main voiceprints from the non-overlapping speech and extract voiceprint features from the overlapping speech; A speech analysis module, configured to determine a target main voiceprint corresponding to the overlapping speech from the plurality of main voiceprints according to the voiceprint features; A speech segmentation module is used to segment the non-overlapping speech according to the multiple main voiceprints to obtain multiple first speech segments, separate the overlapping speech according to the target main voiceprint to obtain multiple second speech segments, and splice the multiple first speech segments and the multiple second speech segments to obtain multiple target speech segments.

13. An electronic device for speech processing, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 11.

14. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

15. A computer program product comprising a computer program, characterized in that When the program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images