Method for training a speech classification model, speech classification method and apparatus

Abstract

The application provides a speech classification model training method, a speech classification method and device, comprising: a plurality of sample speech segments obtained by segmenting and determining a speech stream with a plurality of speakers; inputting each sample speech segment into a pre-constructed speech classification model to generate a target feature vector, a first prediction result and a second prediction result; clustering each sample speech segment according to the target feature vector to obtain a plurality of clustering clusters, and sample speech segments with the same pseudo label belong to speech segments corresponding to the same speaker; calculating a first error of the pseudo label and the first prediction result; calculating a second error of the second prediction result and label information; and training the speech classification model according to the first error and the second error. According to the embodiment of the application, the accuracy of classification can be improved, so that the speakers corresponding to the plurality of speech segments in the speech stream can be clustered without waiting for the entire conversation to end.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a training method for a speech classification model, a speech classification method and apparatus. Background Technology

[0002] With the development of speech classification technology, speech classification has been widely applied in various fields of life, such as multi-speaker speech segmentation and clustering, and extracting target speech from mixed speech. Among them, multi-speaker speech segmentation and clustering technology is widely used in fields such as automatic audio transcription and audio retrieval.

[0003] Currently, multi-speaker speech segmentation and clustering systems in related technologies can only filter out effective speech segments from the speech stream using speech endpoint detection algorithms, and then perform speaker inflection point detection on the effective speech segments to ensure that each speech segment contains only one speaker. After extracting the speech segments, speaker features are extracted, and the speech segments are classified by scoring the features.

[0004] However, the speaker classification in the relevant technology can only be based on the offline speaker log system. That is, the speakers corresponding to multiple speech segments in the speech stream can only be classified after the entire session has ended. It cannot complete the clustering of speakers corresponding to multiple speech segments in the speech stream in real time, resulting in redundancy in the speaker log system. Summary of the Invention

[0005] This application provides a training method for a speech classification model, a speech classification method and apparatus, which can perform speaker clustering in real time, thereby eliminating the need for the speaker log system to store a large amount of speech streams and saving the storage space of the speaker log system.

[0006] In a first aspect, embodiments of this application provide a method for training a speech classification model, the method comprising:

[0007] Obtain a training sample set; the training sample set includes multiple sample speech segments and label information for each of the multiple sample speech segments, the label information being used to represent the speaker corresponding to the sample speech segment, the multiple sample speech segments being segmented and determined from a speech stream with multiple speakers;

[0008] Each sample speech segment is input into a pre-built speech classification model to generate a target feature vector corresponding to each sample speech segment, a first prediction result of the sample speech segment, and a second prediction result of the sample speech segment.

[0009] Based on the target feature vector of each sample speech segment, each sample speech segment is clustered to obtain multiple clusters. Each cluster includes a pseudo-label, an identifier of the sample speech segment, and the target feature vector of the sample speech segment. Sample speech segments with the same pseudo-label belong to the speech segments corresponding to the same speaker.

[0010] Calculate the first error between the pseudo-label and the first prediction result;

[0011] Calculate the second error between the second prediction result and the label information;

[0012] The speech classification model is trained based on the first error and the second error until the speech classification model converges. After the speech classification model converges, the first error is less than a preset error threshold.

[0013] In one possible implementation, the speech classification model is trained based on the first error and the second error until the speech classification model converges:

[0014] The training error is obtained by weighted summation of the first error and the second error.

[0015] The speech classification model is trained based on the training error until the speech classification model converges.

[0016] In one possible implementation, each sample speech segment is clustered based on its target feature vector to obtain multiple clusters, including:

[0017] Calculate the similarity between the target feature vector of the sample speech segment and the audio centroid corresponding to each historical cluster; wherein the audio centroid is used to represent the center of all target feature vectors in the historical cluster;

[0018] The sample speech segment is classified into a target historical cluster, and the similarity between the audio centroid corresponding to the target historical cluster and the target feature vector of the sample speech segment is greater than a preset threshold.

[0019] Alternatively, a new cluster can be generated based on the identifier of the sample speech segment and the feature vector of the sample speech segment, and a pseudo-label corresponding to the new cluster can be generated, wherein the similarity between the target feature vector of the sample speech segment and the audio centroid corresponding to each historical cluster is less than a preset threshold.

[0020] In one possible implementation, prior to obtaining the training sample set, the method further includes:

[0021] Acquire a speech stream containing dialogue between multiple speakers;

[0022] The speech stream is denoised using a speech endpoint detection algorithm to obtain sample speech.

[0023] The sample speech is segmented using a sliding window to obtain multiple sample speech segments.

[0024] In one possible implementation, the method further includes:

[0025] Extract the feature vectors of Mel Frequency Cepstrum Coefficients (MFCCs) from the multiple sample speech segments;

[0026] The step of inputting each sample speech segment into a pre-built speech classification model to generate a target feature vector corresponding to each sample speech segment, a first prediction result for the sample speech segment, and a second prediction result for the sample speech segment includes:

[0027] The MFCC feature vector of each sample speech segment is input into the speech classification model to obtain the target feature vector corresponding to each sample speech segment, the first prediction result of the sample speech segment, and the second prediction result of the sample speech segment.

[0028] In one possible implementation, the speech classification model includes multiple time-delay neural network layers, multiple long short-term memory artificial neural network layers, a statistical layer, a sentence fully connected layer, and a first fully connected layer connected in sequence. The sentence fully connected layer in the speech classification model is also connected to a second fully connected layer.

[0029] The step of inputting each sample speech segment into a pre-built speech classification model to generate a feature vector corresponding to each sample speech segment, a first prediction result for each sample speech segment, and a second prediction result for each sample speech segment includes:

[0030] The multiple time-delay neural network layers generate a first feature vector for each sample speech segment based on each sample speech segment, and the first feature vector includes contextual relationship features between each sample speech segment;

[0031] The multiple long short-term memory artificial neural network layers are based on a first feature vector of each sample speech segment and a second feature vector of each sample speech segment, the second feature vector including speaker transition relationship features between each sample speech segment;

[0032] The statistical layer normalizes the second feature vector of each sample speech segment to obtain the third feature vector of each sample speech segment.

[0033] The fully connected layer of the sentence performs dimensionality reduction on the third feature vector of each sample speech segment to obtain the fourth feature vector of each sample speech segment;

[0034] The first fully connected layer generates a target feature vector for each sample speech segment and a first prediction result for each sample speech segment based on the fourth feature vector of each sample speech segment.

[0035] The second fully connected layer generates a second prediction result for each sample speech segment based on the fourth feature vector of each sample speech segment.

[0036] Secondly, embodiments of this application provide a speech classification method, including:

[0037] Real-time acquisition of target speech stream, wherein the target speech stream is formed by dialogue between multiple speakers;

[0038] The target speech stream is segmented to obtain multiple speech segments;

[0039] Each of the plurality of speech segments is input into the speech classification model as described in any one of claims 1-6 to generate a classification result for the plurality of speech segments.

[0040] In one possible implementation, the step of performing speech segmentation on the target speech stream to obtain multiple speech segments includes:

[0041] The target speech stream is denoised according to the speech endpoint detection algorithm to obtain the denoised target speech stream.

[0042] The denoised target speech stream is segmented using a sliding window to obtain multiple sample speech segments.

[0043] In one possible implementation, after performing speech segmentation on the target speech stream to obtain multiple speech segments, the method further includes:

[0044] Extract the MFCC feature vectors of the multiple speech segments;

[0045] The step of inputting each of the plurality of speech segments into the speech classification model described in the first aspect or any possible implementation thereof, and generating a classification result for the plurality of speech segments, includes:

[0046] The MFCC feature vectors of the multiple speech segments are input into the speech classification model to generate classification results for the multiple speech segments.

[0047] Thirdly, embodiments of this application provide a training apparatus for a speech classification model, comprising:

[0048] An acquisition module is used to acquire a training sample set; the training sample set includes multiple sample speech segments and label information for each of the multiple sample speech segments, the label information being used to represent the speaker corresponding to the sample speech segment, and the multiple sample speech segments being segmented and determined from a speech stream with multiple speakers;

[0049] The generation module is used to input each sample speech segment into a pre-built speech classification model to generate a target feature vector corresponding to each sample speech segment, a first prediction result of the sample speech segment, and a second prediction result of the sample speech segment;

[0050] The clustering module is used to cluster each of the sample speech segments according to the target feature vector of each sample speech segment to obtain multiple clusters. Each of the multiple clusters includes a pseudo-label, an identifier of the sample speech segment, and the target feature vector of the sample speech segment.

[0051] The calculation module is used to calculate a first error between the pseudo-label and the first prediction result, and to calculate a second error between the second prediction result and the label information;

[0052] The training module is used to train the speech classification model based on the first error and the second error until the speech classification model converges, and the first error is less than a preset error threshold after the speech classification model converges.

[0053] In one possible implementation, the clustering module is used for:

[0054] Calculate the similarity between the target feature vector of the sample speech segment and the audio centroid corresponding to each historical cluster; wherein the audio centroid is used to represent the center of all target feature vectors in the historical cluster;

[0055] The sample speech segment is classified into a target historical cluster, and the similarity between the audio centroid corresponding to the target historical cluster and the target feature vector of the sample speech segment is greater than a preset threshold.

[0056] Alternatively, a new cluster can be generated based on the identifier of the sample speech segment and the feature vector of the sample speech segment, and a pseudo-label corresponding to the new cluster can be generated, wherein the similarity between the target feature vector of the sample speech segment and the audio centroid corresponding to each historical cluster is less than a preset threshold.

[0057] In one possible implementation, the acquisition module is further configured to acquire a speech stream having dialogue between multiple speakers;

[0058] The device further includes:

[0059] The noise reduction module is used to perform noise reduction processing on the speech stream according to the speech endpoint detection algorithm to obtain sample speech;

[0060] The segmentation module is used to segment the sample speech according to a sliding window to obtain the multiple sample speech segments.

[0061] In one possible implementation, the device further includes:

[0062] The extraction module is used to extract the Mel-frequency cepstral coefficient (MFCC) feature vectors of the multiple sample speech segments;

[0063] The generation module is used to input the MFCC feature vector of each sample speech segment into the speech classification model to obtain the target feature vector corresponding to each sample speech segment, the first prediction result of the sample speech segment, and the second prediction result of the sample speech segment.

[0064] In one possible implementation, the generation includes a speech classification model and a second fully connected layer. The speech classification model includes multiple time-delay neural network layers, multiple long short-term memory artificial neural network layers, a statistical layer, a sentence fully connected layer, and a first fully connected layer connected in sequence. The sentence fully connected layer in the speech classification model is also connected to a second fully connected layer.

[0065] The multiple time-delay neural network layers generate a first feature vector for each sample speech segment based on each sample speech segment, and the first feature vector includes contextual relationship features between each sample speech segment;

[0066] The multiple long short-term memory artificial neural network layers are based on a first feature vector of each sample speech segment and a second feature vector of each sample speech segment, the second feature vector including speaker transition relationship features between each sample speech segment;

[0067] The statistical layer normalizes the second feature vector of each sample speech segment to obtain the third feature vector of each sample speech segment.

[0068] The fully connected layer of the sentence performs dimensionality reduction on the third feature vector of each sample speech segment to obtain the fourth feature vector of each sample speech segment;

[0069] The first fully connected layer generates a target feature vector for each sample speech segment and a first prediction result for each sample speech segment based on the fourth feature vector of each sample speech segment.

[0070] The second fully connected layer generates a second prediction result for each sample speech segment based on the fourth feature vector of each sample speech segment.

[0071] Fourthly, embodiments of this application provide a voice classification device, including:

[0072] An acquisition module is used to acquire a target speech stream in real time, wherein the target speech stream is formed by a dialogue between multiple speakers;

[0073] The segmentation module is used to segment the target speech stream into multiple speech segments.

[0074] The classification module is used to input each of the plurality of speech segments into the speech classification model described in the first aspect or any possible implementation of the first aspect, and generate a classification result for the plurality of speech segments.

[0075] In one possible implementation, the segmentation module is used for:

[0076] The target speech stream is denoised according to the speech endpoint detection algorithm to obtain the denoised target speech stream.

[0077] The denoised target speech stream is segmented using a sliding window to obtain multiple sample speech segments.

[0078] In one possible implementation, the apparatus further includes an extraction module for extracting MFCC feature vectors of the plurality of speech segments;

[0079] The generation module is used to input the MFCC feature vectors of the multiple speech segments into the speech classification model to generate the classification results of the multiple speech segments.

[0080] Fifthly, embodiments of this application provide a computer device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When executed by the processor, the computer program implements the method provided in the first aspect or any possible implementation of the first aspect, or implements the method provided in the second aspect.

[0081] Sixthly, embodiments of this application provide a computer storage medium storing instructions that, when executed on a computer, cause the computer to perform the method provided in the first aspect or any possible implementation thereof, or to implement the method provided in the second aspect.

[0082] The speech classification model training method, speech classification method, and apparatus provided in this application involve obtaining sample speech segments from multiple speaker speech streams, extracting target feature vectors from the sample speech segments during training, and then clustering each sample speech segment according to its target feature vector to obtain multiple clusters. Each cluster includes a pseudo-label, an identifier for the sample speech segment, and the target feature vector of the sample speech segment. The sample speech segments in each cluster belong to the same speaker. The speech classification model is trained using a first error between the pseudo-label and the prediction result, and a second error between the label information and the prediction result, thereby obtaining a trained speech classification model. Thus, during training, clustering is performed on the target feature vectors of sample speech segments extracted from the pre-built speech classification model, enabling the trained speech classification model to have clustering capabilities, thereby improving classification accuracy. This eliminates the need for subsequent clustering of the classification results generated by the speech classification model, and also eliminates the need to cluster speakers corresponding to multiple speech segments in the speech stream only after the entire conversation has ended. The speech classification model provided in this application embodiment can complete the clustering of speakers corresponding to multiple speech segments in the speech stream in real time, avoiding redundancy in the speaker log system. Attached Figure Description

[0083] Figure 1 A flowchart illustrating a training method for a speech classification model provided in an embodiment of this application is shown.

[0084] Figure 2 This paper shows a schematic diagram of the structure of a speech classification model provided in an embodiment of this application;

[0085] Figure 3 A flowchart illustrating a speech classification method provided in an embodiment of this application is shown;

[0086] Figure 4 A schematic diagram of the structure of a training device for a speech classification model provided in an embodiment of this application is shown;

[0087] Figure 5 This paper shows a schematic diagram of the structure of a speech classification device provided in an embodiment of this application;

[0088] Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of this application is shown. Detailed Implementation

[0089] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be described below with reference to the accompanying drawings.

[0090] In the description of the embodiments of this application, the words "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the words "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a specific manner.

[0091] In the description of the embodiments in this application, the term "and / or" is merely a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, B existing alone, and A and B existing simultaneously. Furthermore, unless otherwise stated, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals.

[0092] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and their variations all mean "including but not limited to," unless otherwise specifically emphasized.

[0093] With the development of speech classification technology, speech classification has been widely applied in various fields of life, such as multi-speaker speech segmentation and clustering, and extracting target speech from mixed speech. Among them, multi-speaker speech segmentation and clustering technology is widely used in fields such as automatic audio transcription and audio retrieval.

[0094] Currently, multi-speaker speech segmentation and clustering systems in related technologies can only filter out effective speech segments from the speech stream using speech endpoint detection algorithms, and then perform speaker inflection point detection on the effective speech segments to ensure that each speech segment contains only one speaker. After extracting the speech segments, speaker features are extracted, and finally, a clustering algorithm is used to aggregate the speech segments of each target speaker.

[0095] For example, currently, speech feature extraction in speaker segmentation mainly uses the x-vector model. Figure 1As shown, the x-vector model includes: a time-delay hidden layer, a statistical layer, and multiple fully connected sentence layers. The time-delay hidden layer consists of multiple parallel Time-Delay Neural Networks (TDNNs). x-vector primarily uses the TDNN network structure, where the input is the concatenation of the current speech frame and its preceding and following frames. The TDNN output corresponds to the number of speakers in the training dataset, and each output speaker label consists of a one-hot vector, where non-zero elements represent the speaker's position. After training, the x-vector model extracts features from each speech segment in a speech stream with multiple speakers, and uses PLDA scoring to calculate the similarity between these segments to construct a similarity matrix. Finally, hierarchical clustering or spectral clustering algorithms are used on the similarity matrix to aggregate the speech segments of each speaker, classifying them to obtain the target speaker for each segment. Therefore, feature extraction and clustering of speech segments are separated in time and space. Furthermore, current speaker log systems are offline, meaning speaker segmentation and clustering are only performed after the entire conversation has ended. Meanwhile, the current modular and cascaded speaker systems also suffer from error accumulation, meaning that each module depends on the precise operation of the previous module. Such speaker log systems are redundant and difficult to optimize.

[0096] However, the speaker segmentation and clustering system in the relevant technology can only be based on the offline speaker log system, that is, the speakers corresponding to multiple speech segments in the speech stream can only be clustered after the entire session has ended. It cannot complete the clustering of speakers corresponding to multiple speech segments in the speech stream in real time, resulting in redundancy of the speaker log system.

[0097] Based on this, embodiments of this application provide a training method for a speech classification model, a speech classification method and apparatus, which can perform real-time clustering of speakers corresponding to multiple speech segments in a speech stream, avoiding redundancy in the speaker log system.

[0098] Figure 1 This is a flowchart illustrating a training method for a speech classification model provided in an embodiment of this application. Figure 1 As shown, the training method for the speech classification model provided in this application embodiment may include S101-S106.

[0099] S101: Obtain a training sample set; the training sample set includes multiple sample speech segments and label information for each of the multiple sample speech segments, the label information being used to represent the speaker corresponding to the sample speech segment, the multiple sample speech segments being segmented and determined from a speech stream with multiple speakers.

[0100] Multiple sample speech segments are determined by segmenting speech streams with multiple speakers. These speech streams with multiple speakers can be speech streams generated from conversations between multiple people in a conference, or speech streams generated during the passage of multiple people's voices, etc.

[0101] In some embodiments, a speech stream containing dialogues between multiple speakers can be obtained from a database. To ensure the performance of the trained speech classification model, the speech stream can be denoised to remove noise and other artifacts, thus obtaining sample speech. For example, a speech endpoint detection algorithm can be used to denoise the speech stream. This algorithm detects valid speech segments in the speech stream and then segments these valid segments using a preset sliding window to obtain sample speech. Here, based on human speaking habits, the window length can be set to 1.5s and the window shift can be set to 0.25s to segment the sample speech, thereby obtaining multiple sample speech segments.

[0102] As an example, the denoised speech stream is "...Have you ever experienced that, where wearing a certain color of clothing makes your skin look darker? Have you ever had that experience? No…". Segmenting it using a sliding window with a window length of 1.5s and a window shift of 0.25s yields three speech segments: "[58.555, 60.458], Have you ever experienced that,?", "[60.845, 65.851], Wearing a certain color of clothing makes your skin look darker, have you ever had that experience?", and "[66.176, 66.752], No." Here, [a, b] represent the corresponding time points of the speech segments within the speech stream.

[0103] After the speech stream is segmented, each speech segment is labeled.

[0104] For example, the audio segments are: "[58.555,60.458], Do you ever experience that?", "[60.845,65.851], Wearing a certain color of clothing makes my skin look darker, have you ever had that experience?", and "[66.176,66.752], No." After annotation, these audio segments can be represented using the format "Time, Position, Gender, Language Standard, Audio Segment." Here, "Language Standard" refers to whether the speech is in standard Mandarin. The above three audio segments can be represented as: "[58.555,60.458"

[0105] G00000698, female, speaking Mandarin, "Have you ever experienced that kind of thing?", "[60.845,65.851] G00000698, female, speaking Mandarin, wearing a certain color of clothing that makes your skin look darker, have you ever had that experience?", "[66.176,66.752] G00000697, male, speaking Mandarin, "No." This is just an example to illustrate the data annotation process; the actual length of the speech stream during training is unlimited.

[0106] S102: Input the sample speech segment into a pre-built speech classification model to generate the target feature vector corresponding to the sample speech segment and the prediction result of the sample speech segment.

[0107] The segmented sample speech segments are input into a pre-built speech classification model to obtain the target feature vector corresponding to each speech segment and the prediction result of the sample speech segment.

[0108] The target feature vector is a vector that can characterize the features of a speech segment. Here, the target feature vector can be the feature vector output by the fully connected layer in the speech classification model.

[0109] In some embodiments, such as Figure 2 As shown, the speech classification model includes multiple time-delay neural network layers, multiple long short-term memory artificial neural network layers, a statistical layer, a sentence fully connected layer, and a first fully connected layer connected in sequence. The sentence fully connected layer in the speech classification model is also connected to a second fully connected layer.

[0110] The multiple time-delay neural network layers generate a first feature vector for each sample speech segment based on each sample speech segment. The first feature vector includes contextual relationship features between each sample speech segment. The multiple long short-term memory artificial neural network layers generate a second feature vector for each sample speech segment based on the first feature vector and the second feature vector, which includes speaker transition relationship features between each sample speech segment. The statistical layer normalizes the second feature vector of each sample speech segment to obtain a third feature vector. The fully connected sentence layer reduces the dimensionality of the third feature vector of each sample speech segment to obtain a fourth feature vector. The first fully connected layer generates a target feature vector and a first prediction result for each sample speech segment based on the fourth feature vector. The second fully connected layer generates a second prediction result for each sample speech segment based on the fourth feature vector.

[0111] In this way, the speech classification model can be trained based on the first and second prediction results.

[0112] In some implementation examples, to save computational resources, after segmenting into multiple sample speech segments, the MFCC feature vector of each sample speech segment can be extracted. This MFCC feature vector is then input into a speech classification model to obtain the target feature vector, a first prediction result, and a second prediction result for each sample speech segment. In this way, by extracting the MFCC feature vector, the features in the sample speech segments are compressed, thereby saving computational resources.

[0113] S103: Based on the target feature vector of each sample speech segment, cluster each sample speech segment to obtain multiple clusters. Each cluster includes a pseudo-label, an identifier of the sample speech segment, and the target feature vector of the sample speech segment. Sample speech segments with the same pseudo-label belong to the speech segments corresponding to the same speaker.

[0114] To avoid temporal and spatial disconnects in speech segment feature extraction and clustering, which can lead to redundancy and difficulty in optimization of the speaker log system, clustering can be performed during model training. Multiple sample speech segments can be clustered based on the target feature vector of each sample speech segment, resulting in multiple clusters. Each cluster includes a pseudo-label, an identifier for the sample speech segment, and the target feature vector of that sample speech segment. Sample speech segments with the same pseudo-label belong to the same speaker.

[0115] For example, the segmented sample speech fragments are as follows: "[58.555,60.458], G00000698, female, Mandarin, have you ever experienced this?", "[60.845,65.851], G00000698, female, Mandarin, wearing a certain color of clothing that makes your skin look dark, have you ever had this experience?", "[66.176,66.752], G00000697, male,

[0116] "Mandarin, no," "[67.440, 67.984], G00000698, female, Mandarin, okay," "[69.805, 70.340],"

[0117] G00000697, male, Mandarin, black; "[70.776,73.328], G00000697, male, Mandarin, black, white and gray are the three colors"; "[74.856,76.112], G00000697, male, Mandarin, the most versatile clothes"; "[77.075,78.312], G00000697, male, Mandarin, so should"; "[78.745,81.396], G00000697, male, Mandarin, buy more black, white and gray clothes". The feature vector output by the first fully connected layer includes the target feature vector of each sample speech segment. Based on each target feature vector, the feature vector corresponding to the pseudo label [1, 1, 2, 1, 2, 2, 2, 2, 2] is generated. Each element in the feature vector corresponding to the pseudo label corresponds one-to-one with each sample speech segment.

[0118] In some embodiments, the similarity between the target feature vector of the sample speech segment and the audio centroid corresponding to each historical cluster is calculated. The audio centroid represents the center of all target feature vectors in the historical cluster. The average vector of all target feature vectors in the historical cluster is calculated to obtain the audio centroid corresponding to the departure cluster. Here, historical clusters refer to already generated clusters.

[0119] If the similarity between the audio centroid of the target historical cluster and the target feature vector of the sample speech segment is greater than a preset threshold, the sample speech segment is classified into the target historical cluster.

[0120] If a new cluster is generated based on the identifier of the sample speech segment and the feature vector of the sample speech segment, and a pseudo-label corresponding to the new cluster is generated.

[0121] As an example, there are n speech segments, from speech segment 1 to speech segment n. Based on the target feature vector 1 corresponding to speech segment 1, a pseudo-label 1 is randomly generated as the pseudo-label for speech segment 1. The speaker number corresponding to pseudo-label 1 is then used as cluster 1. Next, the similarity between the target feature vector 2 and the target feature vector 2 of speech segment 2 is calculated. If the similarity between the target feature vector 2 and the target feature vector 2 of speech segment 2 is greater than a preset threshold, then the pseudo-label for speech segment 2 is pseudo-label 1, and speech segment 2 is classified into cluster 1. If the similarity between the target feature vector 2 and the target feature vector 2 of speech segment 2 is less than the preset threshold, then a pseudo-label 2 is generated for speech segment 2, and pseudo-label 1 and pseudo-label 2 are different. Cluster 2 is generated, and speech segment 2 is classified into cluster 2.

[0122] As another example, speech segments i through n have not been clustered. Currently, there are three historical clusters: cluster 1 through cluster 3. Cluster 1 corresponds to audio centroid 1, cluster 2 to audio centroid 2, and cluster 3 to audio centroid 3. The similarity between the target feature vector i of speech segment i and each of the three audio centroids is calculated. If the similarity between the target feature vector i of speech segment i and audio centroid 2 is greater than a preset threshold, then speech segment i is classified into cluster 3.

[0123] S104: Calculate the first error between the pseudo-label and the prediction result.

[0124] S105: Calculate the second error between the prediction result and the label information.

[0125] S106: Train the speech classification model based on the first error and the second error until the speech classification model converges. After the speech classification model converges, the first error is less than a preset error threshold.

[0126] The speech classification model can be trained based on the first error and the second error until the speech classification model converges. The first error after convergence is less than a preset threshold.

[0127] In some embodiments, the first error and the second error can be weighted and summed to obtain the training error; the speech classification model is trained based on the training error until the speech classification model converges.

[0128] The speech classification model training method provided in this application embodiment obtains sample speech segments from multiple speaker speech streams. During training, it extracts target feature vectors from the sample speech segments and clusters each sample speech segment based on its target feature vector, resulting in multiple clusters. Each cluster includes a pseudo-label, an identifier for the sample speech segment, and the target feature vector of the sample speech segment. The sample speech segments in each cluster belong to the same speaker. The speech classification model is trained using a first error between the pseudo-label and the prediction result, and a second error between the label information and the prediction result, resulting in a trained speech classification model. Thus, by clustering the sample speech segments based on the target feature vectors extracted from the pre-built speech classification model during training, the trained speech classification model possesses clustering capabilities, thereby improving classification accuracy. This eliminates the need for subsequent clustering of the classification results generated by the speech classification model and avoids the need to cluster the speakers corresponding to multiple speech segments in the speech stream only after the entire conversation has ended.

[0129] Figure 3This is a flowchart illustrating a speech classification method provided in an embodiment of this application. Figure 3 As shown, the speech classification method provided in this application embodiment may include S301 to S303.

[0130] S301: Real-time acquisition of target speech stream, wherein the target speech stream is formed by dialogue between multiple speakers.

[0131] S302: Perform speech segmentation on the target speech stream to obtain multiple speech segments.

[0132] In some embodiments, the target speech stream can be denoised according to a speech endpoint detection algorithm to obtain a denoised target speech stream; the denoised target speech stream can be segmented according to a sliding window to obtain the plurality of sample speech segments.

[0133] For example, noise reduction of a target speech stream can be performed using a speech endpoint detection algorithm. This algorithm detects valid speech segments within the target speech stream and then segments these segments using a pre-defined sliding window, resulting in multiple speech fragments. Here, the window length can be set to 1.5 seconds and the window shift to 0.25 seconds, based on human speaking habits, to segment the valid speech and obtain multiple speech fragments.

[0134] S303: Input each of the multiple speech segments into the speech classification model to generate the classification result of the multiple speech segments.

[0135] Speech classification models are achieved through Figure 1 The corresponding example shows a trained model that possesses clustering features, enabling more accurate classification of speech segments.

[0136] In some embodiments, to save computational resources, the MFCC feature vectors of the multiple sample speech segments can be extracted, and the MFCC feature vectors of the multiple speech segments can be input into the speech classification model to generate the classification results of the multiple speech segments.

[0137] The speech classification method provided in this application obtains sample speech segments from multiple speaker speech streams and extracts target feature vectors from these sample speech segments during training. Based on the target feature vectors of each sample speech segment, multiple clusters are obtained. Each cluster includes a pseudo-label, an identifier for the sample speech segment, and the target feature vector of that sample speech segment. The sample speech segments in each cluster belong to the same speaker. The speech classification model is trained using a first error between the pseudo-label and the prediction result, and a second error between the label information and the prediction result, resulting in a trained speech classification model. Thus, by clustering the sample speech segments based on the target feature vectors extracted from the pre-built speech classification model during training, the trained speech classification model possesses clustering capabilities, thereby improving classification accuracy. This eliminates the need for subsequent clustering of the classification results generated by the speech classification model, and also eliminates the need to cluster speakers corresponding to multiple speech segments in the speech stream only after the entire conversation has ended. The speech classification model provided in this application can complete the clustering of speakers corresponding to multiple speech segments in the speech stream in real time, avoiding redundancy in the speaker log system.

[0138] Based on the training method of the speech classification model in the above embodiments, this application also provides a training device for a speech classification model. Figure 4 This is a schematic diagram of the structure of a training device for a speech classification model provided in an embodiment of this application. Figure 4 As shown, the training device for the speech classification model provided in this application embodiment may include an acquisition module 401, a generation module 402, a clustering module 403, a calculation module 404, and a training module 405.

[0139] The acquisition module 401 is used to acquire a training sample set; the training sample set includes multiple sample speech segments and label information for each sample speech segment among the multiple sample speech segments, the label information being used to represent the speaker corresponding to the sample speech segment, and the multiple sample speech segments being segmented and determined from a speech stream with multiple speakers;

[0140] The generation module 402 is used to input each sample speech segment into a pre-built speech classification model to generate a target feature vector corresponding to each sample speech segment, a first prediction result of the sample speech segment, and a second prediction result of the sample speech segment;

[0141] Clustering module 403 is used to cluster each sample speech segment according to the target feature vector of each sample speech segment to obtain multiple clusters. Each cluster includes a pseudo label, an identifier of the sample speech segment, and the target feature vector of the sample speech segment.

[0142] The calculation module 404 is used to calculate the first error between the pseudo-label and the first prediction result, and to calculate the second error between the second prediction result and the label information;

[0143] The training module 405 is used to train the speech classification model based on the first error and the second error until the speech classification model converges, and the first error is less than a preset error threshold after the speech classification model converges.

[0144] In one possible implementation, the clustering module is used for:

[0145] Calculate the similarity between the target feature vector of the sample speech segment and the audio centroid corresponding to each historical cluster; wherein the audio centroid is used to represent the center of all target feature vectors in the historical cluster;

[0146] The sample speech segment is classified into a target historical cluster, and the similarity between the audio centroid corresponding to the target historical cluster and the target feature vector of the sample speech segment is greater than a preset threshold.

[0147] Alternatively, a new cluster can be generated based on the identifier of the sample speech segment and the feature vector of the sample speech segment, and a pseudo-label corresponding to the new cluster can be generated, wherein the similarity between the target feature vector of the sample speech segment and the audio centroid corresponding to each historical cluster is less than a preset threshold.

[0148] In one possible implementation, the acquisition module is further configured to acquire a speech stream having dialogue between multiple speakers;

[0149] The device further includes:

[0150] The noise reduction module is used to perform noise reduction processing on the speech stream according to the speech endpoint detection algorithm to obtain sample speech;

[0151] The segmentation module is used to segment the sample speech according to a sliding window to obtain the multiple sample speech segments.

[0152] In one possible implementation, the device further includes:

[0153] The extraction module is used to extract the Mel-frequency cepstral coefficient (MFCC) feature vectors of the multiple sample speech segments;

[0154] The generation module is used to input the MFCC feature vector of each sample speech segment into the speech classification model to obtain the target feature vector corresponding to each sample speech segment, the first prediction result of the sample speech segment, and the second prediction result of the sample speech segment.

[0155] In one possible implementation, the generation includes a speech classification model and a second fully connected layer. The speech classification model includes multiple time-delay neural network layers, multiple long short-term memory artificial neural network layers, a statistical layer, a sentence fully connected layer, and a first fully connected layer connected in sequence. The sentence fully connected layer in the speech classification model is also connected to a second fully connected layer.

[0156] The multiple time-delay neural network layers generate a first feature vector for each sample speech segment based on each sample speech segment, and the first feature vector includes contextual relationship features between each sample speech segment;

[0157] The multiple long short-term memory artificial neural network layers are based on a first feature vector of each sample speech segment and a second feature vector of each sample speech segment, the second feature vector including speaker transition relationship features between each sample speech segment;

[0158] The statistical layer normalizes the second feature vector of each sample speech segment to obtain the third feature vector of each sample speech segment.

[0159] The fully connected layer of the sentence performs dimensionality reduction on the third feature vector of each sample speech segment to obtain the fourth feature vector of each sample speech segment;

[0160] The first fully connected layer generates a target feature vector for each sample speech segment and a first prediction result for each sample speech segment based on the fourth feature vector of each sample speech segment.

[0161] The second fully connected layer generates a second prediction result for each sample speech segment based on the fourth feature vector of each sample speech segment.

[0162] The speech classification model training device provided in this application uses sample speech segments obtained from multiple speaker speech streams. During training, it extracts target feature vectors from the sample speech segments and clusters each sample speech segment based on its target feature vector, resulting in multiple clusters. Each cluster includes a pseudo-label, an identifier for the sample speech segment, and the target feature vector of the sample speech segment. The sample speech segments in each cluster belong to the same speaker. The speech classification model is trained using a first error between the pseudo-label and the prediction result, and a second error between the label information and the prediction result, resulting in a trained speech classification model. Thus, by clustering the sample speech segments based on the target feature vectors extracted from the pre-built speech classification model during training, the trained speech classification model possesses clustering capabilities, thereby improving classification accuracy. This eliminates the need for subsequent clustering of the classification results generated by the speech classification model and avoids the need to cluster the speakers corresponding to multiple speech segments in the speech stream only after the entire conversation has ended.

[0163] The training device for the speech classification model provided in this application embodiment can perform... Figure 1 The steps of the method in the corresponding embodiments are the same and can achieve the same technical effect. To avoid repetition, they will not be described in detail here.

[0164] Based on the speech classification method in the above embodiments, this application also provides a speech classification device. Figure 5 This is a schematic diagram of the structure of a speech classification device provided in an embodiment of this application. Figure 5 As shown, the speech classification device provided in this application embodiment may include an acquisition module 501, a segmentation module 502, and a classification module 503.

[0165] The acquisition module 501 is used to acquire the target speech stream in real time, wherein the target speech stream is formed by the dialogue of multiple speakers;

[0166] The segmentation module 502 is used to segment the target speech stream to obtain multiple speech segments;

[0167] The classification module 503 is used to input each of the plurality of speech segments into the speech classification model as described in any one of claims 1-6, and generate a classification result for the plurality of speech segments.

[0168] In one possible implementation, the segmentation module is used for:

[0169] The target speech stream is denoised according to the speech endpoint detection algorithm to obtain the denoised target speech stream.

[0170] The denoised target speech stream is segmented using a sliding window to obtain multiple sample speech segments.

[0171] In one possible implementation, the apparatus further includes an extraction module for extracting MFCC feature vectors of the plurality of speech segments;

[0172] The generation module is used to input the MFCC feature vectors of the multiple speech segments into the speech classification model to generate the classification results of the multiple speech segments.

[0173] The speech classification method provided in this application obtains sample speech segments from multiple speaker speech streams and extracts target feature vectors from these sample speech segments during training. Based on the target feature vectors of each sample speech segment, multiple clusters are obtained. Each cluster includes a pseudo-label, an identifier for the sample speech segment, and the target feature vector of that sample speech segment. The sample speech segments in each cluster belong to the same speaker. The speech classification model is trained using a first error between the pseudo-label and the prediction result, and a second error between the label information and the prediction result, resulting in a trained speech classification model. Thus, by clustering the sample speech segments based on the target feature vectors extracted from the pre-built speech classification model during training, the trained speech classification model possesses clustering capabilities, thereby improving classification accuracy. This eliminates the need for subsequent clustering of the classification results generated by the speech classification model, and also eliminates the need to cluster speakers corresponding to multiple speech segments in the speech stream only after the entire conversation has ended. The speech classification model provided in this application can complete the clustering of speakers corresponding to multiple speech segments in the speech stream in real time, avoiding redundancy in the speaker log system.

[0174] The speech classification device provided in this application embodiment is capable of performing... Figure 3 The steps of the method in the corresponding embodiments are the same and can achieve the same technical effect. To avoid repetition, they will not be described in detail here.

[0175] The following describes a computer device provided by an embodiment of this application.

[0176] Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 6 As shown, the computer device provided in this application embodiment can be used to implement the training method or speech classification method of the speech classification model described in the above method embodiments.

[0177] The computer device may include a processor 601 and a memory 602 storing computer program instructions.

[0178] Specifically, the processor 601 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0179] Memory 602 may include mass storage for data or instructions. For example, and not limitingly, memory 602 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 602 may include removable or non-removable (or fixed) media. Where appropriate, memory 602 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 602 is non-volatile solid-state memory.

[0180] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the methods according to this application.

[0181] The processor 601 reads and executes computer program instructions stored in the memory 602 to implement any of the speech classification model training methods or speech classification methods in the above embodiments.

[0182] In one example, the electronic device may also include a communication interface and a bus 610. For example, Figure 6 As shown, the processor 601, memory 602, and communication interface are connected through bus 610 and complete communication with each other.

[0183] The communication interface is mainly used to enable communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0184] Bus 610 includes hardware, software, or both, that couples components of an electronic device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 610 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0185] Furthermore, in conjunction with the above embodiments, this application embodiment can be implemented using a computer storage medium. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the speech classification model training methods or speech classification methods described in the above embodiments.

[0186] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0187] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0188] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0189] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A training method for a speech classification model, characterized in that, include: Obtain a training sample set; the training sample set includes multiple sample speech segments and label information for each of the multiple sample speech segments, the label information being used to represent the speaker corresponding to the sample speech segment, the multiple sample speech segments being segmented and determined from a speech stream with multiple speakers; Each sample speech segment is input into a pre-built speech classification model to generate a target feature vector corresponding to each sample speech segment, a first prediction result of the sample speech segment, and a second prediction result of the sample speech segment. Based on the target feature vector of each sample speech segment, each sample speech segment is clustered to obtain multiple clusters. Each cluster includes a pseudo-label, an identifier of the sample speech segment, and the target feature vector of the sample speech segment. Sample speech segments with the same pseudo-label belong to the speech segments corresponding to the same speaker. Calculate the first error between the pseudo-label and the first prediction result; Calculate the second error between the second prediction result and the label information; The speech classification model is trained based on the first error and the second error until the speech classification model converges. After the speech classification model converges, the first error is less than a preset error threshold.

2. The method according to claim 1, characterized in that, The speech classification model is trained based on the first error and the second error until the speech classification model converges: The training error is obtained by weighted summation of the first error and the second error. The speech classification model is trained based on the training error until the speech classification model converges.

3. The method according to claim 1, characterized in that, Based on the target feature vector of each sample speech segment, each sample speech segment is clustered to obtain multiple clusters, including: Calculate the similarity between the target feature vector of the sample speech segment and the audio centroid corresponding to each historical cluster; wherein the audio centroid is used to represent the center of all target feature vectors in the historical cluster; The sample speech segment is classified into a target historical cluster, and the similarity between the audio centroid corresponding to the target historical cluster and the target feature vector of the sample speech segment is greater than a preset threshold. Alternatively, a new cluster can be generated based on the identifier of the sample speech segment and the feature vector of the sample speech segment, and a pseudo-label corresponding to the new cluster can be generated, wherein the similarity between the target feature vector of the sample speech segment and the audio centroid corresponding to each historical cluster is less than a preset threshold.

4. The method according to any one of claims 1-3, characterized in that, Before obtaining the training sample set, the method further includes: Acquire a speech stream containing dialogue between multiple speakers; The speech stream is denoised using a speech endpoint detection algorithm to obtain sample speech. The sample speech is segmented using a sliding window to obtain multiple sample speech segments.

5. The method according to claim 1, characterized in that, The method further includes: Extract the Mel-frequency cepstral coefficients (MFCC) feature vectors from the multiple sample speech segments; The step of inputting each sample speech segment into a pre-built speech classification model to generate a target feature vector corresponding to each sample speech segment, a first prediction result for the sample speech segment, and a second prediction result for the sample speech segment includes: The MFCC feature vector of each sample speech segment is input into the speech classification model to obtain the target feature vector corresponding to each sample speech segment, the first prediction result of the sample speech segment, and the second prediction result of the sample speech segment.

6. The method according to claim 1, characterized in that, The speech classification model includes multiple time-delay neural network layers, multiple long short-term memory artificial neural network layers, a statistical layer, a sentence fully connected layer, and a first fully connected layer connected in sequence. The sentence fully connected layer in the speech classification model is also connected to a second fully connected layer. The step of inputting each sample speech segment into a pre-built speech classification model to generate a feature vector corresponding to each sample speech segment, a first prediction result for each sample speech segment, and a second prediction result for each sample speech segment includes: The multiple time-delay neural network layers generate a first feature vector for each sample speech segment based on each sample speech segment, and the first feature vector includes contextual relationship features between each sample speech segment; The multiple long short-term memory artificial neural network layers are based on a first feature vector of each sample speech segment and a second feature vector of each sample speech segment, the second feature vector including speaker transition relationship features between each sample speech segment; The statistical layer normalizes the second feature vector of each sample speech segment to obtain the third feature vector of each sample speech segment. The fully connected layer of the sentence performs dimensionality reduction on the third feature vector of each sample speech segment to obtain the fourth feature vector of each sample speech segment; The first fully connected layer generates a target feature vector for each sample speech segment and a first prediction result for each sample speech segment based on the fourth feature vector of each sample speech segment. The second fully connected layer generates a second prediction result for each sample speech segment based on the fourth feature vector of each sample speech segment.

7. A speech classification method, characterized in that, include: Real-time acquisition of target speech stream, wherein the target speech stream is formed by dialogue between multiple speakers; The target speech stream is segmented to obtain multiple speech segments; Each of the plurality of speech segments is input into the speech classification model as described in any one of claims 1-6 to generate a classification result for the plurality of speech segments.

8. The method according to claim 7, characterized in that, The target speech stream is segmented to obtain multiple speech segments, including: The target speech stream is denoised according to the speech endpoint detection algorithm to obtain the denoised target speech stream. The denoised target speech stream is segmented using a sliding window to obtain multiple sample speech segments.

9. A training device for a speech classification model, characterized in that, include: An acquisition module is used to acquire a training sample set; the training sample set includes multiple sample speech segments and label information for each of the multiple sample speech segments, the label information being used to represent the speaker corresponding to the sample speech segment, and the multiple sample speech segments being segmented and determined from a speech stream with multiple speakers; The generation module is used to input each sample speech segment into a pre-built speech classification model to generate a target feature vector corresponding to each sample speech segment, a first prediction result of the sample speech segment, and a second prediction result of the sample speech segment; The clustering module is used to cluster each of the sample speech segments according to the target feature vector of each sample speech segment to obtain multiple clusters. Each of the multiple clusters includes a pseudo-label, an identifier of the sample speech segment, and the target feature vector of the sample speech segment. The calculation module is used to calculate a first error between the pseudo-label and the first prediction result, and to calculate a second error between the second prediction result and the label information; The training module is used to train the speech classification model based on the first error and the second error until the speech classification model converges, and the first error is less than a preset error threshold after the speech classification model converges.

10. A speech classification device, characterized in that, include: An acquisition module is used to acquire a target speech stream in real time, wherein the target speech stream is formed by a dialogue between multiple speakers; The segmentation module is used to segment the target speech stream into multiple speech segments. A classification module is used to input each of the plurality of speech segments into the speech classification model as described in any one of claims 1-6, and generate a classification result for the plurality of speech segments.

Images