Speech classification methods, devices, electronic equipment and storage media
By splitting and clustering voice data, and combining voice duration parameters and deep learning models, the problem of voice segment separation and identity determination under the condition of no pre-registered voiceprint features was solved, and the accurate separation of customer service and customer voices in customer service call recordings was achieved.
Patent Information
- Application Number
- CN202210546553.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-18
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2042-05-18
AI Technical Summary
In application scenarios where it is inconvenient to obtain voiceprint features in advance, how can we separate and identify the voice segments of different people without pre-registering voiceprint features, especially for separating customer service voice segments from customer voice segments in customer service call recordings?
By acquiring the speech data to be analyzed, it is split into multiple single-person speech segments and clustered. Using speech duration parameters and voiceprint features, target segments are selected for classification. Combined with a deep learning model, speech segment separation and identity determination are achieved without registering voiceprint features.
Without pre-registering voiceprint features, it can accurately separate voice segments from different people and identify their identities. It is suitable for scenarios such as customer service call recording, reducing the reliance on pre-acquiring voiceprint features.
Smart Images

Figure CN117133273B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of speech processing technology, and in particular to speech classification methods, apparatus, electronic devices and storage media. Background Technology
[0002] Currently, all voiceprint recognition applications require a registration process. Registration means that for each person with an identity ID, their voice needs to be collected first to extract their voiceprint features, and then registered into the database. The database stores the correspondence between the person's identity ID and voiceprint features. During the recognition process, the voiceprint features of the speech to be recognized are matched with the voiceprint features already stored in the database, thereby achieving the differentiation of speech features.
[0003] However, in some application scenarios where it is inconvenient to obtain voiceprint features in advance, such as separating customer service voice segments from customer service voice segments in customer service call recordings, it is impossible to separate the customer service voice segments from the user voice segments by matching them with voiceprint features in a database because it is inconvenient to obtain the voiceprint features of the customer service representatives and customers beforehand. Therefore, how to separate voice segments from different people without pre-registering voiceprint features has become an urgent technical problem to be solved. Summary of the Invention
[0004] The purpose of this disclosure is to provide a speech classification method, apparatus, electronic device, and storage medium to separate speech segments from different individuals without pre-registering voiceprint features. The specific technical solutions are as follows:
[0005] In a first aspect, embodiments of this disclosure provide a speech classification method, the method comprising:
[0006] Acquire the voice data to be analyzed, wherein the voice data includes the voices of a preset number of people;
[0007] The voice data is split into multiple individual voice segments;
[0008] Clustering is performed on each of the individual speech segments to obtain multiple segment classifications;
[0009] For each segment category, the speech duration parameter of that segment category is determined based on the duration of each individual speech segment in that segment category. The speech duration parameter of that segment category is positively correlated with the duration of each individual speech segment in that segment category.
[0010] Select a preset number of segments with the largest speech duration parameter for classification, and obtain a preset number of target segment classifications;
[0011] For each individual speech segment in any segment category other than the aforementioned target segment categories, assign that individual speech segment to a target segment category.
[0012] Optionally, splitting the voice data into multiple individual voice segments includes:
[0013] Endpoint detection is performed on the speech data to obtain the position of each endpoint in the speech data, wherein the endpoint represents the start and end position of the person's speech;
[0014] The voice data is split according to the position of each endpoint, and segments that do not include human voices are removed to obtain multiple single-person voice segments.
[0015] Optionally, the clustering of each individual speech segment to obtain multiple segment classifications includes:
[0016] The voiceprint features of each individual speech segment are extracted respectively;
[0017] Based on the voiceprint features of each individual speech segment, the individual speech segments are clustered to obtain multiple segment classifications.
[0018] Optionally, for each segment classification, the determination of the speech duration parameter for that segment classification based on the duration of each individual speech segment within that segment classification includes:
[0019] For each segment category, count the number of single-person speech segments in that segment category whose duration exceeds the first duration threshold, and obtain the speech duration parameter for that segment category.
[0020] Optionally, the method further includes:
[0021] For each target segment classification, the voiceprint features of that target segment classification are determined based on the voiceprint features of each individual speech segment in that target segment classification.
[0022] For each individual speech segment in any segment category other than the target segment categories, classifying that individual speech segment into a target segment category includes:
[0023] For each individual speech segment in the other segment categories, if the duration of the individual speech segment is less than the second duration threshold, the similarity between the voiceprint features of the individual speech segment and the voiceprint features of each of the target segment categories is calculated, and the individual speech segment is assigned to the target segment category with the highest similarity.
[0024] Optionally, if the identities of the individuals in the voice data are different, the method further includes:
[0025] The identity type of the personnel belonging to each of the target fragment categories is determined, wherein the personnel belonging to different target fragment categories are different;
[0026] For each individual speech segment in any segment category other than the target segment categories, classifying that individual speech segment into a target segment category includes:
[0027] For each individual speech segment in the other segment classifications, if the duration of the individual speech segment is not less than a second duration threshold, the individual speech segment is converted into speech text; the speech text of the individual speech segment is input into a pre-trained deep learning model to obtain the identity type of the person to whom the individual speech segment belongs; according to the identity type of the person to whom the individual speech segment belongs, the individual speech segment is classified into a target segment classification of the same identity type, wherein the deep learning module is trained using speech text of people with known identity types.
[0028] Optionally, the voice data includes the voice of customer service personnel and the voice of customer service personnel, the preset number is 2, and the preset number of target segment classifications includes a first target segment classification and a second target segment classification; the method further includes:
[0029] If the ratio of the total number of single-person speech segments in the first target segment classification to the total number of single-person speech segments in the second target segment classification meets a preset threshold condition, for each single-person speech segment in the first target segment classification and the second target segment classification, the speech text of the single-person speech segment is input into a pre-trained deep learning model to obtain the identity type of the person to whom the single-person speech segment belongs.
[0030] Secondly, embodiments of this disclosure provide a voice classification device, the device comprising:
[0031] The acquisition module is used to acquire the voice data to be analyzed, wherein the voice data includes the voices of a preset number of people;
[0032] The splitting module is used to split the voice data into multiple single-person voice segments;
[0033] The clustering module is used to cluster the individual speech segments to obtain multiple segment classifications.
[0034] The speech duration parameter determination module is used to determine the speech duration parameter of each segment category based on the duration of each individual speech segment in that segment category. The speech duration parameter of the segment category is positively correlated with the duration of each individual speech segment in that segment category.
[0035] The target segment classification determination module is used to select a preset number of segment classifications with the largest speech duration parameter to obtain a preset number of target segment classifications.
[0036] The segmentation module is used to classify each individual speech segment in a target segment category, for each segment category other than the target segment categories.
[0037] Optionally, the splitting module is specifically used for:
[0038] Endpoint detection is performed on the speech data to obtain the position of each endpoint in the speech data, wherein the endpoint represents the start and end position of the person's speech;
[0039] The voice data is split according to the position of each endpoint, and segments that do not include human voices are removed to obtain multiple single-person voice segments.
[0040] Optionally, the clustering module is specifically used for:
[0041] The voiceprint features of each individual speech segment are extracted respectively;
[0042] Based on the voiceprint features of each individual speech segment, the individual speech segments are clustered to obtain multiple segment classifications.
[0043] Optionally, the voice duration parameter determination module is specifically used for:
[0044] For each segment category, count the number of single-person speech segments in that segment category whose duration exceeds the first duration threshold, and obtain the speech duration parameter for that segment category.
[0045] Optionally, the device further includes:
[0046] The voiceprint feature determination module is used to classify each target segment and determine the voiceprint features of the target segment category based on the voiceprint features of each individual speech segment in the target segment category.
[0047] The partitioning module is also used for:
[0048] For each individual speech segment in the other segment categories, if the duration of the individual speech segment is less than the second duration threshold, the similarity between the voiceprint features of the individual speech segment and the voiceprint features of each of the target segment categories is calculated, and the individual speech segment is assigned to the target segment category with the highest similarity.
[0049] Optionally, since the identities of the individuals in the voice data are different, the device further includes:
[0050] The identity type determination module is used to determine the identity type of the personnel to which each of the target segment categories belongs, wherein the personnel to which different target segment categories belong are different;
[0051] The partitioning module is also used for:
[0052] For each individual speech segment in the other segment classifications, if the duration of the individual speech segment is not less than a second duration threshold, the individual speech segment is converted into speech text; the speech text of the individual speech segment is input into a pre-trained deep learning model to obtain the identity type of the person to whom the individual speech segment belongs; according to the identity type of the person to whom the individual speech segment belongs, the individual speech segment is classified into a target segment classification of the same identity type, wherein the deep learning module is trained using speech text of people with known identity types.
[0053] Optionally, the voice data includes the voice of customer service personnel and the voice of customer service personnel, the preset number is 2, and the preset number of target segment classifications includes a first target segment classification and a second target segment classification; the device further includes:
[0054] The identity type re-determination module is used to input the voice text of each individual voice segment into a pre-trained deep learning model to obtain the identity type of the person to which the individual voice segment belongs, provided that the ratio of the total number of individual voice segments in the first target segment category to the total number of individual voice segments in the second target segment category meets a preset threshold condition.
[0055] Thirdly, embodiments of this disclosure provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;
[0056] Memory, used to store computer programs;
[0057] A processor, when executing a program stored in memory, implements any of the speech classification methods disclosed herein.
[0058] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the speech classification methods of this disclosure.
[0059] The speech classification method provided in this disclosure includes: acquiring speech data to be analyzed, wherein the speech data includes the speech of a preset number of people; splitting the speech data into multiple individual speech segments; clustering each individual speech segment to obtain multiple segment categories; for each segment category, determining a speech duration parameter of the segment category based on the duration of each individual speech segment in the segment category, wherein the speech duration parameter of the segment category is positively correlated with the duration of each individual speech segment in the segment category; selecting a preset number of segment categories with the largest speech duration parameter to obtain a preset number of target segment categories; and assigning each individual speech segment in the other segment categories (excluding the target segment categories) to a target segment category. This method can separate speech segments from different people without pre-registering voiceprint features.
[0060] Of course, implementing any product or method of this disclosure does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description
[0061] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other embodiments can be obtained based on these accompanying drawings.
[0062] Figure 1 This is a schematic diagram of a first type of speech classification method provided in an embodiment of the present disclosure;
[0063] Figure 2 This is a second flowchart illustrating the speech classification method provided in an embodiment of the present disclosure;
[0064] Figure 3 This is a schematic diagram of a third process for a speech classification method provided in an embodiment of the present disclosure;
[0065] Figure 4 This is a schematic diagram of the fourth process of the speech classification method provided in the embodiments of this disclosure;
[0066] Figure 5 A fifth flowchart illustrating the speech classification method provided in this embodiment of the disclosure;
[0067] Figure 6 A schematic diagram of the structure of a speech classification device provided in an embodiment of this disclosure;
[0068] Figure 7 This is a schematic block diagram of an electronic device provided in this disclosure. Detailed Implementation
[0069] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art based on this disclosure are within the scope of protection of this disclosure.
[0070] In recent years, with the development of artificial intelligence technology, voiceprint recognition technology has become increasingly well-known. Verifying a person's identity through their voice is one of the key research directions in artificial intelligence technology in recent years. Voiceprint recognition plays an important role in various fields, including security, finance, and public services, primarily in the following aspects:
[0071] Voiceprint acquisition: The voiceprint recognition system acquires speech, adopts a unified acquisition standard, extracts voiceprint features, trains models and establishes a voiceprint model library, and compares the speech to be recognized with the voiceprint model library to achieve the identification of the speaker's identity.
[0072] Voiceprint identification: Voiceprint identification, also known as speech recognition, is a specialized technology for personal identification of spoken speech. For example, the speech of person A and the speech of speaker B in the recording to be tested can be converted into strip-shaped or curve-shaped speech graphs using a speech graph analyzer. By comparing the speech characteristics such as audio frequency, intensity, and time reflected in the two speech graphs, it can be determined whether person A is speaker B in the recording to be tested.
[0073] Fraud prevention: Helps telephone customer service personnel quickly identify incoming callers to detect impersonation and other incidents, thereby improving security and providing customers with friendly and personalized service.
[0074] Voiceprint attendance: Using voiceprint recognition combined with other biometric methods for attendance tracking is low-cost and can effectively reduce proxy attendance. It is particularly suitable for attendance scenarios involving large groups with high mobility.
[0075] Voiceprint access control: Using voiceprint recognition combined with other biometric methods as the unlocking method of the access control system, it can realize a biometric verification process without carrying a key.
[0076] In summary, all current voiceprint recognition applications require a registration process. Registration means that for each person with an identity ID, their voice must first be collected to extract their voiceprint features, which are then registered in a database. The database stores the correspondence between the person's identity ID and voiceprint features. During the recognition process, the voiceprint feature to be recognized is matched with the voiceprint features already in the database. If a match is successful, the identity ID corresponding to the successfully matched voiceprint feature in the database is used as the identity ID of the voiceprint feature to be recognized. However, how to determine identity when the correspondence between identity ID and voiceprint features has not been registered in advance has become a pressing technical problem that needs to be solved.
[0077] For application scenarios where it's inconvenient to obtain voiceprint features in advance, such as separating customer service voice segments from customer service voice segments in customer service call recordings, it's impossible to separate the customer service voice segments from the user voice segments by matching them with voiceprint features in a database because it's inconvenient to obtain their voiceprint features beforehand. Therefore, how to separate voice segments from different individuals and determine their identities without pre-registering voiceprint features has become an urgent technical problem to be solved.
[0078] To address at least one of the aforementioned problems, embodiments of this disclosure provide a speech classification method, see [link to relevant documentation]. Figure 1 The method includes:
[0079] S101. Obtain the voice data to be analyzed;
[0080] The voice data includes the voices of a preset number of individuals. The voice classification method provided in this disclosure can be implemented using an electronic device, specifically a smartphone, personal computer, or server. The voice data to be analyzed can be any data requiring voice segment classification. The voice data to be analyzed can be pre-recorded data stored in a database, or it can be real-time recorded voice data, all within the scope of this disclosure. The voice data to be analyzed includes the voices of a preset number of individuals. This preset number is a known number; for example, for customer service call recordings, the preset number is 2, meaning the customer service call recording includes the voices of both the customer service representative and the customer. Similarly, for one-on-one interview recordings, the preset number is 2, meaning the one-on-one interview recording includes the voice of both the host and the interviewee.
[0081] S102. The voice data is split into multiple single-person voice segments;
[0082] Because the voice data contains the voices of different people, and these voices are interspersed, for example, person A asks a question, person B answers, and then person A comments on person B's answer, it is necessary to split the entire voice data into multiple single-person voice segments, where each single-person voice segment contains only the voice of one person.
[0083] Speech data can be segmented into multiple single-speaker speech segments based on factors such as the continuity, intensity, and frequency of the speech. In some embodiments, segmenting the speech data into multiple single-speaker speech segments includes: performing endpoint detection on the speech data to obtain the positions of each endpoint in the speech data, wherein the endpoints represent the start and end positions of a person's speech; segmenting the speech data according to the positions of each endpoint, and removing segments that do not contain the person's speech to obtain multiple single-speaker speech segments. Specifically, VAD (Voice Activity Detection) technology can be used to perform endpoint detection on the speech data to obtain the start and end endpoints of the speech; and the continuous speech can be segmented into multiple speech segments based on the obtained endpoints, with each speech segment containing only the speech of one speaker.
[0084] S103. Cluster the individual speech segments to obtain multiple segment classifications;
[0085] Each individual speech segment is clustered using methods such as K-means clustering, Gaussian mixture clustering, DBSCAN clustering, or spectral clustering. Each segment classification includes at least one individual speech segment, and individual speech segments within the same classification are considered to belong to the same sound source.
[0086] S104. For each segment category, determine the speech duration parameter of that segment category based on the duration of each individual speech segment in that segment category.
[0087] The speech duration parameter for each segment category is positively correlated with the duration of each individual speech segment within that category. This parameter reflects the duration of each individual speech segment within each category. For example, for each segment category, the speech duration parameter can be the total duration of all individual speech segments within that category. Alternatively, for each segment category, the speech duration parameter can be calculated by counting the number of individual speech segments with a duration exceeding a first duration threshold. The first duration threshold can be set according to actual needs, for example, it can be set to 2 seconds, 3 seconds, or 5 seconds.
[0088] S105. Select a preset number of segments with the largest speech duration parameter to obtain a preset number of target segment classifications;
[0089] The preset number here is the same as the preset number mentioned in step S101. For example, if the voice data includes the voices of two people, then the two segments with the largest voice duration parameters are selected to obtain two target segment categories; for example, if the voice data includes the voices of four people, then the four segments with the largest voice duration parameters are selected to obtain four target segment categories. In some embodiments, the segment categories can be sorted in descending order of voice duration parameters, and a preset number of segment categories in the sorted queue can be selected as target segment categories.
[0090] Ideally, all individual voice clips from the same person would be clustered into a single category. However, due to errors in the clustering algorithm itself and environmental noise, individual voice clips from the same person might be clustered into multiple categories. Therefore, the total number of category clusters might exceed the preset number (the total number of people in the voice data). Since individual voice clips with errors are relatively few, we consider the preset number of category clusters with the largest voice duration parameter to represent the category of each person in the voice data. For example, for customer service call recordings, we consider the two category clusters with the largest voice duration parameter to be the customer service representative's category and the customer's category, respectively.
[0091] S106. For each individual speech segment in other segment categories besides the target segment categories, classify the individual speech segment into a target segment category.
[0092] Other segment classifications refer to segment classifications other than the target segment classification. For each individual speech segment in an other segment classification, the segment can be assigned to the target segment classification with the highest similarity by calculating the similarity between that individual speech segment and individual speech segments in the target segment classification. In one example, a preset similarity calculation method can be used based on the voiceprint features of the speech segments to calculate the similarity between them. This preset similarity calculation method could be: cosine similarity, Pearson correlation coefficient, covariance, or chi-square test, etc.
[0093] In this embodiment of the disclosure, instead of pre-registering the voiceprint features of each person, multiple segment classifications are obtained through clustering. Then, according to the number of people in the voice data, the same number of target segment classifications are selected. The single-person voice segments in other segment classifications are then assigned to the corresponding target segment classifications. Thus, the separation of voice segments of different people is achieved without pre-registering voiceprint features.
[0094] See Figure 2 In some embodiments, clustering the individual speech segments to obtain multiple segment classifications may include the following refinement steps:
[0095] S201. Extract the voiceprint features of each of the individual speech segments;
[0096] Voiceprints are not only specific but also relatively stable. Voiceprint features are often used as information reflecting the identity of a specific person. Specifically, pre-trained neural network models can be used to extract voiceprint features from speech segments.
[0097] The training method for the neural network model used to extract voiceprint features can be found in existing technologies. In one example, the training process of the neural network model includes the following steps: acquiring two single-person speech segments of the same person; inputting the two single-person speech segments into the neural network model respectively to obtain the voiceprint features of the two single-person speech segments; calculating the similarity between the two voiceprint features; judging whether the neural network model has converged based on the similarity; if the neural network model has converged, the trained neural network model is obtained; if the neural network model has not converged, the parameters of the neural network model are adjusted and the neural network model is trained until it converges.
[0098] S202. Based on the voiceprint features of each individual speech segment, cluster each individual speech segment to obtain multiple segment classifications.
[0099] Clustering methods include K-means clustering, Gaussian mixture clustering, DBSCAN clustering, or spectral clustering, etc. Each segment can be classified as speech data from the same sound source.
[0100] In this embodiment of the disclosure, the clustering of individual speech segments is completed based on the voiceprint features of each individual speech segment, which can cluster individual speech segments into multiple speakers.
[0101] See Figure 3 In some embodiments, for each segment category, the speech duration parameter of that segment category is determined based on the duration of each individual speech segment within that segment category. This may include the following refinement steps:
[0102] S301. For each segment category, count the number of single-person speech segments in that segment category whose duration is greater than the first duration threshold, and obtain the speech duration parameter of that segment category.
[0103] The voice duration parameter for this segment category is positively correlated with the duration of each individual voice segment within that category. The first duration threshold can be set according to actual needs. For example, in customer service voice calls, voice segments with a duration of less than 1 second are generally simple replies. It is understandable that there are many such simple replies in conversations; therefore, the first duration threshold can be set to a relatively large value, such as 3 seconds. Since the number of voice segments with a duration of more than 3 seconds is much smaller than the number of voice segments with a duration of more than 0.1 seconds, this facilitates the rapid determination of the voice duration parameter for each segment category.
[0104] The longer the duration of a single-person voice segment, the richer the voiceprint features of that segment, and the more accurate the clustering result. In this disclosure, it is believed that the clustering accuracy of single-person voice segments with a duration greater than the first duration threshold meets the specified error requirements. Moreover, in actual call scenarios, single-person voice segments with a duration greater than the first duration threshold are mostly valid call segments of people. Therefore, it is believed that the more single-person voice segments with a duration greater than the first duration threshold there are in the segment classification, the more accurate the clustering of that segment classification will be.
[0105] In this embodiment of the disclosure, the number of single-person speech segments with a duration greater than a first duration threshold in the segment classification is used as the speech duration parameter of the segment classification. That is, for a segment classification, the more single-person speech segments with a duration greater than the first duration threshold in the segment classification, the larger the speech duration parameter of the segment classification will be, so that the speech duration parameter can reflect the effectiveness and accuracy of the segment classification.
[0106] Customer service calls are mostly mono audio, often starting with a short intelligent voice prompt before connecting to online customer service. The calls also contain numerous very short phrases such as "um," "ah," "okay," and "understand." Furthermore, customer service calls are frequently mixed with environmental noise. Therefore, the characteristics of customer service calls—including intelligent voice prompts, customer service transfers, extremely short phrases and role switching, and environmental noise—make it difficult to use traditional voiceprint recognition or voiceprint clustering to separate and verify the speaker's identity.
[0107] To solve at least one of the above problems, see Figure 4 This disclosure also provides a speech classification method, including:
[0108] S401. Obtain the voice data to be analyzed;
[0109] The voice data includes the voices of a preset number of people, each with a different identity type.
[0110] S402, The voice data is split into multiple single-person voice segments;
[0111] S403. Cluster the individual speech segments to obtain multiple segment classifications;
[0112] S404. For each segment category, determine the speech duration parameter of that segment category based on the duration of each individual speech segment in that segment category.
[0113] Among them, the speech duration parameter of this segment classification is positively correlated with the duration of each individual speech segment in this segment classification.
[0114] S405. Select a preset number of segments with the largest speech duration parameter to obtain a preset number of target segment classifications;
[0115] Steps S401 to S405 are the same as those described above in S101 to S105, and will not be repeated here.
[0116] S406. For each target segment classification, determine the voiceprint features of the target segment classification based on the voiceprint features of each individual speech segment in the target segment classification.
[0117] In some embodiments, the voiceprint features of each individual speech segment in the target segment classification can be weighted and averaged, and the result of the weighted average can be used as the voiceprint features of the target segment classification.
[0118] In some embodiments, since the target segment classification is determined based on a clustering algorithm, the cluster centers in the target segment classification can also be used as the voiceprint features of the target segment classification.
[0119] S407. For each individual speech segment in the other segment classifications, if the duration of the individual speech segment is less than the second duration threshold, calculate the similarity between the voiceprint features of the individual speech segment and the voiceprint features of each of the target segment classifications, and classify the individual speech segment into the target segment classification with the highest similarity.
[0120] The second duration threshold can be customized according to actual conditions to filter out extremely short speech segments. That is, when the duration of a single-person speech segment is less than the second duration threshold, the single-person speech segment is considered extremely short. For example, the second duration threshold can be 500 milliseconds, 300 milliseconds, or 100 milliseconds. For extremely short speech segments, the similarity between their voiceprint features and the voiceprint features of each target segment category is calculated (in one example, cosine distance can also be calculated, where cosine distance represents similarity). The extremely short speech segment is then assigned to the target segment category with the highest similarity (smallest cosine distance).
[0121] S408. Determine the identity type of the personnel to which each of the target segments belongs;
[0122] In this scenario, different target segment categories belong to different personnel. For example, for customer service call audio, there are two target segment categories, corresponding to customer service and customer identities respectively. The audio duration parameters corresponding to each segment category are sorted in descending order, and the segment categories corresponding to the first two audio duration parameters in the sorted results are selected as the target segment categories. Specifically, the identity of the segment category corresponding to the first audio duration parameter in the sorted results is initialized as customer service, and the other is initialized as customer.
[0123] In some embodiments, the identity type of the person to whom the target segment belongs can also be determined by a deep learning model. Specifically, each individual speech segment in the target segment category is first converted into speech text using ASR (Automatic Speech Recognition) technology, and then the identity type of the person to whom the target segment belongs is determined by a deep learning model.
[0124] In one example, taking a customer service call scenario, the deep learning model is trained through the following steps: First, acquire customer service voice text and customer voice text, where the customer service voice text is labeled with a customer service identity tag, and the customer voice text is labeled with a customer identity tag. Then, select one voice text from each of the customer service and customer voice texts, input the selected voice text into the deep learning model, obtain the identity prediction result, calculate the loss of the deep learning model based on the identity prediction result and the identity tag of the selected voice text, and adjust the parameters of the deep learning model based on the loss. Continue selecting voice texts from each of the customer service and customer voice texts to train the deep learning model until the loss of the deep learning model converges. For other scenarios, the deep learning model can be trained by changing the type and identity tag of the sample voice text, which will not be elaborated here.
[0125] S409. For each individual speech segment in the other segment classifications, if the duration of the individual speech segment is not less than the second duration threshold, convert the individual speech segment into speech text; input the speech text of the individual speech segment into a pre-trained deep learning model to obtain the identity type of the person to whom the individual speech segment belongs; classify the individual speech segment into a target segment classification of the same identity type according to the identity type of the person to whom the individual speech segment belongs.
[0126] The deep learning module is trained using voice texts of individuals with known identity types. When the duration of a single-person voice segment is not less than a second duration threshold, it indicates that the single-person voice segment is not an extremely short speech. In this case, ASR (Automatic Speech Recognition) technology can be used to convert the single-person voice segment into voice text. The voice text is then input into a pre-trained deep learning model to obtain the identity type of the individual to whom the single-person voice segment belongs. Based on the identity type of the individual to whom the single-person voice segment belongs, the single-person voice segment is classified into a target segment category of the same identity type.
[0127] The method provided in this disclosure, without prior registration of voiceprint features, can not only separate voice segments from different individuals, but also identify the categories of individuals.
[0128] For customer service call scenarios, to reduce the impact of noise on speech segment separation, see [reference needed]. Figure 5 Another speech classification method provided in this disclosure includes:
[0129] S501, Obtain the voice data to be analyzed;
[0130] The voice data includes the voices of a preset number of people, including the voices of customer service personnel and customer personnel, and the preset number is 2.
[0131] S502, The voice data is split into multiple single-person voice segments;
[0132] S503. Cluster the individual speech segments to obtain multiple segment classifications;
[0133] S504. For each segment category, determine the speech duration parameter of that segment category based on the duration of each individual speech segment in that segment category.
[0134] Among them, the speech duration parameter of this segment classification is positively correlated with the duration of each individual speech segment in this segment classification.
[0135] S505. Select a preset number of segments with the largest speech duration parameter to obtain a preset number of target segment classifications;
[0136] The preset number of target segment classifications include a first target segment classification and a second target segment classification.
[0137] S506. For each target segment classification, determine the voiceprint features of the target segment classification based on the voiceprint features of each individual speech segment in the target segment classification.
[0138] S507. For each individual speech segment in the other segment classifications, if the duration of the individual speech segment is less than the second duration threshold, calculate the similarity between the voiceprint features of the individual speech segment and the voiceprint features of each of the target segment classifications, and classify the individual speech segment into the target segment classification with the highest similarity.
[0139] S508. Determine the identity type of the personnel to which each of the target segments belongs;
[0140] Different target segments belong to different personnel.
[0141] S509. For each individual speech segment in the other segment classifications, if the duration of the individual speech segment is not less than the second duration threshold, convert the individual speech segment into speech text; input the speech text of the individual speech segment into a pre-trained deep learning model to obtain the identity type of the person to whom the individual speech segment belongs; according to the identity type of the person to whom the individual speech segment belongs, classify the individual speech segment into a target segment classification of the same identity type.
[0142] The deep learning module is trained using the voice and text of people with known identity types.
[0143] Steps S501 to S509 are the same as those described above in S401 to S409, and will not be repeated here.
[0144] S510. If the ratio of the total number of single-person speech segments in the first target segment classification to the total number of single-person speech segments in the second target segment classification meets a preset threshold condition, for each single-person speech segment in the first target segment classification and the second target segment classification, the speech text of the single-person speech segment is input into a pre-trained deep learning model to obtain the identity type of the person to whom the single-person speech segment belongs.
[0145] After all individual voice segments are segmented, the number of individual voice segments with the identity label "customer service" (n1) and the number of individual voice segments with the identity label "customer" (n2) are calculated. When the ratio of n1 to n2 meets a preset threshold (generally, if the ratio of n1 to n2 is too large or too small, it can be determined that the clustering effect is poor; for example, the preset threshold can be set to 1000 or 0.0001), ASR (Automatic Speech Recognition) technology is used to convert all individual voice segments into speech text. The speech text is then input into a pre-trained deep learning model to obtain the identity type of the person to whom the individual voice segment belongs, and this identity type is redefined as the identity type of the person to whom the individual voice segment belongs.
[0146] In this embodiment of the disclosure, by inputting a single-person speech segment into a trained deep learning model, the identity type of the single-person speech segment can be re-determined. In the face of a large amount of environmental noise, the problem of clustering environmental noise into one category and other voices into another category can be solved more accurately, thus achieving speaker separation and identity confirmation.
[0147] In another aspect of this disclosure, a speech classification device is also provided, see [link to relevant document]. Figure 6 ,include:
[0148] The acquisition module 601 is used to acquire the voice data to be analyzed, wherein the voice data includes the voices of a preset number of people;
[0149] The splitting module 602 is used to split the voice data into multiple single-person voice segments;
[0150] Clustering module 603 is used to cluster the individual speech segments to obtain multiple segment classifications;
[0151] The speech duration parameter determination module 604 is used to determine the speech duration parameter of each segment category based on the duration of each individual speech segment in the segment category. The speech duration parameter of the segment category is positively correlated with the duration of each individual speech segment in the segment category.
[0152] The target segment classification determination module 605 is used to select a preset number of segment classifications with the largest speech duration parameter to obtain a preset number of target segment classifications.
[0153] The segmentation module 606 is used to classify each individual speech segment in a target segment category for each segment category other than the target segment categories.
[0154] In some embodiments, the splitting module 602 is specifically used for:
[0155] Endpoint detection is performed on the speech data to obtain the position of each endpoint in the speech data, wherein the endpoint represents the start and end position of the person's speech;
[0156] The voice data is split according to the position of each endpoint, and segments that do not include human voices are removed to obtain multiple single-person voice segments.
[0157] In some embodiments, the clustering module 603 is specifically used for:
[0158] The voiceprint features of each individual speech segment are extracted respectively;
[0159] Based on the voiceprint features of each individual speech segment, the individual speech segments are clustered to obtain multiple segment classifications.
[0160] In some embodiments, the voice duration parameter determination module 604 is specifically used for:
[0161] For each segment category, count the number of single-person speech segments in that segment category whose duration exceeds the first duration threshold, and obtain the speech duration parameter for that segment category.
[0162] In some embodiments, the apparatus further includes:
[0163] The voiceprint feature determination module is used to classify each target segment and determine the voiceprint features of the target segment category based on the voiceprint features of each individual speech segment in the target segment category.
[0164] The partitioning module is also used for:
[0165] For each individual speech segment in the other segment categories, if the duration of the individual speech segment is less than the second duration threshold, the similarity between the voiceprint features of the individual speech segment and the voiceprint features of each of the target segment categories is calculated, and the individual speech segment is assigned to the target segment category with the highest similarity.
[0166] In some embodiments, the voice data contains individuals with different identity types, and the device further includes:
[0167] The identity type determination module is used to determine the identity type of the personnel to which each of the target segment categories belongs, wherein the personnel to which different target segment categories belong are different;
[0168] The partitioning module is also used for:
[0169] For each individual speech segment in the other segment classifications, if the duration of the individual speech segment is not less than a second duration threshold, the individual speech segment is converted into speech text; the speech text of the individual speech segment is input into a pre-trained deep learning model to obtain the identity type of the person to whom the individual speech segment belongs; according to the identity type of the person to whom the individual speech segment belongs, the individual speech segment is classified into a target segment classification of the same identity type, wherein the deep learning module is trained using speech text of people with known identity types.
[0170] In some embodiments, the voice data includes the voice of customer service personnel and the voice of customer personnel, the preset number is 2, and the preset number of target segment classifications includes a first target segment classification and a second target segment classification; the device further includes:
[0171] The identity type re-determination module is used to input the voice text of each individual voice segment into a pre-trained deep learning model to obtain the identity type of the person to which the individual voice segment belongs, provided that the ratio of the total number of individual voice segments in the first target segment category to the total number of individual voice segments in the second target segment category meets a preset threshold condition.
[0172] This disclosure also provides an electronic device, such as... Figure 7 As shown, it includes a processor 701, a communication interface 702, a memory 703, and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.
[0173] Memory 703 is used to store computer programs;
[0174] The processor 701, when executing the program stored in the memory 703, implements any of the speech classification methods in this disclosure.
[0175] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.
[0176] The communication interface is used for communication between the aforementioned electronic devices and other devices.
[0177] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.
[0178] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0179] In another embodiment provided in this disclosure, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the speech classification methods in this disclosure.
[0180] In yet another embodiment provided in this disclosure, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the speech classification methods described above.
[0181] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).
[0182] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0183] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.
[0184] The above description is merely a preferred embodiment of this disclosure and is not intended to limit the scope of protection of this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure are included within the scope of protection of this disclosure.
Claims
1. A method of speech classification, characterized by, The method includes: Acquire the voice data to be analyzed, wherein the voice data includes the voices of a preset number of people; The voice data is split into multiple individual voice segments; Clustering is performed on each of the individual speech segments to obtain multiple segment classifications; For each segment category, the number of individual speech segments with a duration greater than the first duration threshold in that segment category is counted to obtain the speech duration parameter of that segment category. The speech duration parameter of that segment category is positively correlated with the duration of each individual speech segment in that segment category. According to the order of the speech duration parameters from largest to smallest, a preset number of segment categories with the largest speech duration parameters are selected from the multiple segment categories to obtain a preset number of target segment categories; For each individual speech segment in any segment category other than the aforementioned target segment categories, assign that individual speech segment to a target segment category; The voice data contains different identity types for each person, and the method further includes: The identity type of the personnel belonging to each of the target fragment categories is determined, wherein the personnel belonging to different target fragment categories are different; For each individual speech segment in any segment category other than the target segment categories, classifying that individual speech segment into a target segment category includes: For each individual speech segment in the other segment classifications, if the duration of the individual speech segment is not less than a second duration threshold, the individual speech segment is converted into speech text; the speech text of the individual speech segment is input into a pre-trained deep learning model to obtain the identity type of the person to whom the individual speech segment belongs; according to the identity type of the person to whom the individual speech segment belongs, the individual speech segment is classified into a target segment classification of the same identity type, wherein the second duration threshold is used to filter out extremely short speech segments; the deep learning model is trained using speech texts of people with known identity types.
2. The method of claim 1, wherein, The step of splitting the voice data into multiple single-person voice segments includes: Endpoint detection is performed on the speech data to obtain the position of each endpoint in the speech data, wherein the endpoint represents the start and end position of the person's speech; The voice data is split according to the position of each endpoint, and segments that do not include human voices are removed to obtain multiple single-person voice segments.
3. The method of claim 1, wherein, The clustering of each individual speech segment yields multiple segment classifications, including: The voiceprint features of each individual speech segment are extracted respectively; Based on the voiceprint features of each individual speech segment, the individual speech segments are clustered to obtain multiple segment classifications.
4. The method of claim 1, wherein, The method further includes: For each target segment classification, the voiceprint features of that target segment classification are determined based on the voiceprint features of each individual speech segment in that target segment classification. For each individual speech segment in any segment category other than the target segment categories, classifying that individual speech segment into a target segment category includes: For each individual speech segment in the other segment categories, if the duration of the individual speech segment is less than the second duration threshold, the similarity between the voiceprint features of the individual speech segment and the voiceprint features of each of the target segment categories is calculated, and the individual speech segment is assigned to the target segment category with the highest similarity.
5. The method according to claim 1 or 4, characterized in that, The voice data includes the voices of customer service personnel and customer personnel. The preset number is 2, and the preset number of target segment classifications includes a first target segment classification and a second target segment classification. The method further includes: when the ratio of the total number of single-person voice segments in the first target segment classification to the total number of single-person voice segments in the second target segment classification meets a preset threshold condition, for each single-person voice segment in the first target segment classification and the second target segment classification, inputting the voice text of the single-person voice segment into a pre-trained deep learning model to obtain the identity type of the person to whom the single-person voice segment belongs.
6. A speech classification apparatus characterized by comprising: The device includes: The acquisition module is used to acquire the voice data to be analyzed, wherein the voice data includes the voices of a preset number of people; The splitting module is used to split the voice data into multiple single-person voice segments; The clustering module is used to cluster the individual speech segments to obtain multiple segment classifications. The speech duration parameter determination module is used to count the number of single-person speech segments with a duration greater than a first duration threshold in each segment category, and obtain the speech duration parameter of the segment category. The speech duration parameter of the segment category is positively correlated with the duration of each single-person speech segment in the segment category. The target segment classification determination module is used to select a preset number of segment categories with the largest speech duration parameter from the multiple segment categories in descending order of the speech duration parameter, so as to obtain a preset number of target segment categories. The segmentation module is used to classify each individual speech segment in a target segment category, except for the target segment categories. The voice data contains different identity types for each person, and the device further includes: The identity type determination module is used to determine the identity type of the personnel to which each of the target segment categories belongs, wherein the personnel to which different target segment categories belong are different; The partitioning module is also used for: For each individual speech segment in the other segment classifications, if the duration of the individual speech segment is not less than a second duration threshold, the individual speech segment is converted into speech text; the speech text of the individual speech segment is input into a pre-trained deep learning model to obtain the identity type of the person to whom the individual speech segment belongs; according to the identity type of the person to whom the individual speech segment belongs, the individual speech segment is classified into a target segment classification of the same identity type, wherein the second duration threshold is used to filter out extremely short speech segments; the deep learning model is trained using speech texts of people with known identity types.
7. The apparatus of claim 6, wherein, The splitting module is specifically used for: Endpoint detection is performed on the speech data to obtain the position of each endpoint in the speech data, wherein the endpoint represents the start and end position of the person's speech; The voice data is split according to the position of each endpoint, and segments that do not include human voices are removed to obtain multiple single-person voice segments.
8. The apparatus of claim 6, wherein, The clustering module is specifically used for: The voiceprint features of each individual speech segment are extracted respectively; Based on the voiceprint features of each individual speech segment, the individual speech segments are clustered to obtain multiple segment classifications.
9. The apparatus of claim 6, wherein, The device further includes: The voiceprint feature determination module is used to classify each target segment and determine the voiceprint features of the target segment category based on the voiceprint features of each individual speech segment in the target segment category. The partitioning module is also used for: For each individual speech segment in the other segment categories, if the duration of the individual speech segment is less than the second duration threshold, the similarity between the voiceprint features of the individual speech segment and the voiceprint features of each of the target segment categories is calculated, and the individual speech segment is assigned to the target segment category with the highest similarity.
10. The apparatus of claim 6 or 9, wherein, The voice data includes the voice of customer service personnel and the voice of customer service personnel. The preset quantity is 2, and the preset quantity of target segment classifications includes a first target segment classification and a second target segment classification. The device also includes: The identity type re-determination module is used to input the voice text of each individual voice segment into a pre-trained deep learning model to obtain the identity type of the person to which the individual voice segment belongs, provided that the ratio of the total number of individual voice segments in the first target segment category to the total number of individual voice segments in the second target segment category meets a preset threshold condition.
11. An electronic device, comprising: It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-5.
12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-5.
Citation Information
Patent Citations
Speaker identity recognition method and device based on speaking content and storage medium
CN110136727A
Voice data processing method and device, computer equipment and storage medium
CN112289323A
Voiceprint clustering method and device, storage medium and electronic device
CN114464194A
