A single-channel speech separation method in a traffic environment

By grouping, segmenting and acoustic feature encoding of call recording materials in a conversation environment, and combining similar voice clips, it solves the time-consuming and labor-intensive problem in the existing technology, and realizes efficient single-channel voice separation, avoiding pre-collecting voice data and multiple steps to build a model.

CN113921033BActive Publication Date: 2024-11-05SICHUAN XW BANK CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111168264.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-29
Publication Date
2024-11-05
Estimated Expiration
2041-09-29

AI Technical Summary

Technical Problem

The prior art requires pre-collecting voice data of each operator in a telephone environment and establishing a separation model multiple times, which makes it time-consuming and labor-intensive and difficult to efficiently perform voice separation.

Method used

By grouping, segmenting, activity detection, pause processing, re-slicing and acoustic feature encoding of the call recording materials, voice fragments are merged according to similarity, single-channel voice separation is achieved, avoiding pre-collecting voice data and multiple models.

Benefits of technology

The efficiency of voice separation is improved, the removal of silent clips and obvious pauses are achieved, and the voice clips are merged according to the acoustic characteristics, and the single-channel recorded voice separation between the operator and the customer is realized, solving the time-consuming and labor-intensive problem.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113921033B_ABST
    Figure CN113921033B_ABST
Patent Text Reader

Abstract

The present invention belongs to the technical field of audio data separation, and specifically relates to a single-channel voice separation method in a telephone environment. The present invention groups recorded call materials with different operators as grouping identifiers, segments the call recordings of each operator, performs activity detection and pause detection on the segmented segments, removes silent segments, and segments segments with obvious pauses; performs re-segmentation processing, encodes different voice segments based on acoustic features, finds the operator's segments based on similarity and merges them, so as to complete the voice separation of the single-channel recording of the operator's call with the customer, and the present invention does not need to collect the voice of each operator in advance, nor does it need to establish models multiple times.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of audio data separation, and in particular relates to a single-channel voice separation method in a telephone traffic environment. Background Art

[0002] At present, in various industries, telephone communication is gradually used for marketing, collection, and after-sales service. With the fierce competition in the same industry, companies hope that telephone services can achieve better results with the investment of manpower and financial resources. Therefore, recording the content of calls for the specific work performance evaluation of operators or to prevent consumer disputes is a common practice for companies. At present, the commonly used recording equipment uses dual-track audio technology, which can store the voice information of both parties of the call separately; but due to the problem of equipment and migration costs, a large number of companies still use single-track recording equipment. For the audio recorded by single-track recording equipment, it is difficult to conduct targeted analysis on the salesperson's speech level and whether taboo words and phrases are used.

[0003] After searching, it was found that the invention patent application with publication number CN111785291A discloses a speech separation method and a speech separation device, which mainly divides the speech into multiple segments, and then classifies them through a separation model to obtain a set of speech segments of each speaker; the establishment of the separation model requires the pre-collection of the voiceprint features of each operator; and the separation model needs to be re-established at least twice, each time it is established based on the current speech database, and any two established separation models are established based on different speech databases. The above method is corrected by reconstruction to ensure that the target speech segment in the speech segment set is the speech of the same operator, thereby improving the accuracy of speech separation.

[0004] However, the above method for voice separation requires obtaining the operator's voice data in advance, and then obtaining the corresponding voiceprint features and applying them to the separation model. For a large-scale call team, it is time-consuming and laborious to collect the voice data of each operator in advance, and it is necessary to build models based on the voice database multiple times to improve the accuracy of voice separation; although it can improve the accuracy of voice separation, building models multiple times will also cause time-consuming and labor-intensive problems. Summary of the invention

[0005] The invention discloses a single-channel speech separation method in a telephone environment, which is intended to solve the time-consuming and labor-intensive problem of pre-collecting speech data of each operator and establishing models multiple times mentioned in the background technology.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0007] A single-channel voice separation method in a traffic environment comprises the following steps:

[0008] Step 1: According to the call recording materials, group them by different operators and integrate the call recording materials of each operator;

[0009] Step 2: Segment the audio of each operator's call recording material; perform activity detection and pause detection on the segmented segments, delete the silent segments in the audio, and segment the parts with obvious pauses;

[0010] Step 3: The segments divided in step 2 are divided again, and different types of speech segments are encoded according to acoustic features of each segmented segment to form multiple matrices;

[0011] Step 4: Find the operator's segments from the segments segmented in step 3 according to the similarity, and merge the similar segments of the operator.

[0012] The present invention performs activity detection and pause detection on the audio after segmentation, thereby eliminating silent segments and segmenting obvious pause parts, and re-segmenting the segmented segments. According to acoustic features, different types of speech are encoded, and the speech segments of the operator are found out according to similarity and then merged, thereby realizing speech separation of single-channel recordings of conversations between the operator and the customer. In addition, the present invention does not need to collect the speech of each operator in advance, nor does it need to establish models multiple times.

[0013] Preferably, the step 2 segments the audio of each operator's call recording material based on the open source pyannote library.

[0014] The present invention segments the audio of each operator's call recording material based on the open source pyannote library, thereby realizing activity detection and pause detection.

[0015] Preferably, step 3 comprises the following steps:

[0016] Step 3.1: Segment the segment segmented in step 2 again with a step length of t seconds;

[0017] Step 3.2: Encode all the segments divided in step 3.1, encode each t seconds of audio segment into a 1*512 vector, and obtain the matrix corresponding to each audio segment.

[0018] Preferably, step 4 selects and updates the centroid based on cosine similarity, calculates the cosine similarity between the matrix and the centroid, determines the speech belonging to the operator in all segments based on a set threshold, and merges them.

[0019] Preferably, step 4 comprises the following steps:

[0020] Step 4.1: Set the similarity threshold k;

[0021] Step 4.2: Calculate X t and X 2t The cosine similarity matrix of , and the corresponding matrix is ​​obtained; the rows with similarity greater than the threshold are taken out, and X is taken out according to the matrix t and X 2t The corresponding rows whose similarity is greater than the threshold;

[0022] Step 4.3: X 3t , X 4t , X 5t …X nt Respectively and X t Calculate the cosine similarity matrix and get the corresponding matrix; take out the rows with similarity greater than the threshold, and take out X according to the matrix 3t , X 4t , X 5t …X nt The corresponding rows whose similarity is greater than the threshold;

[0023] Step 4.4: Compress the row data taken out in step 4.2 and step 4.3 into an initial centroid of a vector with a dimension of 1*512;

[0024] Step 4.5: The initial centroid and X t , X 2t , X 3t , X 4t , X 5t …X nt Calculate the cosine similarity matrix of the matrix, and take out the rows with similarity greater than the threshold in each matrix based on the threshold;

[0025] Step 4.6: Combine the rows taken out in step 4.5 and the initial centroid and compress them into an updated centroid vector of 1*512;

[0026] Step 4.7: Update the centroid and X t , X 2t , X 3t , X 4t , X 5t …X nt The cosine similarity matrix is ​​calculated based on the matrix, and the rows with similarity greater than the threshold are respectively taken out from each matrix based on the threshold; the speech segments corresponding to the rows taken out in this step are merged to obtain the operator's audio.

[0027] The present invention can realize the voice separation of the single-channel recording of the conversation between the operator and the customer by simply updating the centroid multiple times based on the original model, thereby improving the efficiency.

[0028] Preferably, t is equal to 1 second; K is equal to 0.8.

[0029] In summary, due to the adoption of the above technical scheme, the beneficial effects of the present invention are as follows: the present invention performs activity detection and pause detection on the audio after segmentation, thereby eliminating silent segments and segmenting obvious pause parts, and re-segmenting the segmented segments again, encoding different types of speech according to acoustic features, finding the operator's speech segments according to similarity and merging them, thereby achieving speech separation of single-channel recordings of conversations between operators and customers, and the present invention does not need to collect the speech of each operator in advance, nor does it need to establish models multiple times; it effectively solves the time-consuming and laborious problems mentioned in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The present invention will now be described by way of example with reference to the accompanying drawings, in which:

[0031] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0032] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0033] The following is combined with Figure 1 Describe the embodiments of the present invention in detail;

[0034] A single-channel voice separation method in a traffic environment comprises the following steps:

[0035] Step 1: Collect call recording materials and group them by different operators. That is, the recording materials of the same operator and different customers are considered to be in the same category.

[0036] Step 2: Based on the open source pyannote library, segment different audios of the same operator; perform activity detection and pause detection, delete all noise or silent parts in the audio, and segment the parts with obvious pauses;

[0037] Step 3: Segment the segments segmented in step 2, with the segmentation step set to 1 second, and then encode each segment to form multiple feature matrices;

[0038] For example, suppose that operator A has 20 voice records, which are recorded as A1, A2, A3, A4, A5, A6, A7, A8, A9, A10, A110, A20, A120, A130, A140, A150, A160, A170, A180, A190, A210, A220, A230, A240, A250, A260 20 .

[0039] For A1, A2…A 20 , and then divide it again with a step length of 1 second to obtain {A 1,1 , A 1,2 …A 1,50}, {A 2,1 , A 2,2 …A 2,60}…{A 20,1 , A 20,2 …A 20,80}(Assume that the audio segment A1 of operator A is 50 seconds long, and the audio segment A2 is 60 seconds long... 20 Duration: 80 seconds).

[0040] Encode the segmented audio and record it as X1, X2…X 20 , every second of audio segment A i,j can be encoded as a 1*512 vector. Therefore, X1, X2…X 20 They correspond to matrices of 50*512, 60*512…80*512 dimensions respectively.

[0041] The step length t may also be set to 2 seconds, 1.5 seconds, 0.5 seconds, 0.4 seconds, etc.

[0042] Step 4: Perform similarity voice segment matching, select and update the centroid based on cosine similarity, and then calculate the cosine similarity between different matrices and the centroid. According to the set threshold, find the voice belonging to the operator in all small segments and merge them. This realizes the voice separation of the single-channel recording of the operator's conversation with the customer:

[0043] The specific steps include:

[0044] Step 4.1: Set the similarity threshold to 0.8; the similarity threshold can also be set to 0.9 or 0.88, etc. The cosine similarity calculation method is:

[0045]

[0046] Where: A and B are both n-dimensional vectors, cos(θ) is a numerical value, indicating the similarity between vectors A and B, and its value range is 0-1.

[0047] Step 4.2: Calculate the cosine similarity matrix of X2 and X1, denoted as S 1,2 , corresponding to a 50*60 dimensional matrix (i.e., the time length of audio segment A1 * the time length of A2). Take out the rows whose similarity is greater than the threshold, and take out the corresponding rows whose similarity is greater than the threshold in X2 and X1 according to the matrix;

[0048] Step 4.3: Set X3, X4, ... 20 Calculate the cosine similarity matrix with X1 respectively, and get the corresponding matrix; take out the rows with similarity greater than the threshold, and take out X3, X4...X according to the matrix 20 The corresponding rows whose similarity is greater than the threshold;

[0049] Step 4.4: Compress the row data taken out in step 4.2 and step 4.3 into an initial centroid of a vector with a dimension of 1*512;

[0050] Step 4.5: Compare the initial centroid with X1, X2, X3, X4…X 20 Calculate the cosine similarity matrix of the matrix, and take out the rows with similarity greater than the threshold in each matrix based on the threshold;

[0051] Step 4.6: Combine the rows taken out in step 4.5 and the initial centroid and compress them into an updated centroid vector of 1*512;

[0052] Step 4.7: Update the centroid and X1, X2, X3, X4…X 20 The cosine similarity matrix is ​​calculated based on the matrix, and the rows with similarity greater than the threshold are taken out from each matrix based on the threshold; the speech segments corresponding to the rows taken out in this step are merged to obtain the operator's audio; and the speech separation of the single-channel recording of the operator's conversation with the customer is realized.

[0053] The above-mentioned embodiments only express the specific implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the protection scope of the present application. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the technical solution concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A single-channel speech separation method in a traffic environment, characterized in that: The following steps are involved: Step 1: According to the call recording materials, group them by different operators and integrate the call recording materials of each operator; Step 2: Segment the audio of each operator's call recording material; perform activity detection and pause detection on the segmented segments, delete the silent segments in the audio, and segment the parts with obvious pauses; Step 3: The segments divided in step 2 are divided again, and different types of speech segments are encoded according to acoustic features of each segmented segment to form multiple matrices; Step 4: Find the operator's segments from the segments divided in step 3 according to the similarity, and merge the similar segments of the operator; The step 4 selects and updates the centroid according to the cosine similarity, calculates the cosine similarity between the matrix and the centroid, determines the speech belonging to the operator in all the segments according to the set threshold, and merges them; The step 4 comprises the following steps: Step 4.1: Set the similarity threshold K; Step 4.2: Calculate X t and X 2t The cosine similarity matrix of , and the corresponding matrix is ​​obtained; the rows with similarity greater than the threshold are taken out, and X is taken out according to the matrix t and X 2t The corresponding rows whose similarity is greater than the threshold; Step 4.3: Place X 3t , X 4t , X 5t …X nt Respectively and X t Calculate the cosine similarity matrix and get the corresponding matrix; take out the rows with similarity greater than the threshold, and take out X according to the matrix 3t , X 4t , X 5t …X nt The corresponding rows whose similarity is greater than the threshold; Step 4.4: Compress the row data taken out in step 4.2 and step 4.3 into an initial centroid of a vector with a dimension of 1*512; Step 4.5: The initial centroid and X t , X 2t , X 3t , X 4t , X 5t …X nt Calculate the cosine similarity matrix of the matrix, and take out the rows with similarity greater than the threshold in each matrix based on the threshold; Step 4.6: Combine the rows taken out in step 4.5 and the initial centroid and compress them into an updated centroid vector of 1*512; Step 4.7: Update the centroid and X t , X 2t , X 3t , X 4t , X 5t …X nt The cosine similarity matrix is ​​calculated based on the matrix, and the rows with similarity greater than the threshold are respectively taken out from each matrix; the speech segments corresponding to the rows taken out in this step are merged to obtain the operator's audio.

2. The single-channel speech separation method in a traffic environment according to claim 1, characterized in that: The step 2 segments the audio of each operator's call recording material based on the open source pyannote library.

3. A method for separating single-channel speech in a traffic environment according to any one of claims 1 or 2, characterized in that: The step 3 comprises the following steps: Step 3.1: Segment the segment segmented in step 2 again with a step length of t seconds; Step 3.2: Encode all the segments divided in step 3.1, encode each t seconds of audio segment into a 1*512 vector, and obtain the matrix corresponding to each audio segment.

4. The single-channel speech separation method in a traffic environment according to claim 3, characterized in that: Said t is equal to 1 second.

5. The single-channel speech separation method in a traffic environment according to claim 1, characterized in that: The K is equal to 0.

8.

6. The single-channel speech separation method in a traffic environment according to claim 1, characterized in that: The cosine similarity calculation formula is as follows: Where: A and B are both n-dimensional vectors, cos(θ) is a numerical value, indicating the similarity between vectors A and B, and its value range is 0-1.