A time-domain based third-party interjection voice filtering method

Abstract

The application provides a time-domain-based third-party interrupting voice filtering method, which comprises the following steps: a user inputs voice into a third-party interrupting voice filtering system; the third-party interrupting voice filtering system extracts a voiceprint feature vector of the user's voice and stores the voiceprint feature vector in a database of the third-party interrupting voice filtering system; the third-party interrupting voice filtering system receives a mixed voice which is combined by the user's voice and a voice not belonging to the user; and the third-party interrupting voice filtering system filters the voice not belonging to the user in the mixed voice according to the user's voiceprint feature to obtain a pure user voice. The filtering effect of the application is better than that of the SpEx+ model, and the application can meet the filtering performance and reasoning requirements of an intelligent voice customer service system.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and specifically to a time-domain-based method for filtering third-party interruption speech. Background Technology

[0002] Current mainstream speech filtering models, such as the SpEx+ model, have fast inference speeds, but their filtering performance is still unsatisfactory. The DPRNN-Spe-IRA model, an improvement on SpEx+, offers some improvement in filtering performance, but its inference speed is too slow due to the introduction of the BLSTM structure. Currently, intelligent voice customer service scenarios require fast inference speed and good filtering effects for third-party interruption speech. Therefore, existing technologies are difficult to implement in intelligent voice customer service systems. This invention combines the improved SpEx+ model with an iterative optimization mechanism (IRA) to propose a time-domain-based third-party interruption speech filtering method that can meet the filtering performance and inference requirements of intelligent voice customer service systems. This method combines the SpEx+ model framework, employing TCN as the base module in the speech filtering module to extract speech features, which improves inference speed compared to BLSTM. The voiceprint classifier is improved by using a time-delay neural network (TDNN) as the base module, extending the output of each hidden layer in the time domain. Compared to the ResNet network structure used in the original SpEx+ model's voiceprint classifier, the performance of the voiceprint classifier in extracting target speaker voiceprint features is further optimized. Finally, an iterative optimization mechanism (IRA) is used, allowing the voiceprint classifier and speech filtering module to continuously optimize and update each other, enhancing the filtering effect of the extracted target speech. Therefore, the third-party interruption speech filtering method proposed in this invention has better filtering effect than the SpEx+ model and faster inference speed than the DPRNN-Spe-IRA model, meeting the filtering performance and inference requirements of intelligent voice customer service systems. Summary of the Invention

[0003] The purpose of this invention is to provide a time-domain-based method for filtering third-party interruptions in speech. This method has better filtering performance than the SpEx+ model and can meet the filtering performance and inference requirements of intelligent voice customer service systems.

[0004] A time-domain-based method for filtering third-party interruptions in speech includes:

[0005] Users record their voice messages into a third-party interruption filtering system;

[0006] The third-party interruption voice filtering system extracts the voiceprint feature vector of the user's voice and stores it in the third-party interruption voice filtering system database.

[0007] The third-party interruption voice filtering system receives mixed voice messages, combining user voice messages with voice messages that do not belong to the user.

[0008] The third-party interruption voice filtering system filters out non-user voice from mixed speech based on the user's voiceprint characteristics, resulting in pure user voice.

[0009] The third-party interruption voice filtering system extracts the voiceprint feature vector of the user's voice and stores it in the third-party interruption voice filtering system database, including:

[0010] The third-party interruption voice filtering system converts the original time-domain signal of the user's voice into a high-dimensional voice signal.

[0011] The third-party interruption voice filtering system extracts features from high-dimensional voice signals to obtain the user's voiceprint feature vector;

[0012] The third-party interruption voice filtering system stores the user's voiceprint feature vector in the third-party interruption voice filtering system database.

[0013] The third-party interruption voice filtering system converts the raw time-domain signal of the user's recorded voice into a high-dimensional voice signal, including:

[0014] A one-dimensional convolutional network is used to reduce the model size, and the convolution stride is set to half the size of the convolution kernel. Let F be the feature output of the user's original time-domain speech signal x after passing through the encoder. encoder ;

[0015] F encoder =Conv K12C256 (x), where Conv K12C256 (*) indicates a one-dimensional convolutional network with a kernel size of 12 and 256 channels;

[0016] Mixed speech x mix User voice x aux The encoder yields a high-dimensional speech signal representation F. mix and F aux .

[0017] The third-party interruption speech filtering system extracts features from high-dimensional speech signals to obtain user voiceprint feature vectors, including:

[0018] The voiceprint feature extraction model includes: three time-delay neural networks (TDNN), one one-dimensional convolutional network (1D CNN), and two fully connected layers (Linear).

[0019] TDNN consists of a dilated convolutional network and an activation function, F. aux Shallow features are extracted using three cascaded TDNNs and a one-dimensional convolution, defined as follows:

[0020] F Sh allow =Conv K1C1500(TDNN K3D3C512 (TDNN K3D2C512 (TDNN K5DiC512 (F aux ))))

[0021] Among them, TDNN KxDyCz (·) indicates that the kernel size K of the dilated convolution in the module is x, the size of the hole D is y, and the number of channels C is z;

[0022] Calculate shallow features F with temporal relationships sh allow The statistical characteristics are obtained by calculating the mean and standard deviation for the time series dimension, and then concatenating the mean and standard deviation of the channel C to form a 2×C dimensional fusion feature, represented as follows:

[0023] F fuse =Cat[Mean(F Sh allow ),Std(F sh allow )]

[0024] Where Cat[·] represents the expression for F sh allow The statistical characteristics are concatenated, with Mean(·) representing the mean function and Std(·) representing the standard deviation function;

[0025] Feature F is integrated and fused through a fully connected layer. fuse Generate voiceprint features F voiceprint， Represented as F voiceprint =Linear 256 (F fuse )

[0026] Among them, Linear x This represents a fully connected layer with an output dimension of x;

[0027] The user's voiceprint features are mapped to a dimension consistent with the number of categories, num, representing the probability that a sample belongs to each category, denoted as:

[0028] Class = Linear num (F voiceprint )

[0029] The voiceprint classification network calculates the inference bias using the cross-entropy loss function, and then optimizes the model parameters through backpropagation. The cross-entropy loss is expressed as...

[0030]

[0031] Where N represents the number of samples, num represents the number of categories, and y ic p represents a sign function that takes the value 1 when the true class of sample i is c, and 0 otherwise; ic This represents the probability that sample i is assigned to class c.

[0032] The third-party interruption voice filtering system filters out non-user voice from mixed speech based on user voiceprint characteristics, resulting in pure user voice, including:

[0033] Voiceprint features F voiceprint With mixed speech features F mix The concatenation results in a dimension of (Seq, The feature map of the mixed audio, where Seq represents the sequence length of the mixed audio. This represents the number of feature channels after the mixed speech has been converted by the encoder. The dimension representing voiceprint features.

[0034] Dimension is The voiceprint features are copied Seq times and then become dimension (Seq, The voiceprint characteristics F voiceprint_copy ;

[0035] F mix and F voiceprint_copy The mixed features are obtained by concatenating them along the Seq dimension and then input into the TCN, represented as F1 = TCN(Cat[F mix F voiceprint_copy The resulting F1 is then compared with F. voiceprint_copy Concatenate the second TCN block into the input, and repeat three times;

[0036] After three TCN feature extractions, the feature representation F is obtained and input into a convolutional kernel of size 1. The number of convolutional kernels is equal to F. mix A mask F for mixed speech is obtained in a one-dimensional convolutional neural network with an equal number of channels. mask ;

[0037] Mask F mask It is a representation of mixed audio features F mix For matrices with uniform dimensions and values ​​greater than or equal to 0, use F... mask For F mix The noise component in the image is masked to obtain a clean audio feature representation F. clean , represented as

[0038]

[0039] in, This indicates that the elements are multiplied bitwise, and the mask in the formula represents the elimination or enhancement of features at corresponding positions in the mixed audio.

[0040] Speech filtering uses the scale-invariant signal-to-noise ratio (SI-SNR) of the audio as the loss, where the formula for calculating the scale-invariant signal-to-noise ratio is expressed as:

[0041]

[0042] s noise =s(t)-s target

[0043]

[0044] Where ρ(·) represents the signal-to-noise ratio of the predicted audio and the clean audio, and s(t) and s clean (t) represents the predicted audio signal and the clean audio signal, respectively. <·> denotes the summation of element-wise products, and ||·|| denotes the L2 norm. The model minimizes the loss. SI-SNR For the target, it is represented as

[0045] Loss SI-SNR =-ρ(s) target s noise ).

[0046] The third-party interruption speech filtering system extracts features from high-dimensional speech signals to obtain user voiceprint feature vectors. It also optimizes the voiceprint feature extraction model, specifically:

[0047] The F obtained from the first speech filtering clean The input is fed into the voiceprint extraction model to obtain the first new voiceprint feature F. new_voiceprint The voiceprint feature dimension and F voiceprint Consistent;

[0048] F clean and F new_voiceprint The two voiceprint features are concatenated end-to-end and then the feature dimensionality is reduced through a fully connected layer to obtain the sum F. voiceprint The second new voiceprint feature F with consistent dimensions better_voiceprint ;

[0049] The second new voiceprint feature F better_voiceprin t and mixed speech features F mix After concatenation again, features are extracted to obtain a clean audio feature representation F. better_clean .

[0050] The third-party interruption speech filtering system filters out non-user speech from mixed speech based on user voiceprint characteristics to obtain pure user speech. This process also includes decoding the pure user speech, specifically:

[0051] The decoder is composed of a one-dimensional deconvolutional network with a kernel size of 12 and a channel number of 5.

[0052] A one-dimensional deconvolutional network restores the N-dimensional masked signal of mixed speech to a clean 5-dimensional user speech signal.

[0053] A time-domain-based third-party interruption speech filtering system includes:

[0054] The voice receiving module is used for users to input their voice and to receive mixed voices that combine the user's voice with voices that do not belong to the user;

[0055] The feature extraction module is used to extract the voiceprint feature vector of the user's voice and store it in the database of the third-party interruption speech filtering system.

[0056] The feature filtering module is used to filter out non-user voiceprints in mixed speech based on the user's voiceprint features to obtain pure user speech.

[0057] A time-domain-based third-party interruption speech filtering device, comprising:

[0058] An encoder is used to convert received user speech signals and mixed speech signals into high-dimensional speech signals;

[0059] Voiceprint classifier is used to extract voiceprint features from user speech signals;

[0060] Filters are used to filter out speech that does not belong to the user in mixed speech;

[0061] A decoder is used to convert filtered speech into a low-dimensional speech signal.

[0062] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement any of the time-domain-based third-party interjection speech filtering methods described above.

[0063] The third-party interruption speech filtering method proposed in this invention has better filtering effect than the SpEx+ model and faster inference speed than the DPRNN-Spe-IRA model, which can meet the filtering performance and inference requirements of intelligent voice customer service system.

[0064] Based on practical needs, this paper proposes a time-domain-based third-party interruption speech filtering technology. When intelligent voice customer service performs speech recognition, it utilizes pre-recorded user voices from business consultants to filter out non-business consultant voices, providing a clean sound source for intelligent voice customer service, thereby improving speech recognition accuracy and increasing the efficiency of business consultation. Attached Figure Description

[0065] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0066] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0067] Figure 1 This is a flowchart of the present invention;

[0068] Figure 2 This is a diagram of the feature extraction model of the present invention. Detailed Implementation

[0069] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0070] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0071] Furthermore, the use of terms such as "first" and "second" in this invention is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" and "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. When the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed by this invention.

[0072] Example 1

[0073] A time-domain-based method for filtering third-party interruptions in speech includes:

[0074] S100, users can input their voice into the third-party interruption voice filtering system;

[0075] S200, the third-party interruption voice filtering system extracts the voiceprint feature vector of the user's voice and stores it in the third-party interruption voice filtering system database.

[0076] S300, the third-party interruption voice filtering system receives mixed voices that combine user voices with voices that do not belong to the user.

[0077] The S400 third-party interruption speech filtering system filters out non-user speech from mixed speech based on the user's voiceprint characteristics, resulting in pure user speech.

[0078] The S200 third-party interruption voice filtering system extracts the voiceprint feature vector of the user's voice and stores it in the third-party interruption voice filtering system database, including:

[0079] S201, the third-party interruption voice filtering system converts the original time-domain signal of the user's voice into a high-dimensional voice signal.

[0080] S202, the third-party interruption speech filtering system extracts features from high-dimensional speech signals to obtain user voiceprint feature vectors;

[0081] S203, the third-party interruption voice filtering system stores the user's voiceprint feature vector in the third-party interruption voice filtering system database.

[0082] User speech and mixed speech are processed separately by different encoders with shared weights. This is to eliminate the mismatch in latent feature spaces between the user speech encoder and the mixed speech encoder. The two speech encoders share the same network structure and weights, allowing user speech input and mixed speech input to be represented in a unified latent feature space, thus improving speech recognition performance. Unlike the SpEx+ model encoder, which uses three one-dimensional convolutional networks of different scales, this invention uses only a single one-dimensional convolutional network, reducing the model size. The convolution stride is set to half the size of the convolution kernel. Let F be the feature output of the original time-domain speech signal x after passing through the encoder. encoder Then F encoder =Conv K12C256 (x), where Conv K12C256 (*) indicates a one-dimensional convolutional network with a kernel size of 12 and 256 channels. (Mixed speech x) mix User voice x aux The encoder obtains their respective feature representations F. mix and F aux By utilizing a one-dimensional convolutional network, a one-dimensional time-domain signal is transformed into a two-dimensional feature representation, thus replacing the short-time Fourier transform. This high-dimensional audio spatial representation obtained through a data-driven encoder is more suitable for third-party speech filtering tasks.

[0083] The S201 third-party interruption voice filtering system converts the raw time-domain signal of the user-entered voice into a high-dimensional voice signal, including:

[0084] A one-dimensional convolutional network is used to reduce the model size, and the convolution stride is set to half the size of the convolution kernel. Let F be the feature output of the user's original time-domain speech signal x after passing through the encoder. encoder ;

[0085] F encoder =Conv K12C256 (x), where Conv K12C256 (*) indicates a one-dimensional convolutional network with a kernel size of 12 and 256 channels;

[0086] Mixed speech x mix User voice x aux The encoder yields a high-dimensional speech signal representation F. mix and F aux .

[0087] The S202 third-party interruption speech filtering system extracts features from high-dimensional speech signals to obtain user voiceprint feature vectors, including:

[0088] The voiceprint feature extraction model includes: three time-delay neural networks (TDNN), one one-dimensional convolutional network (1D CNN), and two fully connected layers (Linear).

[0089] TDNN consists of a dilated convolutional network and an activation function, F. aux Shallow features are extracted using three cascaded TDNNs and a one-dimensional convolution, defined as follows:

[0090] F Sh allow =Conv K1C1500 (TDNN K3D3C512 (TDNN K3D2C512 (TDNN K5D1C512 (F aux ))))

[0091] Among them, TDNN KxDyCz (·) indicates that the kernel size K of the dilated convolution in the module is x, the size of the hole D is y, and the number of channels C is z;

[0092] Calculate shallow features F with temporal relationships Sh allow The statistical characteristics are obtained by calculating the mean and standard deviation for the time series dimension, and then concatenating the mean and standard deviation of the channel C to form a 2×C dimensional fusion feature, represented as follows:

[0093] F fuse =Cat[Mean(F Sh allow ),Std(F Sh allow )]

[0094] Where Cat[·] represents the expression for F Sh allowThe statistical characteristics are concatenated, with Mean(·) representing the mean function and Std(·) representing the standard deviation function;

[0095] Feature F is integrated and fused through a fully connected layer. fuse Generate voiceprint features F voiceprint， Represented as F voiceprint =Linear 256 (F fuse )

[0096] Among them, Linear x This represents a fully connected layer with an output dimension of x;

[0097] The user's voiceprint features are mapped to a dimension consistent with the number of categories, num, representing the probability that a sample belongs to each category, denoted as:

[0098] Class = Linear num (F voiceprint )

[0099] The voiceprint classification network calculates the inference bias using the cross-entropy loss function, and then optimizes the model parameters through backpropagation. The cross-entropy loss is expressed as...

[0100]

[0101] Where N represents the number of samples, num represents the number of categories, and y ic p represents a sign function that takes the value 1 when the true class of sample i is c, and 0 otherwise; ic This represents the probability that sample i is assigned to class c.

[0102] The voiceprint classifier performs the task of speaker classification. Higher accuracy indicates more precise speaker classification and better voiceprint feature extraction. Each user's voice has a corresponding speaker label; therefore, the user voice feature representation F obtained after encoder conversion... aux The input is fed into a voiceprint classifier, which is trained using cross-entropy loss as the loss function. The trained voiceprint classifier uses the 256-dimensional feature vector output from the first fully connected layer as the voiceprint feature F. voiceprint This information is provided to the speech filtering module. Since the SpEx+ and DPRNN-Spe-IRA models use a residual network composed of one-dimensional convolutional networks as the classification network, this ordinary one-dimensional convolutional structure is not effective for extracting features from speech signals with temporal relationships. Therefore, the speaker classifier of this invention is constructed using three time-delay neural networks (TDNN), one one-dimensional convolutional network (1D CNN), and two fully connected layers (Linear) as its basic modules. The TDNN consists of a dilated convolutional network and an activation function, F... aux First, shallow features are extracted using three cascaded TDNNs and a one-dimensional convolution, defined as follows:

[0103] F Sh allow =Conv K1C1500 (TDNN K3D3C512 (TDNN K3D2C512 (TDNN K5D1C512 (F aux ))))

[0104] Among them, TDNN KxDyCz (·) indicates that the kernel K of the dilated convolution in the module has a size of x, the size of the hole D is y, and the number of channels C is z. Next, the shallow features F with temporal relationships are calculated. Sh allow The statistical characteristics are obtained by calculating the mean and standard deviation for the time series dimension, and then concatenating the mean and standard deviation of the channel C to form a 2×C dimensional fusion feature, represented as follows:

[0105] F fuse =Cat[Mean(F Sh allow ),Std(F Sh allow )]

[0106] Where Cat[·] represents the expression for F Sh allow The statistical features are concatenated, with Mean(·) representing the mean function and Std(·) representing the standard deviation function. Then, the features F are integrated and fused through a fully connected layer. fuse Generate voiceprint features F voiceprint , represented as

[0107] F voiceprint =Linear 256 (F fuse )

[0108] Among them, Linear x This represents a fully connected layer with output dimension x. Finally, the voiceprint features are mapped to a dimension consistent with the number of categories num, representing the probability that a sample belongs to each category, denoted as:

[0109] Class = Linear num (F voiceprint )

[0110] The voiceprint classification network calculates the inference bias using the cross-entropy loss function, and then optimizes the model parameters through backpropagation. The cross-entropy loss is expressed as...

[0111]

[0112] Where N represents the number of samples, num represents the number of categories, and y ic p represents a sign function that takes the value 1 when the true class of sample i is c, and 0 otherwise; icThis represents the probability that sample i is assigned to class c.

[0113] The S400 third-party interruption voice filtering system filters out non-user voice from mixed speech based on user voiceprint characteristics, resulting in pure user voice, including:

[0114] Voiceprint features F voiceprint With mixed speech features F mix The concatenation results in a dimension of (Seq, The feature map of the mixed audio, where Seq represents the sequence length of the mixed audio. This represents the number of feature channels after the mixed speech has been converted by the encoder. The dimension representing voiceprint features.

[0115] Dimension is The voiceprint features are copied Seq times and then become dimension (Seq, The voiceprint characteristics F voiceprint_copy ;

[0116] F mix and F voiceprint_copy The mixed features are obtained by concatenating them along the Seq dimension and then input into the TCN, represented as F1 = TCN(Cat[F mix F voiceprint_copy The resulting F1 is then compared with F. voiceprint_copy Concatenate the second TCN block into the input, and repeat three times;

[0117] After three TCN feature extractions, the feature representation F is obtained and input into a convolutional kernel of size 1. The number of convolutional kernels is equal to F. mix A mask F for mixed speech is obtained in a one-dimensional convolutional neural network with an equal number of channels. mask ;

[0118] Mask F mask It is a representation of mixed audio features F mix For matrices with uniform dimensions and values ​​greater than or equal to 0, use F... mask For F mix The noise component in the image is masked to obtain a clean audio feature representation F. clean , represented as

[0119]

[0120] in, This indicates that the elements are multiplied bitwise, and the mask in the formula represents the elimination or enhancement of features at corresponding positions in the mixed audio.

[0121] Speech filtering uses the scale-invariant signal-to-noise ratio (SI-SNR) of the audio as the loss, where the formula for calculating the scale-invariant signal-to-noise ratio is expressed as:

[0122]

[0123] s noise =s(t)-s target

[0124]

[0125] Where ρ(·) represents the signal-to-noise ratio of the predicted audio and the clean audio, and s(t) and s clean (t) represents the predicted audio signal and the clean audio signal, respectively. <·> denotes the summation of element-wise products, and ||·|| denotes the L2 norm. The model minimizes the loss. SI-SNR For the target, it is represented as

[0126] Loss SI-SNR =-ρ(s) target s noise ).

[0127] The S202 third-party interruption speech filtering system extracts features from high-dimensional speech signals to obtain user voiceprint feature vectors. It then includes S2021, which optimizes the voiceprint feature extraction model, specifically as follows:

[0128] The F obtained from the first speech filtering clean The input is fed into the voiceprint extraction model to obtain the first new voiceprint feature F. new_voiceprint The voiceprint feature dimension and F voiceprint Consistent;

[0129] F clean and F new_voiceprint The two voiceprint features are concatenated end-to-end and then the feature dimensionality is reduced through a fully connected layer to obtain the sum F. voiceprint The second new voiceprint feature F with consistent dimensions better_voiceprint ;

[0130] The second new voiceprint feature F better_voiceprint and mixed speech features F mix After concatenation again, features are extracted to obtain a clean audio feature representation F. better_clean .

[0131] This invention introduces an iterative optimization mechanism, IRA, which significantly improves model performance without substantially increasing model size. Research shows that when people focus on a particular person's voice, their focus increases over time. Inspired by auditory perception, the IRA mechanism learns a particular speaker's voice better as people become more familiar with them during conversations or listening. The more iterations, the better the extracted target speech.

[0132] The S400 third-party interruption speech filtering system filters out non-user speech from mixed speech based on user voiceprint characteristics to obtain clean user speech. Afterwards, the S500 further decodes the clean user speech, specifically:

[0133] The decoder is composed of a one-dimensional deconvolutional network with a kernel size of 12 and a channel number of 5.

[0134] A one-dimensional deconvolutional network restores the N-dimensional masked signal of mixed speech to a clean 5-dimensional user speech signal.

[0135] The decompressor converts the input speech feature representation into a speech time-domain signal. The decoder's structure and function are symmetrical to the encoder's. It consists of a one-dimensional deconvolutional network with a kernel size of 12 and 5 channels, used for time-domain signal reconstruction. The one-dimensional deconvolutional network restores the N-dimensional masked signal after speech filtering into a clean 5-dimensional speech signal.

[0136] The speech filtering system is evaluated. The dataset consists of clean, single-channel, single-speaker audio, artificially synthesized to suit model training, with a sampling rate of 8kHz. Two randomly selected speakers, A1 and B1, are used as labels for model training, and a mixed speech M is synthesized. Two more speakers, A2 and B2, are then selected as the user's speech. The model trains the speaker coding module using the user's speech to improve speaker classification accuracy; the classification loss is calculated using the cross-entropy loss function. The speech filtering module filters third-party speech using speaker features, and the performance of the resulting inference audio is measured using SI-SNR loss. This invention employs a joint training method, jointly training the speaker classification network and the speech filtering network, jointly optimizing the two tasks. The joint loss is represented by...

[0137]

[0138] in, This represents the weighting factor.

[0139] Evaluation indicators

[0140] The filtering results are evaluated using the scale-invariant signal-to-noise ratio (SI-SNR), with a higher value being better. SI-SNR is defined as follows:

[0141]

[0142] In the formula: x represents the estimated signal and the original clean signal, respectively, and ||x|| =<x,x> To represent signal power, and to ensure scale invariance, it is necessary to... With zero mean for x.

[0143] Experimental results

[0144] The experimental results are shown in Table 1. From Table 1, we can conclude that:

[0145] 1) Based on the SI-SNR index results, the method proposed in this invention outperforms the SpEx+ method by 0.18dB, but is lower than the DPRNN-Spe-IRA method, and can meet the performance requirements for third-party interruption filtering in intelligent voice customer service.

[0146] 2) In terms of inference speed, the method proposed in this invention is faster than the DPRNN-Spe-IRA method, with an inference time that is nearly 32.71% faster, but slower than the SpEx+ method, and can meet the time requirements for filtering third-party interruptions in intelligent voice customer service.

[0147] Table 1 Comparison of Filtering Results by Method

[0148] SpEx+ 11.54 8ms DPRNN-Spe-IRA 11.95 21.7ms The method proposed in this invention 11.72 14.6ms

[0149] Example 2

[0150] A time-domain-based third-party interruption speech filtering system includes:

[0151] The voice receiving module is used for users to input their voice and to receive mixed voices that combine the user's voice with voices that do not belong to the user;

[0152] The feature extraction module is used to extract the voiceprint feature vector of the user's voice and store it in the database of the third-party interruption speech filtering system.

[0153] The feature filtering module is used to filter out non-user voiceprints in mixed speech based on the user's voiceprint features to obtain pure user speech.

[0154] Example 3

[0155] A time-domain-based third-party interruption speech filtering device, characterized in that it comprises:

[0156] An encoder is used to convert received user speech signals and mixed speech signals into high-dimensional speech signals;

[0157] Voiceprint classifier is used to extract voiceprint features from user speech signals;

[0158] Filters are used to filter out speech that does not belong to the user in mixed speech;

[0159] A decoder is used to convert filtered speech into a low-dimensional speech signal.

[0160] Example 4

[0161] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement any of the time-domain-based third-party interjection speech filtering methods described above.

[0162] The third-party interruption speech filtering method proposed in this invention has better filtering effect than the SpEx+ model and faster inference speed than the DPRNN-Spe-IRA model, which can meet the filtering performance and inference requirements of intelligent voice customer service system.

[0163] Based on practical needs, this paper proposes a time-domain-based third-party interruption speech filtering technology. When intelligent voice customer service performs speech recognition, it utilizes pre-recorded user voices from business consultants to filter out non-business consultant voices, providing a clean sound source for intelligent voice customer service, thereby improving speech recognition accuracy and increasing the efficiency of business consultation.

[0164] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A time-domain-based method for filtering third-party interruptions in speech, characterized in that, include: Users record their voice messages into a third-party interruption filtering system; The third-party interruption voice filtering system extracts the voiceprint feature vector of the user's voice and stores it in the third-party interruption voice filtering system database. The third-party interruption voice filtering system receives mixed voice messages, combining user voice messages with voice messages that do not belong to the user. The third-party interruption voice filtering system filters out non-user voice from mixed speech based on the user's voiceprint characteristics to obtain pure user voice. The third-party interruption speech filtering system filters out non-user speech from mixed speech based on user voiceprint characteristics to obtain pure user speech, including: Voiceprint features With mixed speech features The dimensions of the assembled components are: Feature map, Indicates the sequence length of the mixed audio. This represents the number of feature channels after the mixed speech has been converted by the encoder. The dimension representing voiceprint features. Dimension is The voiceprint features are copied Seq times and then become dimensional. Voiceprint characteristics ; Will and The combined features are obtained by concatenating along the Seq dimension and then input into the TCN, represented as follows: , obtained Again with Concatenate the second TCN block into the input, and repeat three times; After three TCN feature extractions, the feature representation is obtained. The input is a convolution kernel of size 1, and the number of convolution kernels is... A mask for mixed speech is obtained in a one-dimensional convolutional neural network with an equal number of channels. ; mask It is a representation of mixed audio features A matrix with consistent dimensions and values ​​greater than or equal to 0, using... right The noise portion in the image is masked to obtain a clean audio feature representation. , represented as ； in, This indicates that the elements are multiplied bitwise, and the mask in the formula represents the elimination or enhancement of features at corresponding positions in the mixed audio. Speech filtering uses the scale-invariant signal-to-noise ratio (SI-SNR) of the audio as the loss, where the formula for calculating the scale-invariant signal-to-noise ratio is expressed as: ； ； ； in, This represents the signal-to-noise ratio of the predicted audio and the clean audio. These represent the predicted audio signal and the clean audio signal, respectively. This represents the sum of the elements by multiplying them digit by digit. Describing the L2 norm, the model minimizes... For the target, it is represented as 。 2. The time-domain-based third-party speech filtering method according to claim 1, characterized in that, The third-party interruption voice filtering system extracts the voiceprint feature vector of the user's voice and stores it in the third-party interruption voice filtering system database, including: The third-party interruption voice filtering system converts the original time-domain signal of the user's voice into a high-dimensional voice signal. The third-party interruption voice filtering system extracts features from high-dimensional voice signals to obtain the user's voiceprint feature vector; The third-party interruption voice filtering system stores the user's voiceprint feature vector in the third-party interruption voice filtering system database.

3. The time-domain-based third-party speech filtering method according to claim 2, characterized in that, The third-party interruption speech filtering system converts the original time-domain signal of the user-recorded speech into a high-dimensional speech signal, including: A one-dimensional convolutional network is used to reduce the model size, and the convolution stride is set to half the size of the convolution kernel. The original time-domain signal of the user's speech is assumed to be... The features output by the encoder are ; ,in This represents a one-dimensional convolutional network with a kernel size of 12 and 256 channels. Mixed speech and user voice The encoder produces a high-dimensional speech signal representation. and .

4. The time-domain-based third-party speech filtering method according to claim 2, characterized in that, The third-party interruption speech filtering system extracts features from the high-dimensional speech signal to obtain the user's voiceprint feature vector, including: The voiceprint feature extraction model includes: three time-delay neural networks (TDNN), one one-dimensional convolutional network (1D CNN), and two fully connected layers (Linear). TDNN consists of a dilated convolutional network and activation functions. Shallow features are extracted using three cascaded TDNNs and a one-dimensional convolution, defined as follows: ； in, Let x be the size of the kernel K of the dilated convolution in the module, y be the size of the hole D, and z be the number of channels C. Calculate shallow features with temporal relationships The statistical characteristics are obtained by calculating the mean and standard deviation for the time series dimension, and then concatenating the mean and standard deviation of the channel C to form a 2×C dimensional fusion feature, represented as follows: ； in, Indicates to Statistical features spliced ​​together This represents the mean function. This represents the function for calculating the standard deviation. Integrating and fusing features through a fully connected layer Generate voiceprint features , represented as ； in, This represents a fully connected layer with an output dimension of x; Map user voiceprint features to the number of categories A consistent dimension represents the probability that a sample belongs to each category, denoted as... ； The voiceprint classification network calculates the inference bias using the cross-entropy loss function, and then optimizes the model parameters through backpropagation. The cross-entropy loss is expressed as... ； Where N represents the number of samples, Indicates the number of categories. Represent a symbolic function, when the sample The real category is The value is 1 if the condition is met, otherwise it is 0. Indicates sample Lost by category The probability of.

5. A time-domain-based third-party speech filtering method according to claim 4, characterized in that, The third-party interruption speech filtering system extracts features from the high-dimensional speech signal to obtain the user's voiceprint feature vector. It also optimizes the voiceprint feature extraction model, specifically: The first result of voice filtering The input is fed into the voiceprint extraction model to obtain the first new voiceprint feature. The voiceprint feature dimensions and Consistent; Will and The two voiceprint features are concatenated end-to-end and then the feature dimensionality is reduced through a fully connected layer to obtain the sum. Second new voiceprint features with consistent dimensions ; The second new voiceprint feature and mixed speech features After concatenation again, features are extracted to obtain a clean audio feature representation. .

6. The time-domain-based third-party speech filtering method according to claim 1, characterized in that, The third-party interruption speech filtering system, after filtering out non-user speech from mixed speech based on user voiceprint features to obtain pure user speech, also includes decoding the pure user speech, specifically: The decoder is composed of a one-dimensional deconvolutional network with a kernel size of 12 and a channel number of 5. A one-dimensional deconvolutional network restores the N-dimensional masked signal of mixed speech to a clean 5-dimensional user speech signal.

7. A time-domain-based third-party interruption speech filtering system, characterized in that, include: The voice receiving module is used for users to input their voice and to receive mixed voices that combine the user's voice with voices that do not belong to the user; The feature extraction module is used to extract the voiceprint feature vector of the user's voice and store it in the database of the third-party interruption speech filtering system. The feature filtering module is used to filter out non-user voiceprints in mixed speech based on the user's voiceprint features to obtain pure user speech; The third-party interruption speech filtering system filters out non-user speech from mixed speech based on user voiceprint characteristics to obtain pure user speech, including: Voiceprint features With mixed speech features The dimensions of the assembled components are: Feature map, Indicates the sequence length of the mixed audio. This represents the number of feature channels after the mixed speech has been converted by the encoder. The dimension representing voiceprint features. Dimension is The voiceprint features are copied Seq times and then become dimensional. Voiceprint characteristics ; Will and The combined features are obtained by concatenating along the Seq dimension and then input into the TCN, represented as follows: , obtained Again with Concatenate the second TCN block into the input, and repeat three times; After three TCN feature extractions, the feature representation is obtained. The input is a convolution kernel of size 1, and the number of convolution kernels is... A mask for mixed speech is obtained in a one-dimensional convolutional neural network with an equal number of channels. ; mask It is a representation of mixed audio features A matrix with consistent dimensions and values ​​greater than or equal to 0, using... right The noise portion in the image is masked to obtain a clean audio feature representation. , represented as ； in, This indicates that the elements are multiplied bitwise, and the mask in the formula represents the elimination or enhancement of features at corresponding positions in the mixed audio. Speech filtering uses the scale-invariant signal-to-noise ratio (SI-SNR) of the audio as the loss, where the formula for calculating the scale-invariant signal-to-noise ratio is expressed as: ； ； ； in, This represents the signal-to-noise ratio of the predicted audio and the clean audio. These represent the predicted audio signal and the clean audio signal, respectively. This represents the sum of the elements by multiplying them digit by digit. Describing the L2 norm, the model minimizes... For the target, it is represented as 。 8. A time-domain-based third-party interruption speech filtering device, characterized in that, include: An encoder is used to convert received user speech signals and mixed speech signals into high-dimensional speech signals; Voiceprint classifier is used to extract voiceprint features from user speech signals; Filters are used to filter out speech that does not belong to the user in mixed speech; A decoder is used to convert filtered speech into low-dimensional speech signals. The third-party interruption voice filtering system filters out non-user voice from mixed speech based on user voiceprint characteristics, resulting in pure user voice, including: Voiceprint features With mixed speech features The dimensions of the assembled components are: Feature map, Indicates the sequence length of the mixed audio. This represents the number of feature channels after the mixed speech has been converted by the encoder. The dimension representing voiceprint features. Dimension is The voiceprint features are copied Seq times and then become dimensional. Voiceprint characteristics ; Will and The combined features are obtained by concatenating along the Seq dimension and then input into the TCN, represented as follows: , obtained Again with Concatenate the second TCN block into the input, and repeat three times; After three TCN feature extractions, the feature representation is obtained. The input is a convolution kernel of size 1, and the number of convolution kernels is... A mask for mixed speech is obtained in a one-dimensional convolutional neural network with an equal number of channels. ; mask It is a representation of mixed audio features A matrix with consistent dimensions and values ​​greater than or equal to 0, using... right The noise portion in the image is masked to obtain a clean audio feature representation. , represented as ； in, This indicates that the elements are multiplied bitwise, and the mask in the formula represents the elimination or enhancement of features at corresponding positions in the mixed audio. Speech filtering uses the scale-invariant signal-to-noise ratio (SI-SNR) of the audio as the loss, where the formula for calculating the scale-invariant signal-to-noise ratio is expressed as: ； ； ； in, This represents the signal-to-noise ratio of the predicted audio and the clean audio. These represent the predicted audio signal and the clean audio signal, respectively. This represents the sum of the elements by multiplying them digit by digit. Describing the L2 norm, the model minimizes... For the target, it is represented as 。 9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements a time-domain-based third-party interruption speech filtering method according to any one of claims 1 to 6.

Images