Noise elimination method, device, electronic device and storage medium for voice call
Through voice endpoint detection, feature extraction and clustering scoring technology, the background voice in voice calls are separated and deleted, which solves the problem of poor background voice noise cancellation in the existing technology, and improves the quality of voice calls and the success of the dialogue system. Rate.
Patent Information
- Application Number
- CN202010570483.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-06-19
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2040-06-19
AI Technical Summary
The existing noise cancellation technology has poor effect on background human noise noise cancellation, resulting in poor voice calls and affecting the success rate of the dialogue system.
Through voice endpoint detection, voice feature extraction, clustering processing and scoring algorithms, the first speaking voice and the second speaking voice are separated, and the background voice is judged and deleted based on the duration to improve the quality of voice calls.
Effectively remove background vocals in voice calls, improve the success rate and voice call quality of the dialogue system, and save and dynamically update the audio characteristics of speakers and background vocals in real time.
Smart Images

Figure CN111754982B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of voiceprint recognition, and in particular to a method, device, electronic device and computer-readable storage medium for eliminating noise in a voice call. Background Art
[0002] Customer service systems, especially smart outbound call systems, often have to deal with background noise interference in the customer's environment. Among all noises, the background human voice noise interference is the strongest. The automatic speech recognition of the smart outbound call system will also recognize the background human voice and treat it as the target of the conversation, which greatly affects the success rate of the overall conversation.
[0003] However, current noise cancellation technology mainly eliminates non-human background noise, but has a poor effect on eliminating background human voice noise, resulting in poor voice call quality. Summary of the invention
[0004] The present invention provides a method, device, electronic device and computer-readable storage medium for noise elimination in voice calls, the main purpose of which is to delete background human voices in voice calls and improve the success rate of the dialogue system.
[0005] To achieve the above object, the present invention provides a method for eliminating noise in a voice call, comprising:
[0006] Perform voice endpoint detection on the call audio to obtain a human voice set;
[0007] Extracting speech features from the human voice set to obtain a speech feature set;
[0008] Extracting a speech feature set to be detected whose cumulative duration is a preset duration threshold from the speech feature set in chronological order to obtain a plurality of speech feature sets to be detected, clustering each of the speech feature sets to be detected, and scoring the obtained clustering results using a preset evaluation algorithm to obtain a scoring value for each of the speech feature sets to be detected;
[0009] According to the score value, the human voice set is divided into a first speaking voice and a second speaking voice;
[0010] The duration of the first speaking voice and the second speaking voice is calculated, and the background voice in the voice speech set is determined according to the duration of the first speaking voice and the second speaking voice, and the background voice is deleted from the voice speech set.
[0011] Optionally, extracting speech features from the human voice set to obtain a speech feature set includes:
[0012] Pre-emphasize, frame and window the human voice set to obtain a voice frame sequence;
[0013] For each frame of speech in the speech frame sequence, a corresponding spectrum is obtained by fast Fourier transform;
[0014] Converting the spectrum into a Mel spectrum by a Mel filter bank;
[0015] Cepstrum analysis is performed on the Mel spectrum to obtain a speech feature set corresponding to the human voice set.
[0016] Optionally, the clustering process for each of the speech feature sets to be detected includes:
[0017] Step a, randomly selecting two feature vectors from the speech feature set to be detected as category centers;
[0018] Step b, for each feature vector in the speech feature set to be detected, by calculating the distance from the feature vector to each of the category centers, clustering the feature vector with the category center closest to it to obtain two initial categories;
[0019] Step c, updating the category centers of the two initial categories;
[0020] Step d: repeat the above steps b and c until the number of iterations reaches a preset threshold, and two standard categories are obtained.
[0021] Optionally, dividing the human voice set into a first speaking voice and a second speaking voice according to the score value comprises:
[0022] Select one of the speech feature sets to be detected and obtain the corresponding score value;
[0023] Comparing the score value with a preset score threshold;
[0024] When the score value is greater than a preset score threshold, merging the two standard categories of the selected speech feature set to be detected into a single speech category, calculating the category center of the single speech category, and generating a first speaking voice according to the single speech category and the category center;
[0025] When the score value is less than or equal to a preset score threshold, generating a first speaking voice and a second speaking voice according to the two standard categories;
[0026] A next speech feature set to be detected is selected, a corresponding score value is obtained, and two standard categories in the speech feature set to be detected are classified into the first speaker's voice or the second speaker's voice according to the score value.
[0027] Optionally, the step of classifying the two standard categories in the to-be-detected speech feature set into the first speaking voice or the second speaking voice according to the score value comprises:
[0028] If the score value is greater than the score threshold, merging the two standard categories of the speech feature set to be detected into a single speech category, calculating the category center of the single speech category, and classifying the single speech category into the first speaker or the second speaker according to the cosine distance between the category center of the single speech category and the category centers of the first speaker and the second speaker;
[0029] If the score value is less than or equal to the score threshold, the two standard categories are respectively classified into the first speaker and the second speaker according to the cosine distance between the category centers of the two standard categories in the speech feature set to be detected and the category centers of the first speaker and the second speaker.
[0030] Optionally, the classification includes:
[0031] Merging the single speech category with the first speaker or the second speaker, and recalculating the merged category center, and accumulating the number of frames of the single speech category with the duration of the first speaker or the second speaker; or
[0032] The two standard categories are merged with the first speaker and the second speaker respectively, and the merged category center is recalculated, and the number of frames of the standard category is accumulated with the duration of the first speaker or the second speaker.
[0033] Optionally, deleting the background human voice from the human voice set includes:
[0034] Calculate the duration ratio of the background voice in the current call using a preset duration algorithm;
[0035] Comparing the duration ratio with a preset ratio threshold;
[0036] When the duration ratio is greater than the ratio threshold, the background human voice is deleted from the human voice set, and the background human voice in the call audio is removed.
[0037] In order to solve the above problem, the present invention also provides a noise elimination device for voice calls, the device comprising:
[0038] The voice endpoint detection module is used to perform voice endpoint detection on the call audio to obtain a human voice set;
[0039] A speech feature extraction module, used for extracting speech features from the human voice set to obtain a speech feature set;
[0040] A cluster scoring module is used to intercept the speech feature set to be detected whose cumulative duration is a preset duration threshold from the speech feature set in chronological order, obtain multiple speech feature sets to be detected, perform clustering processing on each of the speech feature sets to be detected, and score the obtained clustering results using a preset evaluation algorithm to obtain a score value for each of the speech feature sets to be detected;
[0041] A human voice classification module, used for classifying the human voice set into a first speaking human voice and a second speaking human voice according to the score value;
[0042] The background vocal removal module is used to calculate the duration of the first speaking voice and the second speaking voice, and determine the background vocals in the vocal speech set according to the duration of the first speaking voice and the second speaking voice, and delete the background vocals from the vocal speech set.
[0043] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:
[0044] a memory storing at least one instruction; and
[0045] The processor executes the instructions stored in the memory to implement any one of the above-mentioned methods for eliminating noise in voice calls.
[0046] In order to solve the above problems, the present invention also provides a computer-readable storage medium, including a data storage area and a program storage area, the data storage area stores the created data, and the program storage area stores a computer program, and when the computer program is executed by a processor, it implements any one of the above-mentioned methods for noise elimination of voice calls.
[0047] The embodiment of the present invention performs voice endpoint detection on call audio, deletes non-human voice noise in the call audio, and reduces the subsequent processing amount of the computer; performs voice feature extraction on the human voice speech set to obtain a voice feature set, which is convenient for separating the background human voice in the call audio in the subsequent process; extracts the voice feature set to be detected with a cumulative duration of a preset duration threshold from the voice feature set in chronological order to obtain multiple voice feature sets to be detected, performs clustering processing on each of the voice feature sets to be detected, and scores the obtained clustering results using a preset evaluation algorithm to obtain a score value for each of the voice feature sets to be detected. The method of combining clustering with scoring can detect fragmented, blurred, and low-volume background human voices; divides the human voice speech set into a first speaker and a second speaker according to the score value, and can save and dynamically update the audio features of the speaker and the background human voice in real time; calculates the duration of the first speaker and the second speaker, and determines the background human voice in the human voice speech set according to the duration of the first speaker and the second speaker, and deletes the background human voice from the human voice speech set to improve the voice call quality. Therefore, the method, device and computer-readable storage medium for voice call noise elimination proposed in the present invention can delete the background human voice in the voice call and improve the success rate of the dialogue system. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 A flowchart of a method for eliminating noise in a voice call provided by an embodiment of the present invention;
[0049] Figure 2 A flowchart of a speech feature extraction method provided by an embodiment of the present invention;
[0050] Figure 3 A schematic diagram of a flow chart of a method for voice separation provided by an embodiment of the present invention;
[0051] Figure 4 A schematic diagram of a module of a noise elimination device for voice calls provided by an embodiment of the present invention;
[0052] Figure 5 A schematic diagram of the internal structure of an electronic device for implementing a method for eliminating noise during voice calls provided by an embodiment of the present invention;
[0053] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0054] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0055] The execution subject of the noise elimination method for voice calls provided in the embodiment of the present application includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided in the embodiment of the present application. In other words, the noise elimination method for voice calls can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc.
[0056] The present invention provides a method for eliminating noise in voice calls. Figure 1 , which is a flow chart of a method for eliminating noise in a voice call provided by an embodiment of the present invention.
[0057] In this embodiment, the method for eliminating noise during a voice call includes:
[0058] S1. Perform voice endpoint detection on the call audio to obtain a human voice set.
[0059] In detail, the call audio in the embodiment of the present invention includes the audio generated by a conversation in a crowd or in an environment with a lot of human voices, such as the call audio generated when a conversation is made through a communication system such as a telephone or instant messaging software in an environment full of background human voices. The call audio can be directly obtained from the communication system, or called and obtained from a database for storing voice conversation information. It should be emphasized that in order to further ensure the privacy and security of the above call audio, the above call audio can also be stored in a node of a blockchain.
[0060] The voice endpoint detection refers to distinguishing the human voice data and non-human voice data (silence and environmental noise) in the call audio in an environment with noise or other interference, and determining the starting point and ending point of the human voice data to delete the non-human voice noise in the call audio, reduce the subsequent processing volume of the computer, improve efficiency, and provide necessary support for subsequent signal processing.
[0061] In a preferred embodiment of the present invention, the voice endpoint detection model may be a voice activity detection (VAD) model based on a deep neural network (DNN).
[0062] S2. Extracting speech features from the human voice set to obtain a speech feature set.
[0063] For details, see Figure 2 As shown, the S2 includes:
[0064] S21, pre-emphasize, frame and window the human voice set to obtain a voice frame sequence;
[0065] Among them, the pre-emphasis is to use a high-pass filter to enhance the high-frequency part of the speech signal in the human voice set, so that the spectrum of the speech signal becomes flat; the framing is to use a movable finite-length window for weighting to divide the speech signal into several short segments, so that the speech signal has stability; the windowing is to make the non-periodic speech signal present some characteristics of the periodic function, so as to facilitate the subsequent Fourier expansion.
[0066] S22, for each frame of speech in the speech frame sequence, obtaining a corresponding spectrum by fast Fourier transform;
[0067] Preferably, since it is usually difficult to see the characteristics of the speech signal from the transformation of the speech signal in the time domain, the speech signal is usually converted into energy distribution in the frequency domain. Different energy distributions can represent the characteristics of different speech.
[0068] S23, converting the spectrum into a Mel spectrum through a Mel filter bank;
[0069] The Mel filter bank is a group of Mel-scale triangular filter banks, through which the spectrum can be converted into a Mel spectrum, and the Mel frequency can accurately reflect the auditory characteristics of the human ear.
[0070] S24, performing cepstrum analysis on the mel spectrum to obtain a speech feature set corresponding to the human voice set.
[0071] Furthermore, the cepstrum analysis includes taking logarithms and discrete cosine changes and outputting feature vectors. The speech feature set includes feature vectors corresponding to the speech frame sequence output after cepstrum analysis.
[0072] S3. Extract, in chronological order, from the speech feature set a speech feature set to be detected whose cumulative duration is a preset duration threshold, to obtain a plurality of speech feature sets to be detected, perform clustering processing on each of the speech feature sets to be detected, and score the obtained clustering results using a preset evaluation algorithm to obtain a score value for each of the speech feature sets to be detected.
[0073] In the embodiment of the present invention, when the accumulated duration of the speech feature set reaches a preset duration threshold, a detection calculation is performed, and the speech feature set accumulated this time is referred to as a speech feature set to be detected.
[0074] In detail, the clustering process for each of the speech feature sets to be detected includes:
[0075] Step a, randomly selecting two feature vectors from the speech feature set to be detected as category centers;
[0076] Step b: for each feature vector in the speech feature set to be detected, by calculating its distance to each of the category centers, clustering the feature vector with the category center closest to it to obtain two initial categories.
[0077] In detail, the embodiment of the present invention uses the following distance algorithm to calculate the distance between the feature vector and each of the category centers:
[0078]
[0079] Where L(X,Y) is the distance value, X is the category center, Y i is the feature vector in the speech feature set to be detected.
[0080] Step c, updating the category centers of the two initial categories;
[0081] Preferably, the embodiment of the present invention calculates the mean of all feature vectors under each of the initial categories, and updates the mean as the category center of the category.
[0082] Step d: repeat the above steps b and c until the number of iterations reaches a preset threshold, and two standard categories are obtained.
[0083] Furthermore, the present invention uses a preset evaluation algorithm to score the two standard categories obtained to obtain the score values of the standard categories. Preferably, in an embodiment of the present invention, the evaluation algorithm is as follows:
[0084]
[0085] Among them, n1 and n2 are the category centers of two standard categories, respectively, and H s is the assumption that the standard categories belong to the same category, H d is the hypothesis that the standard category belongs to different categories; P(n1,n2|H s ) is the likelihood function that n1 and n2 come from the same space; P(n1|H d ), P(n2|H d ) are the likelihood functions of n1 and n2 from different spaces. The likelihood function is a function of the statistical model parameters, which is used to test whether a hypothesis is valid.
[0086] Preferably, the higher the score value is, the greater the possibility that the speech corresponding to the two standard categories belongs to the same speaker; the lower the score value is, the smaller the possibility that the speech corresponding to the two standard categories belongs to the same speaker.
[0087] S4. Divide the human voice set into a first speaking voice and a second speaking voice according to the score value.
[0088] For details, see Figure 3 As shown, the S4 includes:
[0089] S40, selecting one of the speech feature sets to be detected and obtaining its corresponding score value;
[0090] S41, comparing the score value with a preset score threshold;
[0091] When the score value is greater than a preset score threshold, executing S42, merging the two standard categories of the selected speech feature set to be detected into a single speech category, calculating the category center of the single speech category, and generating a first speaking voice according to the single speech category and the category center;
[0092] The first speaking voice includes speech features and duration, the speech features include the single speech category and the category center, and the duration includes the number of frames of the single speech category.
[0093] When the score value is less than or equal to a preset score threshold, executing S43, generating a first speaking voice and a second speaking voice according to the two standard categories;
[0094] Similarly, the first speaker's voice and the second speaker's voice include speech features and durations, the speech features include the standard category and the category center, and the duration includes the number of frames of the standard category.
[0095] S44, selecting a next speech feature set to be detected, obtaining its corresponding score value, and classifying the two standard categories in the speech feature set to be detected into the first speaker's voice or the second speaker's voice according to the score value;
[0096] S45, judging whether each of the speech feature sets to be detected has been selected, and repeating S44 until each of the speech feature sets to be detected has been selected, thereby obtaining the first speaker's voice and the second speaker's voice.
[0097] In detail, the step of classifying the two standard categories in the to-be-detected speech feature set into the first speaking voice or the second speaking voice includes:
[0098] In one of the embodiments of the present invention, if the score value of the speech feature set to be detected is greater than a score threshold, the two standard categories of the speech feature set to be detected are merged into a single speech category, and the category center of the single speech category is calculated, and the cosine distance between the category center of the single speech category and the category centers of the first speaker voice and the second speaker voice is calculated, and based on the cosine distance, the single speech category is classified into the first speaker voice or the second speaker voice.
[0099] Among them, if the cosine distance between the category center of the single voice category and the category center of the above-mentioned first speaker's voice is closer, the single voice category is classified into the first speaker's voice; if the cosine distance between the category center of the single voice category and the category center of the above-mentioned second speaker's voice is closer, the single voice category is classified into the second speaker's voice.
[0100] The classification includes: merging the single voice category with the first speaker's voice or the second speaker's voice, and recalculating the merged category center; and accumulating the number of frames of the single voice category and the duration of the first speaker's voice or the second speaker's voice.
[0101] In another embodiment of the present invention, if the score value is less than or equal to the score threshold, the cosine distance between the category center of each standard category in the speech feature set to be detected and the category center of the first speaker and the second speaker is calculated, and the two standard categories are respectively classified into the first speaker and the second speaker according to the cosine distance.
[0102] Among them, assuming that the speech feature set to be detected includes a standard category A and a standard category B, if the cosine distance between the category center of the standard category A and the category center of the first speaker's voice is closer and the cosine distance between the category center of the standard category B and the category center of the second speaker's voice is closer, then the standard category A is classified into the first speaker's voice and the standard category B is classified into the second speaker's voice.
[0103] Similarly, the classification includes: merging the standard category A and the standard category with the first speaker or the second speaker respectively, and recalculating the merged category center; and accumulating the number of frames of the standard category A and the standard category B with the duration of the first speaker or the second speaker.
[0104] S5. Calculate the duration of the first speaking voice and the second speaking voice, and determine the background voice in the voice collection according to the duration of the first speaking voice and the second speaking voice, and delete the background voice from the voice collection.
[0105] Preferably, in the call audio, the duration of the target speaker is generally longer than that of the background speaker. Therefore, the embodiment of the present invention uses the speaker with the longer duration between the first speaker and the second speaker as the target speaker, and uses the remaining speakers as background speakers.
[0106] In detail, deleting the background human voice from the human voice set includes:
[0107] Calculate the duration ratio of the background voice in the current call using a preset duration algorithm;
[0108] Comparing the duration ratio with a preset ratio threshold;
[0109] When the duration ratio is greater than the ratio threshold, the background human voice is deleted from the human voice set, and the background human voice in the call audio is removed.
[0110] The duration algorithm is as follows:
[0111] R=t / T
[0112] Among them, R is the proportion of the background voice in this call, t is the duration of the background speaker, and T is the total duration of the call, that is, the sum of the duration of the target speaker and the background speaker.
[0113] Preferably, when the duration ratio is less than the ratio threshold, it means that the background vocal noise interference with the current call is very small, and the call audio does not need to be processed; when the duration ratio is greater than the ratio threshold, it means that the current call is subject to relatively serious background vocal noise interference. Deleting the background vocal from the vocal speech set can reduce the misrecognition caused by the background vocal and improve the quality of the voice call.
[0114] The embodiment of the present invention performs voice endpoint detection on call audio, deletes non-human voice noise in the call audio, and reduces the subsequent processing amount of the computer; performs voice feature extraction on the human voice speech set to obtain a voice feature set, which is convenient for separating the background human voice in the call audio in the subsequent process; extracts the voice feature set to be detected with a cumulative duration of a preset duration threshold from the voice feature set in chronological order to obtain multiple voice feature sets to be detected, performs clustering processing on each of the voice feature sets to be detected, and scores the obtained clustering results using a preset evaluation algorithm to obtain a score value for each of the voice feature sets to be detected. The method of combining clustering with scoring can detect fragmented, blurred, and low-volume background human voices; divides the human voice speech set into a first speaker and a second speaker according to the score value, and can save and dynamically update the audio features of the speaker and the background human voice in real time; calculates the duration of the first speaker and the second speaker, and determines the background human voice in the human voice speech set according to the duration of the first speaker and the second speaker, and deletes the background human voice from the human voice speech set to improve the voice call quality. Therefore, the method, device and computer-readable storage medium for voice call noise elimination proposed in the present invention can delete the background human voice in the voice call and improve the success rate of the dialogue system.
[0115] like Figure 4, which is a functional module diagram of the noise elimination device for voice calls of the present invention.
[0116] The noise elimination device 100 for voice calls of the present invention can be installed in an electronic device. According to the functions to be implemented, the noise elimination device for voice calls can include a voice endpoint detection module 101, a voice feature extraction module 102, a clustering scoring module 103, a human voice classification module 104 and a background human voice removal module 105. The module described in the present invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.
[0117] In this embodiment, the functions of each module / unit are as follows:
[0118] The voice endpoint detection module 101 is used to perform voice endpoint detection on the call audio to obtain a human voice set.
[0119] In detail, the call audio in the embodiment of the present invention includes the audio generated by a conversation in a crowd or in an environment with a lot of human voices, such as the call audio generated when a conversation is made through a communication system such as a telephone or instant messaging software in an environment full of background human voices. The call audio can be directly obtained from the communication system, or called and obtained from a database for storing voice conversation information. It should be emphasized that in order to further ensure the privacy and security of the above call audio, the above call audio can also be stored in a node of a blockchain.
[0120] The voice endpoint detection refers to distinguishing the human voice data and non-human voice data (silence and environmental noise) in the call audio in an environment with noise or other interference, and determining the starting point and ending point of the human voice data to delete the non-human voice noise in the call audio, reduce the subsequent processing volume of the computer, improve efficiency, and provide necessary support for subsequent signal processing.
[0121] In a preferred embodiment of the present invention, the voice endpoint detection model may be a voice activity detection (VAD) model based on a deep neural network (DNN).
[0122] The speech feature extraction module 102 is used to extract speech features from the human voice set to obtain a speech feature set.
[0123] In detail, the speech feature extraction module 102 specifically performs:
[0124] Pre-emphasize, frame and window the human voice set to obtain a voice frame sequence;
[0125] For each frame of speech in the speech frame sequence, a corresponding spectrum is obtained by fast Fourier transform;
[0126] Converting the spectrum into a Mel spectrum by a Mel filter bank;
[0127] Cepstrum analysis is performed on the Mel spectrum to obtain a speech feature set corresponding to the human voice set. The pre-emphasis is to use a high-pass filter to enhance the high-frequency part of the speech signal in the human voice set, so that the spectrum of the speech signal becomes flat; the framing is to use a movable finite-length window for weighting to divide the speech signal into several short segments, so that the speech signal has stability; the windowing is to make the speech signal without periodicity present some characteristics of the periodic function, so as to facilitate the subsequent Fourier expansion.
[0128] Preferably, since it is usually difficult to see the characteristics of the speech signal from the transformation of the speech signal in the time domain, the speech signal is usually converted into energy distribution in the frequency domain. Different energy distributions can represent the characteristics of different speech.
[0129] The Mel filter bank is a group of Mel-scale triangular filter banks, through which the spectrum can be converted into a Mel spectrum, and the Mel frequency can accurately reflect the auditory characteristics of the human ear.
[0130] Furthermore, the cepstrum analysis includes taking logarithms and discrete cosine changes and outputting feature vectors. The speech feature set includes feature vectors corresponding to the speech frame sequence output after cepstrum analysis.
[0131] The cluster scoring module 103 is used to extract the speech feature set to be detected whose cumulative duration is a preset duration threshold from the speech feature set in chronological order, obtain multiple speech feature sets to be detected, perform clustering processing on each of the speech feature sets to be detected, and use a preset evaluation algorithm to score the obtained clustering results to obtain a scoring value for each of the speech feature sets to be detected.
[0132] In the embodiment of the present invention, when the accumulated duration of the speech feature set reaches a preset duration threshold, a detection calculation is performed, and the speech feature set accumulated this time is referred to as a speech feature set to be detected.
[0133] In detail, the clustering process for each of the speech feature sets to be detected includes:
[0134] Step a, randomly selecting two feature vectors from the speech feature set to be detected as category centers;
[0135] Step b: for each feature vector in the speech feature set to be detected, by calculating its distance to each of the category centers, clustering the feature vector with the category center closest to it to obtain two initial categories.
[0136] In detail, the embodiment of the present invention uses the following distance algorithm to calculate the distance between the feature vector and each of the category centers:
[0137]
[0138] Where L(X,Y) is the distance value, X is the category center, Y i is the feature vector in the speech feature set to be detected.
[0139] Step c, updating the category centers of the two initial categories;
[0140] Preferably, the embodiment of the present invention calculates the mean of all feature vectors under each of the initial categories, and updates the mean as the category center of the category.
[0141] Step d: repeat the above steps b and c until the number of iterations reaches a preset threshold, and two standard categories are obtained.
[0142] Furthermore, the present invention uses a preset evaluation algorithm to score the two standard categories obtained to obtain the score values of the standard categories. Preferably, in an embodiment of the present invention, the evaluation algorithm is as follows:
[0143]
[0144] Among them, n1 and n2 are the category centers of two standard categories, respectively, and H s is the assumption that the standard categories belong to the same category, H d is the hypothesis that the standard category belongs to different categories; P(n1,n2|H s ) is the likelihood function that n1 and n2 come from the same space; P(n1|H d ), P(n2|H d ) are the likelihood functions of n1 and n2 from different spaces. The likelihood function is a function of the statistical model parameters, which is used to test whether a hypothesis is valid.
[0145] Preferably, the higher the score value is, the greater the possibility that the speech corresponding to the two standard categories belongs to the same speaker; the lower the score value is, the smaller the possibility that the speech corresponding to the two standard categories belongs to the same speaker.
[0146] The human voice classification module 104 is used to classify the human voice set into a first speaking voice and a second speaking voice according to the score value.
[0147] In detail, the human voice classification module 104 is specifically used for:
[0148] Select one of the speech feature sets to be detected and obtain its corresponding score value;
[0149] Comparing the score value with a preset score threshold;
[0150] When the score value is less than or equal to a preset score threshold, generating a first speaking voice and a second speaking voice according to the two standard categories;
[0151] Selecting a next speech feature set to be detected, obtaining a corresponding score value thereof, and classifying two standard categories in the speech feature set to be detected into the first speaker voice or the second speaker voice according to the score value thereof;
[0152] It is determined whether each of the speech feature sets to be detected has been selected, until each of the speech feature sets to be detected has been selected, and the first speaker's voice and the second speaker's voice are obtained.
[0153] When the score value is greater than a preset score threshold, the human voice classification module 104 merges the two standard categories of the selected speech feature set to be detected into a single speech category, calculates the category center of the single speech category, and generates a first speaking human voice according to the single speech category and the category center;
[0154] The first speaking voice includes speech features and duration, the speech features include the single speech category and the category center, and the duration includes the number of frames of the single speech category.
[0155] Similarly, the first speaker's voice and the second speaker's voice include speech features and durations, the speech features include the standard category and the category center, and the duration includes the number of frames of the standard category.
[0156] In detail, the step of classifying the two standard categories in the speech feature set to be detected into the first speaking voice or the second speaking voice includes:
[0157] In one of the embodiments of the present invention, if the score value of the speech feature set to be detected is greater than a score threshold, the two standard categories of the speech feature set to be detected are merged into a single speech category, and the category center of the single speech category is calculated, and the cosine distance between the category center of the single speech category and the category centers of the first speaker voice and the second speaker voice is calculated, and based on the cosine distance, the single speech category is classified into the first speaker voice or the second speaker voice.
[0158] Among them, if the cosine distance between the category center of the single voice category and the category center of the above-mentioned first speaker's voice is closer, the single voice category is classified into the first speaker's voice; if the cosine distance between the category center of the single voice category and the category center of the above-mentioned second speaker's voice is closer, the single voice category is classified into the second speaker's voice.
[0159] The classification includes: merging the single voice category with the first speaker's voice or the second speaker's voice, and recalculating the merged category center; and accumulating the number of frames of the single voice category and the duration of the first speaker's voice or the second speaker's voice.
[0160] In another embodiment of the present invention, if the score value is less than or equal to the score threshold, the cosine distance between the category center of each standard category in the speech feature set to be detected and the category center of the first speaker and the second speaker is calculated, and the two standard categories are respectively classified into the first speaker and the second speaker according to the cosine distance.
[0161] Among them, assuming that the speech feature set to be detected includes a standard category A and a standard category B, if the cosine distance between the category center of the standard category A and the category center of the first speaker's voice is closer and the cosine distance between the category center of the standard category B and the category center of the second speaker's voice is closer, then the standard category A is classified into the first speaker's voice and the standard category B is classified into the second speaker's voice.
[0162] Similarly, the classification includes: merging the standard category A and the standard category with the first speaker or the second speaker respectively, and recalculating the merged category center; and accumulating the number of frames of the standard category A and the standard category B with the duration of the first speaker or the second speaker.
[0163] The background vocal removal module 105 is used to calculate the duration of the first speaking voice and the second speaking voice, and determine the background vocals in the vocal speech set according to the duration of the first speaking voice and the second speaking voice, and delete the background vocals from the vocal speech set.
[0164] Preferably, in the call audio, the duration of the target speaker is generally longer than that of the background speaker. Therefore, the embodiment of the present invention uses the speaker with the longer duration between the first speaker and the second speaker as the target speaker, and uses the remaining speakers as background speakers.
[0165] In detail, the background vocal removal module 105 deletes the background vocal from the vocal speech set by the following method, including:
[0166] Calculate the duration ratio of the background voice in the current call using a preset duration algorithm;
[0167] Comparing the duration ratio with a preset ratio threshold;
[0168] When the duration ratio is greater than the ratio threshold, the background human voice is deleted from the human voice set, and the background human voice in the call audio is removed.
[0169] The duration algorithm is as follows:
[0170] R=t / T
[0171] Among them, R is the proportion of the background voice in this call, t is the duration of the background speaker, and T is the total duration of the call, that is, the sum of the duration of the target speaker and the background speaker.
[0172] Preferably, when the duration ratio is less than the ratio threshold, it means that the background vocal noise interference with the current call is very small, and the call audio does not need to be processed; when the duration ratio is greater than the ratio threshold, it means that the current call is subject to relatively serious background vocal noise interference. Deleting the background vocal from the vocal speech set can reduce the misrecognition caused by the background vocal and improve the quality of the voice call.
[0173] like Figure 5 FIG. 1 is a schematic diagram of the structure of an electronic device for implementing a noise elimination method for voice calls according to the present invention.
[0174] The electronic device 1 may include a processor 10 , a memory 11 , and a bus, and may also include a computer program stored in the memory 11 and executable on the processor 10 , such as a noise cancellation program 12 for voice calls.
[0175] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. The memory 11 may be an internal storage unit of the electronic device 1 in some embodiments, such as a mobile hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Further, the memory 11 may also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 may not only be used to store application software and various types of data installed in the electronic device 1, such as the code of the noise elimination program 12 for voice calls, but also be used to temporarily store data that has been output or is to be output.
[0176] The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules stored in the memory 11 (for example, executing a noise elimination program for voice calls, etc.), and calls data stored in the memory 11 to execute various functions of the electronic device 1 and process data.
[0177] The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.
[0178] Figure 5 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 5The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.
[0179] For example, although not shown, the electronic device 1 may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, etc. The electronic device 1 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.
[0180] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices.
[0181] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.
[0182] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.
[0183] The noise elimination program 12 for voice calls stored in the memory 11 in the electronic device 1 is a combination of multiple instructions, and when executed in the processor 10, it can achieve:
[0184] Perform voice endpoint detection on the call audio to obtain a human voice set;
[0185] Extracting speech features from the human voice set to obtain a speech feature set;
[0186] Extracting a speech feature set to be detected whose cumulative duration is a preset duration threshold from the speech feature set in chronological order to obtain a plurality of speech feature sets to be detected, clustering each of the speech feature sets to be detected, and scoring the obtained clustering results using a preset evaluation algorithm to obtain a scoring value for each of the speech feature sets to be detected;
[0187] According to the score value, the human voice set is divided into a first speaking voice and a second speaking voice;
[0188] The duration of the first speaking voice and the second speaking voice is calculated, and the background voice in the vocal speech set is determined according to the duration of the first speaking voice and the second speaking voice, and the background voice is deleted from the vocal speech set.
[0189] Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).
[0190] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.
[0191] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0192] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.
[0193] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.
[0194] Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any associated diagram mark in the claims should not be regarded as limiting the claims involved.
[0195] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claim can also be implemented by one unit or device through software or hardware. The second and other words are used to indicate names, but not to indicate any particular order.
[0196] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.
Claims
1. A method for eliminating noise during voice calls, characterized in that: The method comprises: Perform voice endpoint detection on the call audio to obtain a human voice set; Extracting speech features from the human voice set to obtain a speech feature set; Extracting a speech feature set to be detected whose cumulative duration is a preset duration threshold from the speech feature set in chronological order to obtain a plurality of speech feature sets to be detected, clustering each of the speech feature sets to be detected to obtain two corresponding standard categories, and scoring the obtained clustering results using a preset evaluation algorithm to obtain a scoring value for each of the speech feature sets to be detected; According to the score value, the human voice set is divided into a first speaking voice and a second speaking voice; Calculating the duration of the first speaking voice and the second speaking voice, and determining the background voice in the voice collection according to the duration of the first speaking voice and the second speaking voice, and deleting the background voice from the voice collection; Wherein, the method of dividing the human voice set into a first speaker and a second speaker according to the score value comprises: selecting one of the voice feature sets to be detected and obtaining a corresponding score value; comparing the score value with a preset score threshold; when the score value is greater than the preset score threshold, merging the two standard categories of the selected voice feature set to be detected into a single voice category, calculating the category center of the single voice category, and generating a first speaker according to the single voice category and the category center; when the score value is less than or equal to the preset score threshold, generating a first speaker and a second speaker according to the two standard categories; selecting the next voice feature set to be detected, obtaining a corresponding score value, and classifying the two standard categories in the voice feature set to be detected into the first speaker or the second speaker according to the score value; The step of classifying the two standard categories in the speech feature set to be detected into the first speaker or the second speaker according to the score value comprises: if the score value is greater than the score threshold, merging the two standard categories in the speech feature set to be detected into a single speech category, calculating the category center of the single speech category, and classifying the single speech category into the first speaker or the second speaker according to the cosine distance between the category center of the single speech category and the category centers of the first speaker and the second speaker; if the score value is less than or equal to the score threshold, classifying the two standard categories into the first speaker and the second speaker respectively according to the cosine distance between the category centers of the two standard categories in the speech feature set to be detected and the category centers of the first speaker and the second speaker; The higher the score value is, the more likely it is that the speech corresponding to the two standard categories belongs to the same speaker; the lower the score value is, the less likely it is that the speech corresponding to the two standard categories belongs to the same speaker.
2. The method for eliminating noise during voice calls according to claim 1, wherein: The extracting speech features from the human voice set to obtain a speech feature set includes: Pre-emphasize, frame and window the human voice set to obtain a voice frame sequence; For each frame of speech in the speech frame sequence, a corresponding spectrum is obtained by fast Fourier transform; Converting the spectrum into a Mel spectrum by a Mel filter bank; Cepstrum analysis is performed on the Mel spectrum to obtain a speech feature set corresponding to the human voice set.
3. The method for eliminating noise during voice calls according to claim 1, wherein: The clustering process for each of the speech feature sets to be detected includes: Step a, randomly selecting two feature vectors from the speech feature set to be detected as category centers; Step b, for each feature vector in the speech feature set to be detected, by calculating the distance from the feature vector to each of the category centers, clustering the feature vector with the category center closest to it to obtain two initial categories; Step c, updating the category centers of the two initial categories; Step d: repeat the above steps b and c until the number of iterations reaches a preset threshold, and two standard categories are obtained.
4. The method for eliminating noise during voice calls according to claim 1, wherein: The classifications include: Merging the single speech category with the first speaker or the second speaker, and recalculating the merged category center, and accumulating the number of frames of the single speech category with the duration of the first speaker or the second speaker; or The two standard categories are merged with the first speaker and the second speaker respectively, and the merged category center is recalculated, and the number of frames of the standard category is accumulated with the duration of the first speaker or the second speaker.
5. The method for eliminating noise during voice calls according to any one of claims 1 to 4, characterized in that: Deleting the background human voice from the human voice set includes: Calculate the duration ratio of the background voice in the current call using a preset duration algorithm; When the duration ratio is greater than a preset ratio threshold, the background human voice is deleted from the human voice set, and the background human voice in the call audio is removed.
6. A noise elimination device for voice calls, characterized in that: The device comprises: The voice endpoint detection module is used to perform voice endpoint detection on the call audio to obtain a human voice set; A speech feature extraction module, used for extracting speech features from the human voice set to obtain a speech feature set; A cluster scoring module is used to intercept the speech feature set to be detected whose cumulative duration is a preset duration threshold from the speech feature set in chronological order to obtain a plurality of speech feature sets to be detected, perform clustering processing on each of the speech feature sets to be detected to obtain two corresponding standard categories, and score the obtained clustering results using a preset evaluation algorithm to obtain a score value for each of the speech feature sets to be detected; A human voice classification module, used for classifying the human voice set into a first speaking human voice and a second speaking human voice according to the score value; A background vocal removal module, configured to calculate the duration of the first speaking voice and the second speaking voice, and determine the background vocals in the vocal speech set according to the duration of the first speaking voice and the second speaking voice, and delete the background vocals from the vocal speech set; Wherein, the method of dividing the human voice set into a first speaker and a second speaker according to the score value comprises: selecting one of the voice feature sets to be detected and obtaining a corresponding score value; comparing the score value with a preset score threshold; when the score value is greater than the preset score threshold, merging the two standard categories of the selected voice feature set to be detected into a single voice category, calculating the category center of the single voice category, and generating a first speaker according to the single voice category and the category center; when the score value is less than or equal to the preset score threshold, generating a first speaker and a second speaker according to the two standard categories; selecting the next voice feature set to be detected, obtaining a corresponding score value, and classifying the two standard categories in the voice feature set to be detected into the first speaker or the second speaker according to the score value; The step of classifying the two standard categories in the speech feature set to be detected into the first speaker or the second speaker according to the score value comprises: if the score value is greater than the score threshold, merging the two standard categories in the speech feature set to be detected into a single speech category, calculating the category center of the single speech category, and classifying the single speech category into the first speaker or the second speaker according to the cosine distance between the category center of the single speech category and the category centers of the first speaker and the second speaker; if the score value is less than or equal to the score threshold, classifying the two standard categories into the first speaker and the second speaker respectively according to the cosine distance between the category centers of the two standard categories in the speech feature set to be detected and the category centers of the first speaker and the second speaker; The higher the score value is, the more likely it is that the speech corresponding to the two standard categories belongs to the same speaker; the lower the score value is, the less likely it is that the speech corresponding to the two standard categories belongs to the same speaker.
7. An electronic device, characterized in that: The electronic device comprises: a memory storing at least one instruction; and A processor executes the instructions stored in the memory to perform the noise cancellation method for voice calls as claimed in any one of claims 1 to 5.
8. A computer-readable storage medium, comprising a data storage area and a program storage area, wherein the data storage area stores created data and the program storage area stores a computer program, wherein: When the computer program is executed by a processor, the noise elimination method for voice calls as claimed in any one of claims 1 to 5 is implemented.
