Speech enhancement algorithm evaluation method and device, electronic equipment, chip and storage medium

Abstract

The invention provides a speech enhancement algorithm evaluation method and device, electronic equipment, a chip and a storage medium, and the method comprises the steps: carrying out the noise reduction of a noisy speech through employing a speech enhancement algorithm, and obtaining a noise-reduced speech; the noise reduction processing process is evaluated through a plurality of evaluation indexes, a plurality of evaluation results are obtained, the evaluation results correspond to the evaluation indexes, and the evaluation indexes comprise one or more of parameter quantity, floating point operand FLOPs, perceptual speech quality PESQ and short-term objective intelligibility STOI; the multiple evaluation indexes are converted to the same order of magnitude, multiple conversion results are obtained, and the conversion results correspond to the evaluation results; according to the method, the multiple objective indexes are used for evaluating the speech enhancement algorithm, the multiple objective indexes are converted to the same order of magnitude for fusion processing, and the comprehensive and objective evaluation result is obtained.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to a method and device for evaluating a speech enhancement algorithm, an electronic device, a chip, and a storage medium. Background Art

[0002] With the development of artificial intelligence, smart wearable devices are becoming more and more popular. These devices have limited computing resources and often work in noisy environments. In order to ensure the normal operation of their voice interaction functions, it is particularly important to design a speech enhancement algorithm with low complexity and good noise reduction effect. Summary of the invention

[0003] The present invention provides a method, device, electronic device, chip and storage medium for evaluating a speech enhancement algorithm. The speech enhancement algorithm is evaluated using multiple objective indicators, and the multiple objective indicators are converted to the same order of magnitude for fusion processing to obtain a comprehensive and objective evaluation result.

[0004] The first aspect of the present disclosure provides an evaluation method for a speech enhancement algorithm, the method comprising: using a speech enhancement algorithm to perform noise reduction processing on noisy speech to obtain noise-reduced speech; evaluating the noise reduction process through multiple evaluation indicators to obtain multiple evaluation results, wherein the evaluation results correspond to the evaluation indicators, and the multiple evaluation indicators include one or more of parameter quantity, floating-point operations FLOPs, perceived speech quality PESQ, and short-time objective intelligibility STOI; converting the multiple evaluation results to the same order of magnitude to obtain multiple conversion results, and the conversion results correspond to the evaluation results; fusing the multiple conversion results to determine the comprehensive evaluation result of the speech enhancement algorithm.

[0005] In some embodiments of the present disclosure, the method further includes: acquiring noisy audio and clean speech; and generating noisy speech according to the noisy audio and the clean speech.

[0006] In some embodiments of the present disclosure, multiple evaluation indicators include parameter quantities and / or floating-point operations, and the noise reduction process is evaluated using the multiple evaluation indicators to obtain multiple evaluation results including: in the process of the speech enhancement algorithm performing noise reduction processing on noisy speech, the parameter quantities and / or floating-point operations of the speech enhancement algorithm are counted to obtain parameter quantity evaluation results and / or floating-point operation evaluation results.

[0007] In some embodiments of the present disclosure, multiple evaluation indicators include perceived speech quality, and the noise reduction processing process is evaluated by multiple evaluation indicators to obtain multiple evaluation results including: inputting pure speech and noise-reduced speech into a perceived speech quality evaluation function to obtain a perceived speech quality evaluation result of the noise-reduced speech.

[0008] In some embodiments of the present disclosure, multiple evaluation indicators include short-time objective intelligibility, and the noise reduction processing process is evaluated by multiple evaluation indicators to obtain multiple evaluation results including: inputting pure speech and noise-reduced speech into a short-time objective intelligibility evaluation function to obtain a short-time objective intelligibility evaluation result of the noise-reduced speech.

[0009] In some embodiments of the present disclosure, multiple evaluation results are converted to the same order of magnitude to obtain multiple conversion results, including: for each evaluation result, obtaining a reference value of the evaluation result through a baseline algorithm; based on the reference value, performing order of magnitude conversion processing on the evaluation result to obtain a conversion result.

[0010] In some embodiments of the present disclosure, multiple conversion results are fused to determine a comprehensive evaluation result of a speech enhancement algorithm, including: determining a weight for each conversion result; performing weighted summation processing on the multiple conversion results based on the weight of each conversion result to determine a weighted summation result; and determining the weighted summation result as a comprehensive evaluation result of the speech enhancement algorithm.

[0011] The second aspect of the present disclosure proposes an evaluation device for a speech enhancement algorithm, which includes: a processing module: using a speech enhancement algorithm to perform noise reduction processing on noisy speech to obtain noise-reduced speech; an evaluation module: evaluating the noise reduction process through multiple evaluation indicators to obtain multiple evaluation results, wherein the evaluation results correspond to the evaluation indicators, and the multiple evaluation indicators include one or more of parameter quantity, floating-point operation number, perceived speech quality, and short-time objective intelligibility; a conversion module: converting multiple evaluation indicator results to the same order of magnitude to obtain multiple conversion results, and the conversion results correspond to the evaluation results; a fusion module: fusing multiple conversion results to determine the comprehensive evaluation result of the speech enhancement algorithm.

[0012] The third aspect embodiment of the present disclosure proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the first aspect embodiment of the present disclosure.

[0013] The fourth aspect embodiment of the present disclosure proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in the first aspect embodiment of the present disclosure.

[0014] The fifth aspect embodiment of the present disclosure proposes a chip, which includes one or more interface circuits and one or more processors; the interface circuit is used to receive code instructions and transmit them to the processor, and the processor is used to run the code instructions to execute the method described in the first aspect embodiment of the present disclosure.

[0015] In summary, the present invention discloses an evaluation method for a speech enhancement algorithm, which includes: using a speech enhancement algorithm to perform noise reduction processing on noisy speech to obtain noise-reduced speech; evaluating the noise reduction process through multiple evaluation indicators to obtain multiple evaluation results, wherein the evaluation results correspond to the evaluation indicators, and the multiple evaluation indicators include one or more of parameter quantity, floating-point operations, perceived speech quality, and short-time objective intelligibility; converting the multiple evaluation results to the same order of magnitude to obtain multiple conversion results; fusing the multiple conversion results to determine the comprehensive evaluation result of the speech enhancement algorithm. The method uses multiple objective indicators to evaluate the speech enhancement algorithm, and converts the multiple objective indicators to the same order of magnitude for fusion processing to obtain a comprehensive and objective evaluation result.

[0016] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0018] Figure 1 A flowchart of a method for evaluating a speech enhancement algorithm provided by an embodiment of the present disclosure;

[0019] Figure 2 A flowchart of a method for evaluating a speech enhancement algorithm provided by an embodiment of the present disclosure;

[0020] Figure 3 A flowchart of a method for evaluating a speech enhancement algorithm provided by an embodiment of the present disclosure;

[0021] Figure 4 A schematic diagram of the structure of an evaluation device for a speech enhancement algorithm provided in an embodiment of the present disclosure;

[0022] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0024] The evaluation methods for speech enhancement algorithms in related technologies mostly use an objective evaluation indicator in isolation, without comprehensively considering the overall effect represented by multiple indicators, and without considering the impact of parameter quantity and complexity on algorithm deployment.

[0025] To this end, the present disclosure provides a method, device, electronic device, chip and storage medium for evaluating a speech enhancement algorithm, which uses multiple objective indicators to evaluate the speech enhancement algorithm, taking into account the parameter quantity and complexity of the speech enhancement algorithm, including evaluating the speech enhancement algorithm from multiple aspects such as noise reduction effect, speech distortion, computational complexity and parameter quantity, and converting multiple objective indicators to the same order of magnitude for fusion processing to obtain a comprehensive and objective evaluation result.

[0026] Figure 1 The following is a flow chart of a method for evaluating a speech enhancement algorithm provided by an embodiment of the present disclosure. Figure 1 As shown, the evaluation method of the speech enhancement algorithm includes steps 101-104.

[0027] Step 101: Use a speech enhancement algorithm to perform noise reduction processing on noisy speech to obtain noise-reduced speech.

[0028] In some embodiments of the present disclosure, the method further includes acquiring noisy speech.

[0029] In some embodiments of the present disclosure, noisy speech refers to speech containing noise. It can be achieved by collecting clean speech and noisy audio and generating noisy speech based on the clean speech and the noisy audio. It can also be achieved by directly selecting a data set including noisy speech and denoised comparison speech in the training data of the speech enhancement algorithm. The present disclosure is not limited to this.

[0030] In some embodiments of the present disclosure, the tasks of the speech enhancement algorithm include speech denoising, speech separation, and speech de-reverberation.

[0031] In some embodiments of the present disclosure, a speech enhancement algorithm is used for a speech interaction function. For a machine, the training goal of the speech enhancement algorithm is to improve the intelligibility of speech. For a human, the training goal of the speech enhancement algorithm is to improve the quality of speech.

[0032] In some embodiments of the present disclosure, noisy speech is input into a speech enhancement algorithm to be evaluated, and the speech enhancement algorithm to be evaluated performs speech noise reduction, speech separation, and speech de-reverberation on the noisy speech, and the output result is noise-reduced speech.

[0033] Step 102, the noise reduction process is evaluated by multiple evaluation indicators to obtain multiple evaluation results, wherein the evaluation results correspond to the evaluation indicators, and the multiple evaluation indicators include one or more of parameter quantity, floating point operation number FLOPs, perceived speech quality PESQ, and short-time objective intelligibility STOI.

[0034] In the embodiments of the present disclosure, the number of parameters and floating-point operations are important indicators for measuring the size and complexity of deep learning algorithms / models, wherein the number of parameters (Parameters) of an algorithm refers to the total number of parameters contained in the algorithm / model, which can be used to measure the size of memory resources occupied by the speech enhancement algorithm, and the floating-point operation number represents the amount of calculation of the algorithm, which can be used to measure the computational complexity of the speech enhancement algorithm.

[0035] In the embodiments of the present disclosure, perceptual evaluation of speech quality (PESQ) is an important indicator for evaluating speech quality, which can be used to measure the quality of noise-reduced speech obtained by processing with a speech enhancement algorithm.

[0036] In the embodiments of the present disclosure, short-time objective intelligibility (STOI) is an important indicator for evaluating speech intelligibility, and the indicator can be used to measure the intelligibility of noise-reduced speech obtained by processing with a speech enhancement algorithm.

[0037] In the embodiments of the present disclosure, the multiple evaluation indicators may include other evaluation indicators besides parameter quantity, floating-point operation number, perceived speech quality, and short-time objective intelligibility, which are not limited by the present disclosure.

[0038] Step 103, converting the multiple evaluation results to the same order of magnitude to obtain multiple conversion results, and the conversion results correspond to the evaluation results.

[0039] In some embodiments of the present disclosure, multiple evaluation indicators include one or more of parameter quantity, floating-point operation number, perceived speech quality and short-time objective intelligibility, and the evaluation results correspond to the evaluation indicators. The multiple evaluation results include parameter quantity evaluation results, floating-point operation number evaluation results, perceived speech quality evaluation results and short-time objective intelligibility evaluation results, and these evaluation results have different orders of magnitude. For example, the numerical value of the parameter quantity evaluation result is usually large, and the numerical value may reach millions or more. The value range of the perceived speech quality evaluation result is usually quantized to [-0.5, 4.5]. The short-time objective intelligibility evaluation result is usually expressed in the form of percentage, and the value range is quantized to [0, 1]. The multiple evaluation results are converted to the same order of magnitude to obtain a comprehensive evaluation result.

[0040] Step 104: fuse multiple conversion results to determine a comprehensive evaluation result of the speech enhancement algorithm.

[0041] In the embodiments of the present disclosure, when evaluating the effect of a speech enhancement algorithm, the overall effect represented by multiple evaluation indicators is considered, the multiple evaluation indicators are converted to the same order of magnitude, and the converted multiple evaluation indicators are fused to obtain a comprehensive evaluation result.

[0042] In some embodiments of the present disclosure, fusing multiple conversion results includes weighted summing of the multiple conversion results, and also includes other fusing methods, which are not limited by the present disclosure.

[0043] In summary, according to the evaluation method of the speech enhancement algorithm proposed in the present disclosure, the method includes: using a speech enhancement algorithm to perform noise reduction processing on noisy speech to obtain noise-reduced speech; evaluating the noise reduction process through multiple evaluation indicators to obtain multiple evaluation results, wherein the multiple evaluation indicators include one or more of parameter quantity, floating-point operations, perceived speech quality, and short-time objective intelligibility; converting the multiple evaluation results to the same order of magnitude to obtain multiple conversion results; fusing the multiple conversion results to determine the comprehensive evaluation result of the speech enhancement algorithm. This method uses multiple objective indicators to evaluate the speech enhancement algorithm, and converts the multiple objective indicators to the same order of magnitude for fusion processing to obtain a comprehensive and objective evaluation result.

[0044] based on Figure 1 The embodiment shown, Figure 2 The following is a flow chart of a method for evaluating a speech enhancement algorithm proposed in the present disclosure. Figure 3 The present invention is a flowchart of a method for evaluating a speech enhancement algorithm disclosed in the present invention.

[0045] like Figure 2 As shown, the method includes the following steps.

[0046] Step 201: Use a speech enhancement algorithm to perform noise reduction processing on noisy speech to obtain noise-reduced speech.

[0047] In some embodiments of the present disclosure, the method further includes: acquiring noisy audio and clean speech; and generating noisy speech according to the noisy audio and the clean speech.

[0048] In some embodiments of the present disclosure, Figure 3 As shown, noisy audio and clean speech are obtained, and the noisy audio and clean speech are used to generate noisy speech through a noisy speech generation program, and then the noisy speech is processed by the speech enhancement algorithm to be evaluated to obtain noise-reduced speech.

[0049] Furthermore, in some embodiments of the present disclosure, multiple noisy speeches with different noise reduction ratios can be generated based on noisy audio and clean speech, and the multiple noisy speeches with different noise reduction ratios can be denoised using a speech enhancement algorithm to obtain multiple noise-reduced speeches for subsequent evaluation of the speech enhancement algorithm, thereby obtaining a more comprehensive and objective evaluation result.

[0050] Step 202, evaluating the noise reduction process through multiple evaluation indicators to obtain multiple evaluation results, wherein the evaluation results correspond to the evaluation indicators, and the multiple evaluation indicators include one or more of parameter quantity, floating-point operation number, perceived speech quality, and short-time objective intelligibility.

[0051] In some embodiments of the present disclosure, the evaluation index includes a parameter quantity, and the noise reduction process is evaluated by the parameter quantity to obtain multiple evaluation results including: in the process of the speech enhancement algorithm performing noise reduction processing on noisy speech, the parameter quantity of the speech enhancement algorithm is estimated as the parameter quantity evaluation result, wherein the parameter quantity of the speech enhancement algorithm represents the size of memory resources occupied by the speech enhancement algorithm, and it is generally believed that the smaller the value of the parameter quantity, the better the speech enhancement algorithm.

[0052] In some embodiments of the present disclosure, the evaluation index includes a floating-point operand, and the noise reduction process is evaluated by the floating-point operand to obtain multiple evaluation results including: in the process of the speech enhancement algorithm performing noise reduction on noisy speech, the floating-point operand of the speech enhancement algorithm is estimated as the floating-point operand evaluation result, wherein the floating-point operand of the speech enhancement algorithm represents the complexity of the speech enhancement algorithm, and it is generally believed that the smaller the value of the floating-point operand, the better the speech enhancement algorithm.

[0053] In some embodiments of the present disclosure, the evaluation index includes perceived speech quality, and the noise reduction process is evaluated by the perceived speech quality to obtain multiple evaluation results including: inputting pure speech and noise-reduced speech into a perceived speech quality evaluation function, and calculating the perceived speech quality evaluation result of the noise-reduced speech by the perceived speech quality evaluation function, wherein the perceived speech quality represents the quality of the noise-reduced speech, and the larger the value of the perceived speech quality evaluation result, the better the quality of the noise-reduced speech, and the better the effect of the speech enhancement algorithm on the noise reduction processing of the noisy speech.

[0054] In some embodiments of the present disclosure, the evaluation index includes short-time objective intelligibility, and the noise reduction process is evaluated by the short-time objective intelligibility, and multiple evaluation results are obtained, including: inputting pure speech and noise-reduced speech into the short-time objective intelligibility evaluation function, and obtaining the short-time objective intelligibility evaluation result of the noise-reduced speech by calculating the objective intelligibility evaluation function, wherein the short-time objective intelligibility represents the intelligibility of the noise-reduced speech, and the larger the numerical value of the perceived speech quality evaluation result, the higher the intelligibility of the noise-reduced speech, and the better the effect of the speech enhancement algorithm on the noise reduction processing of the noisy speech.

[0055] It should be noted that the multiple evaluation indicators used in this method may also include other indicators for evaluating the speech noise reduction effect or the quality of the algorithm, which is not limited in this disclosure.

[0056] Step 203: For each evaluation result, a reference value of the evaluation result is obtained through a baseline algorithm.

[0057] Step 204 , based on the reference value, the evaluation result is subjected to order of magnitude conversion processing to obtain a conversion result, wherein the conversion result corresponds to the evaluation result.

[0058] In some embodiments of the present disclosure, Figure 3 As shown, for the parameter quantity evaluation result of the speech enhancement algorithm, the reference value of the parameter quantity is obtained through the baseline algorithm, and the parameter quantity evaluation result of the speech enhancement algorithm is converted by order of magnitude using function 1 to obtain the reference value conversion result. The formula of function 1 is as follows:

[0059]

[0060] Among them, parameters represents the parameter evaluation results, parameters baseline The reference value f1 (parameters) indicating the parameter quantity indicates the parameter quantity conversion result.

[0061] In some embodiments of the present disclosure, Figure 3 As shown, for the floating-point operation evaluation result of the speech enhancement algorithm, the reference value of the floating-point operation is obtained through the baseline algorithm, and the floating-point operation evaluation result of the speech enhancement algorithm is converted by the function 2 to obtain the floating-point operation conversion result. The formula of function 2 is as follows:

[0062]

[0063] Among them, FLOPs represents the evaluation result of floating point operations. baseline It indicates the reference value of floating-point operations, and f2(FLOPs) indicates the floating-point operation conversion result.

[0064] In some embodiments of the present disclosure, Figure 3 As shown, for the perceived speech quality evaluation result of the speech enhancement algorithm, a reference value of the perceived speech quality is obtained through a baseline algorithm, and function 3 is used to perform an order of magnitude conversion on the perceived speech quality evaluation result of the speech enhancement algorithm to obtain a perceived speech quality conversion result. The formula of function 3 is as follows:

[0065] f3(PESQ)=PESQ-PESQ baseline

[0066] Among them, PESQ represents the perceived speech quality evaluation result, PESQ baseline It represents the reference value of perceived speech quality, and f3(PESQ) represents the conversion result of perceived speech quality.

[0067] In some embodiments of the present disclosure, Figure 3 As shown in FIG. 1 , for the short-time objective intelligibility evaluation result of the speech enhancement algorithm, the reference value of the short-time objective intelligibility is obtained through the baseline algorithm, and the short-time objective intelligibility evaluation result of the speech enhancement algorithm is converted by the function 4 to obtain the short-time objective intelligibility conversion result. The formula of function 4 is as follows:

[0068] f4(STOI)=STOI-STOI baseline

[0069] Among them, STOI represents the short-term objective intelligibility evaluation result, STOI baseline It represents the reference value of short-time objective intelligibility, and f4(STOI) represents the conversion result of short-time objective intelligibility.

[0070] Step 205: determine the weight of each conversion result.

[0071] In some embodiments of the present disclosure, the weight of each conversion result is determined according to the application scenario of the speech enhancement algorithm to be evaluated. For example, the speech enhancement algorithm is used for voice interaction of a smart wearable device. If there is insufficient memory reserved for the speech enhancement algorithm in the device design, the parameter quantity of the speech enhancement algorithm is given priority when evaluating the speech enhancement algorithm. It can be determined that the weight of the parameter quantity conversion result is greater.

[0072] Step 206: Perform weighted sum processing on the multiple conversion results based on the weight of each conversion result to determine a weighted sum result.

[0073] Step 207: determine the weighted sum result as the comprehensive evaluation result of the speech enhancement algorithm.

[0074] In some embodiments of the present disclosure, four evaluation indicators, namely, parameter quantity, floating-point operation number, perceived speech quality, and short-time objective intelligibility, are used. Assuming that the weights of the conversion results of parameter quantity, floating-point operation number, perceived speech quality, and short-time objective intelligibility are -a, -b, c, and d, respectively, the formula for weighted summation of the conversion results of the four evaluation indicators is as follows:

[0075] score=c*f3(PESQ)+d*f4(STOI)-a*f1(parameters)-b*f2(FLOPs)

[0076] The score is used to represent the comprehensive evaluation result. The higher the score, the better the performance of the speech enhancement algorithm, and vice versa.

[0077] In summary, the evaluation method of the speech enhancement algorithm proposed in the present invention uses a speech enhancement algorithm to perform noise reduction processing on noisy speech to obtain noise-reduced speech; the noise reduction processing process is evaluated through multiple evaluation indicators to obtain multiple evaluation results, wherein the evaluation results correspond to the evaluation indicators, and the multiple evaluation indicators include parameter quantity, floating-point operations FLOPs, perceived speech quality PESQ, short-time objective intelligibility STOI or more; for each evaluation result, a reference value of the evaluation result is obtained through a baseline algorithm; based on the reference value, the evaluation result is subjected to an order of magnitude conversion processing to obtain a conversion result, and the conversion result corresponds to the evaluation result; the weight of each conversion result is determined; based on the weight of each conversion result, a weighted summation processing is performed on multiple conversion results to determine a weighted summation result; the weighted summation result is determined as a comprehensive evaluation result of the speech enhancement algorithm. This scheme uses multiple objective indicators to evaluate the speech enhancement algorithm, taking into account the parameter quantity and complexity of the speech enhancement algorithm, including evaluating the speech enhancement algorithm from multiple aspects such as noise reduction effect, speech distortion, computational complexity and parameter quantity, and converting multiple objective indicators to the same order of magnitude for fusion processing to obtain comprehensive and objective evaluation results.

[0078] Figure 4 FIG. 3 is a schematic diagram of a structure of an evaluation device 300 for a speech enhancement algorithm provided in an embodiment of the present disclosure. Figure 5 As shown, the evaluation device of the speech enhancement algorithm includes:

[0079] Processing module 310: using a speech enhancement algorithm to perform noise reduction processing on the noisy speech to obtain noise-reduced speech;

[0080] Evaluation module 320: evaluates the noise reduction process through multiple evaluation indicators to obtain multiple evaluation results, wherein the evaluation results correspond to the evaluation indicators, and the multiple evaluation indicators include one or more of parameter quantity, floating-point operation number, perceived speech quality, and short-time objective intelligibility;

[0081] Conversion module 330: converting the multiple evaluation results to the same order of magnitude to obtain multiple conversion results, where the conversion results correspond to the evaluation results;

[0082] Fusion module 340: Fusion multiple conversion results to determine the comprehensive evaluation result of the speech enhancement algorithm.

[0083] In some embodiments, the device further includes a generation module, which is used to obtain noisy audio and clean speech; and generate noisy speech based on the noisy audio and clean speech.

[0084] In some embodiments, multiple evaluation indicators include parameter quantities and / or floating-point operations, and the evaluation module 320 is specifically used to: during the process of the speech enhancement algorithm performing noise reduction processing on noisy speech, count the parameter quantities and / or floating-point operations of the speech enhancement algorithm to obtain parameter quantity evaluation results and / or floating-point operation evaluation results.

[0085] In some embodiments, the multiple evaluation indicators include perceived speech quality, and the evaluation module 320 is specifically used to: input the clean speech and the noise-reduced speech into the perceived speech quality evaluation function to obtain the perceived speech quality evaluation result of the noise-reduced speech.

[0086] In some embodiments, the multiple evaluation indicators include short-time objective intelligibility, and the evaluation module 320 is specifically used to: input the clean speech and the noise-reduced speech into the short-time objective intelligibility evaluation function to obtain the short-time objective intelligibility evaluation result of the noise-reduced speech.

[0087] In some embodiments, the conversion module 330 is specifically used to: for each evaluation result, obtain a reference value of the evaluation result through a baseline algorithm; and perform an order of magnitude conversion process on the evaluation result based on the reference value to obtain a conversion result.

[0088] In some embodiments, the fusion module 340 is specifically used to: determine the weight of each conversion result; based on the weight of each conversion result, perform weighted summation processing on multiple conversion results to determine a weighted summation result; and determine the weighted summation result as a comprehensive evaluation result of the speech enhancement algorithm.

[0089] Since the device provided in the embodiment of the present disclosure corresponds to the methods provided in the above-mentioned embodiments, the implementation of the method is also applicable to the device provided in the embodiment and will not be described in detail in this embodiment.

[0090] In summary, the evaluation device of the speech enhancement algorithm proposed in the present invention uses multiple objective indicators to evaluate the speech enhancement algorithm through a processing module, an evaluation module, a conversion module and a fusion module, and takes into account the parameter quantity and complexity of the speech enhancement algorithm. For example, the speech enhancement algorithm is comprehensively evaluated from multiple aspects such as noise reduction effect, speech distortion, computational complexity and parameter quantity, and multiple objective indicators are converted to the same order of magnitude for fusion processing to obtain a comprehensive and objective evaluation result.

[0091] In the embodiments provided in the present application, the methods and devices provided in the embodiments of the present application are introduced. In order to implement the functions in the methods provided in the embodiments of the present application, the electronic device may include a hardware structure and a software module, and implement the functions in the form of a hardware structure, a software module, or a hardware structure plus a software module. A function of the functions may be executed in the form of a hardware structure, a software module, or a hardware structure plus a software module.

[0092] Figure 5 4 is a block diagram of an electronic device 400 for implementing the above-mentioned speech enhancement algorithm evaluation method according to an exemplary embodiment. For example, the electronic device 400 may be a mobile phone, a computer, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0093] Reference Figure 5 , the electronic device 400 may include one or more of the following components: a processing component 402 , a memory 404 , a power component 406 , a multimedia component 408 , an audio component 410 , an input / output (I / O) interface 412 , a sensor component 414 , and a communication component 416 .

[0094] The processing component 402 generally controls the overall operation of the electronic device 400, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 402 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 402 may include one or more modules to facilitate the interaction between the processing component 402 and other components. For example, the processing component 402 may include a multimedia module to facilitate the interaction between the multimedia component 408 and the processing component 402.

[0095] The memory 404 is configured to store various types of data to support operations on the electronic device 400. Examples of such data include instructions for any application or method operating on the electronic device 400, contact data, phone book data, messages, pictures, videos, etc. The memory 404 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0096] The power supply component 406 provides power to the various components of the electronic device 400. The power supply component 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 400.

[0097] The multimedia component 408 includes a screen that provides an output interface between the electronic device 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 408 includes a front camera and / or a rear camera. When the electronic device 400 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0098] The audio component 410 is configured to output and / or input audio signals. For example, the audio component 410 includes a microphone (MIC), and when the electronic device 400 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 404 or sent via the communication component 416. In some embodiments, the audio component 410 also includes a speaker for outputting audio signals.

[0099] I / O interface 412 provides an interface between processing component 402 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0100] The sensor assembly 414 includes one or more sensors for providing various aspects of status assessment for the electronic device 400. For example, the sensor assembly 414 can detect the open / closed state of the electronic device 400, the relative positioning of components, such as the display and keypad of the electronic device 400, and the sensor assembly 414 can also detect the position change of the electronic device 400 or a component of the electronic device 400, the presence or absence of user contact with the electronic device 400, the orientation or acceleration / deceleration of the electronic device 400, and the temperature change of the electronic device 400. The sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 414 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 414 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0101] The communication component 416 is configured to facilitate wired or wireless communication between the electronic device 400 and other devices. The electronic device 400 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G LTE, 5G NR (NewRadio) or a combination thereof. In an exemplary embodiment, the communication component 416 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 416 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0102] In an exemplary embodiment, the electronic device 400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0103] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 504 including instructions, and the instructions can be executed by a processor 520 of an electronic device 500 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0104] The embodiments of the present disclosure further provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the evaluation method of the speech enhancement algorithm described in the above embodiments of the present disclosure.

[0105] The embodiments of the present disclosure further provide a computer program product, including a computer program, which, when executed by a processor, performs the evaluation method of the speech enhancement algorithm described in the above embodiments of the present disclosure.

[0106] An embodiment of the present disclosure also proposes a chip, which includes one or more interface circuits and one or more processors; the interface circuit is used to receive code instructions and transmit them to the processor, and the processor is used to run the code instructions to execute the evaluation method of the speech enhancement algorithm described in the above embodiment of the present disclosure.

[0107] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0108] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0109] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0110] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processing module, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (control method), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0111] It should be understood that the various parts of the embodiments of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0112] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0113] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0114] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for evaluating a speech enhancement algorithm, characterized in that: The method comprises: Use a speech enhancement algorithm to perform noise reduction processing on the noisy speech to obtain noise-reduced speech; The noise reduction process is evaluated by multiple evaluation indicators to obtain multiple evaluation results, wherein the evaluation results correspond to the evaluation indicators, and the multiple evaluation indicators include one or more of parameter quantity, floating-point operations FLOPs, perceived speech quality PESQ, and short-time objective intelligibility STOI. Converting the multiple evaluation results to the same order of magnitude to obtain multiple conversion results, wherein the conversion results correspond to the evaluation results; The multiple conversion results are fused to determine a comprehensive evaluation result of the speech enhancement algorithm.

2. The method according to claim 1, characterized in that The method further comprises: Get noisy audio and clean speech; The noisy speech is generated according to the noise audio and the clean speech.

3. The method according to claim 1, characterized in that The multiple evaluation indicators include parameter quantities and / or floating-point operations. The noise reduction process is evaluated by using the multiple evaluation indicators to obtain multiple evaluation results including: In the process of the speech enhancement algorithm performing noise reduction processing on the noisy speech, the parameter quantity and / or floating-point operation number of the speech enhancement algorithm are counted to obtain the parameter quantity evaluation result and / or floating-point operation number evaluation result.

4. The method according to claim 2, characterized in that: The multiple evaluation indicators include perceived speech quality, and the noise reduction process is evaluated by the multiple evaluation indicators to obtain multiple evaluation results including: The clean speech and the noise-reduced speech are input into a perceptual speech quality evaluation function to obtain a perceptual speech quality evaluation result of the noise-reduced speech.

5. The method according to claim 2, characterized in that: The multiple evaluation indicators include short-term objective intelligibility, and the noise reduction process is evaluated by the multiple evaluation indicators to obtain multiple evaluation results including: The clean speech and the noise-reduced speech are input into a short-time objective intelligibility evaluation function to obtain a short-time objective intelligibility evaluation result of the noise-reduced speech.

6. The method according to any one of claims 1 to 4, characterized in that The converting the multiple evaluation results to the same order of magnitude to obtain multiple conversion results comprises: For each evaluation result, a reference value of the evaluation result is obtained by using a baseline algorithm; Based on the reference value, the evaluation result is subjected to order of magnitude conversion processing to obtain a conversion result.

7. The method according to claim 6, characterized in that The fusing of the multiple conversion results to determine the comprehensive evaluation result of the speech enhancement algorithm includes: Determine the weight of each conversion result; Based on the weight of each conversion result, performing weighted sum processing on the multiple conversion results to determine a weighted sum result; The weighted sum result is determined as the comprehensive evaluation result of the speech enhancement algorithm.

8. A device for evaluating a speech enhancement algorithm, characterized in that: The device comprises: Processing module: Use speech enhancement algorithm to reduce noise in noisy speech to obtain noise-reduced speech; Evaluation module: evaluates the noise reduction process through multiple evaluation indicators to obtain multiple evaluation results, wherein the evaluation results correspond to the evaluation indicators, and the multiple evaluation indicators include one or more of parameter quantity, floating point operation number FLOPs, perceived speech quality PESQ, and short-time objective intelligibility STOI; A conversion module: converting the multiple evaluation results to the same order of magnitude to obtain multiple conversion results, wherein the conversion results correspond to the evaluation results; Fusion module: Fusion the multiple conversion results to determine the comprehensive evaluation result of the speech enhancement algorithm.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

11. A chip, characterized in that: The method comprises one or more interface circuits and one or more processors; the interface circuit is used to receive code instructions and transmit them to the processor, and the processor is used to run the code instructions to execute the method according to any one of claims 1 to 7.

Images