A method, device and readable medium for speech data enhancement based on equalizer

Abstract

The present invention discloses a method, device and readable medium for speech data enhancement based on an equalizer, which comprises obtaining a first speech with a first frequency response curve; using at least one different equalizer to process the first speech to obtain at least one second speech, each second speech having a second frequency response curve different from the first frequency response curve, and the first speech and the second speech corresponding to a first speaker; repeating the above steps to obtain the first speech and the second speech corresponding to a plurality of different speakers, and obtaining training data of a speech model based on the first speech and the second speech of the plurality of different speakers. The speech model obtained by training with the training data can effectively improve the accuracy across devices and reduce the error caused by different audio response curves of the devices.

Description

Technical Field

[0001] The present invention relates to the field of speech signal processing, and in particular to a speech data enhancement method, device and readable medium based on an equalizer. Background Art

[0002] Existing speech processing technologies, such as voiceprint recognition and speech recognition, often perform data enhancement on speech data to improve the generalization ability (or robustness) of the model during the training process to obtain more effective training data. Common speech data enhancement methods include adding noise, background voice, reverberation, speeding up or slowing down speech, and adding codecs. These methods simulate the impact of the outside world on speech and can effectively improve the generalization ability of speech processing technology.

[0003] However, the impact of the frequency response curve of the pickup system on speech has not been taken into account. The frequency response curve of the pickup system reflects its response characteristics to speech of different frequencies. The microphone components and the acoustic space in which the microphone is located will cause the frequency response curve of the pickup system to be different. In addition, in order to obtain special effects, some pickup systems often deliberately adjust their frequency response curves, such as low-frequency enhancement. Due to the differences in the frequency response curves of each pickup system, the frequency spectrum of the same person's voice will be different after being recorded, which will eventually cause the accuracy of speech processing such as speech recognition and voiceprint recognition to decrease, and the accuracy will decrease under different pickup systems. Summary of the invention

[0004] In view of the above-mentioned problems such as spectrum differences caused by using different sound pickup devices to collect voice, the embodiment of the present application aims to propose a voice data enhancement method, device and readable medium based on an equalizer to solve the technical problems mentioned in the above background technology part.

[0005] In a first aspect, an embodiment of the present application provides a method for speech data enhancement based on an equalizer, comprising the following steps:

[0006] S1, obtaining a first speech having a first frequency response curve;

[0007] S2, using at least one different equalizer to process the first speech to obtain at least one second speech, each second speech having a second frequency response curve different from the first frequency response curve, the first speech and the second speech corresponding to the first speaker;

[0008] S3, repeat steps S1-S2 to obtain first speech and second speech corresponding to multiple different speakers, and obtain training data of the speech model based on the first speech and second speech of the multiple different speakers.

[0009] In some embodiments, step S1 specifically includes: collecting a first speech spoken by a first speaker.

[0010] In some embodiments, step S1 specifically includes: extracting a first speech from speech data of a first speaker.

[0011] In some embodiments, step S2 further includes: generating at least one different equalizer by adjusting gain values ​​of different frequency bands.

[0012] In some embodiments, the method further includes: an equalizer adjusting the first speech having the first frequency response curve to become the second speech having the second frequency response curve.

[0013] In some embodiments, the speech model includes a deep learning-based speech recognition model or a voiceprint recognition model.

[0014] In some embodiments, the equalizer comprises a digital equalizer.

[0015] In a second aspect, an embodiment of the present application provides a speech data enhancement device based on an equalizer, comprising:

[0016] A data acquisition module, configured to acquire a first speech having a first frequency response curve;

[0017] an equalizer processing module, configured to process the first speech using at least one different equalizer to obtain at least one second speech, each second speech having a second frequency response curve different from the first frequency response curve, the first speech and the second speech corresponding to a first speaker;

[0018] The training data generation module is configured to repeatedly execute the data acquisition module and the equalizer processing module to obtain the first speech and the second speech corresponding to multiple different speakers, and obtain the training data of the speech model based on the first speech and the second speech of the multiple different speakers.

[0019] In a third aspect, an embodiment of the present application provides an electronic device comprising one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement a method as described in any implementation manner in the first aspect.

[0020] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0021] Compared with the prior art, the present invention has the following technical effects:

[0022] (1) The present invention uses original data and processed data to train models based on deep learning such as speech recognition and voiceprint recognition to obtain models with better generalization capabilities.

[0023] (2) The speech model trained with the training data obtained by the present invention can effectively improve the accuracy across devices and reduce the errors caused by different audio response curves of devices.

[0024] (3) The present invention can enhance the voice data of multiple persons while using the same sound pickup system, and can also achieve the effect of collecting voice data under different sound pickup systems, and is convenient for adjusting the voice data, with low cost and simple operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 is an exemplary device architecture diagram to which an embodiment of the present application may be applied;

[0027] Figure 2 A schematic diagram of a flow chart of a method for enhancing speech data based on an equalizer according to an embodiment of the present invention;

[0028] Figure 3 A schematic diagram of an equalizer of a speech data enhancement method based on an equalizer according to an embodiment of the present invention;

[0029] Figure 4 A schematic diagram of a speech data enhancement device based on an equalizer according to an embodiment of the present invention;

[0030] Figure 5 It is a schematic diagram of the structure of a computer device suitable for implementing the electronic device of the embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] Figure 1An exemplary device architecture 100 is shown to which an equalizer-based speech data enhancement method or an equalizer-based speech data enhancement device according to an embodiment of the present application can be applied.

[0033] like Figure 1 As shown, the device architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0034] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various applications, such as data processing applications, file processing applications, etc., can be installed on terminal devices 101, 102, 103.

[0035] Terminal devices 101, 102, 103 can be hardware or software. When terminal devices 101, 102, 103 are hardware, they can be various electronic devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc. When terminal devices 101, 102, 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software or software modules (for example, software or software modules used to provide distributed services), or they can be implemented as a single software or software module. No specific limitation is made here.

[0036] The server 105 may be a server that provides various services, such as a background data processing server that processes files or data uploaded by the terminal devices 101, 102, and 103. The background data processing server may process the acquired files or data and generate processing results.

[0037] It should be noted that the equalizer-based speech data enhancement method provided in the embodiment of the present application can be executed by the server 105 or by the terminal devices 101, 102, and 103. Accordingly, the equalizer-based speech data enhancement device can be set in the server 105 or in the terminal devices 101, 102, and 103.

[0038] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided as required. In the case where the processed data does not need to be obtained remotely, the above device architecture may not include a network, but only a server or a terminal device.

[0039] Figure 2A method for enhancing speech data based on an equalizer provided in an embodiment of the present application is shown, comprising the following steps:

[0040] S1, obtaining a first speech having a first frequency response curve.

[0041] In a specific embodiment, the first voice corresponds to the first speaker, that is, the first speaker can be used as a label to form a set of training data with the first voice to train the speech recognition module or the voiceprint recognition model. There are many ways to obtain the first voice. One is to directly collect the first voice spoken by the first speaker through a sound pickup system or a sound pickup device; the other is to extract the first voice from the voice data of the first speaker. The voice data can be voice data that has been collected and is known to correspond to the first speaker. A part or the whole of the voice data is intercepted as the first voice. What's more, one of the following second voices can be used as the first voice again for data enhancement to achieve fission data enhancement.

[0042] S2, using at least one different equalizer to process the first speech to obtain at least one second speech, each second speech having a second frequency response curve different from the first frequency response curve, the first speech and the second speech corresponding to the first speaker.

[0043] In a specific embodiment, the equalizer in step S2 is obtained in the following manner: at least one different equalizer is generated by adjusting the gain values ​​of different frequency bands. Specifically, the equalizer is a digital equalizer. Common digital equalizers such as Figure 3 As shown. The horizontal axis represents the center frequency, and the vertical axis represents the gain. Different gains can be adjusted at different center frequencies to achieve different effects. For example, to enhance the low-frequency effect, the gain of the low frequency can be adjusted to a positive number; to enhance the intermediate frequency effect, the gain of the intermediate frequency can be adjusted to a positive number; to enhance the high-frequency effect, the gain of the high frequency can be adjusted to a positive number. In this step, multiple equalizers can be generated using different parameters such as low-frequency enhancement, intermediate-frequency enhancement, and high-frequency enhancement to simulate the impact of different devices on the original voice. The equalizer can adjust the first voice with a first frequency response curve to a second voice with a second frequency response curve. The voice data processed by multiple different equalizers can replace the original voice collected from multiple different devices, thereby obtaining a second voice with multiple different second frequency response curves without using multiple different pickup devices. Although the second frequency response curve is different from the first frequency response curve, they all belong to the first speaker. Therefore, the data enhancement method proposed in the embodiment of the present application can effectively reduce the equipment cost, and has stronger controllability. It can be adjusted according to the specific situation, and can also be adjusted according to the needs, so that the second voice obtains a second frequency response curve different from the frequency response curve of the first voice.

[0044] S3, repeat steps S1-S2 to obtain first speech and second speech corresponding to multiple different speakers, and obtain training data of the speech model based on the first speech and second speech of the multiple different speakers.

[0045] In a specific embodiment, the speech model includes a speech recognition model or a voiceprint recognition model based on deep learning. The first speech and the second speech are used to train the speech recognition, voiceprint recognition and other deep learning-based models to obtain a model with better generalization ability, which can effectively improve the accuracy of speech processing technologies such as speech recognition and voiceprint recognition across devices. The training process of the speech model is consistent with the conventional speech recognition model training and voiceprint model training, so it will not be repeated here. The speech model obtained by training with the training data can effectively improve the accuracy across devices and reduce the errors caused by different audio response curves of the devices.

[0046] The following is based on the ResNetSe34 network model. Only the ResNetSe34 network model is used as a comparative example. The ResNetSe34 network model is used in combination with the above-mentioned equalizer-based speech data enhancement method as Example 1. The operating results are as follows:

[0047] Network structure EER Comparative Example ResNetSe34 1.35 Example 1 ResNetSe34+ speech data enhancement method based on equalizer 1.21

[0048] As shown in the above table, the EER index of the ResNetSe34 network model using the equalizer-based speech data enhancement method on voxceleb1_test shows that after adding the equalizer-based speech data enhancement method, the EER is relatively reduced by 10.3% under the same network structure, where EER means equal error rate.

[0049] Further references Figure 4 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a speech data enhancement device based on an equalizer, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0050] The embodiment of the present application provides a speech data enhancement device based on an equalizer, comprising:

[0051] A data acquisition module 1 is configured to acquire a first speech having a first frequency response curve;

[0052] an equalizer processing module 2, configured to process the first speech using at least one different equalizer to obtain at least one second speech, each second speech having a second frequency response curve different from the first frequency response curve, the first speech and the second speech corresponding to a first speaker;

[0053] The training data generation module 3 is configured to repeatedly execute the data acquisition module and the equalizer processing module to obtain the first speech and the second speech corresponding to multiple different speakers, and obtain the training data of the speech model based on the first speech and the second speech of the multiple different speakers.

[0054] In a specific embodiment, the first voice corresponds to the first speaker, that is, the first speaker can be used as a label to form a set of training data with the first voice to train the speech recognition module or the voiceprint recognition model. There are many ways to obtain the first voice. One is to directly collect the first voice spoken by the first speaker through a sound pickup system or a sound pickup device; the other is to extract the first voice from the voice data of the first speaker. The voice data can be voice data that has been collected and is known to correspond to the first speaker. A part or the whole of the voice data is intercepted as the first voice. What's more, one of the following second voices can be used as the first voice again for data enhancement to achieve fission data enhancement.

[0055] In a specific embodiment, the equalizer in the equalizer processing module 2 is obtained in the following manner: at least one different equalizer is generated by adjusting the gain values ​​of different frequency bands. Specifically, the equalizer is a digital equalizer. Common digital equalizers such as Figure 3 As shown. The horizontal axis represents the center frequency, and the vertical axis represents the gain. Different gains can be adjusted to achieve different effects at different center frequencies. For example, to enhance the low-frequency effect, the gain of the low frequency can be adjusted to a positive number; to enhance the intermediate frequency effect, the gain of the intermediate frequency can be adjusted to a positive number; to enhance the high-frequency effect, the gain of the high frequency can be adjusted to a positive number. The device can generate multiple equalizers using different parameters such as low-frequency enhancement, intermediate-frequency enhancement, and high-frequency enhancement to simulate the impact of different devices on the original voice. The equalizer can adjust the first voice with a first frequency response curve to a second voice with a second frequency response curve. The voice data processed by multiple different equalizers can replace the original voice collected from multiple different devices, thereby obtaining a second voice with multiple different second frequency response curves without using multiple different pickup devices. Although the second frequency response curve is different from the first frequency response curve, they all belong to the first speaker. Therefore, the data enhancement device proposed in the embodiment of the present application can effectively reduce the equipment cost, and has stronger controllability. It can be adjusted according to the specific situation, and can also be adjusted according to the needs, so that the second voice obtains a second frequency response curve different from the frequency response curve of the first voice.

[0056] In a specific embodiment, the speech model includes a speech recognition model or a voiceprint recognition model based on deep learning. The first speech and the second speech are used to train the speech recognition, voiceprint recognition and other deep learning-based models to obtain a model with better generalization ability, which can effectively improve the accuracy of speech processing technologies such as speech recognition and voiceprint recognition across devices. The training process of the speech model is consistent with the conventional speech recognition model training and voiceprint model training, so it will not be repeated here.

[0057] Reference below Figure 5 , which shows an electronic device (eg Figure 1 A schematic diagram of the structure of a computer device 500 (a server or terminal device as shown). Figure 5 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0058] like Figure 5 As shown, the computer device 500 includes a central processing unit (CPU) 501 and a graphics processing unit (GPU) 502, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 503 or the program loaded from the storage part 509 to the random access memory (RAM) 504. In the RAM 504, various programs and data required for the operation of the device 500 are also stored. The CPU 501, GPU 502, ROM 503 and RAM 504 are connected to each other through a bus 505. An input / output (I / O) interface 506 is also connected to the bus 505.

[0059] The following components are connected to the I / O interface 506: an input section 507 including a keyboard, a mouse, etc.; an output section 508 including a display such as a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 509 including a hard disk, etc.; and a communication section 510 including a network interface card such as a LAN card, a modem, etc. The communication section 510 performs communication processing via a network such as the Internet. A drive 511 can also be connected to the I / O interface 506 as needed. A removable medium 512, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 511 as needed, so that a computer program read therefrom is installed into the storage section 509 as needed.

[0060] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part 510, and / or installed from a removable medium 512. When the computer program is executed by a central processing unit (CPU) 501 and a graphics processing unit (GPU) 502, the above-mentioned functions defined in the method of the present application are executed.

[0061] It should be noted that the computer-readable medium described in the present application may be a computer-readable signal medium or a computer-readable medium or any combination of the above two. The computer-readable medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor device, device or component, or any combination of the above. More specific examples of computer-readable media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution device, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable medium that can send, propagate or transmit a program for use by or in conjunction with an instruction execution device, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0062] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0063] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the device, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based device that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0064] The modules involved in the embodiments described in this application may be implemented by software or hardware, and the modules described may also be set in a processor.

[0065] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or it may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device: obtains a first speech with a first frequency response curve; uses at least one different equalizer to process the first speech to obtain at least one second speech, the at least one second speech has at least one second frequency response curve different from the first frequency response curve, and the first speech and the second speech correspond to a first speaker; repeats the above steps to obtain the first speech and the second speech corresponding to multiple different speakers, and obtains the training data of the speech model based on the first speech and the second speech of the multiple different speakers.

[0066] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A speech data enhancement method based on an equalizer, characterized in that: The following steps are involved: S1, acquiring a first speech with a first frequency response curve by using the same sound pickup system; S2, generating at least two different equalizers by adjusting gain values ​​of different frequency bands, and using at least two different equalizers to process the first speech to obtain at least two second speech, each second speech having a second frequency response curve different from the first frequency response curve, and the first speech and the second speech corresponding to the first speaker; S3, repeat steps S1-S2 to obtain first speech and second speech corresponding to multiple different speakers, and obtain training data of the speech model based on the first speech and second speech of the multiple different speakers.

2. The equalizer-based speech data enhancement method according to claim 1, characterized in that: The step S1 specifically includes: collecting the first speech spoken by the first speaker.

3. The equalizer-based speech data enhancement method according to claim 1, characterized in that: The step S1 specifically includes: extracting the first speech from the speech data of the first speaker.

4. The equalizer-based speech data enhancement method according to claim 1, characterized in that: The equalizer adjusts the first voice having the first frequency response curve to the second voice having the second frequency response curve.

5. The equalizer-based speech data enhancement method according to claim 1, characterized in that: The speech model includes a speech recognition model or a voiceprint recognition model based on deep learning.

6. The equalizer-based speech data enhancement method according to claim 1, characterized in that: The equalizer comprises a digital equalizer.

7. A speech data enhancement device based on an equalizer, characterized in that: include: A data acquisition module is configured to acquire a first speech having a first frequency response curve while using the same sound pickup system; an equalizer processing module, configured to generate at least two different equalizers by adjusting gain values ​​of different frequency bands, and process the first speech with the at least two different equalizers to obtain at least two second speech, each second speech having a second frequency response curve different from the first frequency response curve, and the first speech and the second speech corresponding to a first speaker; The training data generation module is configured to repeatedly execute the data acquisition module and the equalizer processing module to obtain the first speech and the second speech corresponding to multiple different speakers, and obtain the training data of the speech model based on the first speech and the second speech of the multiple different speakers.

8. An electronic device, comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images