Speech enhancement recording device and method

Abstract

The invention provides voice enhancement recording equipment, and belongs to the technical field of recording. Comprising an audio recording module, an audio processing module, a virtual sound card and a loudspeaker. The audio recording module is used for receiving audio and transmitting the audio to the audio processing module; the audio processing module receives recorded original data audio; the audio processing module comprises a voice enhancement algorithm, and processes the original data audio through the voice enhancement algorithm; the processed original data audio is sent to the virtual sound card; the virtual sound card receives and processes original data audio and converts an original data audio signal into an analog audio signal, so that an audio data oscillogram after voice enhancement can be visually seen at a computer end through the virtual sound card generated by the UAC; the output end of the virtual sound card is connected with the loudspeaker, the loudspeaker plays the processed audio signal, and the enhanced audio can be listened through the player and the loudspeaker of the computer. The equipment is not easily limited by recording equipment, and the compatibility is higher.

Description

Technical Field

[0001] The present application relates to the field of recording technology, and in particular to a speech enhancement recording device and method. Background Art

[0002] When developing software and hardware in the fields of intelligent speech recognition, Internet of Things, robot control, etc., it is necessary to collect audio and use a large amount of audio to train the product to read voice commands. However, the actual audio collection scenario is often accompanied by noisy noises such as machine operation. In order to accurately identify the content of the voice or collect audio in a specific range, it is necessary to use audio processing software to process the collected audio. Therefore, existing voice enhancement recording equipment generally first collects digital audio signals through recording equipment, and then uses algorithms to perform noise reduction and other processing on the digital audio signals to improve voice quality. Finally, the processed audio is sent to the input end of the external device, and the external device performs operations such as command recognition.

[0003] For example, the patent with application number CN202122103122.1 discloses a professional recording sound card, including a main control module and a power supply control module, an input audio processing module and an output audio processing module respectively connected to the main control module, the main control module includes a main control chip U2, the model of the main control chip is CM6635, the main control chip is connected to a USB interface, an SPDIF input interface, an SPDIF output interface and an SPI interface, the power supply control module includes a power input voltage stabilizing circuit, a power conversion circuit and a noise reduction circuit, the input audio processing module includes an input audio connection circuit and an input audio conversion circuit, and the input audio conversion circuit includes a chip U11.

[0004] However, when testing audio processing or artificial intelligence equipment, it is necessary to connect to various new types of equipment. When the output end of the voice enhancement equipment is connected to these devices, it is easy to be incompatible with the limitations of the digital audio interface, which affects the efficiency of audio transmission and may even cause problems such as inability to connect and transmit. Furthermore, if a special voice conversion device is added, it will need to be frequently replaced when testing different products, increasing the workload. Summary of the invention

[0005] The purpose of the present invention is to solve the problem in the prior art that the recording data output by the recording device is easily incompatible with the sound processing device. Therefore, the present application provides a speech enhancement recording device and method, which can process the sound after recording to achieve the purpose of noise reduction, etc., and can output analog audio data at the same time, is not easily restricted by the recording device interface, and has higher compatibility.

[0006] The present application embodiment provides a speech enhancement recording device, characterized in that it includes:

[0007] Audio input module, audio processing module, virtual sound card and speaker;

[0008] The audio input module is connected to the audio processing module; the audio input module is used to receive audio and transmit the audio to the audio processing module;

[0009] The audio processing module receives the recorded original data audio;

[0010] The audio processing module includes a speech enhancement algorithm, and processes the original data audio by the speech enhancement algorithm;

[0011] The audio processing module is connected to the virtual sound card and sends the processed raw data audio to the virtual sound card;

[0012] The input end of the virtual sound card is connected to the audio processing module, receives the processed original audio data, and converts the original audio data signal into an analog audio signal;

[0013] The output end of the virtual sound card is connected to the speaker, and the speaker plays the processed audio signal.

[0014] By adopting the above technical scheme, an audio receiving module is used to convert the collected analog audio signal into a digital audio signal, and then the digital audio signal is transmitted to the audio processing module to perform noise reduction and other processing on the audio data. A virtual sound card can be generated according to a script, does not take up space, and is worry-free to operate. The processed digital audio signal can be converted into an analog audio signal and transmitted to the UAC sound card on the PC. With the help of an external speaker, voice-enhanced recording and playback can be achieved. In the prior art, when people use recording equipment to record voice-enhanced sounds, they often need to use special equipment or single-chip microcomputers to enhance the sound effects. The enhanced audio is then transmitted to a computer or other storage device. However, in many cases, the interface or protocol of the storage device is prone to incompatibility with the sound processing device, resulting in the inability to record sound. The present application adds a virtual sound card to a sound processing device such as a device or a single-chip microcomputer so that the sound processing device can obtain digital audio after noise reduction enhancement, which has better recording audio playback quality and compatibility with recording equipment.

[0015] In some embodiments, the audio input module includes a microphone array and a sound card;

[0016] The microphone array collects multi-channel audio data and transmits the multi-channel audio data to the sound card;

[0017] The sound card converts the multi-channel audio data into a multi-channel digital audio signal, and transmits the multi-channel digital audio signal to the audio processing module.

[0018] Using a microphone array for recording helps reduce the blur and distortion of the sound, and improves the clarity and quality of the sound. In addition, the microphone array optimizes and enhances the voice signal, which can improve the accuracy of voice recognition and interaction. The sound card bound to the microphone array converts the recorded analog audio signal into a digital signal, which facilitates the subsequent signal processing.

[0019] In some embodiments, the audio processing module receives multiple channels of digital audio data.

[0020] The speech enhancement algorithm includes a first function;

[0021] The multi-channel digital audio data is processed using the first function, and the multi-channel digital audio data is merged into mono-channel digital audio data.

[0022] The first function is used to process the multi-channel audio data, and the multi-channel audio can be merged into a single-channel audio according to the frame number and channel number of the multi-channel audio data, the distance between microphones and other data, thereby improving the quality of recording.

[0023] In some embodiments, the audio input module is configured as a data interface, the data interface is connected to an external recording device, and receives the original analog digital audio recorded by the external recording device.

[0024] The above technical solution can provide a higher degree of freedom and can connect various recording devices to process the audio data to achieve high-quality recording.

[0025] In some embodiments, a message receiving module is also included;

[0026] The message receiving module receives the recording request and connects with the audio input module;

[0027] The message receiving module determines whether to start recording according to the recording request, and sends different signals to the audio recording module;

[0028] The audio recording module receives the signal and starts or stops recording according to the signal.

[0029] In some embodiments, the recording request is transmitted to the message receiving module through the netlink mechanism. The message receiving module uses the recvmsg system call to receive the recording request, and parses the received netlink message according to the uevent format, and determines whether to start recording according to the parsed netlink message.

[0030] In some embodiments, the signal is set to a flag value;

[0031] The message receiving module sets different flag values ​​according to different recording requests;

[0032] The audio recording module reads the value of the flag and starts or stops recording according to the value of the flag.

[0033] In some embodiments, if the message receiving module receives a uevent to start recording, the flag value is set to 0;

[0034] If the message receiving module receives the uevent to stop recording, the flag value is set to 1;

[0035] If the message receiving module receives a uevent that is not a recording, the flag value is not set;

[0036] The audio input module initially sets the flag value to 1;

[0037] When the audio recording module reads that the flag value is set to 1, the recording stops; when the audio recording module reads that the flag value is set to 0, the recording starts.

[0038] By adopting the above technical solution, a quick response mechanism for starting and ending recording is established, and the netlink mechanism is used to achieve a quick response for starting and ending recording.

[0039] In some embodiments, further comprising a development board;

[0040] The audio input module and the audio processing module are integrated into the development board;

[0041] The development board has a UAC module, and the virtual sound card is generated based on the UAC module built into the development board;

[0042] The development board is connected to the speaker, and the speaker plays the processed audio.

[0043] In the prior art, the development board can be used to develop and test software or hardware in the fields of audio equipment, artificial intelligence, speech recognition, etc. Some of the components need to collect high-quality audio data, but they are prone to incompatibility with existing audio processing equipment. This application can use the UAC module that comes with the development board to generate a virtual sound card, and use the virtual sound card to convert the processed digital audio signal into an analog audio signal, so that the development board has greater compatibility during actual use and can be connected to various types of recording equipment. At the same time, it does not take up space and can simply realize voice enhancement recording.

[0044] In one embodiment, the speech enhancement recording process is as follows:

[0045] A. collecting a first digital audio signal through the audio receiving module;

[0046] B. transmitting the first digital signal to the audio processing module;

[0047] C. Processing the first digital audio signal to form a second digital audio signal, and transmitting the second digital audio signal to the virtual sound card;

[0048] D. converting the second digital audio signal into an analog audio signal through the virtual sound card;

[0049] E. sending the analog audio signal to the speaker;

[0050] F. The speaker receives and plays the analog audio signal.

[0051] Through the above steps, the noise-reduced and enhanced audio data can be obtained after sound processing, thereby improving the recording sound quality.

[0052] Other features and corresponding beneficial effects of the present invention are described in the latter part of the specification, and it should be understood that at least part of the beneficial effects become obvious from the description in the specification of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 Flowchart provided for the embodiment of the present application;

[0054] Figure 2 The waveform diagram of the original 4-channel audio provided by the embodiment of the present application;

[0055] Figure 3 This is a 1-channel audio after speech enhancement processing provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The following specific embodiments illustrate the implementation of the present invention, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Although the description of the present invention will be introduced in conjunction with the preferred embodiment, this does not mean that the features of this invention are limited to this implementation. On the contrary, the purpose of introducing the invention in conjunction with the implementation is to cover other options or modifications that may extend based on the claims of the present invention. In order to provide a deep understanding of the present invention, the following description will include many specific details. The present invention can also be implemented without using these details. In addition, in order to avoid confusion or blurring the focus of the present invention, some specific details will be omitted in the description. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0057] It should be noted that in this specification, similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0058] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] In the description of the present invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc. indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description created by the present application, unless otherwise specified, "multiple" means two or more.

[0060] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0061] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0062] See also Figure 1-3 , Figure 1 Flowchart provided for the embodiment of the present application; Figure 2 The waveform diagram of the original 4-channel audio provided by the embodiment of the present application; Figure 3 The 1-channel audio after speech enhancement processing provided in the embodiment of the present application

[0063] like Figure 1As shown, the embodiment of the present application provides a speech enhancement recording device, including:

[0064] Audio input module, audio processing module, virtual sound card and speaker;

[0065] The analog audio signal audio input module is connected to the analog audio signal audio processing module; the analog audio signal audio input module is used to receive audio and transmit the analog audio signal audio to the analog audio signal audio processing module;

[0066] The analog audio signal audio processing module receives the recorded raw data audio;

[0067] The analog audio signal audio processing module includes a speech enhancement algorithm, and processes the original data audio of the analog audio signal through the analog audio signal speech enhancement algorithm;

[0068] The analog audio signal audio processing module is connected to the analog audio signal virtual sound card, and sends the processed analog audio signal data audio to the analog audio signal virtual sound card;

[0069] The output end of the analog audio signal virtual sound card is connected to the analog audio signal speaker, and the analog audio signal speaker records the processed audio signal.

[0070] Among them, in the prior art, conventional recording equipment records sound, processes it, and then directly outputs digital audio data for transmission or storage. However, digital audio is easily restricted by various protocols and interfaces, and incompatibility issues may occur, resulting in the inability to generate or store audio data. The present application utilizes the virtual sound card generated by UAC to perform algorithmic noise reduction and enhancement processing on the recorded audio, and ultimately obtains audio data after voice enhancement.

[0071] like Figure 2 and Figure 3 As shown, in one embodiment, the analog audio signal audio input module includes a microphone array and a sound card;

[0072] The analog audio signal microphone array collects 4-channel audio data and transmits the analog audio signal 4-channel audio data to the analog audio signal sound card;

[0073] The analog audio signal sound card converts the analog audio signal multi-channel audio data into a multi-channel digital audio signal, and transmits the analog audio signal multi-channel digital audio signal to the analog audio signal audio processing module.

[0074] Specifically, the sound card may be a physical sound card or a virtual sound card. The sound card here is used to convert analog audio data recorded by a microphone into digital audio data to facilitate subsequent sound processing.

[0075] In one embodiment, the analog audio signal audio input module is configured as an audio interface, which can be connected to an external recording device and receive a digital audio signal recorded by the recording device.

[0076] Specifically, the external recording device should ensure that the audio data sent to the audio processing module is digital audio data.

[0077] In one implementation, the analog audio signal audio processing module receives multiple channels of digital audio data.

[0078] The analog audio signal speech enhancement algorithm includes a first function;

[0079] The analog audio signal first function is used to process the multi-channel digital audio data, and the multi-channel digital audio data of the analog audio signal are combined into mono-channel digital audio data.

[0080] Specifically, the first function is as follows:

[0081] void test(short*pcm,short*out,int num_sample,int num_channel,floatdistance);

[0082] The meanings of the various parameters are as follows: pcm: pointer to the input 16-bit 16k four-channel audio data;

[0083] out: pointer to the 16-bit 16k-channel audio data processed by the output algorithm;

[0084] num_sample: the number of frames of audio data processed at one time;

[0085] num_channel: the number of channels of the input audio;

[0086] distance: The distance between microphones.

[0087] During the processing, the first function constructed according to various related data can improve the sound quality, reduce the noise of the sound, and improve the audio quality.

[0088] In one embodiment, it also includes a message receiving module;

[0089] The analog audio signal message receiving module receives the recording request and connects with the analog audio signal audio recording module;

[0090] The analog audio signal message receiving module determines whether to start recording according to the analog audio signal recording request, and sends different signals to the analog audio signal audio recording module;

[0091] The analog audio signal audio recording module receives the analog audio signal and starts or stops recording according to the analog audio signal.

[0092] In one embodiment, an analog audio signal recording request is transmitted to an analog audio signal message receiving module through a netlink mechanism. The analog audio signal message receiving module uses a recvmsg system call to receive the analog audio signal recording request, and parses the received netlink message according to the uevent format, and determines whether to start recording based on the parsed netlink message.

[0093] Specifically, the netlink mechanism is mainly used for the transmission of network configuration and management information. It supports multiple protocols, has better compatibility, and has higher transmission efficiency.

[0094] Specifically, there are two types of uevent that need to be processed:

[0095] One is that the uevent for recording starts is:

[0096] strs[0] = ACTION = change;

[0097] strs[1] = DEVPATH = / devices / virtual / u_audio / UAC2_Gadget 0 (the path will be different depending on the device path);

[0098] strs[2]=SUBSYSTEM=u_audio;

[0099] strs[3]=USB_STATE=SET_INTERFACE;

[0100] strs[4]=STREAM_DIRECTION=IN;

[0101] strs[5]=STREAM_STATE=ON;

[0102] The other is the uevent at the end of recording

[0103] strs[0] = ACTION = change

[0104] strs[1] = DEVPATH = / devices / virtual / u_audio / UAC2_Gadget 0 (the path will be different depending on the device path);

[0105] strs[2]=SUBSYSTEM=u_audio;

[0106] strs[3]=USB_STATE=SET_INTERFACE;

[0107] strs[4]=STREAM_DIRECTION=IN;

[0108] strs[5]=STREAM_STATE=OFF.

[0109] In one embodiment, the analog audio signal is set to a flag value;

[0110] The analog audio signal message receiving module sets different flag values ​​according to different recording requests;

[0111] The analog audio signal audio recording module reads the value of the analog audio signal flag and starts or stops recording according to the value of the analog audio signal flag.

[0112] In one embodiment, if the analog audio signal message receiving module receives a uevent to start recording, the flag value is set to 0;

[0113] If the analog audio signal message receiving module receives the uevent to stop recording, the flag value is set to 1;

[0114] If the analog audio signal message receiving module receives a uevent that is not a recording, the flag value will not be set;

[0115] The analog audio signal audio input module initially sets the flag value to 1;

[0116] When the analog audio signal audio input module reads the analog audio signal flag value as 1, the recording stops; when the analog audio signal flag value is set to 0, the recording starts.

[0117] In one embodiment, a development board is also included.

[0118] The analog audio signal audio input module and the analog audio signal audio processing module are integrated into the analog audio signal development board;

[0119] The analog audio signal development board has a UAC module, and the analog audio signal virtual sound card is generated based on the UAC module built into the analog audio signal development board;

[0120] The analog audio signal development board is connected to the analog audio signal speaker, and the analog audio signal speaker records the processed audio.

[0121] Specifically, the development board can be the RK3308 development board

[0122] In one embodiment, a speech enhancement recording device for simulating an audio signal according to any one of claims 1 to 8 is provided, and the speech enhancement recording process is as follows:

[0123] A. collecting a first digital audio signal through an analog audio signal audio receiving module;

[0124] B. transmitting the first digital signal to the analog audio signal audio processing module;

[0125] C. Processing the analog audio signal and the first digital audio signal to form a second digital audio signal, and transmitting the analog audio signal and the second digital audio signal to the analog audio signal virtual sound card;

[0126] D. converting the analog audio signal and the second digital audio signal into an analog audio signal through an analog audio signal virtual sound card;

[0127] E. sending the analog audio signal to the analog audio signal speaker;

[0128] F. Analog audio signal The speaker receives the analog audio signal and records it.

[0129] In the actual operation of this embodiment, the PC or other processor sends a uevent requesting time and terminal, and the message receiving module parses the uevent attribute to learn the event process. When the uevent of the recording request to start or end is parsed, the flag is set. The audio receiving module judges the flag value. When the flag value symbolizing the start of recording is read, the 4-channel data is first recorded from the actual physical sound card of the microphone array, and then processed using the speech enhancement algorithm. The waveforms before and after the processing are shown in the figure below. Figure 2 and Figure 3 As shown, after the audio data is processed, the audio is sent to the virtual sound card, which converts the digital audio signal into an analog audio signal, and then sends the analog audio signal to the speaker of the PC or other processor, thereby realizing the voice enhanced recording process.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A speech enhancement recording device, characterized in that: include: Audio input module, audio processing module, virtual sound card and speaker; The audio input module is connected to the audio processing module; the audio input module is used to receive audio and transmit the audio to the audio processing module; The audio processing module receives the recorded original data audio; The audio processing module includes a speech enhancement algorithm, and processes the original data audio by the speech enhancement algorithm; The audio processing module is connected to the virtual sound card and sends the processed raw data audio to the virtual sound card; The input end of the virtual sound card is connected to the audio processing module, receives the processed original audio data, and converts the original audio data signal into an analog audio signal; The output end of the virtual sound card is connected to the speaker, and the speaker records the processed audio signal.

2. A speech enhancement recording device as claimed in claim 1, characterized in that: The audio input module includes a microphone array and a sound card; The microphone array collects multi-channel audio data and transmits the multi-channel audio data to the sound card; The sound card converts the multi-channel audio data into a multi-channel digital audio signal, and transmits the multi-channel digital audio signal to the audio processing module.

3. A speech enhancement recording device as claimed in claim 2, characterized in that: The audio processing module receives multiple channels of digital audio data. The speech enhancement algorithm includes a first function; The multi-channel digital audio data is processed using the first function, and the multi-channel digital audio data is merged into mono-channel digital audio data.

4. A speech enhancement recording device as claimed in claim 1, characterized in that: The audio input module is configured as a data interface, the data interface is connected to an external recording device, and receives the original analog digital audio recorded by the external recording device.

5. A speech enhancement recording device as claimed in claim 1, characterized in that: Also includes a message receiving module; The message receiving module receives the recording request and connects with the audio input module; The message receiving module determines whether to start recording according to the recording request, and sends different signals to the audio recording module; The audio recording module receives the signal and starts or stops recording according to the signal.

6. A speech enhancement recording device as claimed in claim 5, characterized in that: The recording request is transmitted to the message receiving module through the netlink mechanism. The message receiving module uses the recvmsg system call to receive the recording request, parses the received netlink message according to the uevent format, and determines whether to start recording according to the parsed netlink message.

7. A speech enhancement recording device as claimed in claim 4, characterized in that: The signal is set to a flag value; The message receiving module sets different flag values ​​according to different recording requests; The audio recording module reads the value of the flag and starts or stops recording according to the value of the flag.

8. A speech enhancement recording device as claimed in claim 7, characterized in that: If the message receiving module receives the uevent to start recording, the flag value is set to 0; If the message receiving module receives the uevent to stop recording, the flag value is set to 1; If the message receiving module receives a uevent that is not a recording, the flag value is not set; The audio input module initially sets the flag value to 1; When the audio recording module reads that the flag value is set to 1, the recording stops; when the audio recording module reads that the flag value is set to 0, the recording starts.

9. The speech enhancement recording device according to claim 1, characterized in that: Also includes development boards, The audio input module and the audio processing module are both integrated into the development board; The development board has a UAC module, and the virtual sound card is generated based on the UAC module built into the development board; The development board is connected to the speaker, and the speaker plays the processed audio.

10. A method for voice enhancement recording, characterized in that: The speech enhancement recording device according to any one of claims 1 to 9 is characterized in that the speech enhancement recording process is as follows: A. collecting a first digital audio signal through the audio receiving module; B. transmitting the first digital signal to the audio processing module; C. performing algorithm processing on the first digital audio signal to form a second digital audio signal, and transmitting the second digital audio signal to the virtual sound card; D. converting the second digital audio signal into an analog audio signal through the virtual sound card; E. sending the analog audio signal to the speaker; F. The speaker receives and plays the analog audio signal.

Images