Audio data processing method, device and storage medium for dual-channel communication link

Abstract

The present invention discloses an audio data processing method, device and storage medium of a dual-channel communication link, the method comprising: acquiring audio data, the audio data comprising sound data collected by at least two microphones; judging whether there is faulty audio data in the audio data; if so, removing the faulty audio data from the audio data to obtain target audio data, otherwise, selecting the best sound data from the audio data as the target audio data; removing non-human voice data in the target audio data to obtain denoised audio data; performing sound enhancement processing on the denoised audio data to obtain enhanced audio data; and outputting the enhanced audio data to a mixing console. The present invention can not only ensure that when one microphone fails, the output of audio data can still be achieved, but also effectively avoid the problem of affecting the audio quality due to damage to a certain microphone, thereby improving the audio quality of a conference system and the redundancy capacity of the conference system.

Description

Technical Field

[0001] The present invention belongs to the technical field of audio processing, and in particular relates to an audio data processing method, device and storage medium for a dual-channel communication link. Background Art

[0002] The digital conference discussion system is a conference system that integrates computer, communication, control, multimedia, image and audio technologies. It can organically connect independent subsystems such as conference registration, speech, voting, translation, video, audio, display and network access into one. The central control computer coordinates the work of each subsystem according to the meeting agenda, thereby providing the most accurate and timely information and services for various large-scale international conferences, academic seminars and remote conferences.

[0003] At present, in digital conference discussion systems, single-channel communication is mostly used. Although dual-communication links have appeared, they can only realize the function of one in use and one in backup, and it is impossible to use the dual-channel link to transmit audio at the same time, and there is no audio processing method for the dual-channel communication link; at the same time, smart devices with remote interaction capabilities in conference systems mostly use microphone arrays, and the microphone array includes multiple microphones. Due to the life of components, dust blockage or other human factors, one or more microphones in the microphone array may malfunction, such as being unable to collect sound or having continuous interference sound. This will cause the signal-to-noise ratio of the collected sound to drop significantly, and the clarity of the collected sound will be greatly affected, thereby affecting the voice recognition ability of the smart device; therefore, it is urgent to provide an audio data processing method based on a dual-channel communication link to improve the quality of audio data in the conference system. Summary of the invention

[0004] The purpose of the present invention is to provide an audio data processing method, device and storage medium for a dual-channel communication link to solve the problem that the existing conference system cannot use the dual-channel communication link for audio transmission and the uploaded audio quality is poor.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The present invention provides an audio data processing method for a dual-channel communication link, comprising:

[0007] Acquiring audio data, wherein the audio data includes sound data collected by at least two microphones;

[0008] Determining whether there is faulty audio data in the audio data;

[0009] If yes, then remove the faulty audio data from the audio data to obtain the target audio data; otherwise, select the best sound data from the audio data as the target audio data;

[0010] Removing non-human voice data from the target audio data to obtain denoised audio data;

[0011] Performing sound enhancement processing on the de-noised audio data to obtain enhanced audio data;

[0012] The enhanced audio data is output to a mixing console to complete the processing of the audio data.

[0013] Based on the above disclosed content, the present invention processes the sound data collected by two microphones, which is equivalent to processing the sound data collected by dual channels, that is, first determine whether there is faulty audio data in the audio data of the dual channels. If so, remove the faulty audio data and leave only the normal audio data for output. In this way, it can be ensured that when one channel fails, the output of audio data can still be achieved to ensure the normal operation of the conference system; and if there is no faulty audio data, the optimal sound data is selected from the audio data of the dual channels for output, which is equivalent to selecting the best audio quality for output; thus, the method of audio preferential output can effectively avoid the problem of audio quality being affected by damage to a certain microphone, thereby improving the audio quality of the conference system and the redundancy capacity of the conference system; in addition, the present invention can also perform non-human voice denoising and sound enhancement on the screened data (i.e., the target audio data), thereby further improving the quality of the output audio and ensuring the clarity of the audio of the conference system.

[0014] Through the above design, the present invention can remove faulty audio data in the audio data of dual channels, and when the audio of dual channels are both normal audio data, perform preferential output of audio data. Therefore, the present invention can not only ensure that when one channel fails, the output of audio data can still be achieved, but also effectively avoid the problem of audio quality being affected by damage to a certain microphone, thereby improving the audio quality of the conference system and the redundancy capacity of the conference system.

[0015] In one possible design, determining whether there is faulty audio data in the audio data includes:

[0016] Acquire the amplitude of each sound data in the audio data;

[0017] Calculating the similarity between each sound data and the preset audio data;

[0018] Determining whether the amplitude of each sound data is less than a preset amplitude and determining whether the similarity between each sound data and preset audio data exceeds a preset threshold range;

[0019] If so, the sound data with an amplitude smaller than the preset amplitude is used as the fault audio data and / or the sound data with a similarity exceeding a preset threshold range is used as the fault audio data.

[0020] Based on the above disclosed content, the present invention discloses a method for judging faulty audio data, that is, as long as any one of the above two conditions is met, it can be used as faulty audio data, that is, sound data with an amplitude less than a preset amplitude and / or a similarity exceeding a preset threshold range can be used as faulty audio data.

[0021] In a possible design, calculating the similarity between each sound data and the preset audio data includes:

[0022] Calculating the speech similarity between each sound data and the preset audio data, and calculating the frequency spectrum similarity between each sound data and the preset audio data;

[0023] Accordingly, determining whether the similarity between each sound data and the preset audio data exceeds a preset threshold range includes:

[0024] Determining whether the voice similarity between each sound data and the preset audio data exceeds a first preset threshold range;

[0025] If yes, determining whether the frequency spectrum similarity between each sound data and the preset audio data exceeds a second preset threshold range;

[0026] If yes, the sound data whose frequency spectrum similarity exceeds the second preset threshold range is used as the fault audio data.

[0027] Based on the above disclosed content, the present invention performs a double judgment by calculating the voice similarity and spectrum similarity between each sound data and the preset audio data. That is, when the voice similarity exceeds the first threshold range, the spectrum similarity between the two will be judged again. Only when the spectrum similarity exceeds the second threshold range, the corresponding sound data will be identified as faulty audio data. Through the above design, the accuracy of faulty audio data detection can be improved.

[0028] In one possible design, calculating the voice similarity between each sound data and the preset audio data includes:

[0029] Sampling each sound data according to a preset sampling frequency to obtain sampled data corresponding to each sound data;

[0030] Performing feature extraction on the sampled data corresponding to each sound data to obtain a speech feature vector corresponding to each sound data;

[0031] The cosine distance between the speech feature vector corresponding to each sound data and the speech feature vector of the preset audio data is calculated as the speech similarity.

[0032] In a possible design, calculating the frequency spectrum similarity between each sound data and the preset audio data includes:

[0033] Extracting environmental noise data corresponding to each sound data from each sound data;

[0034] Extracting features of the environmental noise data corresponding to each sound data to obtain a frequency spectrum feature vector corresponding to each sound data;

[0035] The cosine distance between the frequency spectrum feature vector corresponding to each sound data and the frequency spectrum feature vector of the preset audio data is calculated as the frequency spectrum similarity.

[0036] In a possible design, the optimal sound data is selected from the audio data, including:

[0037] Obtaining a signal-to-noise ratio of each sound data in the audio data;

[0038] The sound data with the largest signal-to-noise ratio is taken as the optimal sound data.

[0039] Based on the above disclosed content, the present invention discloses a specific method for selecting the optimal sound data, which is obtained by using the signal-to-noise ratio. The signal-to-noise ratio represents the ratio of signal to noise. The larger the signal-to-noise ratio, the smaller the noise mixed in the signal, and the higher the sound quality of the sound playback, and vice versa; therefore, the sound data with the largest signal-to-noise ratio represents the optimal sound data.

[0040] In one possible design, the method further includes:

[0041] If it is determined that each sound data in the audio data is faulty audio data, fault alarm information is generated to prompt the user that each of the at least two microphones is faulty.

[0042] Based on the above disclosed content, after determining that all sound data are faulty audio data, the present invention can issue a fault alarm message to prompt the user that the microphone in the conference system has a fault and cannot transmit audio normally, so that the user can perform maintenance in time.

[0043] In a second aspect, the present invention provides an audio data processing device for a dual-channel communication link, comprising: an acquisition unit, a determination unit, a screening unit, a denoising unit, a sound enhancement unit, and a sending unit;

[0044] The acquisition unit is used to acquire audio data, wherein the audio data includes sound data collected by at least two microphones;

[0045] The judging unit is used to judge whether there is faulty audio data in the audio data;

[0046] The screening unit is configured to remove the faulty audio data from the audio data to obtain target audio data when the judging unit judges that the faulty audio data is yes, and select the best sound data from the audio data as the target audio data when the judging unit judges that the faulty audio data is no;

[0047] The denoising unit is used to remove non-human voice data from the target audio data to obtain denoised audio data;

[0048] The sound enhancement unit is used to perform sound enhancement processing on the denoised audio data to obtain enhanced audio data;

[0049] The sending unit is used to output the enhanced audio data to the mixing console to complete the processing of the audio data.

[0050] In a third aspect, the present invention provides another audio data processing device for a dual-channel communication link, comprising a memory, a processor and a transceiver that are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the audio data processing method for the dual-channel communication link as in the first aspect or any possible design of the first aspect.

[0051] In a fourth aspect, the present invention provides a storage medium having instructions stored thereon, which, when executed on a computer, executes the audio data processing method of the dual-channel communication link as in the first aspect or any possible design of the first aspect.

[0052] In a fifth aspect, the present invention provides a computer program product comprising instructions, which, when executed on a computer, causes the computer to execute the audio data processing method of the dual-channel communication link as in the first aspect or any possible design of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A schematic diagram of the system architecture of a conference system with dual-channel communication links provided by the present invention;

[0054] Figure 2 A schematic flow chart of the steps of the audio data processing method of the dual-channel communication link provided by the present invention;

[0055] Figure 3 A flowchart of an audio data processing method for a dual-channel communication link provided by the present invention;

[0056] Figure 4 A flowchart of calculating the similarity between the sound data provided by the present invention and the preset audio data;

[0057] Figure 5 A training block diagram of the RNNnoise neural network model provided by the present invention;

[0058] Figure 6 A schematic diagram of the structure of an audio data processing device for a dual-channel communication link provided by the present invention;

[0059] Figure 7 This is a schematic diagram of the structure of the computer main device provided by the present invention. DETAILED DESCRIPTION

[0060] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. It should be noted that although the description of these embodiments is used to help understand the present invention, it does not constitute a limitation of the present invention. The specific structures and functional details disclosed herein are only used to describe the exemplary embodiments of the present invention. However, the present invention can be embodied in many alternative forms, and it should not be understood that the present invention is limited to the embodiments set forth herein.

[0061] It should be understood that although the terms first, second, etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another unit. For example, a first unit can be referred to as a second unit, and similarly, a second unit can be referred to as a first unit without departing from the scope of the exemplary embodiments of the present invention.

[0062] It should be understood that the term "and / or" that may appear in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, B exists alone, and A and B exist at the same time. The term " / and" that may appear in this article describes another type of association object relationship, indicating that two relationships may exist. For example, A / and B can represent two situations: A exists alone, and A and B exist alone. In addition, the character " / " that may appear in this article generally indicates that the previous and next associated objects are in an "or" relationship.

[0063] Example

[0064] like Figure 1 As shown, the present application provides a system architecture for application, that is, it is applied to a conference system, and the conference system uses the audio data processing method provided in this embodiment to perform audio processing; in this embodiment, the example conference system may include, but is not limited to: a communication-connected conference discussion unit and a host, wherein the conference discussion unit includes a plurality of microphones, and the plurality of microphones are arranged at different speaking positions in the conference room, and each microphone is provided with two microphones (microphone A and microphone B, that is, equivalent to being provided with two audio channels), wherein the microphone A of all the microphones is connected to the host through the first main control to form a first audio channel, and similarly, the microphone B of all the microphones is connected to the host through the second main control to form a second audio channel, so as to transmit the audio collected by the two microphones to the host through the corresponding audio channels; at the same time, when the host receives the audio data transmitted by the two channels, it uses the method provided in this embodiment to perform audio processing, so as to remove the faulty audio data in the audio data of the two channels, and when the audio in the two channels is normal, perform preferential output of the audio data, thereby ensuring that when one microphone fails, the output of the audio data can still be achieved, and avoiding the impact on the audio quality due to the damage of a certain microphone.

[0065] The audio data processing method of the dual-channel communication link provided in the first aspect of this embodiment can perform fault identification of the dual-channel audio data, so that when one audio channel fails, the faulty audio is removed and only the normal audio data is uploaded, thereby ensuring that when one microphone fails, the audio data can still be output; at the same time, when both audio channels are not faulty, the audio data can be preferentially output, thereby effectively avoiding the impact on the audio quality due to damage to a certain microphone, thereby improving the audio quality of the conference system and the redundancy capability of the conference system.

[0066] See also Figure 2 and Figure 3 As shown, the audio data processing method of the dual-channel communication link provided in the first aspect of this embodiment may include but is not limited to the following steps S1 to S6; of course, the aforementioned steps S1 to S6 may be but are not limited to being executed on the host side.

[0067] S1. Acquire audio data, wherein the audio data includes sound data collected by at least two microphones.

[0068] Step S1 is to obtain the sound data collected by the two microphones in any microphone in the conference room, so as to obtain dual-channel audio data; of course, more than two microphones can be set in the microphone to form a multi-channel communication link, and its processing method is consistent with that of the two channels. Therefore, this embodiment is described by taking two channels as an example, as shown in the following steps S2 to S6.

[0069] After receiving the audio data uploaded by the two microphones through the corresponding channels, the host can perform fault identification of the dual-channel audio, and then use different methods to process the audio according to the identification results. The fault identification process is shown in step S2 and its sub-steps.

[0070] S2. Determine whether there is faulty audio data in the audio data.

[0071] In this embodiment, judging whether fault audio data exists or not may include, but is not limited to, the following steps S21 to S24.

[0072] S21. Obtain the amplitude of each sound data in the audio data.

[0073] S22. Calculate the similarity between each sound data and the preset audio data.

[0074] In this embodiment, two indicators are used to determine whether the sound data is faulty audio data, namely: the amplitude of the sound data (that is, the frequency amplitude) and the similarity with the preset audio data. At the same time, as long as one of the above indicators is met, it can be identified as faulty audio data, as shown in the following steps S23 and S24.

[0075] S23. Determine whether the amplitude of each sound data is less than a preset amplitude and determine whether the similarity between each sound data and preset audio data exceeds a preset threshold range.

[0076] S24. If yes, then use the sound data with an amplitude smaller than the preset amplitude as the fault audio data and / or use the sound data with a similarity exceeding a preset threshold range as the fault audio data.

[0077] It can be known from the aforementioned steps S23 and S24 that as long as the amplitude of the sound data is smaller than the preset amplitude and / or the similarity exceeds the preset threshold range, the sound data can be identified as faulty audio data.

[0078] See also Figure 4 As shown, in this embodiment, the similarity between each sound data and the preset audio data is calculated for example, which may include but is not limited to: calculating the voice similarity between each sound data and the preset audio data, and calculating the frequency spectrum similarity between each sound data and the preset audio data. That is to say, this embodiment will calculate the voice similarity and the frequency spectrum similarity, thereby realizing dual detection.

[0079] The following provides a specific calculation process of speech similarity and spectrum similarity, as shown in the following steps A to F.

[0080] A. Sampling each sound data according to a preset sampling frequency to obtain sampling data corresponding to each sound data.

[0081] B. Perform feature extraction on the sampled data corresponding to each sound data to obtain a speech feature vector corresponding to each sound data.

[0082] C. Calculate the cosine distance between the speech feature vector corresponding to each sound data and the speech feature vector of the preset audio data as the speech similarity.

[0083] The principle of the above steps A to C is: sample each sound data according to a preset sampling frequency (such as 16000Hz or 48000Hz, etc.) to obtain sampled data, and then perform speech feature extraction to obtain a speech feature vector; similarly, the same method is used for the preset audio data to obtain its corresponding speech feature vector, and use it as the standard speech feature vector; finally, the cosine distance between the two is calculated to obtain the speech similarity between the two, that is, the value of the cosine distance between the two is used as the speech similarity.

[0084] In this embodiment, speech feature extraction is to extract a multidimensional feature vector from each speech signal (i.e., sound data), and feature extraction can be performed using, but is not limited to, perceptual linear prediction, linear prediction coding, or Mel-spectrogram cepstral coefficients. Taking Mel-spectrogram cepstral coefficient feature extraction as an example, the specific processing process is: pre-emphasize the sound data in turn (essentially, pass the sound data through a high-pass filter to enhance the high-frequency part, make the frequency of the signal flat, remove the spectrum tilt, and compensate for the suppressed high-frequency part of the speech signal), and then perform framing, windowing, and fast Fourier transform. The reason is that since the speech signal is a short-term stationary signal, framing can treat each frame as a stationary signal to reduce the changes between frames. Finally, the data processed as above is processed through Mel filtering and discrete cosine transform to obtain Mel-spectrogram cepstral coefficients. Therefore, speech feature extraction can be achieved based on Mel-spectrogram cepstral coefficients.

[0085] In this embodiment, the speech similarity between the sound data and the preset audio data is described by the cosine distance between the two, and the cosine distance represents the cosine value between the two speech feature vectors, which is calculated using the Euclidean dot product formula, wherein the smaller the cosine distance, the more similar the two are, and vice versa, the greater the difference between the two; therefore, it is only necessary to calculate the cosine distance between the speech feature vector of the sound data and the speech feature vector of the preset audio data to obtain the speech similarity between the sound data and the preset audio data; finally, according to the size of the cosine distance value, it is determined whether it is within the first preset threshold range, and thus it is possible to determine whether the sound data is similar to the preset audio data.

[0086] Similarly, the process of calculating the frequency spectrum similarity between the sound data and the preset audio data is as shown in the following steps D to F.

[0087] D. Extracting the environmental noise data corresponding to each sound data from each sound data.

[0088] E. Perform feature extraction on the environmental noise data corresponding to each sound data to obtain a frequency spectrum feature vector corresponding to each sound data.

[0089] F. Calculate the cosine distance between the frequency spectrum feature vector corresponding to each sound data and the frequency spectrum feature vector of the preset audio data as the frequency spectrum similarity.

[0090] When calculating the spectral similarity between the sound data and the preset audio data, the principle is consistent with the aforementioned speech similarity principle, that is, both calculate the cosine distance of the spectral feature vectors of the two. The difference is that this embodiment calculates the spectral similarity between the ambient noise data in the sound data and the preset audio data. That is to say, if the spectral similarity does not exceed the second preset threshold range, it means that the ambient noise of the sound data has not reached the fault standard, that is, it is not faulty audio data; otherwise, it means that the ambient noise of the sound data has reached the fault standard and can be identified as faulty audio data.

[0091] In this embodiment, the extraction of frequency spectrum features of sound data may be implemented by, but not limited to, using MATLAB software.

[0092] Therefore, when the faulty audio is judged according to the similarity in the aforementioned step S23, it will be divided into two judgments, that is, when the speech similarity exceeds the first preset range, it will be judged again whether the spectral similarity between the sound data and the preset audio data exceeds the second preset threshold range. If it exceeds, the sound data can be determined as faulty audio data; therefore, the similarity judgment process can include but is not limited to the following steps S231 to S233.

[0093] S231. Determine whether the voice similarity between each sound data and the preset audio data exceeds a first preset threshold range.

[0094] S232. If yes, determine whether the frequency spectrum similarity between each sound data and the preset audio data exceeds a second preset threshold range.

[0095] S233. If yes, use the sound data whose frequency spectrum similarity exceeds the second preset threshold range as the fault audio data.

[0096] Through the aforementioned steps S231 to S233, a dual judgment of the similarity between the sound data and the preset audio data can be achieved, thereby improving the accuracy of fault audio data detection.

[0097] In this embodiment, after determining in step S23 that the similarity between the sound data and the preset audio data exceeds the preset threshold range, the microphone corresponding to the sound data can be determined to be a faulty microphone. At this time, an alarm message can be generated to prompt the user to perform repairs in time to ensure the audio transmission quality of the conference system.

[0098] For example, based on the above example, there are two microphones A and B, wherein microphone A corresponds to the first sound data and microphone B corresponds to the second sound data. Therefore, when the similarity between the first sound data and the preset audio data exceeds the preset threshold range, microphone A can be identified as a faulty microphone.

[0099] Therefore, after the faulty audio data in the audio data is determined by judging the above two indicators, different methods can be used to upload the audio data according to the detection results, as shown in the following step S3.

[0100] S3. If yes, remove the faulty audio data from the audio data to obtain target audio data; otherwise, select the best sound data from the audio data as the target audio data.

[0101] In this embodiment, if one of the audio data is faulty audio data, the faulty audio data will be discarded, and only the normal audio data will be retained for output; for example, the audio data includes first sound data and second sound data. When it is detected that the first sound data is faulty audio data, the first sound data will be discarded, and only the second sound data will be retained as the target audio data; thereby, it can be ensured that when a microphone fails, the output of audio data can still be achieved.

[0102] If it is determined that there is no faulty audio data in the audio data, that is, all the sound data are normal audio data, then the best sound data will be selected from all the sound data as the target audio data, that is, preferential output will be performed. In this way, the problem of audio quality being affected by damage to a microphone can be avoided.

[0103] In addition, in this embodiment, if it is determined that each sound data in the audio data is faulty audio data, then a fault alarm message will be generated to prompt the user that each of the at least two microphones is faulty.

[0104] In this embodiment, for example, the best sound data is selected from the audio data, which may be, but is not limited to, using the following methods S31 and S32.

[0105] S31. Obtain the signal-to-noise ratio of each sound data in the audio data.

[0106] S32. Use the sound data with the largest signal-to-noise ratio as the optimal sound data.

[0107] The principle of step S31 and step S32 is: the signal-to-noise ratio represents the ratio of the signal to the noise. The larger the signal-to-noise ratio, the smaller the noise mixed in the signal, and the higher the sound quality of the sound playback, and vice versa; therefore, the sound data with the largest signal-to-noise ratio represents the optimal sound data.

[0108] In this embodiment, after the target audio data is obtained, in order to further improve the quality of the audio, the following steps S4 and S5 are provided to achieve noise removal and sound enhancement.

[0109] S4. Remove non-human voice data from the target audio data to obtain denoised audio data.

[0110] Step S4 is to remove non-human voice data in the target audio data, thereby reducing the noise in the audio; for example, removing non-steady-state noise such as keyboard sounds, knocking on the table, drinking water sounds and / or coughing sounds.

[0111] In this embodiment, the trained RNNnoise neural network model is used as an example to remove non-human voice data, wherein the training process of the RNNnoise neural network model is as follows:

[0112] See also Figure 5 As shown, a clean human voice signal is used as the data input source 1, the clean voice is first generalized by adding noise to obtain the noisy voice, and then the noisy voice is input into the RNNnosie neural network model for training to obtain the trained RNNnoise neural network model; finally, the target audio data can be input into the trained RNNnoise neural network model to obtain the denoised audio data.

[0113] The RNNnoise neural network model is essentially a recurrent neural network (RNN), which uses a GRU (Gate Recurrent Unit) network to extract features, train the neural network, and obtain weight data to achieve denoising.

[0114] The target audio data after the denoising process can be subjected to sound enhancement processing to further improve the sound quality, as shown in the following step S5.

[0115] S5. Perform sound enhancement processing on the noise-removed audio data to obtain enhanced audio data.

[0116] In this embodiment, the sound enhancement may be achieved by using, for example but not limited to, an automatic gain control algorithm (Automatic Gain Control, AGC) and a dynamic range control (Dynamic Range Control, DRC) algorithm.

[0117] The data after sound enhancement can be output to the mixing console, thereby completing the processing of the audio data, as shown in step S6.

[0118] S6. Output the enhanced audio data to the mixing console to complete the processing of the audio data.

[0119] In this embodiment, the processing principle of audio data with more than two channels is the same as that of the aforementioned two-channel audio data, which will not be elaborated herein.

[0120] like Figure 6 As shown, the second aspect of this embodiment provides a hardware device for implementing the audio data processing method of the dual-channel communication link described in the first aspect of the embodiment, including: an acquisition unit, a judgment unit, a screening unit, a denoising unit, a sound enhancement unit and a sending unit.

[0121] The acquisition unit is used to acquire audio data, wherein the audio data includes sound data collected by at least two microphones.

[0122] The judging unit is used to judge whether there is faulty audio data in the audio data.

[0123] The screening unit is used to remove the faulty audio data from the audio data to obtain target audio data when the judging unit judges that the fault is yes, and to select the best sound data from the audio data as the target audio data when the judging unit judges that the fault is no.

[0124] The denoising unit is used to remove non-human voice data from the target audio data to obtain denoised audio data.

[0125] The sound enhancement unit is used to perform sound enhancement processing on the denoised audio data to obtain enhanced audio data.

[0126] The sending unit is used to output the enhanced audio data to the mixing console to complete the processing of the audio data.

[0127] The working process, working details and technical effects of the hardware device provided in this embodiment can be found in the first aspect of the embodiment and will not be described in detail here.

[0128] like Figure 7As shown, the third aspect of this embodiment provides another audio data processing device for a dual-channel communication link. Taking the device as a computer host device as an example, it includes: a memory, a processor and a transceiver that are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the audio data processing method for the dual-channel communication link as described in the first aspect of the embodiment.

[0129] For example, the memory may include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, first-in first-out memory (FIFO) and / or first-in last-out memory (FILO), etc.; the processor may include, but is not limited to, a microprocessor of the STM32F105 series, a reduced instruction set computer (RSIC) microprocessor, an X86 architecture processor, or a processor with an integrated embedded neural network processing unit (NPU); the transceiver may include, but is not limited to, a wireless fidelity (WIFI) wireless transceiver, a Bluetooth wireless transceiver, a general packet radio service technology (GPRS) wireless transceiver, a ZigBee protocol (a low-power local area network protocol based on the IEEE802.15.4 standard, ZigBee) wireless transceiver, a 3G transceiver, a 4G transceiver and / or a 5G transceiver, etc. In addition, the device may also include, but is not limited to, a power module, a display screen, and other necessary components.

[0130] The working process, working details and technical effects of the computer main device provided in this embodiment can be found in the first aspect of the embodiment and will not be described in detail here.

[0131] The fourth aspect of this embodiment provides a storage medium that stores instructions for the audio data processing method for the dual-channel communication link described in the first aspect of the embodiment, that is, the storage medium stores instructions, and when the instructions are run on a computer, the audio data processing method for the dual-channel communication link described in the first aspect is executed.

[0132] The storage medium refers to a carrier for storing data, which may include but is not limited to a floppy disk, a CD, a hard disk, a flash memory, a USB flash drive and / or a memory stick, and the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0133] The working process, working details and technical effects of the storage medium provided in this embodiment can be found in the first aspect of the embodiment and will not be described in detail here.

[0134] A fifth aspect of this embodiment provides a computer program product comprising instructions, which, when executed on a computer, causes the computer to execute the audio data processing method for a dual-channel communication link as described in the first aspect of the embodiment, wherein the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0135] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for processing audio data in a dual-channel communication link, characterized in that: include: Acquiring audio data, wherein the audio data includes sound data collected by at least two microphones; Determining whether there is faulty audio data in the audio data; If yes, then remove the faulty audio data from the audio data to obtain the target audio data; otherwise, select the best sound data from the audio data as the target audio data; Removing non-human voice data from the target audio data to obtain denoised audio data; Performing sound enhancement processing on the de-noised audio data to obtain enhanced audio data; Outputting the enhanced audio data to a mixing console to complete the processing of the audio data; Determining whether there is faulty audio data in the audio data includes: Acquire the amplitude of each sound data in the audio data; Calculating the similarity between each sound data and the preset audio data; Determining whether the amplitude of each sound data is less than a preset amplitude and determining whether the similarity between each sound data and preset audio data exceeds a preset threshold range; If so, taking the sound data whose amplitude is less than the preset amplitude as the fault audio data and / or taking the sound data whose similarity exceeds the preset threshold range as the fault audio data; Calculating the similarity between each sound data and the preset audio data includes: Calculating the speech similarity between each sound data and the preset audio data, and calculating the frequency spectrum similarity between each sound data and the preset audio data; Accordingly, determining whether the similarity between each sound data and the preset audio data exceeds a preset threshold range includes: Determining whether the voice similarity between each sound data and the preset audio data exceeds a first preset threshold range; If yes, determining whether the frequency spectrum similarity between each sound data and the preset audio data exceeds a second preset threshold range; If yes, the sound data whose frequency spectrum similarity exceeds the second preset threshold range is used as the fault audio data.

2. The method according to claim 1, characterized in that Calculating the voice similarity between each sound data and the preset audio data includes: Sampling each sound data according to a preset sampling frequency to obtain sampled data corresponding to each sound data; Performing feature extraction on the sampled data corresponding to each sound data to obtain a speech feature vector corresponding to each sound data; The cosine distance between the speech feature vector corresponding to each sound data and the speech feature vector of the preset audio data is calculated as the speech similarity.

3. The method according to claim 1, characterized in that Calculating the frequency spectrum similarity between each sound data and the preset audio data includes: Extracting environmental noise data corresponding to each sound data from each sound data; Extracting features of the environmental noise data corresponding to each sound data to obtain a frequency spectrum feature vector corresponding to each sound data; The cosine distance between the frequency spectrum feature vector corresponding to each sound data and the frequency spectrum feature vector of the preset audio data is calculated as the frequency spectrum similarity.

4. The method according to claim 1, characterized in that Selecting the best sound data from the audio data includes: Obtaining a signal-to-noise ratio of each sound data in the audio data; The sound data with the largest signal-to-noise ratio is taken as the optimal sound data.

5. The method according to claim 1, characterized in that The method further comprises: If it is determined that each sound data in the audio data is faulty audio data, fault alarm information is generated to prompt the user that each of the at least two microphones is faulty.

6. An audio data processing device for a dual-channel communication link, characterized in that: include: An acquisition unit, a judgment unit, a screening unit, a denoising unit, a sound enhancement unit, and a sending unit; The acquisition unit is used to acquire audio data, wherein the audio data includes sound data collected by at least two microphones; The judging unit is used to judge whether there is faulty audio data in the audio data; The screening unit is configured to remove the faulty audio data from the audio data to obtain target audio data when the judging unit judges that the faulty audio data is yes, and select the best sound data from the audio data as the target audio data when the judging unit judges that the faulty audio data is no; The denoising unit is used to remove non-human voice data from the target audio data to obtain denoised audio data; The sound enhancement unit is used to perform sound enhancement processing on the denoised audio data to obtain enhanced audio data; The sending unit is used to output the enhanced audio data to the mixing console to complete the processing of the audio data; Determining whether there is faulty audio data in the audio data includes: Acquire the amplitude of each sound data in the audio data; Calculating the similarity between each sound data and the preset audio data; Determining whether the amplitude of each sound data is less than a preset amplitude and determining whether the similarity between each sound data and preset audio data exceeds a preset threshold range; If so, taking the sound data whose amplitude is less than the preset amplitude as the fault audio data and / or taking the sound data whose similarity exceeds the preset threshold range as the fault audio data; Calculating the similarity between each sound data and the preset audio data includes: Calculating the speech similarity between each sound data and the preset audio data, and calculating the frequency spectrum similarity between each sound data and the preset audio data; Accordingly, determining whether the similarity between each sound data and the preset audio data exceeds a preset threshold range includes: Determining whether the voice similarity between each sound data and the preset audio data exceeds a first preset threshold range; If yes, determining whether the frequency spectrum similarity between each sound data and the preset audio data exceeds a second preset threshold range; If yes, the sound data whose frequency spectrum similarity exceeds the second preset threshold range is used as the fault audio data.

7. An audio data processing device for a dual-channel communication link, characterized in that: include: A memory, a processor and a transceiver that are sequentially communicatively connected, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the audio data processing method of the dual-channel communication link as described in any one of claims 1 to 5.

8. A storage medium, characterized in that: The storage medium stores instructions, and when the instructions are executed on a computer, the audio data processing method for a dual-channel communication link according to any one of claims 1 to 5 is executed.

Images