A method, system and storage medium for processing multi-channel audio signals

Abstract

The present application discloses a method, system and storage medium for processing multiple audio signals, belonging to the technical field of multi-channel audio processing. The method comprises: when multiple first audio signals are received, multiplexing the multiple first audio signals into one second audio signal; sending the second audio signal to a receiving end so that the receiving end restores the second audio signal into multiple third audio signals, sending the multiple third audio signals to an audio processing module to perform audio recording on each of the third audio signals, so as to realize simultaneous transmission, recording, storage and playback of multiple audio signals.

Description

Technical Field

[0001] The present application relates to the technical field of multi-channel audio processing, and in particular to a method, system and storage medium for processing multi-channel audio signals. Background Art

[0002] In current multimedia technology, audio processing systems mainly support a limited number of channels, usually mono or stereo. The audio processing capabilities of the Android system are also limited to a small number of channels, and cannot meet the processing requirements for multi-channel audio signals. For example, the existing Android hardware interface cannot support the efficient transmission of multi-channel audio signals at the same time; the HAL layer and application layer of the Android system do not provide sufficient support for multi-channel audio signals, and cannot achieve the recording and playback of 32-channel audio signals. The existing technology has low efficiency and poor effect in processing multi-channel audio signals. Summary of the invention

[0003] The present application provides a method, system and storage medium for processing multi-channel audio signals to achieve simultaneous transmission, recording, storage and playback of multi-channel audio signals.

[0004] The present application provides a method for processing multiple audio signals, comprising: when receiving multiple first audio signals, multiplexing the multiple first audio signals into one second audio signal;

[0005] The second audio signal is sent to a receiving end, so that the receiving end restores the second audio signal into a plurality of third audio signals, and the plurality of third audio signals are sent to an audio processing module to perform audio recording on each of the third audio signals.

[0006] Furthermore, the receiving of a plurality of first audio signals is specifically:

[0007] Receiving a first audio signal using a plurality of I2S input interfaces;

[0008] Each of the I2S input interfaces receives a first audio signal; the first audio signal is an I2S signal.

[0009] Furthermore, the multiplexing of the plurality of first audio signals into one second audio signal is specifically:

[0010] Multiplexing a plurality of the first audio signals into one second audio signal using a multiplexer;

[0011] Each of the first audio signals occupies a time segment in the second audio signal; and the second audio signal is a TDM signal.

[0012] Furthermore, the multiplexing of the plurality of first audio signals into one second audio signal further includes:

[0013] The synchronization signal is used to control the allocation of time segments of a plurality of channels of the first audio signals according to a preset transmission timing.

[0014] Further, the second audio signal is restored into a plurality of third audio signals, and the plurality of third audio signals are sent to the audio processing module, specifically:

[0015] The second audio signal is restored into a plurality of third audio signals by using a demultiplexer, and each of the third audio signals is input to an I 2S output interface respectively, so that the plurality of third audio signals are sent to an audio processing module through the I 2S output interface.

[0016] Further, the audio recording of each channel of the third audio signal is specifically:

[0017] Using an audio driver module to receive a plurality of third audio signals; the audio driver module pre-configures audio parameters, the audio parameters including: transmission parameters of a preset channel, sampling rate and audio format;

[0018] The audio interface of the HAL layer is used to access and record the third audio signals of several channels to obtain the target audio track audio signal.

[0019] Furthermore, after receiving a plurality of first audio signals by the audio driver module, the method further includes:

[0020] The third audio signals of several channels are stored in a ring buffer, and the third audio signals in the ring buffer are cyclically read to the HAL layer.

[0021] Furthermore, the audio processing module is also used for:

[0022] Get the target audio track audio signal generated at the HAL layer;

[0023] The target audio track audio signal is encoded according to the channel number of each audio track by using the application layer; and the encoded target audio track audio signal is stored in a target file.

[0024] Furthermore, the audio processing module is also used for:

[0025] Decoding the encoded target audio signal using the application layer to obtain the target audio signal;

[0026] The target audio track audio signal is sent to an audio output device for playback.

[0027] Accordingly, the present application also provides a multi-channel audio signal processing system, comprising: a transmission processing module, a receiving end and an audio processing module;

[0028] Wherein, the audio processing module includes a HAL layer, an application layer and an audio driver module;

[0029] The transmission processing module is used for multiplexing the first audio signals into one second audio signal when receiving the first audio signals, and sending the second audio signal to the receiving end;

[0030] The receiving end is used to restore the second audio signal into a plurality of third audio signals, and send the plurality of third audio signals to the audio processing module;

[0031] The audio processing module is used to perform audio recording on each channel of the third audio signal.

[0032] Accordingly, the present application also provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to execute a method for processing multi-channel audio signals as described in the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a flowchart of an embodiment of a method for processing multi-channel audio signals provided by the present application;

[0034] Figure 2 It is a structural diagram of an embodiment of a multi-channel audio signal processing system provided by the present application. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0036] Embodiment 1

[0037] Please refer to Figure 1 , a method for processing a multi-channel audio signal provided in an embodiment of the present application, comprising steps S101-S102:

[0038] Step S101: when receiving a plurality of first audio signals, multiplexing the plurality of first audio signals into a second audio signal;

[0039] Step S102: sending the second audio signal to a receiving end, so that the receiving end restores the second audio signal into a plurality of third audio signals, and sends the plurality of third audio signals to an audio processing module to perform audio recording on each of the third audio signals.

[0040] Furthermore, the receiving of a plurality of first audio signals is specifically:

[0041] Receiving a first audio signal using a plurality of I2S input interfaces;

[0042] Each of the I2S input interfaces receives a first audio signal; the first audio signal is an I2S signal.

[0043] Furthermore, the multiplexing of the plurality of first audio signals into one second audio signal is specifically:

[0044] Multiplexing a plurality of the first audio signals into one second audio signal using a multiplexer;

[0045] Each of the first audio signals occupies a time segment in the second audio signal; and the second audio signal is a TDM signal.

[0046] As a preferred implementation, the method for processing multi-channel audio signals provided in the present application processes 32 channels of audio signals, and uses 16 I2S interfaces to obtain 16 channels of I2S signals respectively as the first audio signal of each channel, and each channel of I2S signal uses stereo.

[0047] The I2S (Inter-IC Sound) interface is mainly used for mono or stereo audio data transmission, while the TDM (Time-Division Multiplexing) interface can support multiple audio signals on the same transmission line. In this embodiment, a multiplexer is used to convert multiple I2S signals into a single TDM signal, and each I2S signal occupies a time segment of the TDM signal. At the same time, the synchronization signal is used to ensure that the transmission timing of each I2S signal in the TDM signal is correct.

[0048] Furthermore, the multiplexing of the plurality of first audio signals into one second audio signal further includes:

[0049] The synchronization signal is used to control the allocation of time segments of a plurality of channels of the first audio signals according to a preset transmission timing.

[0050] Further, the second audio signal is restored into a plurality of third audio signals, and the plurality of third audio signals are sent to the audio processing module, specifically:

[0051] The second audio signal is restored into a plurality of third audio signals by using a demultiplexer, and each of the third audio signals is input to an I 2S output interface respectively, so that the plurality of third audio signals are sent to an audio processing module through the I 2S output interface.

[0052] As a preferred implementation scheme, this embodiment uses a demultiplexer to parse the data in the TDM frame into multiple time slices according to a pre-set time slice allocation method after receiving the TDM signal, each time slice corresponds to the audio data of one channel, and the parsed audio data is output to the 16-channel I2S interface respectively.

[0053] Further, the audio recording of each channel of the third audio signal is specifically:

[0054] Using an audio driver module to receive a plurality of third audio signals; the audio driver module pre-configures audio parameters, the audio parameters including: transmission parameters of a preset channel, sampling rate and audio format;

[0055] The audio interface of the HAL layer is used to access and record the third audio signals of several channels to obtain the target audio track audio signal.

[0056] As a preferred embodiment, the present application modifies and updates the audio driver program to support the transmission and processing of 32-channel audio signals in the 16-channel I 2S interface.

[0057] Specifically, modify the ALSA audio driver code to support 32-channel audio signal transmission and processing. In the audio_hw_params(struct snd_pcm_substream*substream,struct snd_pcm_hw_params*params) method of cm_nativ.c in the ALSA audio driver, add a method to determine whether the number of channels is 32. If so, set the transmission parameters, sampling rate, and audio format of 32 channels.

[0058] Add support for 32-channel audio recording in the HAL layer audio interface definition, and expose the access interface to the application layer. Add support for 32 channels to audio_stream_in() in the audio_hw.h file, and implement the interface for 32-channel recording.

[0059] Furthermore, after receiving a plurality of first audio signals by the audio driver module, the method further includes:

[0060] The third audio signals of several channels are stored in a ring buffer, and the third audio signals in the ring buffer are cyclically read to the HAL layer.

[0061] As a preferred implementation scheme, the present application initializes a circular buffer for storing 32 channels of audio data, obtains data in a loop in the in_read() method, reads 32 channels of audio data from the circular buffer to the HAL layer for audio access and recording, and the HAL layer accesses and records the 32-channel third audio signal to obtain 32-channel audio track audio signals.

[0062] It should be noted that this embodiment can process data with less than 32 channels. During recording, if one or more channels have no signal input / output, the data of the channel is set to 0, which is equivalent to no signal input / output of the channel.

[0063] As a preferred implementation scheme, the updated HAL layer is tested and verified to ensure that it can stably support 32-channel audio recording. Specifically, an audio_channel 32.sh script is written. This script mainly configures the 32-channel audio recording parameters. At the same time, it detects whether the device supports 32-channel recording and broadcasting. If it does, the recorded audio is saved in the audio-channel 32.wav file. After recording for 10 seconds, it is not detected whether the recording is successful. If the recording is successful, the recorded audio data is played to detect whether the playback is successful.

[0064] Furthermore, the audio processing module is also used for:

[0065] Get the target audio track audio signal generated at the HAL layer;

[0066] The target audio track audio signal is encoded according to the channel number of each audio track by using the application layer; and the encoded target audio track audio signal is stored in a target file.

[0067] As a preferred embodiment, the application layer of the present application can obtain 32-track audio signals from the access interface exposed by the HAL layer. The 32-track audio signals are encoded according to the channel number of each track, and the 32-track audio signals are organized into a multi-track audio file. The audio file is stored in a target file that supports 32 tracks.

[0068] Furthermore, the audio processing module is also used for:

[0069] Decoding the encoded target audio signal using the application layer to obtain the target audio signal;

[0070] The target audio track audio signal is sent to an audio output device for playback.

[0071] As a preferred implementation scheme, an audio file containing 32-channel audio tracks is read from a storage device, the audio signals are decoded according to the channel number information in the file, restored to the original 32-channel audio tracks, and the decoded audio signals are transmitted to an audio output device for playback.

[0072] Implementing the embodiments of the present application has the following effects:

[0073] This application implements TDM multiplexing of the I2S interface, which can support the simultaneous transmission of up to 32 audio signals. This application adds support for multi-channel audio recording in the HAL layer of the Android system. This application implements encoding and decoding of multi-track audio signals at the application layer, and supports the storage and playback of audio data.

[0074] Embodiment 2

[0075] Please refer to Figure 2 , a multi-channel audio signal processing system provided in an embodiment of the present application, comprising: a transmission processing module 201, a receiving end 202 and an audio processing module 203;

[0076] Wherein, the audio processing module includes a HAL layer, an application layer and an audio driver module;

[0077] The transmission processing module 201 is used for multiplexing the first audio signals into one second audio signal when receiving the first audio signals, and sending the second audio signal to the receiving end;

[0078] The receiving end 202 is used to restore the second audio signal into a plurality of third audio signals, and send the plurality of third audio signals to the audio processing module;

[0079] The audio processing module 203 is used to perform audio recording on each channel of the third audio signal.

[0080] The audio recording of each channel of the third audio signal is specifically:

[0081] Using an audio driver module to receive a plurality of third audio signals; the audio driver module pre-configures audio parameters, the audio parameters including: transmission parameters of a preset channel, sampling rate and audio format;

[0082] The audio interface of the HAL layer is used to access and record the third audio signals of several channels to obtain the target audio track audio signal.

[0083] The audio processing module 203 is also used to obtain the target track audio signal generated in the HAL layer;

[0084] The target audio track audio signal is encoded according to the channel number of each audio track by using the application layer; and the encoded target audio track audio signal is stored in a target file.

[0085] The audio processing module 203 is also used to decode the encoded target audio track audio signal using the application layer to obtain the target audio track audio signal;

[0086] The target audio track audio signal is sent to an audio output device for playback.

[0087] The above-mentioned multi-channel audio signal processing system can implement the multi-channel audio signal processing method of the above-mentioned method embodiment. The options in the above-mentioned method embodiment are also applicable to this embodiment and will not be described in detail here. The rest of the contents of the embodiment of the present application can refer to the contents of the above-mentioned method embodiment, and in some preferred embodiments, they will not be described in detail.

[0088] Embodiment 3

[0089] Accordingly, the present application also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the method for processing multi-channel audio signals as described in any one of the above embodiments.

[0090] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the terminal device.

[0091] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0092] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device.

[0093] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the terminal device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the mobile terminal, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Med ia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0094] Wherein, if the module / unit integrated in the terminal device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0095] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the scope of protection of the present application. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A method for processing multi-channel audio signals, characterized in that: include: When receiving a plurality of first audio signals, multiplexing the plurality of first audio signals into a second audio signal; Sending the second audio signal to a receiving end, so that the receiving end restores the second audio signal into a plurality of third audio signals, and sends the plurality of third audio signals to an audio processing module; The audio processing module is used to perform audio recording on each channel of the third audio signal, specifically: Receive a plurality of third audio signals by using an audio driver module; the audio driver module pre-configures audio parameters, the audio parameters including: transmission parameters of preset channels, sampling rate and audio format; access and record the plurality of third audio signals by using an audio interface of the HAL layer to generate a target track audio signal; After receiving the plurality of third audio signals by the audio driver module, the method further includes: storing the plurality of third audio signals in a ring buffer, and cyclically reading the third audio signals in the ring buffer to the HAL layer; Acquire the target audio track audio signal generated in the HAL layer; use the application layer to encode the target audio track audio signal according to the channel number of each audio track; store the encoded target audio track audio signal in the target file; use the application layer to decode the encoded target audio track audio signal to obtain the target audio track audio signal; send the target audio track audio signal to the audio output device for playback.

2. A method for processing multi-channel audio signals as claimed in claim 1, characterized in that: The receiving of a plurality of first audio signals is specifically: Receiving a first audio signal using a plurality of I2S input interfaces; Each of the I 2S input interfaces receives a first audio signal; the first audio signal is an I 2S signal.

3. A method for processing multi-channel audio signals as claimed in claim 1, characterized in that: The multiplexing of the plurality of first audio signals into one second audio signal is specifically: Multiplexing a plurality of the first audio signals into one second audio signal using a multiplexer; Each of the first audio signals occupies a time segment in the second audio signal; and the second audio signal is a TDM signal.

4. A method for processing multi-channel audio signals as claimed in claim 3, characterized in that: The step of multiplexing the plurality of first audio signals into one second audio signal further includes: The synchronization signal is used to control the allocation of time segments of a plurality of channels of the first audio signals according to a preset transmission timing.

5. The method for processing multi-channel audio signals according to claim 1, characterized in that: The restoring the second audio signal into a plurality of third audio signals, and sending the plurality of third audio signals to the audio processing module, specifically comprises: The second audio signal is restored into a plurality of third audio signals by using a demultiplexer, and each of the third audio signals is input to an I 2S output interface respectively, so that the plurality of third audio signals are sent to an audio processing module through the I 2S output interface.

6. A multi-channel audio signal processing system, characterized in that: include: Transmission processing module, receiving end and audio processing module; Wherein, the audio processing module includes a HAL layer, an application layer and an audio driver module; The transmission processing module is used for multiplexing the first audio signals into one second audio signal when receiving the first audio signals, and sending the second audio signal to the receiving end; The receiving end is used to restore the second audio signal into a plurality of third audio signals, and send the plurality of third audio signals to the audio processing module; The audio processing module is used to perform audio recording on each channel of the third audio signal, specifically: Receive a plurality of third audio signals by using an audio driver module; the audio driver module pre-configures audio parameters, the audio parameters including: transmission parameters of preset channels, sampling rate and audio format; access and record the plurality of third audio signals by using an audio interface of the HAL layer to generate a target track audio signal; After receiving the plurality of third audio signals by the audio driver module, the method further includes: storing the plurality of third audio signals in a ring buffer, and cyclically reading the third audio signals in the ring buffer to the HAL layer; Acquire the target audio track audio signal generated in the HAL layer; use the application layer to encode the target audio track audio signal according to the channel number of each audio track; store the encoded target audio track audio signal in the target file; use the application layer to decode the encoded target audio track audio signal to obtain the target audio track audio signal; send the target audio track audio signal to the audio output device for playback.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program; wherein, when the computer program is run, it controls the device where the computer-readable storage medium is located to execute a method for processing multi-channel audio signals as claimed in any one of claims 1 to 5.

Images