Audio transmission method and device, terminal and storage medium

Abstract

The embodiment of the invention discloses an audio transmission method and device, a terminal and a storage medium, and relates to the technical field of audio transmission. The method is used for an audio sending end and an audio receiving end which establish connection based on WebRTC. The method comprises the following steps: carrying out audio acquisition; under the condition that at least two paths of audios are collected, encoding and packaging the at least two paths of audios respectively to obtain RTP data packets corresponding to the at least two paths of audios respectively; and sending the RTP data packets corresponding to the at least two paths of audios to an audio receiving end, so that the audio receiving end independently decodes and plays the at least two paths of audios based on the RTP data packets. According to the embodiment of the invention, different audios on the audio sending end side can be independently played on the audio receiving end side, and the audio playing flexibility of the audio receiving end side is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of audio transmission technology, and in particular to an audio transmission method, device, terminal and storage medium. Background Art

[0002] Web Real-Time Communication (WebRTC) is a real-time communication technology that allows network applications or sites to establish peer-to-peer connections without the help of an intermediary to achieve the transmission of video streams and / or audio streams. Summary of the invention

[0003] The embodiments of the present application provide an audio transmission method, device, terminal and storage medium. The technical solution is as follows:

[0004] On the one hand, an embodiment of the present application provides an audio transmission method, the method is used for an audio sending end, a connection is established between the audio sending end and the audio receiving end based on WebRTC, and the method includes:

[0005] Conduct audio collection;

[0006] When at least two channels of audio are collected, the at least two channels of audio are respectively encoded and packaged to obtain real-time transport protocol (RTP) data packets corresponding to the at least two channels of audio;

[0007] The RTP data packets corresponding to the at least two audio channels are sent to the audio receiving end, so that the audio receiving end can independently decode and play the at least two audio channels based on the RTP data packets.

[0008] On the other hand, an embodiment of the present application provides an audio transmission method, which is used for an audio receiving end, wherein a connection is established between the audio receiving end and the audio sending end through WebRTC, and the method includes:

[0009] Receive the RTP data packet sent by the audio sending end;

[0010] Divide the RTP data packet to obtain RTP data packets corresponding to at least two audio channels;

[0011] Based on the RTP data packets corresponding to each audio channel, the at least two audio channels are independently decoded and played.

[0012] On the other hand, an embodiment of the present application provides an audio transmission device, the device is used for an audio sending end, a connection is established between the audio sending end and the audio receiving end based on WebRTC, and the device includes:

[0013] A collection module, used for audio collection;

[0014] The encoding and packaging module is used to encode and package the at least two channels of audio respectively when at least two channels of audio are collected, so as to obtain RTP data packets corresponding to the at least two channels of audio respectively;

[0015] The sending module is used to send the RTP data packets corresponding to the at least two audio channels to the audio receiving end, so that the audio receiving end can independently decode and play the at least two audio channels based on the RTP data packets.

[0016] On the other hand, an embodiment of the present application provides an audio transmission device, the device is used for an audio receiving end, the audio receiving end is connected to an audio sending end through WebRTC, and the device includes:

[0017] A receiving module, used for receiving the RTP data packet sent by the audio sending end;

[0018] A division module, used for dividing the RTP data packet to obtain RTP data packets corresponding to at least two audio channels;

[0019] The decoding and playing module is used to independently decode and play the at least two audio channels based on the RTP data packets corresponding to each audio channel.

[0020] On the other hand, an embodiment of the present application provides a terminal, which includes a processor and a memory; the memory stores at least one instruction, and the at least one instruction is used to be executed by the processor to implement the audio transmission method described in the above aspect.

[0021] On the other hand, an embodiment of the present application provides a computer-readable storage medium, in which at least one instruction is stored, and the instruction is loaded and executed by a processor to implement the audio transmission method as described in the above aspects.

[0022] On the other hand, an embodiment of the present application provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the terminal reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the terminal executes the audio transmission method provided in the above aspect.

[0023] In an embodiment of the present application, for two ends that establish a connection based on WebRTC, when the audio sending end collects at least two channels of audio, it does not mix the at least two channels of audio, but encodes and packages each channel of audio separately to obtain an RTP data packet corresponding to the at least two channels of audio, and sends the RTP data packet to the audio receiving end, so that the audio receiving end can divide the RTP data packet corresponding to each channel of audio, and then independently decode and play the at least two channels of audio based on the divided RTP data packet, thereby realizing independent playback of different audios on the audio sending end side on the audio receiving end side, and improving the flexibility of audio playback on the audio receiving end side. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 is a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application;

[0026] Figure 2 is a flow chart of an audio transmission method provided by an exemplary embodiment of the present application;

[0027] Figure 3 is a schematic diagram of an audio transmission process shown in an exemplary embodiment of the present application;

[0028] Figure 4 is a flowchart of an audio transmission method provided by another exemplary embodiment of the present application;

[0029] Figure 5 is a flowchart of an audio encoder selection process shown in an exemplary embodiment of the present application;

[0030] Figure 6 is a flowchart of an audio coding and packaging process shown in an exemplary embodiment of the present application;

[0031] Figure 7 is a flowchart of an audio transmission method provided by another exemplary embodiment of the present application;

[0032] Figure 8 is a schematic diagram of an implementation of an audio transmission process shown in an exemplary embodiment of the present application;

[0033] Fig. 9 is a schematic diagram of an interface of an audio volume independent adjustment function shown in an exemplary embodiment of the present application;

[0034] Fig.10 is a flowchart of an audio decoding and playing process shown in an exemplary embodiment of the present application;

[0035] Fig.11 is a structural block diagram of an audio transmission device provided by an exemplary embodiment of the present application;

[0036] Fig.12 is a structural block diagram of an audio transmission device provided by another exemplary embodiment of the present application;

[0037] Fig.13 It is a schematic diagram of the structure of a terminal provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0039] Please refer to Figure 1 , which shows a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application. The implementation environment may include: an audio transmitting end 110 and an audio receiving end 120.

[0040] The audio transmitter 110 is an electronic device for audio collection and transmission. The electronic device may be a smart phone, a tablet computer, a wearable device, or a personal computer, etc. ( Figure 1 In the description, the audio transmitting end 110 is taken as a smart phone as an example), and the embodiments of the present application do not limit the specific type of electronic device.

[0041] In some embodiments, the audio collected by the audio transmitter 110 may be external audio or local audio, wherein the external audio may be a human voice collected through a microphone, and the local audio may be music played through a local audio playback application.

[0042] The audio receiving end 120 is an electronic device for receiving and playing the audio transmitted by the audio transmitting end 110. The electronic device may be a smart phone, a tablet computer, a wearable device, or a personal computer, etc. ( Figure 1 In the description, the audio receiving end 120 is taken as a tablet computer as an example), and the embodiments of the present application do not limit the specific type of electronic device.

[0043] In some embodiments, the audio receiving terminal 120 can play audio through a built-in speaker, or through other external audio playback devices, or through both the built-in speaker and the external audio playback device. The external audio playback device can be connected to the audio receiving terminal 120 through a wired or wireless data connection. Figure 1As shown, the audio receiving end 120 has a built-in speaker 121 and is connected to a Bluetooth headset 130 via a Bluetooth connection.

[0044] A connection is established between the audio transmitter 110 and the audio receiver 120 based on WebRTC, and the connection may be a wireless local area network (WLAN) connection or a peer to peer (P2P) connection.

[0045] In the embodiment of the present application, after collecting multiple audio channels, the audio sending end 110 does not mix the multiple audio channels, and then encode and package the mixed audio and send it to the audio receiving end 120. Instead, it creates multiple independent audio stream sending paths, and encodes and packages the audio in the audio stream sending paths corresponding to each audio, thereby obtaining RTP data packets corresponding to each audio channel.

[0046] Correspondingly, after receiving the RTP data packet sent by the audio transmitting end 110, the audio receiving end 120 can identify the RTP data packets corresponding to different audios, and thus independently decode and play the RTP data packets corresponding to each audio channel.

[0047] Since the multiple audio channels are independent of each other, in some embodiments, the audio receiving end 120 can play the multiple audio channels through the same audio player or can play different audio channels through different audio players. Figure 1 As shown, the RTP data packet sent by the audio sending end 110 includes an RTP data packet corresponding to the voice call audio and an RTP data packet corresponding to the background music. The audio receiving end 120 can play the background music through the built-in speaker 121 and play the call audio through the Bluetooth headset 130 connected by Bluetooth.

[0048] In some embodiments, the audio receiving end 120 can manually set which audio playback device to use to play the audio.

[0049] In some embodiments, the audio receiving end 120 provides a volume adjustment function, through which a user can independently adjust the volume of multiple audio channels.

[0050] It should be noted that the above embodiment only takes audio transmission between a single audio transmitter and a single audio receiver as an example. In other possible application scenarios, a single audio transmitter can simultaneously transmit audio to multiple audio receivers, or a single audio receiver can simultaneously transmit audio to multiple audio transmitters. This embodiment does not limit this.

[0051] See also Figure 2 , Figure 2is a flowchart of an audio transmission method provided by an exemplary embodiment of the present application. Figure 1 Taking the implementation environment shown as an example, the method may include the following steps.

[0052] Step 201: The audio transmitter collects audio.

[0053] In some embodiments, the audio transmitter may perform multi-channel audio collection through a multi-channel collection thread.

[0054] In some embodiments, the audio transmitter may perform audio collection based on a unified collection parameter, or the audio transmitter may collect different audio based on different collection parameters, so as to balance the collected audio quality and collection power consumption.

[0055] Indicatively, Figure 3 As shown, when the audio transmitter plays music, the acquisition module collects the audio of the background music; when an incoming call is received during the audio playback process, the acquisition module collects the audio of the background music and the ringtone at the same time; when an incoming call is answered (the ringtone stops), the acquisition module collects the audio of the background music and the call tone at the same time.

[0056] Step 202: When at least two audio channels are collected, the audio transmitter encodes and packages the at least two audio channels respectively to obtain RTP data packets corresponding to the at least two audio channels.

[0057] When at least two channels of audio are collected, the audio transmitter encodes and packages each channel of audio respectively to obtain an RTP data packet corresponding to each channel of audio.

[0058] In some embodiments, the audio transmitter may adopt a unified encoding strategy to encode audio of different channels, or the audio transmitter may adopt different encoding strategies to encode audio of different channels, so as to balance the audio encoding quality and encoding power consumption.

[0059] For the encoded data, the audio transmitter performs RTP encapsulation and packaging on the data to obtain RTP data packets corresponding to each audio channel. When performing RTP encapsulation and packaging, the audio transmitter needs to write a specific identifier in the header of the RTP data packet to distinguish the RTP data packets corresponding to different audios.

[0060] Furthermore, the RTP data packets corresponding to different audios have their own sequence numbers, that is, at the same time, the sequence numbers in the RTP data packets corresponding to different audios may not be continuous.

[0061] Indicatively, Figure 3As shown, when the audio sending end plays music, the encoding module encodes and packages the background music to obtain the first RTP data packet corresponding to the background music; when an incoming call is received during the audio playback process, the encoding module encodes and packages the background music and the ringtone respectively to obtain the first RTP data packet corresponding to the background music, and the second RTP data packet corresponding to the ringtone; when an incoming call answering operation is received (the ringtone stops), the encoding module encodes and packages the background music and the call tone respectively to obtain the first RTP data packet corresponding to the background music, and the third RTP data packet corresponding to the call tone.

[0062] Step 203: the audio transmitting end sends the RTP data packets corresponding to at least two audio channels to the audio receiving end, so that the audio receiving end can independently decode and play the at least two audio channels based on the RTP data packets.

[0063] In a possible implementation, the audio transmitter writes the RTP data packet into a User Datagram Protocol Socket (UDP Socket) interface and then sends it to the audio receiver, wherein the RTP data packets corresponding to each audio channel are sequentially written into the UDP Socket interface in the order in which they are generated.

[0064] Indicatively, Figure 3 As shown, when the audio sending end is playing music, the sending module only writes the first RTP data packet corresponding to the background music into the UDP Socket interface and then sends it to the audio receiving end; when an incoming call is received during the audio playback process, the sending module writes the first RTP data packet corresponding to the background music and the second RTP data packet corresponding to the ringtone into the UDP Socket interface in sequence and then sends them to the audio receiving end; when an incoming call answering operation is received (the ringtone stops), the sending module writes the first RTP data packet corresponding to the background music and the third RTP data packet corresponding to the call tone into the UDPSocket interface in sequence and then sends them to the audio receiving end.

[0065] Step 204: the audio receiving end receives the RTP data packet sent by the audio sending end.

[0066] The audio receiving end receives the RTP data packet through the connection with the audio sending end.

[0067] In some embodiments, the audio receiving end reads the RTP data packet from the UDP Socket interface.

[0068] Step 205: The audio receiving end divides the RTP data packet to obtain RTP data packets corresponding to at least two audio channels.

[0069] When there are multiple audio channels, since the RTP data packets corresponding to different audio channels are sent to the audio receiving end in sequence, in order to ensure the accuracy of subsequent independent decoding of each audio channel, the audio receiving end needs to divide the received RTP data packets and determine the RTP data packets corresponding to each audio channel in sequence.

[0070] In some embodiments, the audio receiving end distinguishes the RTP data packets corresponding to different audio channels based on a specific identifier carried in the RTP data packet header.

[0071] In some embodiments, after distinguishing the RTP data packets corresponding to different audios, the audio receiving end will also sort the RTP data packets according to the sequence number carried in the RTP data packet header, so as to perform audio decoding in sequence later.

[0072] Indicatively, Figure 3 As shown, when the audio sending end plays music, the receiving module determines the received RTP data packets as the first RTP data packets corresponding to the background music; when an incoming call is received during the audio playback process, the receiving module divides the RTP data packets into the first RTP data packets corresponding to the background music and the second RTP data packets corresponding to the ringtone; when an incoming call answering operation is received (the ringtone stops), the receiving module divides the RTP data packets into the first RTP data packets corresponding to the background music and the third RTP data packets corresponding to the call tone.

[0073] Step 206: The audio receiving end independently decodes and plays at least two audio channels based on the RTP data packets corresponding to each audio channel.

[0074] When there is a need for playback, the audio receiving end independently decodes the RTP data packets corresponding to each audio channel and plays the audio through the corresponding audio playback device.

[0075] In some embodiments, the audio receiving end uses a decoding strategy corresponding to the encoding strategy at the audio playing end to decode the audio, that is, the audio playing end can use a unified decoding strategy for different audios, or can use different decoding strategies for different audios.

[0076] Indicatively, Figure 3As shown, when the audio sending end plays music, the decoding module only decodes the first RTP data packet corresponding to the background music, and hands over the decoding result of the background music to the playback module for playback; when an incoming call is received during the audio playback process, the decoding module simultaneously decodes the first RTP data packet corresponding to the background music and the second RTP data packet corresponding to the ringtone, and hands over the decoding results of the background music and the ringtone to the playback module for playback; when an incoming call answering operation is received (the ringtone stops), the decoding module simultaneously decodes the first RTP data packet corresponding to the background music and the third RTP data packet corresponding to the call tone, and hands over the decoding results of the background music and the call tone to the playback module for playback.

[0077] To summarize, in the embodiments of the present application, for the two ends that establish a connection based on WebRTC, when the audio sending end collects at least two channels of audio, it does not mix the at least two channels of audio, but encodes and packages each channel of audio separately to obtain an RTP data packet corresponding to the at least two channels of audio, and sends the RTP data packet to the audio receiving end, so that the audio receiving end can divide the RTP data packet corresponding to each channel of audio, and then independently decode and play the at least two channels of audio based on the divided RTP data packet, thereby realizing independent playback of different audios on the audio sending end side on the audio receiving end side, and improving the flexibility of audio playback on the audio receiving end side.

[0078] In a possible implementation, the audio transmitter creates a corresponding audio stream transmission path (AudioSendStream) for each audio channel, and configures a corresponding encoder for each audio stream transmission path, so as to encode and package the audio through the audio stream transmission path.

[0079] Please refer to Figure 4 , which shows a flowchart of an audio transmission method provided by another exemplary embodiment of the present application. This embodiment uses the method for Figure 1 Taking the audio transmitting end shown as an example for explanation, the method may include the following steps.

[0080] Step 401, perform audio collection.

[0081] Since different types of audio have different spectral distribution characteristics, if a unified sampling strategy is used for all types of audio, the sampling quality may be affected or the sampling power consumption may be increased. In order to reduce the sampling power consumption while ensuring the sampling quality, the audio transmitter can use different sampling strategies for different types of audio.

[0082] In a possible implementation, the audio transmitter first determines the audio type of the audio to be collected, and then collects the audio based on sampling parameters corresponding to the audio type, wherein the sampling parameters include at least one of a sampling rate and a number of sampling channels.

[0083] In some embodiments, the audio type may include an alarm sound (ALARM), music sound (MUSIC), a ringtone (RING), a system sound (SYSTEM), and a call sound (VOICE), etc. The embodiment of the present application does not specifically limit the audio type.

[0084] Optionally, different audio types may correspond to different sampling parameters, or different types of audio may use the same sampling parameters.

[0085] In some embodiments, since the frequency spectrum of ringtones and music sounds is wider and contains richer information, a higher sampling rate and more channels are required to achieve better sampling effects; since the frequency spectrum of call tones is narrower, a lower sampling rate and fewer channels can achieve good sampling effects.

[0086] In an illustrative example, for music and ringtones, the audio transmitter uses 48kHz dual-channel audio sampling, and for call sounds, the audio transmitter uses 16kHz single-channel sampling. The above correspondence between audio types and sampling parameters is only for illustrative purposes, but does not constitute a limitation thereto.

[0087] Step 402: When at least two audio channels are collected, determine the audio stream sending paths corresponding to the at least two audio channels.

[0088] In a possible implementation, the audio transmitter creates a corresponding audio stream transmission path for each audio channel, which is used to process the collected audio before audio transmission, wherein the processing method includes but is not limited to audio encoding, encapsulation and packaging, etc. Accordingly, before audio encoding, the audio transmitter needs to determine the audio stream transmission path corresponding to each audio channel.

[0089] Regarding the method of determining the audio corresponding to the audio stream sending path, in a possible implementation manner, the process may include the following steps:

[0090] 1. For any one of the at least two audio channels, determine a synchronization source (SSRC) identifier corresponding to the audio based on the audio type of the audio, wherein different audio types correspond to different SSRC identifiers.

[0091] In the embodiment of the present application, the audio transmitter allocates different SSRC identifiers to audios of different audio types. Accordingly, the audio transmitter determines the SSRC identifier corresponding to the audio based on the audio type of the audio.

[0092] In a possible implementation, when the collected audio is pushed from the service layer to the audio transport layer, it carries an audio type parameter indicating the audio type, and the audio transport layer queries the corresponding SSRC identifier according to the audio type parameter.

[0093] In some embodiments, when the service layer sets the audio type parameter for the collected audio, the audio type parameter set for the audio is only used to indicate the audio type, for example, the audio type parameter may include ALARM, MUSIC, RING, SYSTEM, and VOICE. In this case, for multiple audio channels of the same type but from different sources, since the multiple audio channels correspond to the same SSRC identifier, the multiple audio channels will be processed in the same audio stream sending path.

[0094] In other embodiments, for audio of the same type but from different sources, such as audio collected from a music application and audio collected from a video application, both of which are of type MUSIC but from different sources, the audio type parameter set by the service layer for the audio is used to distinguish both the audio type and the audio source, for example, the audio type parameter corresponding to the audio collected from the music application is MUSIC 1, and the audio type parameter corresponding to the audio collected from the video application is MUSIC 2. In this case, for multiple audio channels of the same type but from different sources, since the multiple audio channels correspond to different SSRC identifiers, the multiple audio channels will be processed independently in different audio stream sending paths.

[0095] 2. Determine the audio stream transmission path corresponding to the audio based on the SSRC identifier.

[0096] In this embodiment, each audio stream transmission path corresponds to its own SSRC identifier. Accordingly, the audio transmitter determines the audio stream transmission path corresponding to each audio channel based on the corresponding relationship between the SSRC identifier and the audio stream transmission path and the SSRC identifier corresponding to each audio channel.

[0097] In an illustrative example, the audio sending end determines that the corresponding SSRC identifier is SSRC 1 based on the audio type MUSIC corresponding to the background music, and then determines that the background music corresponds to the audio stream sending path 1; based on the audio type VOICE corresponding to the call tone, the audio sending end determines that the corresponding SSRC identifier is SSRC 2, and then determines that the call tone corresponds to the audio stream sending path 2.

[0098] Optionally, when there are multiple audio channels corresponding to the same audio stream sending path, the multiple audio channels need to be mixed before audio encoding is performed through the audio stream sending path, and then the mixed audio is audio encoded through the audio stream sending path.

[0099] It should be noted that for a newly collected audio, the audio sender cannot query the SSRC identifier corresponding to the audio type. At this time, the audio sender needs to allocate an SSRC identifier based on the audio type of the audio and create an audio stream sending path corresponding to the SSRC identifier. The SSRC identifier is different from the allocated SSRC identifier.

[0100] Step 403: Perform audio encoding through the encoder configured in the audio stream sending path to obtain audio encoding data.

[0101] In the embodiment of the present application, an encoder is configured in each audio stream transmission path for performing audio encoding on the collected audio to obtain audio encoding data. Since different encoders are good at encoding different audio types, in order to improve the audio encoding quality, in a possible implementation, for each audio stream transmission path, the audio transmitting end configures the encoder based on the audio type corresponding to the audio stream transmission path when creating the audio stream transmission path. The audio type corresponding to the audio stream transmission path is determined based on the SSRC identifier corresponding to the audio stream transmission path.

[0102] In some embodiments, when the audio type corresponding to the audio stream sending path is a voice type, the audio sending end configures a Silk encoder for the audio stream sending path;

[0103] When the audio type corresponding to the audio stream sending path is a music type, the audio sending end configures a Celt encoder for the audio stream sending path;

[0104] When the audio type corresponding to the audio stream sending path is a ring tone type, the audio sending end configures an AAC encoder for the audio stream sending path.

[0105] Indicatively, Figure 5 As shown, for the collected audio data, the audio transmitter determines the audio type in turn. When the audio type is a voice type, the Silk encoder is enabled to perform audio encoding to obtain voice encoding data; when the audio type is a music type, the Celt encoder is enabled to perform audio encoding to obtain music encoding data; when the audio type is a ringtone type, the AAC encoder is enabled to perform audio encoding to obtain ringtone encoding data; when the audio type is other types, encoders of other types are enabled to perform audio encoding to obtain audio encoding data of other types.

[0106] It should be noted that the above-mentioned audio types, encoders and the correspondence between the two are only for exemplary descriptions. Those skilled in the art may configure encoders for other audio types, configure other types of encoders and correspondences different from the above-mentioned ones, and this embodiment does not constitute a limitation on this.

[0107] Optionally, since different acquisition strategies are adopted for different types of audio during audio acquisition, audio encoding is also required based on audio parameters corresponding to the audio (such as sampling rate parameters, number of channels, etc.), that is, the audio parameters are used as part of the input to instruct the encoder to encode, which is not elaborated in this embodiment.

[0108] Step 404: Encapsulate and package the audio encoding data using the SSRC identifier corresponding to the audio stream sending path to obtain an RTP data packet, wherein different audio stream sending paths correspond to different SSRC identifiers.

[0109] When encapsulating and packaging the encoded audio encoded data, the audio sending end needs to package the SSRC identifier corresponding to the audio stream sending path into the RTP data packet, so that the audio receiving end can determine which audio stream receiving path to process the RTP data packet based on the SSRC identifier.

[0110] It should be noted that before encapsulation and packaging, the audio stream sending path may also be subjected to redundancy processing to improve the robustness of audio transmission, which will not be elaborated in this embodiment.

[0111] Step 405: Send RTP data packets corresponding to at least two audio channels to the audio receiving end.

[0112] Schematically, the process of audio transmission at the audio transmitter is as follows: Figure 6 As shown. First, the audio receiving end determines the audio type, queries the corresponding SSRC identifier according to the audio type, and queries the corresponding audio stream sending path based on the SSRC identifier, thereby encoding the audio through the audio stream sending path and packaging the audio encoded data. After the encoding is completed, the audio stream sending path encapsulates the audio encoded data based on the SSRC identifier to obtain an RTP data packet, and then calls the UDPSocket interface to transmit the RTP data packet to the audio receiving end.

[0113] In this embodiment, the audio transmitter uses different collection strategies to collect audio for different types of audio, which helps to improve the audio collection quality and further improve the subsequent audio playback quality on the audio receiver side.

[0114] In addition, by creating different audio stream sending paths for audio of different audio types and configuring the encoder in the audio stream sending path based on the audio type, it helps to improve the encoding quality of different types of audio, thereby improving the audio playback quality on the subsequent audio receiving end side.

[0115] Corresponding to the audio stream sending path created by the audio sending end, the audio receiving end needs to create a corresponding audio stream receiving path (AudioReceiveStream), which is used to process the received RTP data packets before audio playback, where the processing methods include but are not limited to RTP unpacking, audio decoding, etc. In addition, when different types of encoders are configured in different audio stream sending paths, in order to ensure correct audio decoding, the audio receiving end needs to configure the corresponding decoder for the audio stream receiving path.

[0116] In a possible implementation, in order to inform the audio receiving end how to create an audio stream receiving path and what decoder to configure for each audio stream receiving path, the audio sending end sends a Session Description Protocol (SDP) message to the audio receiving end. The SDP message includes a correspondence between the audio type and the SSRC identifier, so that the audio receiving end can create an audio stream receiving path based on the correspondence.

[0117] In some embodiments, when there is newly collected audio, the audio sending end needs to send the SDP message to instruct the audio receiving end to create a new audio stream receiving path; when the existing audio is stopped, the audio sending end also needs to send the SDP message to instruct the audio receiving end to remove the audio stream receiving path corresponding to the stopped audio.

[0118] Schematically, before the audio transmitter transmits background music and call audio to the audio receiver, the SDP message sent includes a corresponding relationship as shown in Table 1.

[0119] Table 1

[0120] Audio Type SSRC Identifier MUCIC SSRC 1 VOICE SSRC 2

[0121] Correspondingly, the audio receiving end receives the SDP message sent by the audio sending end, creates an audio stream receiving path corresponding to the SSRC identifier based on the SSRC identifier contained in the SDP message, and configures the decoder in the audio stream receiving path based on the audio type corresponding to the SSRC identifier, wherein different decoders are used to encode audio of different audio types.

[0122] Different audio stream receiving channels on the audio receiving end side correspond to different SSRC identifiers, and the decoder configuration strategy of the audio receiving end and the encoder configuration strategy of the audio sending end remain the same.

[0123] In some embodiments, when the audio type corresponding to the audio stream receiving path is a voice type, the audio receiving end configures a Silk decoder for the audio stream receiving path;

[0124] When the audio type corresponding to the audio stream receiving path is a music type, the audio receiving end configures a Celt decoder for the audio stream receiving path;

[0125] In the case that the audio type corresponding to the audio stream receiving path is a ring tone type, the audio receiving end configures an AAC decoder for the audio stream receiving path.

[0126] Illustratively, in combination with the corresponding relationship shown in Table 1, the audio receiving end creates a first audio stream receiving path corresponding to SSRC 1 and a second audio stream receiving path corresponding to SSRC 2, and configures a Celt decoder for the first audio stream receiving path and a Silk decoder for the second audio stream receiving path.

[0127] In this embodiment, the audio sending end sends an SDP message containing the correspondence between the audio type and the SSRC identifier to the audio receiving end, so that the audio receiving end creates an audio stream receiving path corresponding to the audio stream sending path, and configures a corresponding decoder for each audio stream receiving path to ensure that the audio can be correctly decoded at the audio receiving end.

[0128] Please refer to Figure 7 , which shows a flowchart of an audio transmission method provided by another exemplary embodiment of the present application. This embodiment uses the method for Figure 1 Taking the audio receiving end shown as an example for explanation, the method may include the following steps.

[0129] Step 701, receiving an RTP data packet sent by an audio sending end.

[0130] The implementation of this step may refer to the above step 204, and this embodiment will not be described in detail here.

[0131] Step 702: divide the RTP data packets based on the SSRC identifiers carried by the RTP data packets to obtain RTP data packets corresponding to at least two audio channels, wherein the RTP data packets corresponding to different audio channels carry different SSRC identifiers.

[0132] Since RTP data packets corresponding to multiple audio channels may be received, in order to ensure that subsequent audio decoding is correct, the audio receiving end can divide the RTP data packets by reading the SSRC identifier carried by the RTP data packets.

[0133] Step 703: Cache the RTP data packets corresponding to each audio channel into the buffer of the audio stream receiving path, wherein different audio channels correspond to different audio stream receiving paths.

[0134] Affected by factors such as network quality, the RTP data packets corresponding to the same audio may not be transmitted to the audio stream receiving end in the order in which they are generated. In order to ensure that the RTP data packets can be decoded in order later, in a possible implementation, the audio receiving end sets corresponding buffers (packet buffers) for different audio stream receiving paths to store the RTP data packets corresponding to different audio stream receiving paths. Accordingly, the audio receiving end writes the RTP data packets corresponding to the divided different audios into the corresponding buffers.

[0135] In some embodiments, the audio receiving end sorts the RTP data packets in the buffer based on the sequence number in the RTP data packet header so as to perform audio decoding in sequence later.

[0136] Step 704: for any one of the at least two audio channels, the RTP data packets cached in the buffer are decoded by a decoder configured in the audio stream receiving path corresponding to the audio channel to obtain audio decoding data.

[0137] In a possible implementation, the audio stream receiving path reads an RTP data packet from a buffer, unpacks the RTP data packet to obtain the audio encoding data contained therein, and then performs audio decoding on the audio encoding data through a configured decoder to obtain audio decoding data.

[0138] In some embodiments, the audio decoded data obtained by decoding can be cached in a to-be-played cache corresponding to the audio stream receiving path. When there is a subsequent playback demand, the audio decoded data is extracted from the to-be-played cache.

[0139] Indicatively, Figure 8As shown, when the audio transmitter needs to transmit background music 810 and call audio 820 to the audio receiver, the audio transmitter creates a first audio stream transmission path 811 and a second audio stream transmission path 821 based on the audio type. The SSRC identifier corresponding to the first audio stream transmission path 811 is SSRC 1, which is used to encode and package music-type audio, so the first audio stream transmission path 811 is configured with a Celt encoder; the SSRC identifier corresponding to the second audio stream transmission path 821 is SSRC 2, which is used to encode and package voice-type audio, so the second audio stream transmission path 821 is configured with a Silk encoder.

[0140] When the first audio stream sending path 811 encapsulates and packages the audio coding data, SSRC 1 is added to the packet header to obtain a first RTP data packet 812 containing SSRC 1 and music coding data; when the second audio stream sending path 821 encapsulates and packages the audio coding data, SSRC 2 is added to the packet header to obtain a second RTP data packet 822 containing SSRC 2 and voice coding data.

[0141] Furthermore, the audio sending end calls the UDP Socket interface 830 to transmit the first RTP data packet 812 and the second RTP data packet 822 to the audio receiving end.

[0142] In addition, the audio transmitter also sends an SDP message to the audio receiver, indicating the correspondence between SSRC 1 and the music type, and the correspondence between SSRC 2 and the voice type. Based on the SDP message, the audio receiver creates a first audio stream receiving channel 814 corresponding to SSRC 1 and a second audio stream receiving channel 824 corresponding to SSRC 2, and configures a Celt decoder corresponding to the music type for the first audio stream receiving channel 814, and configures a Silk decoder corresponding to the voice type for the second audio stream receiving channel 824.

[0143] After receiving the RTP data packet by calling the UDP Socket interface 830, the audio receiving end caches the first RTP data packet 812 to the first buffer 813 corresponding to the first audio stream receiving channel 814 based on the SSRC identifier carried in the packet header, and caches the second RTP data packet 822 to the second buffer 823 corresponding to the second audio stream receiving channel 824.

[0144] After the first audio stream receiving channel 814 unpacks and decodes the first RTP data packet 812, it obtains music decoding data 815. After the second audio stream receiving channel 824 unpacks and decodes the second RTP data packet 822, it obtains voice decoding data 825.

[0145] Step 705: Based on the audio decoding data, the audio is played through the audio playing channel corresponding to the audio.

[0146] Since the audio decoding data corresponding to different types of audio are independent of each other, in a possible implementation, the audio receiving end may create an audio playback channel (AudioTrack) based on the audio types corresponding to each audio channel on the audio sending end side.

[0147] Optionally, the audio receiving end creates an audio playback channel based on the audio type included in the SDP message.

[0148] Accordingly, when audio is played based on the audio decoding data corresponding to each audio channel, the audio receiving end determines the audio playing channel corresponding to the audio based on the audio type of the audio, and loads the audio decoding data through the audio playing channel for audio playback. Different audio types correspond to different audio playing channels.

[0149] In an illustrative example, when the audio transmitting end transmits background music and call audio, the audio receiving end creates a first audio playback channel corresponding to the music type and a second music playback channel corresponding to the voice type.

[0150] Since users have different volume requirements for different types of audio in different scenarios, and different audio playback channels are created for different audio types in the above embodiments, in a possible implementation, the audio receiving end supports an independent audio volume adjustment function, and the user can adjust the volume of each audio channel according to the volume requirements.

[0151] Regarding the implementation method of independent audio volume adjustment, in some embodiments, in response to a volume adjustment operation on a target audio in at least two audio channels, the audio receiving end adjusts the channel volume of the audio playback channel corresponding to the target audio.

[0152] Optionally, the audio receiving end displays a volume adjustment control, and the sub-controls contained in the volume adjustment control correspond to the audio playback channel and are used to set the channel volume of the corresponding audio playback channel.

[0153] Indicatively, Fig. 9 As shown, the audio receiving end receives the background music and call sound transmitted by the audio sending end. When the user needs to adjust different types of audio separately, the voice volume adjustment sub-control and the background sound volume adjustment sub-control in the volume adjustment control 91 can be operated separately. For example, the call sound volume can be increased through the voice volume adjustment sub-control, and the background sound volume can be reduced through the background sound volume adjustment sub-control.

[0154] In addition to being able to independently adjust the volume of each audio channel, the audio receiving end can also provide an audio playback device selection function, that is, the user can manually select the audio playback device corresponding to each audio channel according to needs. Accordingly, when playing audio, the audio receiving end needs to determine the audio playback device corresponding to the audio, and then output the audio decoding data to the audio playback device through the audio playback channel corresponding to the audio based on the audio decoding data, so as to play the audio through the audio playback device.

[0155] The same audio playback channel can output audio decoding data to one or more audio playback devices at the same time.

[0156] The audio playback device corresponding to the audio may be set by the audio receiving end by default based on the audio type, or may be set manually by the user.

[0157] For example, for ringtone-type audio, the audio receiving end determines that the audio playback device of the audio is all devices with audio playback function connected to the audio receiving end. Exemplarily, when the audio receiving end has a built-in speaker and is connected to a Bluetooth headset, the audio receiving end outputs the audio decoding data corresponding to the ringtone to the speaker and the Bluetooth headset simultaneously through the audio playback channel.

[0158] For example, for call-type audio, the audio receiving end determines that the default audio playback device for the audio is all devices with audio playback function connected to the audio receiving end, and the user can choose to play the call voice only through the Bluetooth headset through manual settings. At this time, the audio receiving end only outputs the audio decoding data corresponding to the call voice to the Bluetooth headset through the audio playback channel.

[0159] Indicatively, Fig.10 As shown, after the audio decoding data is obtained, if there is no audio playback channel corresponding to the audio type, the audio receiving end creates an audio playback channel corresponding to the audio type based on the audio type. If it exists, the corresponding audio playback channel is selected. For each audio playback channel, the user can independently set the channel volume and set the output device of the audio playback channel to realize the customization of the audio output volume and channel.

[0160] In this embodiment, by setting the channel volume and output device of the audio playback channel, independent volume adjustment of multiple audio channels and customization of the audio playback device corresponding to each audio channel can be achieved, which helps to improve the flexibility of audio playback.

[0161] In addition to being able to control the volume and output devices of different audios, in some embodiments, the audio receiving end can also provide an audio pause function. In a possible implementation, in response to a pause operation for the target audio in at least two audio channels, the audio receiving end discards the RTP data packet corresponding to the target audio.

[0162] After discarding the RTP data packet corresponding to the target audio, the audio receiving end does not need to decode and play the target audio.

[0163] In some embodiments, in response to a play operation on the target audio, the audio receiving end continues to process the RTP data packets corresponding to the target audio.

[0164] In another possible implementation, when a pause operation is received, the audio receiving end may also send a pause instruction to the audio sending end so that the audio sending end can pause sending the RTP data packets corresponding to the target audio based on the instruction, thereby avoiding continuing to send RTP data packets and causing waste of processing and transmission resources.

[0165] When receiving a play operation for the target audio, the audio receiving end sends a play instruction to the audio sending end so that the audio sending end continues to send the RTP data packets corresponding to the target audio (but there will be a certain recovery delay).

[0166] See also Fig.11 , Fig.11 : is a structural block diagram of an audio transmission device provided by an exemplary embodiment of the present application. The device includes:

[0167] The acquisition module 1101 is used for audio acquisition;

[0168] The encoding and packaging module 1102 is used to encode and package the at least two audio channels respectively when at least two audio channels are collected, so as to obtain RTP data packets corresponding to the at least two audio channels respectively;

[0169] The sending module 1103 is used to send the RTP data packets corresponding to the at least two audio channels to the audio receiving end, so that the audio receiving end can independently decode and play the at least two audio channels based on the RTP data packets.

[0170] The coding packaging module 1102 is used to:

[0171] Determine the audio stream sending paths corresponding to the at least two audio channels respectively;

[0172] Performing audio encoding through an encoder configured in the audio stream sending path to obtain audio encoding data;

[0173] The audio encoding data is encapsulated and packaged using the SSRC identifier corresponding to the audio stream sending path to obtain the RTP data packet, wherein different audio stream sending paths correspond to different SSRC identifiers.

[0174] Optionally, the encoding packaging module 1102 is used to:

[0175] For any one of the at least two audio channels, based on the audio type of the audio channel, determining an SSRC identifier corresponding to the audio channel, wherein different audio types correspond to different SSRC identifiers;

[0176] The audio stream sending path corresponding to the audio is determined based on the SSRC identifier.

[0177] Optionally, the sending module 1103 is further used to:

[0178] An SDP message is sent to the audio receiving end, wherein the SDP message includes a correspondence between an audio type and an SSRC identifier, so that the audio receiving end creates an audio stream receiving path based on the correspondence.

[0179] Optionally, the device further comprises:

[0180] A creation module is used to create the audio stream sending path corresponding to the SSRC identifier; and configure the encoder based on the audio type corresponding to the audio stream sending path.

[0181] Optionally, the creation module is used to:

[0182] When the audio type corresponding to the audio stream sending path is a voice type, configuring a Silk encoder for the audio stream sending path;

[0183] In a case where the audio type corresponding to the audio stream sending path is a music type, configuring a Celt encoder for the audio stream sending path;

[0184] In a case where the audio type corresponding to the audio stream sending path is a ringtone type, an AAC encoder is configured for the audio stream sending path.

[0185] Optionally, the acquisition module 1101 is used to:

[0186] Determine the audio type of the audio to be collected;

[0187] Audio collection is performed based on sampling parameters corresponding to the audio type, where the sampling parameters include at least one of a sampling rate and a number of sampling channels.

[0188] See also Fig.12 , Fig.12 : is a structural block diagram of an audio transmission device provided by another exemplary embodiment of the present application. The device includes:

[0189] The receiving module 1201 is used to receive the RTP data packet sent by the audio sending end;

[0190] A division module 1202 is used to divide the RTP data packet to obtain RTP data packets corresponding to at least two audio channels;

[0191] The decoding and playing module 1203 is used to independently decode and play the at least two audio channels based on the RTP data packets corresponding to each audio channel.

[0192] Optionally, the division module 1202 is used to:

[0193] The RTP data packets are divided based on the SSRC identifiers carried by the RTP data packets to obtain RTP data packets corresponding to at least two audio channels, wherein RTP data packets corresponding to different audio channels carry different SSRC identifiers.

[0194] Optionally, the decoding and playing module 1203 is used to:

[0195] Cache the RTP data packets corresponding to each audio channel into a buffer of the audio stream receiving path, wherein different audio channels correspond to different audio stream receiving paths;

[0196] For any one of the at least two audio channels, the RTP data packet cached in the buffer is audio decoded by a decoder configured in the audio stream receiving path corresponding to the audio, to obtain audio decoding data;

[0197] Based on the audio decoding data, audio is played through an audio playing channel corresponding to the audio.

[0198] Optionally, the receiving module 1201 is further used for:

[0199] Receiving an SDP message sent by the audio sending end, wherein the SDP message includes a correspondence between an audio type and an SSRC identifier;

[0200] The device also includes a first creation module, which is used to create the audio stream receiving path corresponding to the SSRC identifier; and configure the decoder in the audio stream receiving path based on the audio type corresponding to the SSRC identifier, wherein different decoders are used to encode audio of different audio types.

[0201] Optionally, the first creation module is used to:

[0202] When the audio type corresponding to the audio stream receiving path is a voice type, configuring a Silk decoder for the audio stream receiving path;

[0203] When the audio type corresponding to the audio stream receiving path is a music type, configuring a Celt decoder for the audio stream receiving path;

[0204] In a case where the audio type corresponding to the audio stream receiving path is a ring tone type, an AAC decoder is configured for the audio stream receiving path.

[0205] Optionally, the decoding and playing module 1203 is used to:

[0206] Determine the audio playback channel corresponding to the audio based on the audio type of the audio, wherein different audio types correspond to different audio playback channels;

[0207] Loading the audio decoding data through the audio playback channel to perform audio playback;

[0208] The device also includes a second creation module, configured to:

[0209] The audio playback channel is created based on the audio type corresponding to the audio.

[0210] Optionally, the device further comprises:

[0211] The volume adjustment module is used to adjust the channel volume of the audio playback channel corresponding to the target audio in the at least two audio channels in response to the volume adjustment operation on the target audio.

[0212] Optionally, the decoding and playing module 1203 is used to:

[0213] Determine an audio playback device corresponding to the audio;

[0214] Based on the audio decoding data, the audio decoding data is output to the audio playback device through an audio playback channel corresponding to the audio, so that the audio is played through the audio playback device.

[0215] Optionally, the device further comprises:

[0216] The discarding module is used to discard the RTP data packet corresponding to the target audio in response to a pause operation on the target audio in the at least two audio channels.

[0217] See also Fig.13 , Fig.1313 is a schematic diagram of the structure of a terminal provided by an exemplary embodiment of the present application. The terminal can be implemented as an audio transmitter or an audio receiver in the above embodiment. The terminal can be a smart phone, a tablet computer, a smart wearable device, etc. The terminal can also include one or more of the following components: a processor 1310 and a memory 1320.

[0218] Optionally, the processor 1310 uses various interfaces and lines to connect various parts of the entire electronic device, and executes various functions of the electronic device and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 1320, and calling data stored in the memory 1320. Optionally, the processor 1310 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 1310 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a neural network processor (NPU), and a baseband chip. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content that needs to be displayed on the touch display; the NPU is used to implement artificial intelligence (AI) functions; and the baseband chip is used to process wireless communications. It is understandable that the above baseband chip may not be integrated into the processor 1310, but may be implemented by a separate chip.

[0219] The memory 1320 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 1320 includes a non-transitory computer-readable storage medium. The memory 1320 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 1320 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc.; the data storage area may store data created according to the use of the electronic device (such as audio data, a phone book), etc.

[0220] In addition, those skilled in the art will appreciate that the structure of the terminal shown in the above figures does not constitute a limitation on the terminal, and the terminal may include more or fewer components than shown, or a combination of certain components, or a different arrangement of components.

[0221] The embodiment of the present application also provides a computer-readable storage medium, in which at least one instruction is stored, and the at least one instruction is loaded and executed by a processor to implement the method described in the above embodiment. Optionally, the computer-readable storage medium may include: ROM, RAM, solid state drive (SSD, Solid State Drives) or optical disk, etc. Among them, RAM may include resistance random access memory (ReRAM, Resistance Random Access Memory) and dynamic random access memory (DRAM, Dynamic Random Access Memory).

[0222] The embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the audio transmission method provided in various optional implementations of the above aspects.

[0223] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An audio transmission method, characterized in that: The method is used for an audio sending end, and a connection is established between the audio sending end and the audio receiving end based on WebRTC. The method includes: Conduct audio collection; When at least two channels of audio are collected, the at least two channels of audio are respectively encoded and packaged to obtain RTP data packets corresponding to the at least two channels of audio; The RTP data packets corresponding to the at least two audio channels are sent to the audio receiving end, so that the audio receiving end can independently decode and play the at least two audio channels based on the RTP data packets.

2. The method according to claim 1, characterized in that The step of respectively encoding and packaging the at least two channels of audio to obtain RTP data packets corresponding to the at least two channels of audio respectively includes: Determine the audio stream sending paths corresponding to the at least two audio channels respectively; Performing audio encoding through an encoder configured in the audio stream sending path to obtain audio encoding data; The audio encoding data is encapsulated and packaged using the SSRC identifier corresponding to the audio stream sending path to obtain the RTP data packet, wherein different audio stream sending paths correspond to different SSRC identifiers.

3. The method according to claim 2, characterized in that The step of determining the audio stream sending paths corresponding to the at least two audio channels respectively includes: For any one of the at least two audio channels, based on the audio type of the audio channel, determining an SSRC identifier corresponding to the audio channel, wherein different audio types correspond to different SSRC identifiers; The audio stream sending path corresponding to the audio is determined based on the SSRC identifier.

4. The method according to claim 3, characterized in that The method further comprises: An SDP message is sent to the audio receiving end, wherein the SDP message includes a correspondence between an audio type and an SSRC identifier, so that the audio receiving end creates an audio stream receiving path based on the correspondence.

5. The method according to claim 3, characterized in that: The method further comprises: Creating the audio stream sending path corresponding to the SSRC identifier; The encoder is configured based on the audio type corresponding to the audio stream sending path.

6. The method according to claim 5, characterized in that The configuring the encoder based on the audio type corresponding to the audio stream sending path includes: When the audio type corresponding to the audio stream sending path is a voice type, configuring a Silk encoder for the audio stream sending path; In a case where the audio type corresponding to the audio stream sending path is a music type, configuring a Celt encoder for the audio stream sending path; In a case where the audio type corresponding to the audio stream sending path is a ringtone type, an AAC encoder is configured for the audio stream sending path.

7. The method according to any one of claims 1 to 6, characterized in that: The audio collection comprises: Determine the audio type of the audio to be collected; Audio collection is performed based on sampling parameters corresponding to the audio type, where the sampling parameters include at least one of a sampling rate and a number of sampling channels.

8. An audio transmission method, characterized in that: The method is used for an audio receiving end, where a connection is established between the audio receiving end and the audio sending end via WebRTC, and the method includes: Receive the RTP data packet sent by the audio sending end; Divide the RTP data packet to obtain RTP data packets corresponding to at least two audio channels; Based on the RTP data packets corresponding to each audio channel, the at least two audio channels are independently decoded and played.

9. The method according to claim 8, characterized in that The RTP data packet is divided to obtain RTP data packets corresponding to at least two audio channels, respectively, including: The RTP data packets are divided based on the SSRC identifiers carried by the RTP data packets to obtain RTP data packets corresponding to at least two audio channels, wherein RTP data packets corresponding to different audio channels carry different SSRC identifiers.

10. The method according to claim 8, characterized in that The independently decoding and playing the at least two audio channels based on the RTP data packets corresponding to the respective audio channels includes: Cache the RTP data packets corresponding to each audio channel into a buffer of the audio stream receiving path, wherein different audio channels correspond to different audio stream receiving paths; For any one of the at least two audio channels, the RTP data packet cached in the buffer is audio decoded by a decoder configured in the audio stream receiving path corresponding to the audio, to obtain audio decoding data; Based on the audio decoding data, audio is played through an audio playing channel corresponding to the audio.

11. The method according to claim 10, characterized in that The method further comprises: Receiving an SDP message sent by the audio sending end, wherein the SDP message includes a correspondence between an audio type and an SSRC identifier; Creating the audio stream receiving path corresponding to the SSRC identifier; The decoder in the audio stream receiving path is configured based on the audio type corresponding to the SSRC identifier, wherein different decoders are used to encode audio of different audio types.

12. The method according to claim 11, characterized in that The configuring the decoder in the audio stream receiving path based on the audio type corresponding to the SSRC identifier includes: When the audio type corresponding to the audio stream receiving path is a voice type, configuring a Silk decoder for the audio stream receiving path; When the audio type corresponding to the audio stream receiving path is a music type, configuring a Celt decoder for the audio stream receiving path; In a case where the audio type corresponding to the audio stream receiving path is a ring tone type, an AAC decoder is configured for the audio stream receiving path.

13. The method according to claim 10, characterized in that The step of loading the audio decoding data through the audio playback channel corresponding to the audio to perform audio playback includes: Determine the audio playback channel corresponding to the audio based on the audio type of the audio, wherein different audio types correspond to different audio playback channels; Loading the audio decoding data through the audio playback channel to perform audio playback; The method further comprises: The audio playback channel is created based on the audio type corresponding to the audio.

14. The method according to claim 10, characterized in that The method further comprises: In response to a volume adjustment operation on a target audio in the at least two channels of audio, a channel volume of an audio playback channel corresponding to the target audio is adjusted.

15. The method according to claim 10, characterized in that The step of loading the audio decoding data through the audio playback channel corresponding to the audio to perform audio playback includes: Determine an audio playback device corresponding to the audio; Based on the audio decoding data, the audio decoding data is output to the audio playback device through an audio playback channel corresponding to the audio, so that the audio is played through the audio playback device.

16. The method according to claim 8, characterized in that The method further comprises: In response to a pause operation on a target audio in the at least two channels of audio, the RTP data packet corresponding to the target audio is discarded.

17. An audio transmission device, characterized in that: The device is used for an audio sending end, and a connection is established between the audio sending end and the audio receiving end based on WebRTC. The device includes: A collection module, used for audio collection; The encoding and packaging module is used to encode and package the at least two channels of audio respectively when at least two channels of audio are collected, so as to obtain RTP data packets corresponding to the at least two channels of audio respectively; The sending module is used to send the RTP data packets corresponding to the at least two audio channels to the audio receiving end, so that the audio receiving end can independently decode and play the at least two audio channels based on the RTP data packets.

18. An audio transmission device, characterized in that: The device is used for an audio receiving end, and a connection is established between the audio receiving end and the audio sending end via WebRTC. The device includes: A receiving module, used for receiving the RTP data packet sent by the audio sending end; A division module, used for dividing the RTP data packet to obtain RTP data packets corresponding to at least two audio channels; The decoding and playing module is used to independently decode and play the at least two audio channels based on the RTP data packets corresponding to each audio channel.

19. A terminal, characterized in that: The terminal includes a processor and a memory; the memory stores at least one instruction, and the at least one instruction is used to be executed by the processor to implement the audio transmission method as described in any one of claims 1 to 7, or to implement the audio transmission method as described in any one of claims 8 to 16.

20. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the audio transmission method as described in any one of claims 1 to 7, or to implement the audio transmission method as described in any one of claims 8 to 16.

21. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the terminal implements the audio transmission method as described in any one of claims 1 to 7, or implements the audio transmission method as described in any one of claims 8 to 16.

Images