Audio transmission method and apparatus, storage medium, and electronic device

Abstract

The present application discloses an audio transmission method and apparatus, a storage medium, and an electronic device. The audio transmission method comprises: acquiring audio data; performing framing on the audio data to generate a plurality of audio frames; using a first transmission channel to transmit the plurality of audio frames to an audio judgment module, such that the audio judgment module obtains a first audio frame; using a second transmission channel to transmit the plurality of audio frames to the audio judgment module, such that the audio judgment module obtains a second audio frame; and by means of the audio judgment module, determining to output the first audio frame or the second audio frame as the current output audio frame. The solution can reduce audio lagging.

Description

Audio transmission method, device, storage medium and electronic device Technical Field

[0001] The embodiments of the present application relate to the field of audio transmission technology, and specifically to an audio transmission method, device, storage medium, and electronic device. Background Art

[0002] With the popularization of audio technology, a wide variety of applications have emerged. Some specialized applications require high latency in wireless audio transmission, such as e-sports gaming. When a user is playing a game, the game screen must be almost perfectly synchronized with the sound transmitted to the user's headphones.

[0003] However, during wireless audio transmission, packet loss is inevitable due to environmental factors (such as thick walls, metal isolation belts, electronic interference, etc.), which causes audio jams and seriously affects the user experience.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide an audio transmission method, device, storage medium, and electronic device, which can reduce audio freeze.

[0006] In a first aspect, an embodiment of the present application provides an audio transmission method, comprising:

[0007] Get audio data;

[0008] Performing frame processing on the audio data to generate a plurality of audio frames;

[0009] Using a first transmission channel to send a plurality of the audio frames to an audio decision module, so that the audio decision module obtains a first audio frame;

[0010] sending the plurality of audio frames to the audio decision module using a second transmission channel, so that the audio decision module obtains a second audio frame;

[0011] The audio decision module determines whether to output the first audio frame or the second audio frame as a current output audio frame.

[0012] In the audio transmission method provided in an embodiment of the present application, the determining, by the audio arbitration module, to output the first audio frame or the second audio frame as the current output audio frame includes:

[0013] When the audio adjudication module receives the current first audio frame within a preset time, the current first audio frame is used as the current output audio frame;

[0014] When the audio adjudication module does not receive the current first audio frame within a preset time, taking the target second audio frame corresponding to the current first audio frame as the current output audio frame;

[0015] The current output audio frame is output according to a preset strategy.

[0016] In the audio transmission method provided in an embodiment of the present application, outputting the current output audio frame according to a preset strategy includes:

[0017] Determining whether the previous output audio frame and the current output audio frame are of the same type;

[0018] The current output audio frame is subjected to corresponding data processing according to the judgment result, and the processed current output audio frame is output.

[0019] In the audio transmission method provided in an embodiment of the present application, performing corresponding data processing on the current output audio frame according to the judgment result includes:

[0020] When the previous current output audio frame and the current output audio frame are of the same type, performing a first preprocessing on the current output audio frame;

[0021] When the previous current output audio frame and the current output audio frame are of different types, a second preprocessing is performed on the current output audio frame.

[0022] In the audio transmission method provided in an embodiment of the present application, the first preprocessing of the current output audio frame includes:

[0023] The sampling rate of the current output audio frame is adjusted according to the preset sampling rate.

[0024] In the audio transmission method provided in an embodiment of the present application, performing second preprocessing on the current output audio frame includes:

[0025] Adjusting the sampling rate of the current output audio frame according to a preset sampling rate;

[0026] Perform cross-fading processing on the adjusted current output audio frame.

[0027] In the audio transmission method provided in the embodiment of the present application, the framing of the audio data to generate a plurality of audio frames includes:

[0028] Obtaining a transmission frame length of the first transmission channel;

[0029] The audio data is framed according to the transmission frame length.

[0030] In a second aspect, an embodiment of the present application provides an audio transmission device, comprising:

[0031] An audio acquisition unit, configured to acquire audio data;

[0032] An audio framing unit, configured to perform framing processing on the audio data to generate a plurality of audio frames;

[0033] a first sending unit, configured to send the plurality of audio frames to the audio decision module using a first transmission channel, so that the audio decision module obtains a first audio frame;

[0034] a second sending unit, configured to send the plurality of audio frames to the audio decision module using a second transmission channel, so that the audio decision module obtains a second audio frame;

[0035] An audio output unit is configured to determine, through the audio decision module, to output the first audio frame or the second audio frame as a current output audio frame.

[0036] In a third aspect, the present application provides a storage medium storing a plurality of instructions, wherein the instructions are suitable for loading by a processor to execute any of the above-mentioned audio transmission methods.

[0037] In a fourth aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-described audio transmission methods when executing the computer program.

[0038] In summary, the audio transmission method provided in the embodiment of the present application includes obtaining audio data; performing frame processing on the audio data to generate a plurality of audio frames; using a first transmission channel to send the plurality of audio frames to the audio arbitration module so that the audio arbitration module obtains a first audio frame; using a second transmission channel to send the plurality of audio frames to the audio arbitration module so that the audio arbitration module obtains a second audio frame; and determining, by the audio arbitration module, to output the first audio frame or the second audio frame as the current output audio frame. By using two transmission channels to send audio frames, this solution can reduce the phenomenon of audio freeze caused by missing audio frames due to packet loss. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0040] FIG1 is a schematic structural diagram of an audio transmission system provided in an embodiment of the present application.

[0041] FIG2 is another structural diagram of the audio transmission system provided in an embodiment of the present application.

[0042] FIG3 is a flow chart of the audio transmission method provided in an embodiment of the present application.

[0043] FIG4 is a schematic structural diagram of an audio transmission device provided in an embodiment of the present application.

[0044] FIG5 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0046] It should be noted that, in this document, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0047] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0048] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of this application and have no specific meaning. Therefore, "module", "component" or "unit" can be used interchangeably.

[0049] In the description of this application, it should be noted that the terms "upper," "lower," "left," "right," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate the description of this application and simplify the description. They are not intended to indicate or imply that the devices or components referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, terms such as "first" and "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0050] During wireless audio transmission, packet loss is inevitable due to environmental factors (such as thick walls, metal isolation belts, electronic interference, etc.), which causes audio to become stuck and seriously affects the user experience.

[0051] Based on this, the embodiments of the present application provide an audio transmission method, device, storage medium and electronic device. Specifically, the audio transmission device can be integrated into an electronic device, which can be a server or a terminal; wherein the terminal can include a mobile phone, a wearable smart device, a tablet computer, a laptop computer, and a personal computer (PC); the server can be a single server or a server cluster composed of multiple servers, and can be a physical server or a virtual server.

[0052] For example, referring to Figure 1, which is a schematic diagram of the structure of an audio transmission system provided in an embodiment of the present application, the audio transmission system may include a transmitter 1000 and a receiver 2000.

[0053] The transmitting end 1000 may be connected to the receiving end 2000 via a network. In this embodiment, the transmitting end 1000 and the receiving end 2000 may perform the audio transmission method provided in the embodiment of the present application.

[0054] For example, the transmitting end 1000 obtains audio data; the transmitting end 1000 performs frame processing on the audio data to generate multiple audio frames; the transmitting end 1000 uses a first transmission channel to send the multiple audio frames to the audio decision module, so that the audio decision module obtains a first audio frame; the transmitting end 1000 uses a second transmission channel to send the multiple audio frames to the audio decision module, so that the audio decision module obtains a second audio frame; wherein the audio decision module can be integrated into the receiving end 2000. The receiving end 2000 determines, through the audio decision module, to output the first audio frame or the second audio frame as the current output audio frame.

[0055] For another example, refer to Figure 2, which is another structural diagram of the audio transmission system provided in an embodiment of the present application. The audio transmission system may include a transmitter 1000, a receiver 2000, and a server 3000.

[0056] The sending end 1000 and the receiving end 2000 can be connected to the server 3000 via a network respectively. The sending end 1000 and the receiving end 2000 communicate indirectly via the server 3000.

[0057] In this embodiment, the audio transmission method provided in the embodiment of the present application can be executed by the server 3000. Specifically, the server 3000 obtains audio data; the server 3000 performs frame processing on the audio data to generate a plurality of audio frames; the server 3000 uses a first transmission channel to send the plurality of audio frames to the audio decision module so that the audio decision module obtains a first audio frame; the server 3000 uses a second transmission channel to send the plurality of audio frames to the audio decision module so that the audio decision module obtains a second audio frame; and the server 3000 determines, through the audio decision module, to output the first audio frame or the second audio frame as the current output audio frame.

[0058] In the embodiment of the present application, the transmitting end 1000 and the receiving end 2000 may be electronic devices with computing hardware that can support and execute software products corresponding to multimedia. It should be noted that the network may be a wireless network such as a wireless local area network (WLAN), a local area network (LAN), a cellular network, Bluetooth, a 2G network, a 3G network, a 4G network, or a 5G network.

[0059] The following will describe the technical solutions of this application in detail through specific embodiments. It should be noted that the order of description of the following embodiments does not limit the priority order of the embodiments.

[0060] Please refer to Figure 3, which is a flow chart of the audio transmission method provided in an embodiment of the present application. The specific flow of the audio transmission method can be as follows:

[0061] 101. Obtain audio data.

[0062] The audio data refers to data transmitted via audio. It should be noted that the audio data includes but is not limited to original lossless audio recorded in real time by an audio recording device with network function.

[0063] In some embodiments, the audio recording device may be integrated into the transmitting end, and the transmitting end may directly obtain audio data through the audio recording device. The audio recording device may also be a device independent of the transmitting end, and the audio data may be collected by the audio recording device, and then the transmitting end may obtain the audio data from the audio recording device through data exchange.

[0064] 102. Perform frame processing on the audio data to generate a plurality of audio frames.

[0065] The framing process may be to divide the audio data into frames according to a preset frame length, and then encapsulate each frame of data to obtain a plurality of audio frames. Each waiting audio frame may include the entire audio frequency band or a portion of the audio frequency band.

[0066] In one embodiment, several audio frames may carry their identification information, where the identification information is used to represent the position relationship of the corresponding audio frames in the audio data, wherein the identification information may be a frame number or a timestamp.

[0067] Among them, several audio frames can be transmitted through the first transmission channel and / or the second transmission channel.

[0068] It should be noted that the first transmission channel and the second transmission channel use different transmission protocols. In an embodiment of the present application, the first transmission channel may use the UltraWide Band (UWB) transmission protocol. The second transmission channel may use the Bluetooth transmission protocol. That is, the first transmission channel may be a UWB transmission channel, and the second transmission channel may be a Bluetooth transmission channel.

[0069] Bluetooth transmission is a widely used technology. However, due to its limited bandwidth, it typically transmits audio data with a sampling rate below 48K, compressed, and lossy. However, UWB transmission offers the advantages of high bandwidth and low latency, enabling the transmission of high-sampling-rate, uncompressed, and lossless audio data.

[0070] Therefore, in order to reduce the delay of audio transmission and ensure the quality of the output audio, in an embodiment of the present application, the audio frames sent by the UWB transmission channel are mainly output, and the audio frames sent by the Bluetooth transmission channel are used as supplementary data. Therefore, when packet loss occurs in the UWB transmission channel, the audio frames sent by the Bluetooth transmission channel are used for substitute output to avoid audio frame loss and audio freeze.

[0071] Since the audio frames output by the embodiment of the present application are mainly audio frames sent by the UWB transmission channel, when framing the audio data, the transmission frame length of the UWB transmission channel can be used as the preset frame length to frame the audio data.

[0072] That is, the step of “framing the audio data to generate a plurality of audio frames” may include obtaining a transmission frame length of the first transmission channel; and framing the audio data according to the transmission frame length.

[0073] 103. Use a first transmission channel to send a plurality of audio frames to an audio decision module, so that the audio decision module obtains a first audio frame.

[0074] It can be understood that, since the first transmission channel is a UWB transmission channel, the audio frame sent through the first transmission channel is a lossless first audio frame.

[0075] 104. Use the second transmission channel to send a plurality of audio frames to the audio decision module, so that the audio decision module obtains a second audio frame.

[0076] It is understandable that since the second transmission channel is a UWB transmission channel, the audio frames transmitted via the second transmission channel are compressed and lossy second audio frames. Therefore, before transmitting the plurality of audio frames to the audio adjudication module via the second transmission channel, the plurality of audio frames to be transmitted need to be compressed.

[0077] 105. Determine, through an audio decision module, whether to output the first audio frame or the second audio frame as a current output audio frame.

[0078] In the embodiment of the present application, the audio frames sent via the UWB transmission channel are mainly output, while the audio frames sent via the Bluetooth transmission channel are used as supplementary data.

[0079] Therefore, in some embodiments, when the audio arbitration module receives the current first audio frame within the preset time, the current first audio frame is used as the current output audio frame; when the audio arbitration module does not receive the current first audio frame within the preset time, the target second audio frame corresponding to the current first audio frame is used as the current output audio frame; and the current output audio frame is output according to the preset strategy.

[0080] The preset time refers to the total time T required for the preset number of retransmissions N.

[0081] When an audio frame is transmitted via the first transmission channel, if the audio decision module receives the audio frame, it will return a confirmation message to the transmitting end. If the audio decision module receives the audio frame, it will not return the confirmation message to the transmitting end. If the transmitting end does not receive the confirmation message within the preset time period t, the audio frame is retransmitted to the audio decision module via the first transmission channel (retransmission). This continues until the transmitting end receives the returned confirmation message, or the transmitting end does not receive the confirmation message within the preset time period T. At this point, the next audio frame can be sent to the audio decision module via the first transmission channel.

[0082] It is understandable that because the first and second audio frames are of different types, when the currently output audio frame switches from the first audio frame to the second audio frame, or vice versa, a discontinuity may occur between the currently output audio frame and the previously output audio frame, resulting in discontinuous sound heard by the user. To avoid this and enable smooth switching between different audio frame types, corresponding data processing is required when switching between different audio frame types.

[0083] That is, in some embodiments, the step of "outputting the current output audio frame according to a preset strategy" can specifically be: determining whether the previous output audio frame and the current output audio frame are of the same type; performing corresponding data processing on the current output audio frame according to the judgment result, and outputting the processed current output audio frame.

[0084] When the previous current output audio frame and the current output audio frame are of the same type, the current output audio frame is subjected to a first preprocessing; when the previous current output audio frame and the current output audio frame are of different types, the current output audio frame is subjected to a second preprocessing.

[0085] Specifically, when the previous current output audio frame and the current output audio frame are of the same type, the sampling rate of the current output audio frame can be adjusted according to the preset sampling rate. When the previous current output audio frame and the current output audio frame are of different types, the sampling rate of the current output audio frame can be adjusted according to the preset sampling rate first; and then crossfading can be performed on the adjusted current output audio frame.

[0086] Among them, the preset sampling rate refers to the sampling rate of the first audio frame. It can be understood that since the embodiment of the present application uses the first audio frame as the main output, and the second audio frame sent by the Bluetooth transmission channel is used as supplementary data. Therefore, when the second audio frame is output as supplementary data, it is necessary to adjust the sampling rate of the second audio frame so that the sampling rate of the second audio frame is the same as the sampling rate of the first audio frame, thereby avoiding affecting the audio output effect due to the different sampling rates of the previous output audio frame and the current output audio frame.

[0087] In some embodiments, the sampling rate of the second audio frame may be adjusted by interpolation, for example, by inserting the average of two adjacent sampling points between two sampling points, thereby doubling the sampling rate of the second audio frame.

[0088] In some embodiments, assume that the length of the current output audio frame is N, and there are M points at the end of the frame that need to be cross-faded. x is the first audio frame, and y is the second audio frame. Then, the specific process of cross-fading the adjusted current output audio frame can be as follows:

[0089] For(i=0;i<(NM);i++)out[i]=x[i];

[0090] For (i=0; i<(NM); i++)out[i+NM]=x(i+NM)*i / N-M+y(i+NM)*(1-i / (NM)).

[0091] It can be understood that when the second audio frame is output as supplementary data, the second audio frame needs to be decompressed.

[0092] In some embodiments, the target second audio frame corresponding to the current first audio frame may be a second audio frame received by the audio adjudication module and having identification information identical to that of the current first audio frame.

[0093] In another embodiment, when the audio adjudication module does not receive a second audio frame with the same identification information as the current first audio frame, the audio adjudication module may output the second audio frame adjacent to the second audio frame as the current output audio frame. Although this may result in unsmooth audio output, it is much better than missing the entire audio frame.

[0094] For example, if the identification information of the current first audio frame is N, but the audio adjudication module has not received the current first audio frame, then the second audio frame with identification information N can be used as the current output audio frame for output. However, there may also be a case where the audio adjudication module does not receive the second audio frame with identification information N. In this case, the second audio frame with identification information N+1 can be output as the current output audio frame.

[0095] In summary, the audio transmission method provided by the embodiment of the present application includes acquiring audio data; performing frame processing on the audio data to generate a plurality of audio frames; using a first transmission channel to send a plurality of the audio frames to the audio arbitration module so that the audio arbitration module obtains a first audio frame; using a second transmission channel to send a plurality of the audio frames to the audio arbitration module so that the audio arbitration module obtains a second audio frame; and determining by the audio arbitration module whether to output the first audio frame or the second audio frame as the current output audio frame. This solution uses two transmission channels to send audio frames, and the audio frames sent by the first transmission channel are mainly output. If an audio frame sent by the first transmission channel is still lost after multiple retransmissions, the audio frame sent by the first transmission channel is used for output. That is, this solution can reduce the phenomenon of audio freeze caused by missing audio frames due to packet loss.

[0096] To facilitate better implementation of the audio transmission method provided in the embodiment of the present application, the embodiment of the present application also provides an audio transmission device. The meanings of the terms are the same as those in the above-mentioned audio transmission method, and the specific implementation details can be referred to the description in the method embodiment.

[0097] Please refer to Figure 4, which is a schematic diagram of the structure of the audio transmission device provided by the embodiment of the present application. The audio transmission device may include an audio acquisition unit 201, an audio framing unit 202, a first sending unit 203, a second sending unit 204 and an audio output unit 205.

[0098] An audio acquisition unit 201 is used to acquire audio data;

[0099] The audio framing unit 202 is used to perform framing processing on the audio data to generate a plurality of audio frames;

[0100] A first sending unit 203 is configured to send a plurality of audio frames to the audio decision module using a first transmission channel, so that the audio decision module obtains a first audio frame;

[0101] The second sending unit 204 is configured to send a plurality of audio frames to the audio decision module using a second transmission channel, so that the audio decision module obtains a second audio frame;

[0102] The audio output unit 205 is configured to determine, through the audio decision module, whether to output the first audio frame or the second audio frame as the current output audio frame.

[0103] The specific implementation of each of the above units can be found in the above-mentioned embodiment of the audio transmission method, and will not be described in detail here.

[0104] In summary, the audio transmission device provided in the embodiment of the present application can obtain audio data through the audio acquisition unit 201; the audio framing unit 202 performs framing processing on the audio data to generate a number of audio frames; the first sending unit 203 uses the first transmission channel to send a number of audio frames to the audio judgment module, so that the audio judgment module obtains the first audio frame; the second sending unit 204 uses the second transmission channel to send a number of audio frames to the audio judgment module, so that the audio judgment module obtains the second audio frame; the audio output unit 205 determines whether to output the first audio frame or the second audio frame as the current output audio frame through the audio judgment module. This solution uses two transmission channels to send audio frames, and the audio frames sent by the first transmission channel are mainly output. If an audio frame sent by the first transmission channel is still lost after multiple retransmissions, the audio frame sent by the first transmission channel is used for output. That is, this solution can reduce the phenomenon of audio freeze caused by the loss of audio frames due to packet loss.

[0105] The present application also provides an electronic device, which may be integrated with the audio transmission device of the present application. FIG5 shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application. Specifically:

[0106] The electronic device may include components such as a radio frequency (RF) circuit 601, a memory 602 including one or more computer-readable storage media, an input unit 603, a display unit 604, a sensor 605, an audio circuit 606, a wireless fidelity (WiFi) module 607, a processor 608 including one or more processing cores, and a power supply 609. It will be understood by those skilled in the art that the electronic device structure shown in FIG5 does not limit the electronic device and may include more or fewer components than shown, or combine certain components, or arrange the components differently. Among them:

[0107] The RF circuit 601 can be used to receive and send signals during information transmission or calls. In particular, after receiving downlink information from the base station, it is handed over to one or more processors 608 for processing; in addition, uplink data is sent to the base station. Generally, the RF circuit 601 includes but is not limited to an antenna, at least one amplifier, a tuner, one or more oscillators, a Subscriber Identity Module (SIM) card, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF circuit 601 can also communicate with the network and other devices through wireless communication. Wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile Communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0108] The memory 602 can be used to store software programs and modules. The processor 608 executes various functional applications and information processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device (such as audio data, a phone book, etc.). In addition, the memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 602 may also include a memory controller to provide the processor 608 and the input unit 603 with access to the memory 602.

[0109] The input unit 603 can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical, or trackball signal input related to user settings and function control. Specifically, in one embodiment, the input unit 603 may include a touch-sensitive surface and other input devices. A touch-sensitive surface, also known as a touch display or touchpad, can detect user touch operations on or near it (for example, operations performed by a user using a finger, stylus, or any other suitable object or accessory on or near the touch-sensitive surface) and drive corresponding connected devices according to a pre-set program. Optionally, the touch-sensitive surface may include a touch detection device and a touch controller. The touch detection device detects the user's touch direction and detects signals generated by the touch operation, transmitting the signals to the touch controller. The touch controller receives the touch information from the touch detection device, converts it into touch point coordinates, and then sends it to the processor 608. It can also receive and execute commands from the processor 608. In addition, touch-sensitive surfaces can be implemented using various types, such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch-sensitive surface, the input unit 603 may also include other input devices. Specifically, other input devices may include, but are not limited to, one or more of a physical keyboard, function keys (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, and the like.

[0110] The display unit 604 can be used to display information input by the user or information provided to the user and various graphical user interfaces of the electronic device. These graphical user interfaces can be composed of graphics, text, icons, videos and any combination thereof. The display unit 604 may include a display panel. Optionally, the display panel can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch-sensitive surface can cover the display panel. When the touch-sensitive surface detects a touch operation on or near it, it is transmitted to the processor 608 to determine the type of touch event. The processor 608 then provides a corresponding visual output on the display panel based on the type of touch event. Although in Figure 5, the touch-sensitive surface and the display panel are implemented as two independent components to implement input and output functions, in some embodiments, the touch-sensitive surface and the display panel can be integrated to implement input and output functions.

[0111] The electronic device may also include at least one sensor 605, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel according to the brightness of the ambient light, and the proximity sensor may turn off the display panel and / or backlight when the electronic device is moved to the ear. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that recognize the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that the electronic device can also be configured with, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be described in detail here.

[0112] Audio circuit 606, speakers, and microphones provide an audio interface between the user and the electronic device. Audio circuit 606 converts received audio data into electrical signals and transmits them to the speaker, which then converts them into sound signals for output. The microphone, on the other hand, converts collected sound signals into electrical signals, which are then received by audio circuit 606 and converted into audio data. The audio data is then processed by output processor 608 and transmitted via RF circuit 601 to, for example, another electronic device. Alternatively, the audio data is output to memory 602 for further processing. Audio circuit 606 may also include an earphone jack to allow communication between external headphones and the electronic device.

[0113] WiFi is a short-range wireless transmission technology. Electronic devices can help users send and receive emails, browse web pages, and access streaming media through WiFi module 607, providing users with wireless broadband Internet access. Although FIG5 shows WiFi module 607, it is understood that it is not a required component of the electronic device and can be omitted as needed without changing the essence of the invention.

[0114] Processor 608 is the control center of the electronic device, connecting all parts of the phone using various interfaces and circuits. By running or executing software programs and / or modules stored in memory 602 and accessing data stored in memory 602, it performs various functions of the electronic device and processes data, thereby providing overall monitoring of the phone. Optionally, processor 608 may include one or more processing cores; preferably, processor 608 may integrate an application processor and a modem processor, where the application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 608.

[0115] The electronic device also includes a power supply 609 (e.g., a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 608 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 609 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0116] Although not shown, the electronic device may further include a camera, a Bluetooth module, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 608 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 602 according to the following instructions, and the processor 608 will run the application programs stored in the memory 602 to implement various functions, such as:

[0117] Get audio data;

[0118] Performing frame processing on the audio data to generate several audio frames;

[0119] Using a first transmission channel to send a plurality of audio frames to the audio decision module, so that the audio decision module obtains a first audio frame;

[0120] Using the second transmission channel to send a plurality of audio frames to the audio decision module, so that the audio decision module obtains a second audio frame;

[0121] The audio decision module determines whether to output the first audio frame or the second audio frame as the current output audio frame.

[0122] In summary, the electronic device provided by the embodiment of the present application obtains audio data; performs frame processing on the audio data to generate a plurality of audio frames; uses a first transmission channel to send the plurality of audio frames to the audio judgment module so that the audio judgment module obtains a first audio frame; uses a second transmission channel to send the plurality of audio frames to the audio judgment module so that the audio judgment module obtains a second audio frame; and determines through the audio judgment module to output the first audio frame or the second audio frame as the current output audio frame. This solution uses two transmission channels to send audio frames, and the audio frames sent by the first transmission channel are mainly output. If an audio frame sent by the first transmission channel is still lost after multiple retransmissions, the audio frame sent by the first transmission channel is used for output. That is, this solution can reduce the phenomenon of audio freeze caused by missing audio frames due to packet loss.

[0123] In the above embodiments, the description of each embodiment has its own focus. For the part that is not described in detail in a certain embodiment, please refer to the detailed description of the audio transmission method above, and will not be repeated here.

[0124] It should be noted that, for the audio transmission method in the embodiment of the present application, those skilled in the art can understand that all or part of the process of implementing the audio transmission method in the embodiment of the present application can be completed by controlling the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, such as in the memory of the terminal, and executed by at least one processor in the terminal. During the execution process, it may include the process of the embodiment of the audio transmission method.

[0125] For the audio transmission device of the embodiment of the present application, its various functional modules can be integrated into a single processing chip, each module can exist physically separately, or two or more modules can be integrated into a single module. The aforementioned integrated modules can be implemented in the form of hardware or software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium.

[0126] To this end, an embodiment of the present application provides a storage medium storing a plurality of instructions that can be loaded by a processor to execute the steps of any of the audio transmission methods provided in the embodiments of the present application. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), or the like.

[0127] The above describes in detail the audio transmission method, device, storage medium and electronic device provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the core idea of ​​the present application. At the same time, for those skilled in the art, based on the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An audio transmission method, which includes: Obtain audio data; Perform frame splitting on the audio data to generate a number of audio frames; Send a number of the audio frames to an audio adjudication module via a first transmission channel, so that the audio adjudication module obtains a first audio frame; Send a number of the audio frames to the audio adjudication module via a second transmission channel, so that the audio adjudication module obtains a second audio frame; Determine, through the audio adjudication module, to output the first audio frame or the second audio frame as a current output audio frame.

2. The audio transmission method according to claim 1, wherein, The determining, through the audio adjudication module, to output the first audio frame or the second audio frame as a current output audio frame includes: When the audio adjudication module receives a current first audio frame within a preset time, use the current first audio frame as the current output audio frame; When the audio adjudication module does not receive the current first audio frame within a preset time, use a target second audio frame corresponding to the current first audio frame as the current output audio frame; Output the current output audio frame according to a preset policy.

3. The audio transmission method according to claim 2, wherein The outputting the current output audio frame according to a preset policy includes: Judge whether the previous output audio frame and the current output audio frame are of the same type; Perform corresponding data processing on the current output audio frame according to the judgment result, and output the processed current output audio frame.

4. The audio transmission method according to claim 3, wherein The performing corresponding data processing on the current output audio frame according to the judgment result includes: When the previous current output audio frame and the current output audio frame are of the same type, perform a first preprocessing on the current output audio frame; When the previous current output audio frame and the current output audio frame are of different types, perform a second preprocessing on the current output audio frame.

5. The audio transmission method according to claim 4, wherein, The performing a first preprocessing on the current output audio frame includes: Adjust the sampling rate of the current output audio frame according to a preset sampling rate.

6. The audio transmission method according to claim 4, wherein, The performing a second preprocessing on the current output audio frame includes: Adjust the sampling rate of the current output audio frame according to a preset sampling rate; Perform a cross-fade process on the adjusted current output audio frame.

7. The audio transmission method according to claim 1, wherein, The performing frame splitting on the audio data to generate a number of audio frames includes: Obtain the transmission frame length of the first transmission channel; Perform frame splitting on the audio data according to the transmission frame length.

8. An audio transmission device, which includes: An audio acquisition unit for obtaining audio data; An audio frame splitting unit for performing frame splitting on the audio data to generate a number of audio frames; A first sending unit for sending a number of the audio frames to an audio adjudication module via a first transmission channel, so that the audio adjudication module obtains a first audio frame; A second sending unit for sending a number of the audio frames to the audio adjudication module via a second transmission channel, so that the audio adjudication module obtains a second audio frame; An audio output unit for determining, through the audio adjudication module, to output the first audio frame or the second audio frame as a current output audio frame.

9. A storage medium, wherein, The storage medium stores multiple instructions, and the instructions are suitable for being loaded by a processor to execute the audio transmission method according to any one of claims 1-7.

10. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, When the processor executes the computer program, it implements the audio transmission method according to any one of claims 1-7.

Images