Wireless audio transmission method, audio device, and storage medium

Abstract

The present application provides a wireless audio transmission method, an audio device, and a storage medium. The method of the present application comprises: first, acquiring a maximum duration T1 from when a transmission end transmits an audio packet to when the transmission end receives an acknowledgement packet fed back by a receiving end, and a duration difference T0 between the transmissions of two adjacent audio packets by the transmission end; and in the event of a potential packet loss and retransmission, on the basis of the maximum duration T1 and the duration difference T0, determining the number of retransmissions, such that the number of retransmissions is better suited to audio transmission, thereby reducing latency and ensuring audio quality.

Description

Wireless audio transmission method, audio device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on September 15, 2023, with application number CN2023112083080 and application name “Wireless Audio Transmission Method, Audio Device and Storage Medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the technical field of audio transmission, and in particular to a wireless audio transmission method, an audio device, and a storage medium. Background Art

[0003] With the widespread adoption of audio technology, a wide variety of applications have emerged. Some specialized applications require high latency in wireless audio transmission, such as esports gaming. When playing, the game screen must be almost perfectly synchronized with the sound transmitted to the user's headphones. Currently, the mainstream wireless audio transmission solution on the market is Bluetooth, but its latency is generally over 200ms. Even after optimization by some manufacturers, the latency is still around 80ms. This high latency is difficult to meet the requirements of wireless audio transmission, thus affecting the user experience.

[0004] Summary of the Invention

[0005] In view of this, the present application provides a wireless audio transmission method, an audio device, and a storage medium, which can improve the problem of high delay in existing wireless audio transmission.

[0006] This application provides a wireless audio transmission method, comprising:

[0007] S1: The sender sends the current audio packet to the receiver;

[0008] S2: Obtain the maximum duration T1 between the sending end sending an audio packet and receiving the response packet fed back by the receiving end, and the duration difference T0 between the sending end sending two adjacent audio packets;

[0009] S3: The sending end detects whether the response packet is received within the maximum time length T1;

[0010] If yes, then execute S41: the sending end sends the next audio packet to the receiving end;

[0011] If not, then executing S42: the transmitting end determines the number of retransmissions of the current audio packet according to the maximum duration T1 and the duration difference T0, and retransmits the number of times; and,

[0012] S5: After determining that the multiple retransmissions are completed, the transmitting end sends the next audio packet to the receiving end.

[0013] Optionally, the S42 includes: determining that T1≤T0 / 2, then determining that the number of retransmissions of the current audio packet is 1, or (T1*n)≤T0, where n is the number of retransmissions of the current audio packet.

[0014] Optionally, the S42 includes: determining that T1>T0 / 2, and then determining that the number of retransmissions of the current audio packet is 1.

[0015] Optionally, the S42 includes: determining that T1>T0 / 2, then determining that the sum of the audio data memory occupied by the retransmission times is equal to the cache of the receiving end, or the difference with the cache of the receiving end is less than the memory occupied by the audio data in the current audio package.

[0016] Optionally, the S2 includes: obtaining the maximum duration T1 and the duration difference T0 according to a preset corresponding relationship; the S42 includes: determining the number of retransmissions of the current audio packet according to the following relationship: T2 = T0 + ((T1*n) / / T0+1)*T0+T3

[0017] T2 is the delay time, T3 is the time from when the sending end sends an audio packet to when the receiving end receives the audio packet, and n is the number of retransmissions of the current audio packet.

[0018] Optionally, when performing the retransmission in S42, the receiving end does not feed back the response packet, and the time difference between two adjacent retransmissions is T1-T4, where T4 is the transmission time of the response packet.

[0019] Optionally, the method further includes:

[0020] It is determined that the multiple retransmissions are completed, and if the receiving end has not received the current audio packet, the audio data of the previous audio packet is copied and used as the audio data of the current audio packet.

[0021] Optionally, in S42, the transmitting end performs lossless encoding on the audio data of the current audio packet and then retransmits the audio data.

[0022] The present application provides an audio device comprising a memory and a processor, wherein the memory stores an audio transmission program, and when the audio transmission program is executed by the processor, the steps of the wireless audio transmission method as described above are implemented.

[0023] The present application provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the wireless audio transmission method as described in any one of the above items are implemented.

[0024] As described above, the present application first obtains the maximum duration T1 from the sender sending an audio packet to the receiver receiving the response packet fed back by the receiver, as well as the duration difference T0 between the sender sending two adjacent audio packets. When facing possible packet loss and retransmission, the number of retransmissions is determined based on the maximum duration T1 and the duration difference T0, so that the number of retransmissions is more suitable for audio transmission, thereby reducing delays and ensuring audio quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] FIG1 is a schematic diagram of a flow chart of a wireless audio transmission method provided in an embodiment of the present application;

[0026] FIG2 is a schematic diagram of retransmission of an audio packet provided in an embodiment of the present application;

[0027] FIG3 is a schematic diagram of another audio packet retransmission method provided in an embodiment of the present application;

[0028] FIG4 is a schematic diagram of another audio packet retransmission provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] To address the aforementioned issues in the prior art, the present application provides a wireless audio transmission method, an audio device, and a storage medium. These several protected themes are based on the same concept and solve the same or similar problems. The implementation methods of the various protected themes can be referenced in conjunction with each other, and any repetitions will not be repeated here.

[0030] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly described below in conjunction with specific embodiments and corresponding drawings. Obviously, the embodiments described below are only some of the embodiments of this application, not all of them. Unless there is a conflict, the following embodiments and their technical features can be combined with each other and also belong to the technical solutions of this application.

[0031] In the description of the embodiments of the present application, the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", etc. to indicate the orientation or position relationship are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the technical solutions of the corresponding embodiments, and do not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation, and should not be understood as a limitation on the present application.

[0032] Please refer to FIG1 , which shows a wireless audio transmission method provided in an embodiment of the present application. The execution subject of each step may be a transmitting end, and the method at least includes the following steps S1 to S5 .

[0033] S1: The sender sends the current audio packet to the receiver.

[0034] In one example, the method is applied to an audio sending device, and the method is used to achieve pairing and connection with at least one audio receiving device, thereby forming an audio transmission system. The audio transmission system can be a one-to-one (one audio sending device corresponds to one audio receiving device) or a one-to-many (one audio sending device corresponds to multiple audio receiving devices) relationship, wherein the audio sending device can serve as a transmitting end, and the audio receiving device serves as a receiving end. The wireless audio transmission method provided in the embodiment of the present application can be based on UWB (Ultra Wide Band) technology. UWB technology has the advantages of low system complexity, low power spectrum density of transmitted signals, insensitivity to channel fading, low interception capability, high positioning accuracy, etc., and is particularly suitable for high-speed wireless access in dense multipath places such as indoors. Therefore, the devices connected by the wireless audio transmission method provided by the present application can bring high-order, fidelity, and low-latency audio wireless transmission due to the use of UWB ultra-wideband technology.

[0035] In one example, the transmitter can first enter pairing mode and further set the current preamble sequence information (e.g., preamble code). Specifically, the preamble sequence information can be set to a pairing character, such as 30. This preamble sequence information is a series of signals sent before sending a useful signal. Its function is to remind the receiver that a valid signal is about to be sent and to pay attention to reception to avoid missing the useful signal. More specifically, it assists the receiver in frequency synchronization, automatic gain control, and other aspects.

[0036] The audio packet contains the audio data to be transmitted and other necessary information. The specific form and frame structure of the audio packet can be found in the prior art and will not be described in detail here.

[0037] S2: Obtain the maximum duration T1 from when the sender sends an audio packet to when the sender receives a response packet fed back by the receiver, and the duration difference T0 between when the sender sends two adjacent audio packets.

[0038] Please refer to Figures 2 to 4 together. Assuming that the transmitter sends an audio packet every 1ms, 1ms of audio data must be accumulated from the beginning before transmission begins; the 1ms is the time difference T0 between the transmitter and the two adjacent audio packets. After each audio packet is sent by the transmitter, if the receiver receives the audio packet, it will return a response packet (i.e., Ack) to the transmitter; if the transmitter does not receive the response packet from the receiver within a certain period of time, the transmitter will retransmit the same data, i.e., send the current audio packet again. Taking Figure 2 as an example, the maximum time T1 from the transmitter sending an audio packet to receiving the response packet from the receiver is 0.4ms, so the certain time is 0.4ms. In actual scenarios, it can be slightly larger than 0.4ms to allow for fluctuations.

[0039] It should be understood that the so-called maximum duration T1 is the time actually felt by the upper-layer application, including: the duration of the upper-layer application at the sending end to send to the system RF (radio frequency), the duration of the protocol required to complete the actual transmission, the duration of the audio packet transmitted over the air, the duration of the response packet fed back by the receiving end, the duration of the sending end receiving the response packet, and the duration of the sending end's RF returning to its upper-layer application layer.

[0040] During audio transmission, the roles of the sending and receiving devices switch. For example, as shown in Figure 2, the sending device completes sending the audio packet at 1ms and switches roles to become the receiver, waiting for the response packet. After receiving the audio packet, the receiving device switches roles to become the sender and sends the response packet. After completing the response packet transmission process, it switches roles again to become the receiver. This process continues in this way until the audio transmission is complete.

[0041] Retransmission means that after the original sender switches its role to become the receiver, if it does not receive a response packet within a maximum time T1, it will resend the original audio packet.

[0042] S3: The sender detects whether a response packet is received within the maximum time T1.

[0043] If yes, then step S41 is executed: the transmitting end sends the next audio packet to the receiving end.

[0044] If not, then execute S42: the sending end determines the number of retransmissions of the current audio packet based on the maximum duration T1 and the duration difference T0, and retransmits the number of times. And,

[0045] S5: After determining that the multiple retransmissions are completed, the transmitting end sends the next audio packet to the receiving end.

[0046] Based on steps S2 and S42, the present application first obtains the maximum time length T1 from the sender sending an audio packet to receiving the response packet, as well as the time difference T0 between the sender sending two adjacent audio packets. When facing possible packet loss and retransmission, the number of retransmissions is determined based on the maximum time length T1 and the time difference T0, so that the number of retransmissions is more suitable for audio transmission, thereby reducing delays and ensuring audio quality.

[0047] In one example, referring to FIG2 , T1 is 0.4 ms and T0 is 1 ms. It can be determined that the maximum duration T1 and the time difference T0 satisfy: T1 ≤ T0 / 2. It can be determined that the number of retransmissions for the current audio packet is 1, that is, 1 retransmission is performed. The fewer the number of retransmissions, the lower the delay.

[0048] Furthermore, if the receiving end does not receive the retransmitted current audio packet and the receiving end does not feedback the response packet to the sending end, the sending end does not receive the response packet; in order to ensure the audio quality, the receiving end can adopt an adaptive packet loss compensation technology to perform packet loss compensation, for example, using PLC (Package Loss Concealment) technology to copy the audio data of the previous audio packet (the most recent audio packet before the current audio packet is transmitted and the sending end successfully receives the response packet feedback from the receiving end) from the receiving end's buffer and use it as the audio data of the current audio packet.

[0049] In another example, if the audio transmission system has high enough transmission efficiency and sufficient bandwidth, and T0 is several times T1, such as in a UWB-based audio transmission scenario, the number of retransmissions, n, can satisfy (T1*n)≤T0, meaning all retransmissions can be within the time difference T0 (i.e., 1ms). Consequently, the receiver's buffering can be small or even non-existent.

[0050] Please refer to Figure 3, T1 is 0.6ms, T0 is 1ms, it can be determined that the maximum duration T1 and the time difference T0 satisfy: T1>T0 / 2, then it can be determined that the number of retransmissions for the current audio packet is 1, that is, 1 retransmission is performed. The fewer the number of retransmissions, the lower the delay.

[0051] The retransmission in this example occupies the sending time period of the next audio packet. Originally, audio packet Data2 must be sent at 3ms, but due to the retransmission, it is now in the retransmission time of the current audio packet Data1. The sending end does not send audio packet Data2, but continues to wait for receiving the response packet. If the response packet is received within the maximum time length T1, or exceeds the maximum time length T1, regardless of whether the response packet is received, the sending end sends the next audio packet (i.e., audio packet Data2) to the receiving end.

[0052] In another example, referring to FIG4 , when it is determined that the maximum duration T1 and the duration difference T0 satisfy: T1>T0 / 2, it can be determined that the sum of the audio data memory occupied by the number of retransmissions is equal to the cache of the receiving end, or the difference with the cache of the receiving end is less than the memory occupied by the audio data in the current audio packet.

[0053] That is, if a retransmission fails, the sender can continue to retransmit by occupying the transmission time slot of subsequent audio packets, including the next audio packet. However, this will cause the subsequent audio packets to be sent later. When performing multiple retransmissions, in order to ensure that the receiver can continue to play audio data during the transmission time slots of subsequent audio packets, the receiver must have sufficient cache to store the audio data that should have been sent during these time slots. In other words, the sender can retransmit continuously, but the upper limit of the number of retransmissions is subject to the receiver's cache.

[0054] In the examples shown in Figures 2 to 4, the overall delay of audio transmission is: 1ms (i.e., T1, the sender must cache T1 to have audio data) + transmission time + receiving end cache time. Among them, the transmission time is the time from the sender sending an audio packet to the receiver receiving the audio packet. In actual scenarios, the data size of each response packet fed back by the receiver is the same, so the time length of each response packet fed back is the same. Taking 0.07ms as an example, the transmission time is T1-0.07ms. In addition, in order to ensure smooth audio playback, the receiving end's cache time is generally the length of time corresponding to multiple retransmissions.

[0055] This application can determine the number of retransmissions of the current audio packet according to the following relationship: T2 = T0 + ((T1*n) / / T0+1)*T0+T3

[0056] T2 is the delay time, T3 is the time from when the sending end sends an audio packet to when the receiving end receives the audio packet, that is, the transmission time, and n is the number of retransmissions of the current audio packet.

[0057] In the relationship, (T1*n) / / T0 is a positive integer obtained by rounding off the division. For example, T0=1ms, T1=0.3ms, and the number of retransmissions n=4, then the buffering time of the receiving end is 2ms. The calculation process is as follows: ((T1*n) / / T0+1)*T0=((0.3*4) / / 1+1)=(1.2 / / 1+1)=2.

[0058] In actual scenarios, if the duration difference T0 is changed, the maximum duration T1 will change accordingly. The maximum duration T1 and the duration difference T0 do not have a linear relationship, because the overhead of each audio transmission includes not only the actual audio data, but also some other overheads, such as leading sequence information, basic protocols, etc. No matter how much memory the transmitted audio data occupies, the memory occupied by these overheads is the same. For different audio transmission chips, the maximum duration T1 and the duration difference T0 may be different, and must be tested according to actual conditions. In one example, the present application can obtain the maximum duration T1 and the duration difference T0 based on a preset corresponding relationship. The preset corresponding relationship can be obtained based on the actual test results of multiple audio transmission chips, and each of the maximum duration T1 has a corresponding duration difference T0.

[0059] According to the relationship formula for calculating the buffering time of the receiving end, that is, ((T1*n) / / T0+1)*T0, the functional relationship for calculating the delay duration f(n) can be combined with f(n)=((T1*n) / / T0+1)*T0, and the delay duration can be calculated by selecting different retransmission times n, different maximum durations T1 and corresponding duration difference T0. In other words, the present application can calculate the optimal maximum duration T1 and corresponding duration difference T0 based on the delay duration that ensures audio transmission and playback quality by adjusting different retransmission times n.

[0060] When performing the retransmission in S42, the receiving end may not feed back the response packet, and the sending end may continuously retransmit. The time difference between two adjacent retransmissions is T1-T4, where T4 is the transmission time of the response packet.

[0061] Here, according to T2=T0+((T1*n) / / T0+1)*T0+T3, the maximum duration T1 is now reduced to only the transmission time, that is, to T1-T4. In some examples, the result of (T1*n) / T0 is just over a certain integer, for example, the result can be 3.1, then the result of (T1*n) / / T0 is 3; if the maximum duration T1 is reduced to only the transmission time, that is, the time required for the acknowledgment packet is removed, the result of (T1*n) / T0 will become 2.9, then the result of (T1*n) / / T0 will be 2, thereby reducing the delay.

[0062] In the step S42, if it is determined that the receiving end has not received the current audio packet after completing the multiple retransmissions, the audio data of the previous audio packet is copied and used as the audio data of the current audio packet. In other words, to ensure audio quality, the receiving end can adopt an adaptive packet loss compensation technology to perform packet loss compensation, for example, using PLC technology to copy the audio data of the previous audio packet (the most recent audio packet before the current audio packet is transmitted and the transmitting end successfully receives the response packet fed back by the receiving end) from the receiving end's cache and use it as the audio data of the current audio packet.

[0063] During the retransmission in S42 , the transmitting end may also perform lossless encoding (Free Lossless Audio Code, FLAC, also known as lossless audio compression) on the audio data of the current audio packet before retransmitting. Using lossless encoding at the transmitting end can reduce the amount of data required for transmission, effectively reducing the value of the maximum duration T1. Furthermore, the required amount of data to be transmitted is reduced, which improves the transmission success rate and also increases the decoding rate at the receiving end. However, the overall delay includes both encoding and decoding time.

[0064] In actual scenarios, since the transmitter needs to accumulate a certain amount of audio data before encoding, the duration that the encoding system buffers the audio data for encoding (referred to as the encoding buffer duration) can be equal to the optimal time difference T0 between two adjacent audio packets sent by the transmitter. If the encoding buffer duration is greater than the optimal time difference T0, then T0 becomes the encoding buffer duration. If the encoding buffer duration is less than the optimal time difference T0, encoding efficiency is likely to be reduced.

[0065] An embodiment of the present application further provides an audio device, including a memory and a processor. The memory stores an audio transmission program, and when executed by the processor, the audio transmission program implements the steps of the wireless audio transmission method described in any of the above examples. The audio device can implement the aforementioned role of a transmitter or a receiver, and the specific form of the audio device is not limited by this application.

[0066] An embodiment of the present application further provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the wireless audio transmission method as described in any example are implemented.

[0067] The storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0068] Since the instructions stored in the storage medium can execute the steps of any wireless audio transmission method provided in the embodiments of the present invention, the beneficial effects that can be achieved by any wireless audio transmission method provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0069] The above descriptions are only some embodiments of the present application and do not limit the patent scope of the present application. For ordinary technicians in this field, any equivalent structural changes made using the contents of this specification and drawings are also included in the patent protection scope of the present application.

[0070] This document uses step codes such as S1 and S2 for the purpose of expressing the corresponding content more clearly and concisely, and does not constitute a substantial limitation on the order. Those skilled in the art may execute S2 first and then S1 during specific implementation, but these should all be within the scope of protection of this application.

[0071] Although the terms "first," "second," and the like are used herein to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. In addition, the singular forms "a," "an," and "the" are intended to include the plural forms as well. The terms "or" and "and / or" are to be interpreted as inclusive, meaning any one or any combination. Exceptions to this definition occur only when a combination of elements, functions, steps, or operations is inherently mutually exclusive in some manner.

Claims

1. A wireless audio transmission method, characterized in that: include: S1: The sender sends the current audio packet to the receiver; S2: Obtain the maximum time length T1 from when the sender sends an audio packet to when the sender receives a response packet fed back by the receiver, and the time length difference T0 between when the sender sends two adjacent audio packets; S3: The sending end detects whether the response packet is received within the maximum time length T1; If yes, then S41 is executed: the sending end sends the next audio packet to the receiving end; If not, executing S42: the transmitting end determines the number of retransmissions of the current audio packet according to the maximum duration T1 and the duration difference T0, and retransmits the number of times; as well as, S5: After determining that the several retransmissions are completed, the sending end sends the next audio packet to the receiving end.

2. The wireless audio transmission method according to claim 1, characterized in that: The S42 includes: If T1≤T0 / 2 is determined, then the number of retransmissions of the current audio packet is determined to be 1, or (T1*n)≤T0, where n is the number of retransmissions of the current audio packet.

3. The wireless audio transmission method according to claim 1, characterized in that: The S42 includes: If it is determined that T1>T0 / 2, then it is determined that the number of retransmissions of the current audio packet is 1.

4. The wireless audio transmission method according to claim 1, characterized in that: The S42 includes: If T1>T0 / 2 is determined, it is determined that the sum of the audio data memory occupied by the number of retransmissions is equal to the buffer of the receiving end, or the difference between the sum and the buffer of the receiving end is less than the memory occupied by the audio data in the current audio packet.

5. The wireless audio transmission method according to claim 1 or 4, characterized in that: The S2 includes: obtaining the maximum duration T1 and the duration difference T0 according to a preset corresponding relationship; The S42 includes: The number of retransmissions of the current audio packet is determined according to the following relationship: T2=T0+((T1*n) / / T0+1)*T0+T3 T2 is the delay time, T3 is the time from when the sending end sends an audio packet to when the receiving end receives the audio packet, and n is the number of retransmissions of the current audio packet.

6. The wireless audio transmission method according to claim 5, characterized in that: When the retransmission is performed in S42, the receiving end does not feed back the response packet, and the time difference between two adjacent retransmissions is T1-T4, where T4 is the transmission time of the response packet.

7. The audio transmission method according to claim 5, characterized in that: The method further comprises: Determine that the several retransmissions are completed, and if the receiving end has not received the current audio packet, copy the audio data of the previous audio packet and use it as the audio data of the current audio packet.

8. The wireless audio transmission method according to claim 5, characterized in that: In the S42, the transmitting end performs lossless encoding on the audio data of the current audio packet and then retransmits the audio data.

9. An audio device, comprising a memory and a processor, characterized in that: An audio transmission program is stored in the memory, and when the audio transmission program is executed by the processor, the steps of the wireless audio transmission method according to any one of claims 1 to 8 are implemented.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the wireless audio transmission method according to any one of claims 1 to 8 are implemented.

Images