Wireless audio transmission methods, systems and devices

Abstract

This application provides a wireless audio transmission method, system, and device, relating to the field of wireless audio transmission technology. The wireless audio transmission method, applied to the transmitting end of a wireless communication system, includes: dividing the duration of an audio frame into multiple time windows aligned with the wireless transmission time slots based on the duration of the wireless transmission time slots; determining the target encoding time slot and target transmission time slot corresponding to the current frame based on the time windows; obtaining the time slot difference between the first time slot where the current audio data processing is located and the target encoding time slot; adjusting a limiting threshold based on the time slot difference; and performing limiting processing on the input audio signal based on the limiting threshold; and in the target encoding time slot, starting an encoder to encode the limited audio signal, and transmitting the encoded audio data to the receiving end of the wireless communication system within the target transmission time slot.

Description

Technical Field

[0001] This application relates to the field of wireless audio transmission technology. More specifically, it relates to a wireless audio transmission method, system, and device. Background Technology

[0002] Wireless audio transmission technology is widely used in various real-time audio interaction scenarios. In related wireless audio transmission methods, the acquisition and framing of audio signals, the start control of the encoder, and the transmission window of the wireless physical layer typically employ a decoupled asynchronous operating mode. To absorb the latency jitter caused by timing decoupling and wireless channel fluctuations, the system usually needs to set up a flexible buffer on the transmission link. However, the existence of this asynchronous scheduling mechanism and buffer not only introduces a non-negligible fixed transmission delay, but also causes end-to-end latency jitter due to the scheduling randomness of encoding and transmission timing, making it difficult to guarantee high real-time performance of wireless audio transmission.

[0003] Furthermore, in the preprocessing stage before audio encoding, to prevent excessive amplitude of the input audio signal from causing numerical overflow and clipping distortion at the encoder input, a limiter is typically used to limit the peak amplitude of the audio signal. The limiting processing in related technologies mainly adjusts the gain based on the instantaneous amplitude envelope of the audio signal, and its limiting threshold is usually fixed and static, completely decoupled from the transmission time slots of the wireless physical layer and the encoder startup sequence. This decoupling means that the system cannot provide timely suppression protection for transient large signals when the audio processing period is close to the encoder startup time, resulting in overflow and clipping during encoding. Conversely, if the limiting threshold is globally lowered to avoid clipping, the audio signal will be over-compressed during non-encoding periods, severely impairing the dynamic range of the audio. Summary of the Invention

[0004] Exemplary embodiments of this application provide a wireless audio transmission method, system, and device that can overcome the defects in existing wireless audio transmissions, such as delay jitter caused by timing decoupling, and encoding overflow distortion and audio dynamic impairment caused by the disconnect between the limiting threshold and the encoding timing.

[0005] The technical solutions provided by some embodiments of this application are as follows: In a first aspect, some embodiments of this application provide a wireless audio transmission method applied at a transmitting end, comprising: dividing an audio frame into time windows aligned with the time slots according to the duration of the wireless transmission time slot, to determine a target encoding time slot and a target transmission time slot; obtaining the time slot difference between the currently processed first time slot and the target encoding time slot and adjusting the limiting threshold accordingly to perform limiting processing on the input audio signal; encoding the limited audio signal in the target encoding time slot, and transmitting it to the receiving end within the target transmission time slot.

[0006] Corresponding to the above methods, some embodiments of this application also provide: a wireless audio transmitting device for implementing the above-described transmitting end operation; a wireless audio receiving device for receiving encoded audio data, decoding it, and directly performing interpolation compensation to prevent retransmission when packets are lost; a wireless communication system including the above-described transmitting end and receiving end; a wireless communication device including a processor and a memory; and a computer-readable storage medium and a computer program product.

[0007] The wireless audio transmission method, system, and device provided in some embodiments of this application, when processing and transmitting audio signals at the transmitting end, align the boundaries of audio frames, the start and stop times of encoding units, and the timing of the target transmission time slot according to the time slots of the wireless physical layer (wireless transmission time slots), and obtain the time slot difference between the first time slot where the current audio data processing is located and the target encoding time slot at the transmitting end. Then, adjust the limiting threshold according to the time slot difference, and perform limiting processing on the input audio signal based on the limiting threshold. When the bound target encoding time slot arrives, start the encoder to encode the limited audio signal to obtain encoded audio data, and transmit the encoded audio data to the receiving end within the bound target transmission time slot.

[0008] Because the wireless audio transmission methods, systems, and devices provided in some embodiments of this application can divide the duration of an audio frame into multiple time windows aligned with the wireless transmission time slots according to the time slots of the wireless physical layer, and determine the target encoding time slot and target transmission time slot corresponding to the current frame according to the time windows, then obtain the time slot difference between the first time slot where the current audio data processing is located and the target encoding time slot, and adjust the limiting threshold according to the time slot difference to perform limiting processing on the input audio signal to obtain the limited audio signal, the embodiments of this application can dynamically tighten the limiting threshold according to the time slot difference to suppress transient signal overload as the audio data processing gradually approaches the target encoding time slot, and then complete the encoding and one-way transmission within the bound target encoding time slot and target transmission time slot. Therefore, the embodiments of this application can solve the problems of delay jitter caused by timing decoupling in wireless audio transmission, and encoding overflow distortion and audio dynamic damage caused by the disconnect between the limiting threshold and the encoding timing, thereby improving the timing stability, real-time performance of data stream transmission, and dynamic fidelity of audio signals in wireless audio transmission. Attached Figure Description

[0009] Figure 1 The present application shows a schematic diagram of the structure of a wireless communication system provided in some embodiments; Figure 2 A flowchart illustrating a wireless audio transmission method provided in some embodiments of this application is shown; Figure 3 The following is a schematic diagram of the dynamic limiting processing provided in some embodiments of this application; Figure 4 This paper illustrates a detailed flowchart of dynamic time-frequency equalization processing based on signal-to-noise ratio and quantile spectrum deviation provided in some embodiments of this application. Figure 5 A schematic diagram illustrating the time synchronization and state transition between the sending and receiving ends provided in some embodiments of this application; Figure 6 A schematic diagram illustrating the timing relationship and retransmission protection between the transmitting end and the receiving end, provided for some embodiments of this application; Figure 7 The following is a detailed flowchart illustrating the receiver packet loss concealment and audio decoding output provided in some embodiments of this application; Figure 8 A schematic diagram of end-to-end audio transmission timing and delay allocation based on hard time-slot pipeline provided for some embodiments of this application; Figure 9 The diagram illustrates a flowchart of a wireless audio transmission method based on wireless physical layer time slot binding, provided in some embodiments of this application. Figure 10 This is a flowchart illustrating the overall data processing and transmission of a dual-microphone wireless audio system provided in some embodiments of this application. Detailed Implementation

[0010] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0011] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0012] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0013] It should be noted that the terms "first" and "second," etc., used in some embodiments of this application are only used to distinguish different objects and are not used to limit the order or importance of the objects. In some embodiments of this application, "connection" can be a direct connection or an indirect connection implemented through one or more intermediate components; it can be an electrical connection, a communication connection, or a data transmission connection.

[0014] The wireless communication system in some embodiments of this application includes at least one transmitter and one receiver. Optionally, the transmitter can be a wireless microphone, live streaming sound card, conference audio equipment, portable audio acquisition device, headphone transmitter, or other wireless audio transmission device with audio acquisition and wireless transmission functions; the receiver can be a wireless receiver (such as a dongle receiver), wireless audio gateway, wireless speaker, wireless headphone receiver, audio adapter, computer external audio device, or other wireless audio receiving device with wireless reception and audio output functions. The embodiments of this application do not limit the specific product form of the transmitter and receiver.

[0015] The transmitting and receiving ends can use 2.4GHz, 5GHz, or other wireless communication frequency bands suitable for audio data transmission. The wireless communication link adopts a time-division multiplexing method, dividing the frame length of an audio frame into multiple wireless physical layer time slots. A wireless physical layer time slot is a time unit in which the wireless transceiver circuit can perform transmission, reception, or switching operations according to predetermined time boundaries. Different wireless physical layer time slots can be configured as audio processing time slots, encoding time slots, wireless transmission time slots, wireless reception time slots, or decoding time slots.

[0016] The audio frame can be a pulse-code modulation audio frame with a fixed frame length, or a fixed-length audio frame formed using other digital audio formats. For example, the frame length of the audio frame can be 2ms, 2.5ms, 5ms, or 10ms. The encoder can be a low-complexity communication codec, a low-complexity communication enhanced codec, or other codecs suitable for low-latency audio transmission. The embodiments in this application are not limited to specific wireless frequency bands, audio frame lengths, or codec types.

[0017] The following describes the inventive concept of some embodiments of this application, namely the development process of the inventive concept of aligning wireless physical layer time slots with audio processing timing.

[0018] Existing wireless audio transmission systems face a fundamental "two-way street" problem: they need to achieve constant low latency and zero jitter from end to end to meet the needs of real-time interaction, while also ensuring high-fidelity audio quality and preventing sound quality degradation such as clipping distortion.

[0019] A deeper analysis of the technical bottlenecks reveals that the problem stems from two levels: First, at the transmission scheduling level, existing solutions generally employ asynchronous task scheduling and a mechanism relying on ACK confirmation / retransmission. This decoupled timing relationship leads to random waiting delays between audio processing, encoding, and wireless transmission stages. ACK waiting and retransmission further introduce uncontrollable delay fluctuations, causing the end-to-end delay of the entire link to jitter within a large statistical range, making it impossible to achieve calculability and constancy. Second, at the audio processing level, to prevent clipping distortion caused by audio signal peaks exceeding numerical ranges during encoding, Dynamic Range Control (DRC) technology is typically used. However, traditional DRC limiters set a static or slowly varying threshold based solely on the amplitude of the audio signal itself. This threshold setting is completely decoupled from the actual start-up timing of the encoder. This presents a new challenge under fixed-slot scheduling: setting a strict threshold to prevent clipping when the encoding time is far off will cause excessive compression of the signal within the entire processing window, severely damaging the dynamic range of the audio; setting a lenient threshold to preserve dynamics will fail to effectively suppress sudden transient peaks near the start of encoding, still leading to clipping distortion. Therefore, traditional solutions cannot achieve a balance between "anti-clipping" and "preserving dynamics".

[0020] In summary, achieving extremely low latency, high stability, high fidelity, and low power consumption audio transmission is a critical technical bottleneck that urgently needs to be addressed in the research and development of wireless communication systems (such as wireless microphone systems). The inventive concepts of some embodiments in this application evolved and were completed gradually in the process of resolving the aforementioned contradictions, and their specific development process is as follows: Analysis of latency jitter caused by traditional acknowledgment (ACK) retransmission mechanism and asynchronous scheduling: The inventors initially attempted to use an ACK confirmation mechanism from related technologies for wireless audio transmission. This involved the transmitter waiting for an ACK confirmation signal from the receiver after sending each data packet; if no ACK confirmation signal was received or an error was detected in the data packet, the transmitter would retransmit the corresponding data packet. However, in actual testing, it was found that the end-to-end latency of this wireless transmission scheme fluctuated randomly between 50ms and 200ms, failing to meet the stringent requirements for lip-sync in professional live streaming, singing, and conference scenarios (typically requiring an end-to-end latency of less than 10ms and no jitter).

[0021] The inventors initially believed that latency fluctuations were mainly caused by data packet retransmissions due to wireless environmental interference. To verify this hypothesis, the inventors conducted comparative tests in a shielded room. The test results showed that although the latency fluctuation amplitude decreased after the wireless interference was significantly reduced, there was still a fluctuation of about 10ms to 20ms, which still could not achieve the target of constant ultra-low latency.

[0022] To address this phenomenon, the inventors meticulously examined each processing stage at both the transmitting and receiving ends, including audio acquisition, audio preprocessing, encoding, wireless transmission, and decoding. The results revealed that end-to-end audio transmission latency is not solely determined by the encoder's encoding duration or the wireless link's transmission rate. Even when the average execution time of each processing stage is within acceptable limits and data packets are successfully delivered, random waiting can still occur between stages due to each stage operating according to its own software task scheduling mechanism. For example, when audio preprocessing is complete, the target transmission slot may have already closed; or when the encoder finishes encoding the current audio frame, the wireless transmission module is processing other tasks, causing the encoded audio data to wait in the buffer for the next target transmission slot; even without retransmissions, the time it takes for the receiver to process the ACK confirmation signal exhibits slight fluctuations.

[0023] Thus, the inventors discovered that the root cause of latency jitter lies not only in wireless interference and data retransmission, but also in the asynchronous scheduling relationship between various processing stages and the ACK handshake mechanism itself. To eliminate this type of random latency, it is not possible to solve it simply by adjusting traditional parameters such as shortening the audio frame length, increasing the processor operating frequency, increasing the encoding buffer, or increasing the wireless transmission rate. These methods cannot eliminate asynchronous waiting, and continuously increasing the processor frequency or wireless transmission power will significantly increase the device's power consumption.

[0024] The introduction and limitations of packet loss concealment mechanisms: To address the transmission problems caused by wireless interference and asynchronous scheduling, the inventors further explored the introduction of a Packet Loss Concealment (PLC) mechanism. When a lost encoded audio data packet is detected, the receiver does not send a retransmission request to the transmitter (i.e., it does not wait for retransmission). Instead, based on one or more historically decoded audio frames, it generates compensating audio data through waveform repetition, amplitude attenuation, linear prediction, or spectral interpolation, and outputs this compensating audio data at the output time corresponding to the current audio frame. This scheme avoids playback interruptions caused by waiting for retransmission and maintains the continuity of audio output even in the event of occasional packet loss.

[0025] However, further analysis and testing by the inventors revealed that the PLC mechanism, being a result compensation mechanism, is used to mitigate the consequences of data packet loss that has already occurred in the wireless link. It cannot constrain the actual start times of audio acquisition, processing, encoding, and wireless transmission within the transmitting end. When timing uncertainties arise at the transmitting end due to task contention, preemption, or buffering, even if the receiving end has PLC capabilities, the actual transmission time of data packets at the transmitting end will still change, and the end-to-end delay remains unstable. Furthermore, frequent calls to the PLC mechanism for result estimation can reduce the consistency between the output audio and the original audio, affecting sound quality. Therefore, relying solely on the PLC mechanism cannot fundamentally solve the latency jitter problem in wireless audio transmission.

[0026] The core concept was established: time slot binding and time slot linkage dynamic limiting.

[0027] Based on the above analysis, the inventors turned to study the time correspondence of each processing stage and found that the key factor affecting the stability of low-latency wireless audio transmission is whether there is a definite time correspondence between the audio frame boundary, encoder start and stop time, target transmission time slot, wireless reception time, and decoding time.

[0028] To address this, the inventors proposed the primary inventive concept: binding the boundaries of audio frames, the start and stop times of encoding units, the target transmission time slot, the wireless reception time, and the decoding time of the receiving end to a unified fixed time slot reference in the wireless physical layer. This ensures that audio processing, encoding, and transmission at the transmitting end, and reception and decoding at the receiving end, are executed sequentially according to the pre-bound physical decoding and transmission time slots. During the initial synchronization phase, the time references of the transmitting and receiving ends are aligned using synchronization data packets and acknowledgment packets. In the steady-state phase, the transmitting end sends encoded audio data in a one-way broadcast format without waiting for acknowledgment signals from the receiving end. When packet loss occurs at the receiving end, the PLC mechanism is invoked in the bound physical decoding time slot, thus completely avoiding interference from acknowledgment responses and retransmissions on the established timing sequence.

[0029] Building upon this foundation, the inventors further investigated the deep relationship between audio processing (such as Dynamic Range Control (DRC)) and encoding. Tests revealed that, with fixed time-slot scheduling, the remaining time between audio preprocessing and the target encoding time slot is not constant. During the waiting period before encoding, the cumulative effect of subsequent audio data may cause the signal amplitude at the encoder input to exceed the encoder's safe dynamic range. Further experiments showed that the probability of clipping distortion is closely related to the time interval between the audio clipping processing moment and the target encoding moment: the further the processing moment is from the encoding moment, the higher the probability of clipping distortion. However, if a low fixed clipping threshold is used throughout the entire processing period, unnecessary compression will be applied to audio signals far from the encoding moment, thus sacrificing the audio's dynamic range.

[0030] To address this emerging contradiction, the inventors proposed a time-slot-linked dynamic limiting scheme. For example, the time-slot difference between the current audio data processing time slot and the target encoding time slot is obtained, and the limiting threshold is dynamically adjusted based on this difference. Specifically, the smaller the absolute value of the time-slot difference, the lower the corresponding limiting threshold (i.e., the stronger the limitation). This way, as the target encoding time slot approaches, the limitation on transient peaks is enhanced, preventing overload of the encoded input; while when the target encoding time slot is far away, a relatively high limiting threshold can be used to preserve the dynamic range of the input audio signal.

[0031] For complex system environments, the inventors proposed the following collaborative optimization concepts for further expansion and optimization. After establishing the concepts of "wireless physical time slot binding" and "time slot linkage dynamic limiting", the inventors further proposed the following collaborative optimization concepts to address practical engineering challenges such as multi-terminal collaboration, wireless interference, audio source diversity, and chip computing power load.

[0032] Dynamic time slot allocation in multi-terminal collaboration: In scenarios where multiple transmitters (multiple senders) work together, if a fixed Time Division Multiple Access (TDMA) time slot allocation is used, the time slots occupied by silent transmitters cannot be used by active transmitters, while active transmitters may experience data backlog as data volume increases. To address this, the inventors introduced a next frame data volume prediction mechanism: based on the Voice Activity Detection (VAD) results and the current frame signal energy, the next frame data volume of each transmitter is predicted, and the transmission time slots are dynamically allocated on the time base accordingly, thereby improving the utilization rate of time slot resources.

[0033] Frequency hopping and retransmission decisions based on slot-level signal quality indicator (SQI) to combat wireless interference: Wireless interference in the 2.4 GHz band typically exhibits frequency concentration, temporal bursts, and correlation between adjacent frequencies, making fixed frequency hopping sequences unsuitable for time-varying interference. To address this, the inventors propose a method that evaluates the signal quality index (SQI) in real-time within each receiving slot and, based on the slot-level SQI, directly determines the frequency hopping strategy for the next slot or frame, as well as whether to execute a retransmission decision in an idle slot, achieving microsecond-level adaptive anti-interference response.

[0034] Content-Aware Adaptive Frame Length Switching: Different audio scenarios, such as voice, music, and silence, have inherently different requirements for latency, bitrate, and sound quality, which cannot be met by a fixed frame length. To address this, the inventors designed a content-aware variable frame length mechanism: based on the characteristics of real-time audio content, the audio frame length and encoding parameters are switched at defined frame boundaries, and the new frame length configuration is communicated to the receiving end via a transition frame, achieving on-demand transmission while ensuring timing alignment.

[0035] DMA (Direct Memory Access) frame-by-frame data transfer to reduce CPU load: In related technologies, CPU-involved sample-by-sample audio data transfer generates numerous interrupts and data copy operations. In high sampling rates (48kHz) and multi-channel scenarios, CPU load can even reach over 60%, with data transfer overhead accounting for more than half. To address this, the inventors propose using a Direct Memory Access (DMA) channel to transfer processed audio data to the encoding buffer. After the DMA channel has completed transferring the entire frame of audio data, only one interrupt is triggered, thus freeing the CPU from the heavy data transfer task and significantly reducing system power consumption.

[0036] Therefore, the technical concept of this application is not merely to adjust the limiting parameters, encoding parameters, or wireless transmission parameters in isolation. Instead, it starts from the fact that traditional parameter adjustments cannot eliminate asynchronous waiting. Further understanding that the PLC mechanism can mitigate the consequences of packet loss but cannot constrain the timing of the transmitting end, it ultimately binds audio frame processing, encoding, wireless transmission, wireless reception, and decoding to a unified wireless physical layer time slot, and correlates the limiting threshold with the time slot difference between the current processing time slot and the target encoding time slot. This reduces random waiting between processing stages, lowering latency and jitter in wireless audio transmission; it also suppresses transient peaks near the encoding start time and preserves a larger audio dynamic range during other processing periods; and combined with steady-state unidirectional transmission and packet loss concealment at the receiving end, it prevents acknowledgment or retransmission processes from disrupting the established audio processing timing.

[0037] In other words, the technical solutions of some embodiments of this application are not isolated adjustments to limiting parameters, encoding parameters, or wireless transmission parameters. Rather, they have undergone a systematic technological evolution from traditional ACK retransmission mechanisms, packet loss concealment mechanisms, and fixed time point and fixed time slot binding, to time slot-linked dynamic range control, dynamic time slot allocation, time slot-level SQI adaptive frequency hopping, audio content-aware variable frame length, and DMA whole-frame transport. In the final technical solution, the above-mentioned technical features are aligned and work together, all bound to a unified wireless physical layer time slot reference. While ensuring extremely low latency and timing stability in wireless audio transmission, high anti-interference capability, high sound fidelity, and ultra-low device power consumption are achieved, constituting a complete inventive concept.

[0038] Based on the above technical concept, the structure of the wireless communication system provided in the embodiments of this application will be described by way of example.

[0039] Figure 1 A schematic diagram of the structure of a wireless communication system provided in some embodiments of this application is shown. The wireless communication system includes a transmitter 100 and a receiver 200.

[0040] The transmitting end 100 may include at least one of the following: an audio acquisition module 110, an analog-to-digital conversion module 120, a timing control module 130, an amplitude limiting processing module 140, a frequency domain equalization module 150, a direct memory access transport module 160, an encoding module 170, a wireless transmission module 180, a memory 190, and a processor 191.

[0041] For example, the audio acquisition module 110 is used to acquire sound signals. The audio acquisition module 110 may include one or more microphones. The analog-to-digital converter module 120 is connected to the audio acquisition module 110 and is used to convert the analog audio signal output by the audio acquisition module 110 into a digital audio signal, and form an input audio signal according to a set sampling rate, sampling bit width, and number of channels. For example, the sampling rate may be 48kHz, the sampling bit width may be 16-bit, 24-bit, or 32-bit, and the input audio signal may be a mono or multi-channel audio signal.

[0042] The timing control module 130 is used to align the timing of audio frame boundaries, coding unit start and stop times, and target transmission time slots according to the time slots of the wireless physical layer. It should be understood that "alignment" here means that the audio frame boundaries, coding unit start and stop times, and target transmission time slots have a predetermined correspondence with the same wireless physical layer time slot counting reference, rather than that the above objects must start or stop at the same time.

[0043] For example, the timing control module 130 can determine the boundaries of the wireless physical layer time slots based on the time slot counting signal or the timer interrupt signal output by the wireless communication chip, and trigger the audio frame processing and encoding module 170 and the wireless transmission module 180 in the designated time slots according to the pre-configured time slot scheduling table. Thus, the processing, encoding, and transmission of audio frames are performed in a pre-determined time slot order.

[0044] The amplitude limiting processing module 140 is used to obtain the time slot difference between the first time slot where the current audio data processing is located and the target encoding time slot, adjust the amplitude limiting threshold according to the time slot difference, and perform amplitude limiting processing on the input audio signal based on the amplitude limiting threshold.

[0045] In some embodiments of this application, the first time slot is the wireless physical layer time slot currently performing audio data processing, and the target coding time slot is the start time slot pre-allocated to the coding module 170 of the current audio frame. The time slot difference can be determined based on the time slot number of the first time slot and the time slot number of the target coding time slot, or it can be calculated based on the remaining duration between the start time of the first time slot and the start time of the target coding time slot. Any time slot difference that can characterize the timing relationship between the current audio data processing time and the target coding time slot can be used as the time slot difference in some embodiments of this application.

[0046] The frequency domain equalization module 150 can be connected between the limiting processing module 140 and the encoding module 170, and is used to perform dynamic gain compensation on the limited audio signal according to the sub-band signal-to-noise ratio of different frequency bands. In some embodiments, the frequency domain equalization module 150 can also determine the dynamic gain according to the quantile spectrum deviation of the input audio signal.

[0047] The Direct Memory Access (DMI) transfer module 160 is connected between the limiting processing module 140 and the encoding module 170. It is used to transfer the limited audio signal to the input buffer of the encoding module 170 via a DMI channel. When the frequency domain equalization module 150 is configured, the DMI transfer module 160 can transfer the frequency domain equalized audio signal to the input buffer of the encoding module 170. By using the DMI transfer module 160, the read and write operations performed by the processor 191 on the audio sampling data can be reduced, allowing audio data transfer to occur in parallel with the processor 191's performance of limiting, equalization, or wireless timing control.

[0048] The encoding module 170 is used to start at the bound target encoding time slot and encode the clipped audio signal to generate encoded audio data. The wireless transmission module 180 is used to transmit the encoded audio data to the receiving end 200 within the bound target transmission time slot.

[0049] Therefore, as a wireless audio transmission device, the transmitter 100 can coordinate audio frame processing, encoding, and wireless transmission under a unified wireless physical layer time slot reference, and associate the audio clipping threshold with the timing relationship before the target encoding time slot, thereby improving the timing stability of the transmitter's audio processing and the dynamic processing capability of the encoded input signal.

[0050] The receiver 200 may include at least one of the following: a wireless receiving module 210, a timing synchronization module 220, a decoding module 230, an audio processing unit 240, a direct memory access module 250, a digital-to-analog conversion module 260, and an audio output module 270.

[0051] The wireless receiving module 210 is used to receive encoded audio data transmitted by the transmitting end 100 within the bound wireless physical receiving time slot. The timing synchronization module 220, based on the time base determined during the initial synchronization phase, starts the decoding module 230 within the bound physical decoding time slot to decode the received encoded audio data. The decoding module 230 is connected to the wireless receiving module 210 and is used to start decoding the encoded audio data within the bound physical decoding time slot. The audio processing unit 240, upon detecting the loss of an encoded audio data packet, stops generating retransmission requests for that encoded audio data packet and invokes a packet loss concealment algorithm to generate compensating audio data based on previously decoded audio data.

[0052] Therefore, when the receiver 200, as a wireless audio receiving device, detects the loss of encoded audio data packets, it does not need to wait for the retransmission of the lost data packets. It can still perform decoding or packet loss hiding processing in the pre-bound physical decoding time slot, thereby maintaining the decoding timing of subsequent audio frames and the continuity of audio output.

[0053] The direct memory access module 250 is used to transfer audio data between the wireless receiving module 210, the decoding module 230, and the digital-to-analog converter module 260. The digital-to-analog converter module 260 is used to convert the decoded digital audio signal into an analog audio signal. The audio output module 270 may include a speaker, a headphone jack, a line-out interface, or a digital audio output interface.

[0054] Through the above system architecture, the transmitter 100 and receiver 200 can coordinate audio frame processing, encoding, wireless transmission, wireless reception and decoding under a unified wireless physical layer time slot reference. Combined with time slot linkage dynamic limiting, steady-state unidirectional transmission and receiver packet loss concealment, the timing stability, real-time performance and dynamic processing capability of the encoded input signal of wireless audio transmission are improved.

[0055] Based on the above description, the flow of some embodiments of the wireless audio transmission method provided in this application will be illustrated below.

[0056] Figure 2The diagram illustrates a flowchart of a wireless audio transmission method according to some embodiments of this application. This method is applied to a wireless communication system including a transmitter and a receiver, and may include steps S201 to S204.

[0057] S201. Divide the duration of the audio frame into multiple time windows that are aligned with the wireless transmission time slots according to the duration of the wireless transmission time slots, and determine the target encoding time slot and target transmission time slot corresponding to the current frame according to the time windows.

[0058] In some embodiments, the transmitting end generates a unified time slot counting reference using a hardware timer, system timer, or frame synchronization signal of the wireless communication chip to obtain the corresponding wireless transmission time slot duration (hereinafter referred to as wireless physical layer time slot). The transmitting end can divide the duration of the audio frame into multiple time windows aligned with the wireless transmission time slots based on the wireless transmission time slot duration, and determine the target encoding time slot and target transmission time slot corresponding to the current frame based on the time windows. Thus, an audio frame is divided into N wireless physical layer time slots, where N is a positive integer greater than 1, and corresponding time slots are allocated for audio processing, encoding, and wireless transmission, respectively. In other words, the transmitting end associates the start boundary of the audio frame, the start time of the encoding unit, and the start time of the target transmission time slot with the corresponding time slot boundaries in the time slot counting reference.

[0059] For example, an audio frame has a frame length of 2.5ms, which is divided into 8 radio physical layer (PTL) time slots, each with a duration of 0.3125ms. Time slots 0 to 2 can be configured as audio acquisition and preprocessing time slots, time slots 3 and 4 as encoding time slots, time slots 5 and 6 as target transmission time slots, and time slot 7 as a guard time slot or timing adjustment time slot.

[0060] In the example above, the starting boundary of the audio frame is aligned with the starting boundary of time slot 0. The encoding module starts at the starting boundary of time slot 3 and outputs the encoded audio data before the end of time slot 4. The wireless transmission module starts wireless transmission at the starting boundary of time slot 5 and completes the transmission of the current audio frame before the end of time slot 6. The number of time slots and the time slots occupied by different processing stages are only examples. Those skilled in the art can configure other time slot division schemes according to the audio frame length, processor performance, encoding complexity, and wireless transmission rate.

[0061] The receiver receives encoded audio data within the wireless physical receive time slot corresponding to the target transmit time slot, based on a time base aligned with the transmitter, and initiates the decoding module in a pre-configured physical decoding time slot. For example, the receiver can receive encoded audio data in the current frame period and initiate the decoding module in the second time slot of the next frame period, thus forming a transmit, receive, and decode pipeline with a defined processing order.

[0062] S202, obtain the time slot difference between the first time slot where the current audio data processing is located and the target encoding time slot.

[0063] In this step, when processing the sampling points or sampling blocks of the current audio frame, the transmitting end determines the first time slot in which the current audio data processing is taking place by reading the time slot counter, hardware timer, or time slot scheduling table of the wireless communication chip, and obtains the target encoding time slot corresponding to the current audio frame. The transmitting end determines the time slot difference based on the time slot number of the first time slot and the time slot number of the target encoding time slot.

[0064] For example, the target encoding time slot is the 3rd time slot. When audio data processing is in the 0th time slot, the absolute value of the time slot difference is 3; when audio data processing is in the 1st time slot, the absolute value of the time slot difference is 2; when audio data processing is in the 2nd time slot, the absolute value of the time slot difference is 1; and when the 3rd time slot is reached, the absolute value of the time slot difference is 0.

[0065] S203 adjusts the limiting threshold according to the time slot difference and performs limiting processing on the input audio signal based on the limiting threshold.

[0066] In a specific example, the clipping processing module can pre-store the basic clipping threshold and the correspondence between the time slot difference and the threshold correction amount. The clipping processing module determines the threshold correction amount based on the time slot difference and obtains the current clipping threshold based on the basic clipping threshold and the threshold correction amount.

[0067] The relationship between the time slot difference and the threshold can be set as follows: the smaller the absolute value of the time slot difference, the lower the calculated threshold. This correspondence can be implemented using a continuous function, a piecewise function, or a lookup table. For example, the continuous function can be a sigmoid function, an exponential function, or other monotonically changing function; the piecewise function can be a piecewise linear function or a step function; and the lookup table can store the threshold or threshold correction amount corresponding to different discrete time slot differences.

[0068] For example, when the full-scale amplitude of the digital audio signal is normalized to 1, the basic limiting threshold can be set to 0.70. When the target encoding time slot is the 3rd time slot, the following lookup table can be pre-configured: when the absolute value of the time slot difference is 3, the limiting threshold is 0.70; when the absolute value of the time slot difference is 2, the limiting threshold is 0.65; when the absolute value of the time slot difference is 1, the limiting threshold is 0.58; and when the absolute value of the time slot difference is 0, the limiting threshold is 0.50. It should be understood that the above values ​​are only examples, and other limiting thresholds can be determined based on the encoder's input range, the peak-to-average power ratio of the input audio signal, and the allowable dynamic range.

[0069] In some embodiments, the threshold correction amount can be determined based on the time slot difference and a preset adjustment coefficient, and the limiting threshold can be obtained by subtracting the threshold correction amount from the base limiting threshold. When the absolute value of the time slot difference decreases, the threshold correction amount increases, thereby lowering the limiting threshold. The preset adjustment coefficient can be calibrated through offline audio testing or encoder input overload testing.

[0070] The amplitude limiting module compares the amplitude of the input audio signal's sample point with the current amplitude limiting threshold. If the amplitude of the input audio signal's sample point is not greater than the amplitude limiting threshold, the amplitude limiting module can set the gain of that sample point to 1. If the amplitude of the input audio signal's sample point is greater than the amplitude limiting threshold, the amplitude limiting module calculates the attenuation gain according to the ratio of the amplitude limiting threshold to the sample point amplitude, and multiplies the input audio signal by the attenuation gain.

[0071] Specifically, the absolute value of the sample point amplitude can be compared with the limiting threshold. For example, if the current limiting threshold is 0.58 and the amplitude of a sample point of the input audio signal is 0.80, then the attenuation gain can be set to the ratio of 0.58 to 0.80, i.e., 0.725. Multiplying 0.80 by 0.725 yields an output sample point with an amplitude of 0.58. For sample points with negative amplitudes, their sign can be preserved, and the attenuation gain can be calculated based on their absolute amplitude value.

[0072] In other embodiments, to reduce the abrupt gain changes caused by amplitude limiting threshold variations, the attenuation gain of adjacent sampling points or adjacent sampling blocks can be smoothed. Smoothing can be achieved by using first-order filtering with attack and release time parameters, or by weighting using a smoothing window function corresponding to the audio frame.

[0073] Specifically, the limiting processing module obtains a smoothing window function. The smoothing window function has a first set value in the middle of an audio frame and smoothly changes between the first and second set values ​​during the transition segments at the beginning and end of the audio frame. For example, the first set value can be 1, and the second set value can be 0 or a value greater than 0 and less than 1. During the initial transition segment of the audio frame, the window function value smoothly changes from the second set value to the first set value; during the final transition segment of the audio frame, the window function value smoothly changes from the first set value to the second set value.

[0074] Optionally, the smoothing window function can be a Hanning window, a square root window, a sine window, or other window functions that meet the requirements for transition between the beginning and end of the audio frame. The clipping module uses the smoothing window function to perform weighted operations on the clipped audio signal to smooth the gain changes at the switching points of the audio frames.

[0075] For example, the first 20% of the sampling points at the beginning of an audio frame can be set as the starting transition segment, the last 20% as the ending transition segment, and the middle 60% as the intermediate segment. The window function value for the intermediate segment is set to 1, the window function value for the starting transition segment varies from 0.8 to 1, and the window function value for the ending transition segment varies from 1 to 0.8. The proportion of the transition segment to the audio frame, the first setting value, and the second setting value can be set according to the frame length, sampling rate, and gain change.

[0076] S204, in the target encoding time slot, start the encoder to encode the amplitude-limited audio signal, and send the encoded audio data to the receiving end in the target transmission time slot.

[0077] For example, the timing control module outputs an encoding trigger signal to the encoding module at the start boundary of the target encoding time slot. In response to the encoding trigger signal, the encoding module reads the clipped audio signal from the encoding buffer, performs frame encoding on the audio signal, and generates an encoded audio data packet carrying the frame number, time slot identifier, and audio payload.

[0078] The timing control module outputs a transmit trigger signal to the wireless transmit module at the start boundary of the target transmit time slot. In response to the transmit trigger signal, the wireless transmit module transmits the encoded audio data to the receiver within the bound target transmit time slot.

[0079] In some implementations, the transmitting end adjusts the limiting threshold based on the time slot difference between the first time slot where the current audio data processing is taking place and the target encoding time slot, and performs limiting processing on the input audio signal based on the adjusted limiting threshold.

[0080] Specifically, in this example, the difference between the slot index of the first time slot and the slot index of the target encoding time slot is determined as the slot difference value. For example, the slot index of the first time slot where the current audio data processing is located is denoted as S.curr Let S be the slot index of the target coding slot corresponding to the current audio frame. encode Then the time slot difference can be expressed as:

[0081] Wherein, ΔS represents the time slot difference between the first time slot and the target coding time slot, and its unit can be the number of time slots; S curr The time slot index represents the first time slot in which the current audio data processing is taking place; S encode This represents the slot index of the target coding slot. The slot index is an integer, therefore, ΔS can also be an integer.

[0082] For example, when the first time slot is no later than the target coding time slot, a dynamic clipping threshold is calculated based on the time slot difference, such that the dynamic clipping threshold gradually decreases as the first time slot approaches the target coding time slot. Specifically, the first time slot is no later than the target coding time slot, i.e., it satisfies:

[0083] That is, ΔS ≤ 0. Within this pre-encoding processing interval, as the first time slot of the current audio data processing gradually approaches the target encoding time slot, ΔS gradually approaches 0 from a small negative value, and the corresponding |ΔS| gradually decreases. That is, the smaller the absolute value of the time slot difference, the lower the clipping threshold.

[0084] The amplitude limiting module calculates the threshold correction amount based on the time slot difference. The threshold correction amount can be expressed as:

[0085] in, Indicates the first Threshold correction amount corresponding to each sampling point; Indicates the audio sample point index; This represents the maximum downward adjustment depth of the limiting threshold, and can also be called the compression strength coefficient; This represents the steepness coefficient of the Sigmoid curve, used to control the rate at which the threshold changes due to the time slot difference; Represents the natural constant.

[0086] The clipping module subtracts a threshold correction amount from the basic clipping threshold to obtain the first... The dynamic limiting threshold corresponding to each sampling point:

[0087] Substituting equation (3) into equation (4), we obtain the formula for the time-slot linkage dynamic limiting threshold:

[0088] Or it can be expressed as:

[0089] in, Indicates the first The dynamic limiting threshold used for each sampling point; This represents the basic limiting threshold, which is the basic safe upper limit of the encoder input signal. It can be expressed in negative decibel full-scale dBFS. This is the threshold attenuation coefficient, used to represent the maximum downward adjustment of the limiting threshold; The threshold change steepness coefficient represents the rate at which the time slot difference affects the amplitude limiting threshold. Therefore, in Within the interval, with the first time slot Gradually approaching the target coding slot The calculated dynamic limiting threshold It exhibits a monotonous, gradual decreasing trend.

[0090] When the input audio signal is represented using full-scale normalization, the amplitude range of the input audio signal can be... Accordingly, , , as well as These are all dimensionless normalized amplitude values. For example, they can be set as follows:

[0091]

[0092]

[0093] In a specific example, the sampling rate is 48kHz, the audio frame length is 10ms, and the full-scale amplitude of the input audio signal is normalized to 1. This can be set as follows:

[0094] It should be understood that the above parameters are merely examples and do not constitute a limitation on the scope of protection of this application. In practical applications, the above parameters can be calibrated according to the encoder's input quantization range, the peak-to-average power ratio of the input audio signal, the target audio quality, and the allowable compression intensity.

[0095] According to equation (5), in the pre-coding processing interval Within this time slot, the time slot difference exhibits the following variation pattern.

[0096] when When the first time slot of the current audio data processing is far from the target encoding time slot, we have:

[0097] At this point, the amplitude limiting threshold is relatively high, and small signals and general amplitude signals in the input audio signal can maintain a large dynamic range.

[0098] when That is, when the current audio data processing reaches the target encoding time slot, we have:

[0099] When the current processing time slot is far from the fixed coding time slot, the exponential term in the formula reduces the threshold attenuation term, resulting in a relatively lenient dynamic limiting threshold. When the current processing time slot is close to the fixed coding time slot, the dynamic limiting threshold approaches... The limiting strategy is relatively strict, that is, a relatively strict limiting threshold is applied to the audio signal that is about to enter the encoder in order to suppress transient peaks in the encoded input signal.

[0100] In embodiments that allow preprocessing of post-buffered data following the target coding slot, when At that time, there were:

[0101] At this point, the limiting threshold reaches or is close to its minimum value, allowing for stronger limiting of sudden peaks in the back-cached data.

[0102] Therefore, within the pre-encoding processing interval, as the first time slot of the current audio data processing approaches the target encoding time slot, the absolute value of the time slot difference decreases, and the limiting threshold calculated by equation (5) decreases, enabling the transmitter to enhance the limitation of transient peaks when it is close to the target encoding time slot; when it is far from the target encoding time slot, a relatively high limiting threshold is used to reduce unnecessary dynamic compression.

[0103] This processing method differs from traditional dynamic range control methods that only set amplitude limiting parameters based on input amplitude or short-time energy. It associates the audio processing parameters before encoding with the time slot scheduling relationship of the wireless physical layer, which can preserve more audio dynamic range while protecting the safety margin of the encoded input.

[0104] Next, an example illustration will be given of the attenuation gain and the limiting output signal.

[0105] In some implementations, after calculating the dynamic limiting threshold, the limiting processing module performs limiting processing on the input audio signal based on the dynamic limiting threshold. Specifically, this includes: obtaining the absolute amplitude of the sample points of the input audio signal; and if the absolute amplitude of the sample points is greater than the dynamic limiting threshold, calculating the attenuation gain according to the ratio of the dynamic limiting threshold to the absolute amplitude of the sample points, and multiplying the input audio signal by the attenuation gain.

[0106] Specifically, the amplitude limiting module acquires the first... sampling points And the absolute value of the amplitude at that sampling point. With dynamic limiting threshold Compare them.

[0107] When the amplitude of a sample point of the input audio signal is not greater than the dynamic limiting threshold, the limiting gain corresponding to that sample point is set to 1; when the amplitude of a sample point of the input audio signal is greater than the dynamic limiting threshold, the attenuation gain is calculated according to the ratio of the dynamic limiting threshold to the sample point amplitude. The limiting gain can be expressed as:

[0108] in, Indicates the first The limiting gain corresponding to each sampling point is a dimensionless value; Indicates the first The amplitude of the input audio signal at each sampling point; Indicates the first The dynamic limiting threshold corresponding to each sampling point.

[0109] Next, the input audio signal is multiplied by the calculated clipping gain. Without applying a frame boundary smoothing window function, the clipped audio signal can be expressed as:

[0110] in, Indicates the first The output audio signal, sampled at each point, has been amplitude-limited but not yet weighted by a smooth window. This is achieved by inputting the audio signal... Multiply by attenuation gain This allows the excessive signal amplitude to be precisely pulled back to the dynamic limiting threshold boundary.

[0111] According to equations (14) and (15), when At that time, there were:

[0112] That is, small signals are not compressed, thus preserving weak speech, consonants and other low-amplitude audio details.

[0113] when At that time, there were:

[0114] That is, sampling points exceeding the dynamic limiting threshold are proportionally attenuated to the dynamic limiting threshold, while retaining the positive or negative sign of the input sampling points.

[0115] For example, the current dynamic limiting threshold The value is 0.58, and the input sampling point is... If the value is 0.80, then the limiting gain is:

[0116] The corresponding limiting output is:

[0117] If input sampling points for Therefore, the limiting gain remains 0.725, and the limiting output is... .

[0118] By calculating the attenuation gain according to the ratio of the dynamic limiting threshold to the sampling point amplitude, it is possible to avoid directly truncating sampling points that exceed the encoder's safe input range at the full-scale boundary, and to limit the transient peaks entering the encoding module 170, thereby reducing the probability of encoding input overload and quantization clipping distortion.

[0119] Figure 3 The flowchart of dynamic limiting processing provided in some embodiments of this application is shown. To make the time-slot-linked adaptive limiting processing (DRC) process in some embodiments of this application clearer, the following is a detailed explanation... Figure 3 This section provides a detailed explanation of the specific processing flow for each audio sampling point in the limiting module at the transmitting end. For example... Figure 3 As shown, the adaptive clipping process specifically includes the following steps: S301, Input PCM audio sampling point .

[0120] The transmitting end acquires the time-domain discrete pulse code modulation (PCM) audio data after front-end processing (such as gain compensation) and inputs it into the limiting processing module in units of sampling points. This represents the time-domain index of the current sampling point.

[0121] S302, Read the current time slot index With target coding slot index Calculate the time slot difference .

[0122] In this step, the limiting processing module queries the system timing control unit in real time for the time slot index of the first time slot in which the current audio sampling point is being processed. And the slot index of the wireless target coding slot pre-bound to the current audio frame. The difference between the two is defined as the time slot difference. (i.e., satisfy) ).

[0123] S303. Calculate the threshold correction amount based on the Sigmoid function to obtain the adaptive limiting threshold. .

[0124] The limiting processing module will process the time slot difference calculated in step S402. Substituting the preset nonlinear mapping function (such as the Sigmoid function shown in Equation 3), the threshold correction amount corresponding to the current sampling point is calculated. Subsequently, from the basic limiting threshold Subtracting this threshold correction amount from the middle yields an adaptive limiting threshold that is linked to the time slot difference. .

[0125] S304. Obtain the absolute value of the amplitude at the sampling point. .

[0126] The limiting processing module limits the input PCM audio sampling points. Performing absolute value calculations yields the absolute amplitude of the sampling point, which reflects the instantaneous energy of the signal. .

[0127] S305, Judgment Whether it is valid or not.

[0128] The amplitude limiting module will process the absolute amplitude of the sampling points obtained in step S404. The adaptive limiting threshold calculated in step S403 Perform a magnitude comparison. If the absolute amplitude of the sampling point is greater than the adaptive amplitude limiting threshold (step S405, yes), then proceed to step S406; if the absolute amplitude of the sampling point is not greater than the adaptive amplitude limiting threshold (step S405, no), then proceed to step S407.

[0129] S306, Calculate attenuation gain .

[0130] When the input signal amplitude is overloaded and triggers amplitude limiting, the amplitude limiting processing module applies an adaptive amplitude limiting threshold. Absolute amplitude of the current sampling point The attenuation gain (i.e., the limiting gain) is calculated proportionally. This is to ensure that the absolute amplitude of the signal can be precisely compressed to the amplitude limiting threshold boundary.

[0131] S307, Maintain Limiting Gain .

[0132] When the input signal amplitude is within a safe range and does not trigger the limiting function, the limiting processing module will set the limiting gain corresponding to the sampling point. Setting it to 1 means that no amplitude attenuation is applied to this sampling point.

[0133] S308, Loading Smooth Window Function Weighted smoothing is applied to the limiting gain.

[0134] Determine the initial limiting gain corresponding to each sampling point. Then, the amplitude limiting module loads the preset smoothing window function. (As shown in Equation 23, a piecewise window function). The limiting gain is weighted and smoothed using a smoothing window function to eliminate transient steps caused by changes in time slot differences or abrupt changes in limiting action at audio frame switching, thus smoothing the gain changes of the audio frame at the switching point.

[0135] S409, Output the limited audio signal .

[0136] The limiting processing module converts the original input audio signal Calculated limiting gain and smooth window function Perform product weighting operations to generate and output the final amplitude-limited and smoothed audio signal. It is then sent to the subsequent equalization (EQ) or encoding buffer.

[0137] Thus, the adaptive clipping processing flow provided in some embodiments of this application, by hard-binding the physical layer time slot scheduling state with the clipping threshold, actively tightens the threshold when the time slot approaches using nonlinear mapping, effectively preventing large signals near the encoding time from causing encoding overflow or clipping distortion. Secondly, based on precise detection and proportional attenuation per sample point, the dynamic range of the audio is preserved to the maximum extent while preventing overload, reducing nonlinear harmonic distortion. Finally, by weighting the clipping gain through a smooth window function, the step gain jumps caused by time slot switching or clipping abrupt changes are transformed into smooth gradients, completely eliminating discontinuous noise and pumping effects at frame boundaries, significantly improving the smoothness of continuous audio output and subjective listening experience.

[0138] Next, we will provide an illustrative explanation of the smoothing window function and weighted operations.

[0139] After the input audio signal is subjected to amplitude limiting processing according to Equation (14), in order to suppress the sudden gain change caused by time slot switching and dynamic amplitude limiting threshold change, the amplitude limiting processing also includes: obtaining a smoothing window function, wherein the gain value of the smoothing window function in the middle time period of the audio frame is a first set value, and the gain value in the transition section at the beginning and end of the audio frame smoothly changes from the first set value to the second set value; and using the smoothing window function to perform weighted operation on the amplitude-limited audio signal to smooth the gain change of the audio frame at the switching point.

[0140] Specifically, the amplitude limiting module obtains the smoothing window function. The audio signal after amplitude limiting is weighted using a smooth window function.

[0141] The formula for weighted output using a smooth window is:

[0142] Or it can be expressed as:

[0143] in, This represents the output audio signal after dynamic limiting and smoothing window weighting; Let be the smoothing factor for the soft inflection point (i.e., dimensionless), representing the th . The smoothing window function value (i.e., gain value) corresponding to each sampling point; the meanings of other symbols are the same as in the aforementioned embodiments.

[0144] In one implementation, the soft inflection point smoothing factor is derived using the following recursive formula: .

[0145] Wherein W(n) 1) represents the soft inflection point smoothing factor of the previous sampling point; α is the smoothing coefficient, ranging from 0 to 1; |G(n) 1| represents the absolute difference between the current limiting gain coefficient and the unity gain. To avoid sign repetition, in hardware or software implementations, the threshold attenuation coefficient α in the dynamic threshold formula and the smoothing coefficient α in the soft inflection point formula can use different parameter storage locations and be configured with corresponding calibration values.

[0146] In some alternative implementations, the smoothing window function may also employ a Hanning window, a square root synthesis window, a sine window, or other window functions that can smoothly change the gain during the transition between the beginning and end of an audio frame. The smoothing window function may only apply to the beginning and end transitions of the audio frame (e.g., transition intervals each accounting for 20% of the audio frame). During the middle period of the audio frame (i.e., the main body interval), the window function value (i.e., the gain value) remains at a first set value to reduce the impact on the dynamic range of the main body of the audio frame; while during the beginning and end transitions, the gain value smoothly transitions from the first set value to a second set value, thereby avoiding abrupt changes in gain.

[0147] As an optional implementation, an audio frame includes Each sampling point, the beginning and end transition sections respectively include There are 1 sampling point, of which:

[0148] in, This indicates the proportion of the first or last transition segment in the audio frame; for example... 0.2 can be taken; This indicates the floor function.

[0149] The smooth window function can be taken in the following piecewise form:

[0150] in, The first set value representing the middle time segment of an audio frame; This represents the second set value at the beginning and end boundaries of the audio frame; Indicates the number of sample points included in an audio frame; This indicates the number of sampling points included in the beginning or end transition section.

[0151] According to equation (23), in The head transition section, the gain value starts from the second set value at the head boundary. The first set value for a smooth transition to the middle time period ;exist During the middle period, the gain value remains at the first set value. ;exist The gain value at the tail transition section starts from the first set value. The second set value smoothly transitions to the tail boundary. .

[0152] For example, it can be set as follows:

[0153] For example, The value can be 0, 0.5, or 0.8. When overlapping and summing are required, a square root Hanning window can be used, and the overlapping portion of adjacent audio frames should satisfy the constant gain reconstruction condition to avoid overall amplitude attenuation of the continuous output audio. In this way, by performing the weighting operation, the gain abrupt changes of the audio frames at time slot switching and dynamic limiting threshold changes are smoothed, thereby suppressing the pumping effect and intermittent noise at audio frame switching.

[0154] In other implementations, the smoothing window function may not act directly on the audio signal amplitude, but rather on the limiting gain of adjacent sampling points or adjacent audio frames. For example, the smoothed gain can be expressed as:

[0155] in, This represents the smoothed limiting gain; This represents the limiting gain corresponding to the previous audio frame or the previous processing interval. The corresponding output audio signal can be represented as:

[0156] By smoothly changing the window function value or the limiting gain during the transition between the beginning and end of an audio frame, the sudden gain changes caused by time slot switching and dynamic limiting threshold changes can be reduced, thereby suppressing the pumping effect and intermittent noise at the audio frame switching point.

[0157] Thus, this technical solution introduces a time-slot-linked limiting threshold during dynamic range control. The limiting process is no longer determined solely by the amplitude of the input audio signal, but also by the distance between the current processing time slot and the encoding time slot. When the processing time is close to the encoding time, the limiting threshold becomes stricter, which helps prevent the encoder input from exceeding the dynamic range; when the processing time is far from the encoding time, the limiting threshold is relatively looser, which helps preserve the audio dynamic range. By smoothing gain changes using a soft inflection point smoothing factor, the abrupt changes and audible "click" sounds caused by fixed-threshold hard limiting at the threshold can be reduced.

[0158] Next, an example of data movement based on direct memory access channels will be provided.

[0159] In some embodiments, after clipping and before encoding the audio signal, the transmitter 100 moves the clipped audio signal to the encoding buffer via a direct memory access channel.

[0160] Specifically, the processor 191 can pre-configure the source address, destination address, data width, single transfer length, and trigger source of the direct memory access channel. The source address can be the output buffer of the clipping processing module 140, the destination address can be the input buffer of the encoding module 170, and the trigger source can be a clipping processing completion interrupt, a time slot boundary signal, or an event that the amount of data in the buffer reaches a set value.

[0161] In response to a trigger source, the direct memory access transport module 160 transports the clipped audio signal from the output buffer to the encoding buffer, and outputs a transport completion signal to the processor 191 or the encoding module 170 after the transport is completed. Therefore, the processor 191 does not need to perform point-by-point read and write operations for each audio sample point.

[0162] By using direct memory access channels to move audio data, the number of times the processor 191 performs read and write operations on the audio sample data can be reduced, allowing the processor 191 to devote more runtime to limiting, equalization, timing control or other audio processing tasks, thereby reducing the probability of processing exceeding the specified time slot due to processor task contention.

[0163] In an embodiment where the frequency domain equalization module 150 is configured, the source address of the direct memory access channel can be configured as the output buffer of the frequency domain equalization module 150, while the target address remains the input buffer of the encoding module 170. The direct memory access channel can also employ a dual-buffer structure. When the encoding module 170 reads the current audio frame from the first buffer, it moves the next audio frame to the second buffer and swaps the read / write attributes of the first and second buffers before the next target encoding time slot arrives.

[0164] By using a dual-buffered structure, the encoding of the current audio frame and the data transfer of the next audio frame can be carried out in parallel, thereby improving pipeline processing efficiency and reducing the probability of the encoding module 170 waiting for input data.

[0165] Next, we will provide an illustrative example of subband signal-to-noise ratio and dynamic time-frequency equalization.

[0166] In some embodiments, after clipping the input audio signal and before encoding the audio signal, the method may further include: obtaining the subband signal-to-noise ratio (SNR) of the input audio signal in different frequency bands; calculating a dynamic gain based on static equalization parameters and the subband SNR, wherein the dynamic gain is increased in frequency bands where the subband SNR is higher than a set SNR threshold, and decreased in frequency bands where the subband SNR is lower than the set SNR threshold; and performing gain compensation on the audio signal using a filter based on the dynamic gain. That is, the transmitting end also performs dynamic frequency domain equalization on the clipped audio signal.

[0167] For example, the frequency domain equalization module 150 acquires the subband signal-to-noise ratio (SNR) of the input audio signal at different frequency bands, and calculates the dynamic gain based on the static equalization parameters and the subband SNR. Specifically, the dynamic gain is increased in frequency bands where the subband SNR is higher than a set SNR threshold, and decreased in frequency bands where the subband SNR is lower than the set SNR threshold. The frequency domain equalization module 150 uses a filter to perform gain compensation on the audio signal based on the dynamic gain.

[0168] Specifically, the frequency domain equalization module 150 can divide the audio signal into multiple sub-bands using a filter bank or frequency domain transformation, and estimate the signal power and noise power of each sub-band separately. The sub-band signal-to-noise ratio can be determined based on the ratio of the sub-band signal power to the sub-band noise power. The noise power can be obtained from frames without speech, historical low-energy frames, or the output of a noise estimator.

[0169] In some implementations, the transmitting end can use a frequency domain equalization module to divide the input audio signal into different frequency bands using short-time Fourier transform, improved discrete cosine transform, or filter banks, to obtain multiple time-frequency sub-bands (i.e., different frequency bands). The frequency point or sub-band index is denoted as... Record the audio frame number as .

[0170] As an optional implementation, the signal-to-noise ratio of a sub-band can be calculated based on the signal power and noise power of each sub-band. Let the first sub-band... Frame, First The total power of each subband is The estimated noise power is Then the effective signal power of this sub-band can be expressed as:

[0171] The corresponding linear subband signal-to-noise ratio can be expressed as:

[0172] in, This indicates a positive number that prevents the denominator from being zero; its values ​​can be: to The specific value can be determined based on the precision of fixed-point or floating-point operations.

[0173] The subband signal-to-noise ratio, expressed in decibels, can be:

[0174] In the dynamic time-frequency equalization formula, the linear sub-band signal-to-noise ratio (SNR) can be mapped to an SNR weight between 0 and 1. Based on this SNR weight, a set SNR threshold is compared to determine whether dynamic equalization compensation should be performed on the frequency band.

[0175] thus:

[0176] Next, the dynamic gain is calculated based on the static equalization parameters and the sub-band signal-to-noise ratio. The dynamic time-frequency equalization formula (i.e., the dynamic gain formula) is as follows:

[0177] in: This indicates a frequency point index or a time-frequency sub-band index; Indicates the audio frame number; Indicates the first Frame, First Dynamic equalization gain of a frequency point or sub-band; Indicates the first Static equalization parameters corresponding to each frequency point or sub-band are used to compensate for the fixed frequency response of the microphone, analog front end or acoustic cavity. This represents the equalization adjustment coefficient, used to control the compensation strength of dynamic gain; Indicates the first Frame, First The signal-to-noise ratio of a frequency point or sub-band, or the weight determined by that signal-to-noise ratio; Indicates the first The target quantile spectrum corresponding to each frequency point or sub-band; Indicates the input audio signal at the th Frame, First The current quantile spectrum of a frequency point or sub-band.

[0178] According to equation (32), the subband signal-to-noise ratio weights are adjusted accordingly. When the signal-to-noise ratio (SNR) of the corresponding sub-band is higher than the set SNR threshold in the product term, the weights are adjusted accordingly. The system approaches 1 in terms of static equilibrium parameters. Based on this, a dynamic gain compensation term is added, thereby increasing the dynamic gain of this frequency band; conversely, when the sub-band signal-to-noise ratio of this frequency band is lower than the set signal-to-noise ratio threshold, the weights are adjusted. As the value approaches zero, the dynamic gain compensation term is suppressed (i.e. reduced), thereby reducing the dynamic gain of the frequency band, or even degenerating it to be determined solely by the static equalization parameter, thus preventing the synchronous amplification of environmental noise in low signal-to-noise ratio frequency bands.

[0179] In equation (32), and It can be represented using linear amplitude gain; This is a dimensionless adjustment coefficient, and its exemplary value range can be:

[0180] when , , as well as When all parameters are expressed using the linear amplitude domain, they are dimensionless linear amplitude quantities. If the power spectrum is used, the parameters should be uniformly expressed using the linear power domain. When calculating equation (32), it is not advisable to directly add the value expressed in dB to the linear amplitude gain; when the signal-to-noise ratio is expressed in dB, it can be converted into a dimensionless weight first according to equation (30) or other monotonic mapping functions.

[0181] Optionally, the static equalization parameters can be pre-calibrated based on the frequency response of the microphone, speaker, acoustic cavity, or analog front end, and stored in the non-volatile memory of the transmitter 100. The static equalization parameters may include the center frequency, quality factor, and fundamental gain corresponding to different frequency bands.

[0182] In some embodiments, the static equalization parameters can be limited to a preset range. For example, in the linear amplitude domain, the static equalization gain can be configured as follows:

[0183] For example, It can be corresponding to linear gain in dB It can be corresponding to Linear gain in dB. Dynamic equalization gain can also be limited:

[0184] in, and These represent the minimum and maximum allowable dynamic equalization gain, respectively. By limiting the dynamic equalization gain, excessive equalization gain due to instantaneous spectral deviations can be avoided.

[0185] Since the dynamic equalization gain is gated by the subband signal-to-noise ratio, when the subband signal-to-noise ratio is high, the gain compensation can be performed more fully based on the deviation between the target quantile spectrum and the input quantile spectrum; when the subband signal-to-noise ratio is low, the dynamic compensation amount is reduced accordingly, thereby reducing the synchronous amplification of the noise floor by the equalization process.

[0186] For example, the input audio signal can be divided into low-frequency, mid-frequency, and high-frequency sub-bands, with a signal-to-noise ratio (SNR) threshold of 15 dB. When the SNR of the mid-frequency sub-band is 22 dB, a dynamic gain of 1.5 dB can be added to the static equalization gain; when the SNR of the high-frequency sub-band is 8 dB, the dynamic gain can be reduced by 1 dB from the static equalization gain, or the additional dynamic gain for that frequency band can be set to 0. The number of sub-bands, SNR threshold, and gain values ​​mentioned above are only for illustrating specific implementation methods and do not constitute a limitation.

[0187] In some embodiments, the frequency domain equalization module 150 also acquires the quantile spectrum deviation of the input audio signal and calculates the dynamic gain by combining the static equalization parameters, subband signal-to-noise ratio, and quantile spectrum deviation.

[0188] Quantile spectral deviation is used to characterize the difference between the quantile spectral value of the current input audio signal in the corresponding frequency band and the target quantile spectral value. The current quantile spectral value can be obtained by sorting the energy values ​​of a set number of historical audio frames in the corresponding frequency band to obtain the energy value of a specified quantile. The target quantile spectral value can be predetermined based on target sound samples or device calibration samples.

[0189] For example, the energy values ​​of 20 consecutive audio frames in a subband can be acquired, and the 10th energy value after sorting can be used as the 50th quantile spectrum value. Subtracting this 50th quantile spectrum value from the pre-stored target 50th quantile spectrum value yields the quantile spectrum deviation of the subband. When the subband signal-to-noise ratio (SNR) is higher than 15 dB and the quantile spectrum deviation indicates that the current subband energy is lower than the target subband energy, the dynamic gain of the subband is increased; when the subband SNR is lower than 15 dB, the dynamic gain of the subband is decreased to limit the noise component from increasing synchronously with the equalization gain.

[0190] The frequency domain equalization module 150 can implement gain compensation using a dual second-order filter, a finite impulse response filter, or a frequency domain gain multiplier. When using a dual second-order filter, the gain parameters of the dual second-order filter can be updated according to the dynamic gain, while maintaining the center frequency and quality factor at the calibrated values ​​in the static equalization parameters. To limit filter parameter jumps, the dynamic gain of adjacent audio frames can be interpolated or low-pass smoothed.

[0191] In one specific implementation, after calculating the dynamic gain, the frequency domain equalization module 150 uses a filter to perform gain compensation on the clipped audio signal based on the dynamic gain. If frequency domain processing is used, the... Frame, First The frequency domain signal at each frequency point is denoted as The equalized frequency domain signal can be represented as:

[0192] in, This represents the frequency domain signal after dynamic equalization processing.

[0193] If a dual second-order filter is used to achieve dynamic equalization, then the dynamic gain can be used as a reference. or Update the filter gain parameters for the corresponding frequency band and process the audio signal according to the double second-order filter difference equation:

[0194] in, Indicates the index of the time-domain sampling point; , , , as well as This represents the coefficients of a second-order dual filter, determined based on the filter type, center frequency, quality factor, and dynamic gain. This represents the output signal of the filter.

[0195] Next, we will provide an exemplary explanation of quantile spectral deviation and its participation in dynamic gain calculation.

[0196] In some implementations, the dynamic gain is calculated based on static equalization parameters and subband signal-to-noise ratio, and further includes: obtaining the quantile spectrum deviation of the input audio signal, where the quantile spectrum deviation is the difference between the target quantile spectrum and the current input quantile spectrum; and calculating the dynamic gain by combining the static equalization parameters, subband signal-to-noise ratio, and quantile spectrum deviation.

[0197] Specifically, in equation (32), the difference between the target quantile spectrum and the input quantile spectrum constitutes the quantile spectrum bias. This quantile spectrum bias can be expressed as:

[0198] in, Indicates the first Frame, First Quantile spectral deviation at a frequency point or sub-band; Represents the target quantile spectrum; This indicates the current input quantile spectrum.

[0199] Substituting equation (38) into equation (32), and combining the static equalization parameters, subband signal-to-noise ratio, and quantile spectrum deviation to calculate the dynamic equalization gain, the relationship can be expressed as follows:

[0200] in, This represents the calculated dynamic gain; Represents the static equilibrium parameters; This indicates the weight corresponding to the subband signal-to-noise ratio; Indicates quantile spectral deviation; This represents the equilibrium adjustment coefficient.

[0201] The input quantile spectrum can be calculated based on the current audio frame and multiple previous historical audio frames. As an optional implementation, continuous... The audio frame in the 1st The set of spectral amplitude values ​​for each frequency point or sub-band:

[0202] Then according to the preset quantiles Calculate the input quantile spectrum:

[0203] in, This indicates the number of audio frames used to calculate the current quantile spectrum; Indicates the first Frame, First The spectral amplitude or spectral power of a frequency point or sub-band; Let quantiles be quantiles, and satisfy the following conditions:

[0204] For example, It can be 10, 20, 50, or 100. It can be 0.2, 0.5, or 0.8.

[0205] For example, take the sub-band spectral amplitude of 20 consecutive audio frames, and... If set to 0.5, then This indicates that the 20 audio frames are in the [number]th [period]. Median spectral value of each sub-band. Target quantile spectrum. It can be obtained by performing offline statistics on clean speech samples or target audio samples and stored in the non-volatile memory of the transmitting end 100.

[0206] when When the input audio signal has a lower quantile spectrum in the corresponding frequency band than the target quantile spectrum, equation (39) increases the dynamic gain of the frequency band when the subband signal-to-noise ratio is high. When the input audio signal has a higher quantile spectrum in the corresponding frequency band than the target quantile spectrum, equation (39) reduces the dynamic gain of that frequency band. When the subband signal-to-noise ratio is low, The corresponding weights are relatively small, which limits the impact of quantile spectrum deviation on dynamic gain.

[0207] Therefore, by combining static equalization parameters, subband signal-to-noise ratio, and quantile spectrum deviation to calculate dynamic gain, dynamic compensation can be performed based on the current audio signal spectrum state while compensating for fixed hardware frequency response, thereby reducing noise amplification in low signal-to-noise ratio frequency bands.

[0208] Based on the above description Figure 4 This document illustrates a detailed flowchart of dynamic time-frequency equalization processing based on signal-to-noise ratio and quantile spectrum deviation, provided in some embodiments of this application. To make the dynamic time-frequency equalization compensation process based on sub-band signal-to-noise ratio and quantile spectrum deviation in some embodiments of this application more intuitive, the following is a detailed explanation... Figure 4 The specific processing flow of the audio signal by the frequency domain equalization module at the transmitting end is explained in detail. For example... Figure 4 As shown, the dynamic time-frequency equalization processing flow specifically includes the following steps: S401, Input the time-domain PCM audio signal after amplitude limiting processing.

[0209] The frequency domain equalization module receives the anti-clipping time-domain discrete PCM audio signal output by the pre-amplifier (DRC) module, which has eliminated transient overload.

[0210] S402, audio framing, performs time-frequency transformation, and divides into M sub-bands.

[0211] The frequency domain equalization module performs overlapping framing and windowing processing on the input time-domain audio signal, then performs time-frequency transformation (such as Fast Fourier Transform or Short-Time Fourier Transform) to transform the time-domain signal into a frequency-domain signal, and divides the frequency-domain signal into M time-frequency sub-bands according to preset frequency band division rules. It is a positive integer greater than 1.

[0212] S403A, Estimate the noise power of each sub-band and calculate the sub-band signal-to-noise ratio. .

[0213] In the first branch, the frequency domain equalization module uses a noise estimation algorithm to estimate the noise power of each sub-band in real time, and calculates the corresponding linear sub-band signal-to-noise ratio (SNR) by combining the total power of each sub-band. This SNR is then mapped to a sub-band SNR weight between 0 and 1. .

[0214] Step S403B: Sliding statistical subband energy, calculate input quantile spectrum .

[0215] In the second branch, which runs parallel to the first branch, the frequency domain equalization module performs sliding time window statistics on the energy of each sub-band in the time or frequency domain, calculates the sliding energy distribution, and thus obtains the current input quantile spectrum that reflects the spectral characteristics of the current frame length. .

[0216] S404. Calculate the deviation between the target quantile spectrum and the current input quantile spectrum.

[0217] The frequency domain equalization module reads the pre-stored ideal target quantile spectrum. The target quantile spectrum is then compared with the current input quantile spectrum obtained in step S403B. Perform subtraction to calculate the quantile spectral deviation of each sub-band. .

[0218] S405. Calculate the dynamic gain of each subband by combining static equalization parameters, signal-to-noise ratio weights, and spectral bias.

[0219] The frequency domain equalization module is based on static equalization parameters that characterize the inherent frequency response compensation of the hardware. Combined with the signal-to-noise ratio weights obtained in step S403A and the quantile spectrum deviation obtained in step S404 Perform weighted simultaneous calculations to obtain the dynamic gain of each sub-band in the current frame. .

[0220] S406. Multiply the spectrum of each sub-band by the corresponding dynamic equalization gain.

[0221] The frequency domain equalization module uses filters (such as frequency domain multipliers) to multiply the frequency domain signals of each time-frequency sub-band divided in step S402 by the dynamic gain of the corresponding sub-band calculated in step S405. (or dynamic gain after limiting) to achieve gain compensation in the frequency domain.

[0222] S407: Performs inverse frequency domain transform, reconstructs and outputs time-domain equalized audio.

[0223] The frequency domain equalization module performs inverse frequency domain transformation (such as inverse fast Fourier transform or inverse short-time Fourier transform) on each sub-band frequency domain signal after equalization compensation, and reconstructs a continuous denoised and equalized time domain audio signal through the overlapping addition method, and outputs it to the subsequent encoding module.

[0224] Thus, the dynamic time-frequency equalization processing flow provided in some embodiments of this application organically combines static device frequency response correction with dynamic environmental adaptive adjustment, enabling adaptive compensation based on the real-time input spectrum. Secondly, it introduces sub-band signal-to-noise ratio for dynamic gain modulation, providing full compensation in high-SNR frequency bands and automatically suppressing gain in low-SNR frequency bands, effectively avoiding the problem of traditional equalizers indiscriminately amplifying environmental noise. Furthermore, based on quantile spectrum deviation-driven gain calculation, it can dynamically track and approximate the target speech envelope, ensuring consistent and clear listening experience for output audio under different sound field environments. Finally, it employs dual-branch time-frequency transformation and inverse transformation reconstruction, guaranteeing fine-grained frequency domain control and smooth, seamless time domain reconstruction, significantly improving the dynamic performance and subjective fidelity of the audio.

[0225] Next, we will provide an exemplary description of time synchronization and time slot calibration between the transmitting and receiving ends.

[0226] In some implementations, when the system begins wireless audio transmission, the method further includes: in the initial synchronization phase, the transmitter sends a synchronization data packet containing a timestamp to the receiver in a specified synchronization time slot; receives an acknowledgment packet from the receiver within a preset reception time slot, and aligns the time bases of the transmitter and receiver based on the acknowledgment packet; and after the time base alignment is completed, enters a steady-state phase of transmitting encoded audio data in a one-way broadcast manner.

[0227] For example, during the initial synchronization phase, the sending end sends a synchronization data packet containing a timestamp to the receiving end in a specified synchronization time slot. After receiving the synchronization data packet, the receiving end sends an acknowledgment packet back to the sending end in a preset feedback time slot. The sending end aligns its time base with the receiving end based on the acknowledgment packet.

[0228] Specifically, in this application embodiment, the time offset calculation for the above-mentioned time synchronization process is performed using the following optional implementation method. Optionally, during the initial synchronization phase, the sending end records the first transmission timestamp when sending the synchronization data packet. The receiving end records the first received timestamp when it receives the synchronization data packet. The receiving end records a second sending timestamp when sending the acknowledgment packet. When the sending end 100 receives the acknowledgment packet, it records the second receiving timestamp. .

[0229] Assuming that the uplink and downlink wireless propagation delays are approximately symmetrical, the transmitting end uses a time stamp containing the aforementioned timestamps. , , and The acknowledgment packet is used to calculate the time offset between the receiver and the sender. The time offset between the sender and receiver can be expressed as:

[0230] Round-trip transmission delay can be expressed as:

[0231] One-way propagation delay can be approximated as:

[0232] in, This indicates the time offset of the receiver's time base relative to the transmitter's time base; Indicates the round-trip transmission delay of synchronization data packets and acknowledgment response packets; This represents the estimated one-way transmission delay. The units for the above time quantities can be microseconds, nanoseconds, hardware timer counts, or other standardized time units.

[0233] The sending or receiving end can adjust the time offset. Calibrate the local time slot counter. Assume the duration of a single wireless physical layer time slot is... The corresponding integer time slot offset can be expressed as:

[0234] The remaining error after time offset correction can be expressed as:

[0235] in, Indicates the number of integer time slots to be corrected; This indicates rounding to the nearest integer. This indicates the remaining time deviation after integer time slot correction is completed.

[0236] Receiver 200 can first follow Adjust the local time slot counter, and then compensate by adjusting the local timer load value or the start time of the next audio frame. .

[0237] For example, when a 2.5ms audio frame is divided into 8 wireless physical layer time slots, the duration of a single wireless physical layer time slot is:

[0238] In some implementations, after time base alignment is completed, the transmitting and receiving ends enter a steady-state phase of transmitting encoded audio data in a one-way broadcast manner. In this steady-state phase, the transmitting end transmits encoded audio data within the bound target transmission time slot and no longer waits for any acknowledgment signal from the receiving end for the encoded audio data. By using acknowledgment packets only to complete time base alignment during the initial synchronization phase and canceling the packet-by-packet acknowledgment mechanism after entering the steady-state phase, the hard-binding time slot scheduling relationship established between the transmitting and receiving ends can be avoided due to acknowledgment waiting and data retransmission, thus ensuring constant transmission delay.

[0239] Furthermore, some embodiments of this application provide methods that include: in the initial synchronization phase, if the transmitting end does not receive an acknowledgment packet within a preset reception time slot, a synchronization data packet retransmission process is triggered; and if the number of synchronization data packet retransmissions reaches a set upper limit and no acknowledgment packet is still received, the wireless transmitter of the transmitting end is turned off.

[0240] Specifically, let the current number of synchronization data packet retransmissions be . The set upper limit value is When the following conditions are met:

[0241] If the transmitting end does not receive an acknowledgment packet from the receiving end within the preset reception time slot, it indicates that the current wireless channel environment is extremely poor or the receiving end is not working properly. The transmitting end immediately shuts down its wireless transmitter (or puts the wireless transmitter into a standby power-saving state) to terminate meaningless synchronization attempts and reduce power consumption. For example, the set upper limit value... It can be 3, 5 or 10, there is no restriction here.

[0242] By limiting the number of times synchronization data packets can be retransmitted, and shutting down the wireless transmission module 180 or putting it into standby mode after reaching the upper limit, continuous wireless transmission when synchronization fails can be limited, reducing invalid wireless usage and device power consumption.

[0243] Figure 5This is a schematic diagram illustrating the time synchronization and state transition between the sending and receiving ends in some embodiments of this application. To make the logical transitions between the sending and receiving ends in the initial synchronization phase and the steady-state phase in some embodiments of this application clearer, the following is a detailed explanation... Figure 5 The time synchronization and state transition process between the two ends is explained in detail. For example... Figure 5 As shown, the method includes steps T1 to T7 on the sending end (Tx) side and steps R1 to R4 on the receiving end (Rx) side. The specific interaction flow is as follows: During the initial synchronization phase, the sending end executes step T1, generating a high-precision first transmission timestamp. The synchronization data packet is sent to the receiving end via a wireless channel. The receiving end executes step R1, receiving the synchronization data packet and recording the first reception timestamp. Subsequently, the receiving end executes step R2, calculates the clock offset based on the recorded timestamp, and generates an acknowledgment packet (ACK packet) to reply to the sending end.

[0244] The transmitting end executes step T2 to determine whether the ACK response packet has been received within the preset receiving time slot. If received (step T2, yes), then step T6 is executed to lock the transmitting end's time slot reference and align the time slot beat; the receiving end correspondingly executes step R3 to lock the receiving end's time slot reference and synchronize the time slot beat. After the time reference alignment is completed, the transmitting end enters the steady state stage and executes step T7, continuously sending encoded audio data packets in a one-way broadcast manner according to a fixed time slot beat (no ACK feedback required); the receiving end executes step R4, listening within the bound receiving window, receiving the encoded audio data packets, and performing subsequent decoding.

[0245] If the sending end does not receive the ACK response packet within the preset reception time slot (step T2, No), it enters the retransmission process, executing step T3 to increment the retransmission count of the synchronization data packet by 1. Then, it executes step T4 to determine whether the current retransmission count has reached the set upper limit value. If the upper limit value has not been reached (step T4, No), it returns to step T1 to retransmit the synchronization data packet; if the upper limit value has been reached (step T4, Yes), it executes step T5 to immediately shut down the sending end's wireless transmitter and disconnect the connection.

[0246] Thus, this technical solution, by introducing handshake synchronization, locking, and clear state machine transitions without ACK broadcasts, enables the system to achieve high-precision time alignment during the connection establishment phase, and completely eliminates the non-deterministic latency caused by retransmission waiting during the steady-state phase, ensuring constant latency for wireless audio transmission. Furthermore, in the event of connection establishment failure, a retry limit determination and a transmitter active shutdown mechanism are introduced to avoid meaningless RF transmissions by the transmitter in poor channel conditions or when the receiver is abnormal, significantly reducing the system's ineffective power consumption and extending device battery life.

[0247] Figure 6 This document provides a schematic diagram illustrating the timing relationship and retransmission protection of the interaction between the transmitting and receiving ends in some embodiments of this application. To more intuitively demonstrate the air interface interaction timing during the synchronization, broadcast, and retransmission timeout processes described above, the following is combined with... Figure 6 The normal link establishment and transmission timing and the timeout retransmission and disconnection timing are explained in detail. For example... Figure 6 As shown in the figure, the left axis represents the internal state and transmission actions of the transmitting end (Tx), the right axis represents the receiving and processing state of the receiving end (Rx), and the horizontal arrow in the middle represents the data flow direction of the air interface.

[0248] like Figure 6 As shown in the upper part, under normal interaction timing, the sending end first sends 0-1 packets (i.e., synchronization data packets) for ACK verification during the initial synchronization phase and enters the waiting ACK state. After receiving this data packet (the "ACK packet of Tx" at Rx), the receiving end replies with an ACK data packet to the sending end in the corresponding feedback time slot. After successfully receiving the reply packet, the sending end confirms that the time base alignment is complete and enters the steady state phase, and then continuously sends 3-6 data packets (i.e., encoded audio data packets). During this phase, the sending end does not need to wait for any ACK, and the receiving end directly decodes and outputs audio after receiving the broadcast data.

[0249] like Figure 6 As shown in the lower part, under the timeout retransmission disconnection sequence, the transmitter sends a 0-1 packet during the initial synchronization phase and enters the ACK waiting state. If the receiver fails to receive this packet due to channel interference or other reasons ("Not Received" is displayed at Rx), the transmitter will not receive feedback within the preset reception time slot. The transmitter determines that a timeout has occurred and triggers the retransmission process, sending the 0-1 packet again. This process repeats. If the number of retransmissions reaches the set upper limit, the transmitter terminates transmission, performs a disconnection operation, and shuts down the wireless transmitter.

[0250] Next, an exemplary description will be given of initial synchronization and steady-state unidirectional transmission.

[0251] In some embodiments, the wireless audio transmission process includes an initial synchronization phase and a steady-state phase. In the initial synchronization phase, the transmitting and receiving ends perform acknowledgment and response interactions to align the time base; after the time base alignment is completed, the process enters the steady-state phase, in which encoded audio data is transmitted in a one-way broadcast manner.

[0252] Specifically, during the initial synchronization phase, the sending end sends a synchronization data packet containing a timestamp to the receiving end in a specified synchronization time slot. The synchronization data packet may also include at least one of the following: sender identifier, frame sequence number, time slot configuration identifier, audio frame length, and checksum information.

[0253] After receiving the synchronization data packet, the receiving end determines the time offset information based on the timestamp in the synchronization data packet and the local reception time, and sends an acknowledgment packet to the sending end within a preset feedback time slot. The acknowledgment packet may contain at least one of the following: the sequence number of the synchronization data packet, the reception timestamp, the time offset information, and the reception status information.

[0254] The transmitting end receives the acknowledgment packet from the receiving end within a preset reception time slot, and aligns the time bases of the transmitting and receiving ends based on the acknowledgment packet. Time base alignment can be achieved by adjusting the local time slot counter of the transmitting or receiving end, adjusting the start time of the next audio frame, or setting the clock offset compensation amount.

[0255] After completing time base alignment, the transmitter enters a steady-state phase. Within the target transmission time slot, the transmitter uses the wireless transmission module to send the encoded audio data to the receiver in a one-way broadcast manner, without waiting for the receiver to send an acknowledgment signal after sending the encoded audio data.

[0256] It's worth noting that "not waiting for acknowledgment from the receiver" means that during steady-state audio data transmission, the transmitter does not require the receiver to return an acknowledgment signal for the current encoded audio data packet as a prerequisite for sending the next encoded audio data packet. The transmitter continuously sends the encoded audio data of the corresponding audio frame according to the pre-configured target transmission time slot.

[0257] This steady-state unidirectional transmission method can eliminate the waiting overhead caused by packet-by-packet acknowledgment and avoid the retransmission of data packets occupying subsequent wireless transmission time slots, thus helping to maintain the pre-established fixed time slot beat and constant transmission delay.

[0258] During the initial synchronization phase, if the sending end does not receive an acknowledgment packet within the preset receive time slot, it triggers a synchronization data packet retransmission process. The sending end can count the number of synchronization data packet retransmissions and reopen the preset receive time slot after each retransmission.

[0259] If the number of retransmissions of synchronization data packets reaches a set upper limit and no acknowledgment packet is received, the transmitting end shuts down the wireless transmission module. The upper limit can be 3, 5, 10 times, or other positive integers set according to the wireless channel environment. For example, if the upper limit is 10 times, and the transmitting end sends synchronization data packets 10 times consecutively without receiving an acknowledgment packet within a preset reception time slot, it stops sending synchronization data packets and controls the wireless transmission module to enter standby or low-power mode.

[0260] In other embodiments, after the transmitting end shuts down the wireless transmission module, it can restart the wireless transmission module and re-execute the initial synchronization process when the preset waiting time expires or a reconnection command is received.

[0261] Next, we will provide an exemplary description of receiver decoding, packet loss concealment, historical audio interpolation, and attenuation compensation.

[0262] In some implementations, the receiving end performs the following operations after receiving the encoded audio data: starts the decoder in the bound physical decoding time slot to decode the received encoded audio data; and if the current encoded audio data packet is detected to be lost, prevents the sending end from sending a retransmission request and calls the packet loss hiding algorithm to interpolate and generate compensating audio data based on historical audio data for output.

[0263] Specifically, the receiving end receives encoded audio data unidirectionally broadcast by the sending end within a bound wireless physical receive time slot. Upon receiving the encoded audio data, the receiving end starts the decoder in the bound physical decoding time slot to decode the received encoded audio data. During this process, the receiving end can determine (i.e. detect) whether the current encoded audio data packet is lost based on the frame sequence number of the encoded audio data packet, the cyclic redundancy check result, or whether valid data has been received within the predetermined receive time slot. If the loss of the current encoded audio data packet is detected, in order to maintain a fixed beat and constant delay in end-to-end wireless audio transmission, the receiving end prevents sending any retransmission requests to the sending end (i.e., does not execute retransmission requests), and instead immediately and proactively calls the packet loss concealment algorithm to interpolate and generate compensated audio data based on historical audio data for output.

[0264] For example, the receiving end receives encoded audio data broadcast unidirectionally by the sending end within the bound wireless physical receiving time slot and writes the received encoded audio data into the decoding buffer. The timing synchronization module starts the decoding module in the bound physical decoding time slot according to the time base determined in the initial synchronization phase to decode the received encoded audio data.

[0265] For example, the receiver can receive the k-th encoded audio data packet in the wireless physical reception time slot of the current audio frame, and start the decoding module in the physical decoding time slot of the next audio frame to decode the k-th encoded audio data packet. The decoded audio data is then transferred to the digital-to-analog converter module via the direct memory access module and output by the audio output module.

[0266] The receiving end can detect whether the current encoded audio data packet is lost based on the frame sequence number of the encoded audio data packet, the cyclic redundancy check result, the reception status indicator, or whether valid data has been received within the predetermined wireless physical reception time slot.

[0267] When the current encoded audio data packet is detected to be lost, the receiving end stops generating retransmission requests for that encoded audio data packet and calls the packet loss concealment algorithm to generate compensating audio data based on historical audio data interpolation for output. Historical audio data may include one or more decoded audio frames before the current lost frame, and may also include audio frames received after the lost frame.

[0268] Packet loss concealment algorithms can employ waveform repetition, amplitude attenuation, linear prediction, spectral interpolation, or built-in packet loss concealment algorithms of the codec. For example, in the event of a single lost coded audio data packet, the audio processing unit can copy the previously decoded audio frame and apply an attenuation envelope from 1 to 0.8 to the amplitude of the copied audio frame to obtain compensating audio data. In the event of two consecutive lost coded audio data packets, an attenuation envelope from 0.8 to 0.5 can be applied to the second compensating audio frame. After a valid coded audio data packet is received again, a crossfading can be performed between the compensating audio data and the newly decoded audio data.

[0269] In one specific implementation, the packet loss hiding algorithm is invoked to generate compensation audio data based on historical audio data interpolation, including: generating compensation frames by weighted interpolation of multiple frames of historical audio data, or generating compensation frames by applying a frame-by-frame attenuation envelope to the previously decoded audio data.

[0270] It should be noted that the following interpolation and attenuation formulas are optional implementation methods, used to illustrate the packet loss compensation logic that can be implemented in this implementation method.

[0271] As an optional implementation, let the sequence number of the currently lost audio frame be... , No. The and the first The historical decoded audio frames (i.e., multiple frames of historical audio data) are respectively and The generated compensated audio data (i.e., compensated frames) can then be represented as:

[0272] in, Represented as the first Compensated audio data generated from each lost audio frame; Indicates the index of the intra-frame sampling point; Represents the historical audio interpolation weights, and satisfies:

[0273] For example, It can be 0.6, 0.7, 0.8, or dynamically determined based on the correlation between historical audio frames.

[0274] In another alternative implementation, the method of generating a compensation frame by applying a frame-by-frame attenuation envelope to the previously decoded audio data can be to copy the previous historically decoded audio frame (i.e., the previous historically decoded audio frame) and apply an attenuation envelope to the copied audio data. Let the number of consecutively lost audio frames be... Then the first The attenuation coefficient corresponding to a series of lost frames can be expressed as:

[0275] in, Let represent the single-frame attenuation coefficient, and satisfy:

[0276] The corresponding compensated audio data can be represented as:

[0277] For example, It can be set to 0.8. When the first audio frame is lost, the amplitude of the compensation audio frame can be 0.8 times the amplitude of the previous decoded audio frame; when the second consecutive audio frame is lost, the amplitude of the compensation audio frame can be 0.64 times the amplitude of the previous decoded audio frame. By applying a frame-by-frame attenuation envelope, the compensation audio can fade out smoothly during long periods of packet loss, avoiding abnormal intermittent noise.

[0278] In another alternative implementation, a gradual attenuation envelope can be applied within a frame that has lost an audio frame. Let an audio frame include... There are 1 sampling point, and the frame start gain is . The frame tail gain is Then the intra-frame attenuation envelope can be expressed as:

[0279] The corresponding compensated audio data can be represented as:

[0280] in, For example, you can set , This allows the compensated audio data to be smoothly attenuated from the original amplitude of the previous audio frame to 0.8 times its original amplitude within an audio frame.

[0281] Furthermore, in this example, upon receiving valid encoded audio data again, a cross-fade transition is performed between the compensated audio data and the newly decoded audio data.

[0282] Specifically, when the receiving end receives valid encoded audio data again after packet loss, to avoid waveform jumps caused by the discontinuity between the compensation waveform and the newly recovered decoded waveform due to packet loss, a cross-gradient can be performed between the compensation audio data and the newly decoded audio data. Let the cross-gradient interval include... With a sampling point, the cross-gradient output can be expressed as:

[0283] in:

[0284] This refers to the audio signal obtained by decoding the newly received valid encoded audio data (i.e., the newly decoded audio data). This represents the compensated audio data generated during packet loss. By using crossfading, the waveform transition between the compensated audio data and the newly decoded audio data can be reduced, eliminating noise.

[0285] It should be understood that in embodiments employing low-complexity communication codecs or low-complexity communication-enhanced codecs, the packet loss hiding interface provided by the decoder can be directly invoked, and the packet loss identifier can be input into the decoder, which will then generate compensating audio data based on the historical decoding status.

[0286] When the receiving end does not receive the currently encoded audio data packet, it still performs packet loss concealment processing in the physical decoding time slot corresponding to the current audio frame and outputs compensation audio data in the corresponding output time slot. The decoding timing of subsequent audio frames is not changed due to waiting for the retransmission of the currently encoded audio data packet.

[0287] Therefore, the receiver can maintain the continuity of audio output when occasional packet loss occurs, while maintaining a fixed decoding rhythm for subsequent audio frames, avoiding additional delays and jitter caused by retransmission waiting.

[0288] Figure 7 This document illustrates a detailed flowchart of the receiver packet loss concealment and audio decoding output provided in some embodiments of this application. To make the decoding and packet loss concealment processing flow of the receiver after receiving audio data in some embodiments of this application more intuitive, the following is a detailed explanation... Figure 7 The specific processing steps at the receiving end are explained in detail. For example... Figure 7 As shown, the process specifically includes the following steps: S701 listens for and receives wireless data within the bound physical receive time slot.

[0289] Within the pre-agreed physical reception time slot with the transmitter, the receiver enables the wireless transceiver's reception state, listens for and receives wireless audio data packets broadcast by the transmitter through the wireless channel.

[0290] S702. Parse the data packet frame header, verify packet integrity, and identify the frame sequence number.

[0291] The receiving end performs physical layer unpacking on the received wireless data packets, parses the data packet header to obtain transmission control information, and verifies the integrity of the data packets through verification algorithms such as cyclic redundancy check, and identifies the audio frame sequence number carried by the data packets.

[0292] S703, Detect whether the current frame is lost.

[0293] The receiving end determines whether the current encoded audio data packet is lost based on whether the data packet verification in step S702 passes or whether the frame sequence number is interrupted. If it is not lost (step S703, No), then step S704 is executed; if it is lost (step S703, Yes), then step S705 is executed.

[0294] S704. Start the LC3+ decoder for normal decoding in the bound decoding time slot.

[0295] If it is determined that the current frame has not been lost, the receiving end triggers and starts the LC3+ decoder within the bound physical decoding time slot to decompress and decode the received encoded audio data, recover the current normal audio frame, and then executes step S707.

[0296] S705, The physical layer prohibits the generation of retransmission requests.

[0297] If the loss of the current encoded audio data packet is detected, the communication protocol stack of the receiving end controls the physical layer to block and prohibit any retransmission request from being sent back to the sending end, so as to ensure that the air interface channel only performs one-way broadcasting during the steady state phase.

[0298] S706 triggers the PLC algorithm to perform interpolation reconstruction based on historical correct frames.

[0299] While prohibiting retransmission, the receiving end triggers the packet loss concealment algorithm, calls the audio data that has been successfully decoded in multiple frames of history for weighted interpolation, or applies a frame-by-frame attenuation envelope to the audio data that has been decoded in the previous frame to generate compensation audio data (i.e. compensation frame) to replace the lost frame, and then executes step S707.

[0300] S707. Perform cross-gradient smoothing processing on the current normal frame / compensated frame and adjacent frames.

[0301] The receiving end performs a cross-gradient transition process between the current normal frame decoded and output in step S704, or the compensated frame generated in step S706, and adjacent audio frames that are close in time. The phase or amplitude discontinuity at the data splicing boundary between the two frames is eliminated by the gradient weighting.

[0302] S708, via DMA, transfers data to a digital-to-analog converter and outputs a continuous audio stream.

[0303] The receiving end transfers the smoothed audio data in real time through the Direct Memory Access (DMA) channel to the first-in first-out (FIFO) buffer of the Digital-to-Analog Converter (DAC), converts it into an analog audio signal for output, and thus reconstructs a continuous and smooth physical audio stream.

[0304] Thus, this technical solution fundamentally eliminates the backhaul channel overhead and nondeterministic latency jitter caused by wireless retransmissions by prohibiting retransmission requests at the physical layer, achieving constant ultra-low latency wireless audio transmission. Secondly, by utilizing the Packet Loss Concealment (PLC) algorithm combined with adjacent frame crossover fading processing, it can adaptively reconstruct based on historical waveforms when packet loss occurs, effectively eliminating click noise introduced by waveform abrupt changes or phase breaks at frame boundaries, ensuring high-fidelity auditory continuity. Thirdly, the design of using bound hardware decoding time slots in conjunction with DMA to directly transfer data to the DAC achieves a high degree of synergy between the software algorithm and the hardware transmission bus, ensuring real-time audio playback and robustness against packet loss with extremely low system computing power overhead.

[0305] Next, we will provide an exemplary description of direct memory access transport and end-to-end time-slot pipeline.

[0306] After completing dynamic limiting and optional dynamic time-frequency equalization processing, the transmitting end transfers the processed audio signal to the encoding buffer via a direct memory access channel. The source address of the direct memory access channel can be the output buffer of the limiting module or the frequency domain equalization module, and the destination address can be the input buffer of the encoding module 170.

[0307] In one specific implementation, the transmitting end performs high-precision oversampling of the acquired external analog audio signal using an analog-to-digital converter (ADC) to generate digitized pulse-code modulation (PCM) audio data. Subsequently, the transmitting end uses a direct memory access (DMA) channel to automatically transfer the PCM audio data to the system's data buffer. During the transfer process, in order to significantly reduce processor interrupt overhead and avoid context switching caused by sample-by-sample interrupts, the system only triggers a hardware interrupt (i.e., a processor interrupt) to the processor when the accumulated PCM audio data in the data buffer reaches the data size of a complete audio frame. This wakes up the processor, and the interrupt service routine performs unified audio processing on the complete audio frame (e.g., the aforementioned dynamic clipping and adaptive frequency domain equalization).

[0308] In this example, let an audio frame include Each sampling point occupies [number] sampling points. byte, number of channels The amount of data moved per frame in the direct memory access channel. It can be set to:

[0309] in, The unit is bytes. Taking a 48kHz sampling rate, 2.5ms audio frame, dual channels (C=2), and 2 bytes per sample point (B=2, corresponding to 16-bit precision) as an example, the number of sample points per channel is:

[0310] The corresponding data transfer volume per frame is:

[0311] The first "2" indicates the number of dual channels, and the second "2" indicates that each sampling point occupies 2 bytes.

[0312] When using a per-sample-point interrupt, the interrupt frequency at a 48kHz sampling rate is approximately 48,000 times / second; when using a 2.5ms full-frame interrupt, the interrupt frequency is approximately: 1 / (2.5×10⁻⁶). 3 =400 times / second. Therefore, DMA whole-frame transfer can significantly reduce interrupt context switching and data copying operations. In high sampling rate multi-channel scenarios, CPU load can be reduced from about 60% to below 25%, allowing the CPU to be used for core algorithms such as audio content classification, adaptive dynamic range limiting (DRC), and high-efficiency coding.

[0313] In some implementations, a dual-buffered structure can be used in the direct memory access channel (DMI) to achieve parallelization of data transfer and computation. The data buffer specifically includes a first encoding buffer and a second encoding buffer. When the encoding module (such as the LC3+ encoding module) reads the current audio frame from the first encoding buffer, the DMI moves the next audio frame to the second encoding buffer. Before the next target encoding time slot arrives, the system swaps the read / write attributes of the first and second encoding buffers, thus ensuring seamless pipeline operation.

[0314] The timing control module outputs an encoding trigger signal in the bound target encoding time slot, causing the encoder to read the amplitude-limited audio signal from the encoding buffer and generate encoded audio data; the timing control module further outputs a transmission trigger signal in the bound target transmission time slot, causing the wireless transmitter to transmit the encoded audio data in the corresponding target transmission time slot.

[0315] Thus, in this embodiment, by performing audio data transfer through the direct memory access channel, the task occupation caused by the processor performing sample-by-sample read and write operations can be minimized. On this basis, by arranging direct memory access transfer, encoding and wireless transmission in pre-bound wireless physical layer time slots, the processing links of the entire link form a pipeline with a fixed rhythm, reducing timing fluctuations and latency jitter caused by processor task contention or random cache waiting.

[0316] In some implementations, the CPU classifies the audio content of the complete audio frame that has been transferred by DMA into three categories: speech, music, and silence.

[0317] In this example, the transmitter of the wireless communication system can adaptively adjust the frame structure and timing arrangement of the physical layer transmission and application layer processing according to the currently transmitted audio content. Specifically, the method further includes: acquiring the audio content features of the input audio signal; determining the audio content type corresponding to the input audio signal based on the audio content features; determining the target audio frame length from multiple candidate audio frame lengths based on the audio content type; and adjusting the timing correspondence between the boundaries of the audio frame, the start and stop times of the coding units, and the target transmission time slot according to the target audio frame length, and switching the current audio frame length to the target audio frame length at the audio frame switching boundary, and synchronously updating the time slot scheduling information of the transmitter and the receiver.

[0318] In one specific implementation, the step of acquiring the audio content features of the input audio signal and determining the audio content type corresponding to the input audio signal based on the audio content features can be implemented as follows: During system operation, the processor (CPU) at the transmitting end analyzes the complete audio frame processed by the Direct Memory Access (DMA) controller to extract the audio content features. These audio content features include, but are not limited to: fundamental frequency range, spectral energy distribution characteristics, zero-crossing rate (ZCR), and signal energy value. Based on the audio content features, the processor classifies the audio content type corresponding to the input audio signal into three categories: speech, music, and silence. The specific determination criteria are as follows: Voice type: When the fundamental frequency of the detected audio signal is in the range of 85Hz to 255Hz, the spectral energy is mainly concentrated in the mid-low frequency band, and the zero-crossing rate is lower than the preset zero-crossing rate threshold, the current audio content type is determined to be voice. Music type: When the detected audio signal has a wide spectral distribution range (covering approximately 20Hz to 20kHz), a dynamic range exceeding a preset dynamic threshold, and a zero-crossing rate higher than the zero-crossing rate threshold, the current audio content type is determined to be music; Silence type: When the Speech Activity Detection (VAD) algorithm determines that there is no speech activity and the signal energy of the current audio frame is lower than a preset energy threshold, the current audio content type is determined to be silence.

[0319] It should be understood that the fundamental frequency of speech signals is typically between 85Hz and 255Hz, with spectral energy mainly concentrated in the low and mid-frequency range, and a relatively low zero-crossing rate. Music signals have a wider spectral distribution, covering approximately 20Hz to 20kHz, with a larger dynamic range and a relatively higher zero-crossing rate. Silence signals can be detected using VAD (Voice Amplifier) ​​and determined by combining this with the fact that the audio frame energy is below a set energy threshold.

[0320] As a specific implementation, the step of determining the target audio frame length from multiple candidate audio frame lengths based on the audio content type is specifically implemented as follows: The system determines the target audio frame length from a set of preset candidate audio frame lengths based on the determined audio content type. Specifically, if the audio content type indicates that the input audio signal contains speech audio content, a first audio frame length (set to 2.5ms in this embodiment) is selected; if the audio content type indicates that the input audio signal contains music or silence audio content, a second audio frame length (set to 5ms in this embodiment) is selected, and the second audio frame length is longer than the first audio frame length.

[0321] The audio content classification results can be configured in relation to audio frame length and encoding bitrate as follows: Table 1. Audio Frame Length and Encoding Bitrate Corresponding Configuration Relationship

[0322] As shown in Table 1, when the current audio content type is speech, the transmitter selects the first audio frame length (2.5ms) and the encoding bitrate of 96kbps to minimize the air interface waiting latency of the audio frame; when the current audio content type is music, the transmitter selects the second audio frame length (5ms) and the encoding bitrate of 160kbps to improve the encoding compression efficiency and audio fidelity; when the current audio content type is silence, the transmitter uses the second audio frame length (5ms) and only sends the keep-alive packet with the lowest bitrate to maintain the wireless physical connection and time synchronization.

[0323] To avoid frequent frame length switching due to minute changes in audio content within a very short time, the transmitting end employs a switching decision lag (backoff) mechanism: The target audio frame length is only allowed to switch to the second audio frame length (5ms) when the system is currently in voice mode and detects that M consecutive frames of audio data all meet the aforementioned music characteristics, where M can be set to 10 (with a first audio frame length of 2.5ms, the confirmation time for 10 consecutive frames corresponds to approximately 25ms); the target audio frame length is only allowed to switch to the first audio frame length (2.5ms) when the system is currently in music mode and detects that N consecutive frames of audio data all meet the aforementioned voice characteristics, where N is set to 20 (with a second audio frame length of 5ms, the confirmation time for 20 consecutive frames corresponds to approximately 100ms), to avoid incorrect frame length switching caused by short pauses or short audio segments; when VAD detects silence and K consecutive frames meet the silence condition, the system switches to silence mode, where K can be set to 5.

[0324] It should be understood that, in order to avoid decoding failure, audio stuttering, or frame boundary misalignment at the receiving end due to inconsistent frame lengths between the two ends, this implementation requires that the frame length switching action must be performed at the audio frame switching boundary. Before reaching the switching boundary, the transmitting end first sends a synchronization packet to the receiving end through the air interface to notify the receiving end of the new frame length configuration information.

[0325] For example, the synchronization packet includes a specially designed "transition frame" that carries: audio data at the current frame length, a switching marker indicating an upcoming frame length change, new frame length configuration parameters, and the starting position information of the new frame length data in the subsequent transport stream. After successfully receiving and parsing the transition frame, the receiving end synchronously updates its decoder's decoding frame length parameters and decompression encoding parameters at predetermined audio frame switching boundaries, and adjusts its reception and decoding time slot scheduling information.

[0326] Thus, in this embodiment, by performing frame length switching at frame boundaries and utilizing transition frames to synchronize the configurations of the transmitting and receiving ends, audio stuttering or frame boundary misalignment caused by the receiving end decoding with an incorrect frame length can be avoided. Using shorter frame lengths in voice scenarios reduces latency, while using longer frame lengths in music scenarios improves encoding efficiency. In silent scenarios, sending only keep-alive packets reduces bandwidth and power consumption. In other words, this technical solution, through this handshake switching at frame boundaries, achieves synchronized reconstruction and timing alignment of audio frame boundaries, the start and stop times of encoding units (decoders), and the wireless receiving window in the time domain according to the new target audio frame length, ensuring a seamless audio transition experience.

[0327] In some implementations, using a constant-delay scheduling mechanism with frame slot binding, the end-to-end total delay can be calculated from the frame length and slot offset. Let the audio frame length be... The total number of time slots in a single frame is The fixed time slot offset for encoding and decoding at the transceiver end is Then the end-to-end delay can be expressed as:

[0328] in, Indicates end-to-end delay; Indicates the audio frame length; This represents the total number of wireless physical layer time slots divided within the frame length of an audio frame. This represents the pre-configured fixed time slot offset between the encoding / decoding and output at the transmitting and receiving ends.

[0329] With the audio frame length, number of time slots, and time slot offset remaining constant, the delay corresponding to equation (62) is a fixed value. Since the subsequent time slot scheduling relationship is not changed through packet-by-packet acknowledgment or data retransmission during steady-state transmission, the impact of wireless link quality fluctuations on end-to-end delay can be reduced, thus achieving low-jitter audio transmission.

[0330] In one specific embodiment, the audio frame length T The frame time is 2.5ms, and each frame is divided into 8 time slots. The encoding time slots are... S encode=1, decoding time slot is S If decode=5, then: Δ=5 1=4 D =2.5+4×(2.5 / 8)=2.5+1.25=3.75ms Therefore, in this embodiment, the end-to-end latency is 3.75ms. Since the audio frame boundaries, encoding time slots, and decoding time slots are all scheduled according to fixed time slots, the transmission latency does not depend on the packet-by-packet ACK waiting time, and the latency jitter can be controlled within 1ms.

[0331] In another embodiment, the audio frame length is 5ms, and each frame is divided into 14 time slots. When the encoding / decoding time slot offset is fixed at Δ=6: D =5 + 6 × (5 / 14) ≈ 7.14 ms Therefore, a 2.5ms frame length is suitable for latency-sensitive voice and live streaming scenarios, while a 5ms frame length is suitable for music scenarios with high requirements for bit rate and sound quality.

[0332] By using frame-slot binding encoding and decoding scheduling, the end-to-end latency can be expressed as a function of frame length and fixed slot offset, and the latency fluctuations caused by ACK waiting, task preemption and random retransmission can be reduced, thereby achieving constant low latency transmission.

[0333] Figure 8This document provides schematic diagrams illustrating end-to-end audio transmission timing and delay allocation based on hard-slot pipelines in some embodiments of this application. To make the principle of achieving ultra-low end-to-end constant delay based on hard-slot pipeline scheduling in some embodiments of this application more intuitive, the following is combined with... Figure 8 The timing of pipelined collaboration between the transmitter (Tx) and receiver (Rx) within a single audio frame cycle is described in detail.

[0334] like Figure 8 As shown, in a specific implementation, the audio frame length is... Taking a hard-slotted system as an example, timing arrangement is performed. The frame length of each complete audio frame (e.g., the first...) Frame, First The frame is divided into 8 physical time slots in the time domain, labeled as time slot S0 to time slot S7 (i.e., Slot 0 to Slot 7). The transmitting end (Tx) and the receiving end (Rx) lock the time slot clock through high-precision time synchronization and execute pipeline tasks in parallel in their respective bound time slots using hardware interrupt-driven methods.

[0335] Specifically, on the transmitting end (Tx) side, the processing flow of a single audio frame is divided into three pipeline stages: acquisition preprocessing, encoding, and wireless transmission. In the audio acquisition and preprocessing stage, the sending end is currently in the... Within the time slots Slot 0 to Slot 2 of the frame, analog audio signals are acquired in real time through an audio input interface (such as I2S) and converted into PCM audio sampling points, while the aforementioned adaptive limiting (DRC) processing is completed in the time domain. In the LC3+ encoding stage, the sending end is at the... Within slot 3 of the frame, the LC3+ encoder is triggered to compress and encode the entire frame of PCM data acquired and processed in slots 0-2, generating encoded audio data packets. During the wireless transmission phase, the transmitter is in the... During time slots 4 to 5 of the frame, the wireless transmission RF module is enabled to broadcast the generated encoded audio data packets to the outside via the air interface.

[0336] Accordingly, on the receiving end (Rx) side, for this first... The frame data processing flow is divided into three pipeline stages: wireless reception, decoding, and audio output. During the wireless reception phase, the receiver aligns with the transmitter's radio frequency in the [missing information - likely a specific phase or stage]. During frame time slots Slot 4 to Slot 5, the wireless receiving radio frequency module is enabled to receive encoded audio data packets transmitted over the air. During the LC3+ decoding stage, the receiver decodes the next frame (the... Within the time slots Slot 1 to Slot 2 of the frame, the LC3+ decoder is triggered to decode the received data packet (if the packet is lost, the PLC hiding and reconstruction is performed) to recover the time domain PCM audio data. During the audio output phase, the receiver is in the... Within Slot 3 of the frame, the DMA controller is activated to transfer the decoded and recovered PCM audio data to the digital-to-analog converter (DAC) for analog audio signal output.

[0337] like Figure 8 As shown at the bottom, through the above hard time slot pipeline design, from the transmitting end... The start time of frame audio acquisition (the first frame) (the start of frame Slot 0), to the receiving end will... The moment when the frame audio decoding output is completed (the first frame) The end-to-end processing latency of the entire link is strictly limited to 12 time slots (i.e., the end of frame Slot 3). The 8 time slots of the frame plus the first (The frame has 4 time slots).

[0338] Since the frame length of each audio frame (8 time slots) is Therefore, the average duration of each time slot is The calculated end-to-end latency for the 12 time slots is as follows: .

[0339] Thus, from the transmitter starting to acquire audio in Slot 0 of frame N, to the receiver completing the output of the audio for that frame in Slot 3 of frame N+1, the entire end-to-end transmission pipeline spans 12 time slots, achieving an ultra-low physical latency of 3.75ms at the end-to-end. In other words, this technical solution, through a low-overhead 2.5ms frame length design and fine-grained time slot pipeline partitioning, compactly arranges audio acquisition, encoding, wireless transmission and reception, decoding, and output within 12 time slots, locking the end-to-end latency at a constant 3.75ms, greatly improving the real-time performance of wireless audio transmission and achieving ultra-low and constant end-to-end physical latency.

[0340] Based on the above description, this application also provides an audio transmission method based on wireless physical layer time slot binding.

[0341] Figure 9 The diagram illustrates a flowchart of a wireless audio transmission method based on wireless physical layer time slot binding, provided in some embodiments of this application. For example... Figure 9As shown, this wireless audio transmission method is applied to a wireless communication system including a transmitter and a receiver. By hard-binding each stage of audio processing to the physical layer time slots, it ensures constant end-to-end latency and high-fidelity transmission. The specific process steps are as follows: S901, align audio frame boundaries, encoding / decoding start / stop, and wireless window timing according to the wireless physical layer time slots.

[0342] In this step, the system establishes a unified time coordinate system for the entire link, using the wireless physical layer time slot as the reference time scale. The start and end times of audio frames, the start / stop times of encoders and decoders, and the wireless transmission and reception windows are forcibly aligned in the time domain, providing a timing basis for subsequent pipelined processing.

[0343] S902: Acquire audio and convert it into a PCM digital signal.

[0344] In this step, the transmitting end acquires analog audio signals through a microphone and its front-end circuitry, and converts them into PCM (Pulse Code Modulation) digital signals through an ADC (Analog-to-Digital Converter).

[0345] S903. Obtain the difference between the current time slot and the target coding time slot, and perform dynamic clipping processing.

[0346] The sending end obtains the index of the first time slot in real time for the current audio data processing. With target coding slot index Based on the time slot difference The dynamic limiting threshold is calculated using the aforementioned equations (3) to (6), and the audio signal is pre-shaped to suppress possible overload and clipping distortion.

[0347] S904, Perform dynamic time-frequency equalization processing.

[0348] After the amplitude limiting process, the transmitting end further performs dynamic equalization based on the subband signal-to-noise ratio and quantile spectrum deviation to compensate for the hardware frequency response and optimize the vocal performance. The processed data is then passed to the corresponding processing buffer.

[0349] S905: Transfer data to the encoding buffer via the DMA channel.

[0350] The transmitting end uses a DMA (Direct Memory Access) controller to move the pre-processed PCM audio sampling data from the algorithm processing buffer to the input buffer dedicated to the encoding module, in order to reduce processor load and avoid jitter caused by random bus contention.

[0351] S906. Upon reaching the target encoding time slot, start the encoder to complete the encoding.

[0352] When the clock counter reaches the pre-bound target coding time slot At this time, the timing control module triggers the encoding trigger signal, causing the encoder (such as the LC3plus encoder) to read data from the encoding buffer and generate encoded audio data packets.

[0353] S907. Broadcast encoded data within the bound target transmission time slot.

[0354] After encoding is completed, the sending end transmits the encoded audio data to the receiving end in a one-way broadcast within the physical layer target transmission time slot bound to it.

[0355] S908. Receive encoded data within the bound receive time slot.

[0356] The receiver (such as Dongle) activates its radio frequency front-end within its bound wireless physical receive time slot according to a pre-established time synchronization reference, and receives audio data packets broadcast by the transmitter.

[0357] S909, Upon reaching the binding decoding time slot, the decoder is started for decoding.

[0358] S910, Detect whether data packets are lost.

[0359] The receiving end verifies the received data packets.

[0360] If the data is complete (step S910, No), proceed to step S911. If the data is incomplete (step S910, Yes), proceed to step S912.

[0361] S911 outputs the normally decoded PCM audio.

[0362] S912, without requesting retransmission, calls the packet loss hiding algorithm to generate compensation audio.

[0363] If a packet loss or error is detected (step S910, yes), to ensure absolute constant end-to-end delay, the receiver does not initiate any automatic retransmission request to the sender, but immediately calls the built-in packet loss concealment algorithm (PLC). Compensating audio samples are generated using the correlation of historical audio data to fill the current decoding time slot.

[0364] S913 outputs analog audio after DAC conversion.

[0365] Whether it is normally decoded PCM audio or compensated audio, it is sent to DAC (digital-to-analog converter) via DMA, converted into analog audio signal, and then output to the speaker for playback via Lineout interface or power amplifier.

[0366] Thus, through the above steps S901 to S913, this application deeply couples the software logic of audio processing with the hardware timing of the wireless physical layer, which not only ensures constant low latency in the whole-link transmission, but also achieves a balance between audio continuity and fidelity in complex wireless environments through the time-slot linkage limiting and non-retransmission packet loss concealment mechanism.

[0367] Next, we will provide an exemplary description of dynamic time slot allocation.

[0368] In some implementations, when the wireless communication system includes multiple transmitters, in order to avoid air interface time slot conflicts when multiple transmitters transmit data simultaneously and to improve the utilization of wireless spectrum resources, the method further includes: obtaining the audio data transmission requirements of each transmitter in at least one wireless physical layer time slot to be allocated; determining the target number of time slots corresponding to each transmitter based on the audio data transmission requirements of each transmitter; and dynamically allocating wireless transmission time slots to each transmitter based on the target number of time slots.

[0369] In this example, in a wireless communication system with multiple transmitters, each transmitter appends a next frame data volume prediction field to the header of the current audio data packet. The next frame data volume prediction value is calculated based on the VAD detection result and signal energy of the current frame.

[0370] in, Predict the amount of data for the next frame; The result is the voice activity detection result; 0 is used when there is silence, and 1 is used when there is voice or valid audio activity. The energy of the current audio frame signal; Maximum signal energy; This represents the maximum data volume. and It can be represented by the number of bits, the number of bytes, or the number of encoded data packets. and It can be represented by a normalized energy value.

[0371] The receiving end collects the predicted data volume of the next frame sent by each sending end, and forms: .in, For the number of senders, For the first The predicted data volume of the next frame from each transmitter.

[0372] The receiver performs dynamic time slot allocation based on each prediction value. First, the transmitters are arranged from largest to smallest according to the amount of predicted data (i.e., audio data transmission requirements); second, continuous time slots are allocated to transmitters with larger amounts of predicted data first, in order to reduce time slot fragmentation.

[0373] Furthermore, when it is detected that the first transmitting end does not have any encoded audio data to be transmitted within a preset time period (i.e., its predicted data volume P), next When the number of idle time slots is 0, the system releases at least one wireless transmission time slot originally allocated to the first transmitter, for example, allocating only a minimum keep-alive time slot to maintain basic physical connection and link synchronization; the freed-up wireless transmission time slots are then reallocated to the second transmitter whose audio data transmission demand meets the set conditions. Specifically, the remaining idle time slots are allocated to the second transmitter where the predicted data volume is at a moderate level, according to the proportion of the predicted data volume.

[0374] In other words, at least one minimum keep-alive time slot is allocated to the sender with a predicted data volume of 0; finally, the remaining time slots are allocated to the senders with a predicted data volume of medium level according to the proportion of the predicted data volume.

[0375] After completing the dynamic time slot allocation, the receiving end writes the allocation result into the synchronization packet header of the next frame and broadcasts it to all sending ends via the air interface. Upon receiving the synchronization packet, each sending end updates its own transmission time slot table and transmits the encoded audio frame in the next frame according to the updated time slot table. By broadcasting the dynamic time slot allocation result at the beginning of each frame, multiple sending ends can use the same time slot configuration, avoiding transmission conflicts caused by inconsistent time slot configurations.

[0376] To more clearly illustrate the dynamic allocation process described above, this embodiment provides a specific application scenario. In this scenario, the wireless communication system includes three transmitters: a vocal microphone, a guitarist microphone, and a drummer microphone. The detection and time slot calculation process for each transmitter within the current frame period is as follows: The test results for the lead vocal microphone (as one of the secondary transmitters) are as follows: ,and ,therefore: .

[0377] The guitarist's microphone (as the primary transmitter) is currently muted. ,therefore: .

[0378] Drummer microphone (as another secondary transmitter) ,and ,therefore: .

[0379] After collecting the above prediction data, the receiving end sorts and allocates time slots according to the prediction data volume of the lead singer microphone (0.8Pmax), drummer microphone (0.5Pmax), and guitarist microphone (0) from largest to smallest: the lead singer microphone is allocated 6 consecutive transmission time slots because of the largest transmission demand; the drummer microphone is allocated 4 transmission time slots; the guitarist microphone has no audio transmission demand, so its originally fixed time slot is released and only 1 minimum keep-alive time slot is allocated; the remaining time slots are used as system protection intervals or spare time slots.

[0380] This embodiment employs a dynamic time slot allocation scheme based on data volume prediction. This scheme can identify the first transmitting end in a silent state in real time, proactively release its unused physical time slots, and dynamically inject the released time slots into the more active second transmitting end with higher data transmission demand. Through this elastic bandwidth scheduling, the system's time slot utilization rate is significantly increased from 40% to 60% in traditional fixed allocation schemes to 80% to 95%. Without increasing the air interface bandwidth, it supports more transmitting ends simultaneously performing high-fidelity, low-latency operations.

[0381] Next, we will provide an exemplary description of time slot-level signal quality assessment, adaptive frequency hopping, and retransmission strategies.

[0382] In some implementations, to further improve the robustness of wireless audio transmission against multipath fading and electromagnetic interference, the method further includes: acquiring signal quality parameters corresponding to at least one wireless physical layer time slot; determining a signal quality index corresponding to the wireless physical layer time slot based on the signal quality parameters; determining a target channel from a set of candidate channels if the signal quality index satisfies the channel switching conditions; and switching the current working channel to the target channel in a pre-agreed switching time slot with the transmitting end and the receiving end.

[0383] Specifically, obtaining signal quality parameters corresponding to at least one wireless physical layer time slot, and determining the signal quality index (i.e., signal quality index SQI) corresponding to the wireless physical layer time slot based on the signal quality parameters, may include: The receiver (Rx) evaluates the wireless signal quality of the current operating channel (frequency point) within each wireless physical layer receive time slot and obtains the corresponding signal quality parameters. The signal quality parameters include at least one of Received Signal Strength Indication (RSSI), Packet Error Rate (PER), and Signal-to-Noise Ratio (SNR).

[0384] Furthermore, the receiving end normalizes the signal quality parameters and calculates the signal quality index (SQI) based on the normalized signal quality parameters and their corresponding weights. The specific calculation formula is as follows:

[0385] Wherein, SQI is the signal quality index; Normalized received signal strength; PER is the packet error rate; Normalized signal-to-noise ratio; , and These are the weight coefficients for the corresponding parameters, and they satisfy the following constraints: .

[0386] In one embodiment, and All were normalized to interval, The range of values ​​is also... Because a negative correlation transformation was used for the packet error rate in the calculation (i.e., using...) (representing packet accuracy), so that the final calculated... The range of values ​​is also limited to Interval.

[0387] Each weighting coefficient can be flexibly configured according to the actual wireless channel environment and application scenario. For example, in a typical configuration, the weighting coefficients can be set as follows: , , .

[0388] When the signal quality indicators meet the channel switching conditions, the steps of determining the target channel from the candidate channel set and performing channel switching in a pre-agreed switching time slot are specifically implemented as follows: When the receiver detects that the SQI of the current working channel is lower than the set limit threshold (i.e., the channel switching condition is met) within M consecutive physical time slots, the receiver marks the current working channel as an interfered channel and scans a preset set of candidate channels (frequency points) to select the candidate channel with the highest SQI value as the target channel (frequency hopping target frequency point).

[0389] After determining the target channel, the receiving end generates a channel switching command (frequency hopping command) and broadcasts it to the sending end in the synchronization packet header of the next frame. The sending and receiving ends synchronously switch their current operating channel to the target channel according to a pre-agreed switching time slot, thereby avoiding communication interruptions due to inconsistent switching timing.

[0390] In other words, if the SQI of the current frequency point is lower than the set SQI threshold for N consecutive time slots, the receiver marks the current frequency point as an interfered frequency point, scans the candidate frequency point list, and selects the candidate frequency point with the highest SQI as the frequency hopping target frequency point. The frequency hopping command is broadcast to each transmitter in the next frame synchronization packet.

[0391] As a specific example, the threshold value is set to 0.7. When the system detects an SQI of 0.55 in time slot 5 and an SQI of 0.52 in time slot 6, meaning that the SQI in two consecutive time slots (where M=2) is below the threshold of 0.7, the channel switching condition is deemed met. The receiver broadcasts a frequency hopping command in time slot 7 or the immediately following synchronization time slot, instructing each transmitter to synchronously switch to the candidate channel with the highest SQI at the beginning of the next frame.

[0392] In one specific implementation, the method further includes a parameter update and closed-loop control mechanism: after switching to the target channel, the receiver re-evaluates the SQI of the new channel and feeds back the obtained signal quality index or channel state information (CSI) generated based on the index to the transmitter. The transmitter updates at least one of the candidate channel set, frequency hopping sequence, and transmitter transmit power in real time based on the received signal quality index or channel state information. For example, when the fed-back SQI is low but has not reached the switching threshold, the transmitter can actively increase the transmit power to improve link quality; or it can remove channels that are continuously interfered with from the candidate channel set and frequency hopping sequence.

[0393] The transmitting and receiving ends synchronously perform frequency switching at a pre-agreed switching time to avoid operating on different frequencies. The switched frequency can be re-evaluated using SQI, and the candidate frequency list or frequency hopping sequence can be updated based on the evaluation results.

[0394] Furthermore, if the receiver detects a data packet loss, a retransmission decision is made based on the SQI of the current time slot and the number of remaining time slots in the current frame. When the SQI... current SQI threshold When remaining_slots≥1, it indicates that the current channel quality is good and time resources are sufficient. The receiver sends a retransmission instruction to the sender, and the sender immediately retransmits the lost data packet in the remaining time slot of the current frame.

[0395] When SQI current ≤SQI threshold When remaining_slots≥2, it indicates that the current time slot channel is subject to transient interference. The receiver does not require retransmission in the currently interfered time slot, but waits for the signal quality index to recover to above the threshold in subsequent time slots before retransmission. When the remaining time slots are insufficient to complete a reliable retransmission (e.g., remaining_slots<1), retransmission is abandoned, and the decoder's PLC packet loss concealment function is called to generate compensating audio data.

[0396] Thus, this embodiment reduces the frequency hopping response time from hundreds of milliseconds to less than one frame by evaluating the signal quality index in each receiving time slot and performing frequency hopping in the next frame after confirming interference at the current frequency point in multiple consecutive time slots. For example, in a 5ms frame long embodiment, the frequency hopping response time can be less than 5ms; in a dense WiFi environment, the audio packet loss rate can be reduced from 5% to 10% to less than 1%.

[0397] Figure 10 This document presents a flowchart illustrating the overall data processing and transmission process of a dual-microphone wireless audio system according to some embodiments of this application. To further clarify the overall signal processing and transmission paths of the dual-microphone transmitter and receiver in some embodiments of this application, the following is a detailed explanation... Figure 10 A detailed explanation of the end-to-end data processing flow of the entire wireless audio system is provided. For example... Figure 10 As shown, the system includes two symmetrical microphone acquisition branches and one reception and playback branch, with the specific processing flow as follows: At the transmitting end, the first microphone (Mic_1) and the second microphone (Mic_2) synchronously acquire external time-domain analog audio signals. Subsequently, their respective analog-to-digital converters (ADCs) complete the analog-to-digital conversion and output a digital PCM audio data stream. Then, the direct memory access (DMA) controller automatically moves the PCM audio data stream to the transmitting end's internal memory buffer and triggers a hardware interrupt when a frame of data is filled, thereby starting the interrupt service routine (ISR).

[0398] In the Interrupt Service Routine (ISR), the transmitter processor performs conditional configuration processing on the two audio data streams: First, it determines whether gain compensation is enabled on the production line. If so, it calls the preset production line calibration parameters to perform gain compensation on the digital PCM audio data to eliminate microphone sensitivity differences; otherwise, it skips the process. Second, it determines whether dynamic range control (DRC) processing is enabled. If so, it calls the DRC limiter (DRC_limiter) to implement amplitude limiting protection for signals exceeding a given threshold to prevent clipping distortion; otherwise, it skips the process. Then, it determines whether equalization (EQ) processing is enabled. If so, it performs the aforementioned frequency domain adaptive EQ processing to optimize the spectrum; otherwise, it skips the process.

[0399] After the above-described configuration process, the audio data is stored back into the corresponding transmit buffer. Subsequently, the transmitter calls the LC3+ encoder to perform low-latency compression encoding on the data in the buffer, and transmits the encoded data packets to the receiver (Dongle) via the 2.4G wireless radio frequency channel.

[0400] On the receiving end, the dongle receives encoded data packets from two microphones via a 2.4G wireless radio frequency channel and sends them to the LC3+ decoder for decoding and reconstruction. The decoded time-domain audio data is written into the receiver's DMA buffer and then routed to a digital-to-analog converter (DAC) via the lineout bus for processing, ultimately converting it into an analog signal to drive the speaker for sound playback.

[0401] Thus, by introducing conditional decision branches for gain compensation, DRC, and EQ, the system can dynamically enable corresponding algorithms based on the actual acoustic environment and production line calibration requirements, automatically skipping steps when no processing is needed, effectively reducing the operating power consumption of the transmitter MCU. Utilizing direct memory transfer between ADC / DAC and DMA, coupled with interrupt service routine (ISR) triggering, greatly reduces the CPU overhead in data addressing and copying, ensuring high real-time performance for dual-channel audio acquisition and playback. Finally, through pre-processing production line gain calibration and unified post-processing DRC / EQ, the physical differences between the two microphone devices are eliminated, ensuring high fidelity of the audio signal before wireless transmission and consistent listening experience across both channels.

[0402] To make the technical solutions and effects of some embodiments of this application more intuitive and easier to understand, this embodiment provides a specific application example of a live sound card system based on a 2.5ms frame length and an 8-timeslot configuration.

[0403] In this example, a wireless communication system is used in a mobile phone live streaming wireless microphone device to meet the requirements of extremely low lip-sync latency and high-fidelity voice transmission in live streaming scenarios. The specific parameter settings for this system are as follows: the audio sampling rate is set to... The frame period (frame length) of physical transmission. Set as The total number of physical time slots within each frame period Set as That is, the duration of a single time slot is (Corresponding slot numbers are Slot 0 to Slot 7); the codec uses the LC3+ codec, whose encoded frame length is aligned with the physical frame length. (The corresponding number of sampling points per frame is) (samples); physical coding time slots of the transmitter (Tx) Set as Physical decoding time slots at the receiver (Rx, in this example, the Dongle receiver adapter). Set as Time slot offset between the receiver and transmitter Set as (Right now ).

[0404] A constant delay scheduling example based on frame slot binding. In this example, the transmitter and receiver work together based on a strict slot pipelined parallel scheduling mechanism. The specific timing steps are as follows: Slot 0: The transmitter's (Tx) direct memory access (DMA) controller will load the frame slot 0 from the slot 0. Frame audio signal (total) (One PCM sample) is automatically moved from the analog-to-digital converter (ADC) to the memory buffer, and a hardware interrupt is triggered after the transfer is completed; Slot 1: The transmitter processor (CPU) responds to the hardware interrupt and extracts the audio signal, sequentially performing preset gain compensation, adaptive damping (DRC) processing, and frequency domain equalization (EQ) processing, and completes LC3+ encoding compression before the end of the current slot. The encoding slot corresponding to this step is... Slot 2: The transmitter enables the RF transmitter to transmit the encoded and compressed data in slot 2. The audio data packet is transmitted to the air interface via a 2.4 GHz wireless radio frequency channel in a one-way broadcast manner; Slot 5: The receiver (Dongle) starts receiving radio frequency precisely at the pre-bound time slot 5, and transmits the received audio data packet in a one-way broadcast manner. The frame data packet is sent to the LC3+ decoder for decoding. The corresponding decoding time slot for this step... Subsequent time slots: The receiver sends the decoded PCM audio data to the digital-to-analog converter (DAC) via the DMA channel, converts it into an analog audio signal, and routes it to the Lineout interface for output.

[0405] According to the formula for calculating constant end-to-end latency across the entire link:

[0406] Substituting the system parameters set above, we can calculate:

[0407] Therefore, from the moment the microphone at the transmitting end picks up the audio to the moment the analog audio is output from the headphone jack at the receiving end, the total physical delay is constantly locked at 1. This eliminates time delay jitter in the time domain.

[0408] Adaptive Limiting (DRC) and Physical Time Slot Coordination Example. In this example, the basic limiting threshold of the limiting processing module... Set as Mapping sensitivity coefficient Set as Slot scaling factor Set as The system obtains the current processing time slot in real time. The dynamic calculation process is as follows: When the system is in time slot 1 (i.e., the coding time slot), at this time... , When processing data, the corresponding time slot difference is... Substitute the values ​​into the adaptive limiting threshold calculation formula to calculate the dynamic threshold:

[0409] At this point, the threshold tightens to its minimum (i.e.) The system applies the strictest amplitude limit to the signal about to be fed into the encoder to prevent encoding transient overflow; in contrast, when the system is in time slot 0 (i.e., the data transfer time slot, at which time...) , When performing amplitude limiting calculations, the corresponding time slot difference is... Substitute into the formula to calculate the dynamic threshold:

[0410] At this point, the threshold automatically widens to This preserves a wider dynamic range of the signal; when the amplitude of the input signal approaches the dynamic threshold calculated above. At this time, the limiting module abandons the hard limiting mechanism and instead uses a smooth window function. The gain is weighted with a soft inflection point. This ensures that the actual limiting gain decreases non-linearly and gradually, avoiding a step increase in gain at the critical value that could produce clicking noise.

[0411] Example of broadcast transmission without acknowledgment (ACK). In this example, the sender broadcasts a data packet to the air interface in slot 2 of the frame period. The data packet consists of a header and a payload, wherein the payload carries the first... The frame's LC3+ encoded data; the header contains the frame number. The system uses timestamp information, and only the first two data packets sent during the initial connection establishment phase are enabled and include an "ACK request" flag. The specific transmission and exception handling process is as follows: Initial Synchronization Phase: After system startup, the first two data packets sent by the transmitter carry the "ACK request" flag. Upon receiving these data packets, the receiver replies with an ACK confirmation packet to the transmitter within the corresponding feedback physical time slot, and writes the receiver's clock reference into the ACK packet. This completes high-precision physical time alignment between the two ends. Steady-State Broadcast Transmission Phase: After initial synchronization, the "ACK request" flag in the header of subsequent audio data packets sent by the transmitter is cleared (no longer requesting ACK). The transmitter continues to perform one-way broadcast transmission, while the receiver performs one-way reception listening according to the aligned time slot beat. Packet Loss Concealment Handling: If the receiver fails to successfully receive the first data packet due to transient interference in the wireless channel... Since the frame data packet is in a steady-state phase without ACK, the receiver's physical layer prohibits it from sending any retransmission requests to the transmitter. Simultaneously, the receiver triggers the packet loss concealment (PLC) algorithm to extract the first frame data packet. The correct audio data of the frame is used to perform multi-point interpolation and exponential decay compensation to reconstruct the first frame. The frame-alternate waveform output eliminates audio interruptions caused by packet loss; latency savings: compared to related technologies where each packet requires waiting for an ACK response (assuming each wait and processing of an ACK consumes time...). (When multiple packets are sent consecutively, this will accumulate to introduce several milliseconds of additional waiting delay). This embodiment cancels the ACK response during the steady-state phase, eliminating this delay overhead and ensuring... End-to-end ultra-low physical latency.

[0412] In other scenarios, such as single-person live streaming, this embodiment, when applied to a scenario of live streaming with a single wireless microphone, can include a transmitter and a receiver. The system parameters are as follows: audio frame length... Number of time slots per frame The sampling rate is Quantization bit width is Encoding slots Decoding time slot Fixed offset of encoding / decoding time slots Basic amplitude limit threshold Threshold attenuation coefficient Threshold change steepness coefficient The LC3+ coding rate is .

[0413] In time slot 0, the DMA moves 120 sampling points of the current audio frame acquired by the ADC to the data buffer, and triggers an interrupt after the entire frame data is moved. The CPU reads the complete audio frame, and the audio content classification unit determines that the current audio content is speech based on the fundamental frequency, spectral energy, and zero-crossing rate, and maintains this information. Frame length and Encoding bitrate.

[0414] In time slot 1, the transmitting end performs time slot-linked adaptive dynamic range control. Because... ,therefore:

[0415] Subsequently, the encoder completes the LC3+ encoding of the current audio frame within a fixed encoding time slot.

[0416] In time slot 2, the transmitter broadcasts the currently encoded audio frame via 2.4G radio. Since the system includes only one transmitter, the dynamic time slot allocation unit allocates available transmission time slots to that transmitter. The receiver performs SQI evaluation on the received signal in time slots 3 and 4. Under the current environment... When the SQI is higher than the SQI threshold of 0.7, the receiver does not perform frequency hopping. In time slot 5, the receiver performs LC3+ decoding on the received encoded audio frames. The decoded PCM data is buffered by DMA and then converted by DAC before being output. The end-to-end delay is: .

[0417] This embodiment can achieve an end-to-end latency of approximately 3.75ms in moderate WiFi interference environments such as home living rooms, with latency jitter of less than 1ms, making it suitable for live lip-sync scenarios.

[0418] In another scenario, such as a multi-person performance, this embodiment is applied to a three-person band performance. The system includes three transmitting ends: a lead vocal microphone, a guitarist microphone, and a drummer microphone, as well as one receiving end. The system uses 5ms audio frames, each frame is divided into 14 time slots, the sampling rate is 48kHz, the quantization bit width is 24bit, and the LC3+ bit rate is 160kbps.

[0419] The lead vocalist's microphone VAD result is 1, and the ratio of current frame energy to maximum energy is 0.8. Therefore, the predicted data volume for the next frame is: .

[0420] The guitarist's microphone has a VAD value of 0, therefore: .

[0421] The drummer's microphone has a VAD result of 1, and the ratio of current frame energy to maximum energy is 0.5. Therefore: .

[0422] The receiving end arranges the three transmitting ends in descending order of predicted data volume to obtain the allocation order of the lead vocalist microphone, drummer microphone, and guitarist microphone. The lead vocalist microphone is allocated 6 consecutive time slots, the drummer microphone is allocated 4 consecutive time slots, and the guitarist microphone is allocated 1 keep-alive time slot. The remaining time slots are used as guard intervals or spare time slots. The allocation results are broadcast to the three transmitting ends via the next frame synchronization packet.

[0423] In time slot 5 of a certain frame, the receiver detected an SQI of 0.55, which is below the SQI threshold of 0.7; in time slot 6, it detected an SQI of 0.52, with two consecutive time slots below the threshold. The receiver marks the current frequency as an interfered frequency and selects the frequency with the highest SQI from the candidate frequency list as the frequency hopping target. A frequency hopping instruction is broadcast in the synchronization packet of the next frame, causing the three transmitters to switch to the target frequency at the beginning of the next frame.

[0424] After switching to the target frequency, the SQI returned to 0.88. Through dynamic time slot allocation, the time slot utilization rate was increased from approximately 50% with the fixed allocation scheme to approximately 85%, and the frequency hopping response time was controlled within 5ms.

[0425] In another scenario, such as a meeting setting, variable frame length switching occurs as follows: This scenario utilizes a wireless microphone system in a meeting room. The system initially uses a 5ms frame length and is in silent mode.

[0426] When the speaker pauses speaking, VAD detects silence. After five consecutive frames meet the silence condition, the transmitter switches to silence mode, maintains a 5ms frame length, and only sends keep-alive packets with the lowest bitrate.

[0427] When the speaker begins speaking, VAD detects it as speech. After 10 consecutive frames of speech features are detected, the transmitter switches to speech mode, using a 2.5ms frame length and a 96kbps encoding rate. The switching process is performed at the audio frame boundary, and the transition frame carries a switching marker, the new frame length configuration, and the start position of the new frame length data.

[0428] After receiving the transition frame, the receiving end updates the decoder's frame length and encoding parameters at the agreed frame boundaries and performs decoding according to the new decoding time slot. As a result, the end-to-end latency in the voice phase can reach 3.75ms, the bandwidth usage in the silence phase can be reduced by about 90%, and no audio stuttering occurs during the frame length switching process.

[0429] Some embodiments of this application also provide a wireless communication device, which includes a memory and a processor. The memory is used to store an executable program, and the processor is used to run the program. When the program runs, it executes the wireless audio transmission method of any of the above embodiments.

[0430] The processor can be a central processing unit, digital signal processor, microcontroller, application-specific integrated circuit, field-programmable gate array, or other processing circuit capable of executing audio processing and wireless timing control instructions. The memory can include random access memory, read-only memory, flash memory, or other non-transitory computer-readable storage media.

[0431] The executable program may include at least one of the following: a time slot configuration program, a limiting threshold adjustment program, an audio limiting program, an audio encoding program, a wireless transmission control program, an initial synchronization program, a dynamic frequency domain equalization program, and a packet loss concealment program. Different programs may be executed by the same processor, or they may be executed separately by a general-purpose processor, a digital signal processor, and processing units in a wireless communication chip.

[0432] By implementing the above wireless audio transmission method programmatically, the audio frame length, physical layer time slot number, time slot scheduling table, dynamic amplitude limiting parameters, dynamic equalization parameters, and synchronization and packet loss concealment strategies can be flexibly configured according to the specific hardware platform, thereby adapting to different types of wireless audio transmitting and receiving devices.

[0433] In some embodiments, this application also provides an electronic device, including: a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to cause the electronic device to implement the wireless audio transmission method described in any of the above embodiments when executing the computer program.

[0434] This application embodiment may also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps in the above-described wireless audio transmission method.

[0435] In some embodiments, this application provides a computer program product that, when run on a computer, enables the computer to implement the wireless audio transmission method described in any of the above embodiments.

[0436] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the described embodiments and various different variations of embodiments suitable for specific use considerations.

[0437] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A wireless audio transmission method, characterized in that, The method, applied to the transmitter of a wireless communication system, includes: The duration of the audio frame is divided into multiple time windows that are aligned with the wireless transmission time slots based on the duration of the wireless transmission time slots, and the target encoding time slot and target transmission time slot corresponding to the current frame are determined based on the time windows. The process involves: obtaining the time slot difference between the current audio data processing time slot and the target encoding time slot; wherein the time slot difference is determined by the difference between the time slot index of the first time slot and the time slot index of the target encoding time slot; calculating a dynamic limiting threshold based on the time slot difference when the first time slot is not later than the target encoding time slot, such that the dynamic limiting threshold gradually decreases as the first time slot approaches the target encoding time slot; obtaining the absolute amplitude of the sample points of the input audio signal; and calculating an attenuation gain according to the ratio of the dynamic limiting threshold to the absolute amplitude of the sample points when the absolute amplitude of the sample points is greater than the dynamic limiting threshold, and multiplying the input audio signal by the attenuation gain to perform amplitude limiting processing on the input audio signal based on the limiting threshold; and... During the target encoding time slot, the encoder is activated to encode the amplitude-limited audio signal, and the encoded audio data is transmitted to the receiving end of the wireless communication system during the target transmission time slot.

2. The method according to claim 1, characterized in that, Transmitting encoded audio data to the receiving end within the target transmission time slot includes: During the target transmission time slot, the encoded audio data is transmitted to the receiving end in a one-way broadcast format using a wireless transmitter, and no acknowledgment signal is waited for from the receiving end after the encoded audio data is transmitted.

3. The method according to claim 1, characterized in that, After the clipping process and before encoding the audio signal, the method further includes: The clipped audio signal is moved to the encoding buffer via a direct memory access channel.

4. The method according to claim 1, characterized in that, The limiting process also includes: Obtain a smoothing window function, wherein the gain value of the smoothing window function is a first preset value during the middle period of the audio frame, and the gain value of the smoothing window function smoothly transitions from the first preset value to a second preset value during the transition period at the beginning and end of the audio frame; and The smoothing window function is used to perform weighted operations on the amplitude-limited audio signal to smooth the gain changes of the audio frames at the switching points.

5. The method according to any one of claims 1-4, characterized in that, After limiting the input audio signal and before encoding the audio signal, the method further includes: Obtain the subband signal-to-noise ratio of the input audio signal at different frequency bands; Based on static equalization parameters and the sub-band signal-to-noise ratio (SNR), a dynamic gain is calculated. This dynamic gain is increased in frequency bands where the sub-band SNR is higher than a set SNR threshold, and decreased in frequency bands where the sub-band SNR is lower than the set SNR threshold. The audio signal is gain compensated using a filter based on the dynamic gain.

6. The method according to claim 5, characterized in that, The calculation of dynamic gain based on static equalization parameters and the sub-band signal-to-noise ratio also includes: Obtain the quantile spectrum deviation of the input audio signal, wherein the quantile spectrum deviation is the difference between the target quantile spectrum and the current input quantile spectrum; and The dynamic gain is calculated by combining the static equalization parameters, the subband signal-to-noise ratio, and the quantile spectrum deviation.

7. The method according to claim 2, characterized in that, The method further includes: During the initial synchronization phase, the sending end sends a synchronization data packet containing a timestamp to the receiving end in a specified synchronization time slot; Within a preset reception time slot, the sender receives an acknowledgment packet from the receiver, and aligns the time bases of the sender and receiver based on the acknowledgment packet; and After completing time base alignment, the system enters a steady-state phase where encoded audio data is transmitted in a one-way broadcast format.

8. The method according to claim 7, characterized in that, The method further includes: During the initial synchronization phase, if the sending end does not receive the acknowledgment packet within the preset reception time slot, a synchronization data packet retransmission process is triggered; and If the number of retransmissions of the synchronization data packet reaches the set upper limit and no acknowledgment packet is received, the wireless transmitter of the sending end is turned off.

9. The method according to any one of claims 1-4, characterized in that, The method further includes: The acquired audio signal is sampled by an analog-to-digital converter to generate pulse code modulated audio data; The pulse-code modulated audio data is moved to the data buffer via a direct memory access channel; and When the accumulated pulse code modulation audio data in the data buffer reaches the data size of a complete audio frame, a processor interrupt is triggered so that the processor can perform audio processing on the complete audio frame.

10. A wireless audio transmitting device, characterized in that, The method applied to any one of claims 1-9 includes: The timing control module is used to divide the duration of an audio frame into multiple time windows that are aligned with the wireless transmission time slots according to the duration of the wireless transmission time slots, and to determine the target encoding time slot and target transmission time slot corresponding to the current frame according to the time windows. The amplitude limiting module is used to obtain the time slot difference between the first time slot where the current audio data processing is located and the target encoding time slot, adjust the amplitude limiting threshold according to the time slot difference, and perform amplitude limiting processing on the input audio signal based on the amplitude limiting threshold. An encoding module, configured to start in the target encoding time slot, encode the clipped audio signal to generate encoded audio data; and A wireless transmission module is used to transmit the encoded audio data to the receiving device within the target transmission time slot.

11. A wireless audio receiving device, characterized in that, The method applied to any one of claims 1-9 includes: The wireless receiving module is used to receive encoded audio data broadcast unidirectionally by the transmitting end within the wireless physical receiving time slot; A decoding module, connected to the wireless receiving module, is used to start within the physical decoding time slot and decode the encoded audio data; and The audio processing unit is used to prevent the sending of a retransmission request to the sending end when the loss of the encoded audio data packet is detected, and to call the packet loss hiding algorithm to generate compensation audio data by interpolation based on the historically decoded audio data.

12. The receiving device according to claim 11, characterized in that, The method of invoking the packet loss hiding algorithm to generate compensated audio data by interpolation based on historically decoded audio data includes: Compensation frames are generated by weighted interpolation of multiple frames of historical audio data, or by applying a frame-by-frame attenuation envelope to the decoded audio data of the previous frame. Upon receiving valid encoded audio data again, a cross-fade transition is performed between the compensated audio data and the newly decoded audio data.

13. A wireless communication system, characterized in that, It includes a transmitting end and a receiving end, wherein the transmitting end includes the wireless audio transmitting device as described in claim 10, and the receiving end includes the wireless audio receiving device as described in claim 11 or 12.

14. A wireless communication device, characterized in that, include: A memory for storing an executable program; and a processor for running the program, wherein the program, when running, performs the wireless audio transmission method according to any one of claims 1-9.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a computing device, the computing device implements the wireless audio transmission method according to any one of claims 1-9.

Images