Clock Synchronization and Latency Reduction in Audio Wireless Multi-Channel Audio Systems (WMAS)

Abstract

System-wide clock synchronization and latency reduction of audio signals in a wireless multi-channel audio system (WMAS). The apparatus includes a base station and a wireless audio device. The base station includes a master clock source, a framer operative to generate base station frames including audio data and associated audio clock timing derived from the master clock source, and a transmitter operative to transmit the frames via the WMAS. The wireless audio device includes a receiver operative to receive frames from the base station via the WMAS, a frame synchronization circuit operative to generate audio data and associated timing signals from the received frames, and a clock generator circuit operative to receive a local clock signal generated by the frame synchronization circuit and generate therefrom a plurality of clocks derived from the timing signals to synchronize the wireless audio device to the base station frames, thereby enabling communication with the base station according to a predetermined schedule.

Description

[Technical field]

[0001] The subject matter disclosed herein relates to the field of communications, and more particularly to systems and methods for clock synchronization and latency reduction in multi-device two-way communication systems, such as wireless multi-channel audio systems (WMAS), also known as wireless venue area networks (WVANs). [Background technology]

[0002] Wireless audio (and video) (A / V) equipment used for real-time production of audiovisual information, for example for entertainment, live events, and conferences, is denoted by the term Program Production and Special Events (PMSE). Typically, wireless A / V production equipment includes cameras, microphones, in-ear monitors (IEMs), conferencing systems, and mixing consoles. Use cases for PMSE vary, but generally, each is used within a limited local geographic area and for a limited duration. A typical live audio / video production setup requires very low latency and highly reliable transmission to avoid glitches and perceptible corruption of the media content.

[0003] Accurate synchronization is also important to minimize jitter between samples captured by multiple devices and accurately render the audio-video content. For example, consider a live audio performance where microphone signals are streamed over a wireless channel to an audio mixing console where the various incoming audio streams are mixed. The in-ear mixed audio is streamed back to the microphone users via the wireless IEM system. To achieve this, the audio sampling of the microphone signals must be synchronized to a system clock that is typically integrated into the mixing console used to capture, mix, and play back the audio signals.

[0004] Wireless microphones are commonly used today for a variety of applications, including concerts and other events in large venues where the use of wired microphones is not practical or desirable. A wireless microphone has a small battery-powered radio transmitter in the microphone body that transmits the audio signal from the microphone by radio waves to a nearby receiver unit, which recovers the audio. Other audio equipment is connected to the receiver unit by a cable. Wireless microphones are widely used in the entertainment industry, television broadcasting, and public speaking, allowing speakers, interviewers, performers, and entertainers to move around freely while using the microphone without the need for a cable attached to the microphone.

[0005] Wireless microphones usually operate in the VHF or UHF frequency bands, allowing the transmitter to use a small, inconspicuous antenna. Cheaper models use a fixed frequency, but most models allow a choice of several frequency channels in case of channel interference or to allow multiple microphones to be used simultaneously. FM modulation is usually used, but some models use digital modulation to prevent unauthorized reception by scanner radio receivers, and these operate in the 900 MHz, 2.4 GHz, or 6 GHz ISM bands. Some models use antenna diversity (i.e. two antennas) to prevent gaps in transmission due to the movement of the performer.

[0006] Most analog wireless microphone systems use wideband FM modulation, requiring a bandwidth of approximately 200 kHz. The relatively large bandwidth requirement effectively limits the use of wireless microphones to VHF and above. Older wireless microphone systems operate in the VHF portion of the electromagnetic spectrum.

[0007] Many modern wireless microphone products operate in the UHF television band. In the United States, this low range spans from 470 MHz to 614 MHz. Typically, wireless microphones operate in unused TV channels ("white space"), where there is room for one or two microphones for every megahertz of available spectrum.

[0008] Pure digital radio microphone systems are also in use, using a variety of digital modulation methods. Some use the same UHF frequencies used by analog FM systems to transmit digital signals at a fixed bit rate. These systems encode an RF carrier with one, and sometimes two, channels of digital audio. Advantages offered by pure digital systems include lower noise, lower distortion, opportunities for encryption, and improved transmission reliability.

[0009] Some digital systems use frequency-hopping spread spectrum technology similar to that used in cordless phones and radio-controlled models. This can require more bandwidth than a wideband FM signal, so these microphones typically operate in the unlicensed 900 MHz, 2.4 GHz, or 6 GHz bands.

[0010] Some disadvantages of wireless microphones include: (1) limited range (wired balanced XLR microphones can operate up to 300 feet or 100 meters); (2) potential for interference from other radio equipment or other radio microphones; (3) limited operating time relative to battery life (shorter than normal condenser microphones due to the greater battery drain from the transmit circuitry); (4) noise or dead spots, especially in non-diversity systems; (5) a limited number of radio channels (i.e., frequencies), limiting the number of microphones that can operate at the same time and place; and (6) poor sound quality.

[0011] Another important factor with the use of wireless microphones is latency, which is the time it takes for an audio signal to travel from the input (i.e., the microphone) to the audio output (i.e., the receiver or mixing console). In an analog wireless system, the microphone converts the acoustic energy of the sound source into an electrical signal that is then transmitted over radio waves. Because both the electrical and RF signals travel at the speed of light, the latency of an analog wireless system is negligible.

[0012] In a digital radio system, the acoustic to electrical conversion is the same, but the electrical audio signal is converted into a digital bit stream. This analog audio to digital conversion takes time, which introduces latency into the system. The amount of latency in a digital radio system depends on the amount of signal processing involved, and also on the RF mechanism utilized.

[0013] For a typical performer, a latency of 5-10 ms is acceptable in a live performance using stage monitors. Above 10 ms, the signal delay becomes noticeable and can adversely affect the performer's timing and overall performance. Latency is particularly important for certain performers, such as vocalists and drummers in live applications using in-ear monitor systems, because the performer hears himself / herself both from the monitoring system and through bone vibrations. In such situations, round-trip latency should be 6 ms or less to avoid compromising the performance. Summary of the Invention

[0014] This disclosure describes a system and method for clock synchronization and latency reduction in a multi-device two-way communication system, such as an audio wireless venue area network (WVAN), also referred to as a wireless multi-channel audio system (WMAS). The WMAS of the present invention includes a base station and wireless audio devices, such as microphones, in-ear monitors, etc., that may be used at live events, concerts, nightclubs, houses of worship, etc. The WMAS is a multi-channel digital wideband system, in contrast to many commercial narrowband, e.g., GFSK, and analog prior art wireless microphone systems. The system may be designed to provide extremely low latency, e.g., less than 6 milliseconds, for round-trip audio delay from microphone to mixing console and back to in-ear monitor.

[0015] Low latency can be achieved by synchronization of the entire system, including codecs, transmit and receive frames, local clocks, messages, and frame synchronization. In one embodiment, the entire OSI stack is synchronized. The system uses a single master clock in the base station to which all other clocks, both base stations and devices, are locked and derived.

[0016] Also, the size of the TX packet buffer at both the device and the base station may be an integer of the size of the audio compressor buffer at the transmitter, and at the receiver the RX packet buffer may be an integer multiple of the size of the audio enhancer buffer. This allows for the elimination of the audio compressor output buffer (and audio enhancer input buffer), with compressed packets being written directly from the compressor to the TX packet buffer (and directly from the RX packet buffer to the enhancer). Eliminating the audio compressor output buffer (and audio enhancer input buffer) significantly reduces the overall latency of the audio. System-wide synchronization allows for the elimination of the audio compressor output buffer and audio enhancer input buffer.

[0017] Thus, according to an embodiment of the present invention, system-wide clock synchronization and latency reduction of audio signals for use in a wireless multi-channel audio system (WMAS) is provided. The WMAS includes a base station and a wireless audio device. The base station includes a master clock source, a framer operative to generate base station frames including audio data and associated audio clock timing derived from said master clock source, and a transmitter operative to transmit the frames via the WMAS. The wireless audio device includes a receiver operative to receive frames from the base station via the WMAS, a frame synchronization circuit operative to generate audio data and associated timing signals from the received frames, and a clock generator circuit operative to receive a local clock signal generated by said frame synchronization circuit and generate therefrom a plurality of clocks including an audio clock derived from the timing signal, and to synchronize the wireless audio device to the base station frames, thereby enabling communication with the base station according to a predetermined schedule. The predetermined schedule may include uplink and downlink communication over the same channel.

[0018] In a wireless audio device, the frame synchronization circuitry may operate to generate audio data and associated timing using PHY frame boundary timing detected via signal correlation associated with a received frame.

[0019] The wireless audio device may be included in a microphone system. The wireless audio device may further include a first RF modem including an analog-to-digital converter (ADC) for converting an input audio signal to a digital domain using an audio clock generated by a clock generator circuit, a synchronization buffer operative to receive a digital output of the ADC, a compressor and an associated compressor buffer operative to receive an output of the synchronization buffer, and a transmitter and an associated TX packet buffer operative to receive an output of the compressor. Compressed packets may be written directly from the compressor buffer to the TX packet buffer for transmission to the base station. The TX packet buffer size may be an integer number of the size of the compressor buffer.

[0020] The wireless audio device may be included in an in-ear monitor. The wireless audio device may further include an expander and an associated expander buffer, an RF modem and an associated RX packet buffer that operates to output compressed packets that are written directly from the RX packet buffer to the expander buffer, and a digital-to-analog DAC converter that operates to input audio samples from the expander buffer to output an analog audio signal using an audio clock. The RX packet buffer size may be an integer number of the expander buffer size.

[0021] The master clock source may include a local oscillator in the base station or a clock signal from an audio mixing console to a digital interface in the base station.

[0022] In operation, (i) a clock in the WMAS synchronized to and derived from a master clock source in the base station, and (ii) communication on a predetermined schedule, allows for latency to be reduced to 4 milliseconds or less, and in some embodiments, less than 3 milliseconds. Latency is the time interval between receipt of an audio event at a microphone and output of an audio signal from the base station corresponding to the audio event. The wireless audio device may include synchronization circuitry operative to provide digital feedforward synchronization of an audio clock to frame synchronization clock timing, or analog feedback synchronization of an audio clock to frame synchronization clock timing.

[0023] The multiple clocks derived from the clock generator circuit may include an analog-to-digital converter (ADC) clock, a digital-to-analog converter (DAC) clock, a transmitter (TX) clock, a receiver (RX) clock, and / or a radio frequency (RF) clock.

[0024] The frame synchronization circuit may include at least one of a packet detector circuit, a correlator circuit, a phase-locked loop (PLL) circuit, a delay-locked loop (DLL) circuit, and a frequency-locked loop (FLL) circuit.

[0025] According to embodiments of the present invention, various methods are provided for system-wide clock synchronization of audio signals for use in a wireless multi-channel audio system (WMAS) including a base station and wireless audio devices. At the base station, a master clock source is provided and a first clock is generated including a first audio clock synchronized to the master clock source. A frame is generated including audio data and timing derived from the master clock. The frame is transmitted over the WMAS.

[0026] At the wireless audio device, frames are received from a base station via the WMAS and clock timing is generated from the received frames. A second clock is generated, including a second audio clock synchronized to the clock timing.

[0027] A first clock in the base station and a second audio clock in the wireless audio device are synchronized to a master clock source to enable communication with the base station according to a predetermined schedule. The second clock may include at least one of an audio clock, an ADC clock, a DAC clock, a TX clock, an RX clock, and / or an RF clock.

[0028] In the wireless audio device, audio data may be generated and associated clock timing may be detected using PHY frame boundary timing via signal correlation associated with received frames. The predetermined schedule may include uplink and downlink communications over the same frequency channel.

[0029] Synchronization of communications on a predetermined schedule can reduce latency to 4 nanoseconds or less, and in some embodiments, less than 3 nanoseconds. Latency is the time interval between receipt of an audio event at a microphone and output of an audio signal from the base station corresponding to the audio event. In wireless audio devices, the audio clock can be synchronized to the frame sync clock timing using digital feedforward synchronization or using analog feedback synchronization.

[0030] Clock timing from the received frame may be performed using a packet detector circuit, a correlator circuit, a phase-locked loop (PLL) circuit, a delay-locked loop (DLL) circuit, and / or a frequency-locked loop (FLL) circuit.

[0031] According to various embodiments of the present invention, a wireless audio device for use in a multi-channel audio system (WMAS), which may be included in a microphone system or an in-ear monitor, may include a receiver operative to receive frames via said WMAS, the frames including timing derived from a master clock source in said WMAS, a frame synchronization circuit operative to extract clock timing from said received frames, and a clock generator circuit operative to generate a plurality of clocks synchronized to the clock timing generated by the frame synchronization circuit to synchronize the wireless audio device to the received frames, thereby enabling communication according to a predetermined schedule.

[0032] The frame synchronization circuitry may operate to generate audio data and associated timing using PHY frame boundary timing detected via signal correlation associated with received frames. The predetermined schedule may include uplink and downlink communications over the same frequency channel. In operation, (i) a clock synchronized to and derived from said master clock source, and (ii) communications according to a predetermined schedule, may reduce latency to 4 nanoseconds or less, and in some embodiments, less than 3 milliseconds. Latency is the time interval between receipt of an audio event at a microphone and output of an audio signal from the base station corresponding to the audio event.

[0033] Various methods and systems are provided herein for minimizing latency in a wireless multi-channel audio system (WMAS) including a base station and multiple wireless audio devices. In the wireless audio devices, an audio clock is synchronized to a single master clock in the base station. An analog-to-digital converter (ADC) operates to convert an input audio signal to the digital domain using the audio clock. A synchronization buffer operates to receive a digital output from the ADC. A compressor and an associated compressor buffer operate to receive an output of the synchronization buffer. A first RF modem including a transmitter and an associated TX packet buffer operates to receive an output of the compressor. The TX packet buffer size is an integer multiple of the output of the compressor. The base station includes a single master clock, a second audio clock synchronized to the single master clock, a second RF modem including a receiver and an associated RX packet buffer, and an extender and an associated extender output buffer operative to receive an output of the RX packet buffer. The RX packet buffer size is an integer multiple of the input to the extender.

[0034] Provided herein are various methods and systems for minimizing latency in a wireless multi-channel audio system (WMAS) including a base station. In the base station, an audio clock is synchronized to a single master clock. An ADC operates to convert an input audio signal to the digital domain using the audio clock. A synchronization buffer operates to receive the digital output of the ADC. A compressor and an associated compressor buffer operates to receive the output of the synchronization buffer, and an RF modem including a transmitter and an associated TX packet buffer operates to receive a block output of the compressor. The TX packet buffer size is an integer multiple of the block output of the compressor.

[0035] Provided herein are various methods and systems for minimizing latency in a wireless multi-channel audio system (WMAS) including a base station. In the base station, an audio clock is synchronized to a single master clock. An RF modem including a receiver and an associated RX packet buffer operates to receive packets via the WMAS and store them in the RX packet buffer. An extender and an associated extender output buffer operates to receive a block output of the RX packet buffer. The RX packet buffer size is an integer multiple of the block input to the extender.

[0036] According to the present invention, there is also provided an apparatus for minimizing latency for use in a device in a wireless multi-channel audio system (WMAS) including a wireless device and a base station. The wireless device includes an audio clock synchronized to a single master clock in the base station, an analog-to-digital converter (ADC) for converting an input audio signal to the digital domain using the audio clock, a synchronization buffer operative to receive the digital output of the ADC, a compressor and associated compressor buffer operative to receive the output of the synchronization buffer, and an RF modem including a transmitter and associated TX packet buffer operative to receive a block output of the compressor. The TX packet buffer size is an integer multiple of the block output of the compressor.

[0037] According to the present invention there is further provided an apparatus for minimizing latency for use in a base station in a wireless multi-channel audio system (WMAS), the base station comprising an audio clock synchronized to a single master clock, an RF modem including a receiver and associated RX packet buffer operative to receive packets via said WMAS and store them in an RX packet buffer, and an extender and associated extender output buffer operative to receive a block output of the RX packet buffer, the RX packet buffer size being an integer multiple of the block input to the extender.

[0038] An uplink apparatus for system-wide clock synchronization of audio signals for use in a wireless multi-channel audio system (WMAS) is provided. The uplink apparatus includes a base station and a wireless audio device, e.g., a microphone. The base station includes a master clock source, a framer operative to generate frames including audio data and associated audio clock timing derived from the master clock source, and a transmitter operative to transmit the frames via the WMAS. The wireless audio device includes a receiver operative to receive frames from the base station via the WMAS, a frame synchronization circuit operative to generate audio data and associated timing signals from the received frames, a clock generator circuit operative to input a local clock signal from the timing signal generated by the frame synchronization circuit and generate therefrom an audio clock and a PHY clock, both derived from the timing signal, an analog-to-digital converter (ADC) for converting the input audio signal to the digital domain using the audio clock, a synchronization buffer operative to receive a digital output of the ADC, a compressor and associated compressor buffer operative to receive an output of the synchronization buffer, and a transmitter and associated TX packet buffer operative to receive an output of the compressor using the PHY clock. The compressed packets are written from the compressor buffer directly to a TX packet buffer for transmission to the base station.

[0039] A downlink apparatus for system-wide clock synchronization of audio signals is provided for use in a wireless multi-channel audio system (WMAS) including a base station and an in-ear monitor. The base station includes a master clock source, a framer operative to generate frames including audio data and associated audio clock timing derived from the master clock source, and a transmitter operative to transmit the frames via the WMAS. The in-ear monitor includes a wireless audio device including a receiver operative to receive frames from the base station via the WMAS, a frame synchronization circuit operative to receive RF modulated audio data and associated timing signals from the received frames, a clock generator circuit operative to input a local clock signal from the timing signal generated by the frame synchronization circuit and generate therefrom an audio clock and a PHY clock, both derived from the timing signal, an extender and associated extender buffer, an RF modem and associated RX packet buffer operative to output compressed packets that are written directly from the RX packet buffer to the extender buffer using the PHY clock, and a digital-to-analog DAC converter operative to input audio samples from the extender buffer to output an analog audio signal using the audio clock.

[0040] These, additional and / or other aspects and / or advantages of embodiments of the invention will be set forth in the detailed description that follows, and in some cases may be inferred from the detailed description and / or may be learned by practice of embodiments of the invention.

[0041] The present invention will now be described in further detail in the following exemplary embodiments with reference to the drawings in which identical or similar elements may be designated by identical or similar reference numerals, and in which the features of the various exemplary embodiments are combinable. The present invention is herein described, by way of example only, with reference to the accompanying drawings in which: [Brief description of the drawings]

[0042] [Figure 1] FIG. 1 illustrates an exemplary wireless multi-channel audio system (WMAS) incorporating the clock synchronization and latency reduction systems and methods of the present invention. [Diagram 2] FIG. 2 is a high-level block diagram illustrating an exemplary unidirectional link buffering and clocking scheme. [Diagram 3] FIG. 1 is a high-level block diagram illustrating an example uplink device and base station scheme. [Figure 4] FIG. 2 is a block diagram illustrating a first exemplary audio synchronization scheme using analog feedback synchronization. [Diagram 5] 4 is a block diagram illustrating a second exemplary audio synchronization scheme using digital feedback synchronization. [Figure 6] 1 is a high-level block diagram illustrating an example downlink device and base station. [Figure 7] FIG. 2 illustrates timing for an exemplary WMAS system. [Figure 8] FIG. 2 is a high-level block diagram illustrating an example uplink buffering and clocking scheme. [Figure 9] FIG. 2 is a high-level block diagram illustrating an example downlink buffering and clocking scheme. [Figure 10] 1 is a high-level block diagram of an exemplary frame synchronizer; [Figure 11] 1 is a flow diagram illustrating an example clock synchronization method for use in a base station. [Figure 12] 1 is a flow diagram illustrating an example clock synchronization method for use in wireless audio devices. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0043] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the present invention.

[0044] Among these disclosed benefits and improvements, other objects and advantages of the present invention will become apparent from the following description taken in conjunction with the accompanying drawings. Although detailed embodiments of the present invention are disclosed herein, it will be understood that the disclosed embodiments are merely exemplary of the invention that may be embodied in various forms. Also, each of the examples given in connection with various embodiments of the present invention are intended to be illustrative rather than limiting.

[0045] The subject matter which is regarded as the invention is particularly pointed out and distinctly claimed in the concluding portion of this specification. The invention, however, together with its objects, features, and advantages, both as to organization and method of operation, may best be understood by reference to the following detailed description taken in conjunction with the accompanying drawings.

[0046] The drawings constitute a part of this specification and include exemplary embodiments of the present invention and illustrate various objects and features thereof. Moreover, the drawings are not necessarily to scale, and some features may be exaggerated to show details of particular components. Moreover, any measurements, specifications, etc. shown in the drawings are intended to be illustrative rather than limiting. Thus, specific structural and functional details disclosed herein are not intended to be limiting, but merely as a representative basis for teaching those skilled in the art how to variously utilize the present invention. Moreover, where appropriate, reference numerals may be repeated among the drawings to indicate corresponding or similar elements.

[0047] The illustrated embodiments of the invention may, for the most part, be implemented using electronic components and circuits known to those skilled in the art, and will not be described beyond the extent deemed necessary for an understanding and appreciation of the underlying concepts of the invention and in order not to obscure or deviate from the teachings of the invention.

[0048] Any reference herein to a method should also be applied mutatis mutandis to a system capable of carrying out the method. Any reference herein to a system should also be applied mutatis mutandis to a method that may be carried out by the system.

[0049] Throughout this specification and the claims, the following terms have the meanings clearly associated therewith, unless the context clearly indicates otherwise. As used herein, the phrases "in one embodiment," "in an example embodiment," and "in some embodiments" do not necessarily refer to the same embodiment(s), but may. Additionally, as used herein, the phrases "in another embodiment," "in an alternative embodiment," and "in some other embodiments" do not necessarily refer to different embodiments, but may. Thus, as described below, various embodiments of the present invention can be easily combined.

[0050] Also, as used herein, the term "or" is an inclusive "or" operator and is equivalent to the term "and / or," unless the context clearly indicates otherwise. The term "based on" is not exclusive and may be based on additional unlisted factors, unless the context clearly indicates otherwise. Also, throughout this specification, the meanings of "a," "an," and "the" include plural references. The meaning of "in" includes "in" and "on."

[0051] As will be appreciated by those skilled in the art, the present invention may be embodied as a system, a method, a computer program product, or any combination thereof. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, which may be generally referred to herein as a "circuit," "module," or "system." Furthermore, the present invention may take the form of a computer program product embodied in any tangible medium of expression having computer usable program code embodied in the medium.

[0052] The present invention is described below with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. Each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, may be implemented or supported by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing device to manufacture a machine, such that the instructions, executing via a processor of the computer or other programmable data processing device, create means for performing the functions / acts described in one or more blocks of the flowchart illustrations and / or block diagrams.

[0053] These computer program instructions may be stored on a computer readable medium capable of instructing a computer or other programmable data processing apparatus to function in a particular manner to produce an article of manufacture including instruction means for implementing the functions / acts described in one or more blocks of the flowcharts and / or block diagrams, the instructions stored on the computer readable medium.

[0054] Computer program instructions may be loaded into a computer or other programmable data processing apparatus to cause the computer or other programmable apparatus to perform a series of operational steps to generate a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide a process for implementing the functions / operations described in one or more blocks of the flowcharts and / or block diagrams.

[0055] The present invention is operable with numerous special purpose or general purpose computing system environments or configurations. Well-known computing systems, environments, and / or configurations that may be suitable for use with the present invention include, but are not limited to, personal computers, server computers, cloud computing, handheld or laptop devices, multiprocessor systems, microprocessors, microcontrollers or microcomputer-based systems, set-top boxes, programmable appliances, ASIC or FPGA cores, DSP cores, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. System Architecture

[0056] A diagram illustrating an exemplary wireless multi-channel audio system (WMAS) incorporating the clock synchronization system and method of the present invention is shown in Figure 1. The exemplary WMAS, generally designated 10, comprises a base station 14, typically coupled to a mixing console 12 via one or more cables, and wireless devices including a wireless microphone 16, a monophonic in-ear monitor (IEM) 18, and a stereo IEM 20, optionally with an inertial measurement unit (IMU).

[0057] Wireless microphone device 16 includes an uplink (UL) 98 that transmits audio and management information, and a downlink (DL) 180 that receives management information. IEM device 18 includes an uplink 98 that transmits management information, and a downlink 180 that receives mono audio and management information. IEM device 20 includes an uplink 98 that may transmit IMU and management information, and a downlink 180 that may receive stereo audio and management information.

[0058] The WMAS comprises a star topology network (also referred to as the "network") with a central base station unit (BS) 14 that communicates with and controls all devices in the WMAS. The network is intended to provide reliable communications during a phase of the live event called "Showtime". The network at that time is set up and secured in a selected configuration. This minimizes the overhead that would normally be present in existing wireless standards that the network requires.

[0059] In an embodiment, features of the WMAS include: (1) a star topology; (2) point-to-multipoint audio with a predictable schedule including both DL and UL audio on the same channel (typically on the TVB frequency); (3) all devices are time synchronized to the base station frame; (4) support for fixed and prescribed devices; (5) support for frequency division multiplexing (FDM) for enhanced diversity schemes; (6) a TDM network where each device transmits its packets based on an a priori schedule; (7) a wideband base station with one or two transceivers that receive and transmit multiple audio channels (e.g., greater than four); (8) audio transmission in the DL and wideband OFDM(A) and TDM / OFDM for each device's packets in the UL; (10) primary and auxiliary radio channels are supported by all network entities; and (11) all over-the-air (OTA) audio streams are compressed with "zero" latency.

[0060] In terms of latency, the WMAS of the present invention is adapted to provide an extremely low latency system (i.e., audio path to audio path) of up to 4 ms, including a mixing console processing time of 2 ms. Audio events are received by a wireless microphone device. The audio is then wirelessly transmitted to a base station (BS) via an uplink. A wired handover to a general-purpose audio mixing console occurs with a fixed latency of up to 2 ms from receipt of the audio stream to the return audio stream. The processed audio stream returned to the base station is then wirelessly transmitted via a downlink to an IEM device which plays the audio stream to the user. Uplink latency is defined as an audio event received by a wireless microphone device and then wirelessly transmitted to a base station for output via an audio input / output (IO), and must be 2 ms or less.

[0061] In an embodiment of the present invention, a WMAS system may achieve performance with a tolerance supported under realistic conditions including: (1) a low packet error rate (PER) (e.g., 5e-8) where retransmissions are not applied; (2) short intervals of audio loss (e.g., 15 ms) due to successive packet losses and filled by an audio concealment algorithm; and (3) body shadowing.

[0062] The WMAS system is also adapted to operate in the DTV white space UHF channels (i.e., channels 38-51), although the system may use white space channels adjacent to very high power DTV station channels while still complying with performance and latency requirements.

[0063] We note that currently, the requirement for the network to ensure low latency across all device tiers to meet the desired performance (i.e. 100m range and 5e~8 packet error rate (PER)) is not supported by any standard. For example, the latency of the Bluetooth (BT) compander is larger than the overall latency required by itself (~6ms). BT-specific buffering between tiers is measured in milliseconds. We also note that the Wi-Fi 802.11ax standard is capable of supporting a minimum bandwidth of 20MHz and a maximum of eight devices, and that narrowband interferers may cause a complete loss of connectivity. Applying the above solution to the white space of the TV frequency band would not comply with most of the attributes summarized above.

[0064] To meet the desired low round-trip latency, the system of the present invention utilizes several techniques, including: (1) minimizing buffering to negligible levels by synchronizing all network entities to the base station baseband (BB) clock, achieved with a PHY synchronization signal (time allocation calculation) locked to the radio frame time established by the base station; (2) synchronizing all audio components to the baseband clock by a feedback signal from the synchronization buffer; (3) TX / RX PHY packets contain an integer number of compressed audio buffers; (4) efficient network design; and (5) use of a low-latency compander, where input buffer delay is the primary contributor to latency.

[0065] Throughout this document the following definitions apply: Latency (expressed as a time interval) of an audio system refers to the time difference between the moment a signal is fed into the system and the moment the signal appears at the output, keeping in mind that any system compression operations applied may be lossy, meaning that the signal at the output may not be identical to the signal at the input.

[0066] Uplink latency is defined as the latency of the system (i.e., device and base station) from the moment an audio event appears at the input of the ADC to the moment the event appears at the analog or digital audio of the base station. Downlink latency is defined as the latency of the audio system (device and base station) from the moment an audio event appears at the analog or digital audio of the base station to the moment it appears at the DAC output of the wireless device. Round-trip latency is defined as the latency of the audio system while looped back at the base station terminal from the moment an audio event appears at the uplink device input to the moment it appears at the downlink device output. Synchronized clocks are defined as clocks that appear to have no long-term drift between them. Clocks may have short-term jitter differences, but no long-term drift.

[0067] Reference is now made to Figure 2, which shows a high level block diagram of buffering and clocking in a unidirectional communication link. The system 30 includes a centralized base station 66 that communicates with one or more devices 64, such as, for example, wireless microphones, via a radio link 48. The base station 66 includes, among other things, a receiver (RX) 67, a digital-to-analog converter (DAC) buffer 60, and a digital-to-analog converter (DAC) 62. The RX circuit 67 includes an RX packet buffer 50 coupled to an audio extender 54 and an audio clock regenerator circuit 52. The audio extender 54 includes an audio extender unit 56 and an audio extender output 58. However, multiplexing, combining or mixing of several device audio streams at the base station is typically done using analog means (not shown).

[0068] The device 64 includes a clock management circuit 32, an audio clock 34, an ADC 36, and a TX circuit 65. The TX circuit includes an ADC buffer 38, an audio compressor circuit 40, and a TX packet buffer 42. The audio compressor circuit 40 includes an audio compressor input buffer 44 and an audio compressor output buffer 46.

[0069] Note that the typical audio compressor 40 of Figure 2 (e.g., MP3, AAC, LDAC) operates in blocks where each input block is compressed into an output block. Similarly, in the base station 66, the audio extender 54 relies on currently compressed blocks as well as previously received blocks to reconstruct nearby output blocks that correspond to the audio compressor input blocks. Many wireless audio compressors perform lossy compression to significantly reduce bandwidth (e.g., up to a 1:10 compression ratio).

[0070] Also note that the compressed buffer size is directly related to the compression ratio, with higher compression ratios requiring larger buffer sizes. A typical time delay estimate is around 0.2-0.5 ms. Also, from an audio latency perspective, it is preferable to match the size of the audio compressor buffer with the packet buffer size.

[0071] The clock management 32 as shown in FIG. 2 features a free-running clock relative to the base station 66. The main clock of the device 64 is the audio clock 34, and the clock management unit 32 derives the remaining clocks of the device 64 by digitally dividing or using clock multiplication methods such as, for example, a phase-locked loop (PLL), a frequency-locked loop (FLL), or a delay-locked loop (DLL). In the base station 66, each unit locks onto a corresponding clock. The RX PHY locks onto a frame clock (not shown) and reproduces the audio clock in block 52 for the audio enhancer 54 and the DAC component 62 that functions to output the analog audio signal 63. Since the system 30 includes any packet size (i.e., generic or non-deterministic), the audio compressor 40 in the device 64 accepts input audio data blocks from the ADC buffer 38, stores them in the input buffer 44, and compresses them into output audio data blocks that are stored in the output buffer 46. Therefore, an audio compressor 40 that accommodates any packet size must maintain a large output buffer 46 that contributes significantly to the delay and overall uplink latency. Similarly, in the base station 66, a large extender input buffer 56 contributes significantly to the delay and overall uplink latency.

[0072] 2, the components that contribute to latency in system 30 across devices 64 and base stations 66 are shown in Table 1. Latency contributors can be divided into three types: (1) core latency (e.g., buffer delay), which is latency that cannot be minimized by fast clocking or hardware layout; (2) hardware dependencies, which are latency that can theoretically be minimized to zero with fast clocking and / or hardware layout; and (3) media and filter PHY layer delays, although some hardware dependencies are involved, the main contributors are delays that are essential to achieve performance, e.g., receiver rejection. [Table 1]

[0073] As shown, the core latency of this scheme is Δ T1 and the duration Δ T7 , packet duration and other PHY related delays (e.g. filters, etc.) Δ T3 , the audio compressor input buffer duration Δ T6 Other hardware-related delays include the operation duration Δ T2 and Δ T5 Modem latency (e.g., receiver operation) is Δ T4 A summary of the various latencies in the system 30 of FIG.

number

[0074] A high level block diagram illustrating an exemplary uplink device and base station scheme in a multi-device two-way communication system wireless multi-channel audio system (WMAS) is shown in FIG. 3. The WMAS system, generally referenced 70, comprises a base station 74 that communicates with one or more wireless audio devices 72 via a radio link 98. The base station 74 comprises, among other things, a master clock 106, a clock generation circuit 108, a TX circuit 110 including a framer 112, a receiver (RX) 90, an audio block 114 including a DAC buffer 124, a DAC 116 and a digital interface circuit 118. The RX circuit 90 comprises a RX packet buffer 100 coupled to an audio extender 102 including an audio extender output buffer 104. The DAC 116 functions to generate an analog audio output signal 120, and the digital interface 118 generates a digital audio output signal 122 that is transmitted to the mixing console 12.

[0075] The device 72 comprises audio circuitry 81, a local clock source 83 (e.g., TCXO), RX circuitry 76, clock generator 80, synchronization buffer 86, and TX circuitry 88. An ADC 82 converts analog audio input 84 to a digital format that is fed to a synchronization buffer 86. A frame synchronization circuit 78 provides synchronization with the clock generator circuit 80. The output of the synchronization buffer 86 is input to a compressor input buffer 94 in an audio compressor 92. The output of the audio compressor 92 is input to a TX packet buffer 96.

[0076] 3 also illustrates various delays that contribute to system latency. In accordance with a feature of the present invention, the overall latency in the system is minimized by keeping the audio system tightly locked to the RF clock. In operation, a base station (BS) 74 serves multiple devices coexisting throughout the system. The device 72 illustrated in the system 70 functions in the uplink direction, i.e., to transmit audio from the device 72 to the BS 74. An example of the device 72 may include a wireless microphone 16, as described above in connection with FIG. 1.

[0077] Both the device and the base station include receivers and transmitters that aid in the clock recovery and locking process. The BS 74 includes a master clock 106, the output of which is locked to the clocks of the other BSs 74 (e.g., transmitter clock, DAC clock, receiver clock, console clock, etc.). It is understood that the master clock 106 may be selected by the designer without loss of versatility and is not critical to the present invention. In the device 72, the receiver 76 may use a periodic over-the-air time signal, such as a multicast downlink packet generated by the base station 74 and multicast to multiple devices 72 to generate and lock the receiver clocks, transmitter clocks, and ADC clocks in the device 72. An example of a periodic over-the-air time signal is a frame sync signal generated by the frame sync circuit 78 in the RX 76.

[0078] Note that the system may have either an analog output from the DAC 116 or an analog output from the digital console interface 118, which may contain uncompressed audio signals and optionally a master clock 123 for synchronization of the entire system. The use of a digital console interface allows for another Δ T7 Delay is saved, although this delay will be reintroduced if the console outputs its analog output to real speakers. However, if the signal is used for loopback (e.g. performer monitor signal), this delay is saved completely and will not be reintroduced.

[0079] The base station 74 of the system 70 may have an analog audio output 120 from the DAC 116 or a digital audio output 122 from the digital console interface 118, which may contain uncompressed audio signals. Optionally, a master clock 123 may be an input from the mixing console 12 to which all clocks in the system 70 may be synchronized.

[0080] Several key features of this system allow for a significant reduction in overall latency, including (1) the use of a master clock 106 in the base station 74 to which other clocks in both the base station 74 and the devices 72 are locked and / or derived, (2) the system is deterministic and does not involve schedule changes during showtime, and (3) the size of the packets used is an integer multiple of the size of the audio compressor output buffer (as well as the audio expander input buffer).

[0081] By making the size of the TX packet buffer an integer multiple of the size of the compressor output buffer, the elimination of the audio compressor output buffer 46 (and audio enhancer input buffer 56) in the system 70 (FIG. 3), as in the system 30 (FIG. 2), is possible because compressed packets are written directly from the compressor input buffer 92 to the TX packet buffer 96 (and directly from the RX packet buffer 100 to the expander output buffer 102). By eliminating the audio compressor output buffer 46 (and audio enhancer input buffer 56), the latency of the audio uplink is significantly reduced. Note that the synchronization of the base station 74 with the device 72 allows the elimination of the audio compressor output buffer 46 and audio enhancer input buffer 56.

[0082] The core latency of the uplink system 70 is determined by the duration Δ T1 and the duration Δ T7 , packet duration, and other PHY-related delays Δ T3 Note that the audio extender 90 includes a number of buffers (e.g., filters, etc.). However, since the packet size is an integer multiple of the compressor output buffer (and expander input buffer), there is no need for extra buffering. The output blocks generated by the output of the audio compressor 92 are simply inserted into the TX packet buffer 96, and once the last block has been written, the packet is transmitted by the transmitter 88. Conversely, at the base station 74, once the receiver 90 receives a complete packet, the audio extender starts working on the first block therein. This scheme significantly reduces the round-trip latency, which is typically 0.5 ms to 1 ms (assuming a compressor buffer size of 0.25 ms to 0.5 ms). Given the sensitivity of performers and artists to latency, this is a significant and much-appreciated improvement within the industry. A summary of the various latencies in the system 70 of FIG. 3 is provided below.

number

[0083] In different embodiments of the invention, alternative techniques for providing audio synchronization may be used. In some embodiments, the PHY digital clock of the device 72 may be synchronized to the PHY clock in the base station 74 by locking to the transmission frame. To achieve a maximum overall round trip latency target of 6 ms, the system should achieve perfect audio synchronization of the audio devices in the WMAS. The audio codec of the device, which is normally a free-running clock as well as the network PHY-locked (i.e., frame-locked) clock, which may be tagged as an output clock, may be synchronized using one of the following techniques, according to different embodiments of the invention. With reference to FIG. 4, digital feed-forward synchronization is described, where synchronization is performed in the digital domain driven by the output clock. The circuitry generally referenced 130 comprises an audio sampling clock 132, a synchronization buffer 134, a synchronization tracking block 138, and a Farrow polyphase filter 136.

[0084] In operation, the Farrow polyphase circuit 136 can interpolate signals at any fraction of the timing signal 135 with great accuracy, as commanded by the synchronous tracking circuit 138. Because the input and output clocks are independently free-running, the synchronous elastic buffer 134 may include a variable delay whose length varies (i.e., increases or decreases) based on the clock drift between the two clocks. The synchronization and tracking circuit 138 tracks this buffer length and infers the exact number of samples per frame. To compensate for clock drift, the circuit 138 changes the sampling point τ 135 provided to the Farrow polyphase filter 136 and changes the synchronous buffer switch position 137 (via a skip / add process).

[0085] A block diagram illustrating a second example of audio synchronization using analog feedback synchronization is shown in Figure 5. The second technique uses analog feedback synchronization, where a synchronization tracking circuit modifies the audio sampling rate (i.e., the input clock) by a feedback signal. The circuit, generally referenced 140, comprises an audio sampling clock 142, a synchronization buffer 144, and a synchronization tracking block 146.

[0086] In operation, the synchronous tracking 146 generates a feedback signal 145 that controls the audio sampling rate. The output audio clock is derived from the output of the synchronous buffer. Because the input and output clocks are independently free-running, the synchronous elastic buffer 144 may include a variable delay that varies in length (i.e., increases or decreases) based on the clock drift between the two clocks. The synchronization and tracking circuit 146 tracks this buffer length and infers the correct number of samples per frame. A variable input clock 142 is used to compensate for the drift in a form of feedback and ensure that the synchronous buffer 144 does not overflow or underflow.

[0087] A high level block diagram illustrating exemplary downlink devices and base stations in a multi-device two-way communication system is shown in Figure 6. The system, generally referenced 150, comprises a base station 74 that communicates with devices 72 over radio links 180 that constitute a WMAS. The base station 74 functions to serve multiple devices 72, and an exemplary device 72 is shown on the right. The device 72 receives audio in the downlink direction (i.e., from the base station 74 to the device 72). An example of a device 72 is an in-ear monitor (IEM), whether mono or stereo.

[0088] The base station 74 includes, among other things, a master clock 106, a clock generation circuit 108, a TX circuit 110, an RX circuit 160, an audio circuit block 114, and a synchronization buffer 168. The audio circuit 114 includes an ADC 164 and a digital interface circuit 118. The TX circuit 110 includes a framer 112, an audio compressor 174 including an input buffer 176, and a TX packet buffer 178.

[0089] The ADC 164 converts the analog audio input 200 to a digital format that is fed to a synchronization buffer 168. The framer circuit 112 provides synchronization to devices on the network. The output of the synchronization buffer 168 is input to a compressor input buffer 176 in an audio compressor 174. The output of the audio compressor 174 is input to a TX packet buffer 178.

[0090] The device 72 may include a temperature controlled crystal oscillator (TCXO) 83, a clock generator circuit 80, an RX circuit 76, a DAC buffer 194, a TX circuit 88 including a framer 208, and an audio circuit 81. The RX circuit 76 includes an RX packet buffer 186 coupled to an audio enhancer 188 including an audio enhancer output buffer 190. A digital-to-analog converter DAC 198 functions to input digital data from the DAC buffer 194 and generate an analog audio output signal 202. In the receiver RX 76, a frame synchronization circuit 78 derives clock timing from the received frame, which is used by the clock generator circuit 80 to synchronize all clocks in the device 72 to the base station 74.

[0091] The various delays that contribute to the overall latency (i.e., Δ T1 ~Δ T7 ) is shown in Figure 6. By keeping the audio system tightly locked to the RF clock, the system minimizes overall latency.

[0092] Both the device and the base station contain receivers and transmitters that aid in the clock recovery and locking process. The BS contains a master clock 106, the output of which is locked to the remaining clocks of the base station 74 (e.g., transmitter clock, DAC clock, receiver clock, console clock, etc.).

[0093] At the device 72 side, the receiver 76 may use a periodic radio time signal generated and transmitted by the base station 74 to generate and lock the receiver clock, the transmitter clock, and the ADC clock. An example of a periodic radio time signal is the frame sync signal generated by the frame sync circuit 78, transmitted as a downlink packet, and received at the RX 76.

[0094] Note that system 150 may have either an analog audio input 200 to ADC 164 or a digital audio input 201 to digital console interface 118, which may contain uncompressed audio signals and optionally a master clock for synchronization of the entire system.

[0095] Several key features of this system allow for a significant reduction in overall downlink latency, including (1) the use of a single master clock to which all other clocks are locked and derived, (2) the system is deterministic and does not involve schedule changes during showtime, and (3) the packet size used is an integer multiple of the size of the audio compressor output buffer and the audio expander input buffer.

[0096] The core latency of the downlink system 150 is determined by the duration Δ T1 and the duration Δ T7 , packet duration and other PHY related delays Δ T3Note that the audio compressor 174 includes a number of additional buffering mechanisms (e.g., filters, etc.) for the packet size. However, since the packet size is an integer multiple of the size of the compressor output buffer 178 (and the expander input buffer 186), there is no need for extra buffering. The output blocks generated by the audio compressor 174 are simply inserted into the TX packet buffer 178, and once the last block has been written, the packet is sent to the transmitter 110. Conversely, on the device 72 side, once the receiver 76 receives a complete packet, the audio expander 188 starts working on the first block therein. This scheme saves a significant round-trip latency, typically 0.5-1.0 ms (assuming a compressor buffer size of 0.25-0.5 ms). A summary of the various latencies in the system 150 of FIG. 6 is provided below.

number

[0097] A flow diagram illustrating an example of a clock synchronization method used in a base station is shown in Figure 11. Clock synchronization in the system is derived from a master clock source in the base station (step 370). A clock generator circuit uses the master clock (or an externally provided clock) to generate the various clocks used in the system, including the audio clock, all of which are synchronized to the master clock (step 372). The base station generates a frame containing the audio data and timing derived from the master clock (step 374), which is then transmitted via the WMAS to the wireless device (step 376).

[0098] A flow diagram illustrating an example of a clock synchronization method for use in wireless audio devices is shown in Figure 12. A clock source is provided to each wireless audio device (step 380). Frames transmitted from a base station are received at each device via the WMAS (step 382). Clock timing for the device is extracted from the received frame (step 384) using the techniques shown in Figures 4 and 5. The various clock signals required are then generated (step 386), including an audio clock synchronized to the clock timing generated by the frame synchronization circuitry.

[0099] A diagram illustrating timing for an exemplary WMAS system according to an embodiment of the present invention is shown in FIG. 7. The network in this exemplary embodiment comprises a base station and three microphones. The base station determines the PHY frames. These frames are recovered using a frame synchronizer in each microphone and are therefore substantially common to all members in the network. Each frame begins with a downlink transmission from the base station to a device that is used by the frame synchronizer in the microphone to lock onto the PHY frame structure. Following the downlink packet, each microphone transmits an uplink packet in a designated predefined time slot.

[0100] Each microphone executes its own audio block consisting of the time between the start of transmission of its respective UL packet and the start of transmission of the following UL packet. The audio frame duration is the same as the PHY frame duration, but time shifted.

[0101] During each audio block, each microphone processes the captured audio samples, compresses them and stores them in the TX buffer. To minimize latency, an audio block completes its cycle just before its designated TX slot. Thus, the audio blocks of the various microphones are time-shifted with respect to each other.

[0102] Similarly, in the case of an in-ear monitor (IEM), the base station generates multiple audio frames that correspond to the downlink transmission of each device.

[0103] A high level block diagram illustrating an exemplary uplink buffering and clocking scheme is shown in Figure 8. The system, generally referenced 210, comprises a base station 74 that communicates with one or more devices 72 over a radio link constituting a WMAS. The base station 74 comprises, among other things, a master clock 106, a clock generation circuit 108, an RF circuit 270, a TX circuit 110, an RX circuit 90, and an audio circuit block 114. The audio circuit 114 comprises a DAC 116 that generates an analog audio output 120, and a digital interface circuit 118 that may receive an optional external master clock 123, generate an optional output master clock 246 to the clock generator circuit 108, and generate a digital audio output 122. The TX circuit 110 comprises a framer 112 and a modulator 222 for generating RF samples that are output to the RF circuit 270 for transmission. The RX circuitry 90 includes a demodulator 234 and an audio enhancer 102 and receives RF samples from the RF circuitry 270 and generates audio samples that are output to the DAC 116 .

[0104] The device 72 includes an RF circuit 268, a TX circuit 88, an RX circuit 76, an audio circuit block 81, and a clock generation circuit 80. The audio circuit 81 includes an ADC 82. The TX circuit 88 includes a modulator 256 and an audio compressor 92. The RX circuit 76 includes a demodulator 262 and a frame synchronizer 78.

[0105] The ADC 82 functions to convert the analog audio input 84 to digital samples that are input to the TX circuitry 88. The RF samples output of the TX circuitry 88 are input to the RF circuitry 268 for transmission. On the receive side, the RF circuitry 268 outputs the received RF samples to the RX circuitry 76 where they are demodulated. The frame synchronizer 78 generates timing from the received frames and synchronizes the clock with the base station master clock 106. The derived timing is input to the clock generator circuitry 80 and used to generate various clocks within the device, including the audio clock.

[0106] The system shown in Figure 8 highlights a clocking scheme for the base station 74 and uplink device 72 in accordance with the present invention. A master clock 106 in the base station 74 is used to derive and synchronize digital clocks throughout the system 210. This clock may comprise a local oscillator (e.g., a TCXO, etc.) in the base station 74, or may optionally be provided from a digital interface coupled to a mixing console.

[0107] 8, the clock generator circuit 108 generates clocks including, for example, TX, RX, and audio clocks. The TX circuit 110 includes a framer 112 and a modulator 222, while the RX circuit 90 includes a demodulator 234 and an audio extender 102. The audio extender 102 outputs digital samples after the extender process to either a DAC 116 or a digital interface 118 in an audio system 114. The base station 74 also includes an RF circuit 270 that receives RF waves to convert the RF samples from the TX 110 to RF waves and outputs the RF samples to the RX 90.

[0108] The uplink device 72 (e.g., wireless microphone, IEM, etc.) shown on the left includes a receiver RX 76, a transmitter TX 88, an audio subsystem 81, and a clock generator module 80. Note that in one embodiment, the uplink device has bidirectional communication for management and synchronization purposes. The clock generator module 80 functions to generate clocks (e.g., PHY clock, audio clock, etc.) for the RX 76, TX 88, RF 268 circuitry, and audio system by locking and deriving a digital clock from the frame sync in the RX module 76. The RX 76 includes a demodulator 262 and a frame synchronizer 78 that locks to the frame rate and phase using techniques such as packet detection, correlators, PLLs, DLLs, FLLs, etc. The TX 88 includes a modulator 256 and an audio compressor 92, and the audio block 81 includes an ADC 82 that converts the input analog signal to digital audio samples. Additionally, the device 72 includes an RF subsystem 268 operable to convert RF samples from the TX 88 into RF waves and to receive the RF waves for outputting the RF samples to the RX 76 .

[0109] A high level block diagram illustrating an exemplary downlink buffering and clocking scheme is shown in FIG. 9. The system, generally referenced 280, comprises a base station 74 that communicates with one or more devices 72 over a radio link that constitutes a WMAS. The base station 74 comprises, among other things, a master clock 106, a clock generation circuit 108, an RF circuit 270, a TX circuit 110, an RX circuit 90, and an audio circuit block 114. The audio circuit 114 comprises an ADC 164 that converts an analog audio input 200 into digital audio samples, and a digital interface 118. The digital interface circuit 118 receives an optional digital audio input signal 201 from a mixing console and generates output audio samples to a clock generator circuit 108 and an optional master clock 246. The TX circuit 110 comprises a framer 112, an audio compressor 174, and a modulator 222 that receives the audio samples and generates RF samples that are output to the RF circuit 270 for transmission. The RX circuitry 90 includes a demodulator 234 that receives RF samples from the RF circuitry 270 and generates audio samples that are output to a DAC (not shown).

[0110] The device 72 includes an RF circuit 268, a TX circuit 88, an RX circuit 76, an audio circuit block 81, a local clock source (e.g., a TCXO) 83, and a clock generation circuit 80. The audio circuit 81 includes a DAC 198. The TX circuit 88 includes a modulator 256 and an audio compressor (not shown). The RX circuit includes a demodulator 262, an audio expander 188, and a frame synchronizer 78.

[0111] The RF samples output of the TX circuitry 88 are input to the RF circuitry 268 for transmission. On the receive side, the RF circuitry 268 outputs the received RF samples to the RX circuitry 76 where they are demodulated. The frame synchronizer 78 generates timing (frame sync signal) from the received frame to synchronize its own clock with the base station master clock 106, 246. The derived timing is input to the clock generation circuitry 80 and used to generate various clocks within the device 72 including the audio clock.

[0112] The system shown in Figure 9 highlights a clocking scheme for base stations and downlink devices (e.g., IEMs, etc.) in accordance with the present invention. A master clock 106, 246 in the base station 74 is used to derive and synchronize digital clocks throughout the system. The master clock may comprise a local oscillator (e.g., TCXO, etc.) in the base station 74, or may optionally be generated by the digital interface 118 from an input digital audio signal 201 from a mixing console.

[0113] In the base station 74, the clock generator circuit 108 generates clocks including, for example, TX, RX, RF, and audio clocks. The TX circuit 110 includes a framer 112, an audio compressor 174, and a modulator 222, while the RX circuit 90 includes a demodulator 234. An analog audio input 200 is converted to digital audio samples by an ADC. The base station 74 also includes an RF unit 270 that receives RF waves from the TX 110 to convert the RF samples to RF waves and outputs the RF samples to the RX 90.

[0114] The downlink device 72 (e.g., IEM, etc.) shown on the left includes RF circuitry 268, receiver RX 76, transmitter TX 88, audio subsystem 81, and clock generator module 80. Note that in one embodiment, the downlink device has bidirectional communication for management and synchronization purposes. The clock generator module 80 functions to generate clocks (e.g., PHY clock, audio clock, etc.) for the RX, TX, RF circuitry, and audio system by locking and deriving a digital clock from a frame sync module 78 in the RX module 76. The RX module 76 includes a demodulator 262 and a frame synchronizer 78 that locks to the frame rate and phase using techniques such as packet detection, correlators, PLL, DLL, FLL, etc.

[0115] The TX 88 includes a modulator 256. The audio circuitry 81 includes a DAC 198 that converts the audio samples output of the audio enhancer 188 to an analog audio output 202. Additionally, the device 72 includes an RF subsystem 268 that is operable to convert the RF samples from the TX 88 into RF waves and that receives the RF waves for outputting the RF samples to the RX 76.

[0116] A high level block diagram illustrating an exemplary frame synchronizer is shown in Figure 10. Note that the description provided herein assumes that there is at least one downlink packet per frame. Without loss of generality, it is assumed that the first packet in a frame is the downlink packet.

[0117] The exemplary frame synchronizer circuit, generally referenced 340, basically comprises an error detection circuit 342, a loop filter 360, a digitally controlled oscillator (DCO) implemented using a modulo-N counter 362, and a phase-locked loop (PLL) circuit including a comparator 364. The error detector 342 comprises a boundary detection / fine placement circuit 344, a sample and hold circuit 346, an error signal subtractor 350, a multiplexer 354, a sample and hold circuit 356, and a packet end detection 352.

[0118] In operation, the error detector 342 uses a PHY boundary detector and fine placement circuit 344 which functions to detect the exact position within the received packet which can vary based on the type of modulation used. The strobe output of this block functions to provide timing for a sample and hold block 346 which samples the output of the DCO (i.e. the output of modulo N counter 362) and thus holds the counter value at which the boundary detection / fine placement was obtained.

[0119] The target boundary value 348 is expressed as a number indicating the number of samples from the start of the packet to the ideal boundary detection point. This number is subtracted from the output of the sample and hold 346 via subtractor 350 to obtain the raw error expressed as a number of samples. This raw error is input to a multiplexer 354, whose output is determined by a "CRC check OK" signal 358 received from the PHY at the end of the packet. If the CRC check is valid, the raw error is output from the multiplexer, otherwise zeros are inserted (i.e. no correction is input to the loop filter). The multiplexer output is input to another sample and hold 356, which is triggered at the end of the packet 352 since the CRC OK signal is only valid at the end of the packet.

[0120] The error signal 357 is input to a loop filter 360, which may be realized by a bang-bang controller, a first order loop, a second order loop, a PID controller, etc. The loop filter outputs a positive number (i.e., advances or increments the counter), a negative number (i.e., retards or decrements the counter), or zero (i.e., NOP or no operation). Thus, the modulo-N counter advances, retards, or remains unchanged depending on the error output. The DCO modulo-N counter 362 increments by one each clock and is free running using the system local oscillator. A frame strobe is generated each time the counter is reset to zero. The output of the DCO is compared to zero and the output of comparator 364 generates the frame strobe for the rest of the system, which is used to derive various clocks within the device, e.g., audio clock, RF clock, PHY clock, etc.

[0121] Those skilled in the art will recognize that the boundaries between logic blocks and circuit blocks are merely exemplary, and that alternative embodiments may merge logic blocks or circuit elements, or provide alternative decomposed functions for various logic blocks or circuit elements. Thus, it should be understood that the architectures illustrated herein are merely exemplary, and that in fact many other architectures may be implemented which achieve the same functionality.

[0122] Any arrangement of components to achieve the same functionality is effectively "associated" such that the desired functionality is achieved. Thus, any two components of the present invention that are combined to achieve a particular functionality may be considered to be "associated" with one another such that the desired functionality is achieved, regardless of architecture or intervening components. Similarly, any two components that are so associated may be considered to be "operably connected" or "operably coupled" with one another to achieve the desired functionality.

[0123] Moreover, those skilled in the art will recognize that the boundaries between operations described above are merely exemplary, operations may be combined into a single operation, a single operation may be distributed into additional operations, operations may be performed with at least partial overlap in time, and alternative embodiments may include multiple instances of a particular operation, and the order of operations may be changed in various other embodiments.

[0124] The terms used herein are for the purpose of describing particular embodiments only and are not intended to be limiting of the present invention. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Furthermore, it is understood that the terms "comprises" and / or "comprising," as used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0125] In the claims, any reference signs placed in parentheses shall not be construed as limiting the scope of the claims. The use of prefaces such as "at least one" and "one or more" in the claims shall not be construed as implying that the preface of another claim element with the indefinite article "a" or "an" limits any particular claim containing the claim element so prefaced to an invention containing only one of such elements, even if the same claim also contains the preface "one or more" or "at least one" and an indefinite article such as "a" or "an". The same applies to the use of definite articles. Unless an exception is expressly stated, terms such as "first", "second", etc. are used to arbitrarily distinguish the elements they describe. Thus, these terms are not necessarily intended to indicate a chronological or other priority of such elements. The mere fact that certain measures are recited in different claims from one another does not indicate that a combination of these measures cannot be used to advantage.

[0126] Corresponding structures, materials, acts, and equivalents of all means or step-plus-function elements in the following claims are intended to include any structure, material, or act for performing a function in combination with other claimed elements as specifically claimed. The description of the present invention has been presented for purposes of illustration and description, but is not intended to be exhaustive or to limit the invention to the disclosed form. Since numerous modifications and changes will readily occur to those skilled in the art, the invention is not intended to be limited to the limited number of embodiments described herein. The embodiments are selected and described in order to best explain the principles and practical applications of the invention and to enable those skilled in the art to understand the invention in terms of various embodiments with various modifications as suitable for the particular use contemplated.

[0127] All optional and preferred features and modifications of the described embodiments and the dependent claims can be used in all aspects of the invention taught herein. Moreover, the individual features of the dependent claims, together with all optional and preferred features and modifications of the described embodiments, can be combined with each other and substituted for each other. Abbreviation AAC Advanced Audio Coding (AAC) is an audio coding standard for lossy digital audio compression. ADC Analog to Digital Converter BS base station CRC A Cyclic Redundancy Check (CRC) is an error-detecting code commonly used in digital networks and storage devices to detect accidental modifications to digital data. DAC Digital to Analog Converter DCO Digitally Controlled Oscillator DTV Digital Television DL Downlink DLL Delay Lock Loop FDM frequency division multiplexing FLL Frequency Lock Loop GFSK Gaussian Frequency Shift Keying IEM In-ear Monitor IMU Inertial Measurement Unit ISM The ISM radio bands are the part of the radio spectrum reserved internationally for Industrial, Scientific, and Medical (ISM) purposes, excluding telecommunications uses. LDAC LDAC is a proprietary audio coding technology developed by Sony that enables streaming of high-resolution audio over a Bluetooth connection. MP3 MP3 is a coding format for digital audio. MHz, GHz Megahertz, Gigahertz ms milliseconds OFDM Orthogonal Frequency Division Multiplexing (OFDM) is a multi-carrier modulation system in which data is transmitted as a combination of orthogonal narrowband signals known as subcarriers. The OSI Open Systems Interconnection Model (OSI model) is a conceptual model that provides a common basis for the coordination of [ISO] standards development for systems interconnection. In the OSI reference model, communication between computing systems is divided into seven abstract layers: physical, data link, network, transport, session, presentation, and application. OTA Radio PER Packet Error Rate PHY "Physical Layer", the electronic circuitry, usually implemented as an integrated circuit, required to implement the physical layer functions of the OSI model PLL Phase Locked Loop RX Receiver TDM time division multiplexing TCXO Temperature Controlled Crystal Oscillator TVB TX transmitter UHF Ultra-high frequency (UHF) is the ITU designation for radio frequencies in the range of 300 megahertz (MHz) to 3 gigahertz (GHz), also known as the decimeter band, because the wavelengths range from one meter to one-tenth of a meter (1 decimeter). UL Uplink VHF Very High Frequency, a range of radio frequency electromagnetic waves (radio waves) from 30 to 300 Megahertz (MHz) with corresponding wavelengths of 10 meters to 1 meter. Wi-fi 802.11ax Wi-Fi 6 (2.4 GHz and 5 GHz)[6] and Wi-Fi 6E (6 GHz), an IEEE standard for wireless local area networks (WLANs), a successor to 802.11ac. WMAS Wireless Multi-Channel Audio System

Claims

1. A wireless multi-channel audio system (WMAS) comprising a base station and a plurality of wireless audio devices, The base station A master clock source; a first clock generator circuit operative to generate a plurality of first clocks, including a first audio clock synchronized to the master clock source; a framer operative to generate frames containing audio data and timing derived from said master clock source; a base station transmitter operative to transmit the frame to the wireless audio device via the WMAS; Including, The wireless audio devices include a microphone system and an in-ear monitor, and each wireless audio device: a receiver operative to receive frames from the base station via the WMAS, the receiver of the in-ear monitor configured to receive frames containing audio data as downlink audio communications from the base station; a frame synchronization circuit operative to generate clock timing from the received frame using the detected PHY frame boundary timing; a second clock generator circuit operative to generate a plurality of second clocks locked to the clock timing generated by the frame synchronization circuit for synchronizing the wireless audio device to a base station frame; a wireless audio device transmitter, the microphone system transmitter configured to transmit synchronized frames containing audio data as uplink audio communications to the base station; Including, The system, wherein the downlink audio communications and the uplink audio communications are enabled according to a predetermined schedule.

2. The system described in claim 1, wherein the master clock source comprises a local oscillator within the base station or a clock signal from an audio mixing console to a digital interface within the base station.

3. The system described in claim 1, wherein the predetermined schedule includes uplink and downlink communications over the same frequency channel.

4. The system of claim 1, wherein during operation, (i) a clock in the WMAS synchronized to and derived from the master clock source within the base station using detected PHY frame boundary timing, and (ii) downlink and uplink audio communications according to the predetermined schedule, enables a reduction in round-trip latency to 6 milliseconds or less, the round-trip latency being the time interval between receiving an audio event at the microphone system that transmits audio data to the base station and loops it back at the base station, and outputting the audio data from the base station corresponding to the audio event, and between receiving the audio data at the in-ear monitor and outputting an audio signal at the in-ear monitor corresponding to the audio event.

5. The system described in claim 1, further comprising a synchronization circuit in the wireless audio device that operates to provide digital feedforward synchronization of an audio clock to frame synchronization clock timing.

6. The system described in claim 1, further comprising a synchronization circuit in the wireless audio device that operates to provide analog feedback synchronization of an audio clock to frame synchronization clock timing.

7. The system of claim 1, wherein the plurality of second clocks include at least one of an audio clock, an analog-to-digital converter (ADC) clock, a digital-to-analog converter (DAC) clock, a transmitter (TX) clock, a receiver (RX) clock, and a radio frequency (RF) clock.

8. The system of claim 1, wherein the frame synchronization circuit comprises a packet detector circuit, a correlator circuit, a phase-locked loop (PLL) circuit, a delay-locked loop (DLL) circuit, and / or a frequency-locked loop (FLL) circuit.

9. A method of clock synchronization for use in a wireless multi-channel audio system (WMAS) including a base station and a plurality of wireless audio devices, the plurality of wireless audio devices including a microphone system and in-ear monitors, the method comprising: In the base station, providing a master clock source; generating a plurality of first clocks including a first audio clock synchronized to the master clock source; generating frames containing audio data and timing derived from said master clock; transmitting the frame via the WMAS; In the wireless audio device, receiving frames from the base station via the WMAS, thereby configuring the in-ear monitor to receive frames containing audio data as downlink audio communications from the base station; generating clock timing from the received frame using the detected PHY frame boundary timing; generating a plurality of second clocks, including a second audio clock synchronized to the clock timing, by synchronizing the first clock in the base station and the second audio clock in the wireless audio device to the master clock source; configuring the microphone system to transmit synchronized frames containing audio data as uplink audio communications to the base station; enabling the downlink audio communications and the uplink audio communications according to a predetermined schedule; A method comprising:

10. The method described in claim 9, wherein the master clock source comprises a local oscillator within the base station or a clock signal from an audio mixing console to a digital interface within the base station.

11. The method of claim 9, wherein the predetermined schedule includes uplink and downlink communications over the same frequency channel.

12. Synchronizing all clocks in the WMAS from the master clock source within the base station, wherein (i) synchronizing the communications using detected PHY frame boundary timing, and (ii) downlink and uplink audio communications according to the predetermined schedule, enables a reduction in round-trip latency to 6 milliseconds or less, the round-trip latency being the time interval between receiving an audio event at the microphone system transmitting audio data to the base station and looping it back at the base station, and outputting the audio data from the base station to the in-ear monitor corresponding to the audio event, and between receiving the audio data at the in-ear monitor and outputting an audio signal at the in-ear monitor corresponding to the audio event. The method of claim 9 comprising:

13. The method of claim 9, further comprising synchronizing an audio clock to the clock timing in the wireless audio device using digital feedforward synchronization.

14. The method of claim 9, further comprising synchronizing an audio clock to the clock timing in the wireless audio device using analog feedback synchronization.

15. The method of claim 9, wherein the plurality of second clocks include at least one of an audio clock, an analog-to-digital converter (ADC) clock, a digital-to-analog converter (DAC) clock, a transmitter (TX) clock, a receiver (RX) clock, and a radio frequency (RF) clock.

16. The method of claim 1, wherein generating the clock timing from the received frame is performed using a packet detector circuit, a correlator circuit, a phase-locked loop (PLL) circuit, a delay-locked loop (DLL) circuit, and / or a frequency-locked loop (FLL) circuit.

17. A wireless audio device configured as a microphone system or in-ear monitor, the wireless audio device being for use within a wireless multi-channel audio system (WMAS), the wireless audio device comprising: a receiver operative to receive frames via the WMAS, the frames containing timing derived from a master clock source within the WMAS; a frame synchronization circuit operative to extract clock timing from received frames using detected PHY frame boundary timing, the in-ear monitor being configured to receive frames containing audio data as downlink audio communications; and a clock generator circuit operative to generate a plurality of clocks synchronized to the clock timing generated by the frame synchronization circuit for synchronizing the wireless audio device to the received frames; a wireless audio device transmitter, the microphone system transmitter configured to transmit synchronized frames containing audio data as uplink audio communications; Equipped with A wireless audio device, wherein the downlink audio communication and the uplink audio communication are enabled according to a predetermined schedule.

18. A wireless audio device as described in claim 17, wherein the predetermined schedule includes uplink and downlink communications over the same frequency channel.

19. The wireless audio device of claim 17, wherein during operation (i) a clock synchronized to and derived from the master clock source using detected PHY frame boundary timing, and (ii) downlink and uplink audio communications according to the predetermined schedule, enables a reduction in round-trip latency to 6 milliseconds or less, the round-trip latency being the time interval between receiving an audio event at the microphone system that transmits audio data to the base station and loops it back at the base station, outputting the audio data from the base station to the in-ear monitor corresponding to the audio event, and receiving the audio data at the in-ear monitor and outputting an audio signal at the in-ear monitor corresponding to the audio event.

Images