Communication between networked audio devices

Abstract

An audio device may be connected to a communication network. The audio device may transmit or receive audio data via a network based on a network clock that may be synchronized with other audio devices connected to the network. The audio device may buffer, convert, encrypt, decrypt, pack, unpack, compress and / or decompress audio data between digital and analog audio signals using a local asynchronous media clock that uses a relatively lower precision timing technique, such as a crystal-based oscillator.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. patent application serial number 18 / 223,766 filed on July 19, 2023, which claims priority to U.S. provisional patent application serial number 63 / 391,061 filed on July 21, 2022, the entire contents of each of which are incorporated herein by reference. Background Art

[0003] Some audio systems are designed to provide high-performance Internet Protocol (IP) audio capabilities. For example, the AES-67 standard provides a way for audio equipment manufacturers to interoperate in an IP audio solution that sends and receives professional-quality audio, such as uncompressed PCM 24-bit audio sampled at 48 kilohertz (KHz), with extremely low latency. Such professional quality and low latency may come at a price. Specifically, such systems involve relatively complex circuitry and / or software, a large number of expensive components, high network bandwidth utilization, and other costs. Summary of the invention

[0004] The following summary presents a simplified summary of certain features. This summary is not an exhaustive overview and is not intended to identify key or critical elements.

[0005] While the above costs for professional quality audio with very low latency may be worth incurring in some cases, for some market segments the required audio quality and latency requirements may be lower or less restrictive. Thus, the costs may be too high for the perceived benefits. One example may be the audio conferencing space. The audio requirements for an average voice-grade audio conference call may differ from the more stringent requirements of a major pop star's tour. Off-the-shelf solutions, such as standards-based Voice over IP (VoIP), may not provide the fidelity that users expect. These standards may typically be telephone-grade standards and follow IT or Internet standards. Furthermore, precise phase-accurate and sample-accurate audio transmission often requires the design of precision-timed hardware modules in the audio product. These types of hardware modules can sometimes be extremely difficult or expensive to source for a variety of reasons (e.g., the ongoing semiconductor shortage currently being experienced and / or supply chain or other economic conditions). It is desirable to provide an audio solution that can be between high-performance audio solutions and telephone-grade or IT-based solutions, and that does not incur the design and build costs that are not appropriate for a given required audio quality and latency level. Such solutions may not use such precision timing hardware modules at all, while still meeting the needs of various audio contexts such as teleconferencing, video conferencing, public address systems, etc. For example, such solutions may selectively and carefully relax one or more bandwidth, quality, and / or latency requirements required by existing high-performance IP audio solutions. Such solutions may continue to provide best-in-class audio performance, while achieving this at a potentially much lower implementation, complexity, and / or usage cost.

[0006] For example, in some high-performance systems, the network clock is tightly synchronized with the global network clock, and other internal clocks are derived from the tightly synchronized network clock. This is usually achieved using a complex and expensive precise hardware timing module. Such highly synchronized timing mechanisms are used to control analog-to-digital conversion and digital-to-analog conversion processes, as well as the packaging and unpacking of packets. This helps the system to have high performance in terms of latency. However, as an alternative, one or more local clocks can simply be allowed to run in an asynchronous manner, as will be described herein. For example, a local clock (such as a local media clock) can be asynchronous with all other clocks (e.g., can be generated independently), and can be used to drive one or more processes, such as analog-to-digital audio conversion, digital-to-analog audio conversion, packaging and / or unpacking. Using one or more such local asynchronous clocks can achieve significantly simpler, cheaper audio equipment, while still being able to achieve audio quality and latency expectations suitable for certain types of audio applications.

[0007] For example, some aspects described herein may relate to an audio system in which a plurality of audio devices can communicate with each other. For example, a first audio device may transmit data (e.g., audio and / or other information) to a second audio device and / or receive the data from the second audio device, and the second audio device may transmit data (e.g., audio and / or other information) to the first audio device and / or receive the data from the first audio device. These audio devices may be connected to each other via a communication medium, which may involve a direct connection between audio devices, an indirect connection between audio devices, and / or a communication network. For example, the connection between these audio devices may be based on IP. For example, the one or more audio devices may transmit data to another audio device in the audio devices in a plurality of packets (such as IP packets) via the communication medium and / or receive data from the other audio device. One or more audio devices in the audio devices may operate according to a plurality of clocks. For example, the transmission and / or reception of packets between the audio devices may be performed according to a first clock (e.g., transmitted and / or received based on the frequency and / or phase of the first clock). For example, the first clock may be based on a master clock shared by the audio device such as via the communication medium. The one or more audio devices may further convert analog signals (such as analog audio signals) into digital data according to a second clock, and / or convert received digital data (such as digital audio or other data) into analog signals (e.g., transmitted and / or received based on the frequency and / or phase of the second clock). The second clock may be asynchronous with the first clock (e.g., generated independently of the first clock). The one or more audio devices may further packetize digital data to be transmitted, and / or unpack digital data received according to the first clock or the second clock (e.g., transmitted and / or received based on the frequency and / or phase of the first clock or the second clock).

[0008] According to other aspects described herein, a method may be performed by an audio device. The method may include: receiving an analog audio signal based on a detected sound; and generating a local asynchronous media clock using, for example, a local oscillator (such as a crystal-based oscillator, a microelectromechanical system oscillator (MEMS), a ceramic resonator, a surface acoustic wave (SAW) oscillator, an inductor / capacitor (LC) oscillator, or another type of asynchronous timing implementation). The local asynchronous media clock may not need to have high accuracy. For example, the frequency variation of the local asynchronous media clock may be at least one part per million, or at least ten parts per million, or at least one hundred parts per million. For example, the frequency variation of an off-the-shelf crystal-based oscillator at room temperature (e.g., at about 20 degrees Celsius) is typically in the range of ten parts per million to one hundred parts per million. As discussed below, the local asynchronous media clock does not need to be extremely accurate, which may be expected to effectively translate into "expensive" and / or "complex". For example, an extremely accurate chip atomic clock is not required to implement the local asynchronous media clock. Instead, cheaper, simpler and / or more readily available technologies can be used to implement the local asynchronous media clock for sending and receiving audio. The method may also include: using the local asynchronous media clock to generate digital audio data based on the analog audio signal. A network clock may be generated using a master clock of a network connected to the audio device. For example, the network clock may be synchronized with the master clock. The audio device may transmit the digital audio data based on the network clock via the network.

[0009] According to other aspects described herein, a method may be performed by an audio device. The method may include: receiving digital audio data via a network based on a network clock synchronized with a master clock of the network. The method may also include: generating a local asynchronous media clock using, for example, a local oscillator (such as a crystal-based oscillator, MEMS, ceramic resonator, SAW oscillator, LC oscillator, or another type of asynchronous timing implementation). As discussed above and further discussed herein, the local asynchronous media clock may have a certain accuracy so that the frequency variation of the local asynchronous media clock may be at least one millionth, or at least ten millionths, or at least one hundred millionths. The method may also include: using the local asynchronous media clock to generate an analog audio signal based on the digital audio data. The method may also include: generating sound based on the analog audio signal, such as by using a speaker.

[0010] Other aspects described herein relate to audio devices that perform the above-described methods and other methods, systems that include two or more of these audio devices, and computer-executable instructions (e.g., software and / or firmware) that, when executed, cause an audio device to perform the above-described methods and other methods.

[0011] These and other features and potential advantages are described in more detail below. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Certain features are illustrated by way of example and not limitation in the accompanying drawings.In the drawings, like numerals represent similar elements.

[0013] Figure 1 is a block diagram of an example audio system.

[0014] Figure 2 is a block diagram showing example details of an audio device, which may be an audio system such as Figure 1 part of an audio system.

[0015] Figure 3 is a block diagram showing another example of details of an audio device, which may be an audio system such as Figure 1 part of an audio system.

[0016] Figure 4 is an example audio system (such as Figure 1 FIG. 1 is a block diagram of an audio system including example details of two audio devices in the audio system.

[0017] Figure 5 is a block diagram showing example details of an audio device, which may be an audio system such as Figure 1 part of an audio system. DETAILED DESCRIPTION

[0018] The accompanying drawings, which form a part of the description herein, illustrate examples of the present disclosure. It should be understood that the examples shown in the accompanying drawings and / or discussed herein are non-exclusive and that there are other examples of how the present disclosure may be practiced.

[0019] Figure 11 is a block diagram of an example audio system 100. The audio system 100 may include a plurality of audio devices, such as an audio device 101 and an audio device 102. The plurality of audio devices may be communicatively coupled to each other via a communication medium, such as a communication network 103. An audio device may be any type of device capable of transmitting, receiving and / or processing audio (e.g., modifying, storing and / or operating in response to audio). Non-limiting examples of audio devices include certain devices that act as or include microphones, speakers, conference equipment, recorders, personal computers, servers, display devices (e.g., televisions or computer displays), network devices, audio mixers, and musical instruments. Thus, for example, the audio device 101 may be or otherwise include a microphone, and the audio device 102 may be or otherwise include a speaker. Audio data generated based on the sound detected by the microphone may be transmitted by the audio device 101 to at least the audio device 102 via the communication network 103. The audio device 102 may accordingly cause its speaker to generate sound based on the received audio data. This is just one example—as another example, each of audio devices 101 and 102 may include both a microphone and a speaker. As yet another example, audio device 101 may include a microphone, and audio device 102 may include a computing device configured to store audio data received from audio device 101. As yet another example, audio devices 101 and 102 may each be an element of a teleconferencing or videoconferencing system. As yet another example, audio devices 101 and 102 may each be an element of a public broadcasting system. Although Figure 1 Two audio devices are shown in FIG. 1 , but this is merely an example, and the system 100 may include any plurality of audio devices interconnected via the communication network 103 , such as three audio devices, four audio devices, or more audio devices.

[0020] The communication network 103 can be any type of network (including simple connections between audio devices) using any one or more protocols. For example, the communication network 103 can utilize the Internet Protocol (IP) to carry data (such as audio data) in IP datagrams. The communication network 103 can use a specific data link layer protocol (such as Ethernet) to transmit these IP datagrams. This combination of IP and Ethernet is called IP over Ethernet (IPoE), where data (such as audio data) is placed in IP datagrams, and these IP datagrams are encapsulated in Ethernet frames. The term "packet" will be used herein to include various organized groups of data, such as, but not limited to, datagrams (e.g., User Datagram Protocol (UDP) datagrams) and frames.

[0021] Each audio device in these audio devices can be configured to transmit data to one or more other audio devices via communication network 103. Each audio device in these audio devices can be further configured to receive data from one or more other audio devices via network 103. Any audio device in these audio devices can be configured to transmit and receive data, or transmit data specifically, or receive data specifically. For example, audio device 101 can be configured to transmit data to audio device 102 via network 103 and / or receive data from this audio device, and audio device 102 can be configured to transmit data to audio device 101 via network 103 and / or receive data from this audio device. The data transmitted between audio devices can include audio data, video data, communication control data, system control data, audio processing parameter data and / or the data of any other type.

[0022] Figure 2 is a block diagram showing example details of an audio device, which may be an audio system such as Figure 1 For example, Figure 2 The audio device shown in the figure can be audio device 101 or audio device 102. In this example, the audio device can include or otherwise be connected to a media source 201. The media source 201 can be inside the housing of the audio device or outside the housing, and the media source can be any type of media source, such as a microphone, a musical instrument, a storage device containing pre-recorded audio, a speaker phone, a telephone, or any other device capable of generating or providing an audio signal (such as an analog audio signal provided to the ADC 202).

[0023] The media source 201 may generate an audio signal representing audio, which may be an analog audio signal. The analog audio signal may be transmitted to an analog-to-digital converter (ADC) 202. The ADC 202 may convert the analog audio signal into a digital audio signal, which may be transmitted to a transmitter buffer 204. The ADC 202 may operate (e.g., be managed by) a local clock, which is asynchronous with any other clock used by the audio device. For example, the ADC 202 may sample the analog audio signal at a sampling rate based on (e.g., equal to) the clock rate of the asynchronous local clock. The asynchronous clock will be referred to herein as a local asynchronous media clock 203. The local asynchronous media clock 203 may be a clock having a specific nominal frequency, such as a nominal frequency of about 32 kHz or a nominal frequency of about 48 kHz, or any other nominal frequency as required. The ADC 202 may generate digital data based on the analog audio signal at the frequency of the local asynchronous media clock 203. For example, if local asynchronous media clock 203 is a clock having a nominal frequency of F hertz (Hz), ADC 202 may sample the analog audio signal at a nominal sampling rate of F Hz, thereby generating data (e.g., one byte of data) for each sample. Thus, ADC 202 may generate a digital audio signal at the frequency of local asynchronous media clock 203 by generating a nominal F bytes of data (or some other amount of data) per second.

[0024] The term "nominal" is used in the above discussion because the local asynchronous media clock 203 does not need to be an extremely accurate clock and does not need to be synchronized with any other accurate clock (or in fact does not need to be synchronized with any other clock at all, regardless of accuracy). For example, the local asynchronous media clock 203 can be implemented as or otherwise include a local oscillator, such as a crystal-based oscillator, for example a piezoelectric crystal and / or circuits for operating the crystal. As other examples, the crystal-based oscillator can be or otherwise include a temperature compensated crystal oscillator (TCXO) or a voltage-controlled crystal oscillator (VCXO). When a voltage is applied to the crystal-based oscillator, the crystal can oscillate at a specific frequency F. The frequency F can be the frequency of the local asynchronous media clock 203, which can be represented as a time-varying voltage signal. The frequency F can be fixed (e.g., by using a TCXO or by using a VCXO with a fixed voltage input to have a nominal stable frequency), or the frequency F can be adjustable, such as by implementing the local asynchronous media clock 203 as a VCXO with an adjustable voltage input. As other examples, local asynchronous media clock 203 may be implemented as a microelectromechanical system oscillator (MEMS), a ceramic resonator, a surface acoustic wave (SAW) oscillator, an inductor / capacitor (LC) oscillator, or another type of asynchronous timing implementation.

[0025] Regardless of the specific implementation, the local asynchronous media clock 203 can have a frequency and phase (offset) that is asynchronous (independent of any other clock) with any other clock used by the audio device and / or by the audio system 100. There are many potential advantages to using a crystal-based oscillator (or, for example, a MEMS, ceramic, SAW or LC oscillator) to implement the local asynchronous media clock 203. When compared with certain other high-precision clock generation devices (such as chip atomic clocks or timing chips synchronized with external precision timing sources), such oscillators can provide clocks with lower precision and / or lower accuracy. Although such oscillators generally have a stable nominal frequency, the use of such timing techniques may cause the frequency of the local asynchronous media clock 203 to vary by at least one part per million, or at least ten parts per million, or at least one hundred parts per million. Their relative changes in clock frequency and phase may be a factor in designing the size of the receiver buffer and / or transmitter buffer to reduce the possibility of buffer overflow. However, these trade-offs may be worthwhile in certain audio applications that do not require extremely high audio quality and extremely high audio synchronization, such as, but not limited to, teleconferencing, video conferencing, public address systems, etc. When compared with such precision clock generating devices, these types of low-precision oscillators are relatively cheap, require less circuit board space, have lower design and manufacturing complexity, and have lower power consumption. Although all of these factors may be advantageous for any audio device, some factors may be particularly advantageous for portable and / or battery-powered and therefore space and / or power-constrained audio devices. In addition, as will be discussed below, using an oscillator with lower design or expected accuracy can provide simplicity, because the exact frequency of the local asynchronous media clock 203 may not be very important for realizing the intended purpose of the audio device and / or audio system. This simplicity and flexibility of implementing an asynchronous local media clock in each audio device 101 and / or 102 can allow the audio device to not include, for example, a phase-locked loop (PLL) that may otherwise be used to synchronize local media clocks with each other. In a system where multiple devices are communicating (such as a first device 101 transmitting audio data, and a second device 102 receiving the transmitted audio data), the first device and the second device may have their own local asynchronous media clocks, which are not necessarily synchronized with each other in terms of their nature. Regardless of the type of clock used to implement the local asynchronous media clock, the first device and the second device can still be able to use the techniques described herein to efficiently transmit and receive audio data while managing the receiving buffer to potentially avoid receiving buffer underruns and overruns.

[0026] The digital audio signal generated by the ADC 202 can be received by a transmitter buffer 204, which can temporarily store the audio data of the digital audio signal. The transmitter buffer 204 can be any type of buffer, such as a first-in-first-out (FIFO) buffer. The transmitter buffer 204 can output its stored audio data in multiple parts by packaging each part of the stored audio data into packets (such as IP datagrams); and transmitting each packet as a packetized digital audio to the network stack and controller 205. The transmitter buffer can compress, encrypt and / or package the buffered digital audio data, and transmit the compressed, encrypted and / or packaged digital audio data to the network stack controller 205 at a rate managed by the local asynchronous media clock 203 or another clock (which will be referred to as the network clock 206 in this article). Any compression scheme and / or encryption scheme can be used to compress and / or encrypt digital audio data. For example, the digital audio data can be encrypted using AES-128 counter mode encryption or AES-256 counter mode encryption. Consistent with the above discussion, the local asynchronous media clock 203 can be completely independent of the network clock 206 (asynchronous with the network clock). In other words, the frequency and phase of the local asynchronous media clock 203 can be completely independent of the frequency and phase of the network clock 206, and regardless of the frequency and phase of the network clock, the local asynchronous media clock operates with its own frequency and phase. In addition, since the local asynchronous media clock 203 of each audio device 101 or 102 has a relatively low accuracy, it can be expected that the local asynchronous media clock 203 of the audio device 101 can have a nominal frequency different from the local asynchronous media clock 203 of the audio device 102. The positive and negative magnitudes of the differences between these nominal frequencies may be at least one part per million, or at least ten parts per million, or at least one hundred parts per million. In addition, even if their corresponding nominal frequencies are the same, their actual frequencies over time may also be expected to be different from each other, wherein their instantaneous or average frequencies at each given time may be expected to differ from each other by at least plus or minus one part per million, or at least plus or minus ten parts per million, or at least plus or minus one hundred parts per million.

[0027] The network stack and controller 205 may act as an interface between the audio device and the network 103. Thus, audio data packets (e.g., IP datagrams) received by the network stack and controller 205 from the transmitter buffer 204 may be reformatted for the network 103 and transmitted via the network 103. For example, the network stack and controller 205 may reformat the audio data IP datagrams by at least encapsulating the IP datagrams in Ethernet frames. The network stack and controller 205 may transmit any packets it receives from the transmitter buffer 204 to the network 103. The network stack and controller 205 may further mark the packets (e.g., IP datagrams and / or Ethernet frames) with sequence numbers and / or timestamps based on a network clock 206, wherein the network clock 206 may be generated based on (e.g., generated synchronously with) a master clock associated with the network 103. For example, the network stack and controller 205 may generate the network clock 206 based on the master network clock according to the Precision Time Protocol (PTP) as specified in IEEE 1588-2008 (PTP Version 2) or IEEE 1588-2019. For example, according to PTP, the master clock of the network 103 may serve as a synchronization reference (e.g., a grand master clock), and the network clock 206 may be generated to have a frequency and phase that is based on (e.g., equal to) the frequency and phase of the master clock. Thus, the network clock 206 may be synchronized with the master clock or otherwise generated based on the master clock.

[0028] Consistent with the above discussion, the local asynchronous media clock 203 for any given audio device in the audio system 100 can be completely independent of (asynchronous with) the master clock of the network 103. In other words, the frequency and phase of the local asynchronous media clock 203 can be completely independent of the frequency and phase of the master clock, and the local asynchronous media clock operates at its own frequency and phase regardless of the frequency and phase of the master clock. In addition, the network clock 206 can be generated by the audio device based on the master clock of the network 103 (e.g., synchronized with the master clock of the network in terms of frequency and phase).

[0029] As mentioned above, the packets to be transmitted to the network 103 may be marked with a timestamp and / or a sequence number by, for example, the network stack and the controller 205. The timestamp may be, for example, a PTP timestamp. The timestamp may be generated based on the network clock 206 or the master clock of the network 103, and may be generated according to RFC 7273. For example, each timestamp may have a value based on the value of the network clock 206 associated with the packet, such as the value of the network clock 206 when the packet is generated or transmitted. The network clock 206 does not need to have hardware accurate timestamp capability, and may be implemented, for example, in software to save complexity and external components. For each packet to be transmitted, the sequence number may be incremented (such as incremented by a value of 1). At the receiving end (e.g., a packet generated by the audio device 101 received by the audio device 102), various incoming packets may be organized using timestamps and / or sequence numbers, even if the local asynchronous media clock 203 of each audio device remains free running in an asynchronous manner. Such a hybrid configuration that uses synchronous timing for certain interactions between the audio devices 101 and 102 while using asynchronous timing for certain internal processing within each of the audio devices 101 and 102 can save complexity and can make the audio device hardware and software simpler. When the other audio device 101 or 102 receives packets via the network 103, the timestamps and / or sequence numbers in each received packet can be read and used to determine the order in which the packets should be received, buffered, and / or processed, even in the presence of variable network 103 delays.

[0030] In addition to transmitting audio data to the network 103, or as an alternative to transmitting audio data to the network, the audio device 101 or 102 may receive audio data from the network 103. Figure 2In the example shown in , the network stack and controller 205 can receive network packets containing audio data (e.g., Ethernet frames encapsulating audio data packets (such as IP datagrams)), and can transmit the audio data packets to the receiver buffer 207, which can unpack (e.g., extract audio data from the data packets), decompress as needed, decrypt as needed, and temporarily store the extracted, decompressed and / or decrypted audio data. The receiver buffer 207 can be any type of buffer (such as a FIFO buffer) and can be combined with the transmitter buffer 204 or implemented as a completely independent buffer. The receiver buffer 207 can unpack the audio data packets (e.g., IP datagrams) received from the network stack and controller 205 in the following manner: extract the audio data from the audio data packets; and store the audio data in the receiver buffer 207. The unpacking can be performed according to the network clock 206 (e.g., managed by the network clock) or by the local asynchronous media clock 203. As discussed above, the audio data packets can include timestamps and / or sequence numbers, which can be used by the receiving audio device to determine the correct order for processing the audio packets. For example, the audio data extracted from a given audio data packet may be stored in the receiver buffer 207 at a location corresponding to (e.g., indexed by or otherwise associated with) the timestamp and / or sequence number of the audio data packet, and / or may be stored in an order corresponding to the timestamp and / or sequence number of the audio data packet in the receiver buffer. In this way, the receiver buffer 207 may ensure that the contents of the various packets are processed (e.g., converted to analog audio signals) in the correct order, even if a later transmitted packet is received before an earlier transmitted packet due to network 103 delays. In other examples, the receiver buffer 207 may store the audio data in the order in which the audio data was received, such as storing the audio data in a first-in, first-out (FIFO) arrangement.

[0031] The receiver buffer 207 may communicate a portion of its stored audio data to a digital-to-analog converter (DAC) 208, which may convert the digital audio data into an analog audio signal. The DAC 208 may operate in accordance with (e.g., its operation may be governed by) the local asynchronous media clock 203. For example, the DAC 208 may receive (e.g., pull or extract) digital data stored in the receiver buffer 207 at a rate based on the frequency of the local asynchronous media clock 203. For example, if the local asynchronous media clock 203 is a clock having a frequency of F Hz, the DAC 208 may receive digital audio data from the receiver buffer 207 and convert the digital audio data into an analog audio signal at the frequency of the local asynchronous media clock 203 by converting F bytes of data per second into an analog audio signal. Although the local asynchronous media clock 203 may be clocked at a frequency of F Hz, the DAC 208 may be clocked at a frequency of F Hz. Figure 2 208, but the local asynchronous media clock signal can be provided to any one or more components of the audio device 101 or 102 as needed. In addition, although the local asynchronous media clock 203 is provided in Figure 2 Although shown in the figure as connected to other elements in a particular manner and as the only element in the figure, the local asynchronous media clock 203 can be located anywhere inside or outside the transmit chain (which includes at least elements 201, 202, 204, and 205), anywhere inside or outside the receive chain (which includes at least elements 205, 207, 208, and 209), and / or as a Figure 2 a part of any other element among the other elements.

[0032] DAC 208 can transmit the generated analog audio signal to a media receiver 209. Media receiver 209 can be inside a housing of the audio device or outside of the housing, and can be any type of media receiver, such as a speaker, an audio storage device, a speaker phone, a telephone, or any other device capable of receiving and / or processing an audio signal (such as the analog audio signal generated by DAC 208). Media receiver 209 can be a separate device from media source 201, or the two devices can be integrated into a single identical device. For example, a speaker phone can include both a media source (e.g., its microphone and associated circuitry) and a media receiver (e.g., its speaker and associated circuitry). In some examples, media source 201 and media receiver 209 can be co-packaged as the same device as the rest of the circuitry of audio device 101 or 102. For example, a single housing can enclose or at least partially enclose the media source 201. Figure 2209. In other examples, the media source 201 and / or the media receiver 209 may be physically separate from a device containing any of the remaining elements 202 to 208 while being communicatively connected to the device. In such examples, analog audio signals from the media source 201 to the ADC 202 and / or analog audio signals from the DAC 208 to the media receiver 209 may be communicated via external ports and / or cables. In addition, the audio device 101 or 102 may only include Figure 2 For example, audio device 101 or 102 may be configured to transmit audio to network 103 without receiving audio from network 103, or audio device 101 or 102 may be configured to receive audio from network 103 without transmitting audio to network 103. In these examples, audio device 101 or 102 may include at least elements 202 to 206 (and possibly 201) but not elements 207 to 209, or audio device 101 or 102 may include at least elements 203 and 205 to 208 (and possibly 209) but not elements 201, 202, and 204.

[0033] As previously mentioned, when the audio device receives packets via the network 103, the timestamps and / or sequence numbers in each received packet can be read and used to determine the order in which each packet should be read, buffered, and / or processed relative to other received packets, even in the presence of variable network 103 delays. In addition, the timestamps and / or sequence numbers can also be used to detect dropped packets. For example, assume that the audio device 101 transmits two packets to the audio device 102, wherein the first of the packets has a first timestamp and / or a first sequence number and is transmitted by the audio device 101 before transmitting a second of the packets having a second timestamp and / or a second sequence number. Even if the audio device 102 receives the second packet before receiving the first packet (e.g., due to variable latency in the network 103), the audio device 102 will be able to correctly reorder the incoming first and second packets in the audio device's receiver buffer 207 based on the respective timestamps and / or sequence numbers of the incoming first and second packets, such that the audio device 102 can buffer the first packet (e.g., in the receiver buffer 207) before the second packet so that the first packet can be processed (e.g., converted from digital to analog using the DAC 208) before the second packet. To achieve this, the recipient audio device 102 can store the audio data of each of the packets in the receiver buffer 207 in an order and / or storage location based on the respective timestamps and / or sequence numbers of the packets. In addition, the buffered audio data can be retrieved from the receiver buffer 207 and transmitted to the DAC 208 in an order based on the respective storage locations of the buffered audio data within the receiver buffer 207. For example, if the recipient audio device 102 determines that the second sequence number differs from the first sequence number by more than one sequence number (e.g., one or more skipped sequence number values ​​between the first sequence number value and the second sequence number value), the recipient audio device 102 may determine that one or more packets have been discarded (e.g., lost in the route to the recipient audio device 102 or otherwise not received by the recipient audio device 102). The recipient audio device 102 may determine the amount of discarded packets based on the number of lost sequence numbers in the received packets. For example, if the received packets include sequence numbers [1, 2, 6, 7, 8, 9, …], the recipient audio device 102 may determine that packets with sequence numbers 3, 4, and 5 are lost, and therefore determine that these three packets have been discarded. The recipient audio device 102 may further determine that one or more packets are discarded based on a time period that has passed after which packets containing one or more sequence numbers are not received as expected.For example, if there is a third sequence number value between the first sequence number value and the second sequence number value, and after a threshold time period, no packet containing the third sequence number value is received from a packet containing the first sequence number or the second sequence number, then the receiving audio device 102 can determine that the packet containing the expected third sequence number has been discarded. If the audio device 102 determines that the packet has been discarded, the audio device 102 can fill its receiving buffer with a manufactured set of data at the location where the data from the discarded packet should have been stored, or can perform some other action, such as generating a signal indicating the discarded packet, for example, where the signal can be used to indicate the discarded packet status to a user of the audio device 102.

[0034] As another example, the recipient audio device 102 may use the timestamp to determine whether one or more packets have been dropped. The recipient audio device 102 may be configured with an expected time between packets, or may be configured with an expected packet send rate from which the expected time between packets may be derived (based on the inverse of the expected packet send rate). The expected packet send time between packets may be predetermined, or may be determined dynamically by the recipient audio device 102, such as by measuring the packet rate and / or the time between packets and averaging these values ​​over a sliding time window. However, to determine the expected time between packets, the recipient audio device 102 may store a value for the expected time between packets, referred to herein as T EP If the receiving audio device 102 receives a message in which the time interval indicated by the timestamp is approximately T EP If the receiving audio device 102 determines that the interval between the two timestamps is greater than T EP If the receiving audio device 102 receives a packet having a timestamp between the two timestamps at a time T, the receiving audio device 102 may determine that at least one packet has been discarded. In addition, the receiving audio device 102 may determine the number of the one or more discarded packets as T. EP In other words, the number of discarded packets between the two received packets can be determined to be equal to T EP / T, which can round up or down as needed.

[0035] In order to increase the confidence of the determination of the discarded packets, the receiving audio device 102 can use both the packet sequence number and the timestamp as discussed above to determine one or more discarded packets. For example, if the packet sequence number is used to indicate the determination of one or more discarded audio packets, and the timestamp is used to indicate the determination of one or more discarded audio packets, then based on the fact that both indicate one or more discarded packets, the receiving audio device 102 can determine that one or more packets have been discarded. For another example, if the packet sequence number is used to indicate the determination of a specific amount of one or more discarded audio packets (e.g., three packets), and the timestamp is used to indicate the determination of the same specific amount of one or more discarded audio packets (e.g., three packets), then based on the fact that both indicate the same amount of one or more discarded packets, the receiving audio device 102 can determine that a specific amount of packets (e.g., three packets) in one or more packets have been discarded. If only one of the two determinations (e.g., using a sequence number without a timestamp, or using a timestamp without a sequence number) indicates one or more discarded packets, the receiving audio device 102 may not determine that one or more packets have been discarded. If both determinations (using sequence numbers and using timestamps) indicate that packets have been discarded, but the numbers of discarded packets determined using the two methods are different, the recipient audio device 102 can determine the number of discarded packets based on one of the methods, such as using a smaller number or a larger number of discarded packets as desired. For example, for a given time period between two received packets, if the recipient audio device 102 uses sequence numbers to determine that two packets between those packets have been discarded, and if the recipient audio device 102 also uses timestamps to determine that three packets between those packets have been discarded, the recipient audio device 102 can determine that a smaller amount (two packets) or a larger amount (three packets) have been discarded as desired.

[0036] In addition to determining the order in which the data in the received packets is buffered and / or converted to analog signals, the receiving audio device 102 can also use the received timestamps to determine the latency of the network 103 for each received packet. This is because of the way the timestamps are generated. For example, each timestamp can be generated based on the value of the network clock 206 known by the network stack and controller 205 of the transmitting audio device 101, where the network clock 206 of the transmitting audio device 101 can be synchronized with the master clock of the network 103. The receiving audio device 102 can have its own network clock 206, which is also synchronized with the master clock of the network 103. Therefore, the receiving audio device 102 can determine the latency of the received packet (e.g., the time that has passed since the transmitting audio device 102 transmitted the packet) based on the timestamp of the received packet and the network clock 206 generated by itself. For example, the receiving audio device 102 can compare the timestamp of the received packet with the value of its network clock 206 (or with a value derived from the value of the network clock 206) when the packet is received. The receiving audio device 102 may perform an action based on the determined latency. For example, if the latency becomes very large (e.g., if the latency is determined to exceed a known threshold), the receiving audio device 102 may take a first action, such as adding filler data (e.g., zero or interpolated audio data) to the audio data stored in the receiver buffer 207, or the receiving audio device 102 may transmit a signal to the transmitting audio device 101 to indicate to the transmitting audio device 101 that the receiving audio device 102 is experiencing high latency, or the audio device 102 may present a message to the user (e.g., via a display or other user interface) to indicate that the audio device 102 is experiencing high latency. Similarly, if the latency is determined to be below a threshold, the receiving audio device 102 may take a second action, such as modifying the data in the receiver buffer 207, for example, by removing (e.g., deleting, ignoring, or overwriting) a subset of the data from the buffer 207. The receiving audio device 102 can also use the timestamps to determine the overall system audio latency, and can adjust the amount of audio stored in its buffer 207 (and / or use sampling rate conversion) to achieve the target latency. For example, the receiving audio device 102 can be configured with a set point for how full the buffer 207 should be. The receiving audio device 102 can determine latency based on the received timestamps by comparing the timestamps in the received packets with its own clock or with a network clock. The receiving audio device 10 can respond to the determined latency by discarding data from the buffer 207 or otherwise time-correcting the data in the buffer and / or storing the generated, time-corrected and / or interpolated data in the buffer 207 to maintain the buffer 207 at a level close to or at the set point.By maintaining the fullness of the buffer 207 at or near a set point, the recipient audio device 102 can provide audio with approximately the same latency regardless of the source of the audio or the network path of the audio. For example, if the recipient audio device 102 receives audio from two sender audio devices, and the audio from one of the sender audio devices is delayed (experiences a latency) greater than the other of the sender audio devices, one or more portions of the audio data of the sender audio device with a smaller latency (less delay) may be foregone or otherwise time-corrected in the buffer 207, and / or one or more portions of the audio data may be produced, time-corrected, and / or interpolated by the recipient audio device 102 to fill in latency gaps that would otherwise appear in the audio data stored in the buffer 207. The recipient audio device 102 may also select a target latency, such as a target latency based on a known or expected network latency, such as a target latency that is slightly greater than the known or expected network latency.

[0037] The receiving audio device 102 may also determine the rate (e.g., sampling rate or packet rate) at which data is received by the audio device 102, and the receiving audio device 102 may take one or more actions based on the determined data rate. For example, the receiving audio device 102 may measure the rate at which packets are received at the network stack and controller 205 over time, and / or the receiving audio device 102 may measure the rate of audio samples received by or stored in the receiver buffer 207 over time. The receiving audio device 102 may compare the measured rate with a threshold rate, and may take one or more actions based on the comparison. For example, the receiving audio device 102 may compare the measured packet rate with a threshold packet rate, or compare the measured sampling rate with a threshold sampling rate, or compare any other measurement of the incoming data rate with a threshold data rate. The threshold sampling rate may be any rate, such as 96 samples per two milliseconds (or 48 samples per millisecond), which may be equivalent to an expected 48kHz audio rate. If the audio rate is expected to be another rate, the threshold may be a different value. For example, if the expected audio rate is 32kHz, the threshold sampling rate can be 32kHz (e.g., 32 samples per millisecond or 64 samples per two milliseconds). In other examples, the threshold data rate can be equal to the nominal clock rate of the local asynchronous media clock 203. If the measured data rate (e.g., sampling rate or packet rate) is lower than the threshold, the receiving audio device 102 can packetize the receiver buffer 207, such as by adding data to the receiver buffer 207, which is sufficient to roughly make up for the lost expected data and roughly achieve the expected data rate. The added data can be a predetermined value (such as all zeros), or it can be other data, such as audio data interpolated from the actual received audio data. If the measured data rate (e.g., sampling rate or packet rate) is higher than the threshold, the receiving audio device 102 can abandon saving the data in the receiver buffer 207, which is sufficient to roughly produce the expected data rate. If the threshold data rate is equal to (or otherwise based on) the nominal clock rate of the local asynchronous media clock 203, the DAC 208 (which may operate in accordance with the local asynchronous media clock 203) may be able to continuously extract data from the receiver buffer 207 at a rate controlled by the local asynchronous media clock 203. Ideally, this process may generally result in the receiver buffer 207 being in a partially full state at all times, while potentially avoiding an underflow condition or an overflow condition of the receiver buffer 207. Additionally, if the measured data rate (e.g., sampling rate or packet rate) is above or below the threshold, the audio device 102 may perform a sampling rate conversion, such as described below with respect to the sampling rate converter 301 and with respect to Figure 3 discussed.

[0038] Thus, each audio device in the audio system 100 can operate using a combination of asynchronous clocks and synchronous clocks. Specifically, for example, each audio device in the audio system 100 can generate its network clock 206 based on the master clock of the network 103, for example, by synchronizing its own network clock 206 with the master clock according to IEEE 1588-2008 or IEEE 1588-2019. The network clock 206 can be used for one or more aspects of communication between the audio devices 101 and 102 via the network 103 (for example, the network clock 206 can be used to generate timestamps for packets). In addition, each audio device in the audio system 100 can generate its own local asynchronous media clock 203, which is synchronized with both the network clock 206 and the master clock of the network 103. The local asynchronous media clock 203 can be used to manage communications and / or processing within the audio device and / or relative to the media source 102 and / or the media receiver 209. Specifically, for example, local asynchronous media clock 203 may be used to control the rate at which analog audio signals from media source 201 are converted by ADC 202 to digital audio signals stored in transmitter buffer 204. Additionally or alternatively, local asynchronous media clock 203 may be used to control the rate at which digital audio signals from receiver buffer 207 are converted by DAC 208 to analog audio signals received by media receiver 209.

[0039] Figure 3 is a block diagram showing another example of details of an audio device, which may be an audio system such as Figure 1 In this example, the audio device 101 or 102 may include the Figure 2Any of the elements discussed, and may also include a sampling rate converter (SRC) 301. SRC 301 can help reduce the amount of overload or underload that the receiver buffer 207 may otherwise experience. SRC 301 can receive digital audio data stored in the receiver buffer 207 and convert the digital audio data to a different sampling rate. For example, if the digital audio data represents audio sampled at a first rate R1, SRC 301 can convert the digital audio data to represent audio sampled at a second rate R2. The rate of R2 may be faster than R1, or the rate of R2 may be slower than R1. If SRC 301 converts from a slower rate R1 to a faster rate R2, SRC 301 can perform an upsampling process, such as by inserting additional digital audio data between existing R1 rate samples. The inserted digital audio data may, for example, have one or more predetermined values ​​(e.g., all zeros, sometimes referred to as "zero stuffing"), or may be an interpolated value calculated based on the original digital audio data value. If SRC 301 is converted from a faster rate R1 to a slower rate R2, then SRC 301 can perform a downsampling process, such as by removing a selected subset of the original digital audio data value / sample. However, other upsampling processes or downsampling processes can be used. SRC 301 can be used to effectively convert digital audio data to be sampled at a rate consistent with a timing domain (e.g., frequency and / or phase) into digital audio data to be sampled at a rate consistent with another different timing domain (e.g., another different frequency and / or phase). For example, rate R1 can be the rate at which the receiver buffer 207 receives packets from the network stack and controller 205 (wherein R1 can be based on the frequency of the local asynchronous media clock 203 of the transmitting party audio device), and rate R2 can be the nominal frequency of the local asynchronous media clock 203 of the receiving party audio device. Therefore, SRC 301 can be used to at least partially compensate for the mismatch in the two local asynchronous media clocks 203 of the transmitting party audio device and the receiving party audio device. Although SRC 301 is shown after receiver buffer 207 (e.g., between receiver buffer 207 and DAC 208), SRC 301 may alternatively be located before receiver buffer 207 (e.g., between network stack and controller 205 and receiver buffer 207). In addition to or as an alternative to providing a receiver-side SRC that works in conjunction with the receiver buffer, a transmitter-side SRC may be added to work in conjunction with the transmitter buffer 204, such as between ADC 202 and transmitter buffer 204 or between transmitter buffer 204 and network stack and controller 205.In this case, the transmit side SRC may convert the sampling rate of the digital audio received from the ADC 202 to a different sampling rate of the audio data to be stored in the transmitter buffer 204 and / or to be transmitted to the network stack and controller 205 .

[0040] Figure 4 is an example audio system (such as Figure 1 1, including example details of two audio devices in the audio system. In the illustrated example, audio device 101 can be communicatively coupled to audio device 102 via network 103. Each of audio devices 101 and 102 can be, for example, based on the embodiments described herein with respect to Figure 2 and / or Figure 3 102. Thus, for example, audio device 101 may transmit audio data packets to audio device 102 via network 103. In addition, audio device 102 may transmit audio data packets to audio device 101 via network 103. Although bidirectional communication is shown between two audio devices 101 and 102, in other examples, audio device 101 may transmit audio data packets to audio device 102 without receiving any audio data packets from audio device 102, or vice versa. In such examples, audio device 101 may not include, for example, elements 207 to 209, and audio device 102 may not include, for example, elements 201, 202, and 204.

[0041] about Figure 4An example of how the system can operate is described as follows. The audio device 101 may include or otherwise be connected to its media source 201. The media source 201 of the audio device 201 may be, for example, a microphone and associated circuitry for operating the microphone. The microphone may generate an analog audio signal, and the ADC 202 of the audio device 101 may receive the analog audio signal and convert it into a digital audio signal. The analog-to-digital conversion of the ADC 202 may be performed at a rate (frequency) and / or phase (e.g., synchronized with the rate and / or phase) of a local asynchronous media clock 203 of the audio device 101. The digital audio signal may be received by a transmitter buffer 203 of the audio device 101, which may store digital audio data based on the digital audio signal. The transmitter buffer 204 may package the stored digital audio into packets (e.g., IP datagrams) and transmit these packets to the network stack and controller 205 of the audio device 101. The transmission of the packetization and / or packets of the stored digital audio data may be performed at a rate and / or phase based on (e.g., synchronized with) the rate (frequency) and / or phase of the local asynchronous media clock 203 or the network clock 206. The network stack and controller 205 of the audio device may also packetize the packets (such as by encapsulating the IP datagrams in Ethernet frames) and transmit those final processed packets to the audio device 102 via the network 103. Transmitting the packets via the network 103 may be performed at a rate and / or phase based on (e.g., synchronized with) the rate (frequency) and / or phase of the master clock.

[0042] After the packets traverse the network 103, the packets may be received by the network stack and controller 105 of the audio device 102, which may at least partially unpack the received packets (e.g., by extracting IP datagrams from encapsulating Ethernet frames) and transmit the resulting audio data packets (e.g., IP datagrams) to the receiver buffer 207 of the audio device 102. Receiving packets from the network 103 may be performed at a rate and / or phase based on a rate (frequency) and / or phase of a master clock (e.g., synchronized with the rate and / or phase). The receiver buffer 207 of the audio device 102 may also unpack the audio packets received from the network stack and controller 205, and extract the audio data and store it in the audio packets. For example, the receiver buffer 207 may extract the audio data stored in the IP datagram received from the network stack and controller 205. The receiver buffer 207 of the audio device 102 may perform unpacking of the audio packets and storing the digital audio data at a rate and / or phase based on the rate (frequency) and / or phase (e.g., synchronized with the rate and / or phase) of the network clock 206 of the audio device 102 or the local asynchronous media clock 203 of the audio device 102. The DAC 208 of the audio device 102 may receive the stored digital audio data from the receiver buffer 207 and may convert the received digital audio data into an analog audio signal, which may be transmitted to the media receiver 209 of the audio device 102. The digital-to-analog conversion may be performed at a rate and / or phase based on the rate (frequency) and / or phase (e.g., synchronized with the rate and / or phase) of the local asynchronous media clock 203 of the audio device 102. The media receiver 209 of the audio device 102 may then process the received analog audio signal. For example, if the media receiver 209 of the audio device 102 is a speaker, the media receiver 209 may generate sound based on the analog audio signal.

[0043] The audio stream can also be transmitted from audio device 102 to audio device 101, and its operation is the same as above with respect to Figure 4 The operations described are the same, except that references to audio device 101 and audio device 102 may be reversed. Furthermore, audio device 101 may transmit audio to audio device 102 while receiving audio from audio device 102, or vice versa. Figure 4Only two audio devices 101 and 102 are shown in the figure, but the audio system 100 may include more than two audio devices, such as three audio devices, four audio devices, or more audio devices, interconnected via the network 103. If there are three or more audio devices in the audio system 100, any given audio device can transmit audio to two or more other audio devices (simultaneously or otherwise), and any audio device can receive audio from two or more other audio devices (simultaneously or otherwise). For example, an audio device can transmit packets addressed to one or more other audio devices via the network 103. Audio packets between any one or more audio devices and another one or more audio devices can be transmitted via one or more streams (such as one or more IP streams).

[0044] The local asynchronous media clock 203 of each of the audio devices 101 and 102 can be asynchronous with each other and with any other clock in the system 100. Thus, for example, the local asynchronous media clock 203 of the audio device 101 can include a first oscillator, and the local asynchronous media clock 203 of the audio device 102 can include a second oscillator independent of the first oscillator. Each of the two local asynchronous media clocks 203 can be implemented using a technology that generally results in a lower precision clock, such as using a crystal-based oscillator, a ceramic resonator, a MEMS oscillator, a SAW oscillator, or an LC oscillator as a non-limiting example. The local asynchronous media clocks 203 of a plurality of audio devices 101 and 102 can have the same nominal frequency, or have different nominal frequencies. For example, the local asynchronous media clock 203 of the audio device 101 can have a nominal frequency of 32kHz, and the local asynchronous media clock 203 of the audio device 102 can have a nominal frequency of 48kHz. Alternatively, local asynchronous media clocks 203 of audio devices 101 and 102 may both have a nominal frequency of 32 kHz, or both may have a nominal frequency of 48 kHz. The specific frequency values ​​mentioned here are merely examples; any one or more nominal frequencies of local asynchronous media clocks 203 may be used.

[0045] Multiple audio devices (e.g., audio devices 101 and 102) in the audio system 100 may transmit non-audio data in addition to audio data via the network 103. Examples of such non-audio data may include configuration settings, status indications, capability indications, or handshake protocol signaling. For example, audio devices may communicate with each other to indicate the nominal frequency of their local asynchronous media clock 203, or to indicate one or more configured or preferred audio compression settings (e.g., configured, available, or preferred one or more compression ratios) or audio compression methods (e.g., configured, available, or preferred one or more types of decoders / decoders (CODECs) to be used). Any type of indication may be used. For example, to indicate a 48kHz clock rate, a data packet including the number "48" or "48,000" may be transmitted by the audio device. Alternatively, an audio device may transmit a data packet indicating a specific shorthand value known to other audio devices in the audio system 100, for example, 32kHz may be represented by a specific bit set to 0, and 48kHz may be represented by a specific bit set to 1. Such non-audio data may be transmitted in a data packet (e.g., a datagram) dedicated to non-audio data, in which case the non-audio data packet may be distinguished from the audio data packet, such as by including a first information in a packet header to indicate the non-audio data packet, and including a different second information in the packet header to indicate the audio data packet. Alternatively, both audio data and non-audio data may be combined together in the same data packet. In either case, the audio data and the non-audio data may each be included in one or more payload portions of one or more packets. One potential advantage of audio devices communicating such information to each other is that the audio devices may use the communicated information to configure themselves in a specific manner so that other audio devices in the audio system 100 configure themselves in a specific manner, or generally negotiate one or more specific configurations so that the audio devices in the audio system 100 will operate and communicate with each other in a compatible manner. For example, two audio devices in the audio system 100 may have two different local asynchronous media clock rates, and may use the exchanged clock rate information or other configuration information to negotiate a specific audio compression ratio based on one or both of the corresponding local asynchronous media clock rates. Such negotiation may be performed automatically between audio devices in the audio system 100. This may provide simplicity for a user of the audio system 100, as the user may not need to be concerned with the local asynchronous media clock rates of the various audio devices in the audio system 100, thereby potentially providing flexibility in selecting audio devices to interact with each other in the audio system 100.

[0046] Figure 5 is a block diagram showing example details of an audio device, which may be an audio system such as Figure 1For example, the audio device may be audio device 101 or audio device 102. The audio device may be implemented as or may otherwise include, for example, a computing device that executes stored instructions and / or may be implemented as hardwired circuits and / or one or more processors that may execute stored computer-readable instructions. In the example shown, the computing device may include or be connected to any of the following: one or more processors 501; storage 502 (which may include one or more computer-readable media, such as memory); an external interface, such as a network interface 503 (which may be configured to communicate with the network 103); a user interface 504; one or more microphones and / or associated circuits 505, the one or more microphones and / or associated circuits configured to detect sound and convert the detected sound into an audio signal, such as an analog audio signal or a digital audio signal; one or more digital signal processors 506, the one or more digital signal processors configured to implement one or more digital signal processing features of an audio device; one or more speakers and / or associated circuits 507, the one or more speakers and / or associated circuits configured to generate sound in response to received audio signals (such as analog audio signals or digital audio signals); and / or a local oscillator 508. The one or more processors 501 may be communicatively connected to any of the other elements 502 to 508 via one or more data buses and / or via one or more other types of connections.

[0047] exist Figure 5In the example of , media source 201 is shown as one or more microphones of element 505, and media receiver 209 is shown as one or more speakers of element 507. However, media source 201 and media receiver 209 can be any other type of media source and media receiver as discussed above. ADC 202 and / or transmitter buffer 204 can be implemented by circuits of element 505 and / or one or more processors 501, and DAC 208 and / or receiver buffer 207 can be implemented by circuits of element 507 and / or one or more processors 501. As desired, the circuits of elements 505 and 507 can be separate circuits or a single instance of a combined circuit. Network stack and controller 205 and / or network clock 206 can be implemented by network interface 503 and / or one or more processors 501. Local asynchronous media clock 203 can be implemented by local oscillator 508. In the example shown, local oscillator 508 can provide a local asynchronous media clock signal to one or more processors 501, circuitry of element 505 (e.g., to control the operation of ADC 202 and / or transmitter buffer 204), and circuitry of element 507 (e.g., to control the operation of DAC 208 and / or receiver buffer 207). However, the local asynchronous media clock signal can be provided to other processors as desired. Figure 5 In one example, the one or more processors 501 may receive a signal from the local oscillator 508, and the one or more processors 501 may generate an asynchronous local media clock based on the signal from the local oscillator 508. For example, the one or more processors 501 may include a phase-locked loop (PLL) circuit, and the signal from the local oscillator 508 may be an input to (e.g., for driving) the PLL circuit.

[0048] The one or more processors 501 may be configured to execute instructions stored in the storage device 502. The instructions, when executed by the one or more processors 501, may cause the computing device (and therefore the audio device) to perform any functionality described herein as being performed by an audio device (such as the audio device 101 or the audio device 102). For example, the one or more processors 501 may control the operation of any of the other elements 502 to 508 of the audio device and / or may direct various signals (such as audio signals and / or clock signals) between the various elements 502 to 508 of the audio device.

[0049] The audio device and / or any of the elements of the audio device (eg, any of elements 501 to 508) may be powered as needed. Although not explicitly shown, the audio device may include an internal battery and / or an external power connection.

[0050] In the following, various features will be highlighted in a set of numbered clauses or paragraphs. These features should not be construed as limitations on the invention or inventive concept, but are merely provided as an emphasis on some features as described herein without implying a particular order of importance or relevance of such features.

[0051] Clause 1. A method comprising:

[0052] receiving digital audio data via a network based on a network clock synchronized with a master clock of the network;

[0053] Generates local asynchronous media clock;

[0054] comparing a rate of the received digital audio data to a threshold data rate, wherein the threshold data rate is based on a nominal rate of the local asynchronous media clock;

[0055] storing at least a portion of the received digital audio data in a buffer, wherein the at least a portion of the received digital audio data is based on the comparison;

[0056] generating an analog audio signal based on the at least a portion of the digital audio data stored in the buffer using the local asynchronous media clock; and

[0057] A speaker is used to generate sound based on the analog audio signal.

[0058] Clause 2. A method according to clause 1, wherein generating the local asynchronous media clock includes using low-precision timing technology, such as using at least one of the following to generate the local asynchronous media clock: a crystal-based oscillator, a MEMS oscillator, a ceramic resonator, a SAW oscillator, or an LC oscillator.

[0059] Clause 3. A method according to clause 1 or 2, wherein receiving the digital audio data based on the network clock includes receiving the digital audio data based on multiple timestamps, and the multiple timestamps are generated based on the network clock and / or based on multiple sequence numbers included in the digital audio data.

[0060] Clause 4. The method according to any one of clauses 1 to 3, further comprising: buffering the digital audio data in a plurality of buffer locations, the plurality of buffer locations being based on the plurality of timestamps and / or based on the plurality of sequence numbers.

[0061] Clause 5. The method of any one of clauses 1 to 4, further comprising: based on the comparison, forgoing storing at least some of the received digital audio data in the buffer.

[0062] Clause 6. The method of any one of clauses 1 to 4, further comprising: packing additional data into the buffer based on the comparison.

[0063] Clause 7. A method according to any one of clauses 1 to 6, wherein receiving the digital audio data comprises receiving a plurality of data packets, the plurality of data packets comprising the digital audio data, the method further comprising: extracting the digital audio data from the plurality of data packets.

[0064] Clause 8. The method of clause 7, wherein the plurality of data packets comprises one or both of: a plurality of Internet Protocol datagrams or a plurality of Ethernet frames.

[0065] Clause 9. The method of clause 7 or 8, wherein the extraction is managed by the local asynchronous media clock.

[0066] Clause 10. The method of any one of clauses 1 to 9, wherein the receiving is performed by a first audio device, the method further comprising:

[0067] receiving a second analog audio signal based on the detected sound;

[0068] generating second digital audio data based on the second analog audio signal using the local asynchronous media clock; and

[0069] The second digital audio data is transmitted via the network.

[0070] Clause 11. A method according to any one of clauses 1 to 10, the method further comprising: decompressing at least a portion of the digital audio data, wherein generating the analog audio signal based on the at least a portion of the digital audio signal comprises generating the analog audio signal based on the decompressed digital audio data.

[0071] Clause 12. A method according to any one of clauses 1 to 11, the method further comprising: decrypting at least a portion of the digital audio data, wherein generating the analog audio signal based on the at least a portion of the digital audio signal comprises generating the analog audio signal based on the decrypted digital audio data.

[0072] Clause 13. The method of any one of clauses 1 to 12, wherein the storage is managed by the local asynchronous media clock.

[0073] Clause 14. The method of any one of clauses 1 to 13, wherein generating the analog audio signal based on the digital audio data comprises using a digital-to-analog converter to convert the at least a portion of the digital audio data stored in the buffer into the analog audio signal.

[0074] Clause 15. The method of Clause 14, further comprising: performing a sampling rate conversion on the at least a portion of the digital audio data stored in the buffer.

[0075] Clause 16. A method according to any one of clauses 1 to 15, the method further comprising: determining one or more discarded packets based on received packet sequence numbers and / or received packet timestamps; and adjusting data in a receiving buffer based on the determination of one or more discarded packets.

[0076] Clause 17. A first audio device, the first audio device comprising:

[0077] one or more processors; and

[0078] One or more computer-readable media storing instructions which, when executed by the one or more processors, cause the first audio device to perform the method of any one of clauses 1 to 16.

[0079] Clause 18. A non-transitory computer-readable medium storing instructions that, when executed, cause a first audio device to perform the method of any one of clauses 1 to 16.

[0080] Clause 19. A method comprising:

[0081] receiving an analog audio signal based on the detected sound;

[0082] Generates local asynchronous media clock;

[0083] generating digital audio data based on the analog audio signal using the local asynchronous media clock;

[0084] generating a network clock based on a master clock of the network; and

[0085] The digital audio data is transmitted over the network based on the network clock.

[0086] Clause 20. The method of claim 19, wherein generating the local asynchronous media clock comprises generating the local asynchronous media clock using a low-precision timing technique, such as using at least one of the following to generate the local asynchronous media clock: a crystal-based oscillator, a MEMS oscillator, a ceramic resonator, a SAW oscillator, or an LC oscillator.

[0087] Clause 21. The method of clause 19 or 20, further comprising: packetizing the digital audio data into a plurality of data packets, wherein the transmitting comprises transmitting the plurality of data packets.

[0088] Clause 22. The method of any one of clauses 19 to 21, wherein the plurality of data packets comprises one or both of: a plurality of Internet Protocol datagrams or a plurality of Ethernet frames.

[0089] Clause 23. A method according to any one of clauses 19 to 22, wherein the packetization is managed by the local asynchronous media clock.

[0090] Clause 24. A method according to any one of clauses 19 to 23, wherein transmitting the digital audio data based on the network clock comprises transmitting the digital audio data in a plurality of packets, each packet comprising a timestamp based on the network clock and / or each packet comprising a sequence number.

[0091] Clause 25. The method of any one of clauses 19 to 24, wherein transmitting the digital audio data comprises transmitting the digital audio data in a plurality of packets at a rate based on a frequency of the local asynchronous media clock.

[0092] Clause 26. The method of any one of clauses 19 to 25, wherein the transmitting is performed by a first audio device, the method further comprising:

[0093] receiving, by the first audio device via the network, second digital audio data;

[0094] generating a second analog audio signal based on the digital audio data using the local asynchronous media clock; and

[0095] Sound is generated based on the second analog audio signal using a speaker associated with the first audio device.

[0096] Clause 27. The method of any one of clauses 19 to 26, further comprising: compressing the digital audio data, wherein the transmitting comprises transmitting the compressed digital audio data.

[0097] Clause 28. The method of any one of clauses 19 to 27, further comprising: encrypting the digital audio data, wherein the transmitting comprises transmitting the encrypted digital audio data.

[0098] Clause 29. The method of any one of clauses 19 to 28, further comprising: storing the digital audio data in a buffer, wherein the storing is managed by the local asynchronous media clock.

[0099] Clause 30. The method of any one of clauses 19 to 29, wherein the transmitting is performed by a first audio device, the method further comprising:

[0100] generating a second local asynchronous media clock using a second crystal-based oscillator;

[0101] receiving, by a second audio device via the network, the digital audio data; and

[0102] A second analog audio signal is generated based on the received digital audio data using the second local asynchronous media clock.

[0103] Clause 31. The method of clause 30, further comprising: buffering the received digital audio data in a buffer location based on a timestamp and / or sequence number associated with the received digital audio data.

[0104] Clause 32. The method of clause 31, further comprising generating a second network clock based on the master clock of the network, wherein the buffer position is based on both the timestamp and the second network clock.

[0105] Clause 33. The method of any one of clauses 30 to 32, further comprising: generating sound based on the second analog audio signal using a speaker associated with the second audio device.

[0106] Clause 34. A method according to any one of clauses 30 to 33, wherein the local asynchronous media clock has a nominal first clock frequency and the second local asynchronous media clock has a nominal second clock frequency, and wherein the nominal first clock frequency is different from the nominal second clock frequency.

[0107] Clause 35. A method according to any one of clauses 30 to 33, wherein the local asynchronous media clock has a nominal first clock frequency of one of 32 kHz or 48 KHz, and wherein the second local asynchronous media clock has a different nominal second clock frequency of the other of 32 KHz or 48 KHz.

[0108] Clause 36. A first audio device, the first audio device comprising:

[0109] one or more processors; and

[0110] One or more computer-readable media storing instructions which, when executed by the one or more processors, cause the first audio device to perform the method of any of clauses 19 to 29.

[0111] Clause 37. A system comprising:

[0112] A first audio device, the first audio device comprising:

[0113] one or more processors; and

[0114] one or more computer-readable media storing instructions which, when executed by the one or more processors of the first audio device, cause the first audio device to perform a method according to any one of clauses 19 to 29; and

[0115] A second audio device, wherein the second audio device comprises:

[0116] one or more processors; and

[0117] One or more computer-readable media storing instructions that, when executed by the one or more processors of the second audio device, cause the second audio device to perform the steps further recited in any of clauses 30 to 35.

[0118] Clause 38. A non-transitory computer-readable medium storing instructions that, when executed, cause a first audio device to perform the method of any one of clauses 19 to 29.

[0119] Although various examples are described above, the features and / or steps of those examples may be combined, divided, omitted, rearranged, modified and / or expanded in any desired manner. Those skilled in the art will readily appreciate various changes, modifications and improvements. Although not explicitly stated herein, such changes, modifications and improvements are intended to be part of this specification and are intended to fall within the spirit and scope of the present disclosure. Therefore, the foregoing description is merely exemplary and not restrictive.

Claims

1. An audio device, comprising: a local asynchronous media clock having a frequency variation of at least one part per million; buffer; one or more processors; and One or more computer-readable media storing instructions that, when executed by the one or more processors, configure the audio device to: receiving digital audio data via a network based on a network clock synchronized with a master clock of the network; comparing a rate of the received digital audio data to a threshold data rate, wherein the threshold data rate is based on a nominal rate of the local asynchronous media clock; storing at least a portion of the received digital audio data in the buffer, wherein the at least a portion of the received digital audio data is based on comparing the rate of the received digital audio data to the threshold data rate; generating an analog audio signal based on said at least a portion of said digital audio data stored in said buffer using said local asynchronous media clock; as well as The analog audio signal is transmitted for generation of sound by a speaker.

2. The audio device of claim 1, wherein the local asynchronous media clock comprises one or more of: a crystal-based oscillator, a microelectromechanical system oscillator (MEMS) oscillator, a ceramic resonator, a surface acoustic wave (SAW) oscillator, or an inductor / capacitor (LC) oscillator.

3. The audio device of claim 1 , wherein the instructions, when executed by the one or more processors, configure the audio device to forgo storing at least some of the received digital audio data in the buffer based on the comparison.

4. The audio device of claim 1 , wherein the digital audio data comprises one or both of a plurality of time stamps or a plurality of packet sequence numbers; and wherein the instructions, when executed by the one or more processors, configure the audio device to: based on one or both of the plurality of sequence numbers or the plurality of time stamps, perform at least one of the following: abandoning storing at least some of the received digital audio data in the buffer; storing locally generated audio data in the buffer; interpolating the data in the buffer; or The data in the buffer is time corrected.

5. The audio device of claim 1 , wherein the instructions, when executed by the one or more processors, configure the audio device to receive the digital audio data by at least: receiving a plurality of data packets, the plurality of data packets comprising the digital audio data; and The digital audio data is extracted from the plurality of data packets.

6. The audio device of claim 1, wherein the instructions, when executed by the one or more processors, configure the audio device to extract the digital audio data from a plurality of data packets at a timing governed by the local asynchronous media clock.

7. An audio device according to claim 1, wherein the instructions, when executed by the one or more processors, configure the audio device to: store at least a portion of the received digital audio data in the buffer using timing managed by the local asynchronous media clock.

8. An audio device according to claim 1, wherein the instructions, when executed by the one or more processors, configure the audio device to: decompress at least a portion of the digital audio data; and generate the analog audio signal based on the at least a portion of the digital audio signal by generating the analog audio signal based at least on the decompressed at least a portion of the digital audio data.

9. An audio device according to claim 1, wherein the instructions, when executed by the one or more processors, configure the audio device to: decrypt at least a portion of the digital audio data; and generate the analog audio signal based on the at least a portion of the digital audio signal by generating the analog audio signal based at least on the decrypted at least a portion of the digital audio data.

10. The audio device of claim 1, wherein the instructions, when executed by the one or more processors, configure the audio device to perform sample rate conversion on the at least a portion of the digital audio data stored in the buffer.

11. An audio device, comprising: a local asynchronous media clock having a frequency variation of at least one part per million; one or more processors; and One or more computer-readable media storing instructions that, when executed by the one or more processors, configure the audio device to: receiving an analog audio signal based on the detected sound; generating digital audio data based on the analog audio signal using the local asynchronous media clock; Generate network clock based on the master clock of the network; as well as The digital audio data is transmitted over the network based on the network clock.

12. The audio device of claim 11, wherein the local asynchronous media clock comprises one or more of: a crystal-based oscillator, a microelectromechanical system oscillator (MEMS) oscillator, a ceramic resonator, a surface acoustic wave (SAW) oscillator, or an inductor / capacitor (LC) oscillator.

13. The audio device of claim 11, wherein the instructions, when executed by the one or more processors, configure the audio device to packetize the digital audio data into a plurality of data packets using timing managed by the local asynchronous media clock.

14. The audio device of claim 11, wherein the instructions, when executed by the one or more processors, configure the audio device to transmit the digital audio data based on the network clock by transmitting the digital audio data in at least a plurality of packets, each packet including a timestamp based on the network clock.

15. The audio device of claim 11, wherein the instructions, when executed by the one or more processors, configure the audio device to transmit the digital audio data by transmitting the digital audio data in a plurality of packets at a rate that is at least based on a frequency of the local asynchronous media clock.

16. The audio device of claim 11, wherein the instructions, when executed by the one or more processors, configure the audio device to: compress the digital audio data; and transmit the digital audio data at least by transmitting the compressed digital audio data.

17. The audio device of claim 11, wherein the instructions, when executed by the one or more processors, configure the audio device to: encrypt the digital audio data; and transmit the digital audio data at least by transmitting the encrypted digital audio data.

18. The audio device of claim 11, wherein the instructions, when executed by the one or more processors, configure the audio device to store the digital audio data in a buffer using timing managed by the local asynchronous media clock.

19. A method comprising: receiving digital audio data from a sender audio device via a network based on a network clock synchronized with a master clock of the network; comparing a rate of the received digital audio data to a threshold data rate, wherein the threshold data rate is based on a nominal rate of a local asynchronous media clock, and wherein a frequency of the local asynchronous media clock is different than a frequency of a local asynchronous media clock of the sender audio device; storing at least a portion of the received digital audio data in a buffer, wherein the at least a portion of the received digital audio data is based on comparing the rate of the received digital audio data to the threshold data rate; generating an analog audio signal based on said at least a portion of said digital audio data stored in said buffer using said local asynchronous media clock; as well as A speaker is caused to generate sound based on the analog audio signal.

20. The method according to claim 19, further comprising: The digital audio data is extracted from a plurality of data packets at a timing governed by the local asynchronous media clock.

21. The method according to claim 19, further comprising: The at least a portion of the received digital audio data is stored in a buffer using timing managed by the local asynchronous media clock.

Images