Method for synchronizing electronic devices and wireless devices for wireless connections

Abstract

To synchronize wirelessly connected audio playback devices with high precision, a first audio device (310, 410), e.g. a soundbar, has a first and a second radio transmitter (414, 415). A plurality of second audio devices (320, 420-422), e.g. loudspeaker boxes, each have a first and a second radio receiver (4204, 4205). The first radio transmitter (414) sends a first data stream (3301) comprising synchronization packets (811-816) over a network with high time precision. The second radio transmitter (415) sends a second data stream (3311) comprising audio data packets (821-826) over a different network with high bandwidth. The first audio device (310, 410) plays back first audio data. Each of the second audio devices (320, 420-422) receives the first and second data streams (3301, 3311) and plays back the received audio data based on synchronization information in the synchronization packets, thereby achieving time synchronization with the first audio device.

Description

Technical Field

[0001] This invention relates to synchronizing electronic devices that are wirelessly connected to each other. Background Technology

[0002] Various methods are known for synchronizing wirelessly connected devices. However, high-precision synchronization remains challenging to achieve. This is particularly important in the field of audio playback devices designed to play back audio content in a coordinated manner, as the human ear is sensitive to small time deviations of a few milliseconds. For example, multiple audio playback devices (such as speaker cabinets) located in the same room and designed to play back multi-channel audio content (such as object-based immersive 3D audio) require very precise synchronization. Such synchronization can be easily achieved as long as such audio playback devices can receive signals via wires, since only the acoustic signal travel time due to sound propagation needs to be considered. These can be calculated or measured and then compensated for. Therefore, for wired speakers, only the very small time delay due to sound propagation needs to be compensated for.

[0003] However, if a wired signal for synchronization is unavailable, synchronization must be based on wirelessly received signals. Therefore, in addition to acoustic sound propagation, the transmission delay, buffering, and processing of the electrical audio signal itself must be considered for wireless audio streaming transmission. This is challenging because radio signals are typically unstable due to unknown signal run times and jitter.

[0004] For a small number of playback devices and a total bandwidth of up to 2 Mbps, Bluetooth Low Energy (LE) Audio (BLE) is a known solution. It uses isochronous channels to wirelessly synchronize up to five individual channels with high precision. This allows for synchronized audio playback through a main speaker or soundbar and up to five additional speakers. However, the limited number of speaker devices and bandwidth necessitate compression of the audio signal for transmission. Furthermore, retransmissions mean that not the full bandwidth is available for the audio data.

[0005] Figure 1 A system 100 is shown with two speaker enclosures 120 and 121 wirelessly connected to a first source device 110 via a Bluetooth network (e.g., BLE). The first source device 110 generates a multicast BLE connection, specifically for connecting a BLE audio broadcast stream (BIS) 130 to each of the speaker enclosures 120 and 121. Using the latest BLE standard, device 110 can broadcast up to five independent audio channels if the audio data is compressed via an audio codec. Multiple speaker enclosures can be connected, each of which must select the correct audio channel and stream, and the total bandwidth is limited to 2 Mbps.

[0006] On the other hand, other network protocols such as Wi-Fi are known to offer higher bandwidth and therefore higher sampling or bit rates, but they have unpredictable latency due to varying traffic, error correction, and other factors.

[0007] Figure 2 A system 200 with two speaker enclosures 220 and 221 is illustrated. The speaker enclosures 220 and 221 are wirelessly connected to an access point (AP) device 210 via a network with very high bandwidth (e.g., Wi-Fi (IEEE 802.11x) or Wi-Fi Direct). The second AP device 210 generates separate unicast Wi-Fi connections 230 and 231 or Wi-Fi multicast connections to each speaker enclosure 220 and 221. The number of speaker enclosures that can be connected is limited only by Wi-Fi bandwidth, and each link can provide significantly higher bandwidth than a BLE connection. The problem is that packet runtime cannot be reliably predicted and is often variable due to high jitter. Therefore, synchronized playback from multiple speakers is difficult to achieve.

[0008] US2023 / 0276515 A1 describes a low-latency wireless audio transmission and reception method that uses a Synchronous Streaming (CIS) link to establish a connection based on a high-speed physical layer, which has a higher data transmission rate than the BLE physical layer. Audio data is transmitted to an audio receiving device via the CIS link based on the high-speed physical layer. For this purpose, the CIS protocol is used in Wi-Fi networks operating in a point-to-point connection. However, this solution is not suitable for standard BLE and standard Wi-Fi protocols. Therefore, modifications to the BLE / CIS protocol are needed to adapt it to Wi-Fi, or modifications to the standard Wi-Fi protocol are needed to adapt it to BLE / CIS. Summary of the Invention

[0009] The purpose of this invention is to provide an improved solution for synchronizing electronic devices, particularly a group of audio playback devices, while wirelessly connecting them via a high-bandwidth connection.

[0010] This objective is achieved by the method of claim 1. In some embodiments, the invention relates to the audio playback device of claim 8. In other embodiments, the invention relates to the wireless speaker device of claim 13.

[0011] This invention is based on the use of two different wireless networks, wherein at least the first network provides packets with more accurate timing than the second network, and wherein the second network provides at least higher bandwidth than the first network. This dual-topology radio system architecture is suitable for combining the advantages of networks providing low-latency wireless links with the advantages of networks providing higher data throughput.

[0012] Various aspects and advantageous embodiments are provided in the following detailed description and claims. Attached Figure Description

[0013] The invention will be better understood from the following detailed description taken in conjunction with the accompanying drawings, in which the same reference numerals denote the same parts. The drawings are shown below.

[0014] Figure 1 These are two wireless speaker boxes connected to the playback device via BLE;

[0015] Figure 2 These are two wireless speaker boxes connected to the playback device via Wi-Fi;

[0016] Figure 3 It is based on the principle of two wireless speaker boxes connected to the playback device via two different networks in the implementation method;

[0017] Figure 4 This is a schematic diagram of an audio playback system according to an embodiment;

[0018] Figure 5 This is a schematic diagram of an audio playback system according to another embodiment;

[0019] Figure 6 It is a schematic block diagram showing the components of the transmitter and receiver equipment;

[0020] Figure 7 It is a sequence diagram with synchronous (sync) groups;

[0021] Figure 8 It is a timing diagram showing the time relationship between packets in the first and second networks and audio broadcast packets;

[0022] Figure 9 This is a timing diagram illustrating the network latency in the first example;

[0023] Figure 10 This is a timing diagram illustrating network latency in the second example;

[0024] Figure 11 This is a schematic block diagram showing the components of the transmitter equipment; and

[0025] Figure 12 It's a flowchart. Detailed Implementation

[0026] In an embodiment of the present invention Figure 3A system 300 is shown having two wireless speaker enclosures 320, 321 connected to an audio hub or source device 310 via two different wireless networks. The source device 310 includes at least two transmitters, one for each network. In the example shown, a first transmitter of the source device 310 provides a multicast or broadcast BLE connection 3301 to the speaker enclosures 320, 321, and a second transmitter of the source device 310 provides a unicast or multicast Wi-Fi connection 3311, 3312 to the speaker enclosures 320, 321. The BLE connection 3301 includes at least one isochronous channel. Each of the speaker enclosures 320, 321 has at least two wireless receivers, one corresponding to each of the transmitters in the source device. The source device provides a synchronization signal at least via the first network connection 3301 and audio data via the second network connections 3311, 3312. Each speaker enclosure receives the synchronization signal and audio data respectively through its first and second receivers and combines them before playing back the audio data. The first network connection 3301 provides a very stable and accurate time reference (such as a "heartbeat") for playback of audio data provided via the second network connections 3311 and 3312. For example, the Bluetooth (BT) standard from BT version 5.2 and above provides isochronous channels that can be used for this synchronization. Due to the lower latency and higher bandwidth of the second network connections 3311 and 3312, the speaker enclosures will typically receive audio data packets before the corresponding synchronization packets; therefore, each speaker enclosure buffers the received audio data packets until the expected playback time indicated by the synchronization packets.

[0027] In one implementation, source device 310 can be integrated into an audio playback device such as a soundbar, allowing speaker enclosures 320, 321 to play back audio data simultaneously with and in addition to playback from the soundbar. In an alternative implementation, where source device 310 is a hub without speakers, the audio playback from speaker enclosures 320, 321 can be synchronized with each other. One advantage of this invention is that the audio data played back from the speaker enclosures will be synchronized with each other and with high precision with the audio data played back from the soundbar. Another advantage is that individual speaker enclosures can be added or removed during operation very easily, as synchronization is performed automatically for the group of connected speaker enclosures. Furthermore, the number of speaker enclosures in the group is unlimited. Another advantage is that standard network connections (i.e., normal BLE and Wi-Fi connections) can be used, at least relative to their physical layer.

[0028] Figure 4A schematic diagram of an audio playback system 400 according to an embodiment is shown. It includes an audio hub 410, which may also be a soundbar or main speaker, and one or more separate audio playback devices 420, 421, 422, or speaker enclosures that may act as rear speakers. The hub 410 has two different wireless transmitters for different networks (i.e., different protocols): a first transmitter 414 is capable of sending at least synchronization packets to the audio playback devices using a first network 3301 protocol, while a second transmitter 415 is capable of sending at least audio data packets to the audio playback devices using a different second network 3311 protocol. Each of the audio playback devices 420, 421, 422 has two wireless receivers, one for each of the different networks 3301, 3311. Note that although three audio playback devices 420, 421, 422 are depicted, more or fewer such devices may be used. For example, the first playback device 420 has a first receiver 4204 as a BLE receiver for receiving synchronization data and a second receiver 4205 as a Wi-Fi receiver for receiving audio data. Each playback device can extract its individual audio data from received audio packets from the second network connection 3311 and combine the received packets as further described below to achieve synchronized audio playback. Furthermore, for each of the audio playback devices 420, 421, and 422, its synchronized audio playback can be adjusted in a known manner based on the acoustic signal run time or the location of individual loudspeakers. In this embodiment, both the first network 3301 and the second network 3311 use broadcast or multicast transmission.

[0029] Figure 5 A schematic diagram of an audio playback system 500 according to another embodiment is shown. This is similar to the previous embodiment, except that the hub 510 uses second network unicast (or multicast) transmission with separate channels 3311, 3312, 3313 for each of the audio playback devices 520, 521, 522, while the first network 3301 uses broadcast or multicast as described above. This allows for a different network protocol type for the second network than in the previous embodiment. Each audio playback device includes a corresponding first receiver 5204, 5214, 5224 for the first network and a second receiver 5205, 5215, 5225 for the second network. As previously described, this embodiment may also include generating separate audio data in the hub 510 for transmission to different audio playback devices 520-522. Each audio playback device 520-522 can extract its separate audio data from received audio packets from its respective channel of the second network connections 3311-3313, as in the audio playback devices 420-422 of the previous embodiment.

[0030] In a variation of the above embodiment, in addition to other audio playback devices 420-422, 520-522, at least one additional subwoofer (not shown) receives audio packets directly via the first network 3301. The subwoofer connection to the second network 3311 may optionally be omitted. Such an audio playback system would then include a hub or soundbar 410, at least one subwoofer, and at least one of the other audio playback devices 420-422, 520-522.

[0031] Figure 6A schematic block diagram of components having a transmitter device and a receiver device in one embodiment is shown. The components may be part of a system 600 implemented by a hub device and an audio playback device. The hub device includes an audio processing module 610, a first network interface module 620, and a second network interface module 630. Each of the network interface modules 620 and 630 includes at least a corresponding transmitter. In this example, the first network interface module 620 is a Bluetooth module that provides at least a BLE audio broadcast connection to an isochronous channel 614, while the second network interface module 630 is an IEEE 802.11x compatible or Wi-Fi module, such as a UDP server that sends TCP / UDP packets via a Wi-Fi connection 615. However, different Wi-Fi types can be used; that is, in IEEE 802.11x, x can be one of a, ac, ax, ad, b, g, and n. The audio processing module 610 can normally provide the first network interface module 620 with a digital audio signal 612 of any format, such as a dual-channel I2S signal or a multiplexed TDM signal. These are widely used, allowing implementations to be based on standard components. In some implementations, the audio processing module 610 can generate different audio data packets for different receivers or audio playback devices. The audio processing module 610 and the first network interface module 620 can also be implemented on a single chip. However, the digital audio signal or at least one channel thereof includes timestamps in some packets, which are thus marked as synchronization packets, for example, by watermarking one of the channels. Since it is sufficient if one of the two channels includes marked synchronization packets, the other channel can be used for redundancy or for a different purpose. For example, it can include audio signals for the subwoofer channel. The first network interface module 620 transmits at least the synchronization packets via BLE broadcast or multicast connection 614 and preferably via a BLE isochronous channel. This ensures accurate synchronization. The first network interface module 620 can additionally transmit audio data to be played back, at least as much as its bandwidth allows (e.g., the subwoofer channel). The audio processing module 610 also provides audio data and timestamps to the second network interface module 630, which generates audio data packets such as TCP / UDP packets, adds the timestamps to the corresponding audio data packets, and then sends the packets. This ensures high-bandwidth transmission and a robust relationship between audio packets and synchronization packets. In one embodiment, the audio data sent by the second network interface module 630 can be decompressed using the available high bandwidth.

[0032] The receiver module 640 within the audio playback device includes a first receiver 6404 for receiving packets from a BLE broadcast or multicast connection 614, a second receiver 6405 for receiving TCP / UDP packets from a Wi-Fi connection 615, and a combiner module 6401 that can act as a synchronizer module. The first receiver 6404 receives synchronization packets and extracts timestamps and / or timestamp synchronization signals, providing them to the combiner module 6401. The second receiver 6405 may optionally extract a portion of the audio data to be replayed from the received audio data packets and provide at least the extracted or received audio data packets to be replayed to the combiner module 6401. One module, such as the second receiver 6405 or the combiner module 6401, extracts timestamps from the audio data packets, and one of these modules (or a separate module, not shown) buffers the audio data packets. The combiner module 6401 aligns the audio data packets for output according to the timestamps, as further described below. Therefore, the audio data output is synchronized according to the timestamps, and synchronized audio data I2S can be provided. Out For playback. The receiver module 640 can be included in an external speaker enclosure, such as the rear speakers 420-422, 520-522 for a soundbar. Some additional blocks may be needed for audio playback, such as a DAC, amplifier, etc., but... Figure 6 Not shown in the diagram. In this embodiment, the clock signal in receiver module 640 is also synchronized based on a timestamp. For example, it can be generated by a PLL.

[0033] Figure 7 A basic timing diagram is shown for an exemplary explanation of the preceding implementation. However, other suitable network types can be used alternatively. Receiver module 640 receives a first BLE packet marked as a synchronization packet at a first time t1 via a BLE isochronous channel. It extracts synchronization information from the first BLE packet. It can also receive subsequent BLE packets not marked as synchronization packets at regular time intervals dt (e.g., dt = 7.5 ms or 10 ms), which can be ignored or used as additional synchronization. For example, such subsequent BLE packets may include audio or other data for different receivers. The timing information is determined by the time difference T relative to the first time t1. Delay The second time t2 is defined, and the output I2S Out Upward-facing speakers provide T DelayA defined amount of audio data, previously or during which it has been received by the UDP receiver 6405, is provided for playback (“broadcast”). Note that the second time t2 may, but does not necessarily, need to coincide with the received BLE packet, as shown in the figure. The provided audio data has been received in Wi-Fi packets prior to broadcast, wherein at least one of the Wi-Fi packets includes synchronization information that matches the synchronization information of the first BLE packet. This is referred to herein as logical synchronization and ensures that the correct audio data is broadcast at a specific time. A fixed time difference T after receiving the first marked BLE packet. sync At the third time t3, another BLE packet marked as a synchronization packet is received. Therefore, the frequency is 1 / T. sync A stable "heartbeat" (or 1 / dt) can be used by the receiver and for synchronizing audio playback. Playback delay T Delay The playback delay T can be derived from the processing time and / or sound transmission latency required by the current audio playback device. If another audio playback device is connected to the same hub as the current audio playback device, the playback delay T... Delay The processing time and / or audio transmission delay required by another playback device can also be derived to enable synchronized playback of all playback devices connected to the hub. In various implementations, only tagged BLE packets are used for time- and logically synchronized audio data, or all BLE packets are used for time-synchronized audio data, while only tagged BLE packets are used for logically synchronized audio data.

[0034] Figure 8 A more detailed timing relationship between packets from the first and second networks received at the audio playback device and audio broadcast packets is illustrated. A first stream 810, which may be a BLE broadcast isochronous stream (BIS), includes isochronous packets (synchronization packets) from the first network received with high timing accuracy. A first synchronization packet 811 is a marked synchronization packet (ISO packet) that includes synchronization information (e.g., a first sequence number or ID value). A defined number (exemplarily twenty) of subsequent synchronization packets 812, 813, 814 are unmarked ISO packets that do not include dedicated synchronization information, while the next synchronization packet 815 again includes synchronization information. Preferably, each marked synchronization packet 811, 815 includes separate synchronization information.

[0035] The second stream 820 (which may be IEEE 802.11x or Wi-Fi / Wi-Fi Direct Stream) comprises audio data packets from a second network received with low latency and a high data rate (at least higher than the first stream and lower latency on average) but high jitter (e.g., UDP packets). The first UDP packet 821 comprises audio data and synchronization data, such as a sequence number or ID value matching the sequence number or ID value of the first-tagged synchronization packet 811. At time t1 of receiving the first-tagged synchronization packet 811, at least a portion of the first UDP packet 821 has typically been received and buffered at receiver module 640. Depending on the bandwidth and other parameters provided by the first and second networks, the complete first UDP packet 821, and potentially one or more additional UDP packets 822, 823, may have been received and buffered at time t1 of receiving the first-tagged synchronization packet 811. Time t1 of receiving the first-tagged synchronization packet 811 triggers the playback of the first buffered audio data packet 831. For the playback of subsequent audio data packets 832, 833, a predefined constant presentation delay T is maintained. delay Audio data can be directly mapped from UDP packet 821 to broadcast packet 831. This process repeats cyclically, i.e., within a complete synchronization period T. sync Subsequently, the next tagged ISO packet 815 is received in the first stream 810, and the next tagged UDP packet 825 with synchronization data is received in the second stream 820. The synchronization data of the tagged ISO packet 815 is compared with the synchronization data of the tagged UDP packet 825, and if both match, the broadcast packet 835 corresponding to the UDP packet 825 is broadcast at its corresponding broadcast time (i.e., T after the reception of the synchronization packet 815). Delay This is then played back along with a defined number of subsequent groups. In this way, the initial synchronization can be found easily and reliably.

[0036] Note that in the above embodiment, the number of ISO packets in the first stream 810 (per cycle) is equal to the number of UDP packets in the second stream 820 (per cycle). Therefore, the time difference between subsequent UDP packets is on average equal to the time difference between subsequent ISO packets, although it varies due to jitter. However, this is merely illustrative, and in other embodiments, the number of packets per cycle can differ between the two streams, such that each cycle has n synchronization packets and m audio data packets. In this case, resynchronization can be performed, for example, every nth synchronization packet and every mth audio data packet. Furthermore, in this example, all audio data is sent in the second stream 820, while the first stream 810 is not used to send audio data. However, in another embodiment, a first portion of the audio data, such as for low data rate audio or audio for a specific channel, can be sent in the first stream 810. As described, the additional audio data sent in the second stream 820 and added to the first audio portion improves audio playback and increases the data rate. Synchronization playback remains as described above.

[0037] To ensure that each packet is mapped to the correct timestamp, if the time T between the received tagged synchronization packet and the broadcast... delay Less than the time T between subsequent markers sync This could be advantageous (though not necessary).

[0038] The broadcast time T between the reception and transmission of broadcast data via BLE synchronization packets (e.g., from the buffer to the DAC). Delay It should be fixed so that the broadcast packets, including the audio data for playback, are synchronized output streams (I2S) after the buffer. Out The audio data is provided at fixed time intervals. In the event that a synchronization packet has not yet been received or has been lost (e.g., due to radio interference), the combiner module 6401 can continue playing the received audio data packet 820 during these fixed time intervals and can be resynchronized later. For this purpose, it can have a backup timer module that can be readjusted by the received tagged ISO packets.

[0039] Receiving a tagged synchronization packet (i.e., an unexpected synchronization packet) within the aforementioned fixed broadcast time frame may cause resynchronization of the audio playback. To resynchronize frames, it can be assumed that the BLE packet tagged is periodic and that the time difference between BLE tagged packets is constant: T recBLE =T recBLE、i-1 -T recBLE、i =T sync (See) Figure 9 In this way, the arrival time T of the synchronization packet relative to the synchronous BLE can be set.recBLE Fixed broadcast time delay T Delay To achieve playback synchronization. The playback time of the corresponding audio group can be determined by T. PB、i =T recBLE、i +T Delay specified.

[0040] Figure 9 The timing diagram shown in the first example has network latency, where the playback time T is... Pb、1 Synchronization of grouped broadcasts is achieved by referencing the BLE synchronization signal. That is, the broadcast delay (or playback delay) T... delay This refers to the time T after the first BLE synchronization packet has been received. recBLE1 Therefore, in T recUDP1 (in T) recBLE1 Previously received audio data in T Pb1 Playback. Subsequent audio data packets are processed in the following period T. sync It is played at fixed intervals.

[0041] Alternatively, if the I2S broadcast time difference T PB Equal to the BLE cycle time, i.e., T recBLE i+1 -T recBLE i =T sync = T PB This ensures that audio playback is synchronized. In this case, the specific broadcast time delay T for each frame is calculated. delay、i This requires T PB i+1 =T PB i +brT PB It is a known value, and according to UDP frame T recUDP The delay time of each packet is calculated based on the arrival time of each packet. T Pb i+1 =T recUDP i+1 +T delay i+1

[0042] This allows the variable playback delay on each frame to be calculated using the following formula: T delay i+1 =T Pb i +T sync -T recUDP i+1 . Figure 10 The playback time synchronization with variable I2S broadcast delay is shown.

[0043] All other mechanisms, such as PLL adjustment to synchronize the different clocks between the transmitter and receiver, remain in place, such that the receiver clock is adjusted to the transmitter clock via BLE based on the synchronization signal, while the BLE transmitter clock adjusts itself toward the receiver path (in this case, I2S). Besides BitTorrent and Wi-Fi, some interfaces can also be implemented differently. For example, transmission between an audio source device and a BLE transmitter can be accomplished via I2S, SPI, USB, or other known audio-enabled interfaces. Instead of Wi-Fi or Wi-Fi directly, LAN or other high-bandwidth networks can also be used.

[0044] In one implementation method Figure 11 A schematic block diagram of a hub device is shown, which is an audio transmission and playback device 1100, such as a soundbar. An input audio stream Ain is received and optionally processed in audio processing block 1110, such as analysis, equalization, etc. The processed audio data is split into two non-overlapping audio data streams Ain1 and Ain2, and can be packaged in audio packing block 1120. In this example, a packet of one of the audio data streams Ain1 intended for local playback is buffered in audio buffer 1130. Upon receiving a playback trigger signal PT from time synchronization block 1180, the buffered packet is retrieved from the buffer, converted into an analog signal (DAC not shown), and provided to amplifier 1140 and speaker 1150 for local playback at the hub device. Parts of blocks 1130, 1140, and 1120 can thus form an audio playback module. Other audio data streams Ain2 packets are wirelessly transmitted via radio interface 1160 in stream 1111, which has low latency and high bandwidth, to one or more remote audio playback devices, such as via Wi-Fi or Wi-Fi Direct UDP packets. Prior to transmission, as described above, some packets are received from time synchronization block 1180 and added to these packets. In BLE radio interface block 1170, synchronization packets are generated, tagged with synchronization data received from time synchronization block 1180 and transmitted via separate wireless synchronization stream 1101, as described above. Time synchronization block 1180 provides a playback trigger signal PT at a predefined time after the transmission of synchronization packets, such that the audio signal played back through local speaker 1150 is synchronized with the audio signal played back through one or more remote external speakers 320, 321 via wireless receiving streams 1101, 1111, as described above. In one embodiment, the delay of the playback trigger signal PT can be adjusted, for example, via a user interface or by automatic measurement (not shown). Alternatively, the broadcast timing can be adjusted by selecting the marked UDP packets as the 1st, 2nd, ..., nth packets in the broadcast packet block. If the distance between device 1100 and external speakers 320, 321 is very low or very high, the adjustment can be based on the acoustic signal delay.

[0045] In one implementation, at least one of the BLE radio interface 1170 and the high-bandwidth radio interface 1160 creates a hidden exclusive hotspot. For example, radio interface 1160 may omit sending announcement messages, causing the identifier (SSID) of Wi-Fi networks 1111, 3311-3313 to be undistributed and undiscoverable by other wireless devices; thus, the network is hidden. Each external speaker enclosure 320, 321 may receive pairing information for connecting to the hub device (e.g., via BLE channel 1101), or may be otherwise configured to connect to the hidden Wi-Fi network 1111, 3311-3313. The network may be exclusive, as it only allows client devices that identify themselves as the (specific) wireless speaker enclosure to connect to the hub device. Each of the wireless speaker devices 320, 321, 420-523 may include memory for storing configuration data required to access the hidden exclusive Wi-Fi (or Wi-Fi Direct, etc.) network. In one implementation, the same mechanism can, in principle, be used for BLE networks.

[0046] Packets can be marked in various ways: for example, UDP packets can be marked by adding one or more bytes to the packet header, where one or more bytes include an identifier or sequence number. These extra one or more bytes are automatically removed at the receiver before actual audio processing. To mark BLE packets, the standard block length generated by LC3 audio encoding can be increased, for example, by adding bytes that also include an identifier or sequence number. The receiver detects this modification and its reception time, and can then remove the additional bytes.

[0047] Figure 12 A flowchart of a method 1200 for synchronizing audio playback of a first audio device and audio playback of a second audio device is shown in one embodiment. The first audio device has a first radio transmitter and a second radio transmitter, and the second audio device has a first radio receiver and a second radio receiver, such as the audio devices 410 and 420 described above. Method 1200 includes sending a first data stream 1210 to the second audio device via the first radio transmitter. The first data stream is an isochronous stream, i.e., it includes synchronization packets at fixed time intervals, wherein the synchronization packets include synchronization information, such as identifiers. The method also includes sending a second data stream 1220 to the second audio device via the second radio transmitter, wherein the second data stream includes audio data packets.

[0048] In this embodiment, the first audio device is also an audio playback device; therefore, at least a portion of the audio data is locally buffered 1270 within the first audio device. Optionally, the audio data can be processed before or after local buffering, for example, by extracting only the audio data required for local playback at the first audio device. The method also includes retrieving audio data 1280 from the buffer, triggered by a playback trigger signal PT for synchronized playback as described above, and providing them to the amplifier and local speaker. These are included in the first audio device to achieve first playback 1290 via the local speaker.

[0049] In the second audio device, the method further includes receiving a first data stream 1230 at a first radio receiver and receiving a second data stream 1240 at a second radio receiver, as already provided, for example, regarding... Figure 4 and Figure 5 The reception 1230 of the synchronization packet of the first data stream includes, for example, recording the reception time in the reception timestamp and extracting synchronization information. Then, at the second audio device, the received synchronization information and the corresponding recorded reception time, along with the audio data packets of the second data stream, are processed 1250 for use in the second playback 1260, which is synchronized with the first playback as described above. That is, the first playback and the second playback are synchronized in time by determining the playback time of the audio data packets at the second audio device from the time of receiving the corresponding synchronization packet of the first audio stream, wherein the correspondence is determined based on the synchronization data included in the synchronization packet.

[0050] In one embodiment, the present invention relates to a method for synchronizing audio playback of a first audio device and audio playback of a second audio device, wherein the first audio device has a first radio transmitter and a second radio transmitter, and wherein the second audio device has a first radio receiver and a second radio receiver. The method includes transmitting a first data stream to the second audio device via the first radio transmitter and transmitting a second data stream to the second audio device via the second radio transmitter; the first data stream includes at least synchronization packets at fixed time intervals, wherein the synchronization packets include synchronization information, and the second data stream includes audio data packets. The method further includes receiving the first data stream at the first radio receiver and receiving the second data stream at the second radio receiver of the second audio device; processing the first audio data at the first audio device according to the synchronization information for a first playback; and processing the second audio data received in the audio data packets of the second data stream at the second audio device for a second playback synchronized with the first playback. The playback time of the audio data packets at the second audio device is determined based on the reception time of the synchronization packets of the first audio stream and based on the synchronization data included in the synchronization packets.

[0051] In another embodiment, the present invention relates to an audio playback device comprising an audio playback module, a first wireless transmitter, a second wireless transmitter, and a synchronization module. The audio playback module is adapted to buffer first audio data and provide the buffered first audio data at an output for playback on a control signal, wherein the first audio data includes at least a first portion of an audio data input stream. The first wireless transmitter is adapted to transmit a first data stream of packetized signals via a first network, wherein the packetized signals of the first network include co-packets. The second wireless transmitter is adapted to transmit a second data stream of packetized signals via a different second network, wherein the packetized signals of the second network include second audio data, and wherein the second audio data includes at least a second portion of the audio data input stream. The synchronization module is adapted to provide a control signal to the audio playback module at a predetermined time following the transmission of a synchronization packet from the first wireless transmitter.

[0052] In another embodiment, the present invention relates to a wireless speaker device comprising: a first radio receiver for receiving packetized signals via a first network; a second radio receiver for receiving packetized audio data signals via a second network; a first extraction module for extracting a first synchronization signal from the packetized signals received from the first network, wherein the reception time of packets including the first synchronization signal is recorded; a second extraction module for extracting second synchronization data and audio data signals from the packetized signals received from the second network; a buffer for buffering the packetized audio data signals from the second network; a combiner module for comparing the second synchronization data received via the second network with the first synchronization signal received via the first network and for indicating a match; and an audio output module for providing audio data for playback, wherein if the combiner module indicates a match, a defined number of audio data packets are provided at a defined time after the recorded reception time.

[0053] In one embodiment, the present invention relates to a computer-readable data carrier having instructions stored thereon, which, when executed, cause a computer to perform the method described above.

[0054] It is clear that, if applicable, the various implementation methods mentioned in the specification can be combined, even if such combinations are not explicitly mentioned.

Claims

1. A method (1200) for synchronizing audio playback of a first audio device and audio playback of a second audio device, wherein, The first audio device (310, 410) has a first radio transmitter (414) and a second radio transmitter (415), and wherein the second audio device (320, 420) has a first radio receiver (4204) and a second radio receiver (4205), the method comprising: According to the Low Latency Isochronous Wireless Network Protocol, a first data stream (3301) is transmitted via the first radio transmitter (414) to the second audio device (320, 420). The first data stream includes at least synchronization packets (811-816) at fixed time intervals, wherein the synchronization packets include synchronization information, and the synchronization information includes first timestamp data. According to the High Bandwidth Wireless Network Protocol, a second data stream (3311) is transmitted via the second radio transmitter (415) to the second audio device (320, 420). The second data stream includes audio data packets (821-826) and second timestamp data, wherein the High Bandwidth Wireless Network has a higher bandwidth than the Low Latency Isochronous Wireless Network. The first data stream (3301) is received (1230) at the first radio receiver (4204), and the second data stream (3311) is received (1240) at the second radio receiver (4205) of the second audio device (320, 420). At the first audio device (310, 410), first audio data for first playback is processed (1270, 1280) according to the synchronization information, the processing including buffering (1270); and At the second audio device (320, 420), based on the first timestamp data and the second timestamp data, for a second playback (1260) synchronized in time with the first playback, the second audio data received in the audio data packets of the second data stream (3311) is processed and buffered (1250) for the second playback (1260) that is synchronized in time with the first playback. The playback time (T) of the audio data packets at the second audio device (320, 420) is specified. PB1 ) is based on the reception time (T) of the synchronization packet (811) of the first audio stream (3301). recBLE1 And determined based on the synchronization data included in the synchronization group.

2. The method according to claim 1, wherein, The first radio transmitter (414) transmits the first data stream (3301) via an isochronous channel of a Bluetooth Low Energy (BLE) broadcast stream according to at least Bluetooth v5.2, and wherein the second data stream (3302) is not a BLE stream.

3. The method according to claim 2, wherein, The second data stream (3302) is a Wi-Fi or Wi-Fi direct stream according to IEEE 802.11x, where x is one of a, ac, ax, ad, b, g or n.

4. The method according to any one of claims 1 to 3, wherein, The received audio data is stored in a buffer in the second audio device (320, 420), and wherein the stored audio data is delayed by a predefined delay (T) relative to the reception time of the synchronization packet. Delay (It was replayed.) 5. The method according to any one of claims 1 to 4, wherein, The third audio device (321, 421) has a third radio receiver (4214) and a fourth radio receiver (4215), and the method further includes the following steps: The first data stream (3301) is also received at the third radio receiver (4214) of the third audio device (321, 421). The second data stream (3311) is also received at the fourth radio receiver (4215) of the third audio device (321, 421). A third playback of third audio data received in at least a portion of the audio data packets of the second data stream (3311) is prepared at the third audio device, wherein the third playback is time-synchronized with the first playback and the second playback, and wherein the third audio device (321, 421) determines the playback time of the audio data packets based on the reception time of the synchronization packets of the first audio stream (3301) and the synchronization data included in the synchronization packets.

6. The method according to claim 5, wherein, The second data stream (3311) is a multicast or broadcast stream.

7. The method according to any one of claims 1 to 6, wherein, The synchronization information in the synchronization packet is the value in the packet header.

8. An audio playback device (410, 510, 1100), comprising: An audio playback module (1120, 1130, 1140) is adapted to buffer first audio data, the first audio data including at least a first portion (Ain1) of an audio data input stream (Ain), and is adapted to provide the buffered first audio data at an output (1150) for playback upon receiving a control signal (PT); A first wireless transmitting device (414, 1170) is adapted to transmit a first data stream (1101) of a packetized signal of a first network (3301) according to a low-latency isochronous wireless network protocol of the first network, the packetized signal including a synchronization packet containing first timestamp data. A second wireless transmitting device (415, 1160) is adapted to transmit different second data streams (1111) of packetized signals according to a high-bandwidth wireless network protocol of a second network (3311), the packetized signals of the second network including second audio data and second timestamp data, the second audio data including at least a second portion (Ain2) of the audio data input stream (Ain); and A synchronization module (1180) is adapted to provide the control signal (PT) to the audio playback module (1130) at a predefined time after the transmission of a synchronization packet by the first wireless transmitting device (414, 1170).

9. The audio playback device according to claim 8, wherein, The first network is an isochronous channel of a Bluetooth Low Energy (BLE) network based on at least Bluetooth v5.2, wherein the synchronization packets are transmitted in broadcast mode of the BLE network, and the second network is a Wi-Fi or Wi-Fi Direct network based on IEEE 802.11x, wherein x is one of a, ac, ax, ad, b, g, or n.

10. The audio playback device according to claim 8 or 9, wherein, The device is a soundbar comprising at least one amplifier and multiple speakers, wherein the audio content of the first portion (Ain1) and the second portion (Ain2) of the audio data input stream does not overlap.

11. The audio playback device according to any one of claims 8 to 10, wherein, The first wireless transmitting device (414, 1170) or the second wireless transmitting device (415, 1160) creates a hidden exclusive hotspot, wherein no network announcement message is sent, so that the first network connection or the second network connection (3301, 3311) can only be accessed by the designated device and cannot be discovered by devices other than the designated device.

12. The audio playback device according to any one of claims 8 to 11, wherein, The first wireless transmitting device (414, 1170) is further adapted to transmit packetized audio signals for the subwoofer in the first data stream (1101).

13. A wireless speaker device (420), comprising: A first radio receiver (4204) is used to receive packetized signals according to a low-latency isochronous wireless network protocol of a first network (3301, 614); A second radio receiver (4205) is used to receive packetized audio data signals according to a high-bandwidth wireless network protocol of a second network (3311, 615), wherein the high-bandwidth wireless network has a higher bandwidth than the low-latency isochronous wireless network. The first extraction module (6404) is used to extract a first synchronization signal from the packetized signal received by the first network (3301, 614), the first synchronization signal including first timestamp data, wherein the reception time of the packet including the first synchronization signal is recorded. The second extraction module (6405) is used to extract second synchronization data from the packetized signals received by the second network (3311, 615), the second synchronization data including second timestamp data; A buffer for buffering the packetized audio data signal of the second network (3311, 615); A combiner module (6401) is configured to compare the second synchronization data received via the second network (3311, 615) with the first synchronization signal received via the first network (3301, 614), and to indicate a match; and An audio output module (6401) is used to provide audio data for playback, wherein, if the combiner module indicates a match, a defined number of audio data packets are provided at a defined time after the recorded reception time.

14. The wireless speaker device according to claim 13, wherein, The first network (3301, 614) is an isochronous channel of a Bluetooth Low Energy (BLE) network based on at least Bluetooth v5.2, the packetized signal of the first network (3301, 614) is transmitted in broadcast mode, and the second network (3311, 615) is a Wi-Fi or Wi-Fi Direct network based on IEEE 802.11.x, where x is one of a, ac, ax, ad, b, g or n.

15. The wireless speaker device according to claim 13 or 14, further comprising a memory having configuration data for accessing a hidden exclusive hotspot of the second network (3311, 615), wherein, No announcement message was sent for the second network, and other devices could not discover the second network.

Images