Wireless conference system and method for transmitting delay-sensitive audio data packets

Abstract

A wireless conference system (100) includes an access point (101; 300) and a plurality of conference units (111-115; 200), configured for bidirectional TDMA-based wireless communication of delay-sensitive audio data packets. The access point (101; 300) and the conference units (111-115; 200) include an actively synchronized clock (203; 303), the clock (203; 303) generating a local audio clock signal for processing audio data packets and a local synchronization clock signal for TDMA-based wireless communication. The conference units (111-115; 200) and the access point (101; 300) include a transceiver (201; 301). The receiver (220; 320) comprises a packet loss detection unit (222; 322) configured to detect the loss of an audio data packet, the packet loss detection unit comprising: means (223; 323) for determining an expected arrival time of the audio data packet based on a local synchronization clock signal and a predetermined expected transmission delay, and means (224; 324) for detecting the loss of the audio data packet if the audio data packet does not arrive before the expected arrival time. The receiver (220; 320) further comprises a packet loss concealment unit (225; 325) configured to generate a replacement packet for the lost audio data packet.

Description

Technical Field

[0001] The present invention generally relates to a wireless conferencing system for use in a conference room for enabling multiple users to participate in a conference. Such a conferencing system typically includes: a central access point under the control of a host or chairperson of the conference; and a plurality of conferencing units for use by the users, the plurality of conferencing units being mounted on or integrated into user tables in the conference room. The present invention more particularly relates to delay-sensitive two-way audio transmission between conferencing units and the access point in such a conferencing system. Background Art

[0002] Conference rooms are often equipped with conference systems that enable a large number of users, such as tens to hundreds of users, to participate in a single conference. A wireless conference system often includes a central access point and a plurality of conference units coupled to the central access point. For example, a separate conference unit may be provided for each user, or a conference unit may be shared by two users in adjacent seats in a conference room. Each conference unit often has a microphone connector, a built-in speaker, and one or more headphone connectors, the microphone connector being used to connect to a microphone such as a gooseneck microphone, and the headphone connector enabling connection to headphones. Each conference unit also has a controller, a data processor, and a transceiver configured for wireless two-way data transmission with a similar transceiver in the central access point. The transceiver of the conference unit includes a transmitter capable of uplinking digital data packets to a receiver, which forms part of the transceiver in the central access point. Similarly, the transceiver of the central access point includes a transmitter capable of downlinking digital data packets to a receiver, which forms part of the transceiver of the conference unit. In a typical situation where a single participant is speaking and other participants are listening, the audio resulting from the participant's speech, captured by the conference unit's microphone, is digitized, packetized, and transmitted upstream to a central access point. The access point processes the received audio packets, such as mixing the received audio packets with other audio input sources, and distributes the processed audio downstream to all other conference units in the conference room, where receivers in the other conference units should receive the audio and processors should process the received audio packets for playback through headphones. This includes transmitting the audio downstream to the initiating conference unit from which the audio packets were received so that the speaker can hear his / her own speech through the headphones.

[0003] Wireless two-way audio communication between conference units and access points needs to meet low latency requirements, sometimes also referred to as real-time requirements. Depending on the applicable quality standards, the end-to-end delay of the round-trip time (RTT), i.e. the maximum delay for an audio packet to be transmitted back and forth between a conference unit and a central access point in a conference system (including all processing performed in the conference unit and the central access point), can, for example, be limited to a value in the range from 10 milliseconds to 30 milliseconds, such as 15 milliseconds. Audio packets that arrive later than the maximum acceptable delay will not be used for playback. When an audio packet does not arrive in time (packet loss), the lost packet will be replaced by a packet determined by a packet loss concealment algorithm to avoid audible artefacts that can be perceived by conference participants whose conference units did not receive the audio packet in time. In order for this packet loss concealment algorithm to be able to generate replacement packets in time, it is important to detect packet loss as early as possible.

[0004] In the example conference system, audio packets are transmitted in the time slots of the TDMA frame via the Wi-Fi connection between the conference unit and the central access point. The TDMA frame can, for example, have a length of 5 milliseconds, which is subdivided into 10 time slots, each of which has a length of 500 microseconds. The conference unit should capture 5 milliseconds of audio, which is usually the speech of the conference participant, digitize the 5 milliseconds of audio into a digital audio packet, and transmit the digital audio packet in a single time slot of the TDMA frame. The packet generation in this example has introduced a delay of 5 milliseconds. In the case of applying soft TDMA to avoid interfering with other users of the channel, soft TDMA introduces variable delay or jitter, that is, introduces so-called Listen-Before-Talk jitter or LBT jitter, wherein the above-mentioned soft TDMA is such a TDMA scheme: in this scheme, the time slot is fixedly assigned to the transceiver, but the transceiver listens to the interference information flow in such a time slot in the channel before transmitting data in such a time slot to determine whether the channel is idle. The interrupt mechanisms that control packet processing on both the transmitter and receiver sides, packet and audio processing times, packet propagation times through the wireless channel, and any synchronization inaccuracies between the transmitter and receiver introduce further delays, which combined with LBT jitter can be as high as 2 milliseconds for one-way transmission. Therefore, in a conferencing system implementing a soft TDMA scheme with a 5 millisecond TDMA frame length, the one-way delay experienced by the audio packets can be as high as 7 milliseconds.

[0005] A straightforward method of detecting packet loss to initiate packet loss concealment relies on sequence numbering of digital audio packets. When a receiver receives an audio packet with the following sequence number, then the packet is considered lost. However, when relying on sequence numbering, the earliest moment when a packet loss can be detected is the point in time when the next packet is received. In the above example with a TDMA frame of 5 milliseconds, in order to generate and transmit the next audio packet, at least another 5 milliseconds will pass, leaving insufficient time for the receiver to apply packet loss concealment and generate a replacement packet for the previously lost audio packet.

[0006] A solution to enhance packet loss detection that relies on the sequence numbering of audio packets can be obtained by reducing the TDMA frame length and, therefore, the length of the audio segments embedded in a single audio packet. However, reducing the packet length will have a negative impact on the capacity of the wireless channel. The overhead / payload ratio will increase because: it can be assumed that the overhead per packet remains constant, so the effective capacity of the wireless link will be reduced. In addition, when packets are shortened, the packet processing speed must be increased, requiring a more expensive processor.

[0007] U.S. patent application US2015 / 0201289 A1, entitled "Method and Apparatus for Rendering Audio in Wireless Hearing Instruments," recognizes the problem of audio packet loss over a wireless communication link in a different application, and more specifically, recognizes the problem of audio packet loss over a wireless communication link in a hearing instrument application, where the hearing instrument is worn on one or both sides of a person's head to assist a patient with hearing loss. As described in paragraph

[0030] of US2015 / 0201289 A1, packets may be retransmitted from an audio source device in an effort to improve the overall packet error rate performance of the wireless link. Retransmissions may occur due to the audio source device failing to receive an acknowledgment from an audio sink device, or they may be sent unconditionally in the form of multiple retransmissions. The suggestion to send acknowledgments and retransmissions indicates that latency constraints in hearing instrument applications are not as stringent as in conferencing systems. As shown in the above example, the quality constraints of the conferencing system do not allow for retransmissions. Paragraphs

[0029] and

[0031] of US2015 / 0201289 A1 also describe that link layer information shared between the radio of the wireless communication circuit and the DSP (digital signal processor) of the processing circuit can be used to determine the packet concealment strategy. The TDMA mechanism deployed by the radio has an inherently good timing mechanism that can plan the arrival of packets. If the planned reception event occurs and no packet is received, the radio notifies the DSP of the lost packet to allow the DSP to insert a packet loss concealment information frame.

[0008] US patent application US2019 / 0104423 A1, entitled "Ultra-Low Latency Audio over Bluetooth", recognizes the low latency requirement for transmitting audio packets over a wireless connection in a different application, more precisely, in a Bluetooth audio transmission application between a device and a wireless headset or wireless earbuds. In addition, US2019 / 0104423 A1 relies on confirmation and retransmission, and teaches to enhance confirmation by combining BTC (classic Bluetooth) packets and BLTE (low power Bluetooth) packets in a single Bluetooth frame, by limiting the number of retransmissions and packet masking per frame period to an upper limit, and by using time-efficient audio encoding and decoding to implement FEC (forward error correction) such as RS (Reed-Solomon), to reduce the latency of wireless audio packet transmission. This is described, for example, in paragraph

[0005] of US2019 / 0104423 A1.

[0009] It is an object of the present invention to disclose embodiments of a conferencing system that solves or mitigates one or more of the above-mentioned disadvantages of existing solutions. More specifically, it is an object of the present invention to disclose embodiments of a conferencing system in which audio packet loss due to wireless transmission from a conferencing unit to a central access point or from an access point to a conferencing unit is detected more quickly so that packet masking can be initiated more quickly to allow the conferencing system to meet latency requirements and quality standards applicable to conferencing systems. It is a further object of the present invention to disclose such embodiments of a conferencing system without negatively impacting the effective bandwidth of the wireless link. Summary of the invention

[0010] According to an embodiment of the present invention, a wireless conference system suitable for enabling multiple users to participate in a conference in a conference room achieves the above-mentioned purpose. The wireless conference system includes an access point and multiple conference units.

[0011] - wherein the access point and one or more of the conference units include a transceiver configured for bidirectional TDMA-based wireless communication of delay-sensitive audio data packets between the one or more conference units and the access point, the transceiver including a transmitter and a receiver;

[0012] - wherein the access point and the one or more conference units include actively synchronized respective clocks, the clocks in the respective clocks being configured to generate a local audio clock signal and a local synchronization clock signal, the local audio clock signal being used to locally process audio data packets, the local synchronization clock signal being used for TDMA-based wireless communication, wherein the one or more conference units include a clock synchronization unit, the clock synchronization unit being configured to: actively synchronize the respective clocks of the one or more conference units with the clock in the access point based on a timestamp periodically broadcast by the access point;

[0013] - wherein the receiver comprises a packet loss detection unit, the packet loss detection unit being configured to: detect the loss of an audio data packet transmitted from the conference unit to the access point; or detect the loss of an audio data packet transmitted from the access point to the conference unit, the packet loss detection unit comprising:

[0014] - means configured to determine the expected arrival time of the audio data packets based on the local synchronous clock signal and a predetermined expected transmission delay, and

[0015] - means configured to detect that an audio data packet is lost if the audio data packet does not arrive before its expected arrival time; and

[0016] - wherein the receiver comprises a packet loss concealment unit configured to generate a replacement packet for the audio data packet detected as lost by the packet loss detection unit.

[0017] Therefore, according to an embodiment of the present invention, the access point and one or more conference units are equipped with clocks or timers that are actively synchronized with high accuracy. Therefore, the receiver knows the transmission time of the data packet, and the receiver can determine the expected arrival time of the data packet. Due to the actively synchronized clock, all transceivers know the start and end times of the TDMA frames and the time slots within these TDMA frames, which depends on a certain clock synchronization tolerance. Therefore, the receiver can derive from the local clock signal of the receiver what the expected arrival time of the audio data packet is, which is the transmission time of the audio data packet plus the expected transmission delay. The transmission time is known from the synchronization clock used by the transmitter and receiver for TDMA-based wireless communication. In fact, each conference unit knows the TDMA schedule, and its synchronization clock allows to derive their position in the TDMA frame. The expected transmission delay takes into account the total time required for interrupt handling on the transmitter side and the receiver side, the propagation of the audio data packet on the wireless link, jitter of various properties and inaccuracies of various properties, and therefore the expected transmission delay represents an upper limit on the total time between the packet processor on the transmitter side that releases the audio data packet for transmission and the packet processor on the receiver side that receives the same audio data packet for processing. If a data packet is not received before the expected arrival time, packet loss concealment is enabled at the receiver side to generate a replacement packet.

[0018] Since the transmission delay in one direction in a conference system, i.e. the total delay due to propagation, jitter, interrupt handling, processing and synchronization inaccuracies, is in the range of several milliseconds, typically between 1 and 3 milliseconds, packet loss concealment in an embodiment according to the present invention can be activated faster than in the following known conference systems: the known conference system is a known conference system that relies on confirmation, packet retransmission and receiver-side timeout for packet loss detection; or a known conference system that relies on audio packet numbering and based on the number of audio packets lost on the receiver side for packet loss detection. Starting packet loss concealment faster allows reducing the overall end-to-end system latency and reduces the risk that packet loss concealment cannot generate replacement packets for insertion and playback of the audio stream in time, thereby reducing the number of audible artifacts. In addition, embodiments of the conference system according to the present invention do not require reducing the TDMA frame length and / or audio data packet length. Measures such as reducing the TDMA frame length and / or the audio data packet length can also enable faster detection of packet loss and thus faster initiation of packet loss concealment, but such measures come at the expense of an increased overhead / payload ratio and therefore a reduction in the effective bandwidth of the wireless link (the amount of useful data that can be transmitted per unit time, i.e., the amount of audio samples that can be transmitted per unit time).

[0019] A conference unit in the context of the present invention includes any unit mounted on or integrated on a user's table in a conference room. Such a conference unit typically includes a built-in microphone array or an audio input connector, such as a connector for a gooseneck microphone, which can be used by a single user or can be shared between two users when the conference unit is installed between the seats of two adjacent users. A conference unit typically also includes one or more audio output connectors, such as a headphone connector for one or more users, and the conference unit typically also includes a built-in speaker. It should be noted that the audio input connector and the audio output connector can also be integrated into a single connector. The conference unit may also include a connector for a camera or other sensor, and may be equipped with a display and physical or virtual (i.e., displayed) buttons to control audio input (e.g., mute a microphone), to control audio output (e.g., control the volume of headphones), to control other sensors, to interact with the chairman (e.g., request to speak), and / or to serve as a voting button. The conference unit may also include an indicator, such as a colored LED that indicates to the chairman, the user of the conference unit, and / or other conference participants what the status of the conference unit is. The conference unit also has a processor for digitizing and packetizing the audio captured by a microphone connected to an audio input connector of the processor, and a wireless transmitter for transmitting the audio data packets to a central unit, the so-called access point. The conference unit also has a receiver for receiving audio data packets from the central unit, and a processor for unpacking the received audio data packets and generating an (analog or digital) audio stream output via the audio output connector. The transmitter and the receiver together form a transceiver for two-way communication with the central unit. The processor generating the audio data packets for transmission and the processor processing the received audio data packets can be integrated to form a single physical processor.

[0020] The access point in the context of the present invention constitutes a central unit, which is managed and controlled by the chairman or the conference organization. The access point provides two-way wireless communication with all conference units in the conference room in a TDMA-based manner. The access point includes a transmitter and a receiver, which transmit and receive audio packets in the time slots of the TDMA frame respectively. The audio data packets received in different time slots come from different conference units. The access point usually has a processor, which is used to process the audio data packets received from different conference units, and select or combine (usually process) the audio packets from one or more conference units into a single audio stream for its transmitter to transmit to the conference unit. The access point can receive input from the chairman, who controls the meeting and decides which conference participants are allowed to speak at any point in time. In addition to the conference unit, the audio stream can also be provided to the interpreter unit to enable the interpreter or translator to upload the interpretation or translation of the audio stream, which is further distributed to the conference unit by the central access point.

[0021] The wireless conference system in the context of the present invention comprises: a group of conference units installed in a conference room; and access points to which the conference units are connected in a multipoint-to-point manner via wireless bidirectional links, which are shared in a TDMA-based manner.

[0022] Packet loss concealment in the context of the present invention includes any algorithm or technique that generates audio samples to replace lost audio packets, i.e., audio data packets that never arrive at the receiver or that arrive at the receiver late (due to the late arrival, the audio data packets can no longer be processed and integrated into the audio stream in a timely manner). Packet loss concealment techniques typically use the most recently received audio packets to generate replacement packets for the lost audio data packets. Packet loss concealment techniques typically seek to avoid or minimize audible effects caused by the replacement, and packet loss concealment techniques may be based, for example, on the frequency or pitch of the most recent audio samples.

[0023] In an embodiment of the wireless conferencing system according to the present invention, the access point and one or more conferencing units are configured not to acknowledge receipt of audio data packets.

[0024] In fact, the preferred embodiment of the wireless conference system according to the present invention implements the following protocol in which the reception confirmation for the audio data packets is not required or is disabled. The confirmation or the absence of confirmation will trigger the retransmission, but any attempt to recover the lost audio data packets by retransmission will delay the activation of the packet loss concealment and thus increase the overall latency of the wireless conference system.

[0025] In an embodiment of the wireless conferencing system according to the present invention, the access point and one or more conferencing units are configured not to retransmit lost audio data packets.

[0026] In fact, the preferred embodiment of the wireless conference system according to the present invention implements a protocol that does not require retransmission of audio data packets in the uplink and downlink directions. As mentioned above, retransmission attempts will delay the activation of packet loss concealment and thus increase the overall latency of the wireless conference system.

[0027] In an embodiment of the wireless conferencing system according to the present invention, for wireless transmission from the conferencing unit to the access point and from the access point to the conferencing unit, delay-sensitive audio data packets have a round-trip time delay limit of 25 milliseconds.

[0028] Therefore, an embodiment of the wireless conference system may set a limit of 25 milliseconds on the round trip time of audio data packets. In other words, this limit sets the maximum acceptable delay for a conference participant between speaking through a microphone connected to his conference unit and hearing his / her own speech in headphones connected to the same conference unit. It should be understood by those skilled in the art that alternative embodiments of the wireless conference system according to the present invention may achieve any other round trip time delay limit of less than 25 milliseconds. Such alternative embodiments set higher quality standards for audio, provided that underlying technologies such as packet loss concealment algorithms are able to meet lower round trip time limits.

[0029] In an embodiment of the wireless conferencing system according to the present invention, for wireless transmission from the conferencing unit to the access point and from the access point to the conferencing unit, delay-sensitive audio data packets have a round-trip time delay limit of 15 milliseconds.

[0030] In fact, the preferred embodiment of the wireless conferencing system sets a 15 millisecond limit on the round-trip time of audio packets between the conference units and the central access point. It will be appreciated by those skilled in the art that a round-trip time limit lower than 15 milliseconds may be implemented to minimize audible artifacts, but this would risk having to deploy more expensive higher capacity packet processors and / or having to implement shorter TDMA frames and shorter audio packet lengths, which would negatively impact the effective bandwidth of the wireless link.

[0031] In an embodiment of the wireless conference system according to the present invention, the TDMA-based wireless communication uses a 5 millisecond TDMA frame.

[0032] Therefore, the preferred embodiment implements a TDMA frame with a length of 5 milliseconds. This means that the audio data packet also includes audio samples spanning 5 milliseconds. At an audio sampling rate of 48kHz, this means that each audio data packet includes 240 audio samples. These 240 audio samples constitute the payload portion of the audio data packet. In addition, the audio data packet includes overhead. It should be understood by those skilled in the art that shorter TDMA frames and shorter audio data packets have a negative impact on the effective bandwidth of the wireless link: the payload portion of the audio data packet will be reduced while its overhead portion will remain unchanged. It should also be understood by those skilled in the art that longer TDMA frames and longer audio data packets will complicate the task of packet processors and packet loss concealment techniques for timely generating audio streams without audible artifact components. In other words, the preferred 5 millisecond TDMA frame length is the result of a trade-off between the effective bandwidth on the wireless link between the conference unit and the access point and the quality standards that the conference system must meet.

[0033] In an embodiment of the wireless conference system according to the present invention, the transmitter is configured to: before transmitting the audio data packet in the transmitter, listen to the interference information flow in the designated time slot in the TDMA frame.

[0034] In practice, a preferred embodiment of the present invention implements a so-called soft TDMA scheme, in which the time slots forming part of a TDMA frame are fixedly assigned to the conference units according to a predefined scheme, but in which the transmission time within each time slot of the TDMA frame is flexibly determined by the conference unit based on the LBT (Listen Before Talk) mechanism. The advantage brought by the LBT mechanism is that the wireless channel can be used simultaneously by different wireless systems, thereby making more efficient use of the overall available bandwidth of the wireless channel. On the other hand, the LBT mechanism introduces jitter or uncertainty to the receiver in terms of the arrival time of the audio data packets, because the transmitter will only transmit the audio data packets after it is sure that the time slot is not used by any other transmitter inside or outside the wireless conference system. The LBT jitter in the conference system is usually around 1 to 2 milliseconds.

[0035] In an embodiment of the wireless conference system according to the present invention, the wireless communication uses Wi-Fi (IEEE802.11).

[0036] In fact, the Wi-Fi protocol can be used very well as a wireless technology to connect conference units in a conferencing system to a central access point. Wi-Fi enables coverage of the conference room area with a single access point (or at most a few access points) and provides multiple channels or frequency bands to cope with interference.

[0037] In an embodiment of the wireless conferencing system according to the present invention, one or more conferencing units include a clock synchronization unit, which is configured to actively synchronize the corresponding clock of the conferencing unit with the clock in the access point based on a timestamp inserted in a beacon message broadcast periodically by the access point.

[0038] Therefore, a preferred way of achieving active synchronization between the clocks of the access points and the clocks of the conference units relies on beacon messages that are broadcast periodically by the access points. In the case where the access points rely on the Wi-Fi protocol, the beacon messages may, for example, correspond to messages in which the access point broadcasts its SSID periodically to enable devices to detect the presence of the access point and establish a connection. Such beacon messages are usually transmitted with a lower modulation scheme compared to other messages, i.e. using a less complex constellation scheme and increased redundancy, making these beacon messages more robust: they have a greater range and a reduced risk of being lost before reaching the receiver. The use of beacon messages to transmit time values ​​or timestamps between the access points of the conference system and the conference units thus makes the active clock synchronization essential to the present invention more robust. In the case of Wi-Fi, an additional advantage is that the beacon messages transmitted therein are backwards compatible with earlier flavors or versions of Wi-Fi technology.

[0039] In an embodiment of the wireless conference system according to the present invention, the predetermined expected transmission delay is determined as the sum of propagation delay, jitter, interrupt handling delay, processing delay and clock synchronization inaccuracy.

[0040] In fact, in a preferred embodiment, the overall inaccuracy of the reception time of the audio data packet includes a first contribution caused by effective propagation through the air, typically in the range of 100 to 200 microseconds. In an embodiment implementing a soft TDMA scheme, the overall inaccuracy of the reception time of the audio packet may also include a second contribution caused by LBT jitter. The second contribution is significant, and for a 5 millisecond TDMA frame, the second contribution is typically in the range of 1 to 1.5 milliseconds. The overall inaccuracy of the reception time of the audio packet also includes a third, fourth and fifth contribution, which are respectively caused by: interrupt handling on the transmitter side and the receiver side, the above-mentioned interrupt handling is the processing of interrupt signals indicating that an event such as receiving a packet has occurred; the processing time of packets and audio; and the synchronization inaccuracy between the transmitter clock and the receiver clock, which may depend on the active synchronization mechanism deployed.

[0041] In an embodiment of the wireless conferencing system according to the present invention, the jitter comprises a listen-before-talk jitter contribution.

[0042] As mentioned above, jitter of various natures may lead to overall inaccuracy in the timing of audio packet reception. In case of applying soft TDMA schemes, a significant jitter contribution comes from the listen-before-talk behavior of the transmitter, where the transmitter waits until the channel is idle before transmitting in the scheduled time slot.

[0043] In an embodiment of the wireless conferencing system according to the present invention, the predetermined expected transmission delay is set to a value between 1.5 milliseconds and 2 milliseconds.

[0044] Tests have shown that in a conferencing system using Wi-Fi and a soft TDMA scheme with a time frame of 5 milliseconds for communication between conferencing units and a central access point of the conferencing system and using timestamps in Wi-Fi beacons for active synchronization, an overall expected transmission delay set to 1.5 milliseconds allows the present invention to be implemented in a manner that achieves substantial benefits in detecting and early enabling of packet loss concealment without sacrificing the effective bandwidth on the wireless link.

[0045] According to a second aspect, the present invention relates to a method for transmitting delay-sensitive audio data packets between one or more conference units and an access point in a conference system, the conference system being adapted to enable a plurality of users to participate in a conference in a conference room, the method for transmitting delay-sensitive audio data packets comprising bidirectional TDMA-based wireless communication of the audio data packets, the method further comprising:

[0046] - Actively synchronize one or more conference units with corresponding clocks in an access point, wherein the clocks in the corresponding clocks are configured to generate a local audio clock signal and a local synchronization clock signal, wherein the local audio clock signal is used to process audio data packets locally, and the local synchronization clock signal is used for wireless communication based on TDMA, wherein the one or more conference units include a clock synchronization unit, wherein the clock synchronization unit is configured to: actively synchronize the corresponding clocks of the one or more conference units with the clock in the access point based on a timestamp periodically broadcast by the access point;

[0047] - Detecting the loss of an audio data packet transmitted from a conference unit to an access point, or detecting the loss of an audio data packet transmitted from an access point to a conference unit, the detection comprising:

[0048] - determining the expected arrival time of the audio data packets based on the local synchronous clock signal and a predetermined expected transmission delay, and

[0049] - detecting audio packet loss if an audio packet does not arrive before its expected arrival time; and

[0050] Packet loss concealment is used to generate replacement packets for audio data packets detected as lost. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 Schematically illustrates an implementation of a wireless conference system 100 according to the present invention;

[0052] Figure 2 is a functional block diagram of an embodiment of a conference unit 200 forming part of an embodiment of a wireless conference system 100 according to the present invention;

[0053] Figure 3 is a functional block diagram of an embodiment of an access point 300 forming part of an embodiment of a wireless conferencing system 100 according to the present invention;

[0054] Figure 4 is a pseudo-flow chart illustrating an embodiment of a method for transmitting delay-sensitive audio data packets according to the present invention; and

[0055] Figure 5 An example implementation of a suitable computing system 500 for performing one or more steps in embodiments of the invention is shown. DETAILED DESCRIPTION

[0056] Figure 1 A wireless conferencing system is shown that includes an access point AP or 101 and five conferencing units CU1-CU5 or 111-115. The conferencing units 111-115 capture audio signals from various conference participants using microphones, digitize the audio signals into audio data packets, and transmit the audio data packets to the access point 101. The access point 101 receives audio packets from one or more conferencing units, processes the audio packets, and generates new audio packets for distribution to all conferencing units 111-112, so that the conferencing units 111-115 receive an audio stream corresponding to an audio signal generated by a single selected conference participant (the selection may be made by a chairperson controlling the access point 101), so that the conferencing units 111-115 receive a combined audio signal generated by multiple selected conference participants (again, the selection may be made by a chairperson controlling the access point 101), or so that the conferencing units 111-115 receive processed audio signals, such as interpreted or translated audio signals, interpretations or translations superimposed on original audio signals, etc. In Figure 1 In an embodiment, the conference units 111-115 use Wi-Fi for wireless and bidirectional communication with the access point 101. The Wi-Fi channels selected for the uplink direction, that is, the direction from the conference units 111-115 to the access point 101, are shared using a soft time division multiple access (TDMA) protocol. The soft TDMA protocol requires that the transmitters in the conference units first listen to see if the channel is idle before transmitting in the designated time slot. Assume Figure 1The TDMA frame in the conferencing system 100 has a length of 5 milliseconds. The TDMA frame consists of 10 time slots of 0.5 milliseconds. If the channel is occupied, the transmitter will wait until the channel is free. This listen-before-talk mechanism or LBT mechanism introduces jitter, i.e., uncertainty in the time when a packet arrives at the access point 101 within a specified time slot. In the downlink direction, i.e., from the access point 101 to the conferencing units 111-115, audio data packets are also transmitted in the time slots of the TDMA frame. Typically, the same TDMA frame is used for uplink and downlink transmissions, and the same Wi-Fi channel is used for uplink and downlink transmissions. The transmitter in the access point 101 also implements the LBT mechanism to avoid interference with other Wi-Fi networks using the same channel.

[0057] Figure 2 The functional building blocks of a conference unit CU or 200 relevant to the present invention are shown. Figure 2 The conference unit 200 indicates Figure 1 A possible implementation of each conference unit 111 - 115 is depicted in FIG. The conference unit 200 includes a transceiver 201 , an antenna 202 for wireless transmission, a clock 203 , a clock synchronization logic 204 , an audio input connector or microphone connector 205 , and an audio output connector or headphone connector 206 . Figure 2Also schematically shown are a microphone 250 and a headset 260, the microphone 250 being connected to the microphone connector 205 to capture audio signals such as speech from conference participants, and the headset 260 being connected to the audio output connector 206 or a built-in speaker to enable conference participants to follow the conference conversation. The transceiver 201 includes a transmitter 210 and a receiver 220. The transmitter 210 includes a processor 211, which is configured to receive an audio signal captured by the microphone 250, digitize the audio signal, and package the digitized audio signal into audio data packets, each audio data packet containing 5 milliseconds of captured speech. The processor 211 forwards the generated audio data packets to the antenna 202, i.e., the radio interface of the antenna, which transmits the audio data packets in the uplink direction to the access point of the conference system using the Wi-Fi protocol with soft TDMA and LBT mechanisms as explained above. In the downlink direction, the antenna 202 periodically receives Wi-Fi beacon messages from the access point associated with the antenna 202. The beacon message includes a timestamp or time value that the clock synchronization unit 204 uses to synchronize the clock 203 with the clock in the access point. The estimated time of arrival unit 223 estimates the arrival time of the next audio data packet using a clock signal generated locally by the clock 203 in the conference unit 200 but actively synchronized with the clock in the access point through the beacon message. Since the clock 203 is actively synchronized with the clock used in the access point for the transmission of audio data packets, the ETA unit 223 can accurately determine the time when the next audio data packet is ready for transmission at the access point. The ETA unit 223 increases this time by a predetermined transmission delay, which is an acceptable time required to cope with interruptions on the transmitter and receiver sides, propagation over the air through the Wi-Fi channel, and possible jitter due to, for example, the LBT mechanism implemented on the transmitter side. The transmission delay may further indicate an inaccurate clock synchronization between the clock 203 and the clock in the access point with which it is actively synchronized. In Figure 2In the example of , the transmission delay is predetermined to correspond to 1.5 milliseconds. The locally generated clock signal and the predetermined transmission delay enable the ETA unit 223 to defensively estimate the arrival time of the next audio data packet. The loss detection unit 224, which receives an interrupt signal from the processor 221 when the audio data packet arrives, verifies whether the audio data packet is received before the next estimated arrival time of the audio data packet. If the audio data packet is received in time, the processor 221 will process the received audio data packet so that the audio samples contained therein can be output as part of the audio stream through the audio connector 206. Each time an audio data packet is not received before the estimated arrival time of the audio data packet, the loss detection unit 224 triggers a packet masking unit 225 formed as part of the receiver 220 to generate a replacement packet and provide the replacement packet to the processor 221 for insertion into the audio stream output via the audio connector 206. The packet masking unit 225 applies a processing-intensive masking algorithm, typically using an earlier received audio data packet to generate a replacement audio data packet, so that when such a replacement audio data packet is inserted into the received audio stream to replace the lost audio data packet, no audible artifact components are generated. Due to active clock synchronization and arrival time estimation of audio packets, packet loss concealment can be triggered early. This enables the conference unit 200 to meet the latency requirements set for conference systems, which are typically in the range of 15 ms to 25 ms for the round trip time of audio packets transmitted back and forth between the conference unit 200 and the access point.

[0058] Figure 3 The functional building blocks of an access point AP or 300 associated with the present invention are shown. Figure 3 The access point 300 indicates Figure 1300. The access point 300 comprises a transceiver 301, an antenna 302 for wireless transmission, a clock 303 and an interface 304 for the chairperson or organization of the conference. The transmitter 301 comprises a transmitter 310 and a receiver 320. The transmitter 310 comprises a processor 311, which is configured to generate audio data packets, each of which contains 5 milliseconds of audio. The content of these audio data packets is controlled by the chairperson of the conference through the interface 304. The content of these audio data packets can be, for example, audio received from a single conference unit, audio received from multiple conference units, audio received from an interpreter or translator, etc. The processor 311 forwards the generated audio data packets to the antenna 302, i.e., the radio interface, which transmits the audio data packets in the downlink direction to the conference units of the conference system using the Wi-Fi protocol with soft TDMA and LBT mechanisms as described above. In the downlink direction, the antenna 302 also periodically sends Wi-Fi beacon messages containing, for example, an access point identifier to enable the conference units to associate with the access point 300. These beacon messages also include a timestamp or time value from the clock 303, enabling the conference unit to actively synchronize the conference unit's clock with the clock 303 in the access point 300. The estimated time of arrival unit 323 uses the clock signal generated locally by the clock 303 in the access point 300 to estimate the arrival time of the next audio data packet from the conference unit. Since the clock 303 is actively synchronized with the clock in the conference unit for the transmission of audio data packets, the ETA unit 323 can accurately determine the time when the next audio data packet is ready for transmission at the conference unit. The ETA unit 323 increases this time by a predetermined transmission delay, which is an acceptable time required to cope with interruptions on the transmitter and receiver sides, propagation over the air through the Wi-Fi channel, and possible jitter due to, for example, the LBT mechanism implemented on the transmitter side. The transmission delay may further indicate an inaccurate clock synchronization between the clock 303 and the clock in the conference unit with which it is actively synchronized. In Figure 3In the example of , the transmission delay is predetermined to correspond to 1.5 milliseconds. The locally generated clock signal and the predetermined transmission delay enable the ETA unit 323 to defensively estimate the arrival time of the next audio data packet of the conference unit. The loss detection unit 324, which receives an interrupt signal from the processor 321 when the audio data packet arrives, verifies whether the audio data packet is received before the next estimated arrival time of the audio data packet. If the audio data packet is received in time, the processor 321 will process the received audio data packet so that the audio samples contained therein can be output and used for downlink transmission. Each time an audio data packet is not received before the estimated arrival time of the audio data packet, the loss detection unit 324 triggers a packet masking unit 325 formed as part of the receiver 320 to generate a replacement packet and provide the replacement packet to the processor 321 for insertion into the output audio stream. The packet masking unit 325 applies a processing-intensive masking algorithm, typically using an earlier received audio data packet to generate a replacement audio data packet, so that when such a replacement audio data packet is inserted into the received audio stream to replace the lost audio data packet, no audible artifact components are generated. Due to active clock synchronization and arrival time estimation of audio packets, packet loss concealment can be triggered early. This enables the access point 300 to meet the latency requirements set for conferencing systems, which are typically in the range of 15 ms to 25 ms for the round trip time of audio packets transmitted back and forth between the conferencing unit and the access point 300.

[0059] Figure 4 A pseudo flow chart illustrating an embodiment of a method for transmitting delay-sensitive audio data packets according to the present invention is shown. Figure 4 The pseudo-flowchart of illustrates the steps performed on the receiver side. In a first step 401, the expected arrival time of the audio data packet is determined based on the expected transmission delay 412 and the time signal received from the active synchronous clock 411. The expected transmission delay 412 is predetermined and recorded in, for example, a computer memory or register. In a second step 402, it is verified whether the audio data packet is received before the expected arrival time determined in the first step 401. If the audio data packet is received before the expected arrival time, the received audio data packet is processed normally in step 403. If the audio data packet is not received before the expected arrival time, packet loss concealment is enabled in step 404 to generate a replacement audio packet for the lost audio data packet. Once the expected arrival time is reached, the packet loss concealment of step 404 is enabled. Finally, the processed audio from the received audio data packet or the audio generated by concealment is streamed in step 405 to form a continuous audio stream without audible aliasing components.

[0060] Figure 5A suitable computing system 500 is shown, which is capable of performing one or more steps in an embodiment of a method for transmitting delay-sensitive audio data packets according to the present invention. The computing system 500 can generally be formed as a suitable general-purpose computer and includes a bus 510, a processor 502, a local memory 504, one or more optional input interfaces 514, one or more optional output interfaces 516, a communication interface 512, a storage element interface 506, and one or more storage elements 508. The bus 510 may include one or more conductors that allow communication between the components of the computing system 500. The processor 502 may include any type of conventional processor or microprocessor that interprets and executes programming instructions. The local memory 504 may include a random access memory (RAM) or another type of dynamic storage device, which stores information and instructions for execution by the processor 502, and / or the local memory 504 may include a read-only memory (ROM) or another type of static storage device, which stores static information and instructions for use by the processor 502. Input interface 514 may include one or more conventional mechanisms that allow an operator or user to input information to computing device 500, such as keyboard 520, mouse 530, pen, voice recognition and / or biometric mechanisms, camera, etc. Output interface 516 may include one or more conventional mechanisms that output information to an operator or user, such as display 540, etc. Communication interface 512 may include any transceiver-like mechanism, such as, for example, one or more Ethernet interfaces that enable computing system 500 to communicate with other devices and / or systems (e.g., with other computing devices 581, 582, 583). The communication interface 512 of computing system 500 may be connected to such additional computing systems via a local area network (LAN) or a wide area network (WAN), such as, for example, the Internet. The storage element interface 506 may include a storage interface, such as, for example, a Serial Advanced Technology Attachment (SATA) interface or a Small Computer System Interface (SCSI), for connecting the bus 510 to one or more storage elements 508, such as one or more local disks, such as SATA disk drives, and controlling writing data to and / or reading data from these storage elements 508. Although the storage element 508 above is described as a local disk, in general any other suitable computer-readable medium may be used, such as a removable disk, an optical storage medium (such as a CD or DVD), a ROM disk, a solid-state drive, a flash memory card, ... The computing system 500 may therefore correspond to Figure 2 The processing circuits 211, 221 in the embodiment shown, or corresponding to Figure 3Processing circuits 311, 321 in the embodiments shown. The processing circuits used in the transmitter and receiver, respectively, of these embodiments may obviously form part of a single processor or computer as indicated by the dashed rectangles in these figures.

[0061] Although the present invention has been described with reference to specific embodiments, it is obvious to those skilled in the art that the present invention is not limited to the details of the aforementioned illustrative embodiments, and the present invention can be implemented with various changes and modifications without departing from its scope. Therefore, the current embodiments are considered to be illustrative rather than restrictive in all aspects, and the scope of the present invention is indicated by the appended claims rather than by the foregoing description, and all changes falling within the meaning and scope of the equivalent content of the claims are therefore intended to be included in the claims. In other words, it is expected to cover any and all modifications, variations or equivalents that fall within the scope of the basic principles and whose essential attributes are claimed in this patent application. The reader of this patent application will also understand that the words "comprise" or "include" do not exclude other elements or steps, the singular words "one" or "an" do not exclude plural numbers, and a single element such as a computer system, a processor or another integrated unit can implement the functions of several devices recorded in the claims. Any figure mark in the claims should not be interpreted as limiting the relevant individual claims. The terms "first", "second", "third", "a", "b", "c", etc. are introduced to distinguish similar elements or steps when used in the specification or claims, rather than necessarily describing the order or time sequence. Similarly, the terms "top", "bottom", "above", "below", etc. are introduced for descriptive purposes and do not necessarily indicate relative positions. It is understood that the terms so used are interchangeable under appropriate circumstances and that embodiments of the invention are capable of operation in accordance with the invention in other sequences or in different orientations than described or illustrated above.

Claims

1. A wireless conference system (100), the wireless conference system (100) being adapted to enable a plurality of users to participate in a conference in a conference room, the wireless conference system (100) comprising an access point (101; 300) and a plurality of conference units (111-115; 200), -in, The access point (101; 300) and one or more of the conference units (111-115; 200) include a transceiver (201; 301), wherein the transceiver (201; 301) is configured for bidirectional TDMA-based wireless communication of delay-sensitive audio data packets between the one or more conference units (111-115; 200) and the access point (101; 300), the transceiver (201; 301) comprising a transmitter (210; 310) and a receiver (220; 320); - wherein the access point (101; 300) and the one or more conference units (111-115; 200) include actively synchronized corresponding clocks (203; 303), the clocks (203; 303) in the corresponding clocks are configured to generate a local audio clock signal and a local synchronization clock signal, the local audio clock signal is used to process the audio data packet locally, and the local synchronization clock signal is used for the TDMA-based wireless communication, wherein the one or more conference units (111-115; 200) include a clock synchronization unit (204), and the clock synchronization unit (204) is configured to: based on the clock synchronization signal generated by the access point (101; 300) regularly broadcast timestamps to actively synchronize the corresponding clocks (203) of the one or more conference units with the clock (303) in the access point (101; 300); - wherein the receiver (220; 320) comprises a packet loss detection unit (222; 322), the packet loss detection unit (222; 322) being configured to: 200) to the access point (101; 300); or detecting the loss of an audio data packet transmitted from the access point (101; 300) to a conference unit (111-115; 200), the packet loss detection unit (222; 322) comprising: - means (223; 323) configured to determine the expected arrival time of the audio data packet based on the local synchronous clock signal and a predetermined expected transmission delay, and - means (224) configured to detect that the audio data packet is lost if the audio data packet does not arrive before the expected arrival time. 324); and - wherein the receiver (220; 320) comprises a packet loss concealment unit (225; 325), the packet loss concealment unit (225; 325) being configured to generate a replacement packet for the audio data packet detected as lost by the packet loss detection unit (222; 322).

2. The wireless conference system (100) according to claim 1, wherein: The access point (101; 300) and the one or more conference units (111-115; 200) are configured not to acknowledge receipt of audio data packets.

3. The wireless conference system (100) according to claim 1, wherein: The access point (101; 300) and the one or more conference units (111-115; 200) are configured not to retransmit lost audio data packets.

4. The wireless conference system (100) according to any one of claims 1 to 3, wherein: The delay-sensitive audio data packets have a round-trip time delay limit of 25 milliseconds for wireless transmission from the conference unit (111-115; 200) to the access point (101; 300) and for wireless transmission from the access point (101; 300) to the conference unit (111-115; 200).

5. The wireless conference system (100) according to any one of claims 1 to 3, wherein: The delay-sensitive audio data packets have a round-trip time delay limit of 15 milliseconds for wireless transmission from the conference unit (111-115; 200) to the access point (101; 300) and for wireless transmission from the access point (101; 300) to the conference unit (111-115; 200).

6. The wireless conference system (100) according to any one of claims 1 to 3, wherein: The TDMA-based wireless communication uses a 5 millisecond TDMA frame.

7. The wireless conference system (100) according to claim 6, wherein: The transmitter (210; 310) is configured to: before transmitting the audio data packet in the transmitter (210; 310), listen for interference information flow in a designated time slot in the TDMA frame.

8. The wireless conference system (100) according to any one of claims 1 to 3, wherein: The wireless communication uses Wi-Fi.

9. The wireless conference system (100) according to any one of claims 1 to 3, wherein: The one or more conference units (111-115; 200) include a clock synchronization unit (204), the clock synchronization unit (204) being configured to: based on a clock signal inserted in a communication channel of the access point (101; 300) to actively synchronize the corresponding clocks (203) of the one or more conference units with the clock (303) in the access point (101; 300).

10. The wireless conference system (100) according to any one of claims 1 to 3, in, The predetermined expected transmission delay is determined as the sum of propagation delay, jitter, interrupt handling delay, processing delay and clock synchronization inaccuracy.

11. The wireless conference system (100) according to claim 10, in, The jitter includes a listen-before-talk jitter contribution.

12. The wireless conference system (100) according to any one of claims 1 to 3, in, The predetermined expected transmission delay is set to a value between 1.5 milliseconds and 2 milliseconds.

13. A method for transmitting delay-sensitive audio data packets between an access point (101; 300) and one or more conference units (111-115; 200) in a wireless conference system (100), the wireless conference system (100) being adapted to enable a plurality of users to participate in a conference in a conference room, the method for transmitting delay-sensitive audio data packets comprising performing bidirectional TDMA-based wireless communication on the audio data packets, the method further comprising: - enabling the one or more conference units (111-115; 200) is actively synchronized with a corresponding clock (203; 303) in the access point (101; 300), the clock (203; 303) in the corresponding clock is configured to generate a local audio clock signal and a local synchronization clock signal, the local audio clock signal is used to locally process the audio data packet, and the local synchronization clock signal is used for the TDMA-based wireless communication, wherein the one or more conference units (111-115; 200) include a clock synchronization unit (204), and the clock synchronization unit (204) is configured to: actively synchronize the corresponding clock (203) of the one or more conference units with the clock (303) in the access point (101; 300) based on a timestamp regularly broadcast by the access point (101; 300); - Detecting the loss of an audio data packet transmitted from a conference unit (111-115; 200) to the access point (101; 300), or detecting the loss of an audio data packet transmitted from the access point (101; 300) to the conference unit (111-115; 200), comprising: - determining the expected arrival time of the audio data packet based on the local synchronous clock signal and a predetermined expected transmission delay; and - detecting that the audio data packet is lost if the audio data packet does not arrive before the expected arrival time; and - Generating replacement packets for the audio data packets detected as lost by packet loss concealment.

Images