Audio packet loss concealment via packet duplication at the decoder input - Patents.com

Abstract

The system includes a server that generates a real-time stream of audio packets and a client device that decodes and plays back audio content of the stream. The client device includes a network interface configured to receive the stream of audio packets over a network and a buffer configured to temporarily buffer a subset of the audio packets of the stream. The client device further includes an audio decoder having an input for receiving the audio packets from the buffer and an output for providing a corresponding segment of a decoded audio data stream. The client device further includes a stream monitoring module configured to, in lieu of decoding lost or delayed audio packets, provide a subset of the audio packets in the buffer that were previously decoded by the decoder to an input of the decoder again for iterative decoding.

Description

[Background technology]

[0001] Real-time media streaming services are often susceptible to network issues, such as packet loss or otherwise delayed packets. In the case of audio streams, audio packet loss or delay can be problematic because the audio decoder of the receiving client device usually cannot simply pause the audio playback, since pausing the audio playback would degrade the audio quality and affect the quality of service. To mitigate the impact of delayed / lost audio packets, many systems employ traditional Packet Loss Concealment (PLC) techniques, such as Forward Error Correction (FEC), silence insertion, interpolation analysis, or post-decoding segment duplication. Summary of the Invention [Problem to be solved by the invention]

[0002] In general, FEC-based PLC techniques rely on increasing the number of at least partially redundant packets sent, which allows the client device to reconstruct lost or delayed audio packets. However, this redundancy often consumes extra bandwidth, lowering the overall bit rate of the audio stream. Furthermore, this approach is limited to only client devices with built-in FEC capabilities. Silence insertion techniques effectively provide either zero output or a default defined output for the time slots corresponding to lost / delayed audio packets. Although relatively simple to implement, this default insertion often causes perceptible audio artifacts. In contrast, interpolation analysis-based techniques analyze the preceding audio signal and attempt to reconstruct the content of lost / delayed audio packets through signal estimation, which can result in improved reproduction of the audio signal and less perceptible audio artifacts. Post-decoding replication techniques rely on replicating slices of the audio output of an audio decoder to compensate for late / lost audio packets, and then analyze the reconstructed audio signal and perform additional processing to effectively "stitch" the reconstructed audio signal together in a manner that reduces audio artifacts. However, while post-decoding replication and interpolation analysis techniques can provide improved signal fidelity, these techniques often require complexity and computational resource capabilities that are infeasible for many client devices. [Means for solving the problem]

[0003] Overview of the embodiment In a first embodiment, the electronic device includes a network interface configured to receive a stream of audio packets over a network, a buffer configured to temporarily buffer a subset of the audio packets of the stream, and an audio decoder having an input for receiving the audio packets from the buffer and an output for providing corresponding segments of the decoded audio data stream (i.e., segments of the decoded audio data respectively corresponding to the audio packets from the buffer). The electronic device further includes a stream monitoring module configured to provide the subset of audio packets in the buffer previously decoded by the decoder again to an input of the decoder for iterative decoding in lieu of decoding of lost or delayed audio packets. The stream monitoring module may be further configured to provide the audio packets previously decoded by the decoder again to an input of the decoder for iterative decoding to provide a packet loss concealment (PLC) process in response to determining that the number of consecutive decoded audio packets does not exceed a specified threshold, and may be further configured to trigger an anomaly in response to the number of consecutive decoded audio packets exceeding a specified threshold. The stream monitoring module may also be further configured to implement an alternative packet loss concealment (PLC) process to compensate for lost or delayed audio packets in response to the number of consecutive audio packets being decoded exceeding a specified threshold.

[0004] Triggering the anomaly may include triggering the generation of an anomaly signal, for example, that results in stopping playback of the audio stream (and, in an exemplary embodiment, the video stream, if there is a corresponding video stream) and / or initiating a system performance check.

[0005] Thus, the proposed electronic device may include a network interface for receiving an audio stream from a network, a buffer, e.g., a jitter buffer, for temporarily buffering a subset of audio packets received at the electronic device, and an audio decoder for decoding the audio packets from the buffer to generate corresponding segments of a decoded audio signal. The decoded audio signal may then be mixed with other audio sources, repacketized, further transmitted, or otherwise further processed to ultimately generate one or more analog audio signals used to drive one or more corresponding speakers. In at least one embodiment, the electronic device may employ a proposed packet loss concealment (PLC) technique based on one or more iterative decoding of previously received audio packets to compensate for one or more subsequent audio packets that have been lost or excessively delayed (i.e., "late") by the network. To this end, the electronic device may employ a stream monitoring module to monitor the received audio stream and detect that audio packets of the stream are late or lost. In response to detecting such a delayed or lost audio packet, the stream monitoring module provides a subset of audio packets in the buffer previously decoded by the decoder for a previous time slot to an input of the decoder again for iterative decoding for the current time slot in lieu of decoding the lost or delayed audio packet intended for the current time slot. By repeating the decoding of previously decoded audio packets to fill gaps caused by delayed / lost packets, this PLC technique leverages the decoder's audio synthesis (by following the normal decoding process path) to promote continuity of the decoded audio signal during packet decoding duplication and reduce or eliminate distortions or artifacts that may occur in the absence of this continuity.

[0006] In an exemplary embodiment, the stream monitoring module may be configured to provide, for a first audio packet received at the electronic device in a timely manner, the first audio packet to the audio decoder for decoding into a corresponding first segment of the audio signal decoded for the first time slot, and for a delayed or lost second audio packet, the first audio packet to the audio decoder for decoding into a corresponding second segment of the audio signal decoded for the second time slot, the second segment being contiguous with the first segment. The stream monitoring module may also be further configured to provide, for a delayed or lost third audio packet, the first audio packet to the audio decoder for decoding into a corresponding third segment of the audio signal decoded for the third time slot, the third segment being contiguous with the second segment.

[0007] In any of the above examples, the stream of audio packets may be generated from audio content generated by a video game application running on a server connected to the electronic device over a network.

[0008] Other embodiments may include a method of operating any of the above-mentioned examples of electronic devices, as well as a system including a server generating a stream of audio packets and any of the above-mentioned examples of electronic devices.

[0009] In another embodiment, a proposed computer-implemented method includes receiving a stream of audio packets from a network, temporarily buffering a subset of the audio packets, decoding a first audio packet of the subset in an audio decoder to generate a first segment of a decoded audio data stream, and decoding the first audio packet again in the audio decoder in response to detecting a loss or delay of a second audio packet following the first audio packet in the stream to generate a second segment of the decoded audio signal following the first segment of the decoded audio signal. The method may also include decoding the first audio packet a third time in the audio decoder in response to detecting a loss or delay of a third audio packet following the second audio packet in the stream to generate a third segment of the decoded audio signal following the second segment in the decoded audio signal. The decoding of the first audio packet again may be further responsive to determining that the number of consecutive decoded first audio packets does not exceed a specified threshold. The method may further include triggering an anomaly in response to the number of consecutive decoded first audio packets exceeding a specified threshold. The method may also include performing an alternative packet loss concealment process to compensate for a lost or delayed second audio packet in response to the number of consecutive decoded first audio packets exceeding a specified threshold. In any of the above examples, the audio decoder may generate the second segment to be contiguous with the first segment. Similarly, in any of the above examples, the stream of audio packets may be generated from audio content generated by a video game application executing on the server.

[0010] Other embodiments may include an electronic device comprising a buffer for buffering audio packets and a decoder for decoding the audio packets, the electronic device configured to perform the exemplary method.

[0011] Another embodiment includes a non-transitory computer-readable storage medium storing a set of executable instructions configured to operate a processor to decode a first audio packet of a temporarily buffered subset of audio packets of a stream received over a network to generate a first segment of a decoded audio signal, and in response to detecting that a second audio packet following the first audio packet in the stream has been lost or delayed, decode the first audio packet again in the audio decoder to generate a second segment of the decoded audio signal that follows and has continuity with the first segment of the decoded audio signal.

[0012] BRIEF DESCRIPTION OF THE DRAWINGS The present disclosure may be better understood, and its numerous features and advantages made apparent to those skilled in the art, by referencing the accompanying drawings, in which: The use of the same reference symbols in different drawings indicates similar or identical items. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a block diagram illustrating a real-time media streaming system employing packet loss concealment (PLC) based on pre-decoder packet duplication, according to some embodiments. [Diagram 2] FIG. 1 is a flow diagram illustrating a method for PLC with pre-decoder packet duplication according to some embodiments. [Diagram 3] FIG. 3 illustrates an exemplary implementation of the method of FIG. 2. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Detailed Description 1-3 illustrate various systems and techniques for mitigating the effects of packet loss or packet delay in a real-time audio stream transmitted from a server to a user's electronic device (i.e., a "client device") over one or more networks. The client device includes a network interface that receives the audio stream from the network, a jitter buffer that temporarily buffers a subset of the audio packets received at the client device, and an audio decoder that decodes the audio packets from the jitter buffer to generate corresponding segments of a decoded audio signal. For example, the decoded audio signal may then be mixed with other audio sources, repacketized, further transmitted, or otherwise further processed to ultimately generate one or more analog audio signals that are used to drive one or more corresponding speakers. In at least one embodiment, the client device employs a packet loss concealment (PLC) technique based on one or more iterative decoding of previously received audio packets to compensate for one or more subsequent audio packets that have been lost or excessively delayed (i.e., "late") by the network. To this end, the client device employs a stream monitoring module to monitor the received audio stream and detect delays or losses in audio packets of the stream. In response to detecting such a late or lost audio packet, the stream monitoring module provides a subset of audio packets in the buffer previously decoded by the decoder for the previous time slot to the input of the decoder again for iterative decoding for the current time slot, instead of decoding the lost or late audio packets for the current time slot.By repeating the decoding of previously decoded audio packets to fill in gaps caused by delayed / lost packets, this PLC technique leverages the decoder's audio synthesis (by following the normal decoding processing path) to promote continuity of the decoded audio signal during packet decoding duplication, reducing or eliminating distortion or artifacts that may occur in the absence of this continuity.

[0015] 1 illustrates a real-time media streaming system 100 employing PLC technology with pre-decoder packet replication, according to some embodiments. The system 100 includes a server 102 coupled to user electronic devices 104 (hereinafter "client devices 104") via one or more networks 106. The one or more networks 106 may include, for example, the Internet or other public access networks, a wired or wireless wide area network (WAN), a wired or wireless local area network (LAN), a wired or wireless personal area network (PAN), or a combination thereof.

[0016] The server 102 includes a network interface 108 coupled to the network 106, a real-time media source 110, and an audio encoder 112. The real-time media source 110 generates or otherwise provides real-time media content for transmission to the client device 104. By way of example, the real-time media source 110 may include, for example, a cloud-based video game running on the server 102 based on player input received from the client device 104 over the network 106, the video game generating both a stream of video frames and an accompanying stream of audio frames for transmission to the client. As another example, the real-time media source 110 may include a video conferencing application that distributes video and audio streams between various participant client devices. As yet another example, the real-time media source 110 may include a forwarding transmission of audio content for a Voice-over-Internet Protocol (VoIP) or other packet-based voice call in a mobile cellular system. The audio encoder 112 operates to encode the audio content stream from the real-time media source 110 and provide the resulting encoded audio stream to the network interface 108, which accordingly packetizes the encoded audio stream and transmits the resulting audio packets to the client device 104 over the network 106 as part of the packetized audio stream 114.

[0017] Client device 104 represents any of a variety of electronic devices utilized to play the audio content of audio stream 114 or to decode and forward the audio content for playback by yet another electronic device. Examples of client device 104 include mobile phones, desktop computers, laptop computers, tablet computers, gaming consoles, “smart” televisions, “smart” watches, automobile information / entertainment systems, and the like. The client device 104 includes a network interface 116 that receives audio packets of an audio stream 114, a jitter buffer 118 (e.g., a circular buffer) for temporarily buffering a sliding subset of recently received audio packets, and an audio decoder 120 that operates to sequentially decode the audio packets from the jitter buffer 118 in a specified order (e.g., in the order they were received, in order based on a timestamp, etc.) to generate an output decoded audio signal 122 (e.g., a Pulse-Code-Modulation (PCM) digital signal) that may be directly converted to one or more analog audio signals used to drive at least one speaker 124 (e.g., via a digital-to-analog converter or DAC) or may be further processed, such as by a digital amplifier / mixer 127, before being converted to one or more analog speaker signals for driving the at least one speaker 124. In one embodiment, the audio decoder 120 is implemented as one or more processors 126 executing audio decoding software 128 stored in system memory 130 or other non-transitory computer-readable medium. By way of example, the audio decoding software 128 may be implemented as, for example, the Opus Interactive Audio Codec or other well-known or proprietary software-based codec.In other embodiments, audio decoder 120 may be implemented as hard-coded or programmable logic, such as an Application-Specific Integrated Circuit (ASIC) or Field-Programmable Gate Array (FPGA) configured to perform the functions described herein. In yet other embodiments, audio decoder 120 may be implemented as a combination of a processor running software and specific hard-coded / programmable logic.

[0018] In at least one embodiment, the network 106 is a combination of one or more packet-switched networks and is therefore subject to congestion, routing errors, buffer overflows and other network problems that may result in one or more of the audio packets of the audio stream 114 being lost (i.e., never received by the client device 104) or delayed (i.e., not received by the client device 104 in time to be processed for playback in its corresponding decoding timeslot). A late audio packet has the same consequence as a lost audio packet in that neither can be used to provide playback of the audio content represented in the corresponding timeslot, and is therefore effectively "lost," and unless otherwise specified, references herein to a "lost audio packet" are intended to include either an audio packet that is lost in the network 106 or an audio packet that arrives too late at the client device 104. Without a compensation mechanism, a lost audio packet means that the audio decoder 120 does not have the intended audio content to decode to generate a corresponding segment of the decoded audio signal 122 for the corresponding time slot, thus introducing significant discontinuities in the resulting decoded audio signal 122 that mar the listener's experience. Conventional PLC techniques, such as FEC, silence insertion, interpolation analysis, and repetition of decoded signal segments, attempt to mitigate the effects of lost audio packets, but these conventional techniques are either overly complex and resource intensive, or do not sufficiently eliminate the discontinuities in the resulting decoded audio signal.

[0019] Thus, in at least one embodiment, the client device 104 employs a PLC technique based on pre-decoder packet replication (i.e., replication of audio packet content at the input of the decoder). To this end, the client device 104 further includes a stream monitoring module 132 coupled to the network interface 116 and the jitter buffer 118, and coupled to a packet selector 134 that operates to select audio packets buffered in the jitter buffer 118 and provide them to an input 136 of the audio decoder 120. In some embodiments, one or both of the stream monitoring module 132 and the packet selector 134 are implemented at least in part as one or more processors 126 executing software 138 stored in the system memory 130 or other non-transitory computer-readable medium. In other embodiments, one or both of the stream monitoring module 132 and the packet selector 134 are implemented at least in part as hard-coded or programmable logic, or a combination of processor-executing software and programmable / hard-coded logic.

[0020] As a general overview of operation, the stream monitoring module 132 monitors the audio stream 114 received via the network interface 116 or the jitter buffer 118 to detect a lost audio packet. While an audio packet for a corresponding decoding time slot is received in time, the stream monitoring module 132 controls the packet selector 134 to access the received audio packet from the jitter buffer 118 and provide the accessed audio packet to the audio decoder 120 for decoding of its audio content to generate a segment of the decoded audio signal 122 for the corresponding time slot. In response to detecting a lost audio packet, when the decoding time slot of the lost packet is approaching, the stream monitoring module 132 controls the packet selector 134 to access an audio packet decoded by the audio decoder 120 for a previous decoding time slot and provide this same audio packet to an input 136 of the audio decoder 120 for iterative decoding, and the resulting decoded segment is used in the decoded audio signal 122 as a replacement for the decoded segment that the audio decoder 120 would have generated if the lost audio packet had not been lost.

[0021] To illustrate, assume that the server 102 transmits audio packets 140, 141, and 142 of the audio stream 114 in that order over the network 106. In this example, audio packets 140 and 142 are successfully received and buffered in time by the client device 104, but audio packet 141 is lost in the network 106. Thus, with audio packet 140 received and buffered in time, the packet selector 134 provides audio packet 140 from the jitter buffer 118 to the input 136 of the audio decoder 120 in the corresponding time slot X, and the audio decoder 120 accordingly decodes the audio content of audio packet 140 to generate segment Y of the decoded audio signal 122. In response to determining that the next audio packet, audio packet 141, for the next time slot X+1 has been "lost", the stream monitoring module 132 controls the packet selector 134 to again provide the previously decoded audio packet, i.e., audio packet 140, to the input 136 of the audio decoder 120, where it is again decoded to generate the corresponding segment Y+1 of the decoded audio signal 122. It should be noted that typical audio synthesis properties exhibited by the audio decoder 120 result in segment Y+1 being generated by the audio decoder 120 to seamlessly stitch with segment Y previously generated by the audio decoder 120 from the same audio content, thus ensuring continuity of the decoded audio signal 122 between segment Y and segment Y+1, even though both segments were generated from the same input packet (audio packet 140).Subsequently, when audio packet 142 is received in time, for time slot X+2, the packet selector 134 provides the audio packet 142 to the input 136 of the audio decoder 120, and accordingly, the audio decoder 120 decodes the audio packet 142 to generate segment Y+2 for the decoded audio signal 122, which is also seamlessly stitched with segment Y+2 by the audio decoder 120 via standard audio synthesis procedures employed by the audio decoder 120.

[0022] FIG. 2 is a diagram illustrating in more detail a method 200 illustrating a pre-decoder packet repetition PLC technique employed by the client device 104 of the system 100 of FIG. 1 according to some embodiments. As represented by block 202, during initialization of the client device 104 to process the audio stream 114, the stream monitoring module 132 sets several parameters. One such parameter initialization includes setting a variable REP_PACKET_COUNT, which represents an ongoing count of the number of packets to be repeated, to zero or some other initialization value. Another parameter initialization includes setting a variable THRESHOLD to a specified number, which represents the maximum number of times an audio packet may be consecutively duplicated for decoding (or the number of times duplicated decoding is performed) before either triggering an anomaly or switching to a different PLC process, depending on the implementation. Thus, THRESHOLD is a tuning parameter that may be used to balance between robustness in terms of packet loss and audio quality. Thus, the value of THRESHOLD may be set by a user, a provider of the audio stream 114, a provider of the client device 104, etc.

[0023] In block 204, the client device 104 starts receiving audio packets of the audio stream 114 from the server 102 via the network interface 116 and buffers a sliding subset of the audio packets in the jitter buffer 118 (wherein the maximum size of the current buffered subset is based on the number of entries in the jitter buffer 118). Once the audio packets are received and buffered, in block 206, the stream monitoring module 132 monitors the incoming audio packets to determine whether an audio packet has been lost. If no lost packets are detected, in block 208, the stream monitoring module 132 sets or otherwise maintains the variable REP_PACKET_COUNT to zero, and for the next decoding time slot, the packet selector 134 accesses the buffered audio packets corresponding to that decoding time slot from the jitter buffer 118 and provides the accessed audio packets as incoming audio packets to the input 136 of the audio decoder 120 in block 210.

[0024] To illustrate, in some implementations, each audio packet is assigned a sequence number that identifies the corresponding audio packet's location in the intended playback sequence during the encoding process at the server 102. As a result, when the stream monitoring module 132 accesses the next audio packet from the jitter buffer 118, the stream monitoring module 132 compares the sequence number of the accessed audio packet with the expected sequence number. If these numbers match, the stream monitoring module 132 determines that the audio packet for the corresponding decoding timeslot is timely. Conversely, if the actual sequence number of the accessed audio packet does not match the expected sequence number, the stream monitoring module 132 identifies the audio packet for the corresponding decoding timeslot as lost.

[0025] Returning to block 206, if a lost audio packet is detected, at block 212, the stream monitoring module 132 determines whether the maximum number of replicas of consecutive packet decodings for the same audio packet has been executed by comparing the variable REP_PACKET_COUNT with THRESHOLD. If so (i.e., REP_PACKET_COUNT = THRESHOLD), further iterations of packet decoding for the interrupted sequence of lost packets are not permitted. Thus, at block 214, the stream monitoring module 132 triggers an anomaly to stop the playback of the audio stream (and, if any, the corresponding video stream) and initiate a system performance check. Alternatively, instead of triggering an anomaly, the client device 104 can switch to using different PLC techniques such as silence insertion, interpolation analysis, etc. However, if the number of executions is less than the maximum number of iterations of consecutive packet decodings (i.e., REP_PACKET_COUNT < THRESHOLD), then at block 216, the stream monitoring module 132 instructs the packet selector 134 to access the previously decoded audio packet from the jitter buffer 118 for the latest time slot and provide this accessed audio packet as the input audio packet to the input 136 of the audio decoder for the next decoding time slot. Also, the stream monitoring module 132 increments REP_PACKET_COUNT to reflect that the packet decoding replica of this audio packet has been executed.

[0026] In block 218, the audio decoder 120 decodes the input audio packet selected via the process represented by blocks 206-216, i.e., either the audio packet associated with the current decoding time slot (if this packet was not lost) or the previously decoded audio packet for the previous time slot (if the audio packet targeted for this time slot was lost). The audio decoder 120 generates a corresponding segment of the decoded audio signal 122 from the encoded audio content of the input audio packet. As explained above, the audio synthesis techniques typically employed by the audio decoder 120 result in seamless stitching of each generated segment with the next segment, thus providing a decoded audio signal 122 without substantial discontinuity even if one or more packets are consecutively lost. Concurrent with the decoding of the audio packets to generate corresponding segments of the decoded audio signal 122, in block 220, the decoded audio signal 122 may be further processed (e.g., by mixing with other audio signals) and then converted to one or more analog signals used to drive one or more speakers 124 to affect the playback of the audio content represented by the audio stream 114.

[0027] Figure 3 illustrates a chart 300 depicting an example operation of the pre-decoder packet iterative PLC process represented by the method 200 of Figure 2. The seven columns of chart 300 represent seven decoding time slots of the audio decoder of the client device 104, identified as time slots A, B, C, D, E, F, and G, with time slot A being the earliest time slot and time slot G being the latest time slot. Row 310 represents the operation of the server 102 in generating an audio packet for each corresponding time slot A-G, row 312 represents the operation of the client device 104 in receiving an audio packet from the server 102 for each corresponding time slot, and row 314 represents the operation of the client device in decoding the audio packets and providing playback of the resulting decoded audio signal segment.

[0028] As illustrated by line 310, the server generates and transmits audio packets 301, 302, 303, 304, 305, 306, and 307 for time slots A, B, C, D, E, F, and G, respectively. As represented by lines 312 and 314, at the client device, for time slot A, audio packet 301 is received (without loss or delay), buffered, decoded, and the resulting segment of the decoded audio signal is played according to normal operation. Similarly, for time slot B, audio packet 302 is received (without loss or delay), buffered, decoded, and the resulting segment of the decoded audio signal is played according to normal operation. However, for time slot C, the associated audio packet 303 is lost by the network 106. Thus, the stream monitoring module 132 notices the lost status of audio packet 303 and accordingly instructs the packet selector 134 to access the audio packet decoded for the previous time slot B, i.e., audio packet 302, and provide this audio packet 302 to the audio decoder 120 to re-decode into the corresponding segment of the decoded audio signal for time slot C. In this manner, the audio content of audio packet 302 is decoded twice to generate two successive segments of the resulting decoded audio signal, once for its associated time slot B and then again to reconstruct the audio signal for the segments corresponding to time slots B and C, to fill in the lost audio packet 303 for the next time slot C.

[0029] Thereafter, for time slot D, the associated audio packet 304 is received on time and thus provided to the corresponding segment of the decoded audio signal for decoding by the audio decoder 120. For the following two time slots E and F, the associated audio packets 305 and 306 are not received on time for decoding in their respective time slots, but instead are received late during the following time slot G. Thus, for time slot E, the client device 104 reuses the audio content of the audio packet 304 to decode the corresponding segment of the decoded audio signal, and the audio packet 304 is again selected for decoding a third time to generate another segment of the decoded audio signal for time slot F. Then, although all three audio packets 305, 306, 307 have been received in time for time slot G, the audio packets 305 and 306 are discarded from the jitter buffer 118 since they are associated with an earlier time slot, and the audio decoder 120 decodes the audio packet 307 to generate the segment of the decoded audio signal corresponding to time slot G.

[0030] In some embodiments, some aspects of the techniques described above may be implemented by one or more processors of a processing system executing software. The software comprises a set of one or more executable instructions stored or otherwise tangibly embodied in a non-transitory computer-readable storage medium. The software may include instructions and specific data that, when executed by one or more processors, operate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer-readable storage medium may include, for example, magnetic or optical disk storage devices, solid-state storage devices such as flash memory, caches, Random Access Memory (RAM) or other non-volatile storage devices, etc. The executable instructions stored on the non-transitory computer-readable storage medium may be source code, assembly language code, object code, or other instruction formats that are interpreted or executable by one or more processors.

[0031] A computer-readable storage medium may include any storage medium or combination of storage media that can be accessed by a computer system during use to provide instructions and / or data to the computer system. Such storage media may include, but are not limited to, optical media (e.g., Compact Disc (CD), Digital Versatile Disc (DVD), Blu-ray Disc), magnetic media (e.g., floppy disk, magnetic tape, or magnetic hard drive), volatile memory (e.g., Random Access Memory (RAM) or cache), non-volatile memory (e.g., Read-Only Memory (ROM) or flash memory), or Microelectromechanical Systems (MEMS)-based storage media. A computer-readable storage medium may be incorporated into a computing system (e.g., system RAM or ROM), permanently attached to a computing system (e.g., a magnetic hard drive), removably attached to a computing system (e.g., an optical disk or Universal Serial Bus (USB)-based flash memory), or coupled to a computer system via a wired or wireless network (e.g., Network Accessible Storage (NAS)).

[0032] It should be noted that not all of the activities or elements described above in the general description are required, that some of the specific activities or devices may not be required, and that one or more additional activities may be performed or elements may be included in addition to those described. Furthermore, the order in which the activities are listed is not necessarily the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, those skilled in the art will appreciate that various modifications and changes can be made without departing from the scope of the present disclosure, as set forth in the following claims. Thus, the specification and drawings should be regarded in an illustrative rather than restrictive sense, and all such modifications are intended to be included within the scope of the present disclosure.

[0033] Benefits, other advantages, and solutions to problems have been described above with respect to specific embodiments. However, the benefits, advantages, solutions to problems, and any features that may give rise to or make more prominent any benefit, advantage, or solution should not be construed as critical, necessary, or essential features of any or all claims. Moreover, the specific embodiments disclosed above are illustrative only, as the disclosed subject matter may be modified and practiced in different but equivalent manners apparent to those skilled in the art having the benefit of the teachings herein. No limitations are intended to the details of construction or design shown herein, except as set forth in the following claims. It is therefore apparent that the specific embodiments disclosed above may be modified or improved upon, and all such variations are deemed to be within the scope of the disclosed subject matter. Accordingly, the protection sought herein is as set forth in the following claims.

Claims

1. a network interface configured to receive a stream of audio packets over a network; a buffer configured to temporarily buffer a subset of audio packets of said stream; an audio decoder having an input for receiving audio packets from the buffer and an output for providing a corresponding segment of a decoded audio data stream; a stream monitoring module configured to, in lieu of decoding a lost or delayed second audio packet following a first audio packet of the subset in the buffer, provide the first audio packet decoded by the audio decoder back to the input of the audio decoder for iterative decoding, The stream monitoring module includes: configured, in response to determining that the number of times the first audio packet has been successively decoded does not exceed a specified threshold, to provide again the first audio packet previously decoded by the audio decoder to the input of the audio decoder for iterative decoding; An electronic device configured to, in response to the number of times the first audio packet is successively decoded exceeding the specified threshold, perform, as a process separate from the decoding of the first audio packet, a process for triggering an anomaly or an alternative packet loss concealment process for compensating for a lost or delayed second audio packet.

2. The stream monitoring module includes: for a first audio packet received in a timely manner at the electronic device, providing the first audio packet to the audio decoder for decoding into a corresponding first segment of a decoded audio signal for a first time slot; 2. The electronic device of claim 1, configured to provide the first audio packet to the audio decoder for decoding into a corresponding second segment of the decoded audio signal for a second time slot for the second audio packet that is delayed or lost.

3. The electronic device of claim 2 , wherein the audio decoder is configured to provide continuity between the first segment and the second segment.

4. 4. The electronic device of claim 2 or claim 3, wherein the stream monitoring module is further configured to, for a delayed or lost third audio packet, provide the first audio packet to the audio decoder for decoding into a corresponding third segment of the decoded audio signal for a third time slot.

5. The electronic device of claim 4 , wherein the audio decoder is configured to provide continuity between the second segment and the third segment.

6. 6. The electronic device of claim 1, wherein the stream of audio packets is generated from audio content generated by a video game application running on a server connected to the electronic device via the network.

7. A system comprising a server generating the stream of audio packets and an electronic device according to any one of claims 1 to 6.

8. receiving a stream of audio packets from a network; temporarily buffering the subset of audio packets; in an audio decoder, decoding a first audio packet of the subset to generate a first segment of a decoded audio data stream; in response to detecting that a second audio packet following the first audio packet in the stream has been lost or delayed and determining that the number of times the first audio packet has been successively decoded does not exceed a specified threshold, re-decoding, in the audio decoder, the first audio packet to generate a second segment of the decoded audio signal following the first segment in the decoded audio signal; and in response to the number of times the first audio packet is successively decoded exceeding the specified threshold, performing a process for triggering an anomaly or an alternative packet loss concealment process for compensating for a lost or delayed second audio packet as a process separate from the decoding of the first audio packet.

9. 9. The method of claim 8, further comprising the step of: in response to detecting that a third audio packet that follows the second audio packet in the stream is lost or delayed, decoding, in the audio decoder, the first audio packet for a third time to generate a third segment of the decoded audio signal that follows the second segment in the decoded audio signal.

10. The method of claim 8 or 9, wherein the audio decoder generates the second segment so as to have continuity with the first segment.

11. The method of any of claims 8 to 10, wherein the stream of audio packets is generated from audio content generated by a video game application running on a server.

12. a buffer for buffering audio packets; a decoder for decoding audio packets, An electronic device configured to perform the method according to any of claims 8 to 11.

13. A computer program, the computer program being executed by a processor to cause the processor to: decoding a first audio packet of the temporarily buffered subset of audio packets of a stream received over a network to generate a first segment of a decoded audio signal; in response to detecting that a second audio packet following the first audio packet in the stream has been lost or delayed and determining that the number of times the first audio packet has been consecutively decoded does not exceed a specified threshold, re-decoding the first audio packet to generate a second segment of the decoded audio signal that follows and is contiguous with the first segment in the decoded audio signal; A computer program product that, in response to the number of times the first audio packet has been consecutively decoded exceeding the specified threshold, causes a process to be executed, separate from the decoding of the first audio packet, for triggering an anomaly or for performing an alternative packet loss concealment process to compensate for a lost or delayed second audio packet.

Images