METHOD AND DEVICE FOR UPDATING AN OUTPUT TIME OF AT LEAST ONE DATA PACKET OF AN AUDIOVISUAL STREAM

Abstract

The present invention relates to a method and a device for updating an output time of at least one data packet of an audiovisual stream transferred from a transmitter to a receiver via at least one communication network. According to the invention, the transmitter: - transfers (E500) a first data packet and receives a second data packet comprising time markers, - calculates (E503), from the time markers, an average travel time, a weighted average value and a deviation, - checks (E507) whether the average travel time is greater than a predetermined threshold, - takes (E508) the average travel time into account to update the output time of at least one data packet of the audiovisual stream if at least the average travel time is less than or equal to the predetermined threshold. Fig. 5

Description

Title of the invention: METHOD AND DEVICE FOR UPDATING AN OUTPUT TIME OF AT LEAST ONE DATA PACKET OF AN AUDIOVISUAL STREAM Technical field

[0001] The present invention relates to a method and device for updating a time output of at least one data packet from an audiovisual stream. STATE OF PRIOR ART

[0002] The transmission of video data packets is subject to time constraints so as to allow good reproduction of the video at the receiver.

[0003] These time constraints are subject to the performance of the communication network connecting the transmitter of the video stream and the receiver.

[0004] For example, variations in the video stream transfer latency and desynchronization of the transmitter and receiver clocks affect the quality of the video stream reproduction. Jitter represents the difference in arrival time between different packets sent at the same time. Jitter occurs in the event of network congestion, timing drift, or routing changes.

[0005] Jitter is highly variable over time and is highly dependent on the type of communication network used for transferring data packets from the transmitter to the receiver.

[0006] This is particularly difficult to compensate for when multiple communication networks are used simultaneously to transmit the video stream.

[0007] Video reproduction at the receiver level corresponds to a reproduction at the receiver level of the video at the same rate as at the transmitter level. This makes it possible to avoid jumps linked to a phase shift and also to respect the latency without an end-to-end time drift. In conventional systems, the phase between the transmitter and the receiver is respected but not the latency.

[0008] The invention aims to propose a solution which makes it possible to take into account the variations in the characteristics of the communication network(s) used for the transmission of the video stream as well as the desynchronization of the clocks of the transmitter and the receiver. Statement of the invention

[0009] To this end, the invention relates to a method for updating an output time of at least one data packet of an audiovisual stream transferred from a transmitter to a receiver via at least one communication network, characterized in that that the process includes the steps carried out by the issuer of:

[0010] - transfer of a first data packet comprising a first time marker corresponding to the time of transmission of the first packet,

[0011] - receiving a second data packet comprising the first marker time, a second time marker corresponding to the time of reception of the first packet and a third time marker corresponding to the time of transmission of the second packet,

[0012] - storing the time of reception of the second packet,

[0013] - calculation, from the three time markers and the time of reception of the second package, with an average travel time,

[0014] - calculation of a weighted average value and a deviation between the travel time current and average travel time,

[0015] - checking whether the average travel time is greater than a predetermined threshold, the predetermined threshold being a function of the weighted average value and the deviation between the current travel time and the average travel time,

[0016] - taking into account the average travel time to update the exit time of at at least one data packet of the audiovisual stream if at least the average travel time is less than or equal to the predetermined threshold,

[0017] - average travel time not taken into account to update exit time of at least one data packet from the audiovisual stream if the average travel time is greater than the predetermined threshold.

[0018] The invention also relates to a device for updating an output time of at least one data packet of an audiovisual stream transferred from a transmitter to a receiver via at least one communication network, characterized in that the device is included in the transmitter and comprises:

[0019] - means for transferring a first data packet comprising a first time marker corresponding to the instant of transmission of the first packet,

[0020] - means for receiving a second data packet comprising the first time marker, a second time marker corresponding to the time of reception of the first packet and a third time marker corresponding to the time of transmission of the second packet,

[0021] - means for storing the time of reception of the second packet,

[0022] - means of calculation, from the three time markers and the time of reception of the second packet, with an average travel time,

[0023] - means for calculating a weighted average value and a deviation between the current travel time and average travel time,

[0024] - means of verifying whether the average travel time is greater than a pre-set threshold determined, the predetermined threshold being a function of the weighted average value and the deviation between current travel time and average travel time,

[0025] - means for taking into account the average travel time to update the time output of at least one data packet from the audiovisual stream if at least the average travel time is less than or equal to the predetermined threshold,

[0026] - means of not taking into account the average travel time for updating the output time of at least one data packet from the audiovisual stream if the average travel time is greater than the predetermined threshold.

[0027] Thus, the present invention provides a solution which makes it possible to take into account the variations in the characteristics of the communication network(s) used for the transmission of the video stream as well as the desynchronization of the clocks of the transmitter and the receiver. The filtering carried out makes the system resistant to the increase in transmission times linked to network congestion.

[0028] According to a particular embodiment, taking into account the average travel time to update the output time of at least one data packet of the audiovisual stream is further dependent on a comparison of the deviation between the current travel time and the average travel time with a minimum time.

[0029] Thus, increases in latency caused by a change in network condition and not by congestion are taken into account and do not cause the system to block.

[0030] According to a particular embodiment, the data packets of the audiovisual stream are transferred from the transmitter to the receiver via a plurality of communication networks and the method is carried out for each communication network.

[0031] Thus, the use of several network interfaces simultaneously allows a precise measurement, for example of the order of a millisecond, of the difference between the clocks of the transmitter and the receiver without resorting to a reference point such as for example an NTP server (acronym for the English term Network Time Protocol).

[0032] According to a particular mode, if the average travel time is greater than the predetermined threshold, the update of the exit time of the at least one packet is carried out from the average travel time of another communication network.

[0033] Thus, the present invention makes it possible to overcome errors linked to a problem in one of the communication networks.

[0034] According to a particular embodiment, the communication networks are Ethernet, Wi-Fi, xDSL, KA-SAT, BGAN or cellular type networks.

[0035] According to a particular mode, the weighted average value and the deviation between the current travel time and the average travel time are calculated from the following formulas:

[0036] rttcuir = (er - ee) where ee is the first time stamp and er is the time of receipt of the second packet,

[0037] _ _ ) ttmeancurr — aqlm

[0038] hifimcurr'j ? ^dev — «77

[0039] where aquo, as and ad are three coefficients used to weight the values ​​used, rtt meancurr is the average calculated at the present time, the variable rttmeanprec is the average calculated at the previous iteration and rttCUIT corresponds to the current value of the travel time. Brief description of the drawings

[0040] The above-mentioned features of the invention, as well as others, will appear more clearly on reading the following description of an exemplary embodiment, said description being made in relation to the attached drawings, among which:

[0041] [Fig-1] schematically illustrates an example of a communication system in which the present invention is implemented;

[0042] [Fig.2] schematically illustrates an example of distribution of data packets video between different communication networks connecting a transmitter and a receiver;

[0043] [Fig.3] illustrates an example of a timing diagram for sending holding packets according to the present invention;

[0044] [Fig.4] illustrates an example of architecture of a transmitter according to a mode of rea lization;

[0045] [Fig.5] illustrates an example of a method executed by a transmitter according to a mode of realization.

[0046] DETAILED DESCRIPTION OF EMBODIMENTS

[0047] [Fig.l] schematically illustrates an example of a communication system in which the present invention is implemented.

[0048] The communication system comprises a transmitter Em and a receiver Re connected to each other by at least one communication network Resl. In the example of [Fig.l], the transmitter Em and the receiver Re are connected by a plurality of communication networks Resl, Res2 to ResN.

[0049] The communication networks Resl, Res2 to ResN are, for example, Ethernet, Wi-Fi, xDSL (Digital Subscriber Line), KA-SAT, BGAN or cellular networks.

[0050] The data transmitted between the transmitter Em and the receiver Re are data packets of a video stream.

[0051] The transmission of video data packets is subject to time constraints so as to allow good reproduction of the video at the receiver.

[0052] Conventionally, the time constraint is managed at the transmitter level by a Time window. The time window is the time that must elapse between the entry and exit of a packet in the system. This constraint is important for several reasons. The main one is to compensate for network jitter.

[0053] Jitter refers to the variation in latency or the variation in the time taken to transfer information. Concretely, jitter represents the difference in arrival time between different packets sent at the same time. Jitter appears in the event of network congestion, timing drift or routing changes.

[0054] Jitter is highly variable over time and is highly dependent on the type of communication network used for transferring data packets from the transmitter to the receiver.

[0055] When a video encoder generates data packets, it does so at a precise rate corresponding to the number of frames per second in the video. It is important to be able to ensure that the packets leave the system at the same rate as they entered. However, the network introduces jitter, causing the interval between packets to vary. Strict adherence to the time window overcomes jitter, which is made all the more significant by the use of different networks.

[0056] [Fig.2] schematically illustrates an example of distribution of video data packets between different communication networks connecting a transmitter and a receiver.

[0057] The transferred data packets are distributed over the different communication networks Resl to ResN. Each time period corresponds to a configurable time interval, for example 1 ms. The transmitter Em permanently keeps a reference to the time interval corresponding to the current time, called now. The data packets are therefore stored in the time interval corresponding to now + the value of the time window. Indeed, this time interval corresponds to the moment from which the packet is considered late and must therefore no longer be transmitted. When they enter the transmission system, a sequence number is assigned to each packet. A time stamp is also affixed to each packet in order to be able to determine the time it spends in the transmission system.

[0058] The transmitter Em has a table of links, each link is representative of the flow rate of one of the communication networks. At each time interval, the transmitter Em scans the table of links. For each link, the transmitter Em determines the quantity of packets to send, depending on the call frequency and the capacity of the communication network.

[0059] The data packets to be sent are retrieved from the timeline following the priority principle according to the deadline of the latter, the tasks whose deadline is close receiving the highest priority. The data packets are retrieved first from the time slots whose expiration is closest. When a time slot is emptied, the next one is used up until the capacity of the communication network is exhausted. Data packets are distributed alternately on each communication network as long as the communication network throughput allows it.

[0060] In the example of [Fig.2], data packet 1 is transmitted over the communication network Resl, data packet 2 is transmitted over the communication network Res2, data packet 3 is transmitted over the communication network ResN, data packet 4 is transmitted over the communication network Resl, data packet 5 is transmitted over the communication network Res2, data packet 6 is transmitted over the communication network ResN and, since the entire bandwidth of the communication networks Res2 and ResN is reached, data packets 7 and 8 are transmitted over the communication network Resl.

[0061] Only data packets that have a chance of being received by the receiver Re in time are transmitted. This makes it possible to avoid a “snowball” effect, causing the collapse of the system if the quantity of data packets to be transmitted is greater than the capacity of the available communication networks. Having at its disposal the capacity of each communication network, the transmitter Em can determine the quantity of audio / video data that it can transmit at each instant. This value is made available to the system providing the video so that it can adapt the video rate to the network constraints of an instant T.

[0062] Since data packets transiting over different communication networks have different latencies, they do not arrive in the correct order. An algorithm must be implemented to overcome this problem. This algorithm must also be responsible for delivering the data packets while respecting the time constraints. Based on the sequence number affixed to each data packet by the transmitter, the receiver stores the packets in a table. Each packet is associated with an output time ts, obtained by the following formula:

[0063] ts = te + timewindow - clockdiff

[0064] te corresponds to the time of entry of the packet into the transmitter Em and ts corresponds to the time of exit of the packet into the receiver Re. At regular intervals, the receiver Re scans the packets present in the table, and compares their theoretical exit time with the current time. When the two match, the data packet is transmitted.

[0065] [Fig.3] illustrates an example of a timing diagram for sending hold packets according to the present invention.

[0066] In order to overcome the clock differences between the transmitter and the receiver, a synchronization mechanism is implemented according to the present invention. This makes it possible to determine the difference between the two clocks in real time, in order to allow the output of packets respecting the time window.

[0067] This mechanism is based on the transmission of data packets called keep-alive packets. These have, according to the present invention, a dual function. The first function is to ensure that the communication network is always operational by regularly sending data packets. The second function is the transmission of time markers allowing the calculation of the clock difference between the transmitter and the receiver, called here clockdiff. The packets contain time markers.

[0068] A first hold packet is sent by the transmitter to the receiver. This contains a time marker, denoted here ee, which corresponds to the time of transmission of said packet. This hold packet is then received by the receiver, which notes the reception time (rr). The receiver then responds with a hold packet, in which it reports the two time markers ee and rr, adding the time marker of the hold packet, denoted re. The transmitter receives the hold packet and stores its arrival time (er). It thus has 4 values, allowing the following calculations to be made:

[0069] emean = (er - ee) / 2

[0070] rmean = (rr - re) / 2

[0071] clockdiff = emean hnean

[0072] The values ​​emean and rmean correspond to the time of half of the transmission. Conventionally, on an unsaturated network, a packet takes as much time to travel the uplink as the downlink, these times being in theory identical. Any difference can be attributed to a clock difference between the transmitter and the receiver. However, it may happen that the networks do not behave symmetrically. This calculation is, according to the present invention, accompanied by statistical calculations in order to filter out aberrant time marker values.

[0073] rttcuir =(er - ee)

[0074] *rtrmecmpvei mountain bike — ..... / ..................................................... ' ™meancurr aqWt

[0075] =----------5------------

[0076] rto = rttmeancuiT +4 * rttdev

[0077] aquo, as and ad are three coefficients used to weight the values ​​used in the calculation. In one implementation, the values ​​used are for example aquo = 256, as = 32 and ad = 64. The variable rttmeancuiT is an average calculated at the present time and the variable rttmeanprec is an average calculated at the previous iteration.

[0078] rttcuir corresponds to the current value of the travel time. We calculate its weighted average value (rttmeancurr) and the deviation between the current travel time and the time average path length (rttdev). The values ​​contained in a holding packet are considered unusable when the rttcur exceeds the value of rto, i.e. its deviation from the average is 4 times greater.

[0079] This mechanism makes it possible to avoid errors in calculating clock differences (clockdiff) in poor network conditions. The transmitter can thus effectively filter unusable hold packets, and obtain a more reliable measurement. Since the clock difference (clockdiff) can vary over time due to a possible clock drift between the transmitter and the receiver, it is important to perform these exchanges and calculations regularly throughout the video transmission, for example every 50 ms.

[0080] [Fig.4] illustrates an example of architecture of a transmitter according to one embodiment

[0081] According to the example of hardware architecture represented in [Fig.4], the transmitter Em comprises, connected by a communication bus 401: a processor Proc 400; a random access memory RAM (Random Access Memory) 403; a read only memory ROM (Read Only Memory) 402; N network interfaces IF Res 1 4041, IF Res2 4042 to IF ResN 404N allowing the transmitter to communicate with the receiver Re and an input interface 406 receiving the data stream to be transmitted.

[0082] The processor 400 is capable of executing instructions loaded into the RAM 403 from the ROM 402, from an external memory (not shown), from a storage medium (such as an SD card), or from a communication network. When the transmitter Em is powered on, the processor 400 is capable of reading instructions from the RAM 403 and executing them. These instructions form a computer program causing the processor 400 to implement all or part of the method described in relation to [Fig.5].

[0083] The method described below in relation to [Fig. 5] can be implemented in software form by executing a set of instructions by a programmable machine, for example a DSP ("Digital Signal Processor" in English) or a microcontroller, or be implemented in hardware form by a machine or a dedicated component, for example an FPGA ("Field-Programmable Gate Array" in English) or an ASIC ("Application-Specific Integrated Circuit" in English). In general, the transmitter Em comprises electronic circuitry configured to implement the method described in relation to [Fig. 5].

[0084] [Fig.5] illustrates an example of a method performed by a transmitter according to one embodiment.

[0085] The present algorithm is executed periodically, for example with a periodicity of the order of a millisecond and for each communication network Res1, Res2 to ResN.

[0086] In step E500, the transmitter Em commands the transmission of a holding packet. The holding packet comprises a time marker ee which corresponds to the time of transmission of said packet.

[0087] In step E501, the transmitter Em receives from the receiver Re a holding packet which comprises the two time markers ee and rr and the time marker of the holding packet noted re received from the transmitter Em.

[0088] In step E502, the transmitter Em stores its arrival time(er).

[0089] In step E503, the transmitter Em performs the following calculations:

[0090] emean = (er - ee) / 2

[0091] rmean = (rr - re) / 2

[0092] rttcuir =(er - ee)

[0093] 1 ttmeancurr «77

[0094] (a ~ad)rltd^ r»deV =----------577-----------

[0095] rto = rttmean +4 * rttdev

[0096] In step E504, the transmitter Em checks whether the calculated value of rttdev is greater than a value rttdeVmin-

[0097] If so, the transmitter Em goes to step E506. If not, the transmitter Em goes to step E505.

[0098] In step E506, the transmitter calculates a new value of rttdeVmin according to the following formula:

[0099] rttdev min = rttdev min + min(1000, rttdev*0.02)

[0100] In this way, the rttdev min is increased by 2% of the current rttdev value, with a maximum of 1ms. This increase allows the local minimum of rttdev to be calculated rather than its global minimum. By doing so, latency increases caused by a change in network conditions and not by congestion are taken into account and do not cause the system to crash.

[0101] Once this operation has been carried out, the transmitter Em interrupts the present algorithm when only one communication network is used or takes in step E510 the value of the clockdiff calculated for another of the communication networks when several communication networks are used.

[0102] In step E505, the transmitter Em sets the value of rttdevmin equal to rttdev.

[0103] In the next step E507, the transmitter Em checks whether the value of rttcur is greater than the value of rto. If so, the transmitter Em interrupts this algorithm when only one communication network is used or takes in step E510 the value of the clockdiff calculated for another of the communication networks when several communication networks are used.

[0104] If not, the transmitter EM goes to step E508 and calculates a new clockdiff value according to the following formula:

[0105] Clockdiff = emean - rmean.

[0106] In step E510, the transmitter Em updates each packet output time with ts = te + timewindow - clockdiff.

Claims

Claims

1. Method for updating an output time of at least one data packet of an audiovisual stream transferred from a transmitter to a receiver via at least one communication network, characterized in that the method comprises the steps carried out by the transmitter of: - transferring (E500) a first data packet comprising a first time marker corresponding to the time of transmission of the first packet, - receiving (E501) a second data packet comprising the first time marker, a second time marker corresponding to the time of reception of the first packet and a third time marker corresponding to the time of transmission of the second packet, - storing (E502) the time of reception of the second packet, - calculating (E503), from the three time markers and the time of reception of the second packet, an average travel time,- calculation (E503) of a weighted average value and of a deviation between the current travel time and the average travel time, - verification (E507) whether the average travel time is greater than a predetermined threshold, the predetermined threshold being a function of the weighted average value and of the deviation between the current travel time and the average travel time, - taking (E508) into account the average travel time to update the output time of at least one data packet of the audiovisual stream if at least the average travel time is less than or equal to the predetermined threshold, - not taking (E510) into account the average travel time to update the output time of at least one data packet of the audiovisual stream if the average travel time is greater than the predetermined threshold.,

2. Method according to claim 1, characterized in that taking into account the average travel time to update the output time of at least one data packet of the audiovisual stream is further dependent on a comparison of the deviation between the current travel time and the average travel time with a minimum time.

3. Method according to claim 1 or 2, characterized in that the data packets of the audiovisual stream are transferred from the transmitter to the receiver via a plurality of communication networks and in that the method is carried out for each communication network. communication.

4. Method according to claim 3, characterized in that if the average travel time is greater than the predetermined threshold, the update of the exit time of the at least one packet is carried out from the average travel time of another communication network.

5. Method according to any one of claims 2 to 4, characterized in that the communication networks are Ethernet, Wi-Fi, xDSL, KA-SAT, BGAN or cellular type networks.

6. Method according to any one of claims 1 to 5, characterized in that the weighted average value and the deviation between the current travel time and the average travel time are calculated from the following formulas: rttcun- =(er - ee) where ee is the first time marker and er is the time of reception of the second packet, rttmeancurr ^de^^d rneancîîrr) rtf T — —i---d.--------—i--------------t aev ciyiu, where aquo, as and ad are three coefficients for weighting the values ​​used in the calculation, rttmeanCurr is the average calculated at the present time, the variable rttmeanprec is the average calculated at the previous iteration and rttCUIT corresponds to the current value of the travel time.

7. Device for updating an output time of at least one data packet of an audiovisual stream transferred from a transmitter to a receiver via at least one communication network, characterized in that the device is included in the transmitter and comprises: - means for transferring a first data packet comprising a first time marker corresponding to the time of transmission of the first packet, - means for receiving a second data packet comprising the first time marker, a second time marker corresponding to the time of reception of the first packet and a third time marker corresponding to the time of transmission of the second packet, - means for storing the time of reception of the second packet, - means for calculating, from the three time markers and the time of reception of the second packet, an average travel time, - means for calculating a weighted average value and a deviation between the current journey time and the average journey time, - means of verifying whether the average journey time is greater than a predetermined threshold, the predetermined threshold being a function of the weighted average value and the deviation between the current journey time and the average journey time, - means for taking into account the average travel time to update the output time of at least one data packet of the audiovisual stream if at least the average travel time is less than or equal to the predetermined threshold, - means of ignoring the average travel time to update the output time of at least one data packet of the audiovisual stream if the average travel time is greater than the predetermined threshold.

8. A computer program product characterized in that it comprises instructions for implementing, by equipment, the method according to any one of claims 1 to 6, when said program is executed by a processor of a node.

9. A storage medium characterized in that it stores a computer program comprising instructions for implementing, by equipment, the method according to any one of claims 1 to 6, when said program is executed by a processor of a node.

Images