A priority nack based rtc transmission optimization technique

Abstract

The application discloses an RTC transmission optimization technology based on Priority NACK, which comprises the following steps: firstly, a data structure PrioritySeqBuffer with a packet sequence number as a storage object is designed at a receiving end; secondly, a NACK request protocol with a priority function is designed at the receiving end; and finally, a protocol processing module PriorityNackModule is designed. According to the application, all lost data packets have the same priority by using a general NACK request protocol realized internally in WebRTC, and the rule is to perform packet loss retransmission once upon receiving a NACK request. Compared with the general NACK request protocol realized internally in WebRTC, the Priority NACK request can greatly increase the packet integrity and continuity in the scenario that only one lost data packet needs to be supplemented to make a large number of subsequent data packets continuous, thereby improving the decoding and rendering frame rate of a video frame and reducing lag.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of RTC transmission optimization technology, and particularly relates to a Priority NACK-based RTC transmission optimization technology. BACKGROUND

[0002] RTC (Real-Time Communications) is a real-time communication technology, of which WebRTC is a representative, which allows network applications or sites to establish a point-to-point (Peer-to-Peer) connection between browsers without the help of an intermediate medium, to realize the transmission of video stream and / or audio stream or other arbitrary data.

[0003] In order to ensure the real-time nature of communication, the UDP data transmission mode is often used in WebRTC. UDP is connectionless and has no congestion control, and the congestion of the network will not reduce the sending rate of the source host, so it is very suitable for RTC real-time communication scenarios, but the reliable transmission of data packets and congestion control need to be implemented by the application layer. SUMMARY

[0004] This part aims to summarize some aspects of the embodiments of the present application and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this part and the abstract and title of the specification to avoid obscuring the purpose of the abstract and title of the specification, and such simplifications or omissions cannot be used to limit the scope of the present application.

[0005] In view of the above problems of the existing Priority NACK-based RTC transmission optimization technology, the present application is proposed.

[0006] Therefore, the purpose of the present application is to provide a Priority NACK-based RTC transmission optimization technology, which is suitable for solving the problem that the reliable transmission of data packets and congestion control need to be implemented by the application layer.

[0007] To solve the above technical problems, the present application provides the following technical solutions: a Priority NACK-based RTC transmission optimization technology, comprising:

[0008] First, the receiving end designs a data structure PrioritySeqBuffer taking packet sequence number as the storage object;

[0009] Second, the receiving end designs a NACK request protocol with priority function;

[0010] Finally, a protocol processing module PriorityNackModule is designed;

[0011] The PrioritySeqBuffer stores the packet sequence number in GOP units, the Priority NACK request protocol is 32 bits, the RTP is a real-time transport protocol, and the WebRTC encapsulates packets using this protocol.

[0012] As a preferred solution of the Priority NACK-based RTC transmission optimization technology, the PrioritySeqBuffer structure supports real-time cleaning capability to clean decoded rendering data or data that has timed out in real time, so as to avoid sending a NACK retransmission request for data that has timed out.

[0013] As a preferred solution of the Priority NACK-based RTC transmission optimization technology, the PrioritySeqBuffer supports analyzing the continuity of the data packets and the priority of the missing data packets, and the calculation rule of the priority is that the sequence number of the largest received data packet minus the sequence number of the missing packet is the priority of the missing packet.

[0014] As a preferred solution of the Priority NACK-based RTC transmission optimization technology, the sequence number of the WebRTC packet in the NACK request protocol is 16 bits, the first 16 bits of the sequence number in the Priority NACK request protocol are the version number, the first 3 bits in the remaining 13 bits are the priority, and the maximum value of the 13 bits in the decimal system is 8192.

[0015] As a preferred solution of the Priority NACK-based RTC transmission optimization technology, the maximum cache data amount of the default data packet in the WebRTC is 2048, so the 13-bit priority meets the existing demand and leaves some room for future data cache optimization.

[0016] As a preferred solution of the Priority NACK-based RTC transmission optimization technology, the PriorityNackModule module is responsible for the encapsulation and analysis functions of the Priority NACK protocol, the receiving end encapsulates the Priority NACK protocol according to the priority of the missing data packet, the sending end analyzes the sequence number and priority of the missing data packet according to the Priority NACK protocol, converts the priority into the retransmission number of the data packet, and requests the existing packet loss retransmission module to retransmit the packet loss, wherein the conversion rule of the priority to the retransmission number is to use bit operation, and the leftmost bit that is 1 is found from right to left, and the position of the bit is the retransmission number.

[0017] As a preferred scheme of the Priority NACK-based RTC transmission optimization technology, in the RTC scenario, B frames are generally closed to reduce the delay because the delay requirement is relatively high, so an I frame is followed by a plurality of P frames in a GOP, the I frame is a key frame, and the P frame is a difference frame.

[0018] As a preferred scheme of the Priority NACK-based RTC transmission optimization technology, in the RTC scenario, B frames are generally closed to reduce the delay because the delay requirement is relatively high, so an I frame is followed by a plurality of P frames in a GOP, the I frame is a key frame, and the P frame is a difference frame.

[0019] As a preferred scheme of the Priority NACK-based RTC transmission optimization technology, in the RTC scenario, B frames are generally closed to reduce the delay because the delay requirement is relatively high, so an I frame is followed by a plurality of P frames in a GOP, the I frame is a key frame, and the P frame is a difference frame.

[0020] As a preferred scheme of the Priority NACK-based RTC transmission optimization technology, in the RTC scenario, B frames are generally closed to reduce the delay because the delay requirement is relatively high, so an I frame is followed by a plurality of P frames in a GOP, the I frame is a key frame, and the P frame is a difference frame.

[0021] Advantages of the application:

[0022] Compared with the general NACK request protocol implemented in WebRTC, the Priority NACK request can greatly increase the packet integrity and continuity in the scenario of a large number of subsequent packets being continuous, thereby improving the decoding and rendering frame rate of the video frame and reducing the lag. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments description will be briefly introduced as follows. Obviously, the drawings in the following description only constitute some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings. Among them:

[0024] Figure 1 A transmission architecture diagram of the Priority NACK-based RTC transmission optimization technology proposed in the present application;

[0025] Figure 2 A GOP diagram of the Priority NACK-based RTC transmission optimization technology proposed in the present application;

[0026] Figure 3 A GOP packet loss diagram of the Priority NACK-based RTC transmission optimization technology proposed in the present application;

[0027] Figure 4 A NACK request priority diagram of the Priority NACK-based RTC transmission optimization technology proposed in the present application;

[0028] Figure 5 A Priority NACK protocol diagram of the Priority NACK-based RTC transmission optimization technology proposed in the present application. DETAILED DESCRIPTION

[0029] In order to make the above-mentioned features and advantages of the present application more apparent and understandable, the specific embodiments of the present application will be described in detail in conjunction with the drawings in the specification.

[0030] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present application, therefore the present application is not limited by the specific embodiments disclosed below.

[0031] Secondly, the "one embodiment" or "embodiment" referred to herein means that the specific features or characteristics can be included in at least one implementation of the present application. In this specification, "in one embodiment" appearing in different places does not mean the same embodiment, nor does it mean an embodiment that is not alone or selectively excluded from other embodiments.

[0032] Secondly, the present invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of the present invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not according to the usual scale. Furthermore, the schematic diagrams are merely examples and should not limit the scope of protection of the present invention. In addition, actual fabrication should include the three-dimensional spatial dimensions of length, width, and depth.

[0033] Reference Figures 1-5 As one embodiment of the present invention, an RTC transmission optimization technique based on Priority NACK is provided, including:

[0034] First, the receiving end designs a data structure PrioritySeqBuf that stores packet sequence numbers.

[0035] fer;

[0036] Secondly, the receiving end is designed with a NACK request protocol that has a priority function;

[0037] Finally, a protocol processing module, PriorityNackModule, was designed.

[0038] PrioritySeqBuffer stores packet sequence numbers in units of Group of Pages (GOPs). The Priority NACK request protocol is 32-bit. RTP is a real-time transport protocol, which is used to encapsulate data packets in WebRTC.

[0039] Furthermore, the PrioritySeqBuffer structure supports real-time cleanup capabilities to clean up decoded or timed-out data in real time, avoiding the sending of timed-out NACK retransmission requests.

[0040] Furthermore, PrioritySeqBuffer supports analyzing the continuity of data packets and the priority of missing data packets. The priority is calculated by subtracting the sequence number of the missing packet from the sequence number of the largest received data packet.

[0041] Furthermore, the WebRTC packet sequence number in the NACK request protocol is 16 bits, while the Priority NACK request protocol is 32 bits. The first 16 bits are the sequence number, the first 3 bits of the last 16 bits are the version number, which is convenient for future expansion, and the remaining 13 bits are the priority. The maximum value of 13 bits in decimal is 8192.

[0042] Furthermore, the default maximum cache size for data packets in WebRTC is 2048, so a 13-bit priority satisfies current requirements and leaves room for future data caching optimization.

[0043] Further, the PriorityNackModule module is responsible for encapsulation and analysis functions of the Priority NACK protocol, the receiving end encapsulates the Priority NACK protocol according to the priority of the missing data packet, and the sending end parses the sequence number and priority of the missing data packet according to the Priority NACK protocol, converts the priority into the number of data packet retransmissions, and requests the existing packet loss retransmission module to retransmit the packet loss, wherein the conversion rule of the priority to the number of retransmissions is to use bit operation, and each bit in the 13 bits occupied by the priority is taken from right to left, and the position of the leftmost bit is found. The position is the number of retransmissions.

[0044] Further, in the RTC scenario, the B frame is generally closed to reduce the delay due to the relatively high delay requirement, so an I frame starts a GOP, and a plurality of P frames follow the I frame, the I frame is a key frame, and the P frame is a difference frame.

[0045] Further, the GOP is the interval between two I frames, in the conventional implementation of the RTC, the encoded video frame is split into an RTP packet for transmission, and the receiving end ensures the decoding correctness of the decoder, and only when the video frame of a certain data packet is complete and all the video frames before the video frame in the GOP are complete, the data packet is sent into the decoder for decoding and finally rendered to the screen.

[0046] Further, the data packets 4 and 16 are lost in network transmission, the NACK module sends a NACK request to the sending end to request the sending end to retransmit the data packets 4 and 16, since the NACK request has no priority concept, the sending end processes the retransmission of the data packets 4 and 16 with the same logic, when the data packet 4 is retransmitted successfully, the receiving end decodes and renders the P frame to which the data packet 4 belongs and all the P frames before the data packet 16, and when the data packet 16 is retransmitted successfully, the receiving end will only decode and render the P frame to which the data packet 16 belongs.

[0047] Further, the Priority NACK is a NACK request protocol with a priority attribute, the Priority NACK request performs 13 times of packet loss retransmission at most for a data packet.

[0048] Working principle:

[0049] The sending end sends data packets to the receiving end's PacketReceiver module through the PacketSender module. The PacketReceiver module transfers the received data packets to the PrioritySeqBuffer module for packet loss and priority analysis in real time. If there is a Priority NACK packet to be sent, it is transmitted to the PriorityNackModule for protocol packaging and returned to the sending end's PriorityNackModule module. This module analyzes the sequence number and priority request of the lost packet and requests the PacketSender module to retransmit the packet.

[0050] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application, which should be covered in the scope of the claims of the present application.

Claims

1. A Priority NACK request protocol based RTC transmission optimization technique characterized in that, The application relates to a priority NACK request protocol and a priority NACK request protocol processing module. First, a data structure PrioritySeqBuffer taking a packet sequence number as a storage object is designed at a receiving end; Second, a Priority NACK request protocol is designed at the receiving end; Finally, a protocol processing module PriorityNackModule is designed; The PrioritySeqBuffer stores packet sequence numbers in units of GOPs, the Priority NACK request protocol is 32 bits, the RTP is a real-time transmission protocol, and the protocol is used to encapsulate data packets in WebRTC; The PriorityNackModule module is responsible for encapsulation and analysis functions of the Priority NACK request protocol, the receiving end encapsulates the Priority NACK request protocol according to the priority of the missing data packet, the sending end analyzes the sequence number and priority of the missing data packet according to the Priority NACK request protocol, converts the priority into a data packet retransmission number, and requests an existing packet loss retransmission module to retransmit the packet loss, wherein the conversion rule of the priority into the retransmission number is to take each bit in the 13 bits occupied by the priority from right to left, find the position of the leftmost 1, and the position is the retransmission number; An I frame starts a GOP, and a plurality of P frames follow the I frame, the I frame is a key frame, and the P frame is a difference frame; The GOP is an interval between two I frames, in a conventional implementation of the RTC, the coded video frame is split into RTP packets for transmission, and the receiving end ensures the decoding correctness of the decoder, and only when a data packet belongs to a complete video frame and all video frames before the video frame in the GOP are complete, the data packet is sent into the decoder for decoding and finally rendered on the screen.

2. The RTC transmission optimization technique based on Priority NACK request protocol according to claim 1, characterized in that: The PrioritySeqBuffer structure supports real-time cleaning capability, and the decoded and rendered data or the overdue data are cleaned in real time, so that the NACK retransmission request of the overdue data is prevented from being sent.

3. The Priority NACK request protocol based RTC transmission optimization technique of claim 1, wherein: The PrioritySeqBuffer supports analysis of the continuity of data packets and the priority of missing data packets, and the priority calculation rule is that the priority of a missing packet is the value obtained by subtracting the sequence number of the missing packet from the sequence number of the largest received data packet.

4. The Priority NACK request protocol based RTC transmission optimization technique of claim 1, wherein: In the Priority NACK request protocol, the WebRTC data packet sequence number is 16 bits, the first 16 bits in the 32 bits of the Priority NACK request protocol are the sequence number, the first 3 bits in the last 16 bits are the version number, the remaining 13 bits are the priority, and the maximum value of the 13 bits in the decimal system is 8192.

5. The Priority NACK request protocol based RTC transmission optimization technique of claim 1, wherein: In the WebRTC, the maximum cache data amount of a data packet is 2048.

6. The Priority NACK request protocol based RTC transmission optimization technique of claim 1, wherein: The receiving end data packets 4 and 16 are lost in network transmission, the NACK module sends a NACK request to the sending end, requiring the sending end to retransmit the data packets 4 and 16, since the NACK request has no priority concept, the sending end will process the retransmission of the data packets 4 and 16 with the same logic, when the data packet 4 is retransmitted successfully, the receiving end will decode and render the P frame to which the data packet 4 belongs and all the P frames before the subsequent data packet 16, when the data packet 16 is retransmitted successfully, the receiving end will only decode and render the P frame to which the data packet 16 belongs at most.

7. The Priority NACK request protocol based RTC transmission optimization technique of claim 1, wherein: The Priority NACK request protocol is a request protocol with a priority attribute, the Priority NACK request protocol performs 13 times of packet retransmission at most for a certain data packet once.

Images