Common transport architecture for heterogeneous data streams
Through the universal transmission architecture (UTF) of heterogeneous data flows, the system complexity and transmission efficiency problems caused by the use of multiple protocols in the existing technology are solved, and flexible data flow management and resource optimization are realized to adapt to the communication needs of different application scenarios. .
Patent Information
- Application Number
- CN202210259616.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2021-06-22
- Filing Date
- 2022-03-16
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-03-16
AI Technical Summary
The prior art requires the use of multiple protocols in network communication to meet the complex communication needs of different application scenarios, resulting in complex system architecture and network management, and the optimal overall transmission efficiency cannot be achieved in specific network scenarios.
A universal transmission architecture (UTF) of heterogeneous data flows is proposed to manage reliable and unreliable data flow transmission through session layer and connection layer hierarchical protocols, providing a separation model between transmission control and reliability management, and supporting the creation of different priorities and reliable data flows, and simplify application development through APIs, reducing the need for processing, memory, storage resources and network bandwidth.
It simplifies application development, reduces network traffic, reduces resource investment, realizes a flexible transmission architecture, adapts to various transmission requirements, improves transmission efficiency and reliability, and reduces system complexity.
Smart Images

Figure CN115348336B_ABST
Abstract
Description
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application claims priority to U.S. patent application No. 17 / 319,738, filed on May 13, 2021; and U.S. patent application No. 17 / 354,657, filed on June 22, 2021. Technical Field
[0003] The present invention relates to network communication technology, and more specifically, to a universal transmission architecture technology for heterogeneous data streams. Background Art
[0004] In network communication, the transport layer can provide services to the application layer. Depending on the network protocol, the services may include connection-oriented communication, reliable transmission, congestion avoidance, error correction, flow control and other services. The application layer may have different transport layer requirements depending on the application scenario, such as reliable transmission of control commands, unreliable transmission of multimedia streams, etc. The traditional solution is to use existing protocols or design specific protocols to meet the needs of specific scenarios. Therefore, complex systems for various application scenarios usually have to use multiple protocols to meet the complete communication needs, which makes the system architecture and network management more complicated. Summary of the invention
[0005] The present invention provides a method, device and system for implementing a universal architecture for heterogeneous data stream transmission.
[0006] On the one hand, the present invention provides a device for communicating between a sending application and a receiving application of a receiving device. The device includes a processor, wherein the processor establishes a stream for transmitting data between the sending application and the receiving application; receives a first request from the sending application and transmits metadata to the receiving application; receives a second request from the sending application and transmits application data to the receiving application; after determining that a frame including application data and metadata is less than or equal to a maximum frame capacity, the frame will include application data and metadata when constructing the frame; and sends the frame to the receiving device in the form of a data packet.
[0007] In a second aspect, the present invention provides a method for communicating between a sending application and a receiving application of a receiving device. The method includes constructing a stream for transmitting data between the sending application and the receiving application; receiving a first request from the sending application to transmit metadata to the receiving application; receiving a second request from the sending application to transmit application data to the receiving application; after determining that a frame including application data and metadata is less than or equal to a maximum frame capacity, constructing the frame to include application data and metadata; and sending the frame to the receiving device in the form of a data packet.
[0008] In a third aspect, the present invention provides a non-temporary computer-readable storage medium containing executable instructions. When a processor runs the executable instructions, the operations to be performed are to build a transmission architecture for communication between a sending application and a receiving application of a receiving device. These operations include: establishing a stream for transmitting data between a sending application and a receiving application; receiving a first request from the sending application to transmit metadata to the receiving application; receiving a second request from the sending application to transmit application data to the receiving application; determining that a frame including application data and metadata is less than or equal to the maximum frame capacity, then including application data and metadata when constructing the frame; and sending the frame to the receiving device in the form of a data packet. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] It will be helpful to better understand the content of the present invention by referring to the accompanying drawings when reading the following detailed description. It should be emphasized that according to convention, the various parts in the drawings are not drawn to scale. On the contrary, for the sake of clarity, the sizes of various different parts in the drawings are arbitrarily enlarged or reduced.
[0010] Figure 1 It is an example diagram of an environment for building a general transmission architecture for heterogeneous data streams drawn according to an implementation scheme of the present invention.
[0011] Figure 2 2 is a system example diagram for describing a general transmission architecture of heterogeneous data streams drawn according to an embodiment of the present invention, represented by system 200.
[0012] Figure 3 This is an example diagram of the structure of a data packet.
[0013] Figure 4 This is an example diagram of the structure of an initial data packet.
[0014] Figure 5 This is an example diagram of the structure of a data packet.
[0015] Figure 6 This is an example diagram of the structure of a frame.
[0016] Figure 7 This is an example diagram of the structure of a data stream frame.
[0017] Figure 8 The following is an example diagram of the structure of a confirmation frame.
[0018] Fig. 9 This is an example diagram of the structure of a closing frame.
[0019] Fig.10 The following is an example diagram of the structure of a feedback frame.
[0020] Fig.11 is an example flow chart of a technique for transmitting metadata.
[0021] Fig.12 This is an example diagram of piggyback confirmation.
[0022] Fig.13 is an example diagram of program code using a common transport architecture for heterogeneous data streams. DETAILED DESCRIPTION
[0023] Several models for communicating over a network have been implemented in the prior art. Two models are the Open Systems Interconnection (OSI) model and the Transmission Control Protocol / Internet Protocol (TCP / IP) model. Other models exist. These models will not be described in detail herein because they do not contribute to the understanding of the general transmission architecture of heterogeneous data streams described in the present invention. However, a high-level description of these models will be provided to facilitate understanding of the context.
[0024] The OSI model is a well-known seven-layer reference model that describes the functions in which systems or applications can communicate over a network. The model includes the physical layer, data link layer, network layer, transport layer, session layer, presentation layer, and application layer. The physical layer is the lowest layer and refers to the electrical or physical interface that connects a device to the transmission medium. The data link layer receives raw data from the physical layer and packages them into frames. The network layer is responsible for organizing data into packets and transmitting packets across multiple networks. The transport layer handles end-to-end transmission of applications. The session layer is a software layer that manages communication sessions between devices. The presentation layer is used to manage data formats and presentation, such as by converting data formats between applications. The application layer is used to allow communication between applications.
[0025] TCP / IP is also a model for end-to-end communication between devices over a network. The TCP / IP model includes the link layer, network layer, transport layer, and application layer. The role of the link layer can be considered similar to the services provided by the physical layer and data link layer in the OSI model. The role of the network layer can be considered similar to the services provided by the network layer in the OSI model. The role of the transport layer can be considered similar to the services provided by the transport layer in the OSI model. The role of the application layer can be considered similar to the services provided by the session layer, presentation layer, and application layer in the OSI model.
[0026] The transport layer in any of the above models (or other models) can implement mechanisms for error control, network congestion control, data retransmission (such as retransmission of lost data), confirmation of received data, deletion of duplicate data (such as discarding received duplicate data), etc. The transport layer can provide a data channel for the communication needs of the application.
[0027] The communication requirements (such as quality of service) of an application may vary from task to task or from data type to data type. Data types may include, but are not limited to, files, emails, web content, real-time audio / video data streams, control commands, and the like. For example, in a real-time communication (RTC) session, a user may send a text message or transfer a file to another user. For example, users communicating in an RTC session may be able to see (such as through video data) and hear (such as through audio data) each other. Therefore, the data types that can be transmitted by an RTC session include: data types that must be received completely lossless (text messages and files); data types that can be allowed to be lost; and data types that can have a higher priority than other data types. If a portion of a text message or a portion of a file is lost (i.e., the recipient does not receive it), the recipient may not be able to reconstruct or view the data. On the other hand, video and audio decoding can allow data loss. In addition, audio data can have a higher transmission priority than video data.
[0028] The transport layer can provide transport layer protocols suitable for different data types or application needs. For example, the Transmission Control Protocol (TCP) and the User Datagram Protocol (UDP) can be used. The TCP protocol can be used for data transmission (such as data delivery) that requires reliability, order, and error checking; when there is no need to guarantee delivery, order, or avoid duplication, the UDP protocol can be used. For example, for the transmission of files, e-mails, web content, control commands, etc., TCP may be more suitable due to their reliability requirements. On the other hand, for the transmission of time-sensitive real-time audio / video data streams, UDP can be used. In addition, other protocols can also be used, such as AppleTalk Transaction Protocol (ATP), Data Packet Congestion Control Protocol (DCCP), Fibre Channel Protocol (FCP), Stream Control Transmission Protocol (SCTP), Structured Stream Transport (SST), custom protocols for specific applications, and so on.
[0029] To accommodate different communication requirements or tasks of an application (e.g., the application layer of a communication model), application developers may have to select from one or more available protocols and / or design a specific protocol to meet the use case or task of a specific application. An application may use multiple protocols to meet complete communication requirements.
[0030] If multiple protocols must be used simultaneously, it will complicate application development and lead to further complexity of system (such as application) architecture and network management. Those skilled in the art should understand that due to the different implementation principles between different protocols, such as competition and conflict between TCP and UDP, the application may not be able to achieve the best overall transmission efficiency in a specific network scenario.
[0031] The universal transmission framework (UTF) for heterogeneous data streams described in the present invention can meet various transmission requirements (such as reliable and unreliable data stream transmission), thereby solving the above-mentioned problems. The following will also explain how to meet various transmission requirements through flexible configuration and adaptive methods. UTF defines a protocol that can be used to transmit reliable and unreliable data. Even if the underlying protocol used by UTF is an unreliable protocol (such as UDP), lost or potentially lost data can be retransmitted.
[0032] UTF can be a layered protocol consisting of two layers: the session layer and the connection layer, as described below. The session layer manages all application streams and can encapsulate the actual data of the stream into stream frames. The session layer provides an interface for abstract (such as transparent, etc.) management of reliable and unreliable transport streams. The session layer can also be used to manage frame retransmission. The connection layer can handle (such as manage, determine, etc.) network conditions. The connection layer can notify the session layer of lost (or potentially lost) frames. The session layer can also notify the application layer of lost frames.
[0033] UTF can be used to transmit multiple data streams with different requirements (e.g., different reliability, priority or other requirements). UTF can provide a hierarchical model that separates transmission control from reliability management, thereby providing a general transmission protocol that can reuse multiple streams to meet the needs of heterogeneous application streams. Applications (such as application developers) can use UTF through an application programming interface (API). For example, different streams can be created by calling (e.g., requesting, etc.) an API. For example, an API can be used to create a reliable stream. As another example, another API can be used to create an unreliable stream. In one embodiment, an unreliable stream can be used for real-time applications (such as transmission, etc.), such as in scenes such as video, audio or instant messaging. In another embodiment, to meet the requirements of real-time transmission, an unreliable stream can execute (such as adopting, supporting, using, etc.) one or more technologies or strategies to reduce delays, such as forward error correction, retransmission of lost data packets within a predetermined time, or retransmission timeouts predetermined for lost data packets, etc. Unreliable streams configured to meet real-time transmission requirements are referred to as partially reliable streams in the present invention. Other APIs can use the created stream to transmit or receive data.
[0034] By using the UTF described in the present invention, for the created stream and / or for the actual data transmitted using the stream, the application does not need to manage congestion control, such as having to determine the time to send data packets to avoid network congestion, etc.; and there is no need to manage packet loss or potential packet loss.
[0035] UTF uses new technologies and structures to implement the following operations: confirmation of receipt of packets and frames by the receiver; confirmation of confirmations sent by the receiver at the sender; transmission and confirmation of metadata; and congestion control. These technologies and structures can simplify application development and reduce network traffic, thereby reducing investment in processing, memory, storage resources or network bandwidth, thereby achieving emissions reductions, as described in detail below.
[0036] Figure 1 FIG. 1 is a diagram of an example environment in which a general transport architecture can be used with heterogeneous data streams, represented by environment 100. Figure 1 As shown, environment 100 may include multiple devices and networks, such as device 102, device 104, and network 106. The devices may be implemented by any configuration of one or more computers, such as a microcomputer, a mainframe computer, a supercomputer, a general-purpose computer, a dedicated / special purpose computer, an integrated computer, a database computer, a remote server computer, a personal computer, a laptop computer, a tablet computer, a mobile phone, a personal data assistant (PDA), a wearable computing device, or a computing service provided by a computing service provider (such as a network host or a cloud service provider).
[0037] In some embodiments, the environment 100 may be implemented using a general-purpose computer with a computer program, and the corresponding methods, algorithms and / or instructions of the present invention may be executed when the computer program is run. In addition, a dedicated computer / processor comprising dedicated hardware may also be used to run any method, algorithm or instruction of the present invention.
[0038] Device 102 may include a processor 108 and a memory 110. Processor 108 may be any type of device capable of managing or processing data. The terms "signal", "data" and "information" are used interchangeably. Processor 108 may include a central processing unit (such as a central processing unit, i.e., CPU), a graphics processor (such as a graphics processing unit, i.e., GPU), an intellectual property (IP) core, an application-specific integrated circuit (ASIC), a programmable logic array (such as a field programmable gate array, i.e., FPGA), an optical processor, a programmable logic controller, a microcontroller, a microprocessor, a digital signal processor, or any other suitable circuit in any number of combinations. Processor 108 may also be distributed on multiple machines that may be directly adapted or connected via a network (such as each machine or device is equipped with one or more processors).
[0039] The memory 110 may be any temporary or non-temporary device capable of storing code and data that can be accessed by the processor (via, for example, a bus). The memory 110 of the present invention may include any combination of random access memory devices (RAM), read-only memory devices (ROM), firmware, optical disks, magnetic disks, hard disk drives, solid-state drives, flash drives, secure digital (SD) cards, memory sticks, compact flash (CF) cards, or any suitable type of storage devices. The memory 110 may also be distributed across multiple machines or devices, such as network-based storage or cloud-based storage. The memory 110 may include data, an operating system, and one or more applications. The data may be any data for processing (such as an audio stream, a video stream, or a multimedia stream), and the application may include instructions executed by the processor 108 to generate control signals that may be used to perform various functions in the method or process of the present invention.
[0040] In some embodiments, the device 102 may also include an auxiliary storage device (such as an external storage device). In high processing requirements, the auxiliary storage device may provide additional storage space. The auxiliary storage device may be any suitable non-temporary computer-readable medium in the form of a storage device, such as a read-only memory device (ROM), an optical disk, a magnetic disk, a hard disk drive, a solid-state drive, a flash drive, a secure digital (SD) card, a memory stick, or a compact flash (CF) card. In addition, the auxiliary storage device may be a component of the device 102 or a shared device accessed by multiple devices over a network. In some embodiments, the application in the memory 110 may be stored in whole or in part in the auxiliary storage device and loaded into the memory 110 as needed for processing.
[0041] Device 102 may also include input / output (I / O) devices (i.e., I / O devices 112). I / O devices 112 may be any type of input device, such as a keyboard, a numeric keypad, a mouse, a trackball, a microphone, a touch-sensitive device (e.g., a touch screen), a sensor, or a gesture-sensing input device. I / O devices 112 may also be any output device that transmits visual, auditory, or tactile signals to a user, such as a display, a touch-sensitive device (e.g., a touch screen), a speaker, a headset, a light-emitting diode (LED) indicator light, or a vibration motor. For example, I / O device 112 may be a display for presenting graphical data, such as a liquid crystal display (LCD), a cathode ray tube (CRT), an LED display, or an organic light-emitting diode (OLED) display. In some cases, an output device may also serve as an input device, such as a touch screen.
[0042] Device 102 may also include a communication device 114 so that it can communicate with another device through network 106. Network 106 can be any combination of any type of communication network, such as a wireless network or a wired network. Wireless networks may include, for example, Wi-Fi networks, Bluetooth networks, infrared communication networks, near field communication (NFC) networks, cellular data networks, etc. Wired networks may include, for example, Ethernet networks. Network 106 may be a local area network (LAN), a wide area network (WAN), a virtual private network (VPN), or the Internet. Network 106 may include multiple server computers (referred to as "servers"). Servers may be connected to each other, and one or more servers may also be connected to end-user devices, such as device 102 and device 104. Communication device 114 may include any combination of any number of devices for sending and receiving data, such as transponders or transceiver devices, modems, routers, gateways, wired network adapters, wireless network adapters, Bluetooth adapters, infrared adapters, NFC adapters, cellular data antennas, etc.
[0043] Similar to device 102, device 104 is equipped with processor 116, memory 118, I / O device 120 and communication device 122. The operation of elements 116-122 of device 104 can be similar to 108-114 in device 102. For example, device 102 can include (or, for example, execute, run, etc.) a sending application that sends data to device 104, and device 104 can include a receiving application that receives data from device 102, and vice versa. Therefore, both devices 102 and 104 can act as senders and receivers at the same time. Device 102 can communicate with device 104 through network 106. Devices 102 and 104 can also communicate with other devices (not shown) connected to network 106. Devices 102 and 104 can include (or the sending application can use / or the receiving application can use) UTF to communicate through the network. It should be noted that some functions of devices 102 and 104 do not have to operate in the same way.
[0044] It should also be noted that the devices 102, 104, and the components or assemblies of the system 100 are not limited to Figure 1 Without departing from the scope of the present invention, devices 102 and 104 and environment 100 may include more or fewer components, assemblies, and hardware or software modules for performing various other functions related to or other than network communications using UTF.
[0045] Figure 2 200 is an example diagram of a system 200 for illustrating a general transmission architecture for heterogeneous data streams. The system 200 includes a user device 204 and a network 206. The user device (such as an application in the device) is also referred to as a sender in the present invention, and can transmit data to another user device (not shown) using UTF 202 through the network 206. The other device is referred to as a receiver in the present invention. The user device 204 (i.e., the sender) can be Figure 1 If the receiving party is device 102 or device 104 in the network, the receiving party may be the other of device 102 or device 104. Network 206 may be Figure 1 The network 106 in FIG. 100. The sender (such as the user device 204) can also be the receiver of the data. Therefore, the user device 204 can be the sender and the receiver at the same time.
[0046] UTF 202 may be included in user device 204. For example, UTF 202 may be a module of user device 204. A server of network 206 may also include a UTF 202 module. It should be noted that a recipient may also include its own UTF 202, and the same numerical sequence is used to refer to the recipient's UTF 202 module. For example, UTF 202 may be implemented as a software program (e.g., software module, library, etc.) that can be executed by a computing device, such as user device 204, or Figure 1 The software program may include machine-readable instructions that may be stored in a memory (such as memory 110) and executed by a processor (such as Figure 1 When the processor 108 of the computer 100 executes the instruction, the computing device can perform the operations described in the present invention. The UTF 202 can also be implemented using dedicated hardware or firmware, or multiple processors, memories, or both.
[0047] UTF 202 may include a session management module 208 and a connection management module 210. Session management module 208 may include a stream factory 212, a stream management module 214, and a scheduling module 216. Connection management module 210 may include a packet loss detection module 220, a data packet management module 218, a network congestion control management module 224, and an acknowledgement management module 222. In some embodiments, additional modules may be added, modules may be combined, and / or modules may be removed.
[0048] The session management module 208 may be used to manage streams. Applications may use the API of the session management module 208 to create (and destroy) and manage streams. A stream may be defined as a sequence of data elements that an application (e.g., an application executing on a user device 204) sends / receives (in terms of the progress of time) to / from another application executing on another computing device over a network 206. A stream may represent (e.g., implement, be used for, etc.) reliable or unreliable transmission. Retransmission control functionality is encapsulated in each stream itself (or more accurately, in the stream management itself).
[0049] The session management module 208 can encapsulate the actual data of the stream (such as the data to be sent) into a stream frame. The stream frame can be a data container with a specific (such as deterministic, known, predictable, etc.) structure and semantics, so that the receiving party can clearly parse (such as use, read, etc.) the content of the received stream frame. According to the present invention, in addition to the stream frame, other forms of frames can also be used.
[0050] The client application can establish a connection between the client device executing the client application and another device (such as a server or another client device). After the connection is established, the client application can use the API of the stream factory 212 to create one or more streams. Creating a stream can include assigning priorities to data streams. Metadata (also referred to as custom metadata, application metadata, or stream metadata in the present invention) can be associated with the stream. Metadata can be associated with the stream when the stream is created or before the stream is used to send application data. Creating a stream can include specifying customized metadata. In one embodiment, the metadata can be constant. That is, during the period when the sender and the receiver are connected to each other, when the stream is in an open state, the stream metadata sent to the receiver will not be changed at a later stage.
[0051] Without loss of generality, three streams can be created in a multimedia communication system. The signal channel can communicate through a reliable stream with the highest priority. An audio or video stream can use an unreliable stream, in which the priority of the audio stream can be higher than that of the video stream. Custom metadata transmitted through the signaling channel can carry a description of the audio or video stream. For example, metadata can include a standard (such as H.264, H.265, AV1, etc.), according to which video data is encoded before using the video stream for transmission. Without such metadata, the recipient may not be able to correctly decode the video data. In another embodiment, metadata can be included in a data packet of the audio or video stream.
[0052] The stream management module 214 can maintain a corresponding record associated with the stream for each frame in the transmitted stream. The relevant record can include the current state of the frame, etc. The current state can be used to indicate whether the frame is successfully transmitted or lost, as described below.
[0053] The scheduling module 216 prioritizes frames for transmission. The scheduling module 216 can prioritize frames using a custom configuration. The scheduling module 216 can transmit frames to the underlying connection management module 210 based on the priority of the frames, as shown by arrow 226. For example, as described above, audio frames can be transmitted before video frames because the audio stream has a higher priority than the video stream when it is created. Audio frames can also be transmitted more efficiently than video frames, and video frames may require much more bandwidth (such as 10 times or 20 times) than audio data to transmit. Therefore, under poor network connection conditions or low-bandwidth network conditions, by prioritizing the transmission of audio frames, the receiving user can actually hear the voice of the sending user immediately even if the sending user cannot be seen immediately (using video data).
[0054] The scheduling module 216 can determine the priority of the frame based on the configured priority of the corresponding flow. The scheduling module 216 transmits (e.g., provides, switches, transmits, etc.) the frame to the connection management module 210, as shown by arrow 226, which indicates a request for the connection management module 210 to send a frame for transmission through the network 206. The scheduling module 216 can schedule when to send the corresponding next frame. The scheduling module 216 can prioritize the flow frames before transmitting them to the connection management module 210.
[0055] In one embodiment, the scheduling module 216 can use a maximum-minimum fairness strategy to schedule the flow frames according to the priority of each flow. In another embodiment, the scheduling module 216 can use different strategies to use the statistical data collected by the data packet management module 218, the congestion control management module 224, other statistical data or a combination thereof to schedule the flow frames, thereby achieving the best overall transmission.
[0056] The connection management module 210 can encapsulate frames (such as stream frames or other frames) into connection data packets and send the data packets out using the underlying protocol. That is, the connection management module 210 can combine multiple stream frames into a data packet for transmission on the network. The underlying protocol can be an unreliable protocol (such as UDP), a reliable protocol (such as TCP) or some other protocol. When the underlying protocol is an unreliable protocol (such as UDP), retransmission control can be performed by UTF 202. When the underlying protocol is a reliable protocol (such as TCP), retransmission can be handled by the reliable protocol itself, and UTF 202 may not be able to control retransmission at this time.
[0057] The packet management module 218 can serialize (when sending to a receiver) and deserialize (when receiving from a sender) packets. Figure 3-5 As shown, the data packet may have a predefined format (such as a structure) that is used to describe the content contained in the data packet. For example, the data packet may include a flow frame, an acknowledgement frame (or simply an ACK frame), some other frames, or a combination of the above forms.
[0058] The packet management module 218 can manage the caching policy. The packet management module 218 can cache packets sent to the network 206. The packet management module 218 can maintain a record of packets sent by the connection management module 210 (e.g., transmitted via the network 206). When an acknowledgment of a packet is received from the recipient, the packet management module 218 can update the record corresponding to the acknowledgment packet. When a packet is acknowledged, the packet can be deleted from the cache. If the packet is not acknowledged, the connection management module 210 can notify the session management module 208 that the frame contained in the packet has been lost. The session management module 208 can mark the frame as lost. In one embodiment, the session management module 208 can notify the stream (or its application) to which the frame belongs that the frame has been lost.
[0059] A data packet may include zero or more flow frames and zero or more confirmation frames, as described below. Since the confirmation is at the data packet level, when the confirmation is received, the cache can be used to determine which flow frames have been confirmed and / or which confirmation frames have been received by the recipient. After receiving the confirmation, the connection management module 210 can then inform the session management module 208 of the confirmation information, as shown by arrow 228. Informing the session management module 208 of the confirmation information of the data packet means notifying the session management module 208 of the confirmation of the flow frame encapsulated (such as including, etc.) in the data packet. After receiving the notification, the flow management module 214 immediately updates the record of the flow frame, and the record indicates that the frame has been successfully delivered.
[0060] The loss detection module 220 may be used to detect lost packets. Each transmitted packet includes a packet sequence number. The acknowledgment may include the packet sequence number received at the receiving end. Including the packet sequence number in the acknowledgment means that the acknowledgment may include a sequence number range to which the packet sequence number belongs, see Figure 8 In some cases, the loss detection module 220 can determine that the unconfirmed data packets are lost data packets (i.e., not received by the receiving party). The lost data packets can be known based on the confirmed data packets. For example, if data packets with data packet sequence numbers 10, 11, 12, and 13 are sent, but only data packets in the data packet sequence number range 11-13 are confirmed, then the loss detection module 220 can determine that the data packet with sequence number 10 is lost (i.e., not received by the receiving party).
[0061] In one embodiment, the unacknowledged data packets may be retransmitted. When the loss detection module 220 detects the lost data packets, the loss detection module 220 notifies the flow management module 214 of the flow frames encapsulated in the lost data packets, as shown by arrow 230. The buffer described in the data packet management module 218 can be used to determine which flow frames are included in the lost data packets. In one embodiment, the flow management module 214 can notify the flow (or its application) of the lost frames in turn.
[0062] In one embodiment, the stream management module 214 may determine how to further process the lost frame based on the type of stream, custom requirements, other criteria, or a combination of the above. For example, if the lost stream frame belongs to a reliable stream, the stream management module 214 may continue to retransmit the stream frame until the stream frame is successfully received (i.e., confirmed). For another example, if the lost stream frame belongs to an unreliable stream, the stream management module 214 may discard the stream frame. In another embodiment, the stream management module 214 may retransmit the lost stream frame belonging to the unreliable stream based on the stream configuration settings or the UTF configuration settings, and the number of retransmissions is a predetermined number of times. In another embodiment, the API may be used to provide (e.g., by an application developer) instructions (e.g., in the form of a callback function) to the stream management module 214 for taking a customized, application-specific processing approach to the lost frame. In another embodiment, the decision to retransmit is not controlled by the stream management module 214, but the stream management module 214 notifies the stream of the lost frame, and the application using the stream may decide whether to retransmit the lost frame based on the application requirements.
[0063] In some cases, the receiver may have received the data packet, but the acknowledgment itself may be lost. According to the above logic, the loss detection module 220 may incorrectly infer that the data packet is lost. Therefore, the loss of the acknowledgment may lead to inaccurate loss detection, which may in turn lead to unnecessary retransmissions. To prevent inaccurate loss detection, the following and Figure 8 A more effective confirmation mechanism is described in .
[0064] For each data packet sent to the receiver (i.e., the receiver of the streaming content), the data packet management module 218 can maintain a corresponding record, which includes the time when the data packet was sent and the status of the data packet. The status can be a state in transmission (i.e., the data packet has been sent but has not been confirmed by the receiver), a confirmed state (i.e., the data packet has been confirmed to be received by the receiver), a lost state (i.e., the loss detection module 220 believes that the data packet has been lost), a pseudo-lost state (i.e., the data packet is considered lost but has actually been confirmed by the receiver), and other forms of states, which can have one or more states, or a combination of these states. At the connection management module 210 of the receiver, when data packets are received from the underlying network protocol, these data packets can be first processed in the connection management module 210 of the receiver. The confirmation frame generated by the confirmation management module 222 of the receiver can include the received data packet sequence number. The confirmation frame can be sent back to the sender. In one embodiment, the receiver can determine when to send the confirmation frame based on the network conditions. Therefore, the transmission of the confirmation frame can also be dynamically adjusted.
[0065] The connection management module 210 of the receiving party extracts the flow frame from the received data packet and transmits the extracted data packet to the session management module 208 of the receiving party. The flow management module 214 of the receiving party can remove the actual data of the application from the flow frame. If a reliable stream is transmitted, the flow management module 214 of the receiving party can immediately provide the actual data of the application to the application layer. If a reliable stream is transmitted, the flow management module 214 of the receiving party can cache the actual data of the application for reordering.
[0066] The congestion control management module 224 can determine when to transmit a data packet to avoid congestion in the network 206. When the transmission rate is higher than the bandwidth of the network, congestion may occur. In one embodiment, repeated retransmissions of unconfirmed data packets may also cause network congestion. The congestion control management module 224 can collect or calculate network statistics (e.g., statistics at the packet level) to calculate bandwidth estimates and control when to transmit data packets. The statistics at the packet level may include round-trip time, one-way delay and jitter, delay gradient, packet loss rate, packet loss range distribution, available bandwidth estimates, other statistics, or a combination of the above statistics. The congestion control management module 224 can set (e.g., update, determine, calculate, set, select, etc.) congestion control parameters based on statistics. The congestion control parameters may include congestion window, packet sending rate, other parameters, or a combination of the above parameters.
[0067] Thus, in one embodiment, the connection management module 210 can use the congestion control management module 224 (e.g., data received, collected, aggregated, etc. by the congestion control management module 224) to determine when to send a data packet. The loss detection module 220 can be used to determine when a data packet in transit is lost. The loss detection module 220 can provide the loss information to the session management module 208. The session management module 208 can in turn notify the sending stream that data has been lost. The sending stream (or an application using the stream) can then determine (e.g., based on specific application needs) whether to retransmit the data.
[0068] Figure 3 FIG. 3 is a diagram showing an example of a data packet structure, represented by structure 300. In addition, UTF may also adopt other data packet structures. A data packet may be sent by a sender, for example, Figure 2 The connection management module 210 of the sender sends the data packet to the receiver through the network 206. The receiver can receive the data packet, for example, Figure 2 The connection management module 210 of the receiving party receives data from the sending party through the network 206. The data packet cannot contain data that exceeds the maximum data packet size. In other words, the size of the data packet has a maximum limit (e.g., in bytes). It should be noted that a data packet can contain frames from multiple flows.
[0069] Field 302 may be used to indicate the type of data packet. In one embodiment, field 302 may have a first value or a second value. The first value indicates that the data packet is an initial data packet transmitted when a connection is established and before any application data is transmitted. The second value indicates that the data packet is a data packet. Figure 4 and Figure 5 As described above, the first value and the second value may be constant values kInitial and kData.
[0070] Field 304 indicates the sequence number of the data packet. Each transmitted data packet in the flow may have a unique sequence number (as assigned by the sender's data packet management module 218). The data packet sequence numbers are incremented. Field 306 indicates the actual data to be included in the data packet.
[0071] Figure 4 is a diagram showing an example of the structure of an initial data packet, represented by structure 400. UTF may also have other structural forms of initial data packets. Figure 3 The structure of the data packet is further elaborated on, Figure 3 The initial packet type in is kInitial. The initial packet can be used to initialize the connection. As described below, the handshake to establish the connection can be similar to the TCP three-way handshake and can be completed within one round trip time (RTT).
[0072] Field 402 indicates that the value of the packet type is kIinitial. Field 404 can be Figure 3 The same as field 304 in . Figure 3 Field 306 in the packet may be further refined into fields 406-430. Field 406 may indicate the version of the UTF protocol used to generate the data packet. For example, the receiver may use the version information to determine the structure (such as content) and semantics of the data packet. For example, the version may be used for backward compatibility.
[0073] Field 408 includes a set of control bits (s, a, r, o, e, t), each of which can have a Boolean value of 0 or 1. Control bits s, a, and r correspond to the three-way handshake between two TCP stacks. If the underlying protocol used by UTF is the TCP protocol, control bits s, a, and r can be used. Control bits s, a, and r correspond to the values SYN (synchronization sequence number), ACK (acknowledgement), and RST (reset connection) of the three-way TCP handshake, respectively.
[0074] Control bit o indicates whether the "option" field appears later in the actual data of the data packet. UTF can use the "option" field. That is, the "option" field can contain UTF-specific data, but not application-specific data. For example, UTF can use the "option" field to transmit session configuration information. Control bit e indicates whether the "early_data" field appears later in the actual data of the data packet. In one embodiment, the application can use early data to send some application data in the first round trip of the handshake without waiting for the handshake to complete (that is, before the handshake is completed). In addition, "early_data" can also have other uses. Control bit t indicates whether the label appears later in the actual data of the data packet. In one embodiment, UTF can use the label to indicate the configuration of the handshake protocol (for example, whether encryption is used during the handshake). In addition, the label can also have other uses.
[0075] Field 410 contains acked_pkt_no, which is the packet sequence number that the initial packet is to be acknowledged. acked_pkt_no is sent from the receiver to the sender. On the other hand, largest_acked represents the largest acknowledged packet sequence number received by the sender from the receiver. That is, largest_acked is sent from the sender to the receiver.
[0076] If control bit o is set (e.g., has a Boolean value of 1), field 412 indicates the length (e.g., in bytes) of field 414 including options. If control bit e is set, field 416 indicates the length (e.g., in bytes) of field 418 including early data. If control bit t is set, field 420 indicates the number of tags present in the actual data. Any number of tags may be used as long as the packet size remains less than or equal to the size of the maximum packet. Figure 4 4 shows the case of two tags. However, when the t control bit is set, a data packet can contain one or more than two tags. Field 422A indicates the length of field 424A containing data for the first tag. Field 424A contains the data for the first tag. Field 422B indicates the length of field 424B containing data for the second tag. Field 424B contains the data for the second tag.
[0077] Figure 5 The structure of the data packet is shown as a structure 500. In addition, UTF can also have other data packet structures. Figure 3 Based on this, the structure of the data packet is further explained. Figure 3 The type of the data packet is kData. A data packet can encapsulate frames of different types and numbers. Each data packet can contain one or more frames.
[0078] Field 502 indicates the type of data packet, and the value is kData. Field 504 can be Figure 3 . Field 506 contains the maximum packet sequence number confirmed by the receiver. In other words, field 506 contains the value maximum_acked, which is the value of the maximum packet sequence number confirmed received from the receiver. The maximum packet sequence number confirmed received from the receiver can be determined based on the confirmation frame received from the receiver. Each data stream packet received by the receiver contains the maximum (largest_acked) packet sequence number confirmed. The sender can use largest_acked to indicate to the receiver that the sender has received the maximum packet sequence number confirmed from the receiver. Including the largest_acked (confirmed maximum) packet sequence number in the data packet is called an ack-ack mechanism (i.e., confirmation of confirmation). The ack-ack mechanism can be implemented by the confirmation management module 222.
[0079] The ack-ack mechanism can solve the erroneous retransmission caused by feedback link failures, such as packet loss, network jitter, network congestion or other link failures. On the sender, the ack-ack mechanism can embed the maximum confirmed sequence number into each data packet sent from the sender to the receiver. After receiving the maximum confirmed sequence number from the sender, the receiver can carry the distribution information between the current confirmation sequence number and the maximum confirmed sequence number into the confirmation packet sent by the receiver back to the sender. The sender can then update the maximum confirmed sequence number and verify the unconfirmed data packets to avoid erroneous retransmission. The ack-ack mechanism can be dynamically and adaptively effective according to different network conditions. In other words, the ack-ack mechanism can be dynamically enabled and disabled according to the network conditions. If the network condition is good (such as no poor connection condition) and the confirmation (ack) frame is not lost (that is, received by the sender), there is no need to enable the ack-ack mechanism. Enabling the ack-ack mechanism when it is not necessary will result in a waste of bytes. If the sender or receiver detects that the confirmation feedback link connection condition is poor or many confirmation data packets are lost, the ack-ack mechanism can be automatically enabled.
[0080] In one embodiment, the receiver can use the largest_acked packet sequence number to continue to send (and resend) confirmations for packets that have been received but for which the sender has not yet received confirmation. In addition, based on the largest_acked packet sequence number, the receiver no longer confirms packets that are less than or equal to the largest packet sequence number that has been confirmed.
[0081] For example, the sender sends packets with sequence numbers 1 to 100, and the receiver receives packets with sequence numbers 1-70, 75-90, and 93-96. The receiver sends an acknowledgment frame for all received packets to the sender. The acknowledgment frame is as follows: Figure 8 As shown. However, some of the sent confirmation frames may not reach the sender. Assume that the sender has received confirmation frames for data packets with sequence numbers 1-70 and 76-78. Therefore, the sender determines that the maximum confirmed data packet sequence number is 78. If a reliable stream is used, the sender continues to send data packets with sequence numbers 71-75 until they are confirmed. If an unreliable stream is used, the sender will not retransmit data packets with sequence numbers 71-75. In another embodiment, UTF may retransmit data packets with sequence numbers 71-75 of an unreliable stream (or partially reliable stream) a predetermined number of times (e.g., 2, 3, 5 or other times). The receiver will no longer confirm data packets that are less than or equal to the maximum confirmed data packet sequence number. In addition, in this case, field 810 (i.e., largest_observed_packet_number) of the confirmation frame will contain a value of 96, see for details. Figure 8As stated. Figure 8 As shown, the sender can use the missing data packet sequence numbers in the confirmation frame to determine which data packets to retransmit. As mentioned above, in one embodiment, whether to retransmit the frame is determined by the application layer according to application requirements.
[0082] Continuing with the above implementation, the receiver receives the data packet containing largest_acked=79. After receiving the data packet containing largest_acked=79, even if the receiver has already sent the confirmation information, the receiver continues to send the confirmation frames of the data packets 80-90 and 93-96. Figure 8 As described above, the range of the data packet sequence number can be confirmed.
[0083] Field 508 indicates the number of frames included in the data packet. For each frame, a corresponding type field (such as field 510), a corresponding frame length field (such as field 512), and a corresponding frame data field (such as field 514) are all included in the data packet. Ellipsis 518 indicates that fields 510-514 can be repeated according to the number of frames included in the data packet (i.e., the value of field 508). As long as the total size of the data packet remains less than or equal to the maximum data packet size and within the range allowed by the congestion control management module 224, a large number of frames can be included in the data packet.
[0084] The frame can be a stream frame (i.e., kStream type value), an acknowledgment frame (i.e., kAckFrame type value), a close frame (i.e., kClose type value), or a congestion feedback frame (i.e., kFeedback type value). In addition, it can also be other frame types. For details on the structure of a stream frame, see Figure 7 The structure of the confirmation frame is described in Figure 8 The structure of the close frame is described in Fig. 9 The structure of the congestion feedback frame is described in Fig.10 described.
[0085] Figure 6 is an example of a frame structure. UTF may also have other frame structures. The frame may be created by the session management module 208 of the UTF, for example, Figure 2 226. The frame created by the stream management module 214 is transmitted to the connection management module 210, as shown by arrow 226. The packet management module 218 in the connection management module 210 may include the frame, or may combine multiple frames into a packet and transmit it through the network 206, as shown by the send / receive arrow 234. As described above, for the received packet, the packet management module 218 may extract the frames contained in the received packet and transmit at least some of the frames to the session management module 208.
[0086] Field 602 may be used to indicate the type of the frame. The receiver may use the type to determine the semantics of the rest of the frame (such as the rest of the content). In addition, there may be other different frame types. For example, the frame type may include a stream frame type (then field 602 may have a constant value kStream), an acknowledgment frame type (then field 602 may have a constant value kAckFrame), a close frame type (then field 602 may have a constant value kClose), and a congestion feedback frame type (then field 602 may have a constant value kFeedback).
[0087] Examples of fields 604 and 606 are shown in Figure 5 In fields 512 and 514. Figure 7-10 As shown, field 606 may include different actual data and data structures and have different semantics depending on the frame type value (ie, the value of field 602). A frame cannot be larger than the maximum frame size.
[0088] Figure 7 FIG. 7 is a flow frame structure example diagram, represented by structure 700. In addition, UTF may also have other flow frame structures. Figure 6 The frame structure is further explained on this basis. Figure 6 The frame type is kStream. A stream frame is a frame that contains application data, that is, a stream frame can be used to transmit application data to the receiver. A stream frame can optionally contain option data or metadata.
[0089] In the stream frame, the value of the type field (i.e., field 702) is the constant value kStream. Field 704 indicates the length of the stream frame (e.g., in bytes). Field 706 can be used as a unique identifier for the stream that generates the stream frame in the current connection. Field 708 can include two tags (e.g., bits): an option tag (o) and a metadata tag (m). The option tag (o) can be used to indicate whether the stream frame contains option data. That is, when the option tag (o) is set (e.g., with a Boolean value of 1), the fields related to the option (i.e., field 710 and field 712) will be included in the stream frame. The metadata tag (m) can be used to indicate whether the stream frame contains metadata. That is, when the metadata tag (m) is set, the fields related to metadata (i.e., field 714 and field 716) will be included in the stream frame.
[0090] When the option flag (o) is set, field 710 contains the size of field 712, which contains the option data. When the metadata flag (m) is set, field 714 contains the size of field 716, which contains the metadata. Field 718 contains the actual data of the stream. The size of field 718 can be calculated by the receiver as (frame_length-opt_len-meta_len).
[0091] Figure 8 FIG. 8 is a diagram showing an example of a confirmation frame structure, which is represented by structure 800. In addition, UTF may also have other confirmation frame structures. Figure 6 The frame structure is further elaborated on. Figure 6 The frame type is kAckFrame. The acknowledgment frame is created by the receiver (such as the acknowledgment management module 222 of the receiver) and sent to the sender to notify the sender that the receiver has received the data packet. The receiver can confirm the range of data packets. There can be multiple ranges in an acknowledgment frame, and there are gaps in sequence numbers in these ranges (such as data packets that were sent but not received). As described above, the receiver may no longer confirm frames with sequence numbers less than or equal to largest_ack in the acknowledgment frame.
[0092] The confirmation frame according to the present invention enables the sender to obtain feedback from the receiver on whether the data packet is received. The sender can use the obtained information to optimize the transmission of the data packet according to the feedback of the receiver. For example, the sender can analyze the uplink and downlink performance respectively according to the received timestamp and the received range distribution, so as to determine the optimal strategy for data packet transmission. For example, in one embodiment, the congestion control management module 224 of the sender can use the data contained in the confirmation frame to calculate the estimated bandwidth of the receiver and / or the sender; the congestion control management module 224 can also use the timestamp in the received confirmation frame to calculate the jitter (for example, the jitter of the uplink and / or the jitter of the downlink). The uplink refers to the network link used by the sender to transmit the data packet to the receiver. The downlink refers to the network link that receives the confirmation frame, or it can be said that it is the link used by the receiver to send the confirmation frame.
[0093] The uplink performance can be determined by the sender based on uplink statistics, which are maintained by the sender in association with the obtained feedback information. The obtained feedback information enables the analysis of uplink status (such as performance, etc.) to be decoupled from the analysis of downlink status (such as performance, etc.). In this way, the connection management module 210 of the sender can more accurately assess the transmission quality. The difference between the sending rate and the receiving rate can be calculated based on packet-train-based probe techniques.
[0094] By decoupling the state of the uplink from the state of the downlink, it is possible to determine (such as inferring, etc.) whether an unconfirmed data packet is lost due to an uplink performance problem. If so, the data packet will be retransmitted; or it can be determined whether the data packet is not confirmed due to a downlink performance problem that causes the confirmation frame to be lost. In this case, the data packet will not be retransmitted, thereby saving network bandwidth. Otherwise, retransmitting the data packet will waste network bandwidth.
[0095] The feedback information included in the confirmation frame may include the receiving timestamp of each data packet and the range distribution information of the received data packet sequence number. Figure 8 As shown, the timestamp of the largest data packet received and the sequence number of the largest data packet received are recorded. Other information is transmitted as incremental values relative to the time and number of the largest data packet received, as detailed in the implementation of the specific data packet format.
[0096] In the confirmation frame, the value of the type field (i.e., field 802) is the constant value kAckFrame. Field 804 indicates the length of the confirmation frame (e.g., in bytes). Bit 806 indicates whether the confirmation frame contains a data packet reception timestamp (i.e., field 822 and one or more fields 824).
[0097] Field 808 indicates the delay time (ack_delay) for confirming receipt of the packet with the largest packet sequence number. That is, the delay time may be the time difference (e.g., in milliseconds) between the first time the receiver receives the packet with the largest_acked sequence number and the second time the receiver sends an immediate acknowledgment frame.
[0098] Field 810 contains the maximum packet sequence number value received by the receiver (i.e., the value maximum_observed_packet_number). Field 812 contains the timestamp of the packet with the maximum packet sequence number received by the receiver (i.e., largest_observed_recv_time), which may refer to the packet management module 218 of the receiver. For example, the receiver may have received packets with sequence numbers 10-49, 56-72, and 74. Then largest_observed_packet_number is 74.
[0099] Field 814 includes the length of the range information in the confirmation frame. Field 816 includes the number of ranges included in the confirmation frame. Two fields can be used to describe the range, such as fields 818 and 820. Field 818 can represent the gap value of the packet sequence number from the previous range to the current range (i.e., next_range_gap). Field 820 can represent the length of the current range (i.e., range_length). For example, assume that the sender sent packets with sequence numbers 50-100, but the receiver did not receive packets 60-72, 79-83, and 95-100. At this time, largest_observed_packet_number is 94; next_range_gap1 can be 11 (i.e., 94-83); range_length1 can be 5 (i.e., 83-79+1); next_range_gap2 can be 7 (i.e., 79-72); range_length2 can be 13 (i.e., 72-60+1). Depending on the value of the number of frames in range (ie, the value of field 814 ), fields 818 - 820 may be repeated, as indicated by ellipsis 821 .
[0100] Field 822 includes the number of timestamps in the confirmation frame. Field 824 includes the delta (e.g., difference) from the value of largest_observed_packet_number (i.e., the value of field 810) to the current packet sequence number. For example, assuming that the value of largest_observed_packet_number is 100, the confirmation frame only contains arrival timestamps for packets with sequence numbers 80 and 70, and packets 71-79 have been lost. In this case, the value of field 822 is 2; the value of delta_number1 (i.e., the first field 824) can be 20 (i.e., 100-80), and the value of delta_number2 (i.e., the second field 824) can be 30 (i.e., 100-70). Corresponding to field 824, field 826 includes the corresponding delta from the value of largest_observed_recv_time (i.e., the value of field 812) to the reception time of the current packet. Depending on the number of timestamps received for a data packet (ie, the value of field 822 ), fields 824 - 826 may be repeated, as indicated by ellipsis 827 .
[0101] In another embodiment, the value of the first field 824 (e.g., delta_number4) (not shown) may be an increment of the immediately preceding value of the second field 824 (e.g., delta_number3) (not shown), where delta_number1 may be an offset relative to large_observed_packet_number. Continuing with the above embodiment, the value of field 822 may be 2; the value delta_number1 (i.e., the first field 824) may be 20 (i.e., 100-80=20), and the value delta_number2 (i.e., the second field 824) may be 10 (i.e., 80-70=10).
[0102] Acknowledgement frames can also be transmitted in packets that do not contain stream frames. Acknowledgement frames can also be carried in packets that contain other stream frames. Carrying an acknowledgement frame in a packet that contains other stream frames is called piggybacking. If the acknowledgement frame is carried in a packet, and the size of the packet containing the stream frame (of other frames) (such as a packet of type kData) does not exceed the maximum packet size, then the acknowledgement frame can be carried in the packet.
[0103] Acknowledgement piggybacking can effectively reduce the data rate consumption of acknowledgment frames, especially when the network traffic is very large. In addition, as described in the present invention, the specific data packet format of the acknowledgment frame can carry (such as including, etc.) useful information, such as the data packet receiving timestamp, and the acknowledgment piggybacking mechanism and ack-ack mechanism can be dynamically and adaptively adopted according to the real-time network conditions, thereby improving the ack performance in various application scenarios. For more information about piggybacking acknowledgments, please refer to Fig.12 .
[0104] Fig. 9 FIG. 9 is a diagram showing an example of a close frame structure, represented by structure 900. In addition, UTF may also have other close frame structures. Structure 900 is Figure 6 The frame structure is further explained based on Figure 6 The frame type is kClose. The Close frame can be used to close a specific stream or connection. Either party (i.e., sender or receiver) can send a Close frame to the other party. The Close frame indicates that the sender of the Close frame is about to perform a closing action.
[0105] In a close frame, the value of the type field (i.e., field 902) is the constant value kClose. Field 904 indicates the length of the close frame (e.g., in bytes). Field 906 indicates the identifier of the stream to be closed by the sender of the close frame. The close frame can also be used to indicate that a connection (and all streams in the connection) is to be closed. For example, a predefined value (e.g., 0xFFFF or some other value) of field 906 can be set to indicate the end of a connection.
[0106] Field 908 may represent the error code for the shutdown action. That is, field 908 may represent the reason for the shutdown action. Field 910 may be used to provide more details about the reason for the shutdown action.
[0107] In one embodiment, if the sender sends a data packet containing metadata and does not receive an acknowledgment frame within a certain period of time, the sender may send a close frame to the receiver. The sender may send a close frame to the receiver, indicating that a stream that has not yet been initialized has been sent and the sender is closing the stream. The sender may then discard the cached stream frames for the stream that has not yet been initialized.
[0108] Fig.10 FIG. 1 is a diagram showing an example structure of a feedback frame, represented by structure 1000. In addition, UTF may also have other feedback frame structures. Structure 1000 is Figure 6 The frame structure is further elaborated on in the following figure, and its frame type is kFeedback.
[0109] Feedback frames (or congestion feedback frames) can be used to report network statistics to the other party in the session. That is, both the receiver and the sender can periodically send feedback frames to each other. Feedback frames can be created based on network statistics collected by the party sending the feedback frames. The congestion control management module 224 of one party in the connection can collect network statistics and use the statistics to create feedback frames. Feedback frames can be included in data packets and transmitted to the other party in the session with priority. The congestion control management module 224 of the other party can use feedback frames for congestion control.
[0110] In the feedback frame, the value of the type field (i.e., field 1002) is a constant value kFeedback. Field 1004 indicates the length of the feedback frame (e.g., in bytes). Field 1006 may indicate the packet loss rate, i.e., the ratio of the number of lost data packets to the total number of data packets sent. Field 1008 indicates delta_from_begin_ms, which may be the time difference between the current time (e.g., the time when the receiver creates the feedback frame) and the time when the connection is created. That is, field 1008 may indicate the current duration of the session (e.g., length, etc.). Field 1014 indicates the estimated bandwidth (e.g., in kilobits per second), which is an estimate of the amount of data that can be transmitted per unit time (e.g., in seconds) of the uplink of the sender of the feedback frame. Field 1012 may indicate the total number of kilobits sent since the last feedback frame. Field 1014 indicates the number of streams to which the feedback frame containing the statistical data belongs. Field 1016 indicates packet jitter. Packet jitter may indicate a change in the delay of receiving a packet. For example, a sender may send a continuous stream of packets at uniform intervals. However, under several circumstances (such as congestion in the receiving network, improper queuing, configuration errors, other circumstances, or a combination of the above), the receiving party may not receive the packets at uniform intervals; conversely, the delay between packets may also vary rather than remain constant. Field 1016 may represent jitter within two standard deviations from the mean, which accounts for about 95% of the jitter data collected.
[0111] Field 1018 indicates the average queuing time of the data packets. For each flow, a respective field 1020 indicates the flow ID, followed by a field 1022 indicating the number of kilobits (or kilobytes) per second of data transmitted by the flow. For example, sent_kbpsN may indicate the bandwidth of the Nth flow. Ellipsis 1023 indicates that there may be multiple fields 1020-1022.
[0112] In one embodiment, the UTF may report at least a portion of the network statistics to the application layer. For example, the application may modify the behavior of the application based on the statistics. By way of example, but not limitation, in an RTC session, upon learning that the other party's network (i.e., downlink) connection is poor, the application may automatically increase the compression ratio (e.g., by increasing the quantization parameter (QP)), reduce the frame rate, temporarily close the video stream connection, adopt some other strategy, or a combination of the above strategies.
[0113] like Figure 6 As mentioned above, stream frames can contain metadata. Metadata can be used to describe basic characteristics of the stream. Therefore, it is necessary to ensure that metadata can be successfully transmitted even in unreliable streams. For example, video applications can use metadata to carry private video descriptors. Without this descriptor, the video may not be decoded correctly.
[0114] Fig.11 1 is a flow chart of a technique for transmitting metadata, represented by technique 1100. Technique 1100 may be implemented by one or more modules of a general transport architecture, such as Figure 2 The technique 1100 may be implemented by a sender, such as an application of a device that sends stream data to a receiver. For example, the technique 1100 may be implemented by Figure 1 The software program may include a program that can be stored in a memory (such as Figure 1 readable instructions in memory 110 or memory 118), and when the processor (such as Figure 1 When the instructions are executed by the processor 108 or the processor 116 of the corresponding device, the corresponding device can perform the technique 1100. The technique 1100 can be performed in whole or in part by Figure 2 The technique 1100 may be implemented using dedicated hardware or firmware, or may use multiple processors and / or multiple memories.
[0115] Technology 1100 uses a reliable and guaranteed mechanism to transmit metadata in an unreliable stream. Metadata can be a stream identifier or a stream descriptor. For example, in an RTC session, multiple users (more specifically, multiple RTC applications executed on their respective user devices) can use their respective UTFs to connect to a server. The server can send the video stream from each user to all users. When establishing a connection, the server (i.e., the RTC application on the server) can send stream descriptor information (i.e., metadata) that identifies the user (such as user name, user location, etc.) to each user, so that the video stream received by each user can be associated or identified by the user. In addition, other forms of metadata can also be used.
[0116] Technique 1100 can be based on a stream frame (such as Figure 7 How to carry metadata in a stream frame is determined by the storage of metadata in the stream frame described in . In order to reduce the number of frames sent over the network, metadata is included in the stream frame only if there is enough remaining space in the stream frame to carry metadata. Metadata can be aggregated with application data (such as actual data) in the stream frame (i.e., kStream frame). When there is not enough space to carry metadata, metadata can be transmitted in a separate metadata frame. A metadata frame is a kStream frame (i.e., stream frame) that does not contain actual data.
[0117] In addition, the technique 1100 implements a redundant transmission mechanism that can send the same metadata frame multiple times. Redundant transmission can be used to mitigate the situation where metadata may be lost when the receiver's downlink connection is poor, and can increase the chances of the metadata frame being successfully transmitted from the sender (and received by the receiver). The metadata can be retransmitted until the sender receives an acknowledgment frame for the metadata.
[0118] The receiver can cache application data from an unreliable stream before successfully receiving metadata (if any). Caching application data can ensure that application data of an unreliable stream is not blocked from being sent due to loss of the metadata stream. That is, while some traditional technologies may transmit metadata (i.e., metadata frames) before transmitting application data, UTF can send metadata in stream (e.g., application) frames. Transmitting metadata together with application data ensures that the receiver can use the application data immediately when the metadata is received. In addition, sometimes a data packet containing metadata may be lost, while a subsequent data packet containing application data is received by the receiver. In this case, the receiver can cache the application data until the metadata is received. For example, in an RTC session, the application data can be video or audio data, so the receiver can cache the application data received with or before the metadata, so that once the metadata is received, the cached video and audio data can be processed at the receiver.
[0119] The technique 1100 can determine whether to send metadata to the recipient. For example, the stream factory 212 of the recipient receives a request to send metadata and a request to establish (such as create, initialize, etc.) a stream. In one embodiment, metadata can be included in the request to the stream management module 214. Therefore, if there is metadata in the stream, the UTF can process the user data only after receiving the metadata. In other words, if the application receives the application layer data of the stream from the session management module 208 of the UTF, it also receives the application data and metadata (if there is metadata). To reiterate, the protocol mechanism described in the present invention can be used in unreliable streams to ensure reliable transmission of metadata at the stream level and transmit the metadata to the application layer before transmitting the data stream. In addition, by piggybacking the metadata with the application data, the delay that may be caused by traditionally transmitting the metadata first and then the stream data can be avoided.
[0120] At 1102, the technique 1100 transmits metadata to the recipient. In one embodiment, the metadata may be sent in a metadata frame or may be sent in a stream frame together with the application data, as described below in relation to 1106. For example, when the stream management module 214 of the recipient receives a request to send application data to the recipient, the stream management module 214 may decide whether to include the metadata in a stream frame together with the application data.
[0121] At 1104, the stream management module 214 determines whether an acknowledgment has been received regarding the metadata. The acknowledgment may be received in a frame of type kAckFrame, such as Figure 8 The metadata being acknowledged may refer to an acknowledgement that the sender has received the data packet containing the metadata. If an acknowledgement has been received, the technique 1100 terminates the operation. If an acknowledgement has not been received, the technique 1100 proceeds to step 1106.
[0122] At 1106, the stream management module 214 determines whether the metadata can be included in the stream frame used to transmit the application data. As described above, the size of the stream frame cannot be larger than the size of the maximum frame. Therefore, the stream management module 214 can determine whether to include (e.g., store, etc.) the metadata in the stream frame carrying the application data. If the metadata can be included in the stream frame, the technique 1100 proceeds to step 1108 to create a stream frame including the application data and the metadata. The stream frame can be sent to the connection management module 210 and then sent to the recipient, such as Figure 2 As shown by arrow 226. If the metadata cannot be included in the stream frame, the technique 1100 proceeds to step 1110.
[0123] At 1110, technique 1100 determines whether the time increment (i.e., the time difference between the current time and the time of the last metadata transmission) is greater than a maximum threshold time kMeta_Frame_Lost_Interval. In one embodiment, step 1110 can be performed by the stream itself. If the time increment is greater than the maximum threshold time (kMeta_Frame_Lost_Interval), technique 1100 proceeds to step 1112 and then sends a metadata frame containing the metadata. The maximum threshold time can be the maximum time expected to receive an acknowledgment frame from the recipient. In one embodiment, the constant time can be fixed. In another embodiment, the constant time can also be adjusted based on any received feedback frames. If the maximum threshold time has not been reached since the last transmission of metadata, technique 1100 proceeds to step 1114.
[0124] At 1114, the technique 1000 waits for a minimum threshold time (kMeta_Frame_Send_Interval) before retransmitting the metadata in the metadata frame. By waiting for the minimum threshold time, the technique 1100 can avoid retransmitting the metadata too frequently. The minimum threshold time can be constant or can be adjusted by the congestion control management module 224 based on feedback frames received from the receiver. In one embodiment, step 1114 can be performed by the flow itself.
[0125] If the number of retransmissions of the metadata has not been reached (e.g., 4, 5, 10, or some other number of retransmissions), technique 1100 returns to step 1104, which Fig.11 If the number of retransmissions has been reached, then the technique 1100 proceeds to step 1116. If the metadata is not acknowledged after the number of retransmissions, then the technique 1100 may infer that, for whatever reason (e.g., the recipient is offline, the network is down, etc.), an acknowledgement of the metadata will not be received or the metadata will not be received. Therefore, the technique 1100 may cancel the transmission of the metadata and terminate the session. Terminating the session may include transmitting a close frame, such as Fig. 9 described.
[0126] Fig.12 1 is an example diagram of acknowledgment piggybacking, denoted by 1200. Acknowledgment piggybacking 1200 shows a data stream packet 1202, an acknowledgment data packet 1204, and a piggybacked data packet 1206.
[0127] The data stream packet 1202 includes a UDP data packet header, a stream frame header, and actual stream data. In an RTC session, the stream frame header and the actual stream data may be a multimedia frame header and multimedia actual data, respectively. The confirmation data packet 1204 includes a UDP data packet header, a confirmation frame header, and actual confirmation data. For details about the confirmation frame header and the confirmation actual data, see Figure 8 Therefore, if each data packet in the data stream packet 1202 and the confirmation data packet 1204 is transmitted separately, they both contain their own UDP data packet headers.
[0128] In contrast, the piggybacked data packet 1206 contains only one UDP header and includes the multimedia frame header and multimedia actual data of the stream data packet 1202, and the confirmation frame header and confirmation actual data of the confirmation data packet 1204. This can reduce the data rate consumption of the confirmation data packet in the network.
[0129] In one embodiment, the piggybacking mechanism can be adaptively enabled according to the current network conditions. For example, in a two-way video communication session, if the congestion control management module 224 of the sender detects that network congestion occurs in the downlink direction, the piggybacking mechanism can be automatically enabled in that direction (e.g., from the sender to the receiver). In one embodiment, the delayed acknowledgment mechanism (e.g., the TCP delayed acknowledgment mechanism and some other delayed acknowledgment mechanisms) can also be dynamically configured and adjusted.
[0130] Fig.13 The program code example of a general transmission architecture using heterogeneous data streams is represented by 1300. The program code 1300 may be executed in a device (such as Figure 1 Device 102 or device 104, Figure 2 Source code of an application executed on a user device 204, etc.
[0131] The second line indicates Figure 2 API call to the stream factory 212. Line 4 is an API call that configures UTF to communicate using UDP as the underlying network protocol. Lines 10-11 show API calls to the stream factory 212 (such as requests, etc.) to create a reliable stream and set the metadata of the stream, which will be transmitted to the recipient. For illustrative purposes only, the metadata is a simple string (i.e., "this is my metadata" shown in line 11). However, the form of metadata in the present invention is not limited to this, but can be any type that is serialized and transmitted to the recipient. Line 13 shows an API call to the stream management module 214 to transmit application data (i.e., "application payload to send"). Application data can be any type of data. Line 13 also illustrates that the application can set a priority for application data (i.e., "HighestPriority"). The scheduling module 216 can use the priority to determine when application data is sent. Lines 15-17 show a callback that UTF can use to send the received stream data to the application.
[0132] Lines 20-21 show another API call (such as request, etc.) to the stream factory 212 to create an unreliable stream and set metadata for the stream, which will be transmitted to the recipient. For illustrative purposes only, the metadata is a simple string (i.e., "this is my metadata" shown in line 21). Line 23 shows an API call to the stream management module 214 to transmit application data (i.e., "application payload to send"). Application data can be any type of data. Line 23 also illustrates that the application can set a priority for the application data (i.e., "LowestPriority"). Lines 25-27 show a callback that the UTF can use to send the received stream data to the application.
[0133] Based on the foregoing, another aspect of the present invention is a set of operations, including a first operation of a session management module of a transport architecture and a second operation of a connection management module of the transport architecture. The set of operations can be stored in a memory (such as a non-transitory computer-readable storage medium) as instructions executable by a processor.
[0134] The first operation may include: receiving a data frame of a connection session flow from a sending application for transmission to a receiving party; transmitting the data frame to a connection management module; and maintaining respective states of the data frames. The second operation may include: receiving an acknowledgment frame from a receiving party, wherein the acknowledgment frame includes a data packet sequence number range, wherein the data packet sequence number range indicates that the receiving party has received a data packet with a sequence number within the data packet sequence number range; and notifying the session management module of a frame number contained in a data packet with a sequence number within the range. The corresponding state of the data frame may be one of a flying state, an acknowledgment state, a lost state, or a suspected lost state.
[0135] In one embodiment, the first operation further includes an operation of receiving a request to create a stream, wherein the request may include whether the stream is a reliable stream, an unreliable stream, or a partially reliable stream. In one embodiment, the second operation may include an operation of notifying a session management module of a data frame confirmed by the receiver based on the confirmation frame; the first operation may also include updating the status of (at least some of) the data frames respectively according to the notification situation.
[0136] In one embodiment, the second operation may further include receiving a feedback frame from the receiver, wherein the feedback frame includes the bandwidth of the stream and the duration of the connection session. In one embodiment, the first operation may further include: notifying the sender application of the lost frame based on the confirmation frame. For example, if a reliable stream or a partially reliable stream is used, the application may determine to retransmit the lost frame. As described above, if an unreliable stream is used, the application may determine to retransmit the lost frame (at least a predetermined number of times).
[0137] As described above, it should be known to those skilled in the art that all or part of the present invention can be implemented using a general-purpose computer or processor with a computer program, which can execute any corresponding technology, algorithm and / or instruction described in the present invention when running. In addition, a special-purpose computer or processor can also be used, which is equipped with special hardware for running the technology, algorithm or instruction described in the present invention.
[0138] The various aspects of the present invention may be described by functional block components and various processing operations. The processes and sequences in the present invention may be performed individually or in any combination. Functional blocks may be implemented by any number of hardware and / or software components that can run specific functions. For example, the content may use various integrated circuit components, such as memory elements, processing elements, logic elements, lookup tables, etc., which perform various functions under the control of one or more microprocessors or other control devices. Similarly, if software programming or software elements are required to implement the various functions of the content, the present invention may be implemented using any programming or scripting language such as C, C++, Java, assembler, etc., and any combination of any data structure, object, process, routine or other programming element may be used to execute various algorithms. Various functions may be implemented by executing algorithms on one or more processors. In addition, the various aspects of the present invention may use any number of conventional techniques for electronic configuration, signal processing and / or control, data processing, etc. The words "mechanism" and "element" are widely used herein and are not limited to mechanical or physical implementations, but may include software routines combined with processors, etc.
[0139] The implementation or partial implementation of the above invention may take the form of a computer program product, for example, the program product may be used by a computer or may be accessed by a computer-readable medium. A computer-usable or computer-readable medium may be any device as long as the device is capable of specifically containing, storing, transmitting or transmitting a program or data structure that may be used by or in conjunction with any processor. The medium may be an electronic, magnetic, optical, electromagnetic or semiconductor device, etc. Other applicable media may also be used. The above-mentioned computer-usable or computer-readable medium may be referred to as a non-temporary memory or medium, and may include RAM or other volatile memory or storage devices that may change over time. The device memory described in the present invention does not have to be physically equipped in the device, but may be remotely accessed by the device and does not have to be adjacent to other physically equipped memories in the device, unless otherwise specifically stated.
[0140] One or more functions performed in the embodiments of the present invention may be implemented using machine-readable instructions, which are in the form of codes for performing operations on the aforementioned one or more combined hardware. The computing code may be implemented in the form of one or more modules, through which one or more combined functions may be performed as computing tools, and when the method and system of the present invention are run, input and output data are transmitted between each module and one or more other modules.
[0141] In the present invention, the terms "signal" and "data" can be used interchangeably. In addition, the functions of different parts of the computing device do not have to be implemented in the same way. Information, data and signals can be represented using various different technologies and methods. For example, any data, instructions, commands, information, signals, bits, symbols and chips mentioned in the present invention can be represented by one or more combinations of voltage, current, electromagnetic waves, magnetic fields or particles, optical fields or particles, etc.
[0142] The term "implementation" in the present invention is used to represent an example, instance or illustration. Any function or design used as an implementation in the present invention does not necessarily mean that it is superior or better than other functions or designs. On the contrary, the term "implementation" is used to present the concept in a concrete way. In addition, the phrases "a function" or "a function" are used many times throughout the text, and unless otherwise specified, they do not mean the same implementation method or the same function.
[0143] The term "or" as used in the present invention is intended to mean an inclusive "or" rather than an exclusive "or" for two or more elements it connects. That is, "X includes A or B" is intended to mean any natural inclusive arrangement, unless otherwise specified or clearly determined from the context. In other words, if X includes A, X includes B, or X includes A and B, then "X includes A or B" is true in any of the foregoing instances. By analogy, "X includes one of A and B" is intended to be used as an equivalent of "X includes A or B". The term "and / or" used in the present invention is intended to mean "and" or a non-exclusive "or". That is, "X includes A, B and / or C" is intended to mean that X may include any combination of A, B and C, unless otherwise specified or the context clearly indicates otherwise. In other words, if X includes A, X includes B, X includes C, X includes both A and B, X includes both B and C, X includes both A and C, or X includes all of A, B, and C, then each or more of the above situations satisfies the description of “X includes A, B, and / or C.” By analogy, “X includes at least one of A, B, and C” is intended to be used as an equivalent to “X includes A, B, and / or C.”
[0144] The term "include" or "have" and their synonyms herein are intended to include the items listed thereafter and their equivalents as well as additional items. The term "if" herein may be interpreted as "when", "when..." or "assuming" depending on the context.
[0145] In the content of the present invention (especially the claims), "one", "an" and "said" and similar demonstrative pronouns should be understood to include both singular and plural forms. In addition, unless otherwise stated, the description of the numerical range in the present invention is only a simple description method, which is intended to represent each individual numerical value included in the range, and each individual value is included in the specification, which is equivalent to being listed separately in the present invention. Finally, the steps of all methods described in the present invention can be performed in any suitable order, unless otherwise stated herein or clearly contradictory to the context. The use of examples or exemplary language (such as "such as") provided by the present invention is intended to better illustrate the present invention, rather than to limit the scope of the present invention, unless otherwise stated.
[0146] The specification uses various titles and subheadings. These are included to enhance readability and simplify the process of finding and citing materials. These titles and subheadings are not intended to, and will not, affect the interpretation of the claims or limit the scope of the claims in any way. The specific embodiments shown and described herein are illustrative examples of the present invention and are not intended to limit the scope of the present invention in any way.
[0147] All references (including publications, patent applications, and patents, etc.) cited herein are hereby incorporated by reference to the same extent as if each reference were individually and specifically indicated to be incorporated by reference and set forth in its entirety.
[0148] Although the present invention has been described herein in conjunction with certain examples and implementations, it should be understood that the present invention is not limited to the disclosed implementations. On the contrary, the present disclosure is intended to cover various modifications and equivalent arrangements within the scope of the claims, which should be given the broadest interpretation to cover all such modifications and equivalent arrangements permitted by law.
Claims
1. A device for communicating between a sending application and a receiving application on a receiving device, include: Processor, which is configured as: Establishing a stream for transmitting data between a sending application and a receiving application; receiving a first request from a sending application to transmit metadata to a receiving application; receiving a second request from the sending application to transmit the application data to the receiving application; After determining that the frame including the application data and the metadata is less than or equal to the capacity of the maximum frame, constructing the frame including the application data and the metadata; and sending the frame to a receiving device in the form of a data packet; Among them, each data packet sent by the processor to the receiving device has a unique serial number that increases in sequence, and under preset network conditions, includes the value of the confirmed maximum data packet serial number received by the processor from the receiving device. At this time, the receiving device no longer confirms data packets that are less than or equal to the confirmed maximum data packet serial number.
2. The apparatus of claim 1, wherein the processor is further configured to: After determining that the frame including the application data and metadata is larger than the capacity of the maximum frame, constructing the frame including the metadata but not including the application data.
3. The apparatus of claim 1, wherein the frame is transmitted via an unreliable transport protocol.
4. The apparatus of claim 1, wherein the processor is further configured to: When no confirmation of the metadata is received from the receiving device, sending a metadata frame including the metadata to the receiving device, in, The metadata frame is sent after waiting a threshold time from sending the frame.
5. The apparatus of claim 1, wherein the processor is further configured to: When confirmation of the metadata is not received from the receiving device and the time difference between the current time and the sending time of the frame does not exceed the threshold time, a metadata frame including the metadata is sent to the receiving device after the threshold time of sending the frame.
6. The apparatus of claim 1, wherein the processor is further configured to: When the number of retransmissions reaches a threshold value: when no confirmation of the metadata is received from the receiving device, the metadata is retransmitted to the receiving device.
7. The apparatus of claim 6, wherein the processor is further configured to: When the acknowledgment is not received after a threshold number of retransmissions, the stream is closed.
8. A method for communicating between a sending application and a receiving application of a receiving device, include: Establishing a stream for transmitting data between a sending application and a receiving application; receiving a first request from a sending application to transmit metadata to a receiving application; receiving a second request from the sending application to transmit the application data to the receiving application; After determining that the frame including the application data and the metadata is less than or equal to the capacity of the maximum frame, constructing the frame including the application data and the metadata; and sending the frame to a receiving device in the form of a data packet; Among them, each data packet sent to the receiving device has a unique serial number that increases in sequence, and contains the value of the confirmed maximum data packet serial number received from the receiving device under preset network conditions. At this time, the receiving device no longer confirms data packets that are less than or equal to the confirmed maximum data packet serial number.
9. The method of claim 8, further comprising: include: After determining that the frame including the application data and metadata is larger than the capacity of the maximum frame, construct the frame including the metadata but not including the application data.
10. The method of claim 8, wherein the frame is transmitted via an unreliable transport protocol.
11. The method of claim 8, further comprising: include: When confirmation of the metadata is not received from the receiving device, a metadata frame including the metadata is immediately sent to the receiving device, wherein the metadata frame is sent after waiting for a threshold time from sending the frame.
12. The method of claim 8, further comprising: include: When confirmation of the metadata is not received from the receiving device and the time difference between the current time and the sending time of the frame does not exceed the threshold time, a metadata frame including the metadata is sent to the receiving device after the threshold time of sending the frame.
13. The method of claim 8, further comprising: include: When the number of retransmissions reaches a threshold value: when no confirmation of the metadata is received from the receiving device, the metadata is retransmitted to the receiving device.
14. The method of claim 13, further comprising: include: When the acknowledgment is not received after a threshold number of retransmissions, the stream is closed.
15. A non-transitory computer-readable storage medium comprising executable instructions, wherein when a processor executes the executable instructions, the operation to be performed is to build a transmission architecture for communication between a sending application and a receiving application of a receiving device, wherein the operation include: Establishing a stream for transmitting data between a sending application and a receiving application; receiving a first request from a sending application to transmit metadata to a receiving application; receiving a second request from the sending application to transmit the application data to the receiving application; After determining that the frame including the application data and the metadata is less than or equal to the capacity of the maximum frame, constructing the frame including the application data and the metadata; and sending the frame to a receiving device in the form of a data packet; Among them, each data packet sent to the receiving device has a unique serial number that increases in sequence, and contains the value of the confirmed maximum data packet serial number received from the receiving device under preset network conditions. At this time, the receiving device no longer confirms data packets that are less than or equal to the confirmed maximum data packet serial number.
16. The non-transitory computer readable storage medium of claim 15, further comprising: include: After determining that the frame including the application data and metadata is larger than the capacity of the maximum frame, constructing the frame including the metadata but not including the application data.
17. The non-transitory computer-readable storage medium of claim 15, wherein the frame is transmitted via an unreliable transport protocol.
18. The non-transitory computer readable storage medium of claim 15, further comprising: include: When confirmation of the metadata is not received from the receiving device, a metadata frame including the metadata is immediately sent to the receiving device, wherein the metadata frame is sent after waiting for a threshold time from sending the frame.
19. The non-transitory computer readable storage medium of claim 15, further comprising: include: When confirmation of the metadata is not received from the receiving device and the time difference between the current time and the sending time of the frame does not exceed the threshold time, a metadata frame including the metadata is sent to the receiving device after the threshold time of sending the frame.
20. The non-transitory computer readable storage medium of claim 15, further comprising: include: When the number of retransmissions reaches a threshold value: when no confirmation of the metadata is received from the receiving device, retransmitting the metadata to the receiving device; and When the acknowledgment is not received after a threshold number of retransmissions, the stream is closed.
Citation Information
Patent Citations
Universal transport framework for heterogeneous data streams
US12206737B2
Universal Transport Framework For Heterogeneous Data Streams
US20220368765A1
Service data transmission method and system and storage medium
CN110312147A
Receiving streams over asynchronous networks
US20030133462A1
Method and apparatus for providing mobile and other intermittent connectivity in a computing environment
US20050223114A1
