Data transmission method, device, system, equipment, medium and program product
By generating detection data packets to identify the available bandwidth of the media stream link, the problem of delay and bandwidth requirements in media stream data transmission is solved, data transmission is optimized, lag is reduced, and user experience is improved.
Patent Information
- Application Number
- CN202310541122.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-12
- Publication Date
- 2024-11-12
AI Technical Summary
Media stream type data transmission services cannot be cached in advance due to the real-time dynamic generation of transmission content, resulting in extreme requirements for delay and bandwidth, and there are problems of data oversend and video stuttering.
By generating a detection data packet associated with the media stream, it is sent to the receiving device to identify the available bandwidth of the link, adjust the congestion window and the transmission rate, and optimize data transmission.
Accurately identify the available bandwidth of the link, reduce lag, improve data transmission efficiency, and improve user experience.
Smart Images

Figure CN118945089A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure generally relates to the field of computer technology, and more particularly to a data transmission method, a data transmission apparatus, a data transmission system, a computing device, a computer-readable storage medium, and a computer program product. Background Art
[0002] In recent years, media streaming data transmission services represented by live broadcast, video conferencing, and short videos have developed rapidly, with a rapid growth trend. Traditional data transmission services include large file data transmission services such as cloud databases and data synchronization, and static media data transmission services such as video on demand and high-latency live broadcast. Compared with traditional data transmission services, media streaming data transmission services have extreme requirements for latency and bandwidth because the transmission content is dynamically generated in real time and cannot be cached in advance. Summary of the invention
[0003] According to some embodiments of the present disclosure, a data transmission method, a data transmission apparatus, a data transmission system, a computing device, a computer-readable storage medium, and a computer program product are provided.
[0004] In a first aspect of the present disclosure, a data transmission method is provided. The data transmission method includes: generating a detection data packet associated with a first device based on traffic characteristics associated with a media stream; sending the detection data packet to a second device; and determining the link available bandwidth for data transmission of the first device based on feedback information received from the second device for the detection data packet. According to an embodiment of the present disclosure, by supplementing the data volume during the trough period of the media stream to perceive the link bandwidth change, the erroneous congestion window or erroneous link bandwidth information can be accurately adjusted, thereby solving the problems of data over-transmission and video freeze. Thus, for example, the transmitting device can accurately evaluate the link available bandwidth and quickly identify bandwidth changes, thereby optimizing data transmission.
[0005] In some embodiments, generating a detection data packet associated with the first device includes: determining additional data for a data packet of the media stream based on the traffic characteristics; and encapsulating the data packet and the additional data for the media stream to generate the detection data packet. Thus, for example, the sending end device can adapt to the media stream by means of additional data, thereby accurately and quickly identifying the actual situation of the link bandwidth change without modifying the transmission architecture.
[0006] In some embodiments, the traffic characteristics include peak and trough information, and determining the additional data for the data packet of the media stream includes: determining whether the media stream meets the link detection condition based on the peak and trough information; and determining to detect the link available bandwidth based on the media stream meeting the link detection condition. Thus, for example, the transmitting end device can identify the media stream with peak and trough characteristics, thereby enabling the link available bandwidth evaluation more accurately and reducing the link bandwidth loss.
[0007] In some embodiments, determining the additional data for the data packet of the media stream includes: determining whether the traffic characteristics include the media stream required bandwidth information, wherein the media stream required bandwidth information indicates whether the media stream has the required bandwidth value; if it is determined that the traffic characteristics include the media stream required bandwidth, determining the additional data based on the media stream required bandwidth information; and if it is determined that the traffic characteristics do not include the media stream required bandwidth, determining the additional data based on the peak and trough information. Thus, for example, the sending end device can flexibly adopt the traffic supplement method or the bandwidth up-probing method to detect the link available bandwidth, thereby enhancing the accuracy and practicality of the link available bandwidth evaluation.
[0008] In some embodiments, generating a detection data packet for the media stream includes: determining whether the current frame is used to send a service data packet based on the frame type information of the current frame of the media stream, wherein the frame type information is included in the traffic characteristics; determining to send the service data packet based on determining that the current frame is used to send a service data packet, wherein the service data packet comes from a queue to be sent or a retransmission queue in a data buffer of the first device; and determining to generate a detection data packet for the media stream based on determining that the current frame is not used to send a service data packet. Thus, by giving priority to sending the current frame service data, all service data of the sender can be completely sent before sending the supplementary or upward probing traffic data, so that the traffic supplement method or bandwidth upward probing method as an optimization means does not affect the normal sending of service data.
[0009] In some embodiments, determining the additional data based on the peak and trough information includes: if it is determined to generate a detection data packet for the media stream, then determining whether the latest transmitted frame is an intra-frame coded frame based on the frame type information of the latest transmitted frame in the media stream, wherein the frame type information is included in the traffic feature; and determining whether the current connection with respect to the media stream is in an application-restricted state based on determining that the latest transmitted frame is an intra-frame coded frame. If the current connection is not in an application-restricted state, it indicates that the business data is backlogged and traffic cannot be supplemented, and therefore traffic supplementation is required when the current connection is in an application-restricted state. As a result, traffic supplementation can be performed only when the connection is in an application-restricted state without affecting the normal transmission of business data.
[0010] In some embodiments, determining the additional data based on the peak and trough information includes: determining whether the amount of data sent about the media stream is less than a first threshold based on the current connection of the media stream being in an application-restricted state, wherein the first threshold indicates the maximum amount of additional data required for the media stream; based on determining that the amount of data sent about the media stream is less than the first threshold, obtaining data less than the first threshold from the waiting-for-response queue in the data buffer as the additional data. In some embodiments, the first threshold is determined based on the ratio of the peak to the trough of the media stream indicated by the peak and trough information and the data amount of the most recently sent frame. Thus, by considering the amount of data sent by the media stream, excessive traffic replenishment is limited based on the peak and trough characteristics of the media stream, thereby reducing traffic loss while accurately evaluating the bandwidth status.
[0011] In some embodiments, determining the additional data based on the media stream required bandwidth information includes: in response to determining to generate a detection data packet for the media stream, determining a detection target bandwidth for detecting the link available bandwidth for data transmission of the first device based on the bandwidth value required for the media stream indicated by the media stream required bandwidth information; based on the detection target bandwidth, determining the additional data by at least one of the following: obtaining data for satisfying the detection target bandwidth from the waiting-for-response queue in the data buffer as the additional data, or using padding bytes as the additional data to fill in to satisfy the detection target bandwidth. Therefore, compared with the embodiment of data transmission based on the traffic supplementation method, the embodiment of data transmission based on the bandwidth up-probing method can purposefully detect by obtaining the actual encoding bit rate or the maximum encoding bit rate, and can achieve accurate detection of the link available bandwidth while further reducing the consumption of bandwidth traffic caused by detecting the link available bandwidth.
[0012] In some embodiments, sending the detection data packet to the second device includes at least one of the following: in response to the media stream required bandwidth information indicating an updated media stream required bandwidth, sending the detection data packet to the second device during the transmission of non-intra-frame coded frames; or in response to the absence of a service data packet to be sent in the first device and the time from the last detection data packet to be sent exceeds a predetermined time, sending the detection data packet to the second device. In this way, both active detection and response demand detection can be considered to further accurately determine the timing of bandwidth increase.
[0013] In some embodiments, sending the detection data packet to the second device includes: transmitting the detection data packet generated by the additional data at a transmission rate corresponding to the detection target bandwidth during a period when the service data is not transmitted; and transmitting the service data packet at a transmission rate before detecting the link available bandwidth during a period when the service data packet is transmitted. In this way, the risk of link congestion and packet loss caused by insufficient link available bandwidth when data is transmitted using bandwidth up-probing can be avoided, thereby further optimizing the data transmission logic.
[0014] In some embodiments, determining the link available bandwidth for data transmission of the first device includes: determining the reception time of the second device receiving the detection data packet based on the feedback information for the detection data packet received from the second device; determining the data volume of the data packet transmitted within a predetermined time interval based on the reception time; determining the detected link available bandwidth based on the predetermined time interval and the data volume of the data packet transmitted within the time interval. In some embodiments, determining the link available bandwidth for data transmission of the first device also includes: marking the detection data packet, wherein the marked detection data packet includes a first detection data packet and a second detection data packet; determining the reception time difference between the reception time of the first detection data packet and the reception time of the second detection data packet based on the feedback information for the first detection data packet and the feedback information for the second detection data packet; determining the number of detection data packets transmitted within the reception time difference based on the determined reception time difference; determining the detected link available bandwidth based on the reception time difference and the number of detection data packets. Thus, the real link available bandwidth can be accurately evaluated, thereby quickly completing the bandwidth detection process for data transmission adjustment.
[0015] In some embodiments, determining the link available bandwidth for data transmission of the first device further includes: if the detected link available bandwidth is greater than or equal to the bandwidth before detecting the link available bandwidth, transmitting the service data packet according to the congestion window and transmission rate corresponding to the detected link available bandwidth; and if the detected link available bandwidth is less than the bandwidth before detecting the link available bandwidth, discarding non-intra-frame coded frames, and transmitting the service data packet according to the congestion window and transmission rate corresponding to the detected link available bandwidth. Thus, the service data transmission bit rate can be accurately reduced based on the link available bandwidth, thereby effectively reducing the video playback freeze rate.
[0016] In some embodiments, determining the link available bandwidth for data transmission of the first device further includes: determining the congestion window and the sending rate based on the detected link available bandwidth and the detection target bandwidth; and transmitting the service data packet according to the determined congestion window and the determined sending rate. In some embodiments, determining the link available bandwidth for data transmission of the first device further includes: if the detected link available bandwidth is greater than or equal to the detection target bandwidth, transmitting the service data packet according to the congestion window and the sending rate corresponding to the detection target bandwidth; and if the detected link available bandwidth is less than the detection target bandwidth, transmitting the service data packet according to the congestion window and the sending rate before detecting the link available bandwidth. In this way, the transmission rate can be made as close as possible to the detected real physical bandwidth, so that the available bandwidth of the link can be accurately evaluated during the media stream trough period, the bandwidth change can be quickly identified, and the data transmission can be optimized. At the same time, in the face of media stream bit rate bursts, the bandwidth can be probed in advance, and the bandwidth detection can be quickly completed to converge to the link available bandwidth, avoiding the backlog of data at the sending end and causing jams.
[0017] In some embodiments, the data transmission method further includes: updating the encoding bit rate based on the detected link available bandwidth; and encoding the media stream based on the updated encoding bit rate. Thus, the determined link available bandwidth is provided to the media stream encoding side, enabling the transmission side to be linked with the encoding side to optimize the control of the transmission bit rate of the media stream push.
[0018] In the second aspect of the present disclosure, a data transmission method is provided. The data transmission method comprises: receiving a media stream including a detection data packet associated with the first device from a first device, wherein the detection data packet is generated based on the traffic characteristics associated with the media stream; generating feedback information for the detection data packet of the media stream based on the received media stream, wherein the feedback information is used to determine the link available bandwidth for data transmission of the first device; and sending the feedback information from the second device to the first device. According to an embodiment of the present disclosure, the receiving device can normally feedback the detection data packet to the sending device according to the transmission framework, so that the data transmission scheme of the present disclosure can be simply implemented without changing the overall transmission framework. As a result, the sending device can use the feedback information to accurately evaluate the available bandwidth of the link, avoid the obvious jamming problem caused by inaccurate bandwidth evaluation in application-restricted scenarios and slow convergence speed in scenarios where the bandwidth demand of the media stream changes suddenly, so as to quickly identify bandwidth changes and effectively optimize data transmission.
[0019] In some embodiments, the detection data packet is generated by encapsulating the data packet of the media stream and additional data, wherein the additional data is determined based on the traffic characteristics, wherein the traffic characteristics include peak and trough information, and the determination of the additional data includes: determining whether the traffic characteristics include media stream required bandwidth information, wherein the media stream required bandwidth information indicates whether the media stream has a required bandwidth value; if it is determined that the traffic characteristics include the media stream required bandwidth, determining the additional data based on the media stream required bandwidth information; and if it is determined that the traffic characteristics do not include the media stream required bandwidth, determining the additional data based on the peak and trough information.
[0020] In some embodiments, receiving a media stream including a probe data packet associated with the first device from a first device includes at least one of the following: receiving the probe data packet, wherein the probe data packet is sent by the first device in response to the media stream required bandwidth information indicating an updated media stream required bandwidth during the transmission of non-intra-frame coded frames; or receiving the probe data packet, wherein the probe data packet is sent by the first device in response to the absence of a business data packet to be sent in the first device and the time from the last time the probe data packet was sent exceeds a predetermined time.
[0021] In some embodiments, the feedback information includes the reception time when the second device receives the detection data packet, wherein the feedback information is used by the first device to determine the data volume of the data packets transmitted within a predetermined time interval, so that the first device determines the link available bandwidth for data transmission of the first device based on the predetermined time interval and the data volume of the data packets transmitted within the time interval.
[0022] In a third aspect of the present disclosure, a data transmission device is provided. The data transmission device includes: a data packet generation module configured to generate a detection data packet associated with a first device based on traffic characteristics associated with a media stream; a data packet transmission module configured to send the detection data packet to a second device; and a bandwidth determination module configured to determine a link available bandwidth for data transmission of the first device based on feedback information for the detection data packet received from the second device.
[0023] In a fourth aspect of the present disclosure, a data transmission device is provided. The data transmission device comprises: a data packet receiving module configured to receive a media stream including a detection data packet associated with the first device from a first device, wherein the detection data packet is generated based on a traffic feature associated with the media stream; a feedback information generating module configured to generate feedback information for the detection data packet of the media stream based on the received media stream, wherein the feedback information is used to determine a link available bandwidth for data transmission of the first device; and a feedback information sending module configured to send the feedback information from a second device to the first device.
[0024] In a fifth aspect of the present disclosure, a data transmission system is provided. The data transmission system includes a first device and a second device, wherein the first device generates a detection data packet associated with the first device based on traffic characteristics associated with a media stream, and transmits the media stream including the detection data packet to the second device; the second device receives the media stream from the first device; the second device generates feedback information for the detection data packet of the media stream based on the received media stream, and sends the feedback information from the second device to the first device; and the first device receives the feedback information from the second device, and determines the link available bandwidth for data transmission of the first device based on the received feedback information for the detection data packet.
[0025] In a sixth aspect of the present disclosure, a computing device is provided, comprising: a processor; and a memory, wherein the memory stores instructions, and when the instructions are executed by the processor, the computing device executes the method provided according to the first aspect of the present disclosure.
[0026] In a seventh aspect of the present disclosure, a computing device is provided, comprising: a processor; and a memory, wherein the memory stores instructions, and when the instructions are executed by the processor, the computing device executes the method provided according to the second aspect of the present disclosure.
[0027] In an eighth aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed by a computing device, the computing device executes the method provided according to the first aspect of the present disclosure.
[0028] In a ninth aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed by a computing device, the computing device executes the method provided according to the second aspect of the present disclosure.
[0029] In a tenth aspect of the present disclosure, a computer program product is provided, which includes instructions, and when the instructions are executed by a computing device, the computing device executes the method provided according to the first aspect of the present disclosure.
[0030] In an eleventh aspect of the present disclosure, a computer program product is provided, which includes instructions, and when the instructions are executed by a computing device, the computing device executes the method provided according to the second aspect of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 An exemplary diagram of a data transmission system according to an embodiment of the present disclosure is shown;
[0032] Figure 2 An exemplary diagram of the component structure of a data transmission device according to an embodiment of the present disclosure is shown;
[0033] Figure 3 An example flow chart of a data transmission method according to an embodiment of the present disclosure is shown;
[0034] Figure 4 Another exemplary flow chart of a data transmission method according to an embodiment of the present disclosure is shown;
[0035] Figure 5A An example flow chart of bandwidth detection according to an embodiment of the present disclosure is shown;
[0036] Figure 5B An example diagram of a component structure of data transmission based on a flow supplementation method according to an embodiment of the present disclosure is shown;
[0037] Figure 5C An example flow chart of data transmission based on a flow supplement method according to an embodiment of the present disclosure is shown;
[0038] Figure 5D A schematic diagram of data transmission based on a flow supplementation method according to an embodiment of the present disclosure is shown;
[0039] Fig. 6AAnother example flow chart of bandwidth detection according to an embodiment of the present disclosure is shown;
[0040] Figure 6B An example diagram of a component structure of data transmission based on bandwidth up-probing according to an embodiment of the present disclosure is shown;
[0041] Figure 6C An example flow chart of data transmission based on bandwidth up-probing method according to an embodiment of the present disclosure is shown;
[0042] Fig.6D A schematic diagram of data transmission based on bandwidth up-probing according to an embodiment of the present disclosure is shown;
[0043] Figure 7 Another exemplary flow chart of a data transmission method according to an embodiment of the present disclosure is shown;
[0044] Fig. 8A A schematic block diagram of a data transmission device according to some embodiments of the present disclosure is shown;
[0045] Figure 8B A schematic block diagram of another data transmission device according to some embodiments of the present disclosure is shown;
[0046] Fig. 9A A schematic block diagram showing an example device for implementing an example implementation of the present disclosure; and
[0047] Fig. 9B A schematic block diagram of another example device for implementing an example implementation of the present disclosure is shown. DETAILED DESCRIPTION
[0048] The preferred implementation of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although example embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.
[0049] As used herein, the term "including" and its variations mean open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "based at least in part on". The terms "embodiment" and "some embodiments" both mean "at least some embodiments". Other explicit and implicit definitions may also be included below.
[0050] As described above, compared with traditional data transmission services, media stream type data transmission services cannot be cached in advance because the transmission content is dynamically generated in real time, so they have extreme requirements for latency and bandwidth. As a typical transmission framework of a data transmission service, it includes a sender, a receiver, and a router for routing data transmission between the sender and the receiver. The sender sends the service data to the receiver via the router, and after receiving the service data, the receiver feeds back the receiving status of the receiver to the sender. If the sender does not detect a packet loss signal based on the feedback receiving status, the data transmission amount is increased; conversely, if the sender detects a packet loss signal based on the feedback receiving status, the data transmission amount is halved. The sender continuously repeats the process of increasing and decreasing the data transmission amount. However, the inventor of the present application noted that the above-mentioned data transmission amount processing method is useful for large file transmission scenarios with continuous and stable data transmission, but for media stream transmission scenarios, in the scenario where the link bandwidth changes and the link bandwidth becomes smaller, it will cause congestion packet loss caused by excessive data transmission, so that frequent data over-transmission causes jamming.
[0051] The inventor of the present application further noticed the process of reducing the amount of data transmission by speculating the application-restricted scenario. For example, when the sender speculates that the application layer is in the application-restricted period, even if no packet loss signal is detected, the current transmission rate remains unchanged. In this way, once the link bandwidth changes, the amount of data sent to the link can be reduced to avoid packet loss. However, the inventor of the present application further studied and found that the above-mentioned data transmission method of maintaining the transmission rate still cannot effectively solve the jamming problem. Specifically, during the trough period of media service flow transmission, the sender cannot accurately detect the actual bandwidth of the current network. Therefore, if the network link bandwidth suddenly decreases, the sender still uses the original high transmission rate for data transmission, which will still cause over-transmission packet loss and cause obvious jamming. On the other hand, if the data provided by the application layer is limited, the sender lacks data to detect the actual bandwidth of the current network. Therefore, if the bandwidth required by the media flow suddenly increases, the amount of data to be transmitted provided by the application layer suddenly increases, and the transmission rate can only be slowly and gradually increased, resulting in a backlog of data at the sender in a short period of time, which will also cause obvious jamming. Therefore, in media stream-type data transmission services, due to their latency and bandwidth sensitivity, there are frequent and obvious freezes, resulting in poor user experience.
[0052] To this end, the present disclosure provides a data transmission scheme, which detects the link available bandwidth of the current network based on the delay and bandwidth-sensitive characteristics of media stream type data transmission services, so as to be applicable to the link bandwidth change scenario when application data is insufficient and the media stream bandwidth demand mutation scenario. Among them, the sending end device generates a detection data packet based on the traffic characteristics associated with the media stream and sends it to the receiving end device, and the receiving end device generates feedback information for the detection data packet and sends it to the sending end device. The sending end device determines the link available bandwidth for data transmission based on the received feedback information for the detection data packet. As a result, it can be applicable to the link bandwidth change scenario and the media stream bandwidth demand mutation scenario when application data is insufficient in the link between the sending end device and the receiving end device, accurately identify the link available bandwidth for subsequent data transmission, significantly reduce the occurrence of jamming, improve the stability of media stream data transmission, effectively improve transmission efficiency and significantly improve user experience.
[0053] Figure 1An example diagram of a data transmission system according to an embodiment of the present disclosure is shown, which specifically shows a data transmission system 10 for detecting available link bandwidth. In an embodiment of the present disclosure, the data transmission system 10 may include a transmitting device 110 and a receiving device 120, wherein the transmitting device 110 and the receiving device 120 are connected via a communication network 130. In an embodiment of the present disclosure, the transmitting device 110 may include an application module 111 and a transmission module 112. In some embodiments, the application module 111 of the transmitting device 110 includes one or more applications at the application layer, and can provide the media stream to be transmitted by the application and the traffic characteristics associated with the media stream to the transmission module 112. In some embodiments, the transmission module 112 of the transmitting device 110 sends the media stream to the receiving device 120 via the communication network 130 based on the traffic characteristics associated with the media stream. In some embodiments, the transmitting device 110 generates a detection data packet associated with the transmitting device 110 based on the traffic characteristics associated with the media stream, and sends the detection data packet to the receiving device 120. In an embodiment of the present disclosure, the receiving device 120 may include an application module 121 and a transmission module 122. In some embodiments, the transmission module 122 of the receiving device 120 receives a media stream from the transmitting device 110, determines feedback information for the media stream, and feeds it back to the transmitting device 110. In some embodiments, the transmission module 122 of the receiving device 120 receives a media stream including a detection data packet associated with the transmitting device 110 from the transmitting device 110 via the communication network 130, and generates feedback information of the receiving device 120 for the detection data packet associated with the transmitting device 110 based on the received media stream. In some embodiments, the transmission module 122 of the receiving device 120 may also provide the data of the received media stream to the application module 121, and the application module 121 performs corresponding data processing based on the data of the provided media stream. In an embodiment of the present disclosure, the transmission module 112 of the transmitting device 110 receives feedback information from the receiving device 120, and determines the link available bandwidth for data transmission of the transmitting device 110 based on the received feedback information for the detection data packet. In some embodiments, the transmitting device 110 may be, for example, a client device that needs to use a media streaming service, such as a mobile terminal, a computer, a tablet computer, etc. In some embodiments, the receiving device 120 may be, for example, a server device that provides a media streaming service, such as a server, a network node, a mobile terminal, a computer, a tablet computer, etc. It should be understood that in the embodiments of the present disclosure, there are no particular restrictions on the types and functions of the transmitting device 110 and the receiving device 120, and the two may be the same or different, independent of each other, or interchangeable.In the embodiments of the present disclosure, the communication network 130 between the transmitting device 110 and the receiving device 120 may include, for example, a local area network (LAN), a wide area network (WAN), the Internet, a virtual LAN (VLAN), an enterprise LAN, a layer 3 virtual private network (VPN), an intranet, or any combination thereof. It should be understood that the various structures in the data transmission system 10 may communicate with each other directly via the communication network, or may communicate with each other indirectly via any structure such as a router, a switch, etc. connected to the communication network 130, and of course may not communicate with each other in some cases. It should be understood that the specific configuration of the data transmission system 10 may include more or fewer elements, but is not limited thereto. Figure 1 Contents shown
[0054] Figure 2 The diagram shows an example of a component structure of a data transmission device according to an embodiment of the present disclosure, which specifically shows a data transmission device as a Figure 1 The example layer structure of the sending device 110 is shown. In an embodiment of the present disclosure, the sending device 110 includes an application layer 2100, a transport layer 2200, and a network layer 2300, wherein the application layer 2100 corresponds to the application module 111 and the transport layer 2200 and the network layer 2300 correspond to the transport module 112. In some embodiments, the application layer 2100, the transport layer 2200, and the network layer 2300 of the sending device 110 can be implemented using the protocol and architecture of the Open System Interconnection Model (OSI), for example, the Transmission Control Protocol (TCP), the Quick UDP Internet Connections (QUIC) protocol, etc. can be used for specific implementation. In an embodiment of the present disclosure, as Figure 2As shown, the application layer 2100 provides the traffic characteristics associated with the media stream to the transport layer 2200, wherein the transport layer 2200 includes a traffic characteristics management unit 2201, an additional data determination unit 2202, a data packet generation unit 2203, a data cache management unit 2204, a congestion control unit 2205, a data packet transmission unit 2206, and a feedback information processing unit 2207. In some embodiments, the traffic characteristics management unit 2201 of the transport layer 2200 receives the traffic characteristics associated with the media stream from the application layer 2100 and determines the generation of the detection data packet. In some embodiments, the additional data determination unit 2202 determines the additional data for generating the detection data packet from the data cache management unit 2204 based on the determination of the traffic characteristics management unit 2201 on the generation of the detection data packet. In some embodiments, the data packet generation unit 2203 encapsulates the data packet of the media stream using the additional data determined by the additional data determination unit 2202 to generate the detection data packet. In some embodiments, the congestion control unit 2205 determines the congestion window and the sending rate of the data transmission of the media stream, and indicates them to the data packet transmission unit 2206. In some embodiments, the data packet transmission unit 2206 provides the media stream including the detection data packet to the network layer 2300 based on the congestion window and the sending rate indicated by the congestion control unit 2205. In some embodiments, the network layer 2300 receives feedback information for the media stream including the detection data packet from the receiving end device 120, and provides it to the data packet transmission unit 2206. In some embodiments, the data packet transmission unit 2206 provides the received feedback information to the feedback information processing unit 2207, and the feedback information processing unit 2207 determines information such as packet loss information, link available bandwidth, transmission delay, etc. for optimizing the transmission of media stream data based on the feedback information, and feeds it back to the congestion control unit 2205, so that the congestion control unit 2205 adjusts the congestion window and the sending rate. In some embodiments, the feedback information is, for example, packet loss information, link available bandwidth, and transmission delay, and can also be specifically implemented by positive feedback (Acknowledgement, ACK) and negative feedback (Negative Acknowledgement,) NACK). In some embodiments, the data buffer management unit 2204 includes a queue to be sent, a queue to be responded to, and a retransmission queue. In some embodiments, the queue to be sent contains data packets to be initially sent, the queue to be responded to contains data packets to be initially sent and waiting for the receiving device 120 to feedback whether the data packets are received, and the retransmission queue contains data packets that need to be retransmitted in response to the feedback from the receiving device 120. In some embodiments, the queue to be sent, the queue to be responded to, and the retransmission queue in the data buffer management unit 2204 can perform data transmission in units of frames.In some embodiments, the transmission frame may include at least one of an intra-frame coded frame (I frame), a forward predicted frame (P frame), and a bidirectional interpolated frame (B frame). Among them, the I frame is used to retain the entire picture information and has a larger size, the P frame is used to refer to other frame data for encoding and has a smaller size, and the B frame is used to refer to the data of the previous and next two frames for encoding and has the smallest size.
[0055] Figure 3 1 shows an example flow chart of a data transmission method according to an embodiment of the present disclosure, which specifically shows a data transmission method 300 executed by a transmitting end device 110. Figure 3 As shown, at step 301, the transmitting device 110 generates a detection data packet associated with the transmitting device 110 based on the traffic characteristics associated with the media stream. At step 303, the transmitting device 110 sends the detection data packet to the receiving device 120. At step 305, the transmitting device 110 determines the link available bandwidth for data transmission of the transmitting device 110 based on the feedback information received from the receiving device 110 for the detection data packet. According to an embodiment of the present disclosure, by supplementing the data volume during the trough period of the media stream to perceive the link bandwidth change, the erroneous congestion window or erroneous link bandwidth information can be accurately adjusted, thereby solving the problems of data over-transmission and video freeze. Thus, for example, the transmitting device can accurately evaluate the link available bandwidth and quickly identify bandwidth changes, thereby optimizing data transmission. It should be understood that Figure 3 The data transmission method 300 shown will be more clearly described in detail below in accordance with the embodiments of the present disclosure.
[0056] Figure 4 Another exemplary flow chart of a data transmission method according to an embodiment of the present disclosure is shown. Figure 3 Based on the data transmission method 300 shown in FIG. 1 , a data transmission method 400 performed by the transmitting end device 110 is further specifically shown. Figure 4As shown, at step 401, the transmitting end device 110 obtains the traffic characteristics associated with the media stream from the application layer 2100 to determine the additional data for the data packet of the media stream based on the traffic characteristics. In some embodiments, the traffic characteristics associated with the media stream may include peak and trough information, media stream required bandwidth information, and frame type information. In some embodiments, the transmitting end device 110 may also obtain information characteristics associated with the media stream from the application layer 2100, and the information characteristics may include frame boundary information, etc. At step 403, the transmitting end device 110 determines whether the media stream meets the link detection condition based on the peak and trough information included in the traffic characteristics. At step 405, the transmitting end device 110 determines to detect the available bandwidth of the link based on that the media stream meets the link detection condition. In some embodiments, the link detection condition is used, for example, to detect that there may be a trough in the link in the current network connection, so that there may be a problem of inaccurate available bandwidth of the link. At step 407, the transmitting end device 110 determines whether the traffic characteristics include the required bandwidth information of the media stream based on the determination to detect the available bandwidth of the link. In some embodiments, it is determined whether the traffic feature includes the media stream required bandwidth information, wherein the media stream required bandwidth information indicates whether the media stream has the required bandwidth value. In some embodiments, the media stream required bandwidth information can be implemented in the form of parameters, for example, defined by stream_desire_bandwidth, which has a value to indicate the bandwidth size required by the application layer. At step 409, if it is determined that the traffic feature includes the media stream required bandwidth, the additional data is determined based on the media stream required bandwidth information. In some embodiments, if the parameter of the media stream required bandwidth information has a value, it means that the application layer has a target bandwidth requirement, so as to detect the available bandwidth of the link according to the target bandwidth requirement of the application layer (also referred to as a bandwidth up-probing method). At step 411, if the transmitting end device 110 determines that the traffic feature does not include the media stream required bandwidth, the additional data is determined based on the peak and trough information. In some embodiments, if the parameter of the media stream required bandwidth information does not exist or does not have a corresponding value, it means that the application layer does not have a target bandwidth requirement, so as to detect the available bandwidth of the link according to the target bandwidth determined based on the peak and trough conditions (also referred to as a traffic supplement method). At step 413, the transmitting end device 110 encapsulates the data packet and the additional data for the media stream to generate the detection data packet. At step 415, the transmitting end device 110 determines the link available bandwidth for data transmission based on the feedback information for the detection data packet.
[0057] Figure 5AFIG. 5 shows an example flow chart of bandwidth detection according to an embodiment of the present disclosure, which further shows a bandwidth detection process 500 of a flow supplementation method performed by a transmitting end device 110 based on the data transmission method 300 and the data transmission method 400. Figure 5A As shown, at step 501, based on the frame type information of the current frame of the media stream, it is determined whether the current frame is used to send a service data packet, wherein the frame type information is included in the traffic characteristics. At step 503, based on determining that the current frame is used to send a service data packet, it is determined to send the service data packet, wherein the service data packet comes from a queue to be sent or a retransmission queue in the data buffer of the transmitting end device 110. In some embodiments, the transmitting end device 110 sends service data first to ensure the transmission of normal service data. At step 505, based on determining that the current frame is not used to send a service data packet, it is determined whether the current connection of the media stream is in an application-restricted state. At step 507, based on the current connection of the media stream being in an application-restricted state, it is determined whether the amount of data sent about the media stream is less than a first threshold, wherein the first threshold indicates the maximum amount of additional data required for the media stream. At step 509, based on determining that the amount of data sent about the media stream is less than the first threshold, data is obtained from the queue to be responded to in the data buffer as the additional data. At step 511, the transmitting end device 110 encapsulates the data packet and the additional data to generate the detection data packet. At step 513, the transmitting end device 110 determines the link available bandwidth for data transmission based on the feedback information of the detection data packet. On the other hand, if it is determined at step 505 that the current connection of the media stream is not in the application-restricted state or it is determined at step 507 that the amount of data sent for the media stream is greater than or equal to the first threshold, then return to step 501.
[0058] Figure 5B The diagram shows an example of a component structure of data transmission based on a flow supplementation method according to an embodiment of the present disclosure, which specifically shows an example of a structure of an applicable data transmission method 300, a data transmission method 400, a bandwidth detection process 500, and a data transmission process 550 described later. Figure 5BAs shown, the sending end device 110 includes an application layer 2100, a transport layer 2200 and a network layer 2300. Among them, the application layer 2100 includes a traffic feature providing unit 2101, a flow information providing unit 2102, and a cross-layer transmission interface 2103, and the transport layer 2200 includes a traffic feature management unit 2201, a traffic supplement unit 5202, a data packet generation unit 2203, a data cache management unit 2204, a congestion control unit 2205, a data packet transmission unit 2206, and a feedback information processing unit 2207. In some embodiments, the traffic feature providing unit 2101, the flow information providing unit 2102, and the cross-layer transmission interface 2103 of the application layer 2100 can be implemented in the form of specific modules of the Software Development Kit (SDK). Figure 5B In the embodiment shown, the flow supplement unit 5202 corresponds to Figure 2 The additional data determination unit 2202 shown determines the additional data in a flow supplement manner. It should be understood that Figure 5B The component structure example of data transmission based on the flow supplement method shown can be applied Figure 2 The component structure shown and its related specific embodiments.
[0059] Figure 5C An example flow chart of data transmission based on flow supplementation according to an embodiment of the present disclosure is shown. Figure 5A Based on the bandwidth detection process 500 shown in FIG. 5 , a data transmission process 550 based on the flow supplement method is further specifically shown. Figure 5CAs shown, at step 551, the application layer 2100 provides the traffic characteristics associated with the media stream to the transport layer 2200. In some embodiments, the application layer 2100 can provide the traffic characteristics associated with the media stream provided by the traffic characteristics providing unit 2101 and the media stream information provided by the stream information providing unit 2102 to the traffic characteristics management unit 2201 and the data cache management unit 2204 of the transport layer 2200 via the cross-layer transmission interface 2103. In some embodiments, the traffic characteristics providing unit 2101 extracts frame type information and information from each frame of the media stream as characteristic parameters of the traffic characteristics. In some embodiments, the frame type information is implemented in the form of parameters, for example, it can be defined as frame_type, for example, I frame / P frame, and the peak and valley information can be, for example, the peak and valley ratio of the media stream, which is implemented in the form of parameters, for example, it can be defined as tranffic_peak_and_valley_ratio, for example, it can be determined based on the frame size, for example, the I frame size is 150KB, and the P frame is 40KB. In some embodiments, when the application layer 2100 provides media stream information and traffic characteristics via the cross-layer transmission interface 2103, the cross-layer transmission interface 2103 may call the StreamSend() function interface to implement the characteristic parameters of the traffic characteristics in a manner of carrying them in the media stream information. For example, I frames and P frames may be defined as:
[0060]
[0061] Wherein, stream_id represents the media stream label, data represents the service data, data_len represents the service data length, frame_info represents the frame boundary information, which may include the member parameter frame_type to represent the frame type, and stream_flow_pattern represents the flow pattern feature structure, which may include the member parameter tranffic_peak_and_valley_ratio to represent the peak-to-valley ratio. At step 553, if it is determined that the traffic characteristic includes peak-to-valley information but does not include the required bandwidth of the media stream, the application layer 2200 determines the additional data based on the peak-to-valley information. At step 555, based on the frame type information of the current frame of the media stream, it is determined whether the current frame is used to send a service data packet, wherein the frame type information is included in the traffic characteristic. At step 557, based on determining that the current frame is used to send a service data packet, it is determined to send the service data packet, wherein the service data packet comes from the to-be-sent queue or the retransmission queue in the data buffer of the transmitting end device 110. In this way, by giving priority to sending the current frame service data, all service data of the sender can be completely sent before sending the supplemented traffic data, so that the traffic supplement method as an optimization means does not affect the normal sending of service data. At step 559, based on determining that the current frame is not used to send a service data packet, it is determined to generate a detection data packet for the media stream. If it is determined at step 559 to generate a detection data packet for the media stream, then at step 561, based on the frame type information of the latest sent frame in the media stream, it is determined whether the latest sent frame is an intra-frame coded frame (for example, an I frame), wherein the frame type information is included in the traffic feature. At step 563, based on determining that the latest sent frame is an intra-frame coded frame, it is determined whether the current connection of the media stream is in an application-restricted state. In some embodiments, if the current connection is not in an application-restricted state, it indicates that the service data is backlogged and traffic supplement cannot be performed, so it is necessary to perform traffic supplement when the current connection is in an application-restricted state. In some embodiments, the application-restricted state may indicate a state in which the application layer service data is limited and insufficient to fill the current network pipeline. In some embodiments, if the application is limited, the congestion control unit 2205 cannot detect any congestion information including congestion packet loss, and will limit the operation of the congestion control algorithm. In some embodiments, the transport layer 2200 can determine whether the current connection is in an application-limited state based on the following conditions: the data length in the socket send buffer is less than the maximum message length, there is no data in the send queue, the number of messages sent to the network is less than the congestion window, and all lost messages have been retransmitted.
[0062] At step 565, based on the current connection of the media stream being in an application-restricted state, it is determined whether the amount of data sent about the media stream is less than a first threshold, wherein the first threshold indicates the maximum amount of additional data required for the media stream. At step 567, based on determining that the amount of data sent about the media stream is less than the first threshold, data less than the first threshold is obtained from the queue to be responded in the data buffer as the additional data to generate a detection data packet. In some embodiments, if the statistically sent supplementary traffic data exceeds the first threshold, no traffic supplement is performed, otherwise a segment of data (e.g., 5KB) is encapsulated from the queue to be responded to and sent to supplement the traffic during the trough period of the media stream. In some embodiments, the first threshold is determined based on the ratio of the peak to the trough of the media stream indicated by the peak-to-trough information and the data amount of the latest sent frame. For example, the first threshold may indicate the upper limit of the traffic that needs to be supplemented to avoid bandwidth waste, which may be implemented in the form of a parameter, for example, defined as max_full_up_data_bytes. In some embodiments, if the most recently transmitted frame is a non-I frame such as a P frame, the data size of the most recently transmitted frame (e.g., frame_size) is recorded, and based on the peak-to-valley ratio of the peak-to-valley information provided by the application (e.g., tranffic_peak_and_valley_ratio), the first threshold value can be determined by multiplying the data size of the most recently transmitted frame, the peak-to-valley ratio, and an adjustable parameter (e.g., factor). In some embodiments, the upper limit of the bandwidth change can be controlled by controlling the adjustable parameter. For example, setting the adjustable parameter to 1 can ensure that the detected link bandwidth is sufficient to transmit the data volume during the peak period of the media stream.
[0063] At step 569, the transmitting end device 110 transmits a detection data packet generated based on the additional data, and determines the link available bandwidth for data transmission based on the feedback information of the receiving end device 120 for the detection data packet. In some embodiments, the feedback information processing unit 2207 of the application layer 2200 determines the reception time of the detection data packet received by the receiving end device 110 based on the feedback information for the detection data packet received from the receiving end device 110, and determines the data volume of the data packet transmitted within the predetermined time interval based on the reception time. Then, the feedback information processing unit 2207 determines the detected link available bandwidth based on the predetermined time interval and the data volume of the data packet transmitted within the time interval. In some embodiments, the receiving end device 120 records the reception status of each data packet sent in the feedback information of the detection data packet, and calculates the amount of data passing through the link during the round-trip time (RTT) within a certain period of time, so as to determine the detected link available bandwidth (for example, bytes / times). In some specific embodiments, if the original link bandwidth is 10 Mbps and then the link bandwidth is reduced to 8 Mbps, then for a media bit rate of 10 Mbps, the link becomes congested, the received data transmitted to the receiving end within a fixed time becomes smaller, and the calculated link available bandwidth is 8 Mbps.
[0064] At step 571, the transmitting end device 110 adjusts the data transmission based on the determined link available bandwidth. In some embodiments, if the detected link available bandwidth is greater than or equal to the bandwidth before the detected link available bandwidth, the service data packet is transmitted according to the congestion window and the transmission rate corresponding to the detected link available bandwidth. In other embodiments, if the detected link available bandwidth is less than the bandwidth before the detected link available bandwidth, the non-intra-frame coded frame (e.g., P frame, B frame) is discarded, and the service data packet is transmitted according to the congestion window and the transmission rate corresponding to the detected link available bandwidth. For example, if the link bandwidth becomes smaller, the link congestion window or the transmission rate is not corrected, which will lead to data over-transmission and face the risk of packet loss. In some specific embodiments, the congestion window is adjusted to a new bandwidth-delay product (Bandwidth-Delay Product, BDP), such as min_RTT*8Mbps, and the transmission rate can be updated to the link available bandwidth 8Mbps. In some embodiments, the bandwidth-delay product, for example, represents the product of the capacity of a data link and the round-trip communication delay, which can indicate the maximum amount of data of the link, that is, the data that has been sent but not yet confirmed. In some embodiments, the bandwidth-delay product can be calculated based on the product of the minimum delay of the link and the link bandwidth, for example, BDP (bandwidth-delay product) = minRTT (minimum delay of the link) * estimated_bandwidth (link bandwidth). On the other hand, if it is determined at step 561 that the latest transmitted frame is not an intra-frame coded frame, or, it is determined at step 563 that the current connection with respect to the media stream is not in an application-restricted state, or it is determined at step 565 that the amount of data transmitted with respect to the media stream is greater than or equal to the first threshold, then return to step 555.
[0065] In some embodiments, the application layer 2100 can update the encoding bit rate based on the detected available bandwidth of the link, and encode the media stream based on the updated encoding bit rate. For example, the transmission bit rate is reduced through a frame loss strategy to smoothly transmit data. In some specific embodiments, if the sender caches more than two P frames of data, the last P frame data can be retained, and the excess P frame data can be discarded, thereby reducing the amount of data sent and avoiding network congestion. At step 573, the sending end device 110 updates the encoding bit rate based on the detected available bandwidth of the link, and encodes the media stream based on the updated encoding bit rate. Thus, the determined link available bandwidth is provided to the media stream encoding side, which enables the transmission side and the encoding side to work together to optimize the control of the transmission bit rate of the media stream push.
[0066] Figure 5D A schematic diagram of data transmission based on flow supplementation according to an embodiment of the present disclosure is shown, wherein Figure 5D(A) shows the relationship between the physical bandwidth and the transmission rate in a scenario where the data transmission scheme of the present disclosure is not used. Figure 5D (B) shows the relationship between physical bandwidth and transmission rate in the scenario of using the data transmission solution of the present disclosure. Figure 5D As shown in (A), due to insufficient data in the traffic valley after the first I frame, the bandwidth is estimated too high. In the second I frame, the estimated transmission rate is higher than the actual physical bandwidth, which will lead to over-transmission of data, causing packet loss. In the fourth frame, the estimated transmission rate is lower than the actual physical bandwidth, which leads to insufficient bandwidth utilization. Figure 5D As shown in (B), since traffic is supplemented during the traffic trough period (as shown by the dotted line), the link bandwidth change can be detected in real time to determine the link available bandwidth. For example, the second P frame after the first I frame and the second P frame after the third I frame can detect the link bandwidth change in time. As a result, the estimated transmission rate can be kept as synchronized as possible with the actual physical bandwidth, so that the link available bandwidth can be accurately evaluated during the media stream trough period, bandwidth changes can be quickly identified, and data transmission can be optimized. At the same time, in the face of media stream bit rate bursts, traffic can be supplemented in advance, and bandwidth detection can be quickly completed to converge to the link available bandwidth, avoiding data backlogs at the sending end and causing jams.
[0067] According to the embodiments of the present disclosure, by supplementing the data volume during the trough period of the media stream to perceive the link bandwidth change, the erroneous congestion window or erroneous link bandwidth information can be accurately adjusted, thereby solving the problems of data over-transmission and video freezing. At the same time, the embodiments of the present disclosure can accurately reduce the service data transmission bit rate based on the link available bandwidth, thereby effectively reducing the video playback freezing rate.
[0068] Fig. 6A Another example flow chart of bandwidth detection according to an embodiment of the present disclosure is shown, which further shows a bandwidth detection process 600 of a bandwidth up-detection method performed by the transmitting end device 110 on the basis of the data transmission method 300 and the data transmission method 400. Fig. 6AAs shown, at step 601, the transmitting end device 110 determines the detection target bandwidth for detecting the link available bandwidth for data transmission of the transmitting end device 110 based on the bandwidth value required by the media stream indicated by the media stream required bandwidth information. In some embodiments, the transmitting end device 110 can count the bandwidth requirements of all media streams on the current connection to obtain the total target bandwidth for bandwidth up-probing. In some embodiments, the detection target bandwidth can be implemented in the form of parameters, such as defined by target_bandwidth. In some embodiments, at step 603, data is obtained from the queue to be responded in the data buffer as the additional data. In some embodiments, data for satisfying the detection target bandwidth can be obtained from the queue to be responded in the data buffer as the additional data, such as a data packet to be responded or a data packet for forward error correction (FEC). At step 605, the transmitting end device 110 encapsulates the data packet and the additional data to generate the detection data packet. In some embodiments, the transmitting end device 110 can mark the detection data packet and send it in a pacing manner. In some embodiments, the data to be sent can be queued and sent evenly at predetermined time intervals by pacing. At step 607, the transmitting device 110 counts the receiving rate of the detection data packet based on the feedback information of the detection data packet, and determines the link available bandwidth for data transmission. At step 609, the application layer 2200 of the transmitting device 110 provides the determined link available bandwidth to the application layer 2100 to perform encoding adjustment of the application layer 2100. In some embodiments, the determined link available bandwidth is provided to the media stream encoding side, which enables the transmission side to be linked with the encoding side to optimize the control of the transmission bit rate of the media stream push stream.
[0069] Figure 6B The component structure example diagram of data transmission based on bandwidth up-probing method according to an embodiment of the present disclosure is shown, which specifically shows the structure examples of applicable data transmission method 300, data transmission method 400, bandwidth detection process 600 and data transmission process 650 described later. Figure 6BAs shown, the sending end device 110 includes an application layer 2100, a transport layer 2200 and a network layer 2300. Among them, the application layer 2100 includes a traffic feature providing unit 2101, a flow information providing unit 2102, and a cross-layer transmission interface 2103, and the transport layer 2200 includes a traffic feature management unit 2201, a bandwidth detection unit 6202, a data packet generation unit 2203, a data cache management unit 2204, a congestion control unit 2205, a data packet transmission unit 2206, and a feedback information processing unit 2207. In some embodiments, the traffic feature providing unit 2101, the flow information providing unit 2102, and the cross-layer transmission interface 2103 of the application layer 2100 can be implemented in the form of specific modules of a software development kit (SDK). Figure 6B In the embodiment shown, the bandwidth up-probing unit 6202 corresponds to Figure 2 The additional data determination unit 2202 shown in the figure determines the additional data by means of bandwidth up-probing. It should be understood that Figure 6B The component structure example of data transmission based on bandwidth up-probing method shown in the figure can be applied to Figure 2 The component structure shown and its related specific embodiments.
[0070] Figure 6C An example flow chart of data transmission based on bandwidth up-probing method according to an embodiment of the present disclosure is shown. Fig. 6A Based on the bandwidth detection process 600 shown in FIG. 6 , a data transmission process 650 based on the flow supplement method is further specifically shown. Figure 6CAs shown, at step 651, the application layer 2100 provides the traffic characteristics associated with the media stream to the transport layer 2200. In some embodiments, the application layer 2100 can provide the traffic characteristics associated with the media stream provided by the traffic characteristics providing unit 2101 and the media stream information provided by the stream information providing unit 2102 to the traffic characteristics management unit 2201 and the data cache management unit 2204 of the transport layer 2200 via the cross-layer transmission interface 2103. In some embodiments, the traffic characteristics providing unit 2101 can carry the media stream demand bandwidth information when creating the media stream. In some specific embodiments, the cross-layer transmission interface 2103 can call the StreamCreate() function interface to create the media stream: StreamCreate(conn_id, stream_flow_pattern, stream_flow_qos.stream_desire_bandwidth=4Mbps), where conn_id represents the connection tag, stream_flow_pattern represents the stream pattern feature structure, and stream_flow_qos represents the stream service quality feature, which can include a member parameter stream_desire_bandwidth to indicate the specific value of the bandwidth requirement. In some embodiments, the traffic feature providing unit 2101 may also update the media stream demand bandwidth information when there is a bandwidth switching demand. In some specific embodiments, the cross-layer transmission interface 2103 may call the UpdateStreamAttribute() function interface to update the media stream: UpdateStreamAttribute(conn_id, stream_id, stream_flow_pattern, stream_flow_qos.stream_desire_bandwidth=20Mbps, Report()), where conn_id represents the connection tag, stream_id represents the media stream tag, stream_flow_pattern represents the stream pattern feature structure, stream_flow_qos represents the stream service quality feature, which may include a member parameter stream_desire_bandwidth to indicate the specific value of the bandwidth demand, and Report() represents the callback function pointer to report the detected link available bandwidth change to the application layer. In some embodiments, the traffic feature providing unit 2101 may extract frame type information and information from each frame of the media stream as feature parameters of the traffic feature.In some embodiments, the frame type information is implemented in the form of parameters, for example, it can be defined as frame_type, such as I frame / P frame / B frame, where I frame is used to retain the entire picture information and has a larger size, P frame is used to refer to other frame data for encoding and has a smaller size, and B frame is used to refer to the encoding of the previous and next two frames of data and has the smallest size. In some embodiments, when the application layer 2100 provides media stream information and traffic characteristics via the cross-layer transmission interface 2103, the cross-layer transmission interface 2103 can call the StreamSend() function interface to implement the characteristic parameters of the traffic characteristics in the form of carrying them in the media stream information. For example, for I frame and P frame, they can be defined as: respectively.
[0071] I frame StreamSend(stream_id,data,data_len,frame_info.frame_type=0,stream_flow_pattern) P-frames StreamSend(stream_id,data,data_len,frame_info.frame_type=1,stream_flow_pattern)
[0072] Wherein, stream_id represents a media stream tag, data represents service data, data_len represents service data length, frame_info represents frame boundary information, which may include a member parameter frame_type to represent a frame type, and stream_flow_pattern represents a stream pattern feature structure. At step 653, if it is determined that the traffic feature includes the media stream required bandwidth, it is determined to send the detection data packet to the receiving end device 120, and the additional data is determined based on the media stream required bandwidth information. In some embodiments, in response to the media stream required bandwidth information indicating an updated media stream required bandwidth, the detection data packet is sent to the receiving end device 120 during the transmission of non-intra-frame coded frames. In some specific embodiments, the application layer 2100 may actively call the UpdateStreamAttribute() interface to indicate that there is an update in the required bandwidth, so as to perform an upward probe during the transmission of non-I frame data, thereby enabling the perception of the target rate to be completed more quickly. In other embodiments, in response to the fact that there is no service data packet to be sent in the sending end device 110 and the time from the previous detection data packet to the sending exceeds a predetermined time, the detection data packet is sent to the receiving end device 120. In some specific embodiments, when the transmitting end device 110 has no business data to be sent and retransmitted data, and a certain time has passed since the last upward exploration, the upward exploration is performed, so as to detect whether the change of the link available bandwidth still meets the business needs through periodic detection. At step 655, the transmitting end device 110 determines the detection target bandwidth for detecting the link available bandwidth for data transmission of the transmitting end device 110. In some embodiments, the transmitting end device 110 can count the bandwidth requirements of all media streams on the current connection to obtain the total target bandwidth for bandwidth upward exploration. In some specific embodiments, the cross-layer transmission interface 2103 of the application layer 2100 can trigger the update of the target rate, and count the bandwidth requirements of the entire link as the total target bandwidth target_bandwidth (for example, 20Mbps) for bandwidth upward exploration by traversing the bandwidth requirements of all media streams of the current connection. At step 657, based on the frame type information of the current frame of the media stream, it is determined whether the current frame is used to send a business data packet, wherein the frame type information is included in the traffic characteristics. At step 659, based on determining that the current frame is used to send a service data packet, it is determined to send the service data packet, wherein the service data packet comes from a to-be-sent queue or a retransmission queue in a data buffer of the transmitting end device 110. In this way, by giving priority to sending the service data of the current frame, all service data of the transmitting end can be completely sent before sending the upward probe traffic data, so that the bandwidth upward probe method as an optimization means does not affect the normal transmission of service data.At step 661, based on determining that the current frame is not used to send a service data packet, it is determined to generate and send a detection data packet for the media stream, and data is obtained from the queue to be responded in the data buffer as the additional data to send the detection data packet generated based on the additional data. In some embodiments, in response to determining to generate a detection data packet for the media stream, based on the bandwidth value required for the media stream indicated by the media stream required bandwidth information, a detection target bandwidth is determined for detecting the link available bandwidth for data transmission of the transmitting end device 110. In some embodiments, based on the detection target bandwidth, data for satisfying the detection target bandwidth is obtained from the queue to be responded in the data buffer as the additional data. In other embodiments, padding bytes are used as the additional data to fill in to satisfy the detection target bandwidth. In some specific embodiments, after sending all service data, a certain amount of data (for example, 30 data packets) can be taken from the queue to be responded as a detection data packet. In some embodiments, if there are insufficient data packets in the queue to be responded, padding bytes can be used to supplement the number of data packets.
[0073] At step 663, the transmitting end device 110 transmits the detection data packet generated by the additional data according to the transmission rate corresponding to the detection target bandwidth during the period when the service data packet is not transmitted, and transmits the service data packet according to the transmission rate before detecting the available bandwidth of the link. In some specific embodiments, after completing the transmission of the detection data packet, the previous rate is restored to transmit the service data. For example, if the target bandwidth is continued to be used for data transmission, the link will be congested and packet lost when the link available bandwidth is not enough for the target bandwidth, so the service data cannot be transmitted according to the detection target bandwidth. As some specific implementations, before sending the detection data packet, the congestion control unit 2205 can record the transmission rate before detecting the available bandwidth of the link, for example, it can be implemented in the form of parameters, which can be defined as rate_before_probe (for example, 4Mbps). When sending the detection data packet, the congestion control unit 2205 can adjust the congestion window and the transmission rate to target_bandwidth (for example, 20Mbps). Then, after completing the sending of the probe data packet, the congestion control unit 2205 restores the congestion window and the sending rate to rate_before_probe (4 Mbps) to send the service data packet.
[0074] At step 665, the transmitting device 110 determines the link available bandwidth for data transmission based on the feedback information of the receiving device 120 for the detection data packet. In some embodiments, the feedback information processing unit 2207 of the application layer 2200 determines the reception time of the detection data packet received by the receiving device 110 based on the feedback information for the detection data packet received from the receiving device 110, and determines the data volume of the data packet transmitted within the predetermined time interval based on the reception time. Then, the feedback information processing unit 2207 determines the detected link available bandwidth based on the predetermined time interval and the data volume of the data packet transmitted within the time interval. In some embodiments, the bandwidth probe unit 6202 marks the probe data packets, wherein the marked probe data packets include a first probe data packet and a second probe data packet, and the feedback information processing unit 2207 determines the reception time difference between the reception time of the first probe data packet and the reception time of the second probe data packet based on the feedback information for the first probe data packet and the feedback information for the second probe data packet, determines the number of probe data packets transmitted within the reception time difference based on the determined reception time difference, and determines the detected link available bandwidth based on the reception time difference and the number of probe data packets. In some specific embodiments, the feedback information processing unit 2207 can evaluate the link available bandwidth based on the feedback information of the receiving end device 120, and the link available bandwidth can be implemented in the form of parameters, for example, defined as average_probe_rate. The feedback information of the receiving end device 120 includes the sequence number of the data packet received by the receiving end device 120 and the reception time of the data packet. For example, the feedback information processing unit 2207 can determine the detected average link available bandwidth average_probe_rate based on the receiving time t0 of the first probe data packet, the receiving time t1 of the last probe data packet, and the data volume probe_bytes of all probe data by the receiving end device 120 by calculating probe_bytes / (t1-t0).
[0075] At step 667, the transmitting end device 110 adjusts the data transmission based on the determined link available bandwidth. In some embodiments, based on the detected link available bandwidth and the detected target bandwidth, the congestion control unit 2205 determines the congestion window and the sending rate, and the data packet transmission unit 2206 transmits the service data packet according to the determined congestion window and the determined sending rate. In some embodiments, if the detected link available bandwidth is greater than or equal to the detected target bandwidth, the service data packet is transmitted according to the congestion window and the sending rate corresponding to the detected target bandwidth. In other embodiments, if the detected link available bandwidth is less than the detected target bandwidth, the service data packet is transmitted according to the congestion window and the sending rate before the detected link available bandwidth is detected. In some specific embodiments, the actual link available bandwidth can be implemented in the form of parameters, for example, it is defined as actual_bandwidth. If the actual link available bandwidth actual_bandwidth is greater than or equal to the detection target bandwidth target_bandwidth and no congestion occurs, it indicates that the detected link available bandwidth average_probe_rate is close to the detection target bandwidth target_bandwidth. If the actual link available bandwidth actual_bandwidth is less than the detection target bandwidth target_bandwidth and congestion occurs, it indicates that the detected link available bandwidth average_probe_rate is close to the actual link available bandwidth actual_bandwidth, that is, it reflects the real link available bandwidth. In some embodiments, the required bandwidth of other media streams on the network connection can be removed to determine the remaining bandwidth as the link available bandwidth for the current media stream. In some specific embodiments, if the required bandwidth of other media streams on the network connection is 0Mbps, the link available bandwidth for the current media stream can be provided with average_probe_rate, for example, 20Mbps. In other specific embodiments, if the detected link available bandwidth is 20 Mbps, the corresponding media stream required bandwidth is automatically updated to 20 Mbps, and if the detected link available bandwidth cannot meet user needs, the original required bandwidth of the media stream is maintained unchanged.
[0076] At step 669, the transmitting end device 110 updates the encoding bit rate based on the detected available bandwidth of the link, and encodes the media stream based on the updated encoding bit rate. In some embodiments, the determined available bandwidth of the link is provided to the media stream encoding side, which enables the transmission side and the encoding side to be linked to optimize the control of the transmission bit rate of the media stream push. In some specific embodiments, after the transport layer 2200 adjusts the congestion window and the sending rate to 20Mbps according to the available bandwidth of the link 20Mbps, the bandwidth-delay product (Bandwidth-Delay Product, BDP) can be calculated as 125KB (20Mbps*50ms), and the sending window is updated to 125KB (BDP). At the same time, the transport layer 2200 can call Report (available_bandwidth=20Mbps) through the callback function to send the link available bandwidth to the application layer 2100 and report it to the user. As a result, the user can directly provide the required bandwidth 20Mbps to the media stream encoder for encoding to generate the media bit rate of the required bandwidth.
[0077] Fig.6D A schematic diagram of data transmission based on bandwidth up-probing according to an embodiment of the present disclosure is shown, wherein Fig.6D (A) shows the relationship between the physical bandwidth and the transmission rate in a scenario where the data transmission scheme of the present disclosure is not used. Fig.6D (B) shows the relationship between physical bandwidth and transmission rate in the scenario of using the data transmission solution of the present disclosure. Fig.6D As shown in (A), due to insufficient data in the traffic valley after the first I frame, the bandwidth is estimated too high. In the second I frame, the estimated transmission rate is higher than the actual physical bandwidth, which will lead to over-transmission of data, causing packet loss. In the fourth frame, the estimated transmission rate is lower than the actual physical bandwidth, which leads to insufficient bandwidth utilization. Fig.6D As shown in (B), since the traffic is supplemented during the traffic trough period (as shown by the dotted line), the link bandwidth change can be detected in real time with the target transmission rate as a reference to determine the link available bandwidth. For example, the third P frame after the first I frame and the second P frame after the third I frame can detect the link bandwidth change in time. As a result, the transmission rate can be made as close as possible to the detected real physical bandwidth, so that the link available bandwidth can be accurately evaluated during the media stream trough period, the bandwidth change can be quickly identified, and data transmission can be optimized. At the same time, in the face of media stream bit rate bursts, the bandwidth can be probed in advance, and the bandwidth detection can be completed quickly to converge to the link available bandwidth, avoiding data backlogs at the sending end and causing jams.
[0078] According to the embodiments of the present disclosure, by supplementing the amount of data during the trough period of the media stream to perceive the link bandwidth change, the erroneous congestion window or erroneous link bandwidth information can be accurately adjusted, thereby solving the problems of data over-transmission and video freeze. Compared with the embodiment of data transmission based on the flow supplement method, the embodiment of data transmission based on the bandwidth upward detection method can purposefully detect by obtaining the actual encoding bit rate or the maximum encoding bit rate, which can achieve accurate detection of the link available bandwidth while further reducing the consumption of bandwidth traffic caused by detecting the link available bandwidth.
[0079] In an embodiment of the present disclosure, the cross-layer transmission interface 2103 of the application layer 2100 can be used in the QUIC protocol to implement the stream creation function interface StreamCreate(), the stream sending function interface StreamSend(), the stream attribute update function interface UpdateStreamAttribute(), and the reporting function interface Report() as shown in Tables 1 to 4 below.
[0080] Table 1 - Stream creation function interface semantics
[0081]
[0082] Table 2 - Stream send function interface semantics
[0083]
[0084] Table 3 - Stream attribute update function interface semantics
[0085]
[0086]
[0087] Table 4 – Reporting function interface semantics
[0088] Function interface name Report(available_bandwidth) Function Introduction The reporting function interface is used to report the current link available bandwidth when the transport layer detects that the link available bandwidth has changed. Parameter Description available_bandwidth: The available bandwidth of the link.
[0089] It should be understood that the function interface implementation of the QUIC protocol in the above table is only an example, and the method or parameters of the present disclosure can be modified appropriately to implement the embodiments of the present disclosure. For example, in some embodiments, the stream creation function interface StreamCreate() can also be defined in the form of StreamOpen(conn_id, stream_flow_pattern, stream_flow_qos). As a result, the present invention can be adapted to the transmission protocol in a suitable manner, thereby simply improving the transmission efficiency without changing the overall design.
[0090] Figure 7Another exemplary flow chart of a data transmission method according to an embodiment of the present disclosure is shown, which further specifically shows a data transmission method 700 executed by a receiving end device 120. Figure 7 As shown, at step 701, the receiving device 120 receives a media stream including a detection data packet associated with the sending device 110 from the sending device 110, wherein the detection data packet is generated based on the traffic characteristics associated with the media stream. At step 703, the receiving device 120 generates feedback information of the receiving device 120 for the detection data packet associated with the sending device 110 based on the received media stream, wherein the feedback information is used to determine the link available bandwidth for data transmission of the sending device 110. At step 705, the receiving device 120 sends the feedback information to the sending device 110. It should be understood that in the embodiment of the present disclosure, the data transmission method 700 executed by the receiving device 120 corresponds to the above-mentioned data transmission method executed by the sending device 110, and those skilled in the art can modify, delete and adjust the data transmission method 700 executed by the receiving device 120 based on the above-mentioned data transmission method executed by the sending device 110. In some embodiments, when the receiving device 120 acts as a sender, it can also execute all or part of the data transmission method executed by the above-mentioned sending device 110, or when the sending device 110 acts as a receiver, it can also execute all or part of the data transmission method executed by the receiving device 120. The present disclosure does not impose any special restrictions on this.
[0091] In some embodiments, the detection data packet is generated by encapsulating the data packet of the media stream and additional data, wherein the additional data is determined based on the traffic characteristics, wherein the traffic characteristics include peak and trough information. In some embodiments, in the determination of the additional data, based on the peak and trough information, it is determined whether the media stream meets the link detection condition, and based on the media stream meeting the link detection condition, it is determined to detect the available bandwidth of the link. In some embodiments, in the determination about detecting the available bandwidth of the link, it can be determined whether the traffic characteristics include media stream required bandwidth information, wherein the media stream required bandwidth information indicates whether the media stream has a required bandwidth value, if it is determined that the traffic characteristics include the media stream required bandwidth, then the additional data is determined based on the media stream required bandwidth information, and if it is determined that the traffic characteristics do not include the media stream required bandwidth, then the additional data is determined based on the peak and trough information. In some embodiments, the receiving device 120 receives the detection data packet from the transmitting device 110, wherein the detection data packet is sent by the transmitting device 110 in response to the media stream required bandwidth information indicating the updated media stream required bandwidth during the transmission of non-intra-frame coded frames. In other embodiments, the receiving device 120 receives the detection data packet from the transmitting device 110, wherein the detection data packet is sent by the transmitting device 110 in response to the absence of a service data packet to be sent in the transmitting device 110 and the time from the last detection data packet to being sent exceeds a predetermined time. In some embodiments, the feedback information of the receiving device 120 for the detection data packet includes a reception time based on the receiving device 110 receiving the detection data packet, wherein the feedback information is used by the transmitting device 110 to determine the data volume of the data packet transmitted within the predetermined time interval, so that the transmitting device 110 determines the link available bandwidth for data transmission of the transmitting device 110 based on the predetermined time interval and the data volume of the data packet transmitted within the time interval.
[0092] According to the embodiments of the present disclosure, the receiving device can normally feed back the detection data packet to the sending device according to the transmission framework, so that the data transmission scheme of the present disclosure can be simply implemented without changing the overall transmission framework. As a result, the sending device can use the feedback information to accurately evaluate the available bandwidth of the link, avoid the obvious jamming caused by inaccurate bandwidth evaluation in application-restricted scenarios and slow convergence speed in scenarios with sudden changes in bandwidth demand for media streams, so as to quickly identify bandwidth changes and effectively optimize data transmission.
[0093] Fig. 8A A schematic block diagram of a data transmission device according to some embodiments of the present disclosure is shown, which specifically shows a data transmission device 810 corresponding to the transmitting end device 110, and Figure 8BFIG. 8 is a schematic block diagram of another data transmission device according to some embodiments of the present disclosure, which specifically shows a data transmission device 820 corresponding to the receiving end device 120. Fig. 8A As shown, the data transmission device 810 includes: a data packet generation module 811, configured to generate a detection data packet associated with the data transmission device 810 based on the traffic characteristics associated with the media stream; a data packet transmission module 812, configured to send the detection data packet to the data transmission device 820; and a bandwidth determination module 813, configured to determine the link available bandwidth for data transmission of the data transmission device 810 based on the feedback information for the detection data packet received from the data transmission device 820. Accordingly, as Figure 8B As shown, the data transmission device 820 includes: a data packet receiving module 821, configured to receive a media stream including a detection data packet associated with the data transmission device 810 from the data transmission device 810, wherein the detection data packet is generated based on the traffic characteristics associated with the media stream; a feedback information generating module 822, configured to generate feedback information of the data transmission device 820 for the detection data packet associated with the data transmission device 810 based on the received media stream, wherein the feedback information is used to determine the link available bandwidth for data transmission of the data transmission device 810; and a feedback information sending module 823, configured to send the feedback information to the data transmission device 810.
[0094] In some embodiments, the data packet generation module 811 is further configured to: determine additional data for the data packet of the media stream based on the traffic characteristics; and encapsulate the data packet and the additional data for the media stream to generate the detection data packet. Thus, for example, the sending end device can adapt to the media stream by means of additional data, thereby accurately and quickly identifying the actual situation of the link bandwidth change without modifying the transmission architecture.
[0095] In some embodiments, the traffic characteristics include peak and trough information, and the data packet generation module 811 is further configured to: determine whether the media stream meets the link detection condition based on the peak and trough information; and determine to detect the link available bandwidth based on the media stream meeting the link detection condition. Thus, for example, the transmitting end device can identify the media stream with peak and trough characteristics, thereby enabling the link available bandwidth evaluation more accurately and reducing the link bandwidth loss.
[0096] In some embodiments, the data packet generation module 811 is further configured to: determine whether the traffic characteristics include the media stream required bandwidth information, wherein the media stream required bandwidth information indicates whether the media stream has the required bandwidth value; if it is determined that the traffic characteristics include the media stream required bandwidth, determine the additional data based on the media stream required bandwidth information; and if it is determined that the traffic characteristics do not include the media stream required bandwidth, determine the additional data based on the peak and trough information. Thus, for example, the sending end device can flexibly adopt the traffic supplement method or the bandwidth up-probing method to detect the link available bandwidth, thereby enhancing the accuracy and practicality of the link available bandwidth evaluation.
[0097] In some embodiments, the data packet generation module 811 is further configured to: determine whether the current frame is used to send a service data packet based on the frame type information of the current frame of the media stream, wherein the frame type information is included in the traffic characteristics; determine to send the service data packet based on determining that the current frame is used to send a service data packet, wherein the service data packet comes from a queue to be sent or a retransmission queue in the data buffer of the data transmission device 810; and determine to generate a detection data packet for the media stream based on determining that the current frame is not used to send a service data packet. Thus, by giving priority to sending the current frame service data, all service data of the sender can be completely sent before sending the supplementary or upward detection traffic data, so that the traffic supplement method or bandwidth upward detection method as an optimization means does not affect the normal transmission of service data.
[0098] In some embodiments, the data packet generation module 811 is further configured to: if it is determined to generate a detection data packet for the media stream, determine whether the latest transmitted frame is an intra-frame coded frame based on the frame type information of the latest transmitted frame in the media stream, wherein the frame type information is included in the traffic feature; and determine whether the current connection with respect to the media stream is in an application-restricted state based on the determination that the latest transmitted frame is an intra-frame coded frame. If the current connection is not in an application-restricted state, it indicates that the business data is backlogged and traffic cannot be supplemented, and therefore traffic supplementation is required when the current connection is in an application-restricted state. As a result, traffic supplementation can be performed only when the application is restricted without affecting the normal transmission of business data.
[0099] In some embodiments, the data packet generation module 811 is further configured to: determine whether the amount of data sent about the media stream is less than a first threshold value based on the current connection of the media stream being in an application-restricted state, wherein the first threshold value indicates the maximum amount of additional data required for the media stream; based on determining that the amount of data sent about the media stream is less than the first threshold value, obtain data less than the first threshold value from the waiting-for-response queue in the data buffer as the additional data. In some embodiments, the first threshold value is determined based on the ratio of the peak to the trough of the media stream indicated by the peak and trough information and the data amount of the latest sent frame. Thus, by considering the amount of data sent by the media stream, the excessive flow replenishment is limited based on the peak and trough characteristics of the media stream, thereby reducing flow loss while accurately evaluating the bandwidth status.
[0100] In some embodiments, the data packet generation module 811 is further configured to: in response to determining to generate a detection data packet for the media stream, determine the detection target bandwidth for detecting the link available bandwidth for data transmission of the data transmission device 810 based on the bandwidth value required by the media stream indicated by the media stream required bandwidth information; based on the detection target bandwidth, determine the additional data by at least one of the following: obtain data for satisfying the detection target bandwidth from the waiting-for-response queue in the data buffer as the additional data, or use padding bytes as the additional data to fill in to satisfy the detection target bandwidth. Therefore, compared with the embodiment of data transmission based on the flow supplement method, the embodiment of data transmission based on the bandwidth up-probing method can purposefully detect by obtaining the actual encoding bit rate or the maximum encoding bit rate, and can realize accurate detection of the link available bandwidth while further reducing the consumption of bandwidth traffic caused by detecting the link available bandwidth.
[0101] In some embodiments, the data packet transmission module 812 is further configured to perform at least one of the following: in response to the media stream required bandwidth information indicating the updated media stream required bandwidth, during the transmission of non-intra-frame coded frames, send the detection data packet to the data transmission device 820; or in response to the absence of a service data packet to be sent in the data transmission device 810 and the time from the last detection data packet to be sent exceeds a predetermined time, send the detection data packet to the data transmission device 820. In this way, the timing of bandwidth increase can be further accurately determined by considering both active detection and response demand detection.
[0102] In some embodiments, the data packet transmission module 812 is further configured to: transmit the detection data packet generated by the additional data at a transmission rate corresponding to the detection target bandwidth during the period when the service data is not transmitted; and transmit the service data packet at a transmission rate before detecting the link available bandwidth during the period when the service data packet is transmitted. In this way, the risk of link congestion and packet loss caused by insufficient link available bandwidth when data is transmitted using bandwidth up-probing can be avoided, thereby further optimizing the data transmission logic.
[0103] In some embodiments, the bandwidth determination module 813 is further configured to: determine the reception time of the detection data packet received by the data transmission device 820 based on the feedback information for the detection data packet received from the data transmission device 820; determine the data volume of the data packet transmitted within the predetermined time interval based on the reception time; determine the detected link available bandwidth based on the predetermined time interval and the data volume of the data packet transmitted within the time interval. In some embodiments, determining the link available bandwidth for data transmission of the data transmission device 810 further includes: marking the detection data packet, wherein the marked detection data packet includes a first detection data packet and a second detection data packet; determining the reception time difference between the reception time of the first detection data packet and the reception time of the second detection data packet based on the feedback information for the first detection data packet and the feedback information for the second detection data packet; determining the number of detection data packets transmitted within the reception time difference based on the determined reception time difference; determining the detected link available bandwidth based on the reception time difference and the number of detection data packets. Thus, the real link available bandwidth can be accurately evaluated, thereby quickly completing the bandwidth detection process for data transmission adjustment.
[0104] In some embodiments, the bandwidth determination module 813 is further configured to: if the detected available bandwidth of the link is greater than or equal to the bandwidth before the detected available bandwidth of the link, transmit the service data packet according to the congestion window and transmission rate corresponding to the detected available bandwidth of the link; and if the detected available bandwidth of the link is less than the bandwidth before the detected available bandwidth of the link, discard the non-intra-frame coded frame, and transmit the service data packet according to the congestion window and transmission rate corresponding to the detected available bandwidth of the link. Thus, the service data transmission bit rate can be accurately reduced based on the available bandwidth of the link, thereby effectively reducing the video playback freeze rate.
[0105] In some embodiments, the bandwidth determination module 813 is further configured to: determine the congestion window and the sending rate based on the detected link available bandwidth and the detection target bandwidth; and transmit the service data packet according to the determined congestion window and the determined sending rate. In some embodiments, determining the link available bandwidth for data transmission of the data transmission device 810 also includes: if the detected link available bandwidth is greater than or equal to the detection target bandwidth, then transmitting the service data packet according to the congestion window and the sending rate corresponding to the detection target bandwidth; and if the detected link available bandwidth is less than the detection target bandwidth, then transmitting the service data packet according to the congestion window and the sending rate before detecting the link available bandwidth. In this way, the transmission rate can be made as close as possible to the detected real physical bandwidth, so that the available bandwidth of the link can be accurately evaluated during the media stream trough period, the bandwidth change can be quickly identified, and the data transmission can be optimized. At the same time, in the face of media stream bit rate bursts, the bandwidth can be probed in advance, and the bandwidth detection can be quickly completed to converge to the link available bandwidth, avoiding the backlog of data at the sending end and causing jams.
[0106] In some embodiments, the data transmission device 810 is further configured to: update the encoding bit rate based on the detected available bandwidth of the link; and encode the media stream based on the updated encoding bit rate. Thus, the determined available bandwidth of the link is provided to the media stream encoding side, which enables the transmission side to be linked with the encoding side to optimize the control of the transmission bit rate of the media stream push. In some embodiments, the detection data packet is generated by encapsulating the data packet and additional data of the media stream, wherein the additional data is determined based on the traffic characteristics, wherein the traffic characteristics include peak and trough information, and the determination of the additional data includes: determining whether the traffic characteristics include media stream required bandwidth information, wherein the media stream required bandwidth information indicates whether the media stream has a required bandwidth value; if it is determined that the traffic characteristics include the media stream required bandwidth, then determining the additional data based on the media stream required bandwidth information; and if it is determined that the traffic characteristics do not include the media stream required bandwidth, determining the additional data based on the peak and trough information.
[0107] In some embodiments, the data packet receiving module 821 is further configured to at least one of the following: receive the probe data packet, wherein the probe data packet is sent by the data transmission device 810 in response to the media stream required bandwidth information indicating an updated media stream required bandwidth during the transmission of non-intra-frame coded frames; or receive the probe data packet, wherein the probe data packet is sent by the data transmission device 810 in response to the absence of a business data packet to be sent in the data transmission device 810 and the time from the last time the probe data packet was sent exceeds a predetermined time.
[0108] In some embodiments, the feedback information generated by the feedback information generation module 822 includes the reception time of the detection data packet received by the data transmission device 820, wherein the feedback information is used by the data transmission device 810 to determine the data volume of the data packets transmitted within a predetermined time interval, so that the data transmission device 810 determines the link available bandwidth for data transmission of the data transmission device 810 based on the predetermined time interval and the data volume of the data packets transmitted within the time interval.
[0109] According to the embodiments of the present disclosure, the transmitting device can sense the link bandwidth change by supplementing the data volume during the trough period of the media stream, and can accurately adjust the erroneous congestion window or erroneous link bandwidth information, thereby solving the problems of data over-transmission and video freezing. Accordingly, the receiving device can normally feedback the detection data packet to the transmitting device according to the transmission framework, so that the data transmission scheme of the present disclosure can be simply implemented without changing the overall transmission framework. As a result, the transmitting device can use the feedback information to accurately evaluate the available bandwidth of the link, avoid the obvious freezing problem caused by inaccurate bandwidth evaluation in application-restricted scenarios and slow convergence speed in scenarios where the media stream demand bandwidth suddenly changes, so as to quickly identify bandwidth changes and effectively optimize data transmission.
[0110] Fig. 9A A schematic block diagram of an example device 910 that can be used to implement an exemplary implementation of the present disclosure is shown, which can be used to implement at least a portion of the method or process of an embodiment of the present disclosure and can specifically correspond to the sending end device 110. Fig. 9AAs shown, the device 910 includes a central processing unit (CPU) 911, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 912 or computer program instructions loaded from a storage unit 918 into a random access memory (RAM) 913. In the RAM 913, various programs and data required for the operation of the device 910 can also be stored. The CPU 911, the ROM 912, and the RAM 913 are connected to each other through a bus 914. An input / output (I / O) interface 915 is also connected to the bus 914. Multiple components in the device 910 are connected to the I / O interface 915, including: an input unit 916, such as a keyboard, a mouse, etc.; an output unit 917, such as various types of displays, speakers, etc.; a storage unit 918, such as a disk, an optical disk, etc.; and a communication unit 919, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 919 allows the device 910 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks. The various methods and / or processes described above may be executed by the processing unit 911. For example, in some exemplary implementations, the methods may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 918. In some exemplary implementations, part or all of the computer program may be loaded and / or installed on the device 910 via the ROM 912 and / or the communication unit 919. When the computer program is loaded into the RAM 913 and executed by the CPU 911, one or more actions of the methods described above may be performed.
[0111] Fig. 9B A schematic block diagram of an example device 920 that can be used to implement an exemplary implementation of the present disclosure is shown, which can be used to implement at least a portion of the method or process of an embodiment of the present disclosure and can specifically correspond to the receiving end device 120. Fig. 9BAs shown, the device 920 includes a central processing unit (CPU) 921, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 922 or computer program instructions loaded from a storage unit 928 into a random access memory (RAM) 923. In the RAM 923, various programs and data required for the operation of the device 920 can also be stored. The CPU 921, the ROM 922, and the RAM 923 are connected to each other through a bus 924. An input / output (I / O) interface 925 is also connected to the bus 924. Multiple components in the device 920 are connected to the I / O interface 925, including: an input unit 926, such as a keyboard, a mouse, etc.; an output unit 927, such as various types of displays, speakers, etc.; a storage unit 928, such as a disk, an optical disk, etc.; and a communication unit 929, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 929 allows the device 920 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks. The various methods and / or processes described above may be executed by the processing unit 921. For example, in some exemplary implementations, the methods may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 928. In some exemplary implementations, part or all of the computer program may be loaded and / or installed on the device 920 via the ROM 922 and / or the communication unit 929. When the computer program is loaded into the RAM 923 and executed by the CPU 921, one or more actions of the methods described above may be performed.
[0112] According to an exemplary implementation of the present disclosure, an electronic device is provided, comprising: at least one processor and at least one memory, wherein the at least one memory is coupled to the at least one processor and stores instructions for execution by the at least one processing unit, and when the instructions are executed by the at least one processing unit, the device executes the method described above.
[0113] The present disclosure may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present disclosure.
[0114] Computer readable storage medium can be a tangible device that can keep and store the instructions used by the instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used herein is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic waves, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.
[0115] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.
[0116] The computer program instructions for performing the disclosed operation may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, and conventional procedural programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, executed as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In some exemplary implementations, by utilizing the state information of a computer-readable program instruction to customize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit may execute a computer-readable program instruction, thereby realizing various aspects of the present disclosure.
[0117] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products according to the exemplary implementations of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.
[0118] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0119] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0120] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple exemplary implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.
[0121] Various embodiments of the present disclosure have been described, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. A data transmission method, characterized in that: include: generating a probe data packet associated with the first device based on traffic characteristics associated with the media stream; Sending the detection data packet to the second device; as well as Based on the feedback information received from the second device for the detection data packet, an available link bandwidth for data transmission of the first device is determined.
2. The data transmission method according to claim 1, characterized in that: Generating a probe packet associated with the first device includes: Based on the traffic characteristics, determining additional data for the data packet of the media stream; and For the media stream, the data packet and the additional data are encapsulated to generate the detection data packet.
3. The data transmission method according to claim 2, characterized in that: The traffic characteristics include peak and trough information, and determining the additional data for the data packet of the media stream includes: Based on the peak and trough information, determining whether the media stream meets a link detection condition; Based on the media stream satisfying the link detection condition, it is determined to detect the available bandwidth of the link.
4. The data transmission method according to claim 3, characterized in that: Determining the additional data for the data packet of the media stream includes: Determining whether the traffic characteristic includes media stream required bandwidth information, wherein the media stream required bandwidth information indicates whether the media stream has a required bandwidth value; If it is determined that the traffic characteristics include the media stream required bandwidth, determining the additional data based on the media stream required bandwidth information; and If it is determined that the traffic characteristic does not include the required bandwidth of the media stream, the additional data is determined based on the peak and trough information.
5. The data transmission method according to claim 4, characterized in that: Generating a detection data packet for the media stream includes: Determining whether the current frame is used to send a service data packet based on frame type information of the current frame of the media stream, wherein the frame type information is included in the traffic feature; Determining to send the service data packet based on determining that the current frame is used to send the service data packet, wherein the service data packet comes from a to-be-sent queue or a retransmission queue in a data buffer of the first device; and Based on determining that the current frame is not used to send a service data packet, determining to generate a detection data packet for the media stream.
6. The data transmission method according to claim 4, characterized in that: Determining the additional data based on the peak and trough information includes: If it is determined to generate a detection data packet for the media stream, determining whether the latest transmitted frame in the media stream is an intra-frame coded frame based on frame type information of the latest transmitted frame in the media stream, wherein the frame type information is included in the traffic feature; and Based on determining that the most recently sent frame is an intra-coded frame, it is determined whether a current connection with respect to the media stream is in an application-limited state.
7. The data transmission method according to claim 6, characterized in that: Determining the additional data based on the peak and trough information includes: Based on the current connection of the media stream being in an application-restricted state, determining whether the amount of data sent with respect to the media stream is less than a first threshold, wherein the first threshold indicates a maximum amount of additional data required with respect to the media stream; Based on determining that the amount of data sent about the media stream is less than the first threshold, data less than the first threshold is obtained from the waiting-for-response queue in the data buffer as the additional data.
8. The data transmission method according to claim 7, characterized in that: The method further comprises: The first threshold is determined based on the ratio of the peak to the trough of the media stream indicated by the peak-to-trough information and the data amount of the latest sent frame.
9. The data transmission method according to claim 4, characterized in that: Determining the additional data based on the media stream required bandwidth information includes: In response to determining to generate a detection data packet for the media stream, determining a detection target bandwidth based on a bandwidth value required for the media stream indicated by the media stream required bandwidth information for detecting an available bandwidth of a link for data transmission of the first device; Based on the detection target bandwidth, the additional data is determined by at least one of the following: obtaining data for satisfying the detection target bandwidth from a waiting-for-response queue in the data buffer as the additional data, or using padding bytes as the additional data to fill in to satisfy the detection target bandwidth.
10. The data transmission method according to claim 9, characterized in that: Sending the detection data packet to the second device includes at least one of the following: In response to the media stream required bandwidth information indicating an updated media stream required bandwidth, sending the detection data packet to the second device during the transmission of non-intra-frame coded frames; or In response to the fact that there is no service data packet to be sent in the first device and a predetermined time has passed since the last sending of the detection data packet, the detection data packet is sent to the second device.
11. The data transmission method according to claim 9, characterized in that: Sending the detection data packet to the second device includes: During a period when service data is not transmitted, transmitting a detection data packet generated using the additional data at a transmission rate corresponding to the detection target bandwidth; and During the transmission of the service data packet, the service data packet is transmitted according to the transmission rate before detecting the available bandwidth of the link.
12. The data transmission method according to any one of claims 1 to 11, characterized in that: Determining the link available bandwidth for data transmission of the first device includes: Determine, based on feedback information received from the second device for the detection data packet, a reception time at which the second device receives the detection data packet; determining, based on the reception time, a data volume of the data packet transmitted within a predetermined time interval; The detected link available bandwidth is determined based on the predetermined time interval and the data volume of the data packets transmitted within the time interval.
13. The data transmission method according to claim 12, characterized in that: Determining the link available bandwidth for data transmission of the first device further includes: Marking the detection data packets, wherein the marked detection data packets include a first detection data packet and a second detection data packet; Determine a reception time difference between a reception time of the first detection data packet and a reception time of the second detection data packet based on feedback information for the first detection data packet and feedback information for the second detection data packet; Based on the determined reception time difference, determining the number of probe data packets transmitted within the reception time difference; Based on the reception time difference and the number of the detection data packets, the detected link available bandwidth is determined.
14. The data transmission method according to claim 12, characterized in that: Determining the link available bandwidth for data transmission of the first device further includes: If the detected available bandwidth of the link is greater than or equal to the bandwidth before the detected available bandwidth of the link, transmitting the service data packet according to the congestion window and sending rate corresponding to the detected available bandwidth of the link; and If the detected link available bandwidth is less than the bandwidth before detecting the link available bandwidth, non-intra-frame coded frames are discarded, and service data packets are transmitted according to the congestion window and sending rate corresponding to the detected link available bandwidth.
15. The data transmission method according to claim 12, characterized in that: Determining the link available bandwidth for data transmission of the first device further includes: Determine a congestion window and a sending rate based on the detected link available bandwidth and the detected target bandwidth; and The service data packet is transmitted according to the determined congestion window and the determined sending rate.
16. The data transmission method according to claim 15, characterized in that: Determining the link available bandwidth for data transmission of the first device further includes: If the detected available bandwidth of the link is greater than or equal to the detection target bandwidth, transmitting the service data packet according to the congestion window and sending rate corresponding to the detection target bandwidth; and If the detected available bandwidth of the link is smaller than the detection target bandwidth, the service data packet is transmitted according to the congestion window and the sending rate before the available bandwidth of the link is detected.
17. The data transmission method according to claim 12, characterized in that: The method further comprises: Based on the detected available bandwidth of the link, updating the encoding bit rate; and The media stream is encoded based on the updated encoding bit rate.
18. A data transmission method, characterized in that: include: receiving, from a first device, a media stream including a probe packet associated with the first device, wherein the probe packet is generated based on traffic characteristics associated with the media stream; generating feedback information of the detection data packet for the media stream based on the received media stream, wherein the feedback information is used to determine a link available bandwidth for data transmission of the first device; as well as The feedback information is sent from the second device to the first device.
19. The data transmission method according to claim 18, characterized in that: The detection data packet is generated by encapsulating the data packet of the media stream and the additional data, wherein the additional data is determined based on the traffic characteristics, wherein the traffic characteristics include peak and trough information, and the determination of the additional data includes: Determining whether the traffic characteristic includes media stream required bandwidth information, wherein the media stream required bandwidth information indicates whether the media stream has a required bandwidth value; If it is determined that the traffic characteristics include the media stream required bandwidth, determining the additional data based on the media stream required bandwidth information; and If it is determined that the traffic characteristic does not include the required bandwidth of the media stream, the additional data is determined based on the peak and trough information.
20. The data transmission method according to claim 18, characterized in that: Receiving, from a first device, a media stream including a probe packet associated with the first device comprises at least one of the following: receiving the detection data packet, wherein the detection data packet is sent by the first device in response to the media stream required bandwidth information indicating an updated media stream required bandwidth during transmission of non-intra-frame coded frames; or The detection data packet is received, wherein the detection data packet is sent by the first device in response to the absence of a service data packet to be sent in the first device and a predetermined time has passed since the last detection data packet was sent.
21. The data transmission method according to claim 18, characterized in that: The feedback information includes the reception time when the second device receives the detection data packet, wherein the feedback information is used by the first device to determine the data volume of the data packets transmitted within a predetermined time interval, so that the first device determines the link available bandwidth for data transmission of the first device based on the predetermined time interval and the data volume of the data packets transmitted within the time interval.
22. A data transmission device, characterized in that: include: a data packet generation module configured to generate a detection data packet associated with the first device based on a traffic characteristic associated with the media stream; A data packet transmission module, configured to send the detection data packet to the second device; as well as The bandwidth determination module is configured to determine the link available bandwidth for data transmission of the first device based on the feedback information for the detection data packet received from the second device.
23. A data transmission device, characterized in that: include: a data packet receiving module configured to receive, from a first device, a media stream including a probe data packet associated with the first device, wherein the probe data packet is generated based on a traffic characteristic associated with the media stream; a feedback information generating module, configured to generate feedback information of the detection data packet for the media stream based on the received media stream, wherein the feedback information is used to determine the link available bandwidth for data transmission of the first device; as well as A feedback information sending module is configured to send the feedback information to the first device.
24. A data transmission system, characterized in that: It includes a first device and a second device, wherein: The first device generates a detection data packet associated with the first device based on traffic characteristics associated with the media stream, and transmits the media stream including the detection data packet to the second device; The second device receives the media stream from the first device; The second device generates feedback information of the detection data packet for the media stream based on the received media stream, and sends the feedback information to the first device; and The first device receives the feedback information from the second device, and determines a link available bandwidth for data transmission of the first device based on the received feedback information for the detection data packet.
25. A computing device comprising: processor; as well as A memory storing instructions, which, when executed by the processor, cause the computing device to perform the method according to any one of claims 1 to 17.
26. A computing device comprising: processor; as well as A memory storing instructions, which, when executed by the processor, cause the computing device to perform the method according to any one of claims 18 to 21.
27. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, which, when executed by a computing device, cause the computing device to perform the method according to any one of claims 1 to 17.
28. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions which, when executed by a computing device, cause the computing device to perform the method according to any one of claims 18 to 21.
29. A computer program product, characterized in that The computer program product comprises instructions which, when executed by a computing device, cause the computing device to perform a method according to any one of claims 1 to 17.
30. A computer program product, characterized in that The computer program product comprises instructions which, when executed by a computing device, cause the computing device to perform a method according to any one of claims 18 to 21 .
Citation Information
Cited By
Dynamic data transmission method and device based on human-computer interaction, equipment and medium
CN122437842A
