Bandwidth-adaptive data transmission method and device

Abstract

The invention relates to a bandwidth-adaptive data transmission method and device, and the method comprises the steps: a transmitting end transmits first media data to a receiving end according to a preset period, and receives second media data fed back by the receiving end; wherein the first media data are hierarchical media data with known attribute information, the first media data at least comprise basic layer data, and the first media data in different periods comprise different data layer numbers; the sending end obtains the attribute information of the second media data and the evaluation index of the bandwidth, and determines the current available bandwidth of the network according to the attribute information of the second media data and the evaluation index of the bandwidth; and the sending end sends the layered media data of the target layer number according to the current available bandwidth of the network. According to the data transmission method provided by the invention, the media data can be adaptively transmitted based on the current available bandwidth, the transmission quality of the media data is ensured to the greatest extent, and the jamming phenomenon is reduced.

Description

Technical Field

[0001] This application relates to the technical field of data transmission, and particularly to a bandwidth adaptive data transmission method and device. Background Art

[0002] The audio and video command and dispatch system is an integrated communication platform integrating video, audio, data transmission and control functions, and is widely used in multiple fields such as public security, emergency response, military command, and traffic management. By real-time transmitting audio and video data, the command center can obtain on-site information in real time, so as to achieve reasonable decision-making and resource allocation.

[0003] Currently, the audio and video command and dispatch system generally uses a communication network for data transmission. For example, audio and video data is transmitted through communication systems such as satellite communication systems, scatter communication systems, and microwave communication systems. However, data transmission using a communication network depends on network bandwidth. When the network condition suddenly deteriorates, it will cause image stuttering, delay or interruption, resulting in a poor user experience. Summary of the Invention

[0004] In view of the above problems in the prior art, this application provides a bandwidth adaptive data transmission method and device, which adaptively adjusts the transmitted audio and video data based on the change of network bandwidth, and transmits clear and smooth images to the greatest extent.

[0005] To achieve the above object, the first aspect of this application provides a bandwidth adaptive data transmission method, including: the sending end sends first media data to the receiving end at a preset period, and receives second media data fed back by the receiving end; wherein, the first media data is layered media data with known attribute information, and the first media data includes at least base layer data, and the number of data layers included in the first media data in different periods is different; the sending end obtains the attribute information of the second media data and the evaluation index of the bandwidth, and determines the current available bandwidth of the network according to the attribute information of the second media data and the evaluation index of the bandwidth; the sending end sends layered media data of the target layer according to the current available bandwidth of the network.

[0006] Thus, in this aspect, by using the layered media data as detection data to detect the current available bandwidth of the network in real time, the available resources of the current network can be accurately obtained based on this detection. Then, based on the available resources of the current network, the media data is sent according to the target layer, which can make the most of the current existing network bandwidth. In the case of low bandwidth, less data layers are transmitted to ensure that the video playback is not stuck or interrupted. In the case of high bandwidth, more data layers are transmitted to provide higher resolution and video quality, thereby optimizing the user experience and contributing to the stable and efficient operation of the network environment.

[0007] As an implementation manner of this aspect, the sending end sends first media data to the receiving end according to a preset period, including: in two adjacent periods, the number of data layers of the first media data sent in this period is more than that of the first media data sent in the previous period.

[0008] As described above, in network bandwidth detection, the maximum available network bandwidth can be gradually approximated by gradually increasing the number of data layers transmitted, so as to accurately evaluate the network state.

[0009] As an implementation manner of this aspect, the sending end obtains the attribute information of the second media data and the evaluation index of the bandwidth, and determines the current available bandwidth of the network according to the attribute information of the second media data and the evaluation index of the bandwidth, including: when the evaluation index of the bandwidth includes the data round-trip time, the sending end determines the data round-trip time according to the attribute information of the second media data, and determines the current available bandwidth of the network according to the data round-trip time; when the evaluation index of the bandwidth includes the data packet loss rate, the sending end determines the data packet loss rate according to the attribute information of the second media data, and determines the current available bandwidth of the network according to the data packet loss rate; and / or when the evaluation index of the bandwidth includes the data jitter state, the sending end determines the data jitter state according to the attribute information of the second media data, and determines the current available bandwidth of the network according to the data jitter state.

[0010] As described above, the current available bandwidth of the network can be more accurately evaluated through these three indicators: the data round-trip time, the packet loss rate, and the jitter state.

[0011] As an implementation manner of this aspect, when the evaluation index of the bandwidth includes the data round-trip time, the sending end determines the data round-trip time according to the attribute information of the second media data, and determines the current available bandwidth of the network according to the data round-trip time, including: obtaining the sending time of the first media data and the receiving time of the second media data; determining the data round-trip time according to the sending time of the first media data and the receiving time of the second media data; when the data round-trip time exceeds the first threshold, using the bandwidth corresponding to the first media data in the previous period adjacent to this period as the current available bandwidth of the network.

[0012] As an implementation of this aspect, when the evaluation metric of the bandwidth includes the data packet loss rate, the sender determines the data packet loss rate according to the attribute information of the second media data, and determines the current available bandwidth of the network according to the data packet loss rate, including: obtaining the number of data packets of the first media data and the number of data packets of the second media data; determining the data packet loss rate based on the number of data packets of the first media data and the number of data packets of the second media data; when the data packet loss rate exceeds the second threshold, using the bandwidth corresponding to the first media data in the previous period adjacent to this period as the current available bandwidth of the network.

[0013] As an implementation of this aspect, when the evaluation metric of the bandwidth includes the data jitter state, the sender determines the data jitter state according to the attribute information of the second media data, and determines the current available bandwidth of the network according to the data jitter state, including: obtaining the jitter value of the second media data in this period and the jitter value of the second media data in the previous period; determining the difference between the two according to the jitter value of the second media data in this period and the jitter value of the second media data in the previous period; when the difference exceeds the third threshold, using the bandwidth corresponding to the first media data in the previous period adjacent to this period as the current available bandwidth of the network.

[0014] As described above, by comparing the round-trip time with the first threshold, comparing the packet loss rate with the second threshold, and comparing the jitter value of the network with the third threshold, a bandwidth that can balance at least one of the three can be found, which is conducive to better evaluating the available network bandwidth.

[0015] As an implementation of this aspect, the sender sends the hierarchical media data of the target layer according to the current available bandwidth of the network, including: the sender obtains the target data layer of the hierarchical media data according to the current available bandwidth of the network; the sender sends the target data layer of the hierarchical media data.

[0016] As described above, by sending the media data of the target data layer based on the current available bandwidth of the network, the current available bandwidth can be utilized to the maximum extent.

[0017] As an implementation of this aspect, it further includes: performing hierarchical processing on the media data to divide the media data into multiple data layers, the multiple data layers including a base layer and multiple enhancement layers, and the spatial parameters, temporal parameters, and / or quality parameters of the multiple data layers being different; independently encapsulating the multiple data layers and adding identification information to obtain the encapsulated multiple data layers for the sender to send, where the identification information is used to represent the respective types of the multiple data layers.

[0018] As an implementation of this aspect, it further includes: setting priorities for the multiple data layers respectively, and the data layer with a higher priority is sent first; wherein, the priority of the base layer is set as the highest priority.

[0019] As described above, a method for hierarchical processing of media data is provided. By using multiple parameters such as spatial parameters, temporal parameters, and / or quality parameters to divide data layers and setting priorities for each data layer, target media data can be sent according to the actual bandwidth and requirements, thereby improving the user experience.

[0020] The second aspect of the present application provides a bandwidth adaptive data transmission device, including: a transmission module disposed at the sending end, configured to send first media data to the receiving end at a preset period and receive second media data fed back by the receiving end; wherein, the first media data is hierarchical media data with known attribute information, and the first media data includes at least base layer data, and the number of data layers included in the first media data in different periods is different; a determination module disposed at the sending end, configured to obtain the attribute information of the second media data and the evaluation index of the bandwidth, and determine the current available bandwidth of the network according to the attribute information of the second media data and the evaluation index of the bandwidth; a sending module disposed at the sending end, configured to send hierarchical media data of a target number of layers according to the current available bandwidth of the network.

[0021] The beneficial effects of this aspect can also be referred to the descriptions of the beneficial effects of each part of the first aspect above.

[0022] The third aspect of the present application provides a computing device, including: at least one processor; and at least one memory, which is connected to the processor and stores program instructions, and when the program instructions are executed by the at least one processor, the at least one processor is caused to execute the bandwidth adaptive data transmission method described in any one of the first aspects above.

[0023] The beneficial effects of this aspect can also be referred to the descriptions of the beneficial effects of each part of the first aspect above.

[0024] The fourth aspect of the present application provides a computer-readable storage medium, on which program instructions are stored, and when the program instructions are executed by a computer, the computer is caused to execute the bandwidth adaptive data transmission method described in any one of the first aspects above.

[0025] The beneficial effects of this aspect can also be referred to the descriptions of the beneficial effects of each part of the first aspect above. Description of the Drawings

[0026] The various technical features of the present application and the relationships between them will be further described below with reference to the accompanying drawings. The drawings are exemplary, and some technical features are not shown to scale. Also, in some of the drawings, technical features that are conventional in the technical field to which the present application pertains and are not essential for understanding and implementing the present application may be omitted, or additional technical features that are not essential for understanding and implementing the present application may be shown. That is to say, the combination of the various technical features shown in the drawings is not used to limit the present application. Additionally, throughout the present application, the content referred to by the same reference numerals is the same. The specific description of the drawings is as follows:

[0027] Figure 1 It is a flowchart of a bandwidth adaptive data transmission method provided by an embodiment of the present application;

[0028] Figure 2 It is a flowchart of a specific embodiment of a bandwidth adaptive data transmission method provided by an embodiment of the present application;

[0029] Figure 3 It is a schematic diagram of an encoding structure after hierarchical processing of target media data provided by an embodiment of the present application;

[0030] Figure 4 It is a schematic diagram of the structure of a bandwidth adaptive data transmission device provided by an embodiment of the present application;

[0031] Figure 5 It is a schematic diagram of the structure of a computing device provided by an embodiment of the present application. Detailed implementation manners

[0032] The technical solutions provided by the present application will be further described below with reference to the accompanying drawings and by way of examples. It should be understood that the methods, devices, and service scenarios provided in the embodiments of the present application are mainly used to illustrate possible implementation manners of the technical solutions of the present application and should not be construed as the sole limitation of the technical solutions of the present application. Those of ordinary skill in the art will understand that with the evolution of technology and the emergence of new service scenarios, the technical solutions provided by the present application are equally applicable to similar technical problems.

[0033] It should be understood that the embodiments of the present application provide a bandwidth adaptive data transmission solution. Since the principles of these technical solutions for solving problems are the same or similar, in the following introduction of specific embodiments, some repeated parts may not be elaborated again, but it should be regarded that there are mutual references between these specific embodiments and they can be combined with each other.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs. In case of inconsistency, the meaning described in this specification or the meaning derived from the content recorded in this specification shall prevail. In addition, the terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0035] The embodiments of this application will be described in detail below with reference to the accompanying drawings. First, the application scenarios of the bandwidth adaptive data transmission solution provided by the embodiments of this application will be introduced. This solution can be applied in the field of traffic management. For example, when applied to an intelligent traffic monitoring system, the video stream can be transmitted to the monitoring center in real time through the bandwidth adaptive technology of this application, helping the traffic management department to monitor the traffic situation in real time. This solution can also be applied in the field of public safety and emergency response. For example, when applied to various command and dispatch platforms (such as the command and dispatch of natural disasters such as earthquakes and fires), the on-site video can be transmitted to the command headquarters in real time through the bandwidth adaptive technology of this application to facilitate real-time decision-making based on the on-site situation. This solution can also be applied in the field of industrial monitoring. For example, when applied to factory monitoring, it ensures that managers can view the factory situation in real time, and even in the case of limited network resources, it will not affect the monitoring effect, thus ensuring efficient and safe production.

[0036] It should be understood that the above application scenarios are exemplary descriptions and do not limit the scope of this application.

[0037] Next, referring to the accompanying drawings, a bandwidth adaptive data transmission method provided by the embodiments of this application will be described in detail. In this embodiment, as Figure 1 shown, the bandwidth adaptive data transmission method includes steps S110 - S130:

[0038] S110: The sending end sends the first media data to the receiving end at a preset period and receives the second media data fed back by the receiving end.

[0039] Among them, the first media data is hierarchical media data with known attribute information (how to perform hierarchical processing on media data can be seen in the detailed description below). The attribute information includes but is not limited to the data size of the first media data, the number of data layers of the first media data, the sending time of the first media data, the number of data packets of the first media data, the network bandwidth required to transmit different data layers of the first media data, and the data metrics corresponding to each layer of data in the first media data. The data metrics are used to represent the data quality of each layer of data. For example, the data metrics can include resolution, frame rate, bit rate, etc. In some embodiments, the first media data includes at least base layer data, and the number of data layers of the first media data corresponding to different periods can be different.

[0040] In this step, the first media data is sent as a data probe by the sending end at a preset period, transmitted to the receiving end through a preset communication protocol, and then the receiving end feeds back the received data to the sending end based on the preset communication protocol. At this time, the data fed back by the receiving end and received by the sending end is recorded as the second media data.

[0041] To accurately evaluate the current available bandwidth, in this embodiment, in two adjacent periods, the number of data layers of the first media data in the current period (also referred to as this period) is more than that of the first media data in the previous period, that is: by gradually increasing the number of data layers of the first media data to gradually increase the consumed bandwidth, so as to evaluate the maximum available bandwidth of the network. As an implementation manner, since the base layer data provides basic video content and has the lowest resolution and bit rate, the first media data in this embodiment at least includes the base layer data, so as to ensure the most basic video playback. For different periods, the number of data layers can be gradually increased on the basis of the base layer data for bandwidth testing.

[0042] In some embodiments, the sending end sends the first media data to the receiving end through the network. After receiving the first media data, the receiving end returns the first media data to the sending end through the network. In the embodiments of this application, the media data returned by the receiving end and received by the sending end is called the second media data. Among them, if packet loss occurs to the first media data due to problems such as insufficient network bandwidth during the above network transmission process, the second media data received by the sending end is the data packets of the first media data that have not been lost; if no packet loss occurs during the above network transmission process, the second media data received by the sending end is exactly the same as the first media data.

[0043] S120: The sending end obtains the attribute information of the second media data and the evaluation index of the bandwidth, and determines the current available bandwidth of the network according to the attribute information of the second media data and the evaluation index of the bandwidth.

[0044] In this step, the sending end receives the second media data fed back by the receiving end. Among them, after the sending end sends the first media data and receives the second media data corresponding to the first media data, by comparing and calculating the attribute information of the second media data with the attribute information of the corresponding first media data, the current available bandwidth of the network is determined, which may specifically include one or more of the following:

[0045] A. When the evaluation index of the bandwidth includes the Round-Trip Time (RTT), the round-trip time is calculated through the attribute information of the first media data and the attribute information of the second media data, so as to determine the current available bandwidth of the network.

[0046] Specifically: Obtain the sending time of the first media data and the receiving time of the second media data. The sending time is recorded when the sending end sends the first media data, and the receiving time is recorded when the sending end receives the second media data. For the first media data and the second media data in the same group (i.e., the same cycle), calculate the time elapsed from sending the first media data to receiving the second media data and record it as the data round-trip time. When the data round-trip time exceeds the first threshold, it indicates that the first media data exceeds the current maximum network bandwidth allowed. Therefore, the bandwidth corresponding to the first media data in the previous cycle adjacent to this cycle can be used as the current available bandwidth of the network. In this embodiment, since the method of this embodiment is to gradually increase the number of data layers in each cycle to approach the maximum available bandwidth of the network, the bandwidth corresponding to the first media data in the previous cycle is less than the bandwidth corresponding to the first media data in this cycle. If it can execute to this cycle, it means that the media data corresponding to the previous cycle is within the allowed bandwidth. Therefore, when the data round-trip time exceeds the first threshold, the bandwidth corresponding to the first media data in the previous cycle is used as the current available bandwidth of the network.

[0047] It should be noted here that the sending time of the first media data is obtained from the attribute information of the first media data, and the receiving time of the second media data is obtained from the attribute information of the second media data.

[0048] B. When the evaluation index of the bandwidth includes the data packet loss rate, calculate the data packet loss rate through the attribute information of the first media data and the attribute information of the second media data, so as to determine the current available bandwidth of the network.

[0049] Specifically: Obtain the number of data packets of the first media data and the number of data packets of the second media data. The number of data packets of the first media data is recorded by the sending end, and the number of data packets of the second media data is recorded when the sending end receives the second media data. For the first media data and the second media data in the same group, the data packet loss rate can be calculated according to their respective numbers of data packets. When the data packet loss rate exceeds the second threshold, the bandwidth corresponding to the first media data in the previous test cycle adjacent to this cycle can be used as the current available bandwidth of the network.

[0050] Similarly, since the method of this embodiment is to gradually increase the number of data layers in each cycle to approach the maximum available bandwidth of the network, the bandwidth corresponding to the first media data in the previous cycle is less than the bandwidth corresponding to the first media data in this cycle. If it can execute to this cycle, it means that the media data corresponding to the previous cycle is within the allowed bandwidth. Therefore, when the data packet loss rate exceeds the second threshold, the bandwidth corresponding to the first media data in the previous cycle is used as the current available bandwidth of the network.

[0051] It should be noted here that the number of data packets of the first media data is obtained from the attribute information of the first media data, and the number of data packets of the second media data is obtained from the attribute information of the second media data.

[0052] C. When the evaluation index of the bandwidth includes the data jitter state, the current available bandwidth of the network can be determined according to the network jitter state.

[0053] Specifically: Obtain the jitter value of the second media data in this period and the jitter value of the second media data in the previous period and calculate the difference between the two. When the difference between the two exceeds the third threshold, it indicates that the network jitter has increased significantly during the test in this period. At this time, use the bandwidth corresponding to the first media data in the previous test period adjacent to this period as the current available bandwidth of the network.

[0054] It should be noted here that the jitter value of the second media data is obtained from the attribute information of the second media data.

[0055] It should be understood that the above determination of the current available bandwidth of the network through one or more of the round-trip time of data, data packet loss rate, and data jitter value is an exemplary description. In other embodiments, the current available bandwidth can also be determined through other metrics that can measure the bandwidth (such as the number of data packets waiting to be processed in the network buffer, that is, the queue length, etc.).

[0056] S130: The sending end sends the layered media data of the target layer according to the current available bandwidth of the network.

[0057] The media data in this embodiment is pre-layered media data (how to perform layered processing on the media data can refer to the relevant descriptions below). In this step, obtain the target data layer of the layered media data according to the current available bandwidth of the network. The target data layer can be a single layer, in which case it is the base layer data, or the target data layer can be multiple layers, in which case it is the base layer data and one or more enhancement layer data; the bandwidth occupied by the target data layer is the current available bandwidth. When the target number of layers of the layered media data is obtained, the sending end sends the target data layer of the layered media data, thereby completing the transmission of the media data this time.

[0058] Before step S110 of the present application, there is also a step of layered processing of the media data (such as the first media data in the above text). Next, the implementation process of its layered processing will be described.

[0059] Specifically: First, the media data is hierarchically processed to divide the media data into multiple data layers. The multiple data layers include a base layer and multiple enhancement layers, and the spatial parameters, temporal parameters, and / or quality parameters of each data layer are different. As an implementation method, the hierarchical processing of the media data can be implemented based on Scalable Video Coding (SVC) technology. Then, the multiple data layers are independently encapsulated respectively to ensure that each data layer can be independently received and decoded. In this embodiment, it is also necessary to add identification information to each data layer to represent its respective type, such as indicating that it is the base layer or which enhancement layer it is, and the parameters corresponding to each data layer (such as spatial parameters, temporal parameters, and / or quality parameters, etc.) can also be represented by identification signals.

[0060] In some embodiments, respective priorities can also be set for each data layer, and the data layer with a higher priority is sent first. In this embodiment, since the base layer has the most basic data content and the lowest resolution and bit rate, the priority of the base layer is set to the highest priority, and the priorities corresponding to the enhancement layers are set in sequence according to the bandwidth they occupy.

[0061] Next, refer to Figure 2 the flowchart shown to introduce a specific embodiment of a bandwidth adaptive data transmission method provided by the embodiments of the present application, which includes steps S210 - S240.

[0062] S210: Hierarchically process the media data to be transmitted to obtain media data with multiple data layers.

[0063] Among them, the media data usually includes audio data and / or video data (abbreviated as audio - video data).

[0064] In this embodiment, the media data to be transmitted is hierarchically processed based on Scalable Video Coding (SVC) technology. Each data layer is an independent sub - data stream, and the receiving end can independently decode and play the received data layer. Among them, the base layer provides the lowest resolution, the lowest frame rate, and the lowest video quality, and the enhancement layer provides additional resolution, additional frame rate, and additional video quality. The enhancement layer needs to rely on the base layer to implement decoding. Therefore, each data transmission must include the base layer data, and optionally includes some enhancement layer data.

[0065] In some embodiments, SVC includes the following three scalable parameters:

[0066] The first is the spatial scalability parameter (also called the spatial parameter): It provides different spatial resolutions, such as from 480p to 720p or 1080p, etc.

[0067] The second is the temporal scalability parameter (also known as the temporal parameter): It provides different frame rates, for example, increasing from 5fps to 30fps or 60fps.

[0068] The third is the quality scalability parameter (also known as the quality parameter): It provides different coding qualities, for example, providing a lower bitrate or a higher bitrate.

[0069] Figure 3 A schematic diagram of an encoding structure obtained by hierarchically processing the media data to be transmitted (only for exemplary description, the number of data layers and the corresponding parameters of each layer can be adjusted according to the actual situation). In this example, the media data to be transmitted is encoded into 12 data layers, namely 1 base layer (BASE) and 11 enhancement layers (L1 - L11). Specifically, the resolution corresponding to the BASE layer is CIF, the frame rate is 5 frames, and the bitrate is 128K (i.e., CIF / 5 frames / 128K); the resolutions corresponding to L1 layer and L2 layer are both CIF, the frame rate is 5 frames, and the bitrate is 32K (i.e., CIF / 5 frames / 32K). Among them, the network bandwidth occupied by the BASE layer to L2 layer is 256K. The resolutions corresponding to L3 layer and L4 layer are both 4CIF, the frame rate is 5 frames, and the bitrate is 128K (i.e., 4CIF / 5 frames / 128K). Among them, the network bandwidth occupied by the BASE layer to L4 layer is 512K. The resolutions corresponding to L5 layer to L7 layer are both 720P, the frame rate is 5 frames, and the bitrate is 128K (i.e., 720P / 5 frames / 128K). Among them, the network bandwidth occupied by the BASE layer to L7 layer is 1M. The resolutions corresponding to L8 layer to L11 layer are both 1080P, the frame rate is 5 frames, and the bitrate is 256K (i.e., 1080P / 5 frames / 256K). Among them, the network bandwidth occupied by the BASE layer to L11 layer is 2M. From this structure diagram, the bandwidth occupied by each data layer and the parameter values of each parameter (resolution, frame rate, bitrate) of each data layer can be clearly obtained.

[0070] From the above, the media data divided into multiple data layers is obtained.

[0071] S220: Prepare the multiple data layers of the media data to be transmitted before transmission.

[0072] In this embodiment, the Real-time Transport Protocol (RTP) is used to transmit the layered target media data. It is necessary to perform separate RTP encapsulation on each data layer to ensure that each data layer can be independently received and processed by the receiving end in the network, so as to adapt to different network conditions. During actual transmission, each data layer is usually transmitted through an independent RTP session. The Payload Type field in the RTP header is used to identify different types of layering, for example, it can distinguish different types of layered data such as the base layer, spatial enhancement layer, temporal enhancement layer, and quality enhancement layer.

[0073] In this embodiment, before transmitting the media data, it is necessary to set the priority of each data layer. The priority of the BASE layer is set to the highest, and the priorities of the data layers L1 to L11 are gradually decreased in turn, so that the data layer with a higher priority can be transmitted first to avoid network congestion.

[0074] In this embodiment, the data of each data layer is output in the form of Network Abstraction Layer (NAL) units. Each NAL unit (i.e., each data layer) not only includes the encoded multimedia data (i.e., audio and video data), but also includes layering information in its header, such as layer identification, timing information, etc. The layer identification is used to distinguish the base layer and each enhancement layer, and the timing information is used to represent the time sequence and time hierarchical relationship of video frames, which helps the decoder correctly process video frames at different time points. When these NAL units are transmitted through the RTP protocol, the receiving end can identify the layer information and timing information of the data by parsing the header information of the NAL units in the RTP, so as to realize the transmission of the multimedia data.

[0075] In this embodiment, when transmitting each data layer through the RTP protocol, the RTP header will also include timestamp information and sequence numbers. The timestamp information is used to synchronize the data of different data layers to make the different data layers maintain time consistency; the sequence number is used to detect the loss of data packets. If the receiving end finds that the sequence numbers are not continuous, it can identify the packet loss event, thus avoiding data transmission errors.

[0076] S230: Estimate the network bandwidth by sending media data with different numbers of layers in different cycles to obtain the real-time available bandwidth.

[0077] In this embodiment, the network bandwidth can be estimated based on the round-trip time of data, the data packet loss rate, and / or the data jitter status. Specifically:

[0078] A. Send media data of different layers as probe data in different cycles and transmit it using the User Datagram Protocol (UDP).

[0079] In this embodiment, first select the BASE layer, L1 layer, and L2 layer as the media data to be sent for the first time.

[0080] B. Measure the round-trip time of the media data in each cycle to evaluate the network latency. Specifically, for the media data in each cycle sent, obtain the sending time at the sending end and the time when the sending end receives the feedback data, so as to obtain the round-trip time.

[0081] C. Set an appropriate sending rate according to the measurement result of the round-trip time in the initial (i.e., the first cycle) and gradually increase the data layers to be sent (such as L3, L4...).

[0082] D. Monitor the packet loss rate: During the process of gradually increasing the number of data layers to be sent, continuously monitor the packet loss rate of the data. When the packet loss rate starts to increase significantly, it indicates that network congestion starts to occur, and the media data sent at this time is close to the available bandwidth of the network.

[0083] E. Evaluate the network jitter situation: During the process of gradually increasing the number of data layers to be sent, continuously monitor the network jitter situation. When the network jitter becomes significantly larger, it indicates that the network resource competition intensifies, and the media data sent at this time is close to the available bandwidth of the network.

[0084] F. Determine the available bandwidth of the network (or the maximum throughput of the network): When it is detected that the round-trip time becomes significantly larger, the packet loss rate increases significantly, and the network jitter becomes significantly larger, it indicates that the amount of data that can be transmitted by the network at this time may have exceeded the limit. Next, the number of data layers to be sent can be reduced, and the sending strategy can be adjusted, so as to find the best layered data combination that can ensure a low packet loss rate and maintain an acceptable jitter level, and use this layered data combination as the maximum throughput or available bandwidth of the current network.

[0085] In order to still be able to transmit clear images as much as possible when the available bandwidth changes, this embodiment continuously performs the above network bandwidth evaluation and dynamically feeds back the available bandwidth in real time according to the evaluation results, so that the media data to be transmitted can be adaptively transmitted.

[0086] S240: Based on the real-time available bandwidth, adjust the number of layers of the media data to be transmitted in real time and send the media data to be transmitted based on the adjustment result.

[0087] In this step, based on the configuration of the media data to be transmitted in step S220 and the real-time available bandwidth in step S230, the number of transmission layers of the media data to be transmitted is adjusted in real time, so as to realize the adaptive transmission of the media data to be transmitted according to the real-time available bandwidth, and thus try to transmit higher-quality media data to the receiving end.

[0088] Based on the bandwidth adaptive data transmission method provided in the above embodiments, the media data to be transmitted is first hierarchically encoded, and then the available bandwidth of the current network is evaluated by sending different layers of media data in different periods, and then the media data to be transmitted is sent based on the available bandwidth of the current network. Less data layers can be transmitted in case of low bandwidth or network congestion to ensure smooth and uninterrupted video playback. More data layers can be transmitted under high bandwidth conditions to provide higher resolution and video quality, etc., thus optimizing the user experience.

[0089] Another embodiment of the present application provides a bandwidth adaptive data transmission device 40. It should be understood that Figure 4 only an exemplary structural schematic diagram of a bandwidth adaptive data transmission device 40 is shown, and the present application does not limit the division of functional modules in the bandwidth adaptive data transmission device. As Figure 4 shown, the device 40 can be logically divided into multiple modules, each module can have different functions, and the functions of each module can be implemented by a processor in a computing device reading and executing instructions in a memory. Exemplarily, the device 40 includes a transmission module 410 disposed at the sending end, a determination module 420 disposed at the sending end, and a sending module 430 disposed at the sending end. In one implementation, the bandwidth adaptive data transmission device 40 is used to execute Figure 1 or Figure 2 the content shown.

[0090] Specifically: The transmission module 410 is used to send first media data to the receiving end according to a preset period and receive second media data fed back by the receiving end; wherein, the first media data is hierarchically media data with known attribute information, and the first media data includes at least base layer data, and the number of data layers included in the first media data in different periods is different. The determination module 420 is used to obtain the attribute information of the second media data and the evaluation index of the bandwidth, and determine the current available bandwidth of the network according to the attribute information of the second media data and the evaluation index of the bandwidth. The sending module 430 is used to send hierarchically media data of the target number of layers according to the current available bandwidth of the network.

[0091] Among them, the specific implementation manners of each functional module in this embodiment can refer to the introductions of the above embodiments, and this embodiment will not elaborate on them anymore.

[0092] Figure 5 It is a structural schematic diagram of a computing device 900 provided by an embodiment of the present application. The computing device can execute various alternative embodiments of the above bandwidth adaptive data transmission method. The computing device can be a terminal, or a chip or chip system inside the terminal. As Figure 5 shown, the computing device 900 includes: a processor 910, a memory 920, and a communication interface 930.

[0093] It should be understood that Figure 5 the communication interface 930 in the computing device 900 shown can be used for communication with other devices, and specifically can include one or more transceiver circuits or interface circuits.

[0094] Among them, the processor 910 can be connected to the memory 920. The memory 920 can be used to store the program code and data. Therefore, the memory 920 can be an internal storage unit of the processor 910, an external storage unit independent of the processor 910, or a component including an internal storage unit of the processor 910 and an external storage unit independent of the processor 910.

[0095] Optionally, the computing device 900 can further include a bus. Among them, the memory 920 and the communication interface 930 can be connected to the processor 910 through the bus. The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 4 a line without an arrow is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0096] It should be understood that in the embodiments of the present application, the processor 910 may adopt a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Alternatively, the processor 910 adopts one or more integrated circuits for executing relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0097] The memory 920 may include a read-only memory and a random access memory, and provide instructions and data to the processor 910. A part of the processor 910 may also include a non-volatile random access memory. For example, the processor 910 may also store information about the device type.

[0098] When the computing device 900 is running, the processor 910 executes the computer-executable instructions in the memory 920 to perform any operation step of the above method and any optional embodiment thereof.

[0099] It should be understood that the computing device 900 according to the embodiments of the present application may correspond to the corresponding subject executing the methods according to the embodiments of the present application, and the above and other operations and / or functions of each module in the computing device 900 respectively implement the corresponding processes of the methods in the present embodiments. For the sake of brevity, they will not be described in detail here.

[0100] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A professional technician can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present application.

[0101] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above may refer to the corresponding processes in the foregoing method embodiments and will not be described in detail here.

[0102] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0103] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0104] In addition, in each embodiment of the present application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0105] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0106] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it is used to execute the above method, and the method includes at least one of the solutions described in each of the above embodiments.

[0107] The computer storage medium of the embodiments of the present application may adopt any combination of one or more computer-readable media. The computer-readable media may be computer-readable signal media or computer-readable storage media. The computer-readable storage media may be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, the computer-readable storage media may be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or component.

[0108] The computer-readable signal media may include data signals propagated in a baseband or as part of a carrier wave, which carry computer-readable program codes. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal media may also be any computer-readable media other than the computer-readable storage media, and this computer-readable media can send, propagate, or transmit a program for use by or in combination with an instruction execution system, device, or component.

[0109] The program codes contained on the computer-readable media may be transmitted by any appropriate media, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0110] The computer program codes for performing the operations of the present application may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program codes may be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0111] In addition, the terms "first", "second", "third", etc. or similar terms such as Module A, Module B, Module C, etc. in the specification and claims are only used to distinguish similar objects and do not represent a specific order for the objects. Understandably, the specific order or sequence can be interchanged when permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0112] In the above description, the reference numerals representing steps, such as S110, S120, etc., do not necessarily mean that the steps will be executed in this order. The order of the front and back steps can be interchanged when permitted, or they can be executed simultaneously.

[0113] The term "comprising" used in the specification and claims should not be construed as limited to the content listed thereafter; it does not exclude other elements or steps. Therefore, it should be interpreted as specifying the presence of the recited features, wholes, steps, or components, but does not exclude the presence or addition of one or more other features, wholes, steps, or components and their groups. Thus, the expression "a device comprising device A and B" should not be limited to a device consisting only of components A and B.

[0114] The "one embodiment" or "embodiment" mentioned in this specification means that the specific features, structures, or characteristics described in connection with that embodiment are included in at least one embodiment of the present application. Therefore, the phrases "in one embodiment" or "in an embodiment" that appear throughout this specification do not necessarily all refer to the same embodiment, but may refer to the same embodiment. In addition, in one or more embodiments, the various specific features, structures, or characteristics can be combined in any suitable manner, as will be apparent to those of ordinary skill in the art from this disclosure.

[0115] Note that the above is only the preferred embodiment of the present application and the technical principles applied. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and various obvious changes, re-adjustments, and substitutions can be made by those skilled in the art without departing from the protection scope of the present application. Therefore, although the present application has been described in detail through the above embodiments, the present application is not limited to the above embodiments only. Without departing from the concept of the present application, more other equivalent embodiments can be included, all of which fall within the protection scope of the present application.

Claims

1. A bandwidth adaptive data transmission method, characterized in that: include: The sending end sends first media data to the receiving end according to a preset period, and receives second media data fed back by the receiving end; wherein the first media data is layered media data with known attribute information, the first media data at least includes basic layer data, and the first media data of different periods includes different numbers of data layers; The sending end obtains the attribute information of the second media data and the evaluation index of the bandwidth, and determines the current available bandwidth of the network according to the attribute information of the second media data and the evaluation index of the bandwidth; The sending end sends the layered media data of the target number of layers according to the current available bandwidth of the network.

2. The method according to claim 1, characterized in that: The sending end sends the first media data to the receiving end according to a preset period, including: In two adjacent cycles, the number of data layers of the first media data sent in the current cycle is greater than the number of data layers of the first media data sent in the previous cycle.

3. The method according to claim 1, characterized in that The transmitting end obtains the attribute information of the second media data and the evaluation index of the bandwidth, and determines the current available bandwidth of the network according to the attribute information of the second media data and the evaluation index of the bandwidth, including: When the bandwidth evaluation index includes a data round trip time, the transmitting end determines the data round trip time according to the attribute information of the second media data, and determines the current available bandwidth of the network according to the data round trip time; When the bandwidth evaluation index includes a data packet loss rate, the transmitting end determines the data packet loss rate according to the attribute information of the second media data, and determines the current available bandwidth of the network according to the data packet loss rate; and / or When the bandwidth evaluation index includes a data jitter state, the transmitting end determines the data jitter state according to the attribute information of the second media data, and determines the current available bandwidth of the network according to the data jitter state.

4. The method according to claim 3, characterized in that When the bandwidth evaluation index includes a data round trip time, the transmitting end determines the data round trip time according to the attribute information of the second media data, and determines the current available bandwidth of the network according to the data round trip time, including: Acquire the sending time of the first media data and the receiving time of the second media data; Determine the data round trip time according to the sending time of the first media data and the receiving time of the second media data; When the data round trip time exceeds a first threshold, the bandwidth corresponding to the first media data in a previous cycle adjacent to the current cycle is used as the current available bandwidth of the network.

5. The method according to claim 3, characterized in that: When the bandwidth evaluation index includes a data packet loss rate, the transmitting end determines the data packet loss rate according to the attribute information of the second media data, and determines the current available bandwidth of the network according to the data packet loss rate, including: Acquire the number of data packets of the first media data and the number of data packets of the second media data; Determining the data packet loss rate based on the number of data packets of the first media data and the number of data packets of the second media data; When the data packet loss rate exceeds a second threshold, the bandwidth corresponding to the first media data in a previous cycle adjacent to the current cycle is used as the current available bandwidth of the network.

6. The method according to claim 3, characterized in that When the bandwidth evaluation index includes a data jitter state, the transmitting end determines the data jitter state according to the attribute information of the second media data, and determines the current available bandwidth of the network according to the data jitter state, including: Obtaining a jitter value of the second media data in this cycle and a jitter value of the second media data in the previous cycle; Determine a difference between the jitter value of the second media data in the current cycle and the jitter value of the second media data in the previous cycle; When the difference exceeds a third threshold, the bandwidth corresponding to the first media data in a previous cycle adjacent to the current cycle is used as the current available bandwidth of the network.

7. The method according to claim 1, characterized in that The sending end sends the target number of layered media data according to the current available bandwidth of the network, including: The sending end obtains the target data layer of the layered media data according to the current available bandwidth of the network; The sending end sends the target data layer of the layered media data.

8. The method according to claim 7, characterized in that Also includes: Performing layered processing on the media data to divide the media data into multiple data layers, the multiple data layers comprising a base layer and multiple enhancement layers, the multiple data layers having different spatial parameters, temporal parameters, and / or quality parameters; The multiple data layers are respectively encapsulated independently and identification information is added to obtain multiple encapsulated data layers for sending by the sending end, wherein the identification information is used to indicate the type of each of the multiple data layers.

9. The method according to claim 8, characterized in that Also includes: The priorities of the multiple data layers are set respectively, and the data layers with high priorities are sent first; wherein the priority of the base layer is set to the highest priority.

10. A bandwidth adaptive data transmission device, characterized in that: include: A transmission module provided at a transmitting end is used to send first media data to a receiving end according to a preset period, and receive second media data fed back by the receiving end; wherein the first media data is layered media data with known attribute information, the first media data at least includes basic layer data, and the first media data of different periods includes different numbers of data layers; A determination module provided at the transmitting end, configured to obtain the attribute information of the second media data and the evaluation index of the bandwidth, and determine the current available bandwidth of the network according to the attribute information of the second media data and the evaluation index of the bandwidth; The sending module arranged at the sending end is used to send the layered media data of the target number of layers according to the current available bandwidth of the network.

Images