Method and apparatus for streaming media data transmission, and device and medium

Abstract

According to the embodiments of the present disclosure, provided are a method and apparatus for streaming media data transmission, and a device and a medium. The method comprises: at an intermediate device, receiving from a first device a request for a first data block of streaming media data, wherein the first data block is associated with a first frame of the streaming media data at the first device, and the first data block comprises at least a media presentation description (MPD) associated with the streaming media data, and media data associated with the first frame; in response to it being determined that the first data block is unavailable at the intermediate device, acquiring the first data block on the basis of a comparison between a first time and a second time, wherein the first time is the time required to acquire the first data block on the basis of a first operation associated with a second device, and the second time is the time required to acquire the first data block on the basis of a second operation different from the first operation; and further, sending the first data block to the first device. In this way, the time to first frame can be effectively reduced.

Description

Method, device, equipment and medium for streaming media data transmission

[0001] This application claims priority to the Chinese invention patent application entitled “Methods, devices, equipment and media for streaming media data transmission” and application number 202410137652.3 filed on January 31, 2024, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] Embodiments of the present disclosure generally relate to the field of computers, and more particularly, to methods, devices, apparatuses, and media for streaming media data transmission. Background Art

[0003] In recent years, mobile terminals such as mobile phones and tablet computers have become important tools for people's daily lives, study and work. People can use mobile terminals to watch movies, live broadcasts, hold remote meetings, and so on. These functions can be achieved with the help of streaming media data transmission. The Dynamic Adaptive Streaming over Hypertext Transfer Protocol (DASH) protocol is a common media data distribution protocol for streaming media. As the DASH protocol is increasingly widely used for streaming media data transmission, how to improve the efficiency of streaming media data transmission implemented based on the DASH protocol has become an urgent problem to be solved. Summary of the Invention

[0004] In a first aspect of the present disclosure, a method for streaming media data transmission is provided. In the method, at an intermediate device, a request for a first data block of streaming media data is received from a first device. The first data block is associated with a first frame of the streaming media data at the first device, and the first data block includes at least a media presentation description MPD associated with the streaming media data, and media data associated with the first frame. In response to determining that the first data block is not available at the intermediate device, the first data block is obtained based on a comparison of a first time and a second time. The first time is a time required to obtain the first data block based on a first operation associated with the second device, and the second time is a time required to obtain the first data block based on a second operation different from the first operation. Further, the first data block is sent to the first device.

[0005] In a second aspect of the present disclosure, a device for streaming media data transmission is provided. The device includes a receiving module, an acquisition module and a sending module. The receiving module is configured to: at an intermediate device, receive a request for a first data block of streaming media data from a first device, the first data block being associated with the first frame of the streaming media data at the first device, and the first data block at least including a media presentation description MPD associated with the streaming media data, and media data associated with the first frame. The acquisition module is configured to: in response to determining that the first data block is unavailable at the intermediate device, acquire the first data block based on a comparison of a first time and a second time, the first time being the time required to acquire the first data block based on a first operation associated with the second device, and the second time being the time required to acquire the first data block based on a second operation different from the first operation. The sending module is configured to: send the first data block to the first device.

[0006] In a third aspect of the present disclosure, an electronic device is provided. The electronic device includes: at least one processing unit; and at least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to perform the method according to the first aspect of the present disclosure.

[0007] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which instructions are stored. When the instructions are executed by a processor, the processor is caused to implement the method according to the first aspect of the present disclosure.

[0008] It should be understood that the content described in this summary section is not intended to limit the key features or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent hereinafter with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0010] FIG1 is a schematic diagram illustrating an example environment in which various embodiments of the present disclosure can be implemented;

[0011] FIG2 shows a signaling diagram for streaming media data transmission according to some embodiments of the present disclosure;

[0012] FIG3 shows a flowchart of a method for streaming media data transmission according to some embodiments of the present disclosure;

[0013] FIG4 shows a block diagram of an example apparatus for streaming media data transmission according to some embodiments of the present disclosure; and

[0014] FIG5 illustrates a block diagram of a device in which one or more embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION

[0015] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0016] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". The following may also include other explicit and implicit definitions. As used herein, the term "model" can represent the association relationship between various data. For example, the above-mentioned association relationship can be obtained based on a variety of technical solutions currently known and / or to be developed in the future.

[0017] As used herein, the term "in response to" refers to a state in which a corresponding event occurs or a condition is satisfied. It will be understood that the timing of executing a subsequent action executed in response to the event or condition is not necessarily strongly correlated with the time when the event occurs or the condition is satisfied. For example, in some cases, the subsequent action may be executed immediately when the event occurs or the condition is satisfied; in other cases, the subsequent action may be executed some time after the event occurs or the condition is satisfied.

[0018] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

[0019] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0020] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0021] As an optional but non-limiting embodiment, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also include a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0022] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the embodiments of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the embodiments of the present disclosure.

[0023] As briefly mentioned above, with the diversification of online entertainment, streaming media data transmission based on the DASH protocol is becoming increasingly widespread. A key metric for the streaming viewing experience is time to first frame, which is the time it takes for the first frame of the video to be displayed after the user clicks "Start Play."

[0024] In one existing solution, if a user initiates playback on the client, the client first needs to request a Media Presentation Description (MPD) file from the server. After receiving and parsing the MPD file, the client can request an initialization segment for audio and an initialization segment for video from the server, and further request a media segment for audio and a media segment for video. In this solution, it takes at least three Hypertext Transfer Protocol (HTTP) requests and responses from the time the user initiates viewing on the client to the time the first frame is presented on the client. This results in a longer first frame time and a poorer viewing experience for users.

[0025] Another existing solution combines the MPD file, initialization segment, and media segments into a single tuning-in segment, thus reducing the time required for two requests. However, the inventors discovered through research that, because the tuning-in segment is time-sensitive, it should not be cached for too long in the Content Delivery Network (CDN) device. Consequently, when a user requests the file, the CDN device often needs to re-request the tuning-in segment from the server. This also increases the first frame time.

[0026] To this end, the embodiments of the present disclosure propose a scheme for obtaining a data block for starting broadcasting in a racing manner through two different operations. Specifically, according to the embodiments of the present disclosure, a scheme for streaming media data transmission is proposed. In this scheme, at an intermediate device, a request for a first data block of streaming media data is received from a first device. The first data block is associated with the first frame of the streaming media data at the first device, and the first data block includes at least a media presentation description MPD associated with the streaming media data, and media data associated with the first frame. In response to determining that the first data block is unavailable at the intermediate device, the first data block is obtained based on a comparison between a first time and a second time. The first time is the time required to obtain the first data block based on a first operation associated with the second device, and the second time is the time required to obtain the first data block based on a second operation different from the first operation. Further, the first data block is sent to the first device.

[0027] The following description will provide a clearer understanding of how, according to embodiments of the present disclosure, when a data block required for initiation is unavailable locally on an intermediate device, two different operations are used to obtain the data block in a competitive manner. This approach ensures timely acquisition of the data block required for initiation. Furthermore, the data block required for initiation can be obtained more efficiently, enabling faster responses to data block requests and effectively reducing the time to first frame.

[0028] Various example implementations of the solution will be described in detail below with reference to the accompanying drawings. First, refer to FIG1 , which illustrates a schematic diagram of an example environment 100 in which various embodiments of the present disclosure can be implemented. Example environment 100 may generally include a client 110 , a server 120 , a user 130 , and CDN devices 140 - 1 , 140 - 2 , ..., 140 -N (hereinafter individually or collectively referred to as CDN devices 140 ), where N is any suitable positive integer, such as 1, 5, 70, 500, and so on. Client 110 is communicatively coupled to CDN device 140 , and CDN device 140 is communicatively coupled to server 120 .

[0029] In FIG1 , client 110 is shown as a mobile phone, but client 110 may also be any type of mobile terminal or portable terminal, including a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio receiver, an e-book device, a gaming device, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, client 110 may also support any type of interface for user 130 (such as a "wearable" circuit, etc.).

[0030] As shown in Figure 1, user 130 can, for example, initiate viewing of streaming content, such as live video, online movies, and the like, by operating an application on client 110. Client 110 can send a request for corresponding streaming data to CDN device 140, and then CDN device 140 can obtain corresponding start-up data locally or through communication with server 120 in a competitive manner and send it to client 110. Client 110 can then render and play the first frame of content based on the start-up data for user 130 to watch. This will be described in further detail below. In some embodiments, server 120 can be a device with computing and communication functions, such as a workstation, cloud server, etc. It should be understood that the structure and function of environment 100 are described for exemplary purposes only, and do not imply any limitation on the scope of the present disclosure.

[0031] Example signaling

[0032] FIG2 shows a signaling diagram 200 for streaming media data transmission according to some embodiments of the present disclosure. The signaling diagram 200 generally involves a first device 201, a second device 202, and an intermediate device 203. The first device 201 and the second device 202 are communicatively coupled via the intermediate device 203. In some embodiments, the first device 201 can be implemented as the client 110 in FIG1 , the second device 202 can be implemented as the server 120 in FIG1 , and the intermediate device 203 can be implemented as the CDN device 140 in FIG1 . It should be understood that the first device 201, the second device 202, and / or the intermediate device 203 can also be implemented as any other suitable device. For example, the first device 201 can also be implemented as another server. The scope of the present disclosure is not limited in this respect.

[0033] As shown in Figure 2, the first device 201 sends 205 a request for a first data block of streaming media data to the intermediate device 203. By way of example and not limitation, the streaming media data includes live broadcast data, online video data, and the like. The first data block is associated with the first frame of the streaming media data at the first device 201. For example, the first data block may include some or all of the data required to generate the first frame at the first device 201. In some embodiments, the streaming media data may be transmitted according to the DASH protocol. Exemplarily, the request for the first data block may be an HTTP request, such as a GET request. The request may, for example, include a Uniform Resource Locator (URL) for the first data block.

[0034] The first data block includes at least a media presentation description MPD associated with the streaming media data, and media data associated with the first frame. The MPD file (also referred to as MPD in this article) is used to describe information about the streaming media data, such as bit rate, resolution, characteristics of different media components (such as audio, video, text, etc.) included in the multimedia content, and so on. In one example, the media data may include audio data. In another example, the media data may include video data. In yet another example, the media data may include audio data and video data. It should be understood that the media data may also include any other suitable data, such as text data, etc., and the scope of the present disclosure is not limited in this respect.

[0035] In some embodiments, the data block may be a segment, and the first data block may also be referred to as a start segment. As used herein, a segment refers to the entity body of a response to an HTTP GET request or a portion of an HTTP GET request from the DASH client 110. It should be understood that the data block may also be implemented in any other suitable manner. The scope of the present disclosure is not limited in this respect.

[0036] In some embodiments, the first data block may further include initialization information associated with the decoded media data. Examples of initialization information include, but are not limited to, initialization setting information for the decoder at the first device 201, a picture parameter set (PPS), a sequence parameter set (SPS), and the like. In this way, after receiving the first data block, the first device 201 may generate and display the first frame based on the first data block, so that only one HTTP request is required to present the first frame, thereby further reducing the first frame time and improving the response speed to the start operation of the user 130.

[0037] Alternatively or additionally, the first data block can also include the start time of media data and / or the sequence number of the media data block corresponding to the media data. Exemplarily and non-restrictively, the audio data block can be the media fragmentation (also referred to as audio fragmentation) for audio content, the video data block can be the media fragmentation (also referred to as video fragmentation) for video content, and the audio and video data block can be the media fragmentation (also referred to as audio and video fragmentation) for audio content and video content. By transmitting the start time and / or sequence number, the first equipment 201 can directly determine the start time and / or sequence number of the data block of subsequent required request based on this information, thereby can more specifically initiate request. In this way, data can be requested more efficiently, and the response time at intermediate equipment 203 and / or second equipment 202 places can be shortened, thereby further reducing first frame time.

[0038] Correspondingly, the intermediate device 203 receives 210 a request for a first data block from the first device 201. In response to receiving the request, the intermediate device 203 may determine whether the first data block is available at the intermediate device 203. It should be noted that, in the context of the present disclosure, the first data block being unavailable at the intermediate device 203 includes the case where the first data block does not exist at the intermediate device 203. By way of example and not limitation, if the first data block exists in the cache of the intermediate device 203 and the cache for the first data block has not expired, then the first data block may be determined to be available at the intermediate device 203. If the first data block exists in the cache of the intermediate device 203 but the cache for the first data block has expired, then the first data block may be determined to be unavailable at the intermediate device 203. If the first data block does not exist in the cache of the intermediate device 203, then the first data block may also be determined to be unavailable at the intermediate device 203.

[0039] If it is determined that the first data block is not available at the intermediate device 203, the intermediate device 203 may perform a first operation and a second operation for obtaining the first data block in parallel, and obtain the first data block based on a comparison of a first time and a second time. The first time is a time required to obtain the first data block based on a first operation associated with the second device 202, and the second time is a time required to obtain the first data block based on a second operation different from the first operation.

[0040] In some embodiments, the first operation may be obtaining a first data block from the second device 202. For example, the first operation may include sending 215 a request for the first data block to the second device 202, as shown in Figure 2. In response to receiving 220 the request from the intermediate device 203, the second device 202 may check whether the first data block is available locally on the second device 202. If it is determined that the first data block is available at the second device 202, the second device 202 may directly read the stored first data block. If it is determined that the first data block is not available at the second device 202, the second device 202 may generate the first data block. Exemplarily, the second device 202 may encapsulate the current MPD in the first data block. Additionally, the second device 202 may select a target media data block including media data associated with the first frame from at least one media data block (e.g., a media fragment) encapsulated for streaming media data, and encapsulate the content of the target media data block in the first data block.

[0041] In some embodiments, the second device 202 can determine the currently available latest media data block as the target media data block. In this way, the delay existing when the client 110 plays can be avoided as much as possible, so that the user 130 can obtain the real-time latest media content in time.

[0042] Alternatively, the target media data block can be selected based on the first frame delay. This delay can be predetermined or sent by the first device 201 to the second device 202 via the intermediate device 203. For example, the first device 201 can send its desired first frame delay along with the request for the first data block. It should be understood that the first frame delay can also be determined in any other suitable manner, and the scope of the present disclosure is not limited in this respect.

[0043] As an example, the second device 202 can determine the target time based on the current time and the delay time. Exemplarily, the second device 202 can directly subtract the delay time (e.g., 5 seconds) from the current time (e.g., 01:45:20) to obtain the target time (e.g., 01:45:15). Further, the second device 202 can select the target media data block from the packaged media data blocks based on the target time. For example, the second device 202 can determine the media data block including the media data of the target time as the target media data block. In this way, a buffer can be established between the content played by the client 110 and the real-time content to avoid problems such as playback jams and other problems caused by problems such as network congestion and network speed drop, thereby ensuring the viewing experience of the user 130. It should be understood that the second device 202 can also select the target media data block in any other suitable manner, and the scope of the present disclosure is not limited in this respect.

[0044] Furthermore, the second device 202 may send 230 the first data block it has obtained to the intermediate device 203. Accordingly, the intermediate device 203 receives the first data block it requested from the second device 202. For example, the time elapsed from the intermediate device 203 sending 215 the request for the first data block to the intermediate device 203 receiving 235 the first data block may be considered the first time. It should be understood that the first operation may also include any other suitable operation of obtaining the first data block from the second device 202, and the scope of the present disclosure is not limited in this respect.

[0045] The inventors have discovered through research that in cloud computing or edge computing scenarios such as CDNs, while other users are watching streaming content, the streaming data at the intermediate device 203 is continuously updated. Even if the first data block requested by the current user 130 is not available as a whole at the intermediate device 203, some or all of the data included in the first data block may still be available at the intermediate device 203. Therefore, the response time can be further shortened by attempting to generate the first data block at the intermediate device 203.

[0046] In this regard, in some embodiments, the second operation includes generating 225 the first data block at the intermediate device 203. As an example, the intermediate device 203 can obtain an MPD. To obtain the MPD, the intermediate device 203 can first determine whether the MPD is available at the intermediate device 203. By way of example and not limitation, if the MPD exists in the cache of the intermediate device 203 and the cache of the MPD has not expired, it can be determined that the MPD is available at the intermediate device 203. If the MPD exists in the cache of the intermediate device 203 but the cache of the MPD has expired, it can be determined that the MPD is not available at the intermediate device 203. If the MPD does not exist in the cache of the intermediate device 203, it can also be determined that the MPD is not available at the intermediate device 203.

[0047] If the MPD is determined to be available at the intermediate device 203, the intermediate device 203 may directly read the MPD for subsequent generation of the first data block. If the MPD is determined to be unavailable at the intermediate device 203, the intermediate device 203 may obtain the MPD from the second device 202. For example, the intermediate device 203 may send a request for the MPD to the second device 202. In response to receiving the request for the MPD, the second device 202 may send the MPD to the intermediate device 203 as a response. Correspondingly, the intermediate device 203 may receive the MPD from the second device 202 for subsequent generation of the first data block.

[0048] Furthermore, the intermediate device 203 may select a target media data block including media data associated with the first frame from at least one media data block encapsulated for the streaming media data based on the MPD. For example, the intermediate device 203 may parse the MPD and select an appropriate media data block according to a configuration.

[0049] In some embodiments, the second device 202 may determine the latest media data block currently available indicated in the MPD as the target media data block. In this way, the delay that exists when the client 110 plays can be avoided as much as possible, so that the user 130 can obtain the latest real-time media content in a timely manner. Alternatively, the target media data block can be selected based on the delay time of the first frame. The delay time can be predetermined, or can be sent to the intermediate device 203 by the first device 201. For example, the first device 201 can send its desired first frame delay time along with the request for the first data block. It should be understood that the first frame delay time can also be determined in any other suitable manner, and the scope of the present disclosure is not limited in this respect.

[0050] As an example, the intermediate device 203 can determine the target time based on the current time and the delay time. Exemplarily, the intermediate device 203 can directly subtract the delay time (e.g., 5 seconds) from the current time (e.g., 01:45:20) to obtain the target time (e.g., 01:45:15). Further, the intermediate device 203 can select the target media data block from the packaged media data blocks based on the target time. For example, the intermediate device 203 can determine the media data block including the media data of the target time as the target media data block. In this way, a buffer can be established between the client 110 playback content and the real-time content to avoid problems such as playback jams and network speed drops, thereby ensuring the viewing experience of the user 130. It should be understood that the intermediate device 203 can also select the target media data block in any other suitable manner, and the scope of the present disclosure is not limited in this respect.

[0051] Because the MPD only contains identification information (e.g., sequence number, start time, etc.) of each media data block and does not contain the media data block itself, after selecting the target media data block, the intermediate device 203 needs to obtain the target media data block. To obtain the target media data block, the intermediate device 203 can first determine whether the target media data block is available at the intermediate device 203. By way of example and not limitation, if the target media data block exists in the cache of the intermediate device 203 and the cache for the target media data block has not expired, it can be determined that the target media data block is available at the intermediate device 203. If the target media data block exists in the cache of the intermediate device 203 but the cache for the target media data block has expired, it can be determined that the target media data block is not available at the intermediate device 203. If the target media data block does not exist in the cache of the intermediate device 203, it can also be determined that the target media data block is not available at the intermediate device 203.

[0052] If it is determined that the target media data block is available at the intermediate device 203, the intermediate device 203 can directly read the target media data block for use in subsequently generating the first data block. If it is determined that the target media data block is not available at the intermediate device 203, the intermediate device 203 can obtain the target media data block from the second device 202. For example, the intermediate device 203 can send a request for the target media data block to the second device 202. In response to receiving the request for the target media data block, the second device 202 can send the target media data block to the intermediate device 203 as a response. Correspondingly, the intermediate device 203 can receive the target media data block from the second device 202 for use in subsequently generating the first data block.

[0053] In the case where the first data block also includes other data (for example, initialization information associated with the decoded media data, etc.), the intermediate device 203 can obtain these data in a manner similar to the method described above with reference to obtaining the target media data block for subsequent generation of the first data block, and the present disclosure will not go into details here.

[0054] Furthermore, the intermediate device 203 may generate a first data block based on the MPD and the target media data block. Exemplarily, the intermediate device 203 may encapsulate the obtained MPD into the first data block. Furthermore, the intermediate device 203 may encapsulate the contents of the obtained media data block into the first data block. In the case where the media data includes audio data and video data, the audio data and video data may be interleaved and encapsulated into the first data block based on a preset sorting criterion. Exemplarily, after writing the video data of the first frame, the audio data of the first frame may be written. Then, the video data and audio data of another frame immediately following the first frame based on the sorting criterion may be written, and so on. In one embodiment, the preset sorting criterion may be decoding timestamp order. In other words, the audio data and video data may be encapsulated into the first data block based on decoding timestamp order. For example, after writing the video data of the first frame, the audio data of the first frame may be written. Then, the video data and audio data of another frame immediately following the first frame based on the decoding timestamp order may be written, and so on. This approach effectively avoids the problem in existing solutions where the video content of all frames in the first data block must be decoded before the audio content of the first frame can be decoded. This allows for more efficient decoding of the media content in the first data block at client 110, further reducing the first frame time and improving the response speed to the start of broadcast operations by user 130.

[0055] In some embodiments, the first data block may be encapsulated based on the MP4 format, thereby more efficiently encapsulating data and achieving compatibility with the DASH protocol. It should be understood that the first data block may also be encapsulated based on any other suitable file format, and the scope of the present disclosure is not limited in this respect.

[0056] The above describes an example process in which intermediate device 203 locally generates the first data block. In this way, intermediate device 203 can reuse locally available information as much as possible, thereby improving the cache hit rate of intermediate device 203 and providing the possibility for intermediate device 203 to obtain the first data block more quickly. Exemplarily, the time required for intermediate device 203 to generate the first data block can be used as the second time. It should be understood that intermediate device 203 can also generate the first data block locally using any other suitable method, and the scope of this disclosure is not limited in this respect.

[0057] In some embodiments, in response to determining the shorter of the first time and the second time, the intermediate device 203 may send 240 the first data block obtained through the operation corresponding to the shorter time to the first device 201. For example, if the first time is shorter than the second time, the first data block sent to the first device 201 is obtained based on the first operation. If the second time is shorter than the first time, the first data block sent to the first device 201 is obtained based on the second operation.

[0058] For example, in response to obtaining a first data block (whether obtained through the first operation or the second operation), the intermediate device 203 can send 240 the first data block to the first device 201. In other words, the intermediate device 203 does not need to wait until both the first operation and the second operation are completed before comparing the first time with the second time. Instead, the intermediate device 203 can compare the first time with the second time in real time. Once one of the first operation and the second operation is completed first, the intermediate device 203 can determine that the time corresponding to the operation is shorter, and then immediately send 240 the first data block obtained by the operation to the first device 201. If the first time is the same as the second time, the intermediate device 203 can send either of the two first data blocks obtained through the two operations.

[0059] In this regard, it should be noted that the actions 215, 220, 225, 230, and 235 shown in the dotted box 250 of Figure 2, which are performed when the first data block is unavailable at the intermediate device 203, do not indicate the order and execution time of these actions, and in some embodiments, some actions may be omitted. For example, the sending action 215 and the generating action 225 may be performed simultaneously. For another example, when the intermediate device 203 has already generated the first data block locally, the receiving action 235 may be omitted. The scope of the present disclosure is not limited in this respect.

[0060] As an example, if the intermediate device 203 has not yet generated the first data block locally when the intermediate device 203 receives the first data block from the second device 202, it can be determined that the first time is shorter than the second time, and the intermediate device 203 can send 240 the first data block from the second device 202 to the first device 201. Additionally, the intermediate device 203 can terminate subsequent operations of locally generating the first data block. If the intermediate device 203 has not received the first data block from the second device 202 when the operation of locally generating the first data block is completed, it can be determined that the second time is shorter than the first time, and the intermediate device 203 can send 240 the generated first data block to the first device 201. Additionally, the intermediate device 203 may no longer receive the first data block from the second device 202.

[0061] In this way, the intermediate device 203 can more efficiently obtain the data blocks for starting broadcasting and provide the data blocks to the first device 201 as quickly as possible. In this way, the intermediate device 203 can respond to requests for data blocks more quickly, thereby effectively reducing the first frame time.

[0062] In some embodiments, if it is determined that the first data block is available at the intermediate device 203 , the intermediate device 203 may read the first data block and send 240 it to the first device 201 in response.

[0063] By way of example and not limitation, the second device 202 may send the entire first data block in one HTTP response. When the media data includes audio data and video data, in this manner, the audio data and video data for the first frame may be sent in the same data block, thereby effectively preventing misalignment of audio and video data.

[0064] After receiving 245 the first data block from the intermediate device 203, the first device 201 may display the first frame based on the first data block. For example, the first device 201 may parse the first data block and perform decoding operations on the audio stream and / or video stream, and then render and present the first frame based on the obtained data. Furthermore, the first device 201 may continue to initiate a request for subsequent audio and video content to the second device 202 to continue playing the subsequent content of the streaming media.

[0065] As can be seen from the above description in conjunction with Figures 1 and 2, in the method for transmitting streaming media data according to various embodiments of the present disclosure, when the data block used for starting broadcasting is not available locally on the intermediate device, the data block is obtained in a competitive manner using two different operations. In this way, on the one hand, the data block used for starting broadcasting can be ensured to be obtained in a timely manner. On the other hand, the data block used for starting broadcasting can be obtained more efficiently, thereby responding to requests for data blocks more quickly, and thus effectively reducing the first frame time.

[0066] Example Method

[0067] FIG3 illustrates a flow chart of a method 300 for transmitting streaming media data, according to some embodiments of the present disclosure. In some embodiments, the method 300 may be performed at the CDN device 140 shown in FIG1 . It should be understood that the method 300 may include additional blocks not shown and / or may omit one (or more) of the blocks shown, and the scope of the present disclosure is not limited in this respect.

[0068] In box 302, at an intermediate device, a request for a first data block of streaming media data is received from a first device, the first data block is associated with a first frame of the streaming media data at the first device, and the first data block includes at least a media presentation description MPD associated with the streaming media data, and media data associated with the first frame.

[0069] In block 304, in response to determining that the first data block is not available at the intermediate device, the first data block is obtained based on a comparison of a first time and a second time, the first time being a time required to obtain the first data block based on a first operation associated with the second device, and the second time being a time required to obtain the first data block based on a second operation different from the first operation.

[0070] At block 306 , a first data block is sent to a first device.

[0071] In some embodiments, the first operation includes sending a request for the first data block to the second device.

[0072] In some embodiments, the second operation includes generating the first data block at the intermediate device.

[0073] In some embodiments, generating a first data block at an intermediate device includes: obtaining an MPD; based on the MPD, selecting a target media data block from at least one media data block encapsulated for streaming media data, the target media data block including media data associated with the first frame; obtaining the target media data block; and generating the first data block based on the MPD and the target media data block.

[0074] In some embodiments, obtaining the MPD includes: determining whether the MPD is available at the intermediate device; in response to determining that the MPD is available at the intermediate device, reading the MPD; and in response to determining that the MPD is not available at the intermediate device, performing the following operations: sending a request for the MPD to a second device; and receiving the MPD from the second device.

[0075] In some embodiments, obtaining the target media data block includes: determining whether the target media data block is available at the intermediate device; in response to determining that the target media data block is available at the intermediate device, reading the target media data block; and in response to determining that the target media data block is not available at the intermediate device, performing the following operations: sending a request for the target media data block to the second device; and receiving the target media data block from the second device.

[0076] In some embodiments, the media data includes audio data and video data.

[0077] In some embodiments, the audio data and the video data are encapsulated in the first data block in an order based on a decoding timestamp.

[0078] In some embodiments, if the first time is shorter than the second time, the first data block sent to the first device is obtained based on the first operation, or if the second time is shorter than the first time, the first data block sent to the first device is obtained based on the second operation.

[0079] In some embodiments, the first data block further includes at least one of the following: initialization information associated with decoding the media data, a start time of the media data, and a sequence number of a media data block corresponding to the media data.

[0080] In some embodiments, the streaming media data includes live data.

[0081] In some embodiments, the streaming media data is transmitted according to a Dynamic Adaptive Streaming over Hypertext Transfer Protocol (DASH) protocol, and the first data block is a segment and is encapsulated based on the MP4 format.

[0082] In some embodiments, the first device comprises a client, the second device comprises a server, and the intermediate device comprises a content delivery network (CDN) device.

[0083] Example devices and equipment

[0084] Embodiments of the present disclosure also provide corresponding apparatuses and devices for implementing the above-described methods or processes. FIG4 illustrates a block diagram of an example apparatus 400 for transmitting streaming media data according to some embodiments of the present disclosure. This apparatus 400 can, for example, be used to implement the methods according to some embodiments of the present disclosure. In some embodiments, apparatus 400 can be implemented at CDN device 140, as shown in FIG1 .

[0085] As shown in Figure 4, the apparatus 400 may include a receiving module 402, an acquisition module 404, and a sending module 406. The receiving module 402 is configured to: at an intermediate device, receive a request for a first data block of streaming media data from a first device, the first data block being associated with a first frame of the streaming media data at the first device, and the first data block comprising at least a media presentation description MPD associated with the streaming media data, and media data associated with the first frame. The acquisition module 404 is configured to: in response to determining that the first data block is unavailable at the intermediate device, acquire the first data block based on a comparison of a first time and a second time, the first time being a time required to acquire the first data block based on a first operation associated with the second device, and the second time being a time required to acquire the first data block based on a second operation different from the first operation. The sending module 406 is configured to: send the first data block to the first device.

[0086] In some embodiments, the first operation includes sending a request for the first data block to the second device.

[0087] In some embodiments, the second operation includes generating the first data block at the intermediate device.

[0088] In some embodiments, generating the first data block at the intermediate device includes: obtaining an MPD;

[0089] Based on the MPD, a target media data block is selected from at least one media data block encapsulated for streaming media data, wherein the target media data block includes media data associated with the first frame; the target media data block is obtained; and a first data block is generated based on the MPD and the target media data block.

[0090] In some embodiments, obtaining the MPD includes: determining whether the MPD is available at the intermediate device; in response to determining that the MPD is available at the intermediate device, reading the MPD; and in response to determining that the MPD is not available at the intermediate device, performing the following operations: sending a request for the MPD to a second device; and receiving the MPD from the second device.

[0091] In some embodiments, obtaining the target media data block includes: determining whether the target media data block is available at the intermediate device; in response to determining that the target media data block is available at the intermediate device, reading the target media data block; and in response to determining that the target media data block is not available at the intermediate device, performing the following operations: sending a request for the target media data block to the second device; and receiving the target media data block from the second device.

[0092] In some embodiments, the media data includes audio data and video data.

[0093] In some embodiments, the audio data and the video data are encapsulated in the first data block in an order based on a decoding timestamp.

[0094] In some embodiments, if the first time is shorter than the second time, the first data block sent to the first device is obtained based on the first operation, or if the second time is shorter than the first time, the first data block sent to the first device is obtained based on the second operation.

[0095] In some embodiments, the first data block further includes at least one of the following: initialization information associated with decoding the media data, a start time of the media data, and a sequence number of a media data block corresponding to the media data.

[0096] In some embodiments, the streaming media data includes live data.

[0097] In some embodiments, the streaming media data is transmitted according to a Dynamic Adaptive Streaming over Hypertext Transfer Protocol (DASH) protocol, and the first data block is a segment and is encapsulated based on the MP4 format.

[0098] In some embodiments, the first device comprises a client, the second device comprises a server, and the intermediate device comprises a content delivery network (CDN) device.

[0099] The modules and / or units included in the device 400 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units can be implemented using software and / or firmware, such as machine executable instructions stored on a storage medium. In addition to or as an alternative to machine executable instructions, some or all of the units in the device 400 can be implemented at least in part by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0100] The modules and / or units shown in Figure 4 may be partially or entirely implemented as hardware modules, software modules, firmware modules, or any combination thereof. In particular, in some embodiments, the processes, methods, or procedures described above may be implemented by hardware in a storage system, a host corresponding to the storage system, or other computing devices independent of the storage system.

[0101] FIG5 shows a block diagram of a device 500 in which one or more embodiments of the present disclosure may be implemented. It should be understood that the electronic device 500 shown in FIG5 is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. The electronic device 500 shown in FIG5 can be used to implement the CDN device 140 shown in FIG1 and / or the methods described above.

[0102] As shown in FIG5 , electronic device 500 is a general-purpose electronic device. Components of electronic device 500 may include, but are not limited to, one or more processors or processing units 510, memory 520, storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. Processing unit 510 may be a real or virtual processor and is capable of performing various processes according to programs stored in memory 520. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to enhance the parallel processing capabilities of electronic device 500.

[0103] The electronic device 500 typically includes a plurality of computer storage media. Such media can be any available media accessible to the electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 520 can be a volatile memory (e.g., registers, cache, random access memory (RAM)), a non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 530 can be a removable or non-removable medium and can include a machine-readable medium, such as a flash drive, a disk, or any other medium that can be used to store information and / or data (e.g., training data for training) and can be accessed within the electronic device 500.

[0104] The electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG. 5 , a disk drive for reading from or writing to a removable, non-volatile disk (e.g., a “floppy disk”) and an optical drive for reading from or writing to a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) by one or more data media interfaces. The memory 520 may include a computer program product 525 having one or more program modules configured to perform various methods or actions of various implementations of the present disclosure.

[0105] The communication unit 540 enables communication with other electronic devices via a communication medium. Additionally, the functions of the components of the electronic device 500 can be implemented in a single computing cluster or multiple computing machines that can communicate via a communication connection. Thus, the electronic device 500 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0106] Input device 550 may be one or more input devices, such as a mouse, keyboard, or trackball. Output device 560 may be one or more output devices, such as a display, a speaker, or a printer. Electronic device 500 may also communicate with one or more external devices (not shown) via communication unit 540 as needed, such as a storage device, a display device, or the like, with one or more devices that allow a user to interact with electronic device 500, or with any device that allows electronic device 500 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0107] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is provided, on which a computer program is stored, and when the program is executed by a processor, the method described above is implemented.

[0108] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0109] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0110] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0111] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple implementations of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part for a module, program segment or instruction, and a part for a module, program segment or instruction comprises one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0112] While various implementations of the present disclosure have been described above, the foregoing description is intended to be illustrative, not exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is selected to best explain the principles of the implementations, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for transmitting streaming media data, comprising: Receiving, at an intermediate device, a request for a first data block of streaming media data from a first device, the first data block being associated with a first frame of the streaming media data at the first device, and the first data block comprising at least a media presentation description (MPD) associated with the streaming media data and media data associated with the first frame; In response to determining that the first data block is not available at the intermediate device, obtaining the first data block based on a comparison of a first time and a second time, the first time being a time required to obtain the first data block based on a first operation associated with a second device, and the second time being a time required to obtain the first data block based on a second operation different from the first operation; as well as Send the first data block to the first device.

2. The method according to claim 1, wherein the first operation comprises: A request for the first data block is sent to the second device.

3. The method according to claim 1, wherein the second operation comprises: The first data block is generated at the intermediate device.

4. The method of claim 3, wherein generating the first data block at the intermediate device comprises: Obtain the MPD; Selecting, based on the MPD, a target media data block from at least one media data block encapsulated for the streaming media data, the target media data block including the media data associated with the first frame; Acquire the target media data block; as well as The first data block is generated based on the MPD and the target media data block.

5. The method according to claim 4, wherein obtaining the MPD comprises: determining whether the MPD is available at the intermediate device; In response to determining that the MPD is available at the intermediate device, reading the MPD; as well as In response to determining that the MPD is not available at the intermediate device, performing the following operations: sending a request for the MPD to the second device; and The MPD is received from the second device.

6. The method according to claim 4, wherein obtaining the target media data block comprises: determining whether the target media data block is available at the intermediate device; In response to determining that the target media data block is available at the intermediate device, reading the target media data block; as well as In response to determining that the target media data block is not available at the intermediate device, performing the following operations: sending a request for the target media data block to the second device; and The target media data block is received from the second device. The method of claim 1 , wherein the media data comprises audio data and video data. 8 . The method according to claim 7 , wherein the audio data and the video data are encapsulated in the first data block in a sequence based on a decoding timestamp.

9. The method according to claim 1, wherein if the first time is shorter than the second time, the first data block sent to the first device is obtained based on the first operation, or If the second time is shorter than the first time, the first data block sent to the first device is obtained based on the second operation.

10. The method according to claim 1, wherein the first data block further comprises at least one of the following: initialization information associated with decoding said media data, The start time of the media data, The sequence number of the media data block corresponding to the media data. The method according to claim 1 , wherein the streaming media data comprises live data. 12 . The method according to claim 1 , wherein the streaming media data is transmitted according to a Dynamic Adaptive Streaming over Hypertext Transfer Protocol (DASH) protocol, and the first data block is a segment and is encapsulated based on an MP4 format. 13 . The method according to claim 1 , wherein the first device comprises a client, the second device comprises a server, and the intermediate device comprises a content delivery network (CDN) device.

14. A device for transmitting streaming media data, comprising: a receiving module configured to: receive, at an intermediate device, a request for a first data block of streaming media data from a first device, where the first data block is associated with a first frame of the streaming media data at the first device, and the first data block includes at least a media presentation description (MPD) associated with the streaming media data and media data associated with the first frame; an acquisition module configured to: in response to determining that the first data block is not available at the intermediate device, acquire the first data block based on a comparison of a first time and a second time, the first time being a time required to acquire the first data block based on a first operation associated with a second device, and the second time being a time required to acquire the first data block based on a second operation different from the first operation; as well as The sending module is configured to: send the first data block to the first device.

15. An electronic device comprising: at least one processing unit; as well as At least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 13 when executed by the at least one processing unit. 16 . A computer-readable storage medium having instructions stored thereon, which, when executed by a processor, cause the processor to implement the method according to claim 1 .

17. A computer program product tangibly stored in a computer storage medium and comprising computer executable instructions which, when executed by a device, cause the device to perform the method according to any one of claims 1 to 13.

Images