Media stream playback, media stream processing method, device, equipment and storage medium

By establishing a transmission packet identification mapping relationship in the browser, the problem of lag during the switching of advertisements and main films is solved, the smoothness of video playback and parsing efficiency are improved, and the user experience is improved.

CN114554256BActive Publication Date: 2024-11-22ALIBABA GROUP HOLDING LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011339067.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-11-25
Publication Date
2024-11-22
Estimated Expiration
2040-11-25

AI Technical Summary

Technical Problem

When playing videos in the browser, switching between advertisements and the main video often leads to stuttering, affecting the user's viewing experience.

Method used

By determining whether the media type identification of the current transmission packet of the real-time media stream is the same as the identification of the previous transmission packet, establish a mapping relationship, parse and play the current transmission packet, avoid transmission packet resolution failures of different media resources, and improve playback fluency and parsing efficiency .

Benefits of technology

It realizes seamless switching of video playback, improves the playback fluency and parsing efficiency of media streams, and improves the user's viewing experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114554256B_ABST
    Figure CN114554256B_ABST
Patent Text Reader

Abstract

The embodiments of the present invention provide a media stream playback, media stream processing method, device, equipment and storage medium. The media stream playback method comprises: determining a current transmission packet of a real-time media stream; if the media type identifier of the current transmission packet is the same as the media type identifier of a previous transmission packet, parsing and playing the current transmission packet based on the transmission packet identifier of the previous transmission packet. The solution of the embodiments of the present invention improves the playback fluency of the media stream.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer technology, and in particular to a media stream playing and processing method, device, equipment and storage medium. Background Art

[0002] At present, there are many ways to embed advertisements in the browser's player. For example, a streaming media transmission (HTTP Live Streaming, HLS) protocol can be used to merge the advertisement video with the feature film for playback. HLS is a new generation streaming media transmission protocol, which fragments a large media file, records the fragment file resource path in an m3u8 index file (i.e., playlist) and provides it to the client. The client can obtain the corresponding media resources based on the m3u8 file and play it.

[0003] However, when playing videos in a browser, a freeze often occurs after the advertisement is played, resulting in a poor viewing experience for users. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a media stream playback, media stream processing method, apparatus, device and storage medium to solve or alleviate the above problems.

[0005] According to a first aspect of an embodiment of the present invention, a media stream playback method is provided, comprising: determining a current transmission packet of a real-time media stream; if a media type identifier of the current transmission packet is the same as a media type identifier of a previous transmission packet, parsing and playing the current transmission packet based on the transmission packet identifier of the previous transmission packet.

[0006] According to a second aspect of an embodiment of the present invention, a media stream processing method is provided, comprising: determining a current transmission packet of a real-time media stream; if the media type identifier of the current transmission packet is the same as the media type identifier of a previous transmission packet, creating a mapping relationship between the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet; and associating the mapping relationship with the current transmission packet.

[0007] According to a third aspect of an embodiment of the present invention, a media stream playback device is provided, comprising: a determination module, which determines a current transmission packet of a real-time media stream; and a parsing and playback module, which, if the media type identifier of the current transmission packet is the same as the media type identifier of a previous transmission packet, parses and plays the current transmission packet based on the transmission packet identifier of the previous transmission packet.

[0008] According to a fourth aspect of an embodiment of the present invention, a media stream processing device is provided, comprising: a determination module, which determines a current transmission packet of a real-time media stream; a creation module, which creates a mapping relationship between the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet if the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet; and an association module, which associates the mapping relationship with the current transmission packet.

[0009] According to a fifth aspect of an embodiment of the present invention, a media stream playback method is provided, comprising: determining a playback control instruction corresponding to a playback node according to a playback control operation, and obtaining a real-time media stream of the playback node according to the playback control instruction; determining a previous transmission packet having the same media type identifier as a current transmission packet of the real-time media stream; and parsing and playing the current transmission packet based on the transmission packet identifiers of the current transmission packet and the previous transmission packet.

[0010] According to a sixth aspect of an embodiment of the present invention, a media stream playback device is provided, comprising: an acquisition module, which determines a playback control instruction corresponding to a playback node according to a playback control operation, and acquires the real-time media stream of the playback node according to the playback control instruction; a determination module, which determines a previous transmission packet having the same media type identifier as a current transmission packet of the real-time media stream; and a parsing and playback module, which parses and plays the current transmission packet based on the transmission packet identifiers of the current transmission packet and the previous transmission packet.

[0011] According to a seventh aspect of an embodiment of the present invention, there is provided an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method described in any one of the first to third aspects.

[0012] According to an eighth aspect of an embodiment of the present invention, there is provided a storage medium on which a computer program is stored. When the program is executed by a processor, the method described in any one of the first to third aspects is implemented.

[0013] In the scheme of the embodiment of the present invention, since the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet, it is possible that the current transmission packet and the previous transmission packet are different media resources. Based on the transmission packet identifier of the previous transmission packet, the current transmission packet is parsed and played, thereby avoiding playback and parsing failures of transmission packets from different media resources, thereby improving the playback smoothness of the media stream and the parsing efficiency of the media stream. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0015] Figure 1A A schematic diagram of a network architecture for media streaming playback as an example.

[0016] Figure 1B A schematic diagram of a data frame structure of a media stream file;

[0017] Figure 2 A schematic flow chart of a media stream playing method according to an embodiment of the present invention;

[0018] Figure 3 is a schematic flow chart of a media stream playing method according to another embodiment of the present invention;

[0019] Figure 4 is a schematic flow chart of a media stream playing method according to another embodiment of the present invention;

[0020] Figure 5 is a schematic flow chart of a media stream processing method according to another embodiment of the present invention;

[0021] Fig. 6A is a schematic flow chart of a media stream playing method according to another embodiment of the present invention;

[0022] Figure 6B is a schematic block diagram of a media stream playing device according to another embodiment of the present invention;

[0023] Fig. 7A is a schematic flow chart of a media stream playing method according to another embodiment of the present invention;

[0024] Figure 7B is a schematic block diagram of a media stream playing device according to another embodiment of the present invention;

[0025] Figure 8 is a schematic block diagram of a media stream processing device according to another embodiment of the present invention;

[0026] Fig. 9 is a schematic block diagram of a media stream playing device according to another embodiment of the present invention;

[0027] Fig.10 The present invention is another embodiment of the hardware structure of an electronic device. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in the field based on the embodiments in the embodiments of the present invention should fall within the scope of protection of the embodiments of the present invention.

[0029] The specific implementation of the embodiment of the present invention is further described below in conjunction with the accompanying drawings of the embodiment of the present invention. FIG1 is a schematic diagram of an exemplary network architecture for media stream playback. The user device 100 includes a network interface 110, an application 120, a player module 121, and a human interaction interface 130. The user 300 interacts with the user device 100 through the human interaction interface 130. The media stream resource server 200 includes a network interface 210 and a variety of resources 1 for delivery, such as feature film resources and advertising resources. The application 120 can be a browser application, an audio playback application, a video playback application, or other applications that can play videos. The player module 121 can be installed in the operating system of the user device 100; it can also be installed in the application 120 in a manner such as an SDK (Software Development Kit); it can also be used as a module of the application 120; it can also have a calling relationship with the method of the application 120.

[0030] In addition, in the media stream resource server 200, in one example, the feature film resource and the advertisement resource can adopt the same coding, for example, the whole media stream can be divided into small files based on HTTP (HyperText TransferProtocol, Hypertext Transfer Protocol) to download. When the media stream is being played at the user equipment 100, it is possible to choose to download the same resource at different rates from more and more different standby sources, allowing the streaming media to adapt to different data rates. When starting a streaming media session, the client can download an m3u8 index file file containing source data for finding available media streams.

[0031] Figure 1BThe following is a schematic diagram of the data frame structure of a media stream file. As shown in the figure, the data frame structure of the TS file of the media stream file (HLS) includes the TS (Transport Stream) layer, the pes (Packet Elemental Stream) layer, and the es (Elementary Stream) layer. The es layer can refer to audio and video data, and the pes layer adds timestamps and other description information of the data frame to the audio and video data. The ts layer adds the necessary information for data stream identification and transmission to the pes layer. The TS layer is divided into three parts: TS header, adaptation field, and data (payload) field. The data field is pes data. The content of the TS layer is identified by the PID value, and the main contents include: PAT table, PMT table, audio stream, and video stream. When parsing the TS stream, first determine the PAT table, then find the PMT, and then find the audio and video stream. The pes layer includes the pes header and selects the pes header and pes data. The pes data includes the ES header and other fields. The pes data adds timestamps and other information to each video / audio frame.

[0032] The PMT table may include PID (transmission packet identifier) ​​and stream_type (media resource type identifier). Players such as browser players can find the corresponding TS packets based on PID for parsing. However, in the advertising resource media stream, stream_type may not be consistent with the actual media resource type parsed based on PID. When switching from the main video to the advertisement or from the advertisement to the main video, there will often be a freeze, resulting in a poor viewing experience for users.

[0033] Figure 2 The method of this embodiment can be executed by any appropriate electronic device with data processing capability, including but not limited to terminal devices such as mobile phones, tablet computers and desktop computers. Figure 2 The media stream playback methods include:

[0034] 210: Determine the current transmission packet of the real-time media stream.

[0035] It should be understood that determining the current transmission packet of the real-time media stream can be performed by a browser application or by a player such as in a browser. In addition, each transmission packet in the real-time media stream can be determined or the current transmission packet can be determined from a set of transmission packets of the real-time media stream that have been parsed in advance.

[0036] As an example, a group of transport packets of a real-time media stream may be determined by parsing index information of a media stream file (e.g., a media stream index file). As another example, a group of transport packets of a real-time media stream may be determined based on interruption attribute information. For example, when the interruption attribute information is identified, the transport packet may be initially parsed (e.g., recorded) to obtain a group of transport packets. In addition, a current transport packet may be determined from a group of transport packets based on sequence information of the interruption attribute information.

[0037] 220: If the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet, the current transmission packet is parsed and played based on the transmission packet identifier of the previous transmission packet.

[0038] It should be understood that, in one example, for a TS transport packet, the media type identifier may include stream_type, and the transport packet identifier may be PID. In other examples, the media type identifier may be other media type information, and the transport packet identifier may be other transport packet identifiers.

[0039] In a specific example, the existing parsed PID list may be searched according to stream_type to see whether there is a parsed PID list with the same stream_type.

[0040] It should also be understood that for parsing and playing the current transport packet, the new PID can be used to continue the previous player parsing framework operation. For example, the player parsing framework includes but is not limited to ffmpeg (an open source computer program that can be used to record, convert digital audio, video, and convert them into streams).

[0041] In the scheme of the embodiment of the present invention, since the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet, it is possible that the current transmission packet and the previous transmission packet are different media resources. Based on the transmission packet identifier of the previous transmission packet, the current transmission packet is parsed and played, thereby avoiding playback and parsing failures of transmission packets from different media resources, thereby improving the playback smoothness of the media stream and the parsing efficiency of the media stream.

[0042] In another implementation of the present invention, the method further includes: if there is a first mapping relationship between the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet, determining that the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet.

[0043] Since the existence of the first mapping relationship indicates that the transmission packet identifier of the current transmission packet has not been previously adopted, the media type identifier of the current transmission packet is determined to be the same as the media type identifier of the previous transmission packet based on the existence of the first mapping relationship, thereby improving the efficiency of the determination process.

[0044] As an example, the first mapping relationship can be a PID automatic mapping table. By establishing the mapping table, the PIDs of different transmission packets (audio or audio) are automatically mapped, avoiding confusion of the upper layer of the player (achieving the upper layer of the player being unaware), thereby achieving a completely seamless effect as playing the same video, and the amount of data processing is relatively small. In addition, compatibility with existing parsing frameworks or configurations is also guaranteed.

[0045] Specifically, for example, when each TS transmission packet is parsed for the first time, it is possible to first check whether there is mapping relationship information according to the discontinuity_no field (an example of a resource switching interruption identifier) ​​of the TS transmission packet.

[0046] In another implementation of the present invention, the method further includes: parsing the packet header field of the current transmission packet to determine a second mapping relationship between the transmission packet identifier of the current transmission packet and the media type identifier; and determining the transmission packet identifier of the previous transmission packet based on the second mapping relationship.

[0047] Since the packet header field of the transmission packet is parsed first, the second mapping relationship between the transmission packet identifier and the media type identifier of the current transmission packet is determined based on the packet header field of the current transmission packet. Therefore, the parsing efficiency of the second mapping relationship is improved, and then based on the second mapping relationship, the transmission packet identifier of the previous transmission packet is determined, thereby improving the determination efficiency of the transmission packet identifier.

[0048] In another implementation of the present invention, the method further includes: storing a transmission packet identifier of the previous transmission packet.

[0049] Therefore, storing the transmission packet identifier of the previous transmission packet is beneficial to improving the parsing efficiency of the subsequent transmission packet having the same media type identifier as the previous transmission packet.

[0050] In another implementation of the present invention, determining a current transmission packet of a real-time media stream includes: determining interruption attribute information of each of a group of transmission packets of the real-time media stream; and determining a current transmission packet from the group of transmission packets based on the interruption attribute information.

[0051] Since the interruption attribute information indicates the possibility that the nodes between transmission packets come from different media resources, the current transmission packet is determined from a group of transmission packets based on the interruption attribute information, which improves the efficiency of selecting the current transmission packet as the node, thereby improving the parsing and playback efficiency.

[0052] As an example, the interruption attribute information may include EXT-X-DISCONTINUITY in HLS, which is used to mark that subsequent video formats may be different.

[0053] In another implementation of the present invention, the interruption attribute information of each of a group of transmission packets of a real-time media stream is determined, including: parsing the real-time media stream through a resource switching interruption identifier to obtain a group of transmission packets; and recording the sequence number of the resource switching interruption identifier of each of the group of transmission packets as the interruption attribute information of each of the group of transmission packets.

[0054] Since the sequence number of the resource switching interruption identifier can efficiently distinguish each transmission packet in a group of transmission packets, the sequence number of the resource switching interruption identifier is used as the interruption attribute information of each transmission packet in a group, thereby improving the parsing efficiency of the transmission packets.

[0055] Since not all EXT-X-DISCONTINUITY tags need to rebuild the player, but the tag is used to mark that subsequent video formats may be different, the current transmission packet is determined from a group of transmission packets based on the sequence number of the resource switching interruption identifier.

[0056] Specifically, for example, when each EXT-X-DISCONTINUITY tag is parsed in the m3u8 file, it can be recorded, and a discontinuity_no field can be added to each subsequent TS transport packet.

[0057] In another implementation of the present invention, the method also includes: parsing the packet header field of the current transmission packet to determine a second mapping relationship between the transmission packet identifier of the current transmission packet and the media type identifier; based on the second mapping relationship and the interruption attribute information of the current transmission packet, determining a third mapping relationship between the interruption attribute information, the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet.

[0058] Since both the second mapping relationship and the interruption attribute information can be parsed from the transmission packet, the third mapping relationship is determined based on the second mapping relationship and the interruption attribute information, thereby improving the efficiency of determining the third mapping relationship.

[0059] In another implementation of the present invention, the method further includes: if there is no first mapping relationship between the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet, and the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet, then creating a first mapping relationship.

[0060] Since the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet, it indicates the possibility that the current transmission packet and the previous transmission packet come from different media resources. Therefore, the first mapping relationship between the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet is beneficial to improving the parsing efficiency of the subsequent transmission packet having the same transmission packet identifier as the current transmission packet.

[0061] For example, the subsequent transmission packet may be a packet that jumps back to the current transmission packet according to a user's playback control operation, or may be a subsequent transmission packet of the media stream.

[0062] In another implementation of the present invention, the method further includes: if the media type identifier of the current transmission packet is different from the media type identifier of the previous transmission packet, parsing and playing the current transmission packet based on the transmission packet identifier of the current transmission packet.

[0063] Since the media type identifier of the current transmission packet is different from the media type identifier of the previous transmission packet, it indicates that the possibility of misparsing the current transmission packet and the previous transmission packet is small. Therefore, the current transmission packet is parsed and played based on the transmission packet identifier of the current transmission packet, thereby improving compatibility with conventional parsing frameworks or algorithms.

[0064] In another implementation of the present invention, the method further includes: acquiring the real-time media stream according to a playback control operation.

[0065] Since the playback control operation is random and uncertain, the real-time media stream is obtained according to the playback control operation, which improves the dynamic parsing capability and further improves the user's viewing experience.

[0066] In another implementation of the present invention, the acquiring the real-time media stream according to the playback control operation includes: determining a playback node of the playback control operation; and acquiring the real-time media stream corresponding to the playback node.

[0067] Since the playback node of the playback control operation indicates the playback node that the user expects to jump to, the playback node obtains the real-time media stream corresponding to the playback node, thereby improving the dynamic parsing capability and further improving the user's viewing experience.

[0068] Figure 3 This is a schematic flow chart of a media stream playing method according to another embodiment of the present invention. As shown in the figure, in step S311, the current transmission packet is parsed, the media type identifier and the transmission packet identifier of the current transmission packet are obtained, and the process proceeds to step S312.

[0069] In step S312, the mapping table is searched to determine whether there is a mapping relationship. If yes, the process proceeds to step S315; if no, the process proceeds to step S313. When each TS transport packet is parsed for the first time, the mapping relationship information can be searched based on the discontinuity_no field (an example of a resource switching interruption identifier) ​​of the TS transport packet.

[0070] In step S313, is there a previous transmission packet with the same media type identifier as the current transmission packet? If yes, proceed to step S314, if not, proceed to step S315. Specifically, for example, the existing parsed PID list can be searched according to stream_type to see if there is a parsed stream_type.

[0071] In step S314, a mapping relationship is established in the mapping table, and the process proceeds to step S315. Specifically, a new PID item is added to the PID mapping table, and a mapping relationship between the current discontinuity_no, the old PID and the new PID is established.

[0072] In step S315, the transmission packet identifier of the previous transmission packet is used for subsequent parsing. Specifically, for example, the old PID can be used to continue the original ffmpeg process operation.

[0073] In step S316, the transmission packet identifier of the current transmission packet is used for subsequent parsing. Specifically, for example, the new PIDTS can be used to continue the original ffmpeg process operation.

[0074] Figure 4 This is a schematic flow chart of a media stream playing method according to another embodiment of the present invention. As shown in the figure, in step S411, the index file is initially parsed, and the process proceeds to step S412.

[0075] In step S412, when the interruption attribute information is detected, the sequence number of the current resource switching interruption flag is recorded, and the process proceeds to step S413. Specifically, for example, each EXT-X-DISCONTINUITY tag can be recorded when it is parsed in the m3u8 file, and a discontinuity_no field is added to each subsequent TS transport packet.

[0076] In step S413, is it determined whether the initial analysis is completed? If yes, the process proceeds to step S414; if no, the process returns to step S411.

[0077] In step S414, the transmission packet is parsed.

[0078] Figure 5 This is a schematic flow chart of a media stream processing method according to another embodiment of the present invention. The method of this embodiment can be executed by any appropriate electronic device with data processing capability, including but not limited to terminal devices such as mobile phones, tablet computers, and desktop computers. Figure 5 The media stream processing methods include:

[0079] 510: Determine the current transmission packet of the real-time media stream;

[0080] 520: if the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet, create a mapping relationship between the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet;

[0081] 530: Associate the mapping relationship with the current transmission package.

[0082] In the solution of the embodiment of the present invention, since the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet, indicating the possibility that the current transmission packet and the previous transmission packet are different media resources, a mapping relationship between the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet is created, thereby improving the playback efficiency of transmission packets of different media resources. In addition, associating the mapping relationship with the current transmission packet improves the parsing efficiency of the media stream.

[0083] Fig. 6A This is a schematic flow chart of a media stream playing method according to another embodiment of the present invention. The method of this embodiment can be executed by any appropriate electronic device with data processing capability, including but not limited to terminal devices such as mobile phones, tablet computers, and desktop computers. Fig. 6A The media stream playing method includes:

[0084] 610: Determine a playback control instruction corresponding to the playback node according to the playback control operation, and obtain a real-time media stream of the playback node according to the playback control instruction.

[0085] 620: Determine a previous transmission packet having the same media type identifier as the current transmission packet of the real-time media stream.

[0086] 630: Based on the transmission packet identifiers of the current transmission packet and the previous transmission packet, parse and play the current transmission packet.

[0087] In the scheme of the embodiment of the present invention, since the current transmission packet of the real-time media stream has the same media type identifier as the previous transmission packet, the current transmission packet is parsed and played based on the transmission packet identifier of the previous transmission packet, thereby avoiding the use of transmission packets with the same media type identifier with different transmission packet identifiers, thereby improving the playback smoothness of the media stream.

[0088] Figure 6B This is a schematic block diagram of a media stream playback device according to another embodiment of the present invention. The device of this embodiment can be applied to any appropriate electronic device with data processing capability, including but not limited to terminal devices such as mobile phones, tablet computers, and desktop computers. Figure 6B The media stream playback device comprises:

[0089] A determination module 640 determines a current transmission packet of the real-time media stream;

[0090] The parsing and playing module 650 parses and plays the current transmission packet based on the transmission packet identifier of the previous transmission packet if the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet.

[0091] In the scheme of the embodiment of the present invention, since the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet, it is possible that the current transmission packet and the previous transmission packet are different media resources. Based on the transmission packet identifier of the previous transmission packet, the current transmission packet is parsed and played, thereby avoiding playback and parsing failures of transmission packets from different media resources, thereby improving the playback smoothness of the media stream and the parsing efficiency of the media stream.

[0092] In another implementation of the present invention, the device also includes: a determination module, which determines that the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet if there is a first mapping relationship between the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet.

[0093] In another implementation of the present invention, the determination module is also used to: parse the packet header field of the current transmission packet to determine the second mapping relationship between the transmission packet identifier of the current transmission packet and the media type identifier; and determine the transmission packet identifier of the previous transmission packet based on the second mapping relationship.

[0094] In another implementation of the present invention, the device further includes: a storage module for storing a transmission packet identifier of a previously transmitted packet.

[0095] In another implementation of the present invention, the determination module is specifically used to: determine the interruption attribute information of each of a group of transmission packets of the real-time media stream; and determine the current transmission packet from the group of transmission packets based on the interruption attribute information.

[0096] In another implementation of the present invention, the determination module is specifically used to: parse the real-time media stream through the resource switching interruption identifier to obtain a group of transmission packets; record the sequence number of the resource switching interruption identifier of each of the group of transmission packets as the interruption attribute information of each of the group of transmission packets.

[0097] In another implementation of the present invention, the determination module is specifically used to: parse the packet header field of the current transmission packet to determine the second mapping relationship between the transmission packet identifier of the current transmission packet and the media type identifier; based on the second mapping relationship and the interruption attribute information of the current transmission packet, determine the third mapping relationship between the interruption attribute information, the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet.

[0098] In another implementation of the present invention, the device also includes: a creation module, which creates a first mapping relationship if there is no first mapping relationship between the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet, and the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet.

[0099] In another implementation of the present invention, the parsing and playing module is further used to: if the media type identifier of the current transmission packet is different from the media type identifier of the previous transmission packet, parse and play the current transmission packet based on the transmission packet identifier of the current transmission packet.

[0100] In another implementation of the present invention, the device further includes: an acquisition module, which acquires the real-time media stream according to a playback control operation.

[0101] In another implementation of the present invention, the acquisition module is specifically used to: determine the playback node of the playback control operation; and acquire the real-time media stream corresponding to the playback node.

[0102] The device of this embodiment is used to implement the corresponding methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here. In addition, the functional implementation of each module in the device of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, which will not be repeated here.

[0103] Fig. 7A This is a schematic flow chart of a media stream playing method according to another embodiment of the present invention. The method of this embodiment can be applied to any appropriate electronic device with data processing capability, including but not limited to terminal devices such as mobile phones, tablet computers, and desktop computers.

[0104] 710: Determine a current transmission packet of a first real-time media stream.

[0105] 720: If the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet of the second real-time media stream, the current transmission packet is parsed and played based on the transmission packet identifier of the previous transmission packet.

[0106] In the scheme of the embodiment of the present invention, since the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet, it is possible that the current transmission packet and the previous transmission packet are different media resources. Based on the transmission packet identifier of the previous transmission packet, the current transmission packet is parsed and played, thereby avoiding playback and parsing failures of transmission packets from different media resources, thereby improving the playback smoothness of the media stream and the parsing efficiency of the media stream.

[0107] It should be understood that the first real-time media stream (eg, live media stream) and the second real-time media stream (eg, live media stream) may be different media streams from the same media resource, or may be different media streams from different media resources.

[0108] In the first example, the first real-time media stream and the second real-time media stream are different media streams from different media resources. In response to the live switching operation, the second real-time media stream can be switched to the first embodiment media stream, and the current transmission packet of the first real-time media stream can be determined.

[0109] Since when switching from the second real-time media stream to the first embodiment media stream, the current transmission packet is parsed and played based on the transmission packet identifier of the previous transmission packet, playback and parsing failures of transmission packets of different media streams are avoided, thereby improving the playback smoothness of the media stream and the parsing efficiency of the media stream.

[0110] In the second example, the first real-time media stream and the second real-time media stream are different media streams from the same media resource. The current transmission packet of the first real-time media stream can be obtained in response to the live broadcast reconnection instruction. The above-mentioned live broadcast reconnection instruction is triggered by the interruption of the second real-time media stream, or by the play refresh instruction. In addition, a mapping relationship between the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet can be generated and stored. The mapping relationship between the media type identifier and the transmission packet identifier of the previous transmission packet of the second real-time media stream can be stored.

[0111] When the second real-time media stream is interrupted or triggered by a playback refresh instruction, the current transmission packet is parsed and played based on the transmission packet identifier of the previous transmission packet, thereby avoiding playback and parsing failures of transmission packets of different media streams, thereby improving the playback smoothness of the media stream and the parsing efficiency of the media stream.

[0112] It should be understood that the second real-time media stream may be all real-time media streams preceding the first real-time media stream, and the transmission packet identifier of the previous transmission packet is determined based on the mapping relationship between the media type identifier and the transmission packet identifier of the previous transmission packet of the second real-time media stream. In addition, the second real-time media stream may also be the previous real-time media stream of the first real-time media stream, and only the mapping relationship between the media type identifier and the transmission packet identifier of the previous transmission packet of the above real-time media stream may be stored, and the transmission packet identifier of the previous transmission packet is determined based on the mapping relationship between the media type identifier and the transmission packet identifier of the previous transmission packet of the above real-time media stream.

[0113] Figure 7B This is a schematic block diagram of a media stream playback device according to another embodiment of the present invention. The device of this embodiment can be applied to any appropriate electronic device with data processing capability, including but not limited to terminal devices such as mobile phones, tablet computers, and desktop computers.

[0114] The determination module 730 determines a current transmission packet of the first real-time media stream.

[0115] The parsing and playing module 740 parses and plays the current transmission packet based on the transmission packet identifier of the previous transmission packet if the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet of the second real-time media stream.

[0116] In the scheme of the embodiment of the present invention, since the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet, it is possible that the current transmission packet and the previous transmission packet are different media resources. Based on the transmission packet identifier of the previous transmission packet, the current transmission packet is parsed and played, thereby avoiding playback and parsing failures of transmission packets from different media resources, thereby improving the playback smoothness of the media stream and the parsing efficiency of the media stream.

[0117] Figure 8 This is a schematic block diagram of a media stream processing device according to another embodiment of the present invention. The device of this embodiment can be applied to any appropriate electronic device with data processing capability, including but not limited to terminal devices such as mobile phones, tablet computers, and desktop computers. Figure 8 The media stream processing device comprises:

[0118] A determination module 810 determines a current transmission packet of a real-time media stream;

[0119] A creation module 820, if the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet, creates a mapping relationship between the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet;

[0120] The association module 830 associates the mapping relationship with the current transmission packet.

[0121] In the solution of the embodiment of the present invention, since the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet, indicating the possibility that the current transmission packet and the previous transmission packet are different media resources, a mapping relationship between the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet is created, thereby improving the playback efficiency of transmission packets of different media resources. In addition, associating the mapping relationship with the current transmission packet improves the parsing efficiency of the media stream.

[0122] The device of this embodiment is used to implement the corresponding methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here. In addition, the functional implementation of each module in the device of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, which will not be repeated here.

[0123] Fig. 9This is a schematic block diagram of a media stream processing device according to another embodiment of the present invention. The device of this embodiment can be applied to any appropriate electronic device with data processing capability, including but not limited to terminal devices such as mobile phones, tablet computers, and desktop computers. Fig. 9 The media stream playback device comprises:

[0124] The acquisition module 910 determines the playback control instruction corresponding to the playback node according to the playback control operation, and acquires the real-time media stream of the playback node according to the playback control instruction;

[0125] A determination module 920 determines a previous transmission packet having the same media type identifier as the current transmission packet of the real-time media stream;

[0126] The parsing and playing module 930 parses and plays the current transmission packet based on the transmission packet identifiers of the current transmission packet and the previous transmission packet.

[0127] In the scheme of the embodiment of the present invention, since the current transmission packet of the real-time media stream has the same media type identifier as the previous transmission packet, the current transmission packet is parsed and played based on the transmission packet identifier of the previous transmission packet, thereby avoiding the use of transmission packets with the same media type identifier with different transmission packet identifiers, thereby improving the playback smoothness of the media stream.

[0128] The device of this embodiment is used to implement the corresponding methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here. In addition, the functional implementation of each module in the device of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, which will not be repeated here.

[0129] Fig.10 The hardware structure of the electronic device of another embodiment of the present invention is as follows; Fig.10 As shown, the hardware structure of the electronic device may include: a processor 1001, a communication interface 1002, a storage medium 1003 and a communication bus 1004;

[0130] The processor 1001, the communication interface 1002, and the storage medium 1003 communicate with each other via the communication bus 1004;

[0131] Optionally, the communication interface 1002 may be an interface of a communication module;

[0132] The processor 1001 may be specifically configured to: determine a current transmission packet of a real-time media stream; if a media type identifier of the current transmission packet is the same as a media type identifier of a previous transmission packet, parse and play the current transmission packet based on the transmission packet identifier of the previous transmission packet;

[0133] Alternatively, determining a current transmission packet of the real-time media stream; if the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet, creating a mapping relationship between the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet; and associating the mapping relationship with the current transmission packet;

[0134] Alternatively, according to the playback control operation, a playback control instruction corresponding to the playback node is determined, and according to the playback control instruction, the real-time media stream of the playback node is obtained; a previous transmission packet having the same media type identifier as the current transmission packet of the real-time media stream is determined; based on the transmission packet identifiers of the current transmission packet and the previous transmission packet, the current transmission packet is parsed and played.

[0135] The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present invention may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0136] The storage medium may be, but is not limited to, a random access memory medium (RAM), a read-only memory medium (ROM), a programmable read-only memory medium (PROM), an erasable programmable read-only memory medium (EPROM), an electrically erasable read-only memory medium (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.

[0137] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a storage medium, and the computer program includes a program code configured to execute the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present invention are executed. It should be noted that the storage medium described in the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access storage medium (RAM), a read-only storage medium (ROM), an erasable programmable read-only storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only storage medium (CD-ROM), an optical storage medium, a magnetic storage medium, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, an apparatus or a device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any storage medium other than a computer-readable storage medium, which may send, propagate, or transmit a program configured to be used by or in conjunction with an instruction execution system, an apparatus or a device. The program code contained on the storage medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0138] Computer program code configured to perform the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0139] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of the code contains one or more executable instructions configured to implement the specified logical function. There are specific sequential relationships in the above-mentioned specific embodiments, but these sequential relationships are only exemplary. When the specific implementation is implemented, these steps may be less, more or the execution order may be adjusted. That is, in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0140] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The names of these modules do not limit the modules themselves in some cases.

[0141] As another aspect, the present invention further provides a storage medium having a computer program stored thereon, which implements the method described in the above embodiment when executed by a processor.

[0142] As another aspect, the present invention further provides a storage medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above storage medium carries one or more programs, and when the above one or more programs are executed by the device, the device: determines the current transmission packet of the real-time media stream; if the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet, then parses and plays the current transmission packet based on the transmission packet identifier of the previous transmission packet;

[0143] Alternatively, determining a current transmission packet of the real-time media stream; if the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet, creating a mapping relationship between the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet; and associating the mapping relationship with the current transmission packet;

[0144] Alternatively, according to the playback control operation, a playback control instruction corresponding to the playback node is determined, and according to the playback control instruction, the real-time media stream of the playback node is obtained; a previous transmission packet having the same media type identifier as the current transmission packet of the real-time media stream is determined; based on the transmission packet identifiers of the current transmission packet and the previous transmission packet, the current transmission packet is parsed and played.

[0145] The expressions "first", "second", "the first" or "the second" used in various embodiments of the present disclosure may modify various components regardless of order and / or importance, but these expressions do not limit the corresponding components. The above expressions are only configured for the purpose of distinguishing an element from other elements. For example, a first user device and a second user device represent different user devices, although both are user devices. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the present disclosure.

[0146] When one element (e.g., a first element) is referred to as being "(operably or communicatively) coupled" or "(operably or communicatively) coupled to" or "connected to" another element (e.g., a second element), it is understood that the one element is directly connected to the other element or the one element is indirectly connected to the other element via yet another element (e.g., a third element). Conversely, it is understood that when an element (e.g., a first element) is referred to as being "directly connected" or "directly coupled" to another element (the second element), no element (e.g., a third element) is interposed between the two.

[0147] The above description is only a preferred embodiment of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present invention is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present invention (but not limited to) to form a technical solution.

Claims

1. A media stream playing method, comprising: Through the resource switching interruption flag, the real-time media stream is parsed to obtain a group of transmission packets; Recording the sequence numbers of the resource switching interruption identifiers of the group of transmission packets as the interruption attribute information of the group of transmission packets; Based on the interrupt attribute information, determining a current transmission packet from the group of transmission packets; If there is a first mapping relationship between the transmission identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet, it is determined that the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet, wherein the resource switching interruption identifier is used to find out whether there is a first mapping relationship between the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet; Based on the transmission packet identifier of the previous transmission packet, the current transmission packet is parsed and played.

2. The method according to claim 1, wherein: The method further comprises: Parsing the packet header field of the current transmission packet to determine a second mapping relationship between the transmission packet identifier of the current transmission packet and the media type identifier; Based on the second mapping relationship, a transmission packet identifier of the previous transmission packet is determined.

3. The method according to claim 1, wherein: The method further comprises: The transmission packet identifier of the previous transmission packet is stored.

4. The method according to claim 1, wherein: The method further comprises: Parsing the packet header field of the current transmission packet to determine a second mapping relationship between the transmission packet identifier of the current transmission packet and the media type identifier; Based on the second mapping relationship and the interruption attribute information of the current transmission packet, a third mapping relationship among the interruption attribute information, the transmission packet identifier of the current transmission packet, and the transmission packet identifier of the previous transmission packet is determined.

5. The method according to claim 1, wherein: The method further comprises: If there is no first mapping relationship between the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet, and the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet, then the first mapping relationship is created.

6. The method according to claim 1, wherein: The method further comprises: If the media type identifier of the current transmission packet is different from the media type identifier of the previous transmission packet, the current transmission packet is parsed and played based on the transmission packet identifier of the current transmission packet.

7. The method according to claim 1, wherein: The method further comprises: According to the playback control operation, the real-time media stream is obtained.

8. The method according to claim 7, wherein: The acquiring the real-time media stream according to the playback control operation includes: Determining a playback node for the playback control operation; Obtain the real-time media stream corresponding to the playback node.

9. A media stream processing method, comprising: Through the resource switching interruption flag, the real-time media stream is parsed to obtain a group of transmission packets; Recording the sequence numbers of the resource switching interruption identifiers of the group of transmission packets as the interruption attribute information of the group of transmission packets; Based on the interrupt attribute information, determining a current transmission packet from the group of transmission packets; If the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet, a mapping relationship between the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet is created, wherein the resource switching interruption identifier is used to find out whether there is a mapping relationship between the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet; The mapping relationship is associated with the current transmission packet.

10. A media stream playback device, comprising: a determination module that parses the first real-time media stream through the resource switching interruption identifier to obtain a group of transmission packets, records the sequence numbers of the resource switching interruption identifiers of the group of transmission packets as the interruption attribute information of the group of transmission packets, and determines the current transmission packet from the group of transmission packets based on the interruption attribute information; The parsing and playing module determines that the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet if there is a first mapping relationship between the transmission identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet, and parses and plays the current transmission packet based on the transmission packet identifier of the previous transmission packet, wherein the resource switching interrupt identifier is used to find out whether there is a first mapping relationship between the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet.

11. A media stream processing device, comprising: a determination module, parsing the real-time media stream through the resource switching interruption identifier to obtain a group of transmission packets, recording the sequence number of the resource switching interruption identifier of each of the group of transmission packets as the interruption attribute information of each of the group of transmission packets, and determining the current transmission packet from the group of transmission packets based on the interruption attribute information; a creation module, which creates a mapping relationship between the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet if the media type identifier of the current transmission packet is the same as the media type identifier of the previous transmission packet, wherein the resource switching interrupt identifier is used to find out whether there is a mapping relationship between the transmission packet identifier of the current transmission packet and the transmission packet identifier of the previous transmission packet; An associating module associates the mapping relationship with the current transmission packet.

12. An electronic device, comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the method according to any one of claims 1-9.

13. A storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.