Multimedia data processing method, device, electronic device and computer storage medium
By configuring subclasses of decapsulation and parsing policies in the playback interface, the problem of poor compatibility of streaming media protocols is solved, and the compatibility and processing of multi-protocols are achieved, and the compatibility and scalability of the player are improved.
Patent Information
- Application Number
- CN202410280616.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-12
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-03-12
AI Technical Summary
The existing streaming protocols have poor compatibility, which makes different protocols unable to be compatible at the same time in the same player, limiting the flexibility of video playback and device support.
By configuring the subclasses of the decapsulation policy and the parsing policy in the playback interface, we realize unified management and processing of different streaming protocols, select appropriate execution classes and execution subclasses for parsing and decapsulating, and realize multi-protocol compatibility and ability Extensibility.
It realizes compatibility and processing of different streaming protocols, improves player compatibility and scalability, and improves understanding of packaging and decoding efficiency.
Smart Images

Figure CN118075372B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a multimedia data processing method, device, electronic device and computer storage medium. Background Art
[0002] With the popularity of mobile video playback applications, playing video content on mobile devices is a common demand, especially in the digital age. Users have higher and higher requirements for video playback, such as high definition, smoothness, and fast loading. In order to meet this demand, there are currently various player playback systems.
[0003] By selecting different streaming protocols in different playback scenarios, the video playback requirements of different playback scenarios can be met. However, different streaming protocols are not compatible with each other, and each streaming protocol is often only supported by some playback devices. For example, HLS (HTTP Live Streaming, an adaptive bitrate streaming protocol based on HTTP) and DASH (Dynamic Adaptive Streaming over HTTP, a dynamic adaptive streaming protocol based on HTTP). Each protocol has its own characteristics and requirements, so processing them requires its own specific decapsulation and parsing methods. Summary of the invention
[0004] The embodiments of the present application provide a multimedia data processing method, device, electronic device and computer-readable storage medium, which can solve the technical problem of poor compatibility of current streaming media protocols.
[0005] The present application provides a multimedia data processing method, including:
[0006] In response to a parsing event of the target multimedia data, a playback interface is called, wherein the playback interface is configured with at least one execution class, wherein the execution class includes a decapsulation strategy and at least two execution subclasses using the decapsulation strategy, wherein different execution subclasses encapsulate parsing strategies corresponding to different playback protocols;
[0007] Selecting a target execution class corresponding to the playback protocol of the target multimedia data from the execution class through the playback interface, and selecting a target execution subclass corresponding to the playback protocol of the target multimedia data from the target execution class;
[0008] Parsing the target multimedia data according to the parsing strategy of the target execution subclass to obtain parsed target multimedia data, and decapsulating the parsed target multimedia data according to the decapsulation strategy of the target execution class to obtain multimedia data to be output;
[0009] The multimedia data to be output is output through the playback interface to obtain decapsulated multimedia data corresponding to the target multimedia data.
[0010] Further, the target execution class includes a target parsing class and a target management class, the target parsing class includes a plurality of parsing subclasses, the parsing subclasses encapsulate a first parsing strategy corresponding to the playback protocol, the target management class includes a plurality of management subclasses, the management subclasses encapsulate a second parsing strategy corresponding to the playback protocol;
[0011] The step of selecting a target execution subclass corresponding to the playback protocol of the target multimedia data from the target execution class includes:
[0012] Selecting a target parsing subclass corresponding to the playback protocol of the target multimedia data from the plurality of parsing subclasses;
[0013] Selecting a target management subclass corresponding to the playback protocol of the target multimedia data from the plurality of management subclasses;
[0014] The target parsing subclass and the target management subclass are determined as the target execution subclass.
[0015] Furthermore, the target multimedia data is parsed according to the parsing strategy of the target execution subclass to obtain the parsed target multimedia data, including:
[0016] The target multimedia data is parsed according to the first parsing strategy of the target parsing subclass and the second parsing strategy of the target management subclass to obtain the parsed target multimedia data.
[0017] Further, the plurality of execution subclasses are configured with an execution parent class, and the target multimedia data is parsed according to the parsing strategy of the target execution subclass to obtain the parsed target multimedia data, and further includes:
[0018] Obtain the parsing strategy of the execution parent class;
[0019] The target multimedia data is parsed according to the parsing strategy of the target execution subclass and the parsing strategy of the execution parent class to obtain the parsed target multimedia data.
[0020] Furthermore, after parsing the target multimedia data according to the parsing strategy of the target execution subclass to obtain the parsed target multimedia data, the method further includes:
[0021] Get the preset data output structure;
[0022] The parsed target multimedia data is stored according to the data output structure.
[0023] Furthermore, the multimedia data processing method further includes:
[0024] In response to the playback control event, a new execution subclass is generated according to the parsing strategy corresponding to the new playback protocol, and a new execution class is generated according to the new decapsulation strategy;
[0025] The new execution subclass and the new execution class are configured to the playback interface.
[0026] Furthermore, the parsing strategy of the new playback protocol includes a first parsing strategy for parsing and a second parsing strategy for management and execution;
[0027] The generating of a new execution subclass according to the parsing strategy corresponding to the new playback protocol includes:
[0028] Encapsulating the first parsing strategy into a new parsing subclass, and encapsulating the second parsing strategy into a new management subclass;
[0029] The new parsing subclass and the new management subclass are added as new execution subclasses to the execution class of the playback interface.
[0030] Furthermore, the execution class of the playback interface includes a parsing class and a management class, and the adding of the new parsing subclass and the new management subclass as new execution subclasses to the execution class of the playback interface includes:
[0031] Adding the new parsing subclass as a new execution subclass to the parsing class;
[0032] The new management subclass is added as a new execution subclass to the management class.
[0033] Accordingly, an embodiment of the present application provides a multimedia data processing device, including:
[0034] A calling module, configured to call a playback interface in response to a parsing event of target multimedia data, wherein the playback interface is configured with at least one execution class, wherein the execution class includes a decapsulation strategy and at least two execution subclasses using the decapsulation strategy, and different execution subclasses encapsulate parsing strategies corresponding to different playback protocols;
[0035] A selection module, used to select a target execution class corresponding to the playback protocol of the target multimedia data from the execution class through the playback interface, and to select a target execution subclass corresponding to the playback protocol of the target multimedia data from the target execution class;
[0036] a protocol processing module, configured to parse the target multimedia data according to the parsing strategy of the target execution subclass to obtain the parsed target multimedia data, and to decapsulate the parsed target multimedia data according to the decapsulation strategy of the target execution class to obtain the multimedia data to be output;
[0037] The output module is used to output the multimedia data to be output through the playback interface to obtain the decapsulated multimedia data corresponding to the target multimedia data.
[0038] In addition, an embodiment of the present application also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor is used to run the computer program in the memory to implement the multimedia data processing method provided in the embodiment of the present application.
[0039] In addition, an embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. The computer program is suitable for loading by a processor to execute any multimedia data processing method provided in the embodiment of the present application.
[0040] In addition, an embodiment of the present application also provides a computer program product, including a computer program, and when the computer program is executed by a processor, any multimedia data processing method provided in the embodiment of the present application is implemented.
[0041] In an embodiment of the present application, by responding to a parsing event of target multimedia data, a playback interface is called, the playback interface is configured with at least one execution class, the execution class includes a decapsulation strategy, and at least two execution subclasses using the decapsulation strategy, different execution subclasses encapsulate parsing strategies corresponding to different playback protocols, so that all execution classes can implement a common interface, thereby allowing users to perform standardized management of data processed by the playback protocol in the same way, so that data of different playback protocols can be uniformly processed and output, and compatibility of multiple types of playback protocols is achieved; a target execution class corresponding to the playback protocol of the target multimedia data is selected from the execution class through the playback interface, and a target execution subclass corresponding to the playback protocol of the target multimedia data is selected from the target execution class; and according to the parsing strategy of the target execution subclass, The target multimedia data is briefly parsed to obtain the parsed target multimedia data, and the parsed target multimedia data is decapsulated according to the decapsulation strategy of the target execution class to obtain the multimedia data to be output, so that different target execution classes are selected through the playback interface, and different types of playback protocols can be parsed without understanding the specific details of each decapsulation strategy or streaming media protocol; finally, the multimedia data to be output is output through the playback interface to obtain the decapsulated multimedia data corresponding to the target multimedia data, thereby realizing the compatibility and processing of multiple playback protocols, so that different data processing of different playback protocols of multimedia data can be realized through one playback interface and the configured execution class and execution subclass, thereby improving the compatibility and scalability of the player, and further improving the decapsulation and decoding efficiency of different playback protocols. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0043] Figure 1 It is a flowchart of the multimedia data processing method provided by the present application;
[0044] Figure 2 It is a schematic diagram of the overall framework of the multimedia data processing method provided by the present application;
[0045] Figure 3 It is a schematic diagram of the overall framework of the multimedia data processing method provided by the present application under the multi-play protocol configuration;
[0046] Figure 4a It is a schematic diagram of the composition of the first parsing class provided in an embodiment of the present application;
[0047] Figure 4b It is a schematic diagram of the composition of the first management class provided in an embodiment of the present application;
[0048] Figure 5a It is a schematic diagram of the composition of the parsing class corresponding to the playback protocol provided in the embodiment of the present application;
[0049] Figure 5b It is a schematic diagram of the composition of the management class corresponding to the playback protocol provided in the embodiment of the present application;
[0050] Figure 6 This is a schematic diagram of a video playback process provided by an embodiment of the present application;
[0051] Figure 7 is a structural diagram of a multimedia data processing device provided in an embodiment of the present application;
[0052] Figure 8 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0053] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.
[0054] The embodiments of the present application provide a multimedia data processing method, device, electronic device and computer readable storage medium. The multimedia data processing device can be integrated in an electronic device, which can be a server or a terminal.
[0055] Among them, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), as well as big data and artificial intelligence platforms.
[0056] The terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal and the server may be directly or indirectly connected via wired or wireless communication, and this application does not limit this.
[0057] In addition, the term "plurality" in the embodiments of the present application refers to two or more than two. The terms "first" and "second" in the embodiments of the present application are used to distinguish descriptions and should not be understood as implying relative importance.
[0058] It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.
[0059] The streaming media service framework usually includes two ports and a server, namely the push end, the pull end and the streaming media server. Pushing refers to the process in which the push end transmits the acquired multimedia data to the server; pulling refers to the process in which the server already has multimedia data and the pull end pulls the multimedia data from the server through the streaming media protocol. Therefore, according to different application scenarios, streaming media protocols can be divided into push protocols and pull playback protocols. Some streaming media protocols can be applied to both the push end and the pull end, while some streaming media protocols can only be applied to the push end or the pull end. The streaming media protocol can realize the transmission of multimedia data on the network.
[0060] At present, since each streaming protocol has its own characteristics and requirements, it needs to go through specific decapsulation and parsing methods. Therefore, streaming protocols are often only supported by certain devices or certain scenarios, and usually a player only supports one streaming protocol. For example, HLS protocol and DASH protocol. In some playback scenarios and devices, only HLS protocol is supported; in other playback scenarios and devices, only DASH protocol is supported. As a result, different types of streaming protocols cannot be compatible with the same player at the same time.
[0061] Based on this, an embodiment of the present application proposes a multimedia data processing method, which calls a playback interface in response to a parsing event of target multimedia data, wherein the playback interface is configured with at least one execution class, the execution class includes a decapsulation strategy, and at least two execution subclasses using the decapsulation strategy, and different execution subclasses encapsulate parsing strategies corresponding to different playback protocols, so that all execution classes can implement a common interface, thereby allowing users to perform standardized management of data processed by the playback protocol in the same way, so that data of different playback protocols can be uniformly processed and output, and compatibility of multiple types of playback protocols is achieved; a target execution class corresponding to the playback protocol of the target multimedia data is selected from the execution class through the playback interface, and a target execution subclass corresponding to the playback protocol of the target multimedia data is selected from the target execution class; and according to the target execution The target multimedia data is parsed according to the parsing strategy of the row subclass to obtain the parsed target multimedia data, and the parsed target multimedia data is decapsulated according to the decapsulation strategy of the target execution class to obtain the multimedia data to be output, so that different target execution classes are selected through the playback interface, and different types of playback protocols can be parsed without understanding the specific details of each decapsulation strategy or streaming media protocol; finally, the multimedia data to be output is output through the playback interface to obtain the decapsulated multimedia data corresponding to the target multimedia data, thereby realizing the compatibility and processing of multiple playback protocols, so that through a playback interface and the configured execution class and execution subclass, different data processing of different playback protocols of multimedia data can be realized, thereby improving the compatibility and scalability of the player, and further improving the decapsulation and decoding efficiency of different playback protocols.
[0062] Specifically, the multimedia data processing method is usually set in a multimedia data processing device in the form of a computer program, and the multimedia data processing device is usually set in a computer device in the form of a processor. The computer device (for example, it can be a terminal device of a local client with an application installed, or a remote server of the application) executes the computer program corresponding to the multimedia data processing method through the multimedia data processing device to execute the multimedia data processing method provided in the embodiment of the present application, as follows.
[0063] See also Figure 1 , Figure 1 : is a flowchart of a multimedia data processing method provided by an embodiment of the present application. The multimedia data processing method may include:
[0064] S101. In response to a parsing event of target multimedia data, a playback interface is called, wherein the playback interface is configured with at least one execution class, wherein the execution class includes a decapsulation strategy and at least two execution subclasses using the decapsulation strategy, and different execution subclasses encapsulate parsing strategies corresponding to different playback protocols.
[0065] In this embodiment, the target multimedia data is the data of the multimedia content to be played, such as video data, audio data, or audio and video data. The target multimedia data can be data obtained by a data source, such as audio and video data obtained by a camera, or data containing multimedia content sent by a server.
[0066] The parsing event of the target multimedia data is an event for parsing and decapsulating the target multimedia data. If a parsing instruction for parsing the target multimedia data is triggered, or an instruction for pushing or pulling a stream of the target multimedia data is triggered, or a specific identifier of the target multimedia data is obtained, the parsing event of the target multimedia data is triggered. In response to the parsing event of the target multimedia data, the playback interface is called. Among them, the playback interface is a public interface implemented by multiple execution classes, and the playback interface is used for external calls. The input and output of the target multimedia data can be processed through the playback interface. The playback interface is configured with at least one execution class, and the execution class contains a decapsulation strategy. The data can be decapsulated through the decapsulation strategy of the execution class. The execution class also includes at least two execution subclasses using the decapsulation strategy, and different execution subclasses encapsulate parsing strategies corresponding to different playback protocols. The target multimedia data can be parsed differently through the parsing strategies corresponding to different playback protocols, so that the parsed target multimedia data meets the requirements of the corresponding playback protocol.
[0067] like Figure 2 As shown, Figure 2 The overall framework diagram provided for this embodiment. Among them, the playback interface is configured with multiple execution classes, namely the first execution class, the second execution class to the nth execution class. The first execution class includes two execution subclasses, namely the first execution subclass and the second execution subclass; wherein the first execution subclass includes the first parsing subclass and the first management subclass; the second execution subclass includes the second parsing subclass and the second management subclass, and the data output by the first parsing subclass and the second parsing subclass are output and stored according to the data output structure. In particular, in the first execution subclass and the second execution subclass included in the second execution class, the first execution subclass and the aforementioned first execution class are not classes corresponding to the same playback protocol. For example, the first execution class is an execution class (FFMPEGExtractor) corresponding to the FFMPEG protocol, and the first execution class can include a parsing subclass (such as FFMPEGSParser) corresponding to the FFMPEG protocol and a management subclass (such as FFMPEGManager) corresponding to the FFMPEG protocol; and the first parsing subclass and the first management subclass in the first execution subclass are execution subclasses of the second execution class, such as the first parsing subclass (HLSParser) and the first management subclass (HLSManager).
[0068] In one implementation of a multi-play protocol configuration, such as Figure 3 As shown, Figure 3 It is a schematic diagram of the overall framework under a multi-playback protocol configuration. Among them, Extractor is an execution class, and there are multiple execution classes, namely, the execution class corresponding to the FFMPEG protocol, namely FFMPEGExtractor, the execution class corresponding to the protocol of the Manifest structure, namely ManifestExtractor, and the execution class corresponding to other protocols, namely...Extractor. The execution class corresponding to the protocol of the Manifest structure includes two execution subclasses, which are the execution subclass corresponding to the HLS protocol, namely HLSExtractor, and the execution subclass corresponding to the Dash protocol, namely DashExtractor. Among them, the execution subclass corresponding to the HLS protocol includes the parsing subclass corresponding to the HLS protocol, namely HLSParser, and the management subclass corresponding to the HLS protocol, namely HLSManager; the execution subclass corresponding to the Dash protocol includes the parsing subclass corresponding to the Dash protocol, namely DashParser, and the management subclass corresponding to the Dash protocol.
[0069] Furthermore, in some implementations, the playback interface may be configured with multiple execution classes, one execution class includes a decapsulation strategy and at least one execution subclass using the decapsulation strategy, and one execution subclass encapsulates at least one parsing strategy corresponding to a playback protocol. Thus, by corresponding different playback protocols to an execution class of a decapsulation strategy, unified management of multiple playback protocols can also be achieved.
[0070] This embodiment calls a playback interface, which configures at least one execution class including a decapsulation strategy and at least two execution subclasses using the decapsulation strategy. Different execution subclasses encapsulate parsing strategies corresponding to different playback protocols, so that different playback protocols can be uniformly processed and output through the playback interface, thereby further realizing support for multiple protocols.
[0071] S102: selecting a target execution class corresponding to the playback protocol of the target multimedia data from the execution class through the playback interface, and selecting a target execution subclass corresponding to the playback protocol of the target multimedia data from the target execution class.
[0072] In this embodiment, the playback protocol is a protocol that requires the target multimedia data to be parsed and transmitted in a specific manner in the network, such as streaming media protocols such as HLS and DASH. Different playback protocols can be selected according to different playback scenarios. For example, when multiple requests are required for playback and high network quality requirements are required, the HLS protocol can be selected as the playback protocol for the target multimedia data; when adaptive bit rate streaming and cross-platform output are required, the DASH protocol can be selected as the playback protocol for the target multimedia data. The target execution class corresponding to the playback protocol of the target multimedia data is selected from the execution class through the playback interface.
[0073] Specifically, the execution class contains a decapsulation strategy, and the corresponding decapsulation strategies for multimedia data of different formats are also different. Therefore, the data format of the target multimedia data is obtained, and according to the playback protocol of the target multimedia data, an execution class is selected from the execution class of the playback interface as the target execution class corresponding to the target multimedia data. For example, the playback interface is configured with execution classes for different playback protocols, such as the execution class (FFMPEGExtractor) corresponding to the FFMPEG protocol and the execution class ManifestExtractor to which the Dash protocol and the HLS protocol belong. When the playback protocol of the target multimedia data is obtained, the target execution class is selected from the execution class according to the playback protocol of the target multimedia data.
[0074] Afterwards, a target execution subclass corresponding to the playback protocol of the target multimedia data is selected from the target execution class. Among them, one execution class includes at least two execution subclasses, and one execution subclass includes at least one parsing strategy for the playback protocol. The target execution subclass of the playback protocol of the target multimedia data is selected from multiple execution subclasses. For example, the playback protocol of the target multimedia data is the HLS protocol, and the execution class (ManifestExtractor) to which the Dash protocol and the HLS protocol belong together includes an execution subclass corresponding to the Dash protocol, and an execution subclass corresponding to the HLS protocol. According to the playback protocol (HLS protocol) of the target multimedia data, it can be determined that the execution class corresponding to the playback protocol of the target multimedia data is the Manifest execution class (ManifestExtractor). Then, from the multiple execution subclasses of the execution class (ManifestExtractor), namely the execution subclass (DashExtractor) corresponding to the Dash protocol and the execution subclass (HLSExtractor) corresponding to the HLS protocol, the target execution subclass corresponding to the playback protocol of the target multimedia data is selected as the execution subclass (HLSExtractor) corresponding to the HLS protocol.
[0075] In some embodiments, if the playback interface is configured with multiple execution classes, one execution class includes a decapsulation strategy, and an execution subclass using the decapsulation strategy, and the one execution subclass encapsulates a parsing strategy corresponding to a playback protocol, then when selecting the target execution class and target execution subclass corresponding to the playback protocol of the target multimedia data, the target execution class corresponding to the playback protocol of the target multimedia data can be selected from the multiple execution classes through the playback interface; thereafter, the execution subclass in the target execution class is directly determined as the target execution subclass corresponding to the playback protocol of the target multimedia data.
[0076] This embodiment selects the target execution class corresponding to the playback protocol of the target multimedia data from the execution class through the playback interface, thereby realizing the accurate acquisition of the decapsulation strategy of the playback protocol of the target multimedia data; then, the target execution subclass corresponding to the playback protocol of the target multimedia data is selected from the target execution class, thereby realizing the accurate acquisition of the parsing strategy corresponding to the playback protocol, and further enables data processing of different playback protocols to be realized through the selected decapsulation strategy and parsing strategy.
[0077] S103, parsing the target multimedia data according to the parsing strategy of the target execution subclass to obtain parsed target multimedia data, and decapsulating the parsed target multimedia data according to the decapsulation strategy of the target execution class to obtain multimedia data to be output.
[0078] In this embodiment, different playback protocols correspond to different parsing strategies. By determining the target execution subclass of the playback protocol of the target multimedia data, the parsing strategy encapsulated in the target execution subclass can be obtained. The target multimedia data is parsed according to the parsing strategy of the target execution subclass to obtain the parsed target multimedia data. The decapsulation strategies corresponding to different playback protocols may be the same. According to the decapsulation strategy encapsulated in the target execution class corresponding to the determined playback protocol of the target multimedia data, the parsed target multimedia data is decapsulated to obtain the multimedia data to be output.
[0079] Furthermore, the same playback protocol may include multiple sub-parsing strategies, and the multiple sub-parsing strategies constitute the parsing strategy of the playback protocol. For example, the parsing strategy of the same playback protocol may include a first parsing strategy and a second parsing strategy, and the first parsing strategy and the second parsing strategy constitute the parsing strategy of the playback protocol. Among them, the first parsing strategy is used for parsing, and the second parsing strategy is used for management and execution. Therefore, when the target multimedia data is parsed according to the parsing strategy of the target execution subclass, the target multimedia data can be parsed according to the parsing subclass encapsulated with the first parsing strategy in the target execution subclass, and the management subclass encapsulated with the second parsing strategy in the target execution subclass, to obtain the parsed target multimedia data. A target execution subclass includes a parsing subclass corresponding to a playback protocol and a management subclass of the same playback protocol.
[0080] In another embodiment, the execution class of the playback interface configuration also includes a parsing class and a management class. The parsing class includes a parsing subclass encapsulated by a first parsing strategy corresponding to multiple playback protocols, and the management class includes a management subclass encapsulated by a second parsing strategy corresponding to multiple playback protocols. Therefore, when parsing the target multimedia data according to the parsing strategy of the target execution subclass, the target parsing subclass can be selected from the multiple parsing subclasses in the parsing class according to the playback protocol of the target multimedia data, and the target management subclass can be selected from the multiple management subclasses in the parsing class according to the playback protocol of the target multimedia data. Then, according to the first parsing strategy in the selected target parsing subclass and the second parsing strategy in the target management subclass, the target multimedia data is parsed to obtain the parsed target multimedia data.
[0081] This embodiment parses the target multimedia data according to the parsing strategy of the target execution subclass to obtain the parsed target multimedia data, thereby achieving accurate parsing of the target multimedia data; thereafter, the parsed target multimedia data is decapsulated according to the decapsulation strategy of the target execution class, so that the data can be correctly processed by the corresponding playback protocol during transmission and processing, and then the processed data can be applied to the playback scenario corresponding to the playback protocol through the parsing strategy and decapsulation strategy of the selected playback protocol.
[0082] S104: output the multimedia data to be output through the playback interface to obtain decapsulated multimedia data corresponding to the target multimedia data.
[0083] In this embodiment, the playback interface is a public external interface, and the name, parameter list, and return type of each decapsulation strategy are defined in the playback interface. Therefore, when the multimedia data to be output is obtained, the multimedia data to be output is output through the playback interface, which can achieve standardized output and unified management of the multimedia data to be output, and further ensure the consistency of the data. For example, the playback interface is SourceInterface, and the multimedia data to be output (AVPactet) obtained after decapsulation is obtained through the playback interface, that is, the decapsulated multimedia data corresponding to the target multimedia data is obtained.
[0084] like Figure 6 As shown, in one embodiment, the multimedia data processing method proposed in the embodiment of the present application can be applied to a video playback scenario. For example, based on the recommendation list of the terminal application interface, the user clicks on the playback control of the video note to trigger the parsing event corresponding to the video note; in response to the parsing event, a player is created, and the video resource corresponding to the video note is downloaded; or, the video resource is downloaded through a pre-created player, and the video resource is a multimedia data. Then, the playback interface is called, and the target execution class corresponding to the playback protocol of the video resource is selected from the execution class corresponding to the playback interface, and the target execution subclass corresponding to the playback protocol of the video resource is selected from the target execution class; the video resource is parsed according to the parsing strategy of the target execution subclass to obtain the parsed video resource, and the parsed video resource is decapsulated according to the decapsulation strategy of the target execution class to obtain the multimedia data to be output; the multimedia data to be output is output through the playback interface corresponding to the player to obtain the decapsulated multimedia data, and the decapsulated multimedia data includes an audio stream and a video stream. The user terminal decodes the audio code stream in the decapsulated multimedia data to obtain audio decoded data; decodes the video code stream in the decapsulated multimedia data to obtain video decoded data; performs post-processing on the audio decoded data and the video decoded data, such as quality adjustment and content optimization, to obtain post-processed data; renders the post-processed data, and plays the playback file obtained after rendering on the application interface of the user terminal.
[0085] The multimedia data processing method proposed in the embodiment of the present application calls a playback interface in response to a parsing event of target multimedia data, wherein the playback interface is configured with at least one execution class, wherein the execution class includes a decapsulation strategy and at least two execution subclasses using the decapsulation strategy, and different execution subclasses encapsulate parsing strategies corresponding to different playback protocols, so that all execution classes can implement a common interface, thereby allowing users to perform standardized management of data processed by the playback protocol in the same way, so that data of different playback protocols can be uniformly processed and output, thereby achieving compatibility of multiple types of playback protocols; selecting a target execution class corresponding to the playback protocol of the target multimedia data from the execution class through the playback interface, and selecting a target execution subclass corresponding to the playback protocol of the target multimedia data from the target execution class; and selecting a target execution subclass corresponding to the playback protocol of the target multimedia data from the target execution class according to the target execution subclass ... The target multimedia data is parsed according to the parsing strategy of the class to obtain the parsed target multimedia data, and the parsed target multimedia data is decapsulated according to the decapsulation strategy of the target execution class to obtain the multimedia data to be output, so that different target execution classes are selected through the playback interface, and different types of playback protocols can be parsed without understanding the specific details of each decapsulation strategy or streaming media protocol; finally, the multimedia data to be output is output through the playback interface to obtain the decapsulated multimedia data corresponding to the target multimedia data, thereby realizing the compatibility and processing of multiple playback protocols, so that through a playback interface and the configured execution class and execution subclass, different data processing of different playback protocols of multimedia data can be realized, thereby improving the compatibility and scalability of the player, and further improving the decapsulation and decoding efficiency of different playback protocols.
[0086] In some embodiments of the present application, the target execution class includes a target parsing class and a target management class, the target parsing class includes a plurality of parsing subclasses, the parsing subclasses encapsulate a first parsing strategy corresponding to the playback protocol, the target management class includes a plurality of management subclasses, the management subclasses encapsulate a second parsing strategy corresponding to the playback protocol;
[0087] The step of selecting a target execution subclass corresponding to the playback protocol of the target multimedia data from the target execution class includes:
[0088] Selecting a target parsing subclass corresponding to the playback protocol of the target multimedia data from the plurality of parsing subclasses;
[0089] Selecting a target management subclass corresponding to the playback protocol of the target multimedia data from the plurality of management subclasses;
[0090] The target parsing subclass and the target management subclass are determined as the target execution subclass.
[0091] In this embodiment, the target execution class may also include a target parsing class and a target management class. Among them, the target parsing class includes at least one first parsing strategy corresponding to a playback protocol, and the target management class includes at least one second parsing strategy corresponding to a playback protocol. Among them, a first parsing strategy corresponding to a playback protocol is encapsulated in a parsing subclass of the target parsing class, and a second parsing strategy corresponding to a playback protocol is encapsulated in a management subclass of the target management class. Therefore, after obtaining the target execution class, that is, after selecting the execution class corresponding to the target multimedia file from multiple execution classes, the target parsing class and the target management class of the target execution class are obtained.
[0092] like Figure 4a As shown, Figure 4a This is a schematic diagram of the composition of the first analysis class. Figure 4b Schematic diagram of the composition of the first management class. The first parsing class includes the first parsing subclass and the second parsing subclass, and the first parsing subclass and the second parsing subclass are parsing classes corresponding to different playback protocols. Similarly, the first management class includes the first management subclass and the second management subclass, and the first management subclass and the second management subclass are management classes corresponding to different playback protocols.
[0093] With the Figure 4a Suitable, Figure 5a is a schematic diagram of the composition of the parsing class corresponding to the specific playback protocol; Figure 4b Suitable, Figure 5b It is a schematic diagram of the composition of the management class corresponding to the specific playback protocol. Among them, ManifestParser is the first parsing class, which includes the parsing subclass corresponding to the HLS protocol (HLSParser) and the parsing subclass corresponding to the Dash protocol (DashParser). ManifestManager is the first management class, which includes the management subclass corresponding to the HLS protocol (HLSManager) and the management subclass corresponding to the Dash protocol (DashManager).
[0094] For example, after obtaining the target execution class as ManifestExtractor from multiple execution classes, the target parsing class (ManifestParser) and the target management class (ManifestManager) of the target execution class (ManifestExtractor) can be obtained. The target parsing class (ManifestParser) includes multiple parsing subclasses, such as HLSParser and DashParser, and the HLSParser and DashParser are two different parsing subclasses. Similarly, the target management class (ManifestManager) includes multiple management subclasses, such as HLSManager and DashManager, and the HLSManager and DashManager are two different management subclasses.
[0095] Then, the parsing subclass corresponding to the playback protocol of the target multimedia data is selected from the multiple parsing subclasses of the target parsing class, that is, the target parsing subclass is obtained, and the management subclass corresponding to the playback protocol of the target multimedia data is selected from the multiple management subclasses of the target management class, that is, the target management subclass is obtained. For example, when the playback protocol of the target multimedia data is the HLS protocol, the target parsing subclass selected as HLSParser and the target management subclass selected as HLSManager can be obtained from the above example. The selected target parsing subclass and target management subclass are the target execution subclass selected from the target execution class.
[0096] In this embodiment, when the target execution class includes a target parsing class and a target management class, different types of playback protocols can be efficiently managed through multiple parsing subclasses of the target parsing class and multiple management subclasses of the target management class, and protocols with the same format requirements can be uniformly managed through the target execution class, thereby further improving the parsing efficiency of the playback protocol.
[0097] In some embodiments of the present application, the target multimedia data is parsed according to the parsing strategy of the target execution subclass to obtain the parsed target multimedia data, including:
[0098] The target multimedia data is parsed according to the first parsing strategy of the target parsing subclass and the second parsing strategy of the target management subclass to obtain the parsed target multimedia data.
[0099] In this embodiment, the target parsing subclass and the target management subclass are classes of the same target execution subclass, that is, the target parsing subclass and the target management subclass belong to the execution subclass corresponding to the same decapsulation strategy. For example, if the target execution subclass is HLSExtractor, the target parsing subclass is the parsing subclass of the execution subclass corresponding to the HLS protocol, such as HLSParser, and the target management subclass is the management subclass under the execution subclass corresponding to the HLS protocol, such as HLSManager. The first parsing strategy of the playback protocol corresponding to the target execution subclass is encapsulated in the target parsing subclass, and the second parsing strategy of the playback protocol corresponding to the target execution subclass is encapsulated in the target management subclass. The first parsing strategy and the second parsing strategy constitute the parsing strategy of the playback protocol corresponding to the target execution subclass. Through the first parsing strategy and the second parsing strategy, the decapsulated target multimedia data is parsed to obtain the multimedia data to be output.
[0100] In some embodiments, the target execution class may further include a target parsing class and a target management class. If the target parsing class includes only a first parsing strategy corresponding to a playback protocol, and the target management class includes only a second parsing strategy corresponding to a playback protocol, then the first parsing strategy according to the target execution subclass is obtained from the target parsing class, and the second parsing strategy according to the target execution subclass is obtained from the target management class, and the decapsulated target multimedia data is parsed according to the first parsing strategy and the second parsing strategy to obtain the multimedia data to be output.
[0101] For example, the target execution class is ManifestExtractor, the target execution class includes a target parsing class called ManifestParser, and the target execution class includes a target management class called ManifestManager. Among them, the target parsing class (ManifestParser) includes a first parsing strategy corresponding to a playback protocol, that is, the parsing strategy in HLSParser, and the target management class (ManifestManager) includes a second parsing strategy corresponding to a playback protocol, that is, the parsing strategy in HLSManager. When the selected target execution subclass is the execution subclass corresponding to the HLS protocol in the target execution class (that is, HLSExtractor), the first parsing strategy corresponding to the target execution subclass (HLSExtractor) is obtained from the target parsing class; and the second parsing strategy corresponding to the target execution subclass (HLSExtractor) is obtained from the target management class. In this way, the parsing strategy in HLSParser and the parsing strategy in HLSManager are obtained.
[0102] This embodiment parses the target multimedia data through the first parsing strategy in the target parsing subclass and the second parsing strategy in the target management subclass, which can achieve accurate parsing of specific playback protocols according to their corresponding decapsulation and parsing methods, and improve the decapsulation and decoding efficiency of multimedia data.
[0103] In some embodiments of the present application, the plurality of execution subclasses are configured with an execution parent class, and the target multimedia data is parsed according to the parsing strategy of the target execution subclass to obtain the parsed target multimedia data, further comprising:
[0104] Obtain the parsing strategy of the execution parent class;
[0105] The target multimedia data is parsed according to the parsing strategy of the target execution subclass and the parsing strategy of the execution parent class to obtain the parsed target multimedia data.
[0106] In this embodiment, multiple execution subclasses are configured with an execution parent class, and the execution subclasses under the same execution parent class are classes with the same functional attributes. The execution parent class configured with multiple execution subclasses is obtained, and the execution parent class encapsulates the parsing strategy of the common functions of the multiple execution subclasses, and each execution subclass can implement the method defined in the interface of the execution parent class. This allows data of different protocols to be processed and output in a unified manner, further improving the scalability and maintainability of the playback system. For example, multiple execution subclasses are FFmpegExtractor, HLSExtractor, and DASHExtractor, respectively, then the execution parent class corresponding to the multiple execution subclasses is ExtractorBase, and some public methods are defined in the execution parent class (ExtractorBase) to implement abstract and unified interfaces, and each execution subclass can implement the method defined in the interface of the execution parent class.
[0107] Obtain the parsing strategy of the execution parent class corresponding to multiple execution subclasses, and the parsing strategy is the parsing strategy corresponding to the public method defined in the execution parent class. When the decapsulated target multimedia data is parsed according to the parsing strategy of the target execution subclass, the obtained decapsulated target multimedia data can be parsed according to the parsing strategy of the target execution subclass and the parsing strategy of the execution parent class to obtain the multimedia data to be output. Specifically, the decapsulated target multimedia data is parsed according to the parsing strategy of the target execution subclass to obtain parsed data; then, the parsed data is output through the parsing strategy of the execution parent class to obtain the multimedia data to be output. Alternatively, the decapsulated target multimedia data is parsed according to the parsing strategy of the execution parent class to obtain parsed data; then, the parsed data is further processed and output through the parsing strategy of the target execution subclass to obtain the multimedia data to be output. This embodiment does not limit the execution order of the parsing strategies in the target execution subclass and the execution parent class.
[0108] This embodiment configures an execution parent class for multiple execution subclasses, so that the execution subclasses belonging to the execution parent class can be uniformly managed through the execution parent class; then, according to the parsing strategy of the target execution subclass and the parsing strategy of the execution parent class, the decapsulated target multimedia data is parsed to obtain the multimedia data to be output, so that data of different protocols can be uniformly processed and output through the execution parent class.
[0109] In some embodiments of the present application, after parsing the target multimedia data according to the parsing strategy of the target execution subclass to obtain the parsed target multimedia data, the method further includes:
[0110] Get the preset data output structure;
[0111] The parsed target multimedia data is stored according to the data output structure.
[0112] In the present embodiment, the preset data output structure is a pre-set data storage structure, such as the storage structure corresponding to the manifest file in Android. When the preset data output structure is obtained, the parsed target multimedia data is stored according to the format corresponding to the data output structure, so that the parsed target multimedia data can be conveniently processed and output through a unified data output structure. Taking the storage structure corresponding to the manifest file as an example, the manifest file is stored in the XML format, such as manifest.xml. The basic structure and core characteristics of the stored data are defined in the XML format in the manifest file. The basic structure of the manifest file includes a series of declarations and attributes, etc., so that the parsed target multimedia data can be correctly managed and executed according to the structure of the manifest file.
[0113] This embodiment obtains a preset data output structure and stores the parsed target multimedia data according to the data output structure, thereby achieving a unified structure output of the multimedia data to be output after parsing, so that subsequent processing steps can be independent of the specific playback protocol, thereby simplifying the design and implementation of the playback system.
[0114] In some embodiments of the present application, the multimedia data processing method further includes:
[0115] In response to the playback control event, a new execution subclass is generated according to the parsing strategy corresponding to the new playback protocol, and a new execution class is generated according to the new decapsulation strategy;
[0116] The new execution subclass and the new execution class are configured to the playback interface.
[0117] In this embodiment, the playback control event is an event for adjusting the playback interface, such as adding a new event to the execution class and execution subclass configured by the playback interface, or creating an event for the playback interface. The playback control event can be triggered by a component of the playback control interface. In response to the playback control event, the playback interface is processed according to the type of the playback control event. Specifically, the types of playback control events can be divided into initial creation events and new events. When the playback interface, execution class and execution subclasses of different playback protocols are initially created, the type of the playback control event triggered is an initial creation event. In response to the initial creation event, a playback interface is created, and the playback interface is a public interface, such as sourceInterface. The playback interface is used for external calls, mainly used to call the output of the decapsulated AVPacket. The playback interface includes the names, parameter lists and return types of different decapsulation strategies, and different decapsulation strategies are encapsulated in different execution classes. The playback interface can be implemented by different execution classes. An execution class is generated according to at least one decapsulation strategy, and at least two execution subclasses are generated according to the parsing strategies corresponding to at least two playback protocols, and a parsing strategy for a playback protocol is encapsulated in one execution subclass. The execution subclass and the execution class are configured to the playback interface.
[0118] Furthermore, an execution parent class may be configured for multiple execution subclasses, and the execution parent class defines the common methods of all execution subclasses.
[0119] Furthermore, according to the execution type of the parsing strategy, the execution subclass can be configured as a parsing subclass and a management subclass. The parsing subclass encapsulates a first parsing strategy for parsing, and the management subclass encapsulates a second parsing strategy for management. The first parsing strategy and the second parsing strategy are parsing strategies for a playback protocol.
[0120] When a new playback protocol or a new decapsulation strategy needs to be added, the type of playback control event triggered is a new event. In response to the new event, a new execution subclass is generated according to the parsing strategy corresponding to the new playback protocol, and a new execution class is generated according to the new decapsulation strategy of the new playback protocol, so that in the existing execution class configured by the playback interface, only the new execution class and new execution subclass corresponding to the new playback protocol need to be added to implement the addition of the new playback protocol parsing strategy and decapsulation strategy without modifying the existing code.
[0121] In one embodiment, a new parsing strategy corresponding to a new playback protocol may also be added separately without adding a new encapsulation method. Specifically, a new execution subclass is generated according to the new parsing strategy of the new playback protocol, and the execution subclass is added to the execution class where the decapsulation strategy that is the same as the decapsulation strategy of the new playback protocol is located. This achieves the separate addition of the new parsing strategy corresponding to the new playback protocol.
[0122] In another embodiment, a new decapsulation strategy may be added separately without adding a new parsing strategy. Specifically, a new decapsulation strategy is obtained, a new execution class is generated according to the new decapsulation strategy, and the execution class is configured to the corresponding playback interface. Thus, the new decapsulation strategy is added separately.
[0123] This embodiment generates a new execution subclass according to the parsing strategy corresponding to the new playback protocol and generates a new execution class according to the new decapsulation strategy in response to the playback control event, and then configures the new execution subclass and the new execution class to the playback interface, so that new decapsulation strategies and new playback protocols can be added without modifying the code, which is easy to expand.
[0124] In some embodiments of the present application, the parsing strategy of the new playback protocol includes a first parsing strategy for parsing and a second parsing strategy for management and execution;
[0125] The generating of a new execution subclass according to the parsing strategy corresponding to the new playback protocol includes:
[0126] Encapsulating the first parsing strategy into a new parsing subclass, and encapsulating the second parsing strategy into a new management subclass;
[0127] The new parsing subclass and the new management subclass are added as new execution subclasses to the execution class of the playback interface.
[0128] In this embodiment, the first parsing strategy is a strategy for parsing multimedia data according to the playback protocol, and the second parsing strategy is a strategy for making decisions and executing corresponding management actions based on the parsing results obtained by the first parsing strategy, such as a strategy for scheduling and executing corresponding operations based on the parsing results obtained by the first parsing strategy. According to the type of parsing strategy, the parsing strategies corresponding to each playback protocol can be divided into a first parsing strategy for parsing and a second parsing strategy for management. The first parsing strategy and the second parsing strategy of the new playback protocol are obtained, and the first parsing strategy is encapsulated as a new parsing subclass, and the second parsing strategy is encapsulated as a new management subclass; thereafter, the new parsing subclass and the new management subclass are added to the execution class corresponding to the new playback protocol in the playback interface.
[0129] For example, if the new playback protocol is Dash, the first parsing strategy of the new playback protocol is encapsulated as a new parsing subclass, which is DashParser, and the second parsing strategy of the new playback protocol is encapsulated as a new management subclass, which is DashManager. Then, the new parsing subclass and the new management subclass are added to the execution class of the playback interface, such as forming a new execution subclass DashExtractor with the new parsing subclass and adding it to the corresponding execution class ManifestExtractor.
[0130] This embodiment encapsulates the first parsing strategy into a new parsing subclass and the second parsing strategy into a new management subclass, and then adds the new parsing subclass and the new management subclass to the execution class of the playback interface, thereby realizing the rapid addition of parsing strategies for new playback protocols. Further, through the addition of the new parsing subclass and the new management subclass, when processing multimedia data, rapid parsing can be performed through the new parsing subclass and the new management subclass.
[0131] In some embodiments of the present application, the execution class of the playback interface includes a parsing class and a management class, and adding the new parsing subclass and the new management subclass as new execution subclasses to the execution class of the playback interface includes:
[0132] Adding the new parsing subclass as a new execution subclass to the parsing class;
[0133] The new management subclass is added as a new execution subclass to the management class.
[0134] In this embodiment, the execution class of the playback interface includes a parsing class and a management class, wherein the parsing class and the management class of the same execution class are classes corresponding to the same decapsulation strategy. The parsing class may include parsing subclasses corresponding to the first parsing strategies of multiple playback protocols, and the management class may include management subclasses corresponding to the second parsing strategies of multiple playback protocols. Therefore, when adding a new parsing subclass and a new management subclass to the execution class, the obtained new parsing subclass may also be added to the parsing class as a new execution subclass, and the obtained new management subclass may be added to the management class as a new execution subclass.
[0135] In this embodiment, the new parsing subclass and the new management subclass are added as a new execution subclass to the corresponding parsing class and management class respectively, so that the parsing strategies corresponding to different playback protocols can be uniformly managed and output through the parsing class and management class of the playback interface.
[0136] In order to better implement the multimedia data processing method provided in the embodiment of the present application, the embodiment of the present application also provides a device based on the multimedia data processing method. The meanings of the terms are the same as those in the multimedia data processing method, and the specific implementation details can refer to the description in the method embodiment.
[0137] For example, Figure 7 As shown, the multimedia data processing device may include: a calling module 401, a selecting module 402, a protocol processing module 403 and an output module 404.
[0138] A calling module 401 is used to call a playback interface in response to a parsing event of target multimedia data, wherein the playback interface is configured with at least one execution class, wherein the execution class includes a decapsulation strategy and at least two execution subclasses using the decapsulation strategy, and different execution subclasses encapsulate parsing strategies corresponding to different playback protocols;
[0139] A selection module 402, configured to select a target execution class corresponding to the playback protocol of the target multimedia data from the execution class through the playback interface, and to select a target execution subclass corresponding to the playback protocol of the target multimedia data from the target execution class;
[0140] The protocol processing module 403 is used to parse the target multimedia data according to the parsing strategy of the target execution subclass to obtain the parsed target multimedia data, and decapsulate the parsed target multimedia data according to the decapsulation strategy of the target execution class to obtain the multimedia data to be output;
[0141] The output module 404 is used to output the to-be-output multimedia data through the playback interface, and obtain the decapsulated multimedia data corresponding to the target multimedia data.
[0142] In an embodiment of the present application, the target execution class includes a target parsing class and a target management class, the target parsing class includes a plurality of parsing subclasses, the parsing subclasses encapsulate a first parsing strategy corresponding to the playback protocol, the target management class includes a plurality of management subclasses, the management subclasses encapsulate a second parsing strategy corresponding to the playback protocol; the selection module 402 includes:
[0143] A first selection unit, configured to select a target parsing subclass corresponding to the playback protocol of the target multimedia data from the plurality of parsing subclasses;
[0144] A second selection unit, configured to select a target management subclass corresponding to the playback protocol of the target multimedia data from the plurality of management subclasses;
[0145] A confirmation unit is used to determine the target parsing subclass and the target management subclass as the target execution subclass.
[0146] Furthermore, the protocol processing module 403 includes:
[0147] The first parsing unit is used to parse the target multimedia data according to the first parsing strategy of the target parsing subclass and the second parsing strategy of the target management subclass to obtain the parsed target multimedia data.
[0148] Furthermore, the plurality of execution subclasses are configured with an execution parent class, and the protocol processing module 403 further includes:
[0149] An acquisition unit, used for acquiring the parsing strategy of the execution parent class;
[0150] The second parsing unit is used to parse the target multimedia data according to the parsing strategy of the target execution subclass and the parsing strategy of the execution parent class to obtain the parsed target multimedia data.
[0151] Furthermore, the multimedia data processing device further includes:
[0152] An acquisition module, used to acquire a preset data output structure;
[0153] A storage module is used to store the parsed target multimedia data according to the data output structure.
[0154] In one embodiment of the present application, the multimedia data processing device further includes:
[0155] A response module, used to generate a new execution subclass according to a parsing strategy corresponding to a new playback protocol and a new execution class according to a new decapsulation strategy in response to a playback control event;
[0156] A configuration module is used to configure the new execution subclass and the new execution class to the playback interface.
[0157] Furthermore, the parsing strategy of the new playback protocol includes a first parsing strategy for parsing and a second parsing strategy for management and execution; the response module includes:
[0158] An encapsulation unit, configured to encapsulate the first parsing strategy into a new parsing subclass, and encapsulate the second parsing strategy into a new management subclass;
[0159] The adding unit is used to add the new parsing subclass and the new management subclass as new execution subclasses to the execution class of the playback interface.
[0160] Furthermore, the execution class of the playback interface includes a parsing class and a management class, and the adding unit includes:
[0161] A first adding sub-unit, used to add the new parsing sub-class as a new execution sub-class to the parsing class;
[0162] The second adding sub-unit is used to add the new management sub-class as a new execution sub-class to the management class.
[0163] The multimedia data processing device proposed in the present application realizes the compatibility and processing of multiple playback protocols, so that different data processing of different playback protocols of multimedia data can be realized through a playback interface and configured execution class and execution subclass, thereby improving the compatibility and scalability of the player and further improving the decapsulation and decoding efficiency of different playback protocols.
[0164] In specific implementation, the above modules can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation methods and corresponding beneficial effects of the above modules can be found in the previous method embodiments, which will not be repeated here.
[0165] The present application also provides an electronic device, which may be a server or a terminal, etc. Figure 8 As shown, it shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application, specifically:
[0166] The electronic device may include components such as a processor 601 with one or more processing cores, a memory 602 with one or more computer-readable storage media, a power supply 603, and an input unit 604. Those skilled in the art will appreciate that Figure 8 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.
[0167] The processor 601 is the control center of the electronic device, which uses various interfaces and lines to connect various parts of the entire electronic device, and executes various functions of the electronic device and processes data by running or executing computer programs and / or modules stored in the memory 602, and calling data stored in the memory 602. Optionally, the processor 601 may include one or more processing cores; preferably, the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 601.
[0168] The memory 602 can be used to store computer programs and modules. The processor 601 executes various functional applications and data processing by running the computer programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a computer program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.
[0169] The electronic device also includes a power supply 603 for supplying power to each component. Preferably, the power supply 603 can be logically connected to the processor 601 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 603 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.
[0170] The electronic device may further include an input unit 604, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.
[0171] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 601 in the electronic device will load the executable files corresponding to the processes of one or more computer programs into the memory 602 according to the following instructions, and the processor 601 will run the computer programs stored in the memory 602, thereby realizing various functions, such as:
[0172] In response to a parsing event of the target multimedia data, a playback interface is called, wherein the playback interface is configured with at least one execution class, wherein the execution class includes a decapsulation strategy and at least two execution subclasses using the decapsulation strategy, wherein different execution subclasses encapsulate parsing strategies corresponding to different playback protocols;
[0173] Selecting a target execution class corresponding to the playback protocol of the target multimedia data from the execution class through the playback interface, and selecting a target execution subclass corresponding to the playback protocol of the target multimedia data from the target execution class;
[0174] Parsing the target multimedia data according to the parsing strategy of the target execution subclass to obtain parsed target multimedia data, and decapsulating the parsed target multimedia data according to the decapsulation strategy of the target execution class to obtain multimedia data to be output;
[0175] The multimedia data to be output is output through the playback interface to obtain decapsulated multimedia data corresponding to the target multimedia data.
[0176] The electronic device proposed in the embodiment of the present application realizes the compatibility and processing of multiple playback protocols, so that different data processing of different playback protocols of multimedia data can be realized through a playback interface and configured execution class and execution subclass, thereby improving the compatibility and scalability of the player and further improving the decapsulation and decoding efficiency of different playback protocols.
[0177] The specific implementation methods and corresponding beneficial effects of the above operations can be found in the detailed description of the multimedia data processing method above, which will not be elaborated here.
[0178] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.
[0179] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored, and the computer program can be loaded by a processor to execute the steps in any multimedia data processing method provided in the embodiment of the present application. For example, the computer program can execute the following steps:
[0180] In response to a parsing event of the target multimedia data, a playback interface is called, wherein the playback interface is configured with at least one execution class, wherein the execution class includes a decapsulation strategy and at least two execution subclasses using the decapsulation strategy, wherein different execution subclasses encapsulate parsing strategies corresponding to different playback protocols;
[0181] Selecting a target execution class corresponding to the playback protocol of the target multimedia data from the execution class through the playback interface, and selecting a target execution subclass corresponding to the playback protocol of the target multimedia data from the target execution class;
[0182] Parsing the target multimedia data according to the parsing strategy of the target execution subclass to obtain parsed target multimedia data, and decapsulating the parsed target multimedia data according to the decapsulation strategy of the target execution class to obtain multimedia data to be output;
[0183] The multimedia data to be output is output through the playback interface to obtain decapsulated multimedia data corresponding to the target multimedia data.
[0184] The computer-readable storage medium proposed in the embodiment of the present application realizes the compatibility and processing of multiple playback protocols, so that different data processing of different playback protocols of multimedia data can be realized through a playback interface and configured execution class and execution subclass, thereby improving the compatibility and scalability of the player and further improving the decapsulation and decoding efficiency of different playback protocols.
[0185] The specific implementation methods and corresponding beneficial effects of the above operations can be found in the previous embodiments, which will not be described in detail here.
[0186] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
[0187] Since the computer program stored in the computer-readable storage medium can execute the steps in any multimedia data processing method provided in the embodiments of the present application, the beneficial effects that can be achieved by any multimedia data processing method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.
[0188] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned multimedia data processing method.
[0189] The above is a detailed introduction to a multimedia data processing method, device, electronic device and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. A multimedia data processing method, characterized in that: include: In response to a parsing event of target multimedia data, a playback interface is called, wherein the playback interface is configured with at least one execution class, wherein the execution class includes a decapsulation strategy, and at least two execution subclasses using the decapsulation strategy, wherein different execution subclasses encapsulate parsing strategies corresponding to different playback protocols, and the parsing strategy of the same playback protocol includes a first parsing strategy and a second parsing strategy, wherein the execution subclass includes a parsing subclass and a management subclass, wherein the parsing subclass encapsulates the first parsing strategy, and the management subclass encapsulates the second parsing strategy; Selecting a target execution class corresponding to the playback protocol of the target multimedia data from the execution class through the playback interface, and selecting a target execution subclass corresponding to the playback protocol of the target multimedia data from the target execution class, wherein the target execution subclass includes a parsing subclass and a management subclass corresponding to the playback protocol of the target multimedia data; Parsing the target multimedia data according to the parsing strategy of the target execution subclass to obtain parsed target multimedia data, and decapsulating the parsed target multimedia data according to the decapsulation strategy of the target execution class to obtain multimedia data to be output; The multimedia data to be output is output through the playback interface to obtain decapsulated multimedia data corresponding to the target multimedia data.
2. The multimedia data processing method according to claim 1, characterized in that: The target execution class includes a target parsing class and a target management class. The target parsing class includes a plurality of parsing subclasses. The parsing subclasses encapsulate a first parsing strategy corresponding to the playback protocol. The target management class includes a plurality of management subclasses. The management subclasses encapsulate a second parsing strategy corresponding to the playback protocol. The step of selecting a target execution subclass corresponding to the playback protocol of the target multimedia data from the target execution class includes: Selecting a target parsing subclass corresponding to the playback protocol of the target multimedia data from the plurality of parsing subclasses; Selecting a target management subclass corresponding to the playback protocol of the target multimedia data from the plurality of management subclasses; The target parsing subclass and the target management subclass are determined as the target execution subclass.
3. The multimedia data processing method according to claim 2, characterized in that: The step of parsing the target multimedia data according to the parsing strategy of the target execution subclass to obtain the parsed target multimedia data includes: The target multimedia data is parsed according to the first parsing strategy of the target parsing subclass and the second parsing strategy of the target management subclass to obtain the parsed target multimedia data.
4. The multimedia data processing method according to any one of claims 1 to 3, characterized in that: A plurality of the execution subclasses are configured with an execution parent class, and the target multimedia data is parsed according to the parsing strategy of the target execution subclass to obtain the parsed target multimedia data, and further includes: Obtain the parsing strategy of the execution parent class; The target multimedia data is parsed according to the parsing strategy of the target execution subclass and the parsing strategy of the execution parent class to obtain the parsed target multimedia data.
5. The multimedia data processing method according to claim 1, characterized in that: After parsing the target multimedia data according to the parsing strategy of the target execution subclass to obtain the parsed target multimedia data, the method further includes: Get the preset data output structure; The parsed target multimedia data is stored according to the data output structure.
6. The multimedia data processing method according to claim 1, characterized in that: The method further comprises: In response to the playback control event, a new execution subclass is generated according to the parsing strategy corresponding to the new playback protocol, and a new execution class is generated according to the new decapsulation strategy; The new execution subclass and the new execution class are configured to the playback interface.
7. The multimedia data processing method according to claim 6, characterized in that: The parsing strategy of the new playback protocol includes a first parsing strategy for parsing and a second parsing strategy for management and execution; The generating of a new execution subclass according to the parsing strategy corresponding to the new playback protocol includes: Encapsulating the first parsing strategy into a new parsing subclass, and encapsulating the second parsing strategy into a new management subclass; The new parsing subclass and the new management subclass are added as new execution subclasses to the execution class of the playback interface.
8. The multimedia data processing method according to claim 7, characterized in that: The execution class of the playback interface includes a parsing class and a management class, and the adding the new parsing subclass and the new management subclass as new execution subclasses to the execution class of the playback interface includes: Adding the new parsing subclass as a new execution subclass to the parsing class; The new management subclass is added as a new execution subclass to the management class.
9. A multimedia data processing device, characterized in that: include: A calling module, configured to call a playback interface in response to a parsing event of target multimedia data, wherein the playback interface is configured with at least one execution class, wherein the execution class includes a decapsulation strategy and at least two execution subclasses using the decapsulation strategy, wherein different execution subclasses encapsulate parsing strategies corresponding to different playback protocols, and the parsing strategy of the same playback protocol includes a first parsing strategy and a second parsing strategy, wherein the execution subclass includes a parsing subclass and a management subclass, wherein the parsing subclass encapsulates the first parsing strategy, and the management subclass encapsulates the second parsing strategy; a selection module, configured to select a target execution class corresponding to the playback protocol of the target multimedia data from the execution class through the playback interface, and select a target execution subclass corresponding to the playback protocol of the target multimedia data from the target execution class, wherein the target execution subclass includes a parsing subclass and a management subclass corresponding to the playback protocol of the target multimedia data; a protocol processing module, configured to parse the target multimedia data according to the parsing strategy of the target execution subclass to obtain the parsed target multimedia data, and to decapsulate the parsed target multimedia data according to the decapsulation strategy of the target execution class to obtain the multimedia data to be output; The output module is used to output the multimedia data to be output through the playback interface to obtain the decapsulated multimedia data corresponding to the target multimedia data.
10. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the multimedia data processing method according to any one of claims 1 to 8.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the multimedia data processing method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Analysis method and system of data communication protocol
CN108183890A
HLS protocol-based video playing method and apparatus
CN108683945A
Streaming media playing method and device based on DASH protocol
CN109274696A
Serial port data analysis method and device
CN110032535A
Method and device for analyzing TCP and UDP data and storage medium
CN111786985A
