A media service method, device, equipment and computer storage medium

Abstract

The embodiments of the present invention provide a media service method, apparatus, device and computer storage medium, which obtain a media segment file, wherein the media segment file includes original subtitle information; convert the media segment file into an intermediate media file in a first media format, wherein the original subtitle information is converted into intermediate subtitle information in the first subtitle format and encapsulated in the intermediate media file; if a service request is received, provide a media service according to the intermediate media file, wherein target subtitle information in a target format corresponding to the service request is formed according to the intermediate subtitle information to provide the subtitle service, and in some implementation processes, has the technical effect of enabling the subtitles in the media segment to satisfy the subtitle display of more streaming media protocol types, thereby improving the flexibility of the media service, and can reduce the degree of fragmentation of the media files.

Description

Technical Field

[0001] The embodiments of the present invention relate to, but are not limited to, the field of multimedia services, and specifically, relate to, but are not limited to, a media service method, apparatus, device, and computer storage medium. Background Art

[0002] In recent years, Internet streaming such as IPTV (Internet Protocol Television) that uses IP (Internet Protocol) to distribute images is being standardized. For example, standardization of VoD (Video On Demand) streaming and live streaming methods based on HTTP (HyperText Transfer Protocol) streaming is being applied. In conventional IPTV systems, for example, subtitles are carried in a separate stream in the MPEG TS (Moving Picture Experts Group Transport Stream) file format.

[0003] However, in the related art, media subtitle information, especially the subtitles in the recorded live segments, cannot meet the subtitle display requirements of different streaming media protocol types and has low flexibility. Summary of the invention

[0004] The media service method, device, equipment and computer storage medium provided by the embodiments of the present invention mainly solve the technical problem that in the related art, subtitles cannot meet the subtitle display requirements of different streaming media protocol types and have low flexibility.

[0005] In order to solve the above technical problems, an embodiment of the present invention provides a media service method, including:

[0006] Acquire a media segment file, wherein the media segment file includes original subtitle information;

[0007] Converting the media segment file into an intermediate media file in a first media format, wherein the original subtitle information is converted into intermediate subtitle information in the first subtitle format and encapsulated in the intermediate media file;

[0008] When receiving a service request, providing a media service according to the intermediate media file;

[0009] Providing media service according to the intermediate media file includes: forming target subtitle information in a target format corresponding to the service request according to the intermediate subtitle information to provide subtitle service.

[0010] The embodiment of the present invention further provides a media service device, including:

[0011] An acquisition module, used for acquiring a media segment file, wherein the media segment file includes original subtitle information;

[0012] An encapsulation module, used for converting the media segment file into an intermediate media file in a first media format, wherein the original subtitle information is converted into intermediate subtitle information in the first subtitle format and encapsulated in the intermediate media file;

[0013] A service module, configured to provide a media service according to the intermediate media file upon receiving a service request;

[0014] Providing media service according to the intermediate media file includes: forming target subtitle information in a target format corresponding to the service request according to the intermediate subtitle information to provide subtitle service.

[0015] An embodiment of the present invention further provides a media service device, including a processor, a memory and a communication bus;

[0016] The communication bus is used to realize the connection and communication between the processor and the memory;

[0017] The processor is used to execute one or more computer programs stored in the memory to implement the steps of the media service method described above.

[0018] An embodiment of the present invention further provides a computer storage medium, wherein the computer storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the media service method as described above.

[0019] According to the media service method, apparatus, device and computer storage medium provided by the embodiments of the present invention, a media segment file is obtained, the media segment file includes original subtitle information; the media segment file is converted into an intermediate media file of a first media format, the original subtitle information is converted into intermediate subtitle information of the first subtitle format and encapsulated in the intermediate media file; when a service request is received, target subtitle information of a target format corresponding to the service request is formed according to the intermediate subtitle information to provide a subtitle service; since the media segment file including the original subtitle information is converted into an intermediate media file, and the corresponding original subtitle information is also converted into intermediate subtitle information, and is converted into other required formats when used, in certain implementation processes, technical effects including but not limited to enabling the subtitles in the media to better meet the subtitle display requirements of different streaming media protocol types, thereby improving the flexibility of the media service, can be achieved; at the same time, the media segment file is converted into an intermediate media file so that the originally fragmented segment files are integrated into an intermediate media file, thereby reducing the degree of fragmentation of the media files.

[0020] Other features and corresponding beneficial effects of the present invention are described in the latter part of the specification, and it should be understood that at least part of the beneficial effects become obvious from the description in the specification of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a flow chart of a media service method according to Embodiment 1 of the present invention;

[0022] Figure 2 This is a schematic diagram of the process of obtaining media segment files according to the first embodiment of the present invention;

[0023] Figure 3 This is a flow chart of a media service method according to Embodiment 2 of the present invention;

[0024] Figure 4 This is a schematic diagram of the process of obtaining media segment files according to the second embodiment of the present invention;

[0025] Figure 5 This is a schematic diagram of receiving a UDP MPEG TS stream according to Embodiment 2 of the present invention;

[0026] Figure 6 This is a schematic diagram of classifying and arranging TS packets according to the second embodiment of the present invention;

[0027] Figure 7 This is a schematic diagram of encapsulating a media segment file into an MP4 file according to Embodiment 2 of the present invention;

[0028] Figure 8 This is a schematic diagram of a service module providing media services according to an MP4 file in Embodiment 2 of the present invention;

[0029] Fig. 9 This is a schematic diagram of the structure of a media service device according to Embodiment 3 of the present invention;

[0030] Fig.10 This is a schematic diagram of the structure of a media service device according to Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the following is a further detailed description of the embodiments of the present invention through specific implementation methods combined with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0032] Embodiment 1:

[0033] In order to enable the subtitles of the media to meet the subtitle display requirements of more different streaming media protocol types and improve the flexibility of the media service, this embodiment provides a media service method. Figure 1 , the method comprising:

[0034] S101, obtaining a media segment file, where the media segment file includes original subtitle information;

[0035] In some specific applications, for example, a media segment file can be directly obtained from storage, or a media segment file transmitted can be received, or a received media stream can be divided into media segment files. It can be understood that the media segment file includes video data, audio data, subtitles and other possible information. In a specific application, the media segment file can be a TS (Transport Stream) segment file. It can be understood that in some specific implementations, the media segment file can include more than one subtitle, for example, two or more subtitles to choose from.

[0036] In this embodiment, the original subtitle information refers to the subtitles carried in the media segment file. In the specific implementation process, it can be subtitles including graphic subtitles and / or text subtitles. As a specific example, the graphic subtitles may include but are not limited to DVB (Digital Video Broadcasting) subtitle subtitles, and the text subtitles may include but are not limited to DVB (Digital Video Broadcasting) teltext subtitles. These two subtitles as examples can be used in DVB (Digital Video Broadcasting) code streams.

[0037] S102, converting the media segment files into intermediate media files, converting the original subtitle information into intermediate subtitle information and encapsulating it in the intermediate media files;

[0038] As an example, a media segment file contains video data, audio data, subtitles and may carry other information. It should be noted that, in addition to being able to encapsulate video data and audio data, the intermediate media file of the first media format should also be able to accommodate other data such as subtitles, such as MP4 (Moving Picture Experts Group 4, a digital audio compression technology defined by the Moving Picture Experts Group) files, etc. Intermediate media files of this type of format can encapsulate subtitle data or other data with video data and audio data through adaptive customization or adaptation. Take the media segment file as a TS segment file and the intermediate media file as an MP4 file as an example. In a specific example, the video data, audio data, and subtitles in the MP4 file are stored in different track boxes (data track packagers), and the video data and audio data may be stored in the TS segment file in the form of ES (Elementary Stream, basic code stream) streams. For this type of video data and audio data ES streams, they can be directly transcapsulated into the MP4 file, that is, different track boxes are used to record the ES stream information of the video data and audio data.

[0039] The first subtitle format is a format that can be stored in an intermediate media file. As an example, the subtitle information in XML (Extensible Markup Language) format can be encapsulated in an MP4 file after customizing the standard format of MP4, so the intermediate subtitle information can be XML subtitle information. It can be understood that XML subtitle information refers to subtitle information in XML format. It should be noted that in actual applications, the standard MP4 format does not support all XML subtitle information. When the XML subtitle information is not supported by the standard MP4 format, the XML subtitle information is identified in the MP4 file using a custom field. The XML subtitle information can be smpte-tt (Society of Motion Picture and Television Engineers Time Text) subtitle information, ttml (Time Text Markup Language) subtitle information, etc. For example, graphic subtitles can be converted into smpte-tt subtitle information, and text subtitles can be converted into ttml subtitle information. Exemplarily, in some specific implementations, DVB subtitle subtitles are encoded and decoded to form smpte-tt subtitle information, and DVB teltext subtitles are encoded and decoded to form ttml subtitle information. It should be understood that this example is not limited to the above-mentioned several types of XML subtitle information, and in actual application, other types of XML subtitle information can be formed and encapsulated into MP4 files according to needs. In other examples, subtitle information in non-XML format can be encapsulated into intermediate media files in other formats.

[0040] It can also be understood that storing various data into an intermediate media file according to the standard of the first media format can also avoid the file fragmentation problem caused by the independence of video data, audio data and subtitles in the media segment files. It can be seen that this embodiment converts the media segment files into intermediate media files to facilitate management and is more convenient when corresponding media services need to be provided.

[0041] S103: If a service request is received, provide a media service according to the intermediate media file;

[0042] It should be understood that the service request mentioned in this embodiment is a service request for media content. When the service request is received, the corresponding media content is provided to the end that initiated the service request. It should be noted that providing media services according to the intermediate media file includes: forming target subtitle information in a target format corresponding to the service request according to the intermediate subtitle information to provide subtitle services. In a specific example, the media segment file is converted into an MP4 file, and when the corresponding media service is performed in response to the service request, it is performed based on the MP4 file. XML subtitle information is encapsulated in the MP4 file, and the XML subtitle information in the MP4 file is encapsulated as target subtitle information in a target format corresponding to the service request to provide subtitle services. There may be multiple service requests, and the target formats corresponding to various service requests may not be the same. In some specific implementations, it can also be understood that the formats supported by different service requests may be different. In some cases, the format of the XML subtitle information needs to be converted again to form a new file.

[0043] However, it should be noted that if the target format corresponding to the service request is the same as the format of the intermediate subtitle information, the format of the intermediate subtitle information will not be converted again, and the intermediate subtitle information can be directly extracted from the intermediate media file and formed into a corresponding file for providing subtitle services. If the target format corresponding to the service request is different from the current format of the intermediate subtitle information, or the service request does not support the current format of the intermediate subtitle information, the intermediate subtitle information is encapsulated from the current format into the target subtitle information of the target format corresponding to the service request, and the subtitle service is provided with the re-encapsulated target subtitle information of the target format.

[0044] As a specific example, the service request may be an HLS (HTTP Live Streaming, a streaming media network transmission protocol based on HTTP) service request, a DASH (Dynamic Adaptive Streaming over HTTP, a dynamic adaptive streaming based on HTTP) service request, etc. In some examples, the HLS service request does not support the ttml format, and therefore, when the XML subtitle information is specifically ttml subtitle information, the ttml format is encapsulated into a file in a format supported by the HLS service, such as webvtt (Web Video Text Tracks), to provide subtitle services. In some examples, the HLS service request may also support the ttml format, and the ttml subtitle information may not be encapsulated into files of other formats.

[0045] In some specific implementations, the service request includes an HLS service request, and if the service request is received, providing a media service according to the MP4 file, wherein forming target subtitle information in a target format corresponding to the service request according to the XML subtitle information to provide the subtitle service includes at least one of the following:

[0046] 1. If an HLS service request is received that does not support smpte-tt subtitle information and ttml subtitle information, the ttml subtitle information can be encapsulated as webvtt subtitle information, and subtitle service can be provided directly based on the webvtt subtitle information.

[0047] It should be noted that traditional HLS service requests that do not support smpte-tt subtitle information and ttml subtitle information usually do not support subtitle information in image format. Therefore, in general, smpte-tt subtitle information does not provide subtitle services for the above traditional HLS service requests. It can be seen that the media service method of the embodiment of the present invention can make the subtitles in the media more meet the subtitle display requirements of different streaming media protocol types through the above method, but it should not be understood as being able to meet the subtitle display requirements in all cases.

[0048] 2. If an HLS service request that supports smpte-tt subtitle information and ttml subtitle information is received, at least one of the smpte-tt subtitle information and the ttml subtitle information is encapsulated into an M4S (fragment MP4 format file, M4S for short) file along with the audio and video in the MP4 file, and subtitle service is provided according to the M4S file.

[0049] In some specific implementations, the service request includes a DASH service request, and if the service request is received, providing a media service according to the MP4 file, wherein forming target subtitle information in a target format corresponding to the service request according to the XML subtitle information to provide the subtitle service includes:

[0050] If a DASH service request is received, at least one of the smpte-tt subtitle information and the ttml subtitle information is encapsulated into an M4S file along with the audio and video in the MP4 file, and a subtitle service is provided according to the M4S file.

[0051] The media service method of this embodiment may provide service for only one service request, or may provide service for multiple service requests respectively, for example, providing service for only one of the above-mentioned HLS service request and DASH service request, or may provide service for both.

[0052] Furthermore, in the above specific implementation process, after receiving the service request, providing the media service according to the intermediate media file also includes:

[0053] S104, forming a target audio and video file corresponding to the service request according to the video data and audio data in the intermediate media file to provide audio and video services;

[0054] It should be noted that the target audio and video files include independent audio files and video files, or in some implementations the audio files and video files are not independent.

[0055] The execution order of step S104 and step S103 is not limited. The intermediate media files also include files recording time information. The end initiating the service request can achieve synchronous playback of audio, video and subtitles according to the time information.

[0056] It is understandable that some formats of intermediate media files (such as MP4 files) can be directly provided to the end that initiates the service request. However, in actual applications, the intermediate media files are usually converted into files of other formats according to the requirements of the service request to provide media services.

[0057] In this embodiment, the widely used HLS service request and DASH service request are used as examples. In the example of the HLS service request, an MP4 file corresponds to a TS fragment file, and the audio file and video file in the MP4 file (i.e., the audio and video tracks in MP4) are encapsulated in the MPEG TS format. In the example of the DASH service request, an MP4 file corresponds to an M4S file, and the audio file, video file and subtitle information in the MP4 file (i.e., the audio, video and subtitle tracks in MP4) are encapsulated into an M4S file.

[0058] In some embodiments, Figure 2 As shown, step S101 specifically includes:

[0059] S201, receiving a media stream;

[0060] The media code stream should carry video data, audio data, original subtitle information and other possible data such as time information. As a specific example, the media code stream can be a UDP MPEG TS (User Datagram Protocol Moving Picture Experts Group Transport Stream) code stream, and the video data, audio data, original subtitle information and other data are respectively encapsulated in TS packets.

[0061] S202, classifying different types of data packets in the media stream, the data packets including subtitle data packets, audio data packets and video data packets, the subtitle data packets including original subtitle information;

[0062] Different types refer to video data, audio data, and subtitle data. All data packets carrying video data are classified into one category, all data packets carrying audio data are classified into one category, and all data packets carrying subtitle data are classified into one category.

[0063] S203, aligning the time of various data packets;

[0064] In specific applications, the time information of various data packets can be obtained, and the data packets can be sorted in time order according to the time information. After that, the time of each data packet can be aligned using a related algorithm program.

[0065] S204, slicing the data packets and encapsulating them into media segment files, each of which includes segments of various types of data packets, and each segment corresponds to the same time;

[0066] The duration of each slice may be set according to actual needs, for example, calculated according to video encoding parameters and corresponding configuration parameters, or may also be directly specified.

[0067] It is understandable that in step S203, the time of each data packet is aligned, so the media segment files formed in step S204 are time synchronized. Therefore, the media content in the intermediate media file in step S102 is also synchronized, and the time of the corresponding media service provided is also synchronized.

[0068] It should also be noted that the subtitle information mentioned above in this embodiment, such as original subtitle information, intermediate subtitle information, etc., can be carried in a separate file in actual applications, or can be recorded as part of the information in a file, or exist in the form of a data stream during transmission.

[0069] The media service method provided by the embodiment of the present invention converts a media segment file including original subtitle information into an intermediate media file, and the original subtitle information is encapsulated in the intermediate media file by encoding and decoding to form intermediate subtitle information. If a service request is received, the media service is provided according to the intermediate media file, and the target subtitle information in the target format corresponding to the service request is formed according to the intermediate subtitle information in the intermediate file to provide the subtitle service; the fragmented media segments are encapsulated into an intermediate media file to solve the fragmentation problem. At the same time, when providing subtitle services, it is based on the intermediate subtitle information in the intermediate media file, and the intermediate subtitle information can be converted into subtitle information in the required format according to the service request. In some implementation processes, the compatibility and flexibility of subtitles in the media are guaranteed, and the subtitle display requirements of different streaming media protocol types can be better met.

[0070] Embodiment 2:

[0071] In order to better reflect the media service method of the present invention, this embodiment further illustrates the media service method in combination with a specific implementation process. In this embodiment, the intermediate media file is an MP4 file, and the intermediate subtitle information is XML subtitle information. Figure 3 ;

[0072] S301, obtaining a media segment file, where the media segment file includes original subtitle information;

[0073] See also Figure 4 The specific process of step S301 is described below with a specific example:

[0074] S401, receiving UDP MPEG TS stream;

[0075] like Figure 5 , which is a schematic diagram of receiving UDP MPEG TS code stream, joins UDP (User Datagram Protocol) multicast or binds UDP unicast address to receive UDP MPEG TS code stream.

[0076] S402, classifying different types of TS packets in the bitstream;

[0077] S403, aligning the time of various TS packets;

[0078] After the TS packets are classified, each type of TS packets is sorted in ascending order according to the DTS (Decode Time Stamp) value. It can be understood that each type of TS packet carries corresponding data, for example, an audio TS packet carries audio data. Figure 6 , reorganize and classify the TS packets of audio, video, and subtitles according to the PID value, and arrange each reorganization queue in ascending order according to the DTS value in the PES (Packetized Elementary Stream) packet of the code stream. Among them, the TS packet marked as the start indicator contains the PES packet, which describes the time information of the ES stream, including PTS (Presentation Time Stamp), DTS, etc., among which DTS is used for audio, video, and subtitle playback synchronization. The time of each TS packet can be aligned according to the algorithm.

[0079] S404, slicing the TS packets and encapsulating them into media segment files;

[0080] These types of TS packets are sliced ​​to generate TS fragment files, which contain ES streams of audio, video and subtitles. At the same time, each generated TS file has a Unix timestamp (Unix timestamp, such as 1590821980016, in milliseconds) to record the start time of the TS file, which is stored in a text file. It should be noted that the length of the slice is set according to needs. In practical applications, the slice duration is usually calculated based on video encoding parameters and configuration parameters.

[0081] S302, converting the media segment files into MP4 files;

[0082] Next, the TS segment file is transcapsulated into an MP4 file. The video and audio ES streams are not processed in any way and are transcapsulated intact into the MP4 file. Different track boxes are used in the MP4 file to record different ES stream information. In this example, the subtitles include subtitles in two formats, DVB subtitle and DVB teltext. The two subtitles, DVB subtitle and DVBteltext, are respectively generated into two XML format subtitle information, smpte-tt and ttml, through encoding and decoding operations. In some examples, a DVB subtitle subtitle decoder is used to decode the DVB subtitle PES package to obtain the subtitle display position information and subtitle png (Portable Network Graphics) image binary data of the subtitle; in some examples, a DVB teltext subtitle decoder is used to decode the DVB teltext PES package to obtain the subtitle pageid, subtitle display position information and subtitle text data, wherein the page id is used to find the language information of the subtitle of the page id in the PMT table (Program Map Table). The subtitle decoder can be implemented by software, hardware or a combination of these. The specific decoding method is not limited in this embodiment. Figure 7 , audio, video and decoded subtitles are encapsulated into MP4 files in separate streams. As an example, the subtitle pageid, subtitle display position information and subtitle text data obtained after decoding the DVB teltext PES package are filled into the ttml format, and the ttml format subtitle information is encapsulated into a track of the MP4 format file. The subtitle display position information and subtitle png image binary data after decoding the DVB subtitle PES package are filled into the smpte-tt format, and the subtitle information in the smpte-tt format is encapsulated into a track of the MP4 format file.

[0083] The standard MP4 format does not support subtitle information in the two XML formats of smpte-tt and ttml, so custom fields are used to identify the XML subtitle information in the MP4 file.

[0084] As an example, the process of encapsulating XML subtitle information into an MP4 file is described. A custom handler type is used in the MP4 format. For subtitles that are converted to ttml, the value of the handler type in the moov->trak->mdia->hdlr box in the MP4 is 'subt', indicating that the original bitstream of the subtitle is DVB teltext, which is a plain text subtitle. For subtitles that are converted to smpte-tt, the value of the handler type in the moov->trak->mdia->hdlr box in the MP4 is 'sub', indicating that the original bitstream of the subtitle is DVB subtitle, which is a picture subtitle. It should be noted that the above 'subt' and 'sub' are custom values, and these values ​​can be arbitrarily named according to the specifications allowed by the MP4 format in actual applications.

[0085] At the same time, the difference between the start time of the TS segment file and a fixed UTC (Universal Time Coordinated) time (in milliseconds) is subtracted and split into two parts: seconds and milliseconds. The seconds part is stored in the empty duration of the first entry of trak->edts->elst, and the millisecond part is stored in the empty duration of the second entry of trak->edts->elst. In this embodiment, the smpte-tt subtitle information and / or the ttml subtitle information can be encapsulated into the mp4 file as a sample, and one sample stores the subtitle information. There is no need to split the subtitle information into multiple samples, which has better integrity and higher efficiency during extraction.

[0086] Each subtitle in the smpte-tt and ttml subtitle information has a start and end time. Due to the length limit of the begin and end times in the XML subtitle information, the difference (in milliseconds) of the corresponding start UTC time (in milliseconds) of the subtitle in the live broadcast minus a fixed UTC time (in milliseconds) is used as the begin value, and the difference (in milliseconds) of the corresponding UTC time (in milliseconds) of the end of the TS segment minus a fixed UTC time (in milliseconds) is used as the end value.

[0087] S303, using MP4 files to provide media services;

[0088] After the above processing, an MP4 file is formed based on the received code stream and the generated media segment files. The specific process of using the MP4 file to provide media services is described below.

[0089] like Figure 8 , the service module provides related services, where the MP4 file is the MP4 file formed above, and each MP4 file has a UTC time. The UTC time is used to generate the time information required in the index file of the protocol used by the service request, such as the value of the EXT-X-PROGRAM-DATE-TIME tag in the index file of the HLS protocol and the value of the publishTime tag in the index file of the DASH protocol.

[0090] In one example, a DASH service request is received. At this time, an MP4 file corresponds to an M4S file (fragment mp4 format file). The audio file, video file and XML subtitle information (smpte-tt subtitle information and / or ttml subtitle information) in the MP4 file are encapsulated into an M4S file for external service. It should also be noted that the service can be divided into two scenarios: live broadcast and on-demand. In the case of live broadcast, smpte-tt and ttml subtitle information are served in M4S format files, and the subtitle information content does not need to be changed; in the case of on-demand, the start time of the subtitle is based on 0 as the reference time, which is different from live broadcast. When serving, the begin and end times of the subtitles encapsulated in the M4S format need to be modified. The new begin (milliseconds) = video playback time (milliseconds) + subtitle information begin time (milliseconds) - the sum of the two empty durations in the subtitle trak->edts->elst in the mp4 file (i.e., the difference between the start time of the above-mentioned TS fragment file minus a fixed UTC time).

[0091] In another example, if an HLS service request is received that does not support smpte-tt subtitle information and ttml subtitle information, at this time, one MP4 file corresponds to one TS fragment file, and the audio and video files in the MP4 file are encapsulated in MPEG TS format. It can be understood that the audio and video files in the MP4 file are audio and video track files respectively. The ttml subtitle information is encapsulated in ttml format as subtitle information in webvtt format to provide subtitle service. In another example, an HLS service request that supports both ttml subtitle information and smpte-tt subtitle information is received. At this time, the format of the ttml subtitle information and the smpte-tt subtitle information may not be changed. In this example, as when a DASH service request is received, the audio file, video file and XML subtitle information (smpte-tt subtitle information and / or ttml subtitle information) in an MP4 file are encapsulated into an M4S file for external service. When responding to an HLS service request, the time processing method for the two service scenarios of live broadcast and on-demand is the same as the above-mentioned example of responding to a DASH service request, and is not repeated here.

[0092] In the related art, when the IPTV platform is converted to the OTT platform, the DVB subtitle and DVBteltext subtitles in the media segment files cannot be used by the OTT platform. In the above example of the present application, the media segment files are converted into MP4 files, and the DVB subtitle and DVB teltext subtitles are respectively generated into two XML format subtitle information, smpte-tt and ttml, through encoding and decoding operations, so that the OTT platform can play the audio and video in the media and display subtitles. It can be seen that through the media service method of the present application, in some implementation processes, the subtitles in the media segments can meet the subtitle display of more streaming media protocol types.

[0093] It should be noted that, in the implementation process of the intermediate media file being a file of other formats, the intermediate subtitle information is correspondingly encapsulated in the intermediate media file in a format that the intermediate media file can support (including support by the format standard itself, or support by means of customizing and adapting some contents in the format, etc.). When providing services, based on specific format requirements, the service is converted into the format required by the request (if the requested format is consistent with the current format of the intermediate media file, format conversion may not be performed).

[0094] Embodiment three:

[0095] This embodiment provides a media service device. Fig. 9 , the media service device 1 comprises:

[0096] An acquisition module 11 is used to acquire a media segment file, where the media segment file includes original subtitle information;

[0097] An encapsulation module 12, configured to convert the media segment files into an intermediate media file in a first media format, and the original subtitle information is converted into intermediate subtitle information in the first subtitle format and encapsulated in the intermediate media file;

[0098] A service module 13, configured to provide a media service according to the intermediate media file after receiving a service request;

[0099] Providing media service according to the intermediate media file includes: forming target subtitle information in a target format corresponding to the service request according to the intermediate subtitle information to provide subtitle service.

[0100] In some implementations, the original subtitle information includes subtitles such as graphic subtitles and / or text subtitles.

[0101] In some implementations, the graphic subtitles may include DVB subtitle subtitles, and the text subtitles may include DVB teltext subtitles. The encapsulation module 12 converts the original subtitle information into the intermediate subtitle information, specifically including:

[0102] DVB subtitles are encoded and decoded to form smpte-tt subtitle information;

[0103] DVB teltext subtitles are encoded and decoded to form ttml subtitle information.

[0104] In the specific implementation process of the above-mentioned embodiment, the encapsulation module 12 is also used to customize the first media format or adapt the first subtitle format to the first media format when the standard of the first media format is not compatible with the intermediate subtitle information of the first subtitle format. For example, in a specific example, the smpte-tt subtitle information and / or ttml subtitle information are respectively identified in the MP4 file with custom fields.

[0105] In some implementations, the service request includes an HLS service request, and the service module 13 is specifically configured to implement at least one of the following steps:

[0106] 1. If an HLS service request that does not support smpte-tt subtitle information and ttml subtitle information is received, the ttml subtitle information is encapsulated as webvtt subtitle information, and subtitle service is provided according to the webvtt subtitle information;

[0107] 2. If an HLS service request that supports ttml subtitle information and ttml subtitle information is received, at least one of the smpte-tt subtitle information and the ttml subtitle information is encapsulated into an M4S file along with the audio and video in the MP4 file, and subtitle service is provided according to the M4S file.

[0108] In some specific implementation processes, the service request includes a DASH service request. The service module 13 is specifically used to encapsulate at least one of the smpte-tt subtitle information and the ttml subtitle information into an M4S file along with the audio and video in the MP4 file if a DASH service request is received, and provide subtitle service according to the M4S file.

[0109] In some implementations, the service module 13 is further configured to, after receiving a service request, form a target audio and video file corresponding to the service request according to the video data and audio data in the intermediate media file to provide audio and video services.

[0110] In some implementations, the acquisition module 11 specifically includes:

[0111] A code stream receiving unit, used for receiving a media code stream. In a specific example, the media code stream may be a UDP MPEGTS code stream, wherein video data, audio data, original subtitle information and other data are respectively encapsulated in TS packets;

[0112] A classification unit, used to classify different types of data packets in the media code stream, the data packets include subtitle data packets, audio data packets and video data packets, and the subtitle data packets include original subtitle information;

[0113] A synchronization unit, used to align the time of various classified data packets;

[0114] The slicing unit is used to slice the data packets and encapsulate them into media segment files. Each media segment file includes slices of various types of data packets, and the time corresponding to each slice is the same.

[0115] This embodiment also provides a media service device. Fig.10 As shown, it includes a processor 101, a memory 102 and a communication bus 103, wherein:

[0116] The communication bus 103 is used to realize the connection and communication between the processor 101 and the memory 102;

[0117] The processor 101 is configured to execute one or more computer programs stored in the memory 102 to implement at least one step in the media service method in the first embodiment and the second embodiment.

[0118] The present embodiment also provides a computer storage medium, which includes volatile or non-volatile, removable or non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, computer program modules or other data). Computer storage media include, but are not limited to, RAM (Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable read only memory), flash memory or other memory technology, CD-ROM (Compact Disc Read-Only Memory), DVD (Digital Versatile Disc) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer.

[0119] The computer storage medium in this embodiment can be used to store one or more computer programs, and the one or more computer programs stored therein can be executed by a processor to implement at least one step of the media service method in the above-mentioned embodiment 1 and embodiment 2.

[0120] It should be understood by those skilled in the art that all or some of the steps, systems, and functional modules / units in the above disclosed methods may be implemented as software (which may be implemented using computer program code executable by a computing device), firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit.

[0121] In addition, it is well known to those skilled in the art that communication media generally contain computer readable instructions, data structures, computer program modules or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery media. Therefore, the present invention is not limited to any specific hardware and software combination.

[0122] The above contents are further detailed descriptions of the embodiments of the present invention in combination with specific implementation methods, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A media service method, comprising: Acquire a media segment file, wherein the media segment file includes original subtitle information; Converting the media segment file into an intermediate media file in a first media format, wherein the original subtitle information is converted into intermediate subtitle information in the first subtitle format and encapsulated in the intermediate media file; When receiving a service request, providing a media service according to the intermediate media file; Providing media service according to the intermediate media file includes: forming target subtitle information in a target format corresponding to the service request according to the intermediate subtitle information to provide subtitle service.

2. The media service method according to claim 1, wherein: The obtaining of the media segment files comprises: Receive media stream; Classifying different types of data packets in the media code stream, the data packets including subtitle data packets, audio data packets and video data packets, the subtitle data packets including original subtitle information; Aligning the time of each type of data packets; The data packets are sliced ​​and encapsulated into the media segment files, each of which includes slices of various types of data packets, and the time corresponding to each slice is the same.

3. The media service method according to claim 1 or 2, characterized in that: The original subtitle information includes at least one of the following: graphic subtitles and text subtitles.

4. The media service method according to claim 3, wherein: The graphic subtitles include digital video broadcasting DVB subtitle subtitles, the text subtitles include digital video broadcasting DVB teltext subtitles, the intermediate media file includes a moving picture expert group standard MP4 file, and the original subtitle information is encoded and decoded to form intermediate subtitle information including: The DVB subtitle is encoded and decoded to form the Society of Motion Picture and Television Engineers Timed Text (SMPTE-TT) subtitle information; The DVB teltext subtitles are encoded and decoded to form timed text markup language ttml subtitle information.

5. The media service method according to claim 4, wherein: The service request includes at least one of an HLS service request based on an HTTP streaming media network transmission protocol and a DASH service request based on HTTP, and the forming of target subtitle information in a target format corresponding to the service request according to the intermediate subtitle information to provide a subtitle service includes at least one of the following: If an HLS service request is received that does not support the smpte-tt subtitle information and the ttml subtitle information, encapsulate the ttml subtitle information into webvtt subtitle information, and provide subtitle service according to the webvtt subtitle information; If an HLS service request supporting the smpte-tt subtitle information and the ttml subtitle information is received, at least one of the smpte-tt subtitle information and the ttml subtitle information is encapsulated into an M4S file along with the audio and video in the MP4 file, and a subtitle service is provided according to the M4S file; If a DASH service request is received, at least one of the smpte-tt subtitle information and the ttml subtitle information is encapsulated into an M4S file along with the audio and video in the MP4 file, and a subtitle service is provided according to the M4S file.

6. The media service method according to claim 5, characterized in that: The providing of media service according to the intermediate media file further comprises: If an HLS service request is received, convert the audio and video in the MP4 file into a target format corresponding to the HLS service request to provide audio and video services; If a DASH service request is received, audio and video services are provided according to the M4S file.

7. The media service method according to claim 4, characterized in that: The original subtitle information is encoded and decoded to form XML subtitle information encapsulated in the MP4 file, including: At least one of the smpte-tt subtitle information and the ttml subtitle information is respectively identified in the MP4 file using a custom field.

8. A media service device, comprising: An acquisition module, used for acquiring a media segment file, wherein the media segment file includes original subtitle information; An encapsulation module, used for converting the media segment file into an intermediate media file in a first media format, wherein the original subtitle information is converted into intermediate subtitle information in the first subtitle format and encapsulated in the intermediate media file; A service module, configured to provide a media service according to the intermediate media file upon receiving a service request; Providing media service according to the intermediate media file includes: forming target subtitle information in a target format corresponding to the service request according to the intermediate subtitle information to provide subtitle service.

9. A media service device, the media service device comprising a processor, a memory and a communication bus; The communication bus is used to realize the connection and communication between the processor and the memory; The processor is configured to execute one or more computer programs stored in the memory to implement the steps of the media service method according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The computer storage medium stores one or more computer programs, and the one or more computer programs can be executed by one or more processors to implement the steps of the media service method according to any one of claims 1 to 7.

Images