Video playing method and device, computer device and storage medium

Abstract

The application relates to a video playing method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a first segment information file adapted to a first video format, the first segment information file containing video segment information of a plurality of video segments in the first video format, wherein the video segment information comprises a video segment address; modifying the first segment information file according to the difference between the segment information files in the first video format and a second video format, to obtain a second segment information file adapted to the second video format; acquiring a to-be-played video segment address obtained by a video player parsing the second segment information file; acquiring a to-be-played video segment in the first video format according to the to-be-played video segment address; converting the to-be-played video segment into a target video segment in the second video format, and instructing the video player to play the target video segment. The method can save server storage space.

Description

Technical Field

[0001] This application relates to the field of video technology, and in particular to a video playback method, apparatus, computer device, and storage medium. Background Technology

[0002] With the development of science and technology, more and more video formats have emerged, such as H.265 / MP4, H.265 / MPEG-TS, and H.264 / AVI. However, different playback software and devices on the market support different video formats to varying degrees, resulting in compatibility issues. For example, some software supports playing H.265 / MPEG-TS videos, while others do not. Currently, the main solution is to pre-generate video resources in multiple formats on a server. For instance, for the same video, pre-generate both H.265 / MPEG-TS and H.265 / MP4 video resources. When a terminal requests video resources, the backend will distribute video resources compatible with that device.

[0003] However, pre-storing video resources in multiple formats on the server consumes a significant amount of server storage resources. Due to varying compatibility between different playback devices / software, the same video may require storage in multiple formats, thus greatly increasing storage costs and reducing storage resource utilization. Summary of the Invention

[0004] Therefore, it is necessary to provide a video playback method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can reduce storage space and improve storage resource utilization in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a video playback method, the method comprising:

[0006] Obtain a first segment information file adapted to the first video format. The first segment information file contains video segment information for each of the multiple video segments of the first video format. The video segment information includes video segment addresses. The first video format is a video format that is not supported by the video player.

[0007] Based on the differences between the segment information files of the first video format and the second video format, the first segment information file is modified to obtain a second segment information file adapted to the second video format; the second video format is a video format supported by the video player.

[0008] Obtain the address of the video segment to be played obtained by the video player parsing the second segment information file;

[0009] Based on the address of the video segment to be played, obtain the video segment to be played in the first video format;

[0010] The video segment to be played is converted into a target video segment in the second video format, and the video player is instructed to play the target video segment.

[0011] Secondly, this application also provides a video playback device, the device comprising:

[0012] The file modification module is used to obtain a first segment information file adapted to a first video format. The first segment information file contains video segment information for each of multiple video segments in the first video format, and the video segment information includes video segment addresses. Based on the differences between the segment information files of the first video format and the second video format, the first segment information file is modified to obtain a second segment information file adapted to the second video format. The first video format is a video format that the video player does not support; the second video format is a video format that the video player supports.

[0013] The video segment acquisition module is used to acquire the address of the video segment to be played obtained by the video player parsing the second segment information file; and to acquire the video segment to be played in the first video format according to the address of the video segment to be played.

[0014] The format conversion module is used to convert the video segment to be played into a target video segment of the second video format, and instruct the video player to play the target video segment.

[0015] In one embodiment, the file modification module is further configured to, in response to a first selection operation of multiple video icons displayed by a video playback application, play the target video corresponding to the video icon selected by the first selection operation; in response to a resolution adjustment operation for the target video, display multiple resolution options; in response to a second selection operation for the multiple resolution options, determine the target resolution corresponding to the resolution option selected by the second selection operation; when the video player in the video playback application does not support a first video format corresponding to the target resolution, obtain a video stream link corresponding to the target video having the target resolution; and obtain a first segment information file adapted to the first video format based on the video stream link.

[0016] In one embodiment, the file modification module is further configured to determine the file content that the second segment information file adapted to the second video format should contain; and modify the first segment information file according to the difference between the file content contained in the first segment information file and the file content that the second segment information file adapted to the second video format should contain, so as to obtain the second segment information file adapted to the second video format.

[0017] In one embodiment, the file modification module is further configured to add a file address tag for an initialization fragment to the first segment information file; and point the file address tag of the initialization fragment to a virtual initialization fragment file to obtain a second segment information file adapted to the second video format.

[0018] In one embodiment, the file modification module is further configured to determine a file compatibility version tag in the first segment information file; if the value of the file compatibility version tag is less than or equal to a preset version threshold, modify the value of the file compatibility version tag to be greater than the preset version threshold, so as to obtain a second segment information file adapted to the second video format.

[0019] In one embodiment, the video segment acquisition module is further configured to send the second segment information file to the video player; wherein, the sent second segment information file is used to trigger the video player to parse the second segment information file, obtain the address of the video segment to be played that matches the current video playback progress, and generate a video segment acquisition request based on the address of the video segment to be played; and obtain the video segment acquisition request generated by the video player to obtain the video segment to be played in the first video format that matches the current video playback progress from the address of the video segment to be played.

[0020] In one embodiment, the format conversion module is further configured to extract first audio data from the video segment to be played, and convert the audio header in the first audio data to obtain second audio data adapted to the second video format; allocate memory of a preset size, and write the second audio data and video data from the video segment to be played into the memory; determine the time data corresponding to the video data and the second audio data respectively; write the time data into the memory, and determine the target video segment to be converted to the second video format based on the data written into the memory.

[0021] In one embodiment, the format conversion module is further configured to call a bitstream filter in a video processing tool to extract a first audio header from the first audio data, and determine the audio features of the first audio data based on the first audio header; call the bitstream filter to generate a second audio header adapted to the second video format based on the audio features, insert the second audio header into the first audio data, and delete the first audio header from the first audio data to obtain second audio data adapted to the second video format.

[0022] In one embodiment, either the second audio data or the video data in the video segment to be played is used as the target media data; the time data corresponding to the target media data includes an update timestamp under the target time type; the format conversion module is further configured to determine a first time base corresponding to the target media data according to the first video format; determine a second time base corresponding to the target media data according to the second video format; determine the original timestamp of the data frame in the target media data under the target time type; and determine the update timestamp of the data frame in the target media data under the target time type according to the first time base, the second time base, and the original timestamp.

[0023] In one embodiment, the format conversion module is further configured to, for each data frame in the target media data, multiply the first time base by the original timestamp of the data frame under the target time type to obtain a multiplied timestamp; add the multiplied timestamp to half of the second time base to obtain a summed timestamp; and use the ratio of the summed timestamp to the second time base as the update timestamp of the data frame under the target time type.

[0024] In one embodiment, the format conversion module is further configured to, upon reading the audio encoding format information of the second audio data, write the audio encoding format information into a preset first data unit; wherein, the first data unit is a data unit storing the metadata of the media; and store the first data unit in the memory.

[0025] In one embodiment, the format conversion module is further configured to extract data from the memory, and parse the extracted data according to the data structure corresponding to the second video format to obtain at least one data unit; extract a first data unit and a second data unit from the at least one data unit; write the first data unit and the second data unit into the memory; wherein the first data unit is a data unit storing media metadata, and the second data unit is a data unit storing file type and compatibility information; delete the first data unit and the second data unit from the at least one data unit; modify the remaining data units, and write the modified data units into the memory to obtain a target video segment converted to the second video format.

[0026] In one embodiment, the format conversion module is further configured to determine the time scaling factor corresponding to the second video format and the playback start time of the video segment to be played, and generate a video display time and a video decoding time based on the time scaling factor corresponding to the second video format and the playback start time of the video segment to be played; modify the third data unit in the remaining data units according to the video display time; the third data unit is a unit that stores the video segment index; modify the fourth data unit in the remaining data units according to the video decoding time; the fourth data unit is a unit that stores a time stream description table.

[0027] Thirdly, this application also provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in any of the video playback methods provided in the embodiments of this application.

[0028] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the video playback methods provided in the embodiments of this application.

[0029] Fifthly, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the video playback methods provided in the embodiments of this application.

[0030] The aforementioned video playback method, apparatus, computer equipment, storage medium, and computer program product, by obtaining a first segment information file adapted to a first video format, can modify the first segment information file to obtain a second segment information file adapted to a second video format, even when the video player does not support the first video format but supports a second video format. By generating the second segment information file adapted to the second video format, the second segment information file can be sent to the video player. Since the video player supports the second video format, it can parse the second segment information file adapted to the second video format to obtain the address of the video segment to be played. By obtaining the address of the video segment to be played, the video segment to be played in the first video format can be obtained based on the address, and the video segment to be played in the first video format can be converted into a target video segment in the second video format. Furthermore, since the video player supports the second video format, it can play the target video segment converted to the second video format, thus achieving video transcoding and playback.

[0031] Since video transcoding and playback can be performed on the terminal, there is no need to store video resources of various formats on the server. Even if the video player does not support the video format stored on the server, the format conversion component in the terminal can convert the video format locally and generate video segments with video formats supported by the video player. In this way, the storage resources such as memory on the server are saved and the utilization rate of storage resources is improved.

[0032] Furthermore, since the transcoding process is performed on video segments, the next video segment can be downloaded from the server simultaneously during the format conversion of the current video segment, and the video player can be triggered to play the target video segment that has undergone format conversion. This improves the efficiency of video segment download and format conversion, achieving the effect of real-time transcoding and playback. Attached Figure Description

[0033] Figure 1 This is a diagram illustrating the application environment of a video playback method in one embodiment;

[0034] Figure 2 This is a flowchart illustrating a video playback method in one embodiment;

[0035] Figure 3 This is a schematic diagram illustrating the interaction between the server and the terminal in one embodiment;

[0036] Figure 4 This is a schematic diagram of real-time video transcoding and playback in one embodiment;

[0037] Figure 5 This is an interactive schematic diagram of a video playback system in one embodiment;

[0038] Figure 6 This is a schematic diagram illustrating video playback based on an XR device in one embodiment;

[0039] Figure 7 This is a schematic diagram of the resolution selection in one embodiment;

[0040] Figure 8 This is a schematic diagram of a fragmented information file in one embodiment;

[0041] Figure 9 This is a schematic diagram illustrating data writing to memory in one embodiment;

[0042] Figure 10 A visual schematic diagram of at least one data unit in one embodiment;

[0043] Figure 11 This is a schematic diagram illustrating the viewing of ultra-high-definition video in one embodiment;

[0044] Figure 12 This is a schematic diagram illustrating the use of the playback function in one embodiment;

[0045] Figure 13 This is a schematic diagram illustrating the effect of multiple people watching a video together in one embodiment;

[0046] Figure 14 This is a schematic diagram showing the screen lighting effect in one embodiment;

[0047] Figure 15 This is a schematic diagram of the overall framework of a video playback method in one embodiment;

[0048] Figure 16 This is a structural block diagram of a video playback device in one embodiment;

[0049] Figure 17 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0051] The video playback method provided in this application embodiment can be applied to, for example, Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store data that server 104 needs to process. The data storage system can be set up independently, integrated into server 104, or placed in the cloud or on other servers. Both terminal 102 and server 104 can be used independently to execute the video playback method provided in this embodiment. Terminal 102 and server 104 can also be used collaboratively to execute the video playback method provided in this embodiment. Taking the example of terminal 102 and server 104 collaboratively executing the video playback method provided in this embodiment, the user can launch a video playback application installed on terminal 102 and select a video to watch. Terminal 102 can then obtain the video stream link of the selected video and obtain a first segment information file of the video to be played, adapted to the first video format, from server 104 based on the video stream link. When the player in the video playback application does not support the first video format, a second video format supported by the video player is determined, and the obtained first segment information file is modified to a second segment information file adapted to the second video format. Terminal 102 retrieves the currently playable video segment in the first video format from server 104 based on the second segment information file, converts the currently playable video segment in the first video format into a second video format supported by the player, and plays the converted video segment. Server 104 can be implemented using a standalone server, a server cluster consisting of multiple servers, or a cloud server.

[0052] It should be noted that the terms "first," "second," and similar terms used in this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Unless the context clearly indicates otherwise, the singular forms "a," "one," or "the," etc., do not indicate a quantity limitation, but rather indicate the presence of at least one. The quantities of "multiple" or "multiple copies" mentioned in the embodiments of this application all refer to a quantity of "at least two," for example, "multiple" means "at least two," and "multiple copies" means "at least two copies."

[0053] In order to clearly describe the technical solution of this application and facilitate understanding of the technical solution of this application, the key concepts involved in this application will be explained below.

[0054] Video container formats: Video container formats (also known as container formats) are file formats used to store video data, audio data, and other related information (such as subtitles, metadata, etc.). They act as "containers," packaging video streams, audio streams, and other data into a single file for unified management and playback. Video container formats do not concern themselves with the actual encoding methods of the video or audio; rather, they define how different types of data are combined into a single file. Common video container formats include MP4 (MPEG-4 Part 14, a digital multimedia container format), MPEG-TS (MPEG-2 TransportStream, a standard format for transmitting and storing various data containing video, audio, and communication protocols), and AVI (AudioVideo Interleaved).

[0055] Video encoding formats: Video encoding formats (also known as video codecs or video compression formats) are technologies and standards used to compress and decompress digital video data. An encoding format defines how to compress raw video data (typically uncompressed, very large files) into smaller files for easier storage and transmission, while preserving as much video quality as possible. When playing the video, the decoder decompresses the compressed data back to its original format for display on the screen. Common video encoding formats include H.264 and H.265.

[0056] HLS (HTTP Live Streaming) is a media streaming protocol based on the HTTP (Hypertext Transfer Protocol) and is widely used for video streaming. HLS allows clients (such as web browsers, mobile applications, and smart TVs) to stream video and audio content without waiting for the entire file to download. It achieves efficient and reliable streaming playback by segmenting media content into small pieces and transmitting them using HTTP.

[0057] H.264: H.264 (also known as Advanced Video Coding, AVC) is a widely used video compression standard. Designed for efficient compression of video data, H.264 provides excellent video quality and a high compression ratio, making it widely used in various applications, including streaming media, video conferencing, high-definition television, and Blu-ray discs. Its main features include high compression efficiency, flexibility and scalability, network adaptability, and broad application support.

[0058] H.265: H.265 (also known as High Efficiency Video Coding, HEVC) is an advanced video compression standard. The successor to H.264, H.265 aims to provide higher compression efficiency and better video quality, especially for high-resolution video content such as 4K and 8K video. Key features include: higher compression efficiency, support for higher resolutions, improved prediction and coding techniques, enhanced parallel processing capabilities, and better error recovery.

[0059] MP4 (MPEG-4 Part 14) is a digital multimedia container format widely used for storing video, audio, subtitles, and other data. As part of the MPEG-4 standard, MP4 is a flexible and efficient format widely used in internet video streaming, mobile devices, video-on-demand, and other scenarios. MP4 files typically use the .mp4 extension, but other extensions such as .m4v, .m4a, and .f4v are also possible. MP4 files are based on the ISO Base MediaFile Format (ISOBMFF), whose structure consists of a series of data units called "boxes" or "atoms." Each box has multiple types, each containing a specific type of data, and boxes can contain other boxes, forming a hierarchical structure.

[0060] fMP4 (Fragmented MP4) is a special variant of the MP4 file format designed to support more efficient streaming media delivery and playback, especially in Adaptive Bitrate Streaming (ABR) and real-time streaming media such as DASH and HLS. Compared to traditional MP4 files, fMP4 files are divided into smaller fragments, each of which can be downloaded and played independently of the others, thus improving the efficiency and flexibility of streaming media delivery. The structure of an fMP4 file is similar to that of a standard MP4 file, but it includes some specific features to support fragmentation.

[0061] FFmpeg: FFmpeg is an open-source multimedia processing toolset widely used for processing video, audio, and other multimedia files and streams. FFmpeg consists of a series of libraries and command-line tools for recording, converting, editing, and streaming audio and video. It supports a wide range of formats and codecs, making it an important tool in the field of multimedia development and processing.

[0062] In one embodiment, such as Figure 2As shown, a video playback method is provided, which can be applied to a terminal or a server. The following describes the application of this method to... Figure 1 Taking the format conversion component in the terminal as an example, the explanation includes the following steps:

[0063] Step 202: Obtain the first segment information file adapted to the first video format. The first segment information file contains the video segment information of each of the multiple video segments of the first video format. The video segment information includes the video segment address. The first video format is a video format that is not supported by the video player.

[0064] Specifically, a video playback application can run on the terminal, including a format conversion component and a video player. When format conversion of a target video in a first video format is required, the format conversion component can obtain the first segment information file of the target video in the first video format. Since the target video belongs to the first video format, this first segment information file is a segment information file adapted to the first video format. Specifically, the segment information file is an m3u8 file. m3u8 (MPEG-2 Transport Stream M3U playlist, with the extension .m3u8) is an HTTP-based streaming media network transmission protocol used to transmit audio and video data over the Internet. The m3u8 file is actually a playlist containing the video segment information for each of the multiple video segments obtained by dividing the video. The video segment information may include information such as the video segment address and the video segment duration. The video segment address is the address of the video segment on the server.

[0065] In one embodiment, the first segment information file contains video segment information for each of the multiple video segments obtained by segmenting a target video in a first video format. Since the target video in a first video format is being segmented, the resulting video segments are also in the first video format.

[0066] In one embodiment, the video format may include a video container format and a video encoding format, for example, referencing Figure 3The video format can be H.265 / MPEG-TS, H.264 / fmp4, Dolby / fmp4, etc. H.265 / MPEG-TS means the video encoding format is H.265 and the video container format is MPEG-TS. H.264 / fmp4 means the video encoding format is H.264 and the video container format is fmp4. Dolby / fmp4 means the video encoding format is Dolby (Dolby Digital) and the video container format is fmp4. Specific terminals can include mobile phones (including Android, iOS, HarmonyOS, etc.), televisions (including smart TVs and non-smart TVs), browsers, XR devices (Extended Reality), tablets, IoT devices, and portable wearable devices, etc. XR devices can include VisionPro, Vision Glass, PlayStation VR, etc. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. The terminal can retrieve and play the target video from the server through a web-based or client-based video playback application. Since server storage space is limited, the server typically only stores a subset of video formats. For example, for H.265, it will only store H.265 / MPEG-TS format videos, not formats like H.265 / fmp4 or H.265 / AVI. For a specific target video to be played, if the target video stored on the server is in a first video format that is not supported by the video player in the video playback application, the terminal can trigger the retrieval of the first segment information file adapted to the first video format. Based on this first segment information file, it generates video segments adapted to the video player and plays them. Figure 3 A schematic diagram of server-terminal interaction is shown in one embodiment.

[0067] In one embodiment, the first video format may specifically be H.265 / MPEG-TS.

[0068] In one embodiment, the format conversion component can obtain a video stream link and retrieve the first segment information file from the server address pointed to by the video stream link. For example, in the above example, after the upper-layer business module in the video playback application determines the target video, the upper-layer business module can obtain the video stream link of the target video and send the video stream link to the format conversion component, so that the format conversion component can obtain the first segment information file from the server based on the video stream link.

[0069] Step 204: Based on the differences between the segment information files of the first video format and the second video format, modify the first segment information file to obtain a second segment information file adapted to the second video format; the second video format is a video format supported by the video player.

[0070] Specifically, the second video format is a format supported by the video player; for example, the second video format could be H.265 / fmp4. The format conversion component can modify the first segment information file based on the differences between the segment information files of the first and second video formats to obtain a second segment information file adapted to the second video format. Here, the differences in segment information files refer to the differences between the segment information files. For example, the format conversion component can determine the file content that the segment information file adapted to the second video format should contain, compare the file content of the first segment information file with the file content that the second segment information file adapted to the second video format should contain, and modify the first segment information file based on these differences to obtain a second segment information file adapted to the second video format.

[0071] For example, the format conversion component can obtain a segment information file template adapted to the second video format. This segment information file template is a pre-set standard segment information file. The format conversion component can compare the segment information file template adapted to the second video format with the first segment information file to determine the differences. These differences can then be used to modify the first segment information to obtain a second segment information file adapted to the second video format. For instance, if it is determined that the standard segment information file contains a file address tag for an initialization segment, but the first segment information file does not, then it can be determined that the differences in the segment information files include the file address tag for the initialization segment. Therefore, the file address tag for the initialization segment can be added to the first segment information file.

[0072] Step 206: Obtain the address of the video segment to be played obtained by the video player parsing the second segment information file, and adapt the video player to the second video format.

[0073] Specifically, after the format conversion component generates the second segment information file, it can send the second segment information file to the video player. The video player can parse the second segment information file to extract the address of the video segment to be played. The video player can generate a video segment retrieval request based on the address of the video segment to be played and send the video segment retrieval request. Further, the format conversion component can obtain the video segment retrieval request sent by the video player and respond to the video segment retrieval request to obtain the address of the video segment to be played carried in the video segment retrieval request.

[0074] In one embodiment, the video player can determine the current video playback progress and extract the address of the video segment to be played from the second segment information file based on the current playback progress. For example, when playing the target video from the beginning, the address of the first video segment in the target video can be extracted from the second segment information file and used as the address of the video segment to be played. For example, when playing the target video from the middle, the address of the video segment to be played that matches the intermediate playback progress can be extracted from the second segment information file based on the duration information of each video segment, and used as the address of the video segment to be played. For another example, when N video segments of the target video have been played, the address of the (N+1)th video segment can be extracted from the second segment information file and used as the address of the video segment to be played.

[0075] Step 208: Obtain the video segment to be played in the first video format based on the address of the video segment to be played.

[0076] Specifically, once the address of the video segment to be played is obtained, the format conversion component can retrieve the video segment of the first video format to be played based on that address. For example, the video segments obtained by segmenting the target video of the first video format can be stored on the server in the form of .ts files, and each .ts file can correspond to a video segment address. When the format conversion component obtains the address of the video segment to be played, it can retrieve the video segment of the first video format to be played from the location pointed to by the address.

[0077] Step 210: Convert the video segment to be played into a target video segment in the second video format, and instruct the video player to play the target video segment.

[0078] Specifically, after obtaining the video segment to be played, the format conversion component can convert it into a target video segment in a second video format. For example, the format conversion component can call a video processing tool to convert the video segment to be played into a target video segment in a second video format. The video processing tool could specifically be Ffmpeg. Further, after converting the video segment to be played into a target video segment in a second video format, the format conversion component can send the target video segment in the second video format to a video player for playback.

[0079] In an easy-to-understand way, refer to Figure 4To achieve real-time transcoding and playback, the system can simultaneously download the current video segment to be played from the server, convert the format of the previously downloaded video segment, and simultaneously call a video player to play the transcoded target video segment. Transcoding refers to converting one format to another. For example, the format conversion component can download the i-th video segment to be played from the server, convert the (i-1)-th video segment to be played into a target video segment of a second video format, and simultaneously call a video player to play the target video segment obtained by converting the (i-2)-th video segment to be played. Figure 4 A schematic diagram of real-time video transcoding and playback in one embodiment is shown.

[0080] In one embodiment, the video playback method may be executed by a video playback system. (See reference...) Figure 5 , Figure 5 A schematic diagram of the interaction of a video playback system in one embodiment is shown. The video playback system includes an upper-layer business module, a format conversion component, a video player, and a server. The upper-layer business module can send a video stream link to the format conversion component, so that the format conversion component can obtain a first segment information file adapted to a first video format based on the video stream link, and modify the first segment information file according to a preset segment information modification method to obtain a second segment information file in a second video format. The format conversion component sends the second segment information file to the video player, so that the video player can parse the second segment information file, obtain the address of the video segment to be played, and request the video segment to be played based on the address. The format conversion component can proxy the request of the video segment to be played, obtain the video segment to be played from the server pointed to by the address of the video segment to be played, and convert the video segment to be played in the first video format into a target video segment in the second video format according to a preset video conversion method, and send the target video segment to the video player so that the video player can play the target video segment.

[0081] In the aforementioned video playback method, by obtaining a first segment information file adapted to the first video format, and even if the video player does not support the first video format but supports the second video format, the first segment information file can be modified to obtain a second segment information file adapted to the second video format. By generating the second segment information file adapted to the second video format, the second segment information file can be sent to the video player. Since the video player supports the second video format, it can parse the second segment information file adapted to the second video format to obtain the address of the video segment to be played. Based on the address of the video segment to be played, the video segment to be played in the first video format can be obtained, and the video segment in the first video format can be converted into a target video segment in the second video format. Since the video player supports the second video format, it can play the target video segment converted to the second video format, thus achieving video transcoding and playback.

[0082] Since video transcoding and playback can be performed on the terminal, there is no need to store video resources of various formats on the server. Even if the video player does not support the video format stored on the server, the format conversion component in the terminal can convert the video format locally and generate video segments with video formats supported by the video player. In this way, the storage resources such as memory on the server are saved and the utilization rate of storage resources is improved.

[0083] Furthermore, since the transcoding process is performed on video segments, the next video segment can be downloaded from the server simultaneously during the format conversion of the current video segment, and the video player can be triggered to play the target video segment that has undergone format conversion. This improves the efficiency of video segment download and format conversion, achieving the effect of real-time transcoding and playback.

[0084] In one embodiment, obtaining a first segment information file adapted to a first video format includes: in response to a first selection operation of multiple video icons displayed by a video playback application, playing the target video corresponding to the video icon selected in the first selection operation; in response to a resolution adjustment operation for the target video, displaying multiple resolution options; in response to a second selection operation for the multiple resolution options, determining the target resolution corresponding to the resolution option selected in the second selection operation; when the video player in the video playback application does not support the first video format corresponding to the target resolution, obtaining a video stream link corresponding to the target video with the first video format; and obtaining the first segment information file based on the video stream link.

[0085] Specifically, a video playback application may be installed on the terminal, such as on an XR device. The user can open this video playback application, which will then display multiple video icons. The user can select one from the displayed video icons, and the video playback application, in response to this first selection of the target video icon, will invoke a video player to play the video. Figure 6 The target video icon shown corresponds to the target video at its default resolution. For the same video, different resolutions may correspond to different video formats. The default resolution can be low resolution, such as 1080P. The video format of the target video at the default resolution is a format supported by the video player. For example, the video format of a target video at 1080P resolution could be H.264 / fmp4. Figure 6 A schematic diagram of video playback based on an XR device is shown in one embodiment.

[0086] When a user wants to change the resolution of a target video, they can trigger a resolution adjustment action for the target video, causing the video playback application to display multiple resolution options. For example, see [reference]. Figure 7 Display as shown Figure 7 Multiple resolution options 701 are shown. The user can select one, and the video playback application can respond to a second selection operation for a target resolution option among the multiple resolution options, determining the target resolution corresponding to the target resolution option. Further, if the video player in the video playback application does not support the first video format corresponding to the target resolution, it can obtain the video stream link corresponding to the target video with the first video format and obtain the first segment information file based on the video stream link. For example, the target resolution can be high definition, such as 4K resolution. For the target video at the target resolution, to save storage resources, the server can only store video resources with the first video format; for example, for a 4K resolution target video, it can only store video resources in H.265 / MPEG-TS format. If the video player does not support the first video format but a 4K resolution target video needs to be played, the video playback application can obtain the video stream link corresponding to the target video with the first video format and obtain the first segment information file based on the video stream link. For example, when the first video format is H.265 / MPEG-TS, the video playback application can obtain the video stream link corresponding to the target video in H.265 / MPEG-TS format, and obtain the first segment information file adapted to the H.265 / MPEG-TS format through the video stream link. This first segment information file stores the segment information of each of the multiple video segments obtained by segmenting the target video in H.265 / MPEG-TS format. Figure 7A schematic diagram of sharpness selection in one embodiment is shown.

[0087] In one embodiment, the video player is specifically a system player. (See reference...) Figure 6 When a video playback application is launched on an XR device, it defaults to using a custom-developed player. Since this player has fewer features than the system player, users can switch to the system player using the video player switching element 601. The system player, being a native player, supports a limited number of video formats. Therefore, a format conversion component can be added to assist the system player in playing more video formats, such as high-definition H.265 / MPEG-TS format videos.

[0088] In the above embodiments, by displaying multiple video icons, users can select the desired target video icon from among the multiple video icons, thereby enabling the video playback application to play the target video that the user wants to play. By displaying multiple resolution options, users can select the desired resolution from among the multiple resolution options. Even if the video player in the video playback application does not support the video format corresponding to the target resolution, the format conversion component can be used to convert the video format corresponding to the target resolution to obtain a video format supported by the video player, thereby enabling the video player to play the target video at the target resolution.

[0089] In one embodiment, modifying the first segment information file based on the differences between the segment information files of the first video format and the second video format to obtain a second segment information file adapted to the second video format includes: determining the file content that the second segment information file adapted to the second video format should contain; and modifying the first segment information file based on the differences between the file content contained in the first segment information file and the file content that the second segment information file adapted to the second video format should contain to obtain a second segment information file adapted to the second video format.

[0090] Specifically, the format conversion component can determine the file content that the second segment information adapted to the second video format should include. For example, the terminal can pre-store a file content record text, which records the file content that should be included for each video format. For instance, it records that the second segment information for the second video format should include a file address tag for an initialization segment. Further, the format conversion component can determine the difference between the file content contained in the first segment information file and the file content that the second segment information file adapted to the second video format should include, and modify the first segment information file according to this difference to obtain a second segment information file adapted to the second video format. For example, if the format conversion component determines that the second segment information file adapted to the second video format should include a file address tag for an initialization segment, but the first segment information file does not contain a file address tag for an initialization segment, then the format conversion component can add a file address tag for an initialization segment to the first segment information file.

[0091] In one embodiment, reference Figure 8 Depending on the container format of the video segments, the contents of the segment information files for FMP4 and MPEG-TS differ slightly. The segment information file can be viewed as a playlist, containing information such as the addresses and durations of several video segments. The FMP4 segment information file (i.e., the second segment information file) should contain the file address tag "#EXT-X-MAP:URI" 801 for the initialization segment, the file compatibility version tag "#EXT-X-VERSION" 802, and video segment information for several video segments 803. The MPEG-TS segment information file (i.e., the first segment information file) contains "#EXT-X-VERSION" 804 and video segment information for several video segments 805. Figure 8 A schematic diagram of a fragmented information file is shown in one embodiment.

[0092] In this embodiment, by determining the differences between file contents and modifying the first segment information file based on these differences, the modification can be made more accurate.

[0093] In one embodiment, the first segment information file is modified based on the difference between the file content contained in the first segment information file and the file content that the second segment information file adapted to the second video format should contain, to obtain a second segment information file adapted to the second video format. This includes: adding a file address tag for an initialization segment to the first segment information file; and pointing the file address tag of the initialization segment to a virtual initialization segment file to obtain a second segment information file adapted to the second video format.

[0094] Specifically, when the first fragment information file is adapted to the MPEG-TS format and the second fragment information file is adapted to the FMP4 format, it can be determined that the first fragment information file lacks a file address tag for the initialization fragment. In this case, a file address tag for the initialization fragment can be added to the first fragment information file, and the file address tag for the initialization fragment can be pointed to a virtual initialization fragment file. For example, the file address tag for the initialization fragment can be "#EXT-X-MAP:URI", and pointing the file address tag for the initialization fragment to the virtual initialization fragment file can be #EXT-X-MAP:URI="init.mp4".

[0095] The file address tag for the initialization fragment indicates the address of an initialization fragment file. The initialization fragment file (i.e., init.mp4) contains two boxes: ftyp and moov. ftyp identifies the file type and compatibility information, ensuring that the video player or decoder can correctly recognize and process the file; moov contains the metadata of the entire file, describing the structure and playback method of the media data. Subsequent fMP4 format video segments, such as 0.mp4, need to have the content of the initialization fragment appended to the beginning of the video segment for it to play correctly. Theoretically, the first segment information file adapted to MPEG-TS format could also contain this tag, but MPEG-TS format video segments contain complete video information and can be played independently, so the first segment information file generally does not include the initialization fragment.

[0096] In one embodiment, during the modification of the first segment information file, the file address tag of the initialization segment is set to point to a virtual initialization segment file. When the video player requests the initialization segment file based on the file address tag of the initialization segment in order to extract the initialization segment file from its file storage path, if it is determined that the initialization segment file does not exist at the location pointed to by the file storage path, the format conversion component generates the actual initialization segment file and returns the generated initialization segment file to the video player.

[0097] In the above embodiments, by adding a file address tag for the initialization segment to the first segment information, when the video player plays a video segment of the second video format, it can obtain the initialization segment file based on the file address tag of the initialization segment, and add the file of the initialization segment before the video segment of the second video format to form a video segment that can be played normally. In this way, the video segment can be played normally by the video player.

[0098] In one embodiment, the first segment information file is modified based on the difference between the file content contained in the first segment information file and the file content that the second segment information file adapted to the second video format should contain, to obtain a second segment information file adapted to the second video format. This includes: determining a file compatibility version tag in the first segment information file; and modifying the value of the file compatibility version tag to be greater than the preset version threshold if the value of the file compatibility version tag is less than or equal to a preset version threshold, so as to obtain a second segment information file adapted to the second video format.

[0099] Specifically, the format conversion component can determine the file compatibility version tag in the first segment information file, for example, by determining the "#EXT-X-VERSION" tag in the first segment information file. When the second segment information file is a segment information file adapted to the fmp4 format, since the value of the "#EXT-X-VERSION" tag in the second segment information file should be greater than a preset version threshold, for example, greater than 6, the format conversion component determines whether the value of the "#EXT-X-VERSION" tag in the first segment information file is greater than the preset version threshold. If the value of the "#EXT-X-VERSION" tag in the first segment information file is less than or equal to the preset version threshold, then the value of the "#EXT-X-VERSION" tag in the first segment information file is modified to be greater than the preset version threshold, for example, modified to 7, in order to support the initialization of the segment's file address tag.

[0100] In this embodiment, by modifying the file compatibility version label, the modified file compatibility version label can meet the requirements of the second video format.

[0101] In one embodiment, obtaining the address of the video segment to be played obtained by the video player parsing the second segment information file, and the video player adapting to the second video format, includes: sending the second segment information file to the video player; wherein, the sent second segment information file is used to trigger the video player to parse the second segment information file, obtain the address of the video segment to be played that matches the current video playback progress, and generate a video segment acquisition request based on the address of the video segment to be played; obtaining the video segment to be played in the first video format based on the address of the video segment to be played includes: proxying the video segment acquisition request generated by the video player to obtain the video segment to be played in the first video format that matches the current video playback progress from the address of the video segment to be played.

[0102] Specifically, after the format conversion component generates the second segment information file, it can send the second segment information file to the video player. The video player can then parse the second segment information file and read the address of the video segment to be played that matches the current video playback progress. For ease of description, the address of the video segment to be played will be referred to as the video segment address below. Further, the video player can obtain the video segment to be played from the server based on this video segment address, but this process can be delegated by the format conversion component, which then obtains the video segment to be played from the server based on the video segment address. It is easy to understand that the obtained video segment to be played belongs to the first video format and matches the current video playback progress. For example, when a user switches the resolution of the target video from 1080P to 4K at 1 minute and 30 seconds, if the duration of the video segment is 30 seconds, the format conversion component can obtain the fourth video segment that matches the current video playback progress. Then, the fourth video segment is converted to 4K resolution and played.

[0103] In this embodiment, by setting a proxy function, the format conversion component can proxy the process of the video player obtaining the video segment to be played, thereby obtaining the video segment to be played. In this way, the obtained video segment to be played can be format converted.

[0104] In one embodiment, converting a video segment to be played into a target video segment of a second video format includes: extracting first audio data from the video segment to be played, and converting the audio header in the first audio data to obtain second audio data adapted to the second video format; allocating memory of a preset size, and writing the second audio data and video data from the video segment to be played into the memory; determining the time data corresponding to the video data and the second audio data respectively, and writing the time data into the memory; and determining the target video segment to be converted into the second video format based on the data written into the memory.

[0105] Specifically, since a video segment can contain both video and audio data, the audio data also needs to be converted during the format conversion of the video segment. More specifically, a video processing tool is installed on the terminal, and the format conversion component extracts the first audio data from the video segment to be played by calling the video processing tool. The first audio data can contain a first audio header and an Elementary Stream (ES). The first audio header plays a crucial role in the audio file, containing basic information such as file type identifier, audio format, sampling rate, bit depth, and number of channels. The ES stream is the most basic video or audio data stream, typically containing a single type of media data (such as video, audio, subtitles, etc.) without any encapsulation or wrapping. The video processing tool can convert the first audio header in the first audio data, transforming it into a second audio header adapted to the second video format, thus obtaining the second audio data adapted to the second video format. For example, the audio ADTS header can be converted into an MPEG-4 AudioSpecficConfig (An Audio Specific Config is an audio header that contains important information about the audio encoder, such as encoder type, audio frame rate, number of audio channels, etc.). The audio ADTS header is the first audio header, and the MPEG-4 AudioSpecficConfig is the second audio header.

[0106] Furthermore, the format conversion component can allocate a memory block of a preset size and write the second audio data and the video data from the video segment to be played into the memory. The format conversion component can determine the corresponding time data for each of the second audio data and the video data from the video segment to be played, and write the time data into the memory as well. The time data refers to data related to the media playback time; for example, the time data may include display timestamps, decoding timestamps, playback duration, start playback timestamps, etc.

[0107] Furthermore, once all the data is stored in memory, the format conversion component can determine the target video segment to be converted to the second video format based on the data in memory.

[0108] In one embodiment, during the data writing process—that is, during the transcoding process—if insufficient memory space is encountered, the memory size will be doubled. After the data writing is completed, that is, after the transcoding is completed, the memory size is adjusted, and the memory without filled data is released to obtain the target video segment in the second video format. By releasing the memory without filled data, memory resources can be saved.

[0109] In one embodiment, the required memory size can be estimated based on the size of the video segment to be played. For example, for the same video segment, the ratio between the size of the video segment in the first video format and the size in the second video format can be determined. Based on the size of the video segment to be played and this ratio, the required memory size can be determined, which makes the determined memory size more accurate and avoids the waste of memory resources.

[0110] In the above embodiments, video format conversion can be achieved by requesting a block of memory and writing the corresponding data into that memory.

[0111] In one embodiment, converting the audio header in the first audio data to obtain second audio data adapted to the second video format includes: calling a bitstream filter in a video processing tool to extract the first audio header from the first audio data, and determining the audio features of the first audio data based on the first audio header; calling the bitstream filter to generate a second audio header adapted to the second video format based on the audio features; inserting the second audio header into the first audio data; and deleting the first audio header from the first audio data to obtain second audio data adapted to the second video format.

[0112] Specifically, a bitstream filter in a video processing tool can be used to extract the first audio header from the first audio data. Since the first audio header contains basic information about the audio file, such as file type identifier, audio format, sampling rate, bit depth, and number of channels, the bitstream filter can determine the audio characteristics of the first audio data based on the first audio header. For example, it can determine the encoder type, audio frame rate, number of audio channels, etc., and then generate a second audio header adapted to the second video format based on these audio characteristics. Further, the bitstream filter inserts the second audio header before the ES stream of the first audio data and removes the first audio header from the first audio data, thus obtaining second audio data adapted to the second video format.

[0113] In one embodiment, with the first video format being H.265 / MPEG-TS and the second video format being H.265 / fMP4, to convert the first audio header (ADTS, Audio Data Transport Format) in the video segment to be played into MPEG-4 AudioSpecficConfig during the process of transcoding the video segment into fMP4, the bitstream filter in FFmpeg needs to be set to aac_adtstoasc. Thus, MPEG-4 AudioSpecficConfig and ES constitute a complete audio segment. Here, FFmpeg is a video processing tool, and the bitstream filter is a bitstream filter.

[0114] In one embodiment, the terminal pre-stores the setting parameters required for the video processing tool under different video formats. Then, after determining the first video format and the second video format, the setting parameters required for the video processing tool can be determined based on the first video format and the second video format. Then, the video processing tool can perform corresponding processing on the video segments to be played based on the set parameters, for example, converting the first audio header into a second audio header adapted to the second video format.

[0115] In the above embodiments, the conversion of the audio header can be achieved simply by setting the bitstream filter, which simplifies the audio data conversion process and improves the audio data conversion efficiency.

[0116] In one embodiment, before converting the audio header in the first audio data using a video processing tool, the encoding format in the first video format can be determined. If the encoding format is H.265, the codec tag in the video processing tool FFmpeg is modified to hvc1, not hev1. hvc1 and hev1 represent different packet formats for the H.265 stream. If the encoding format is H.264, no additional processing is required.

[0117] In one embodiment, either the second audio data or the video data in the video segment to be played is used as the target media data; the time data corresponding to the target media data includes an update timestamp under the target time type; the step of determining the update timestamp of the target media data under the target time type includes: determining a first time base corresponding to the target media data according to a first video format; determining a second time base corresponding to the target media data according to a second video format; determining the original timestamp of the data frame in the target media data under the target time type; and determining the update timestamp of the data frame in the target media data under the target time type according to the first time base, the second time base, and the original timestamp.

[0118] The data frames are either video or audio frames. The target time type includes display time type and decoding time type; the raw timestamps under the target time type include the raw display timestamp and the raw decoding timestamp. The updated timestamps under the target time type include the updated display timestamp and the updated decoding timestamp.

[0119] Specifically, both the second audio data and the video data in the video segment to be played need to have their display timestamps and decoding timestamps recalculated. The method for recalculating the display timestamp of the second audio data can be the same as that for the video data, and the method for recalculating the decoding timestamp of the second audio data can be the same as that for the video data. The calculation method for the display timestamp can also be the same as that for the decoding timestamp. Therefore, the following explanation uses either the second audio data or the video data in the video segment to be played as the target media data, and takes updating either the display timestamp or the decoding timestamp as the update timestamp under the target time type as an example. For video frames, the display timestamp indicates the time when the video frame is displayed; for audio frames, the display timestamp indicates the time when the audio frame is played. For video frames, the decoding timestamp indicates the time when the video frame is decoded; for audio frames, the decoding timestamp indicates the time when the audio frame is decoded.

[0120] Furthermore, since different container formats support different time bases—for example, the time base corresponding to the FLV (FLASH VIDEO, a streaming media format) container format is {1, 1000}, while the time base corresponding to the TS (MPEG Transport Stream, a file format for storing and transmitting audio and video data) container format is {1, 90000}—the format conversion component can determine the first time base corresponding to the target media data based on the first video format. For example, if the first video format is H.265 / MPEG-TS, and the target media data is video data, then the first time base corresponding to the target media data can be determined to be the time base corresponding to the MPEG-TS video container format. When the target media data is second audio data, since the audio data in the video segment with the first video format is encapsulated in AAC (Advanced Audio Coding), the first time base corresponding to the target media data can be determined to be the time base corresponding to AAC. Here, the time base refers to the basic unit used to represent time in a digital signal, defining the precision of the timestamp and duration. For example, if the time base is 1 / 25, then the duration of each frame is 0.04 seconds (i.e., 1 divided by 25). Accordingly, the format conversion component can determine the second time base corresponding to the target media data based on the second video format, for example, determine the time base corresponding to FMP4, and use that time base as the second time base.

[0121] Furthermore, the format conversion component can determine the original timestamp of the data frame in the target time type. For example, if the target media data is video data and the target time type is display time type, the format conversion component can read the display timestamp of the video frame from the video segment to be played. This read display timestamp is the original timestamp of the data frame in the target media data under the target time type. If the target media data is second audio data and the target time type is decoding time type, the format conversion component can directly read the decoding timestamp of the audio frame from the first audio data. This read decoding timestamp is the original timestamp of the data frame in the target media data under the target time type.

[0122] Furthermore, the format conversion component can determine the update timestamp of the data frame in the target media data under the target time type based on the first time base, the second time base, and the original timestamp. For example, the original timestamp can be converted from the first time base to the second time base, thereby obtaining the update display timestamp of the video frame in the video data when the target media data is video data and the target time type is display time type; obtaining the update decoding timestamp of the video data when the target media data is video data and the target time type is decoding time type; obtaining the update display timestamp of the second audio data when the target media data is second audio data and the target time type is decoding time type; and obtaining the update decoding timestamp of the second audio data when the target media data is second audio data and the target time type is decoding time type. After obtaining the update timestamp under the target time type, the timestamp under the target time type can be written into memory.

[0123] In one embodiment, determining the update timestamp of a data frame in the target media data under the target time type based on a first time base, a second time base, and an original timestamp includes: for each data frame in the target media data, multiplying the first time base by the original timestamp of the data frame under the target time type to obtain a multiplied timestamp; adding the multiplied timestamp to half of the second time base to obtain a summed timestamp; and using the ratio of the summed timestamp to the second time base as the update timestamp of the data frame under the target time type.

[0124] Specifically, the update timestamp of a data frame under the target time type can be determined using the following formula:

[0125]

[0126] Where n_timestamp is the update timestamp under the target time type; n_timestamp is the original timestamp under the target time type; o_time_base is the first time base; and n_time_base is the second time base.

[0127] In one embodiment, in addition to determining the updated display timestamp and the updated decoding timestamp, the playback duration of the target video segment in the second video format also needs to be recalculated. For example, the original timebase corresponding to the target video segment can be determined according to the first video format, and the updated timebase corresponding to the target video segment can be determined according to the second video format; the original playback duration corresponding to the target video segment can be determined, and the updated playback duration of the target video segment can be determined according to the original timebase, the updated timebase, and the original playback duration, and the updated playback duration can also be written into memory. For example, the timebase corresponding to MPEG-TS can be determined and used as the original timebase, the timebase corresponding to FMP4 can be determined and used as the updated timebase, and the updated playback duration can be determined using the following formula:

[0128]

[0129] Where n_dur is the updated playback duration, o_dur is the original playback duration, o1_time_base is the original time base, and n1_time_base is the updated time base.

[0130] In one embodiment, reference Figure 9 In addition to writing the second audio data, the video data in the video segment to be played, the updated display timestamp, the updated decoding timestamp, and the updated playback duration into memory, the starting playback timestamp of the video segment to be played can also be written into memory, thus obtaining the initial target video segment. Figure 9 A schematic diagram of data being written to memory is shown in one embodiment.

[0131] In the above embodiments, the updated timestamp is determined by using a time base and the original timestamp, making the determined updated timestamp more accurate. Furthermore, by converting the original timestamp into an updated timestamp compatible with the second video format, it is also possible to play the target video segments of the second video format normally based on the updated timestamp during playback.

[0132] In one embodiment, the method further includes: upon reading the audio encoding format information of the second audio data, writing the audio encoding format information into a preset first data unit, wherein the first data unit is a data unit storing the metadata of the media; and storing the first data unit in memory.

[0133] Specifically, during the parameter settings of the video processing tool, the `movflags` property in FFmpeg can be set to `frag_keyframe+empty_moov+delay_moov`. `frag_keyframe+empty_moov` is used to fragment the MP4 video and output it in fMP4 format. After setting `delay_moov`, during the process of converting the first audio data into the second audio data through a bitstream filter and writing the second audio data into memory, if the audio encoding format information of the second audio data is read, the audio encoding format information is written into a preset first data unit and stored in memory. The fmp4 structure consists of a series of data units called "boxes" or "atoms". Each box has multiple types, each containing a specific type of data, and boxes can contain other boxes, forming a hierarchical structure. The first data unit is the data unit that stores the media's metadata information, specifically the `moov` property in fmp4.

[0134] In this embodiment, if the first data unit (e.g., moov) is written too early, the correct audio encoding format information will not be written into the first data unit. Therefore, it is necessary to delay writing the first data unit so that the correct audio encoding format information can be written into the first data unit. Then, when playing the target video segment, the target video segment can be played normally through the audio encoding format information in the first data unit.

[0135] In one embodiment, determining the target video segment to be converted to a second video format based on the data written to memory includes: extracting data from memory; parsing the extracted data according to the data structure corresponding to the second video format to obtain at least one data unit; extracting a first data unit and a second data unit from the at least one data unit; and writing the first data unit and the second data unit into memory; wherein the first data unit is a data unit storing metadata of the media, and the second data unit is a data unit storing file type and compatibility information; deleting the first data unit and the second data unit from the at least one data unit; modifying the remaining data units, and writing the modified data units into memory to obtain the target video segment converted to the second video format.

[0136] Specifically, after writing the second audio data, media data from the video segment to be played, updated display timestamps, updated decoding timestamps, updated playback duration, and the start playback timestamp of the video segment to be played into memory, the initial target video segment can be obtained. However, even though the initial target video segment has been obtained, it may not yet be decoded and played by the video player, so further processing of the initial target video segment is required. The format conversion component can extract all the data in memory, parse the extracted data according to the data structure corresponding to the second video format, and obtain at least one data unit. For example, the terminal can display something like... Figure 10 At least one data unit as shown. Figure 10 A visual schematic diagram of at least one data unit in one embodiment is shown. The extracted data can be parsed using parsing code written according to the structure definition of the second video format, or parsing code from existing open-source libraries such as bento4 can be used.

[0137] Furthermore, a first data unit and a second data unit can be extracted from at least one data unit. The first data unit stores metadata about the media, such as moov; the second data unit stores file type and compatibility information, such as ftyp. Inputting the second data unit and the first data unit sequentially into memory initializes the fragment file. Further, the first and second data units can be deleted from at least one data unit, and the remaining data units can be modified to obtain modified data units. Writing the modified data units into memory yields the target video fragment converted to a second video format, for example, a target video fragment converted to H.265 / fmp4 format.

[0138] In one embodiment, the data in the first data unit and the second data unit can be concatenated to obtain an initialization fragment file, and the initialization fragment file can be stored in memory.

[0139] In the above embodiments, by parsing the data in memory to obtain at least one data unit, the data in memory can be further modified on a unit-by-unit basis, making data modification more convenient and thus improving the efficiency of data modification. Furthermore, by further modifying the data in memory, the modified data can be played normally by the native video player.

[0140] In one embodiment, modifying the remaining data units includes: determining the time scaling factor corresponding to the second video format and the playback start time of the video segment to be played, and generating video display time and video decoding time based on the time scaling factor corresponding to the second video format and the playback start time of the video segment to be played; modifying the third data unit in the remaining data units according to the video display time; the third data unit is a unit that stores the video segment index; modifying the fourth data unit in the remaining data units according to the video decoding time; the fourth data unit is a unit that stores a time stream description table.

[0141] Specifically, the format conversion component can determine the time scaling factor corresponding to the second video format. For example, the time base corresponding to the second video format can be used as the time scaling factor. For instance, when the second video format is H.265 / fmp4, the time base corresponding to fmp4 can be used as the time scaling factor. Further, the format conversion component can generate the video display time and video decoding time based on the time scaling factor corresponding to the second video format and the playback start time of the video segment to be played. For example, the format conversion component can multiply the time scaling factor corresponding to the second video format by the playback start time of the video segment to obtain the video display time and video decoding time. Here, the video display time refers to the start playback timestamp of the video segment, and the video decoding time refers to the start decoding time of the video segment.

[0142] Furthermore, the third data unit in the remaining data units can be modified according to the video display time. For example, the video display time in the third data unit can be modified to the video display time determined above. The third data unit is a unit storing video segment indexes; for example, the third data unit is tfdt. Furthermore, the fourth data unit in the remaining data units can also be modified according to the video decoding time. For example, the video decoding time in the fourth data unit can be modified to the video decoding time determined above. The fourth data unit is a unit storing a time stream description table; for example, the fourth data unit is sidx.

[0143] Furthermore, after modifying the remaining data units, they can be input into memory to obtain the final target video segment in the second video format. This target video segment can be played normally by the native system player. For example, the video player is specifically the system player; see reference [link / reference]. Figure 11Before using the video playback method provided in this application, the native system player did not support 4K resolution H.265 / MPEG-TS, therefore the maximum resolution was only 1080P. After using the video playback method provided in this application, 4K resolution videos can also be viewed in the system player. Furthermore, refer to... Figure 12 After adopting the video playback method provided in the embodiments of this application, the playback functions provided by the system player can still be used normally, such as pausing, fast forwarding, rewinding, and dragging the progress bar. Figure 11 A schematic diagram of viewing ultra-high-definition video in one embodiment is shown. Figure 12 A schematic diagram of a playback function is shown in one embodiment.

[0144] In one embodiment, after using the video playback method provided in this application, the system functions provided by the system player can still be used normally, for example, referring to... Figure 13 The function of multiple people watching videos together can be used normally, and reference is also available. Figure 14 The screen lighting effect function (simulating the screen light reflection effect on the surface of surrounding objects when playing video) can be used normally. Figure 13 This illustration shows a schematic diagram of the effect of multiple people watching a video together in one embodiment. Figure 14 A schematic diagram showing the screen lighting effect in one embodiment is shown.

[0145] In one embodiment, since the decoding time is generally before the display time, the video display time can be obtained by multiplying the time scaling factor corresponding to the second video format by the playback start time of the video segment to be played, and the video decoding time can be obtained by advancing the video display time by a preset time period.

[0146] In the above embodiments, by modifying the video display time and video decoding time, the modified video display time and video decoding time can be made more accurate.

[0147] The video playback method of this application has the following beneficial effects:

[0148] Wider compatibility: The native system player only supports specific video formats. By using the video playback methods described above, the native system player can play more video formats.

[0149] Low cost: By using the terminal to convert video formats, compared with converting video formats on the server, more server computing costs can be avoided. Compared with pre-storing multiple video formats on the server, it can also save server storage space and reduce the complexity of backend services.

[0150] It can support more system features in the future: For example, some unique features on XR devices can only be implemented by the system player. Through the video playback method described above, users can not only use the various functions provided by the system player normally, but also enable the system player to play videos of more video formats.

[0151] High portability: The transpackaging code is written in C / C++ and can be compiled into multi-platform products, suitable for other platforms.

[0152] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0153] This application also provides an application scenario in which the above-described video playback method is applied. Specifically, the video playback method is applied in this scenario as follows:

[0154] refer to Figure 15 , Figure 15A schematic diagram of the overall framework of a video playback method in one embodiment is shown. On the XR device, the native system player can be used to play the video. When the video format of the target video at the target resolution is not supported by the system player, the video stream link corresponding to the target video at the target resolution can be passed to the format conversion component for parsing. Depending on the encapsulation format of the video segments, the m3u8 files (i.e., the segment information files) corresponding to fMP4 (i.e., the video encapsulation format in the second video format mentioned above) and MPEG-TS (i.e., the video encapsulation format in the first video format mentioned above) have slightly different formats. The m3u8 file corresponding to fMP4 (i.e., the second segment information file) has an additional "#EXT-X-MAP:URI" tag, representing the file address of an initial segment. Therefore, a "#EXT-X-MAP:URI" tag needs to be added to the m3u8 file corresponding to MPEG-TS (i.e., the first segment information file), and the #EXT-X-MAP:URI tag should point to a virtual initial segment file. File data will only be generated when the player requests this file path. #EXT-X-VERSION indicates a compatible version of the m3u8 file. The #EXT-X-VERSION value in the fMP4 m3u8 file must be greater than or equal to 6. Therefore, the #EXT-X-VERSION value in the corresponding MPEG-TS m3u8 file also needs to be modified to a version greater than or equal to 6 to support the #EXT-X-MAP:URI tag.

[0155] After the video stream link is passed to the system player, the system player's requests for video segments can be proxied and replaced with converted video resources. For example, the system component AVAssetResourceLoader can be used to relay the system player's requests. When the format conversion component receives the system player's request to retrieve video segments, it can first download the original video segments to be played and then convert them into a format supported by the system player. The video segments to be played can be converted into a format supported by the system player in the following ways: 1. Use FFmpeg to transcode the video: First, determine whether the encoding format of the video segments to be played is H.264 or H.265. If it is H.265, the codec tag of the video stream needs to be changed to hvc1, not hev1. 2. Set movflags in FFmpeg (i.e., the video processing tool mentioned above) to frag_keyframe+empty_moov+delay_moov. frag_keyframe+empty_moov is used to fragment the MP4 video and output it in fMP4 format. `delay_moov` ensures the correct audio encoding format information is written to the `moov` (i.e., the first data unit mentioned above). 3. When encapsulating into fMP4, the audio ADTS header (i.e., the first audio header mentioned above) needs to be converted to MPEG-4 AudioSpecficConfig (i.e., the second audio header mentioned above), and the ADTS header needs to be removed, keeping only the ES stream. Therefore, the bitstream filter `aac_adtstoasc` in FFmpeg needs to be used. 4. Allocate a memory area to store the encapsulated data. The audio and video data in the video segment to be played need to have their display timestamp and decoding timestamp recalculated; different encapsulation formats have different time bases. Write the processed data sequentially into the newly allocated memory area. After processing all data, write the start playback timestamp of the video segment to be played into the new video segment. During the encapsulation process, if insufficient memory space is encountered, the memory will be doubled. After encapsulation, reorganize the memory size, release unfilled memory, and obtain the fMP4 data.

[0156] Although the fMP4 file has been obtained, it cannot be decoded and played by the system player as part of HLS because some information is missing. The parsable memory data is converted into an MP4 structure, which contains multiple boxes (i.e., the data units mentioned above). This parsing step can be done by writing parsing code based on the MP4 structure definition, or by using parsing code from existing open-source libraries, such as bento4. The ftyp (i.e., the second data unit mentioned above) and moov (i.e., the first data unit mentioned above) of the boxes in the structure are extracted and output sequentially into memory, resulting in init.mp4 (i.e., the initialization fragment file mentioned above). Since ftyp and moov represent global information of the video, any fragment can generate a correct init.mp4. Afterwards, tfdt (i.e., the third data unit mentioned above) and sidx (i.e., the fourth data unit mentioned above) are modified. Multiply the playback start time (in seconds) of the video segment to be played by the timescale to obtain the video display time and video decoding time. Modify `sidx.earliest_presentation_time` to the calculated video display time, and modify `tfdt.base_media_decode_time` to the calculated video decoding time. Remove the `ftyp` and `moov` boxes, and output the remaining boxes sequentially into memory. Finally, you will obtain a video that can be decoded by the native system player.

[0157] Compared to the computationally expensive method of decoding H.265 and then re-encoding it to H.264 to support native system players, this application reduces computational costs and improves format conversion efficiency by converting the video container format (re-encapsulation, converting MPEG-TS to fMP4), thus ensuring compatibility with playback software and devices.

[0158] The above application scenarios are merely illustrative. It is understood that the application of the video playback method provided in the embodiments of this application is not limited to the above scenarios.

[0159] Based on the same inventive concept, this application also provides a video playback device for implementing the video playback method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more video playback device embodiments provided below can be found in the limitations of the video playback method described above, and will not be repeated here.

[0160] In one embodiment, such as Figure 16 As shown, a video playback device 1600 is provided, including: a file modification module 1602, a video segment acquisition module 1604, and a format conversion module 1606, wherein:

[0161] The file modification module 1602 is used to obtain a first segment information file adapted to the first video format. The first segment information file contains video segment information for each of the multiple video segments in the first video format, and the video segment information includes video segment addresses. Based on the differences between the segment information files of the first video format and the second video format, the first segment information file is modified to obtain a second segment information file adapted to the second video format. The first video format is a video format that the video player does not support playing; the second video format is a video format that the video player supports playing.

[0162] The video segment acquisition module 1604 is used to obtain the address of the video segment to be played obtained by the video player parsing the second segment information file; and to obtain the video segment to be played in the first video format based on the address of the video segment to be played.

[0163] The format conversion module 1606 is used to convert the video segment to be played into a target video segment in a second video format, and instruct the video player to play the target video segment.

[0164] In one embodiment, the file modification module 1602 is further configured to, in response to a first selection operation of multiple video icons displayed by a video playback application, play the target video corresponding to the video icon selected in the first selection operation; in response to a resolution adjustment operation for the target video, display multiple resolution options; in response to a second selection operation for the multiple resolution options, determine the target resolution corresponding to the resolution option selected in the second selection operation; when the video player in the video playback application does not support the first video format corresponding to the target resolution, obtain a video stream link corresponding to the target video with the target resolution; and obtain a first segment information file adapted to the first video format based on the video stream link.

[0165] In one embodiment, the file modification module 1602 is further configured to determine the file content that the second segment information file adapted to the second video format should contain; and modify the first segment information file according to the difference between the file content contained in the first segment information file and the file content that the second segment information file adapted to the second video format should contain, so as to obtain the second segment information file adapted to the second video format.

[0166] In one embodiment, the file modification module 1602 is further configured to add a file address tag for an initialization segment to the first segment information file; and point the file address tag of the initialization segment to a virtual initialization segment file to obtain a second segment information file adapted to the second video format.

[0167] In one embodiment, the file modification module 1602 is further configured to determine the file compatibility version tag in the first segment information file; if the value of the file compatibility version tag is less than or equal to a preset version threshold, modify the value of the file compatibility version tag to be greater than the preset version threshold, so as to obtain a second segment information file that is compatible with the second video format.

[0168] In one embodiment, the video segment acquisition module 1604 is further configured to send a second segment information file to the video player; wherein the sent second segment information file is used to trigger the video player to parse the second segment information file, obtain the address of the video segment to be played that matches the current video playback progress, and generate a video segment acquisition request based on the address of the video segment to be played; and obtain the video segment acquisition request generated by the video player to obtain the video segment to be played in the first video format that matches the current video playback progress from the address of the video segment to be played.

[0169] In one embodiment, the format conversion module 1606 is further configured to extract the first audio data from the video segment to be played, and convert the audio header in the first audio data to obtain the second audio data adapted to the second video format; allocate a preset size of memory, and write the second audio data and the video data in the video segment to be played into the memory; determine the time data corresponding to the video data and the second audio data respectively; write the time data into the memory, and determine the target video segment to be converted into the second video format based on the data written into the memory.

[0170] In one embodiment, the format conversion module 1606 is further configured to call the bitstream filter in the video processing tool to extract the first audio header from the first audio data, and determine the audio features of the first audio data based on the first audio header; call the bitstream filter to generate a second audio header adapted to the second video format based on the audio features, insert the second audio header into the first audio data, and delete the first audio header from the first audio data to obtain the second audio data adapted to the second video format.

[0171] In one embodiment, either the second audio data or the video data in the video segment to be played is used as the target media data; the time data corresponding to the target media data includes the update timestamp under the target time type; the format conversion module 1606 is further configured to determine the first time base corresponding to the target media data according to the first video format; determine the second time base corresponding to the target media data according to the second video format; determine the original timestamp of the data frame in the target media data under the target time type; and determine the update timestamp of the data frame in the target media data under the target time type according to the first time base, the second time base and the original timestamp.

[0172] In one embodiment, the format conversion module 1606 is further configured to, for each data frame in the target media data, multiply the first time base by the original timestamp of the data frame under the target time type to obtain a multiplied timestamp; add the multiplied timestamp to half of the second time base to obtain a summed timestamp; and use the ratio of the summed timestamp to the second time base as the update timestamp of the data frame under the target time type.

[0173] In one embodiment, the format conversion module 1606 is further configured to write the audio encoding format information into a preset first data unit when the audio encoding format information of the second audio data is read; wherein the first data unit is a data unit that stores the metadata of the media; and store the first data unit in memory.

[0174] In one embodiment, the format conversion module 1606 is further configured to extract data from memory, and parse the extracted data according to the data structure corresponding to the second video format to obtain at least one data unit; extract a first data unit and a second data unit from the at least one data unit; write the first data unit and the second data unit into memory; wherein the first data unit is a data unit storing the metadata of the media, and the second data unit is a data unit storing file type and compatibility information; delete the first data unit and the second data unit from the at least one data unit; modify the remaining data units, and write the modified data units into memory to obtain the target video segment converted to the second video format.

[0175] In one embodiment, the format conversion module 1606 is further configured to determine the time scaling factor corresponding to the second video format and the playback start time of the video segment to be played, and generate video display time and video decoding time according to the time scaling factor corresponding to the second video format and the playback start time of the video segment to be played; modify the third data unit in the remaining data units according to the video display time; the third data unit is a unit that stores the video segment index; modify the fourth data unit in the remaining data units according to the video decoding time; the fourth data unit is a unit that stores a time stream description table.

[0176] Each module in the aforementioned video playback device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0177] In one embodiment, a computer device is provided, which may be a server or a terminal. The following description uses a terminal as an example, and the internal structure diagram of the terminal can be as follows. Figure 17As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a video playback method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0178] Those skilled in the art will understand that Figure 17 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0179] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0180] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0181] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0182] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0183] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0184] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A video playback method, characterized in that, The method includes: Obtain a first segment information file adapted to the first video format. The first segment information file contains video segment information for each of the multiple video segments of the first video format. The video segment information includes video segment addresses. The first video format is a video format that is not supported by the video player. Determine the file content that the second segment information file, which is adapted to the second video format, should contain; Based on the difference between the file content contained in the first segment information file and the file content that the second segment information file adapted to the second video format should contain, the first segment information file is modified to obtain a second segment information file adapted to the second video format; the second video format is a video format supported by the video player. Obtain the address of the video segment to be played obtained by the video player parsing the second segment information file; the video segment address in the first segment information file and the address of the video segment to be played in the second segment information file are both addresses of the video segments in the server; Based on the address of the video segment to be played, obtain the video segment to be played in the first video format; The video segment to be played is converted into a target video segment in the second video format, and the video player is instructed to play the target video segment.

2. The method according to claim 1, characterized in that, The step of obtaining the first segment information file adapted to the first video format includes: In response to a first selection operation on multiple video icons displayed by a video playback application, the target video corresponding to the video icon selected in the first selection operation is played. In response to a resolution adjustment operation for the target video, multiple resolution options are displayed; In response to a second selection operation for the plurality of sharpness options, the target sharpness corresponding to the sharpness option selected by the second selection operation is determined; When the video player in the video playback application does not support the first video format corresponding to the target resolution, a video stream link corresponding to the target video with the target resolution is obtained. Obtain the first segment information file adapted to the first video format based on the video stream link.

3. The method according to claim 1, characterized in that, The step of modifying the first segment information file based on the difference between the file content contained in the first segment information file and the file content that the second segment information file adapted to the second video format should contain, to obtain the second segment information file adapted to the second video format, includes: Add the file address tag of the initialization fragment to the first fragment information file; The file address tag of the initialization segment is pointed to a virtual initialization segment file to obtain a second segment information file adapted to the second video format.

4. The method according to claim 1, characterized in that, The step of modifying the first segment information file based on the difference between the file content contained in the first segment information file and the file content that the second segment information file adapted to the second video format should contain, to obtain the second segment information file adapted to the second video format, includes: Determine the file compatibility version tags in the first fragment information file; If the value of the file compatibility version tag is less than or equal to a preset version threshold, the value of the file compatibility version tag is modified to be greater than the preset version threshold to obtain a second segment information file that is compatible with the second video format.

5. The method according to claim 1, characterized in that, The step of obtaining the address of the video segment to be played obtained by the video player parsing the second segment information file includes: Send the second segment information file to the video player; The second segment information file sent is used to trigger the video player to parse the second segment information file, obtain the address of the video segment to be played that matches the current video playback progress, and generate a video segment acquisition request based on the address of the video segment to be played. The step of obtaining the video segment to be played in the first video format based on the address of the video segment to be played includes: Obtain the video segment retrieval request generated by the video player, so as to obtain the video segment to be played that matches the current video playback progress and is in the first video format from the address of the video segment to be played.

6. The method according to claim 1, characterized in that, The step of converting the video segment to be played into a target video segment in the second video format includes: Extract the first audio data from the video segment to be played, and convert the audio header in the first audio data to obtain the second audio data adapted to the second video format; Allocate a preset amount of memory and write the second audio data and the video data from the video segment to be played into the memory; Determine the time data corresponding to the video data and the second audio data respectively; The time data is written into the memory, and the target video segment to be converted into the second video format is determined based on the data written into the memory.

7. The method according to claim 6, characterized in that, The step of converting the audio header in the first audio data to obtain second audio data adapted to the second video format includes: The first audio header in the first audio data is extracted by calling the bitstream filter in the video processing tool, and the audio features of the first audio data are determined based on the first audio header. The bitstream filter is invoked to generate a second audio header adapted to the second video format based on the audio features. The second audio header is inserted into the first audio data, and the first audio header is deleted from the first audio data to obtain second audio data adapted to the second video format.

8. The method according to claim 6, characterized in that, The target media data is either the second audio data or the video data in the video segment to be played; the time data corresponding to the target media data includes the update timestamp under the target time type. The steps for determining the update timestamp of the target media data under the target time type include: Based on the first video format, determine the first time base corresponding to the target media data; Based on the second video format, determine the second time base corresponding to the target media data; Determine the original timestamp of the data frame in the target media data under the target time type; Based on the first time base, the second time base, and the original timestamp, determine the update timestamp of the data frame in the target media data under the target time type.

9. The method according to claim 8, characterized in that, The step of determining the update timestamp of the data frame in the target media data under the target time type based on the first time base, the second time base, and the original timestamp includes: For each data frame in the target media data, the first time base is multiplied by the original timestamp of the data frame under the target time type to obtain the multiplied timestamp; Add the multiplied timestamp to half of the second time base to obtain the summed timestamp; The ratio of the summed timestamp to the second time base is used as the update timestamp of the data frame under the target time type.

10. The method according to claim 6, characterized in that, The method further includes: Upon reading the audio encoding format information of the second audio data, the audio encoding format information is written into a preset first data unit; wherein, the first data unit is a data unit storing the metadata of the media; The first data unit is stored in the memory.

11. The method according to claim 6, characterized in that, The step of determining the target video segment to be converted to the second video format based on the data written into the memory includes: Extract the data from the memory, and parse the extracted data according to the data structure corresponding to the second video format to obtain at least one data unit; Extract the first data unit and the second data unit from the at least one data unit; The first data unit and the second data unit are written into the memory; wherein, the first data unit is a data unit that stores the metadata of the media, and the second data unit is a data unit that stores file type and compatibility information; Delete the first data unit and the second data unit from the at least one data unit; The remaining data units are modified and written into the memory to obtain the target video segments converted to the second video format.

12. The method according to claim 11, characterized in that, The modification of the remaining data units includes: Determine the time scaling factor corresponding to the second video format and the playback start time of the video segment to be played, and generate the video display time and video decoding time based on the time scaling factor corresponding to the second video format and the playback start time of the video segment to be played; The third data unit in the remaining data units is modified according to the video display time; the third data unit is a unit that stores the video segment index. The fourth data unit in the remaining data units is modified according to the video decoding time; the fourth data unit is a unit that stores a time stream description table.

13. A video playback device, characterized in that, The device includes: The file modification module is used to obtain a first segment information file adapted to a first video format, wherein the first segment information file contains video segment information of multiple video segments of the first video format, and the video segment information includes video segment addresses; determine the file content that a second segment information file adapted to a second video format should contain; modify the first segment information file according to the difference between the file content contained in the first segment information file and the file content that the second segment information file adapted to the second video format should contain, to obtain a second segment information file adapted to the second video format; wherein the first video format is a video format that the video player does not support; and the second video format is a video format that the video player supports. The video segment acquisition module is used to acquire the address of the video segment to be played obtained by the video player parsing the second segment information file; and to acquire the video segment to be played in the first video format according to the address of the video segment to be played; the video segment address in the first segment information file and the video segment address to be played in the second segment information file are both addresses of the video segments in the server; The format conversion module is used to convert the video segment to be played into a target video segment of the second video format, and instruct the video player to play the target video segment.

14. The apparatus according to claim 13, characterized in that, The file modification module is also used to respond to a first selection operation of multiple video icons displayed by the video playback application and play the target video corresponding to the video icon selected by the first selection operation. In response to a resolution adjustment operation for the target video, multiple resolution options are displayed; In response to a second selection operation for the plurality of sharpness options, the target sharpness corresponding to the sharpness option selected by the second selection operation is determined; When the video player in the video playback application does not support the first video format corresponding to the target resolution, a video stream link corresponding to the target video with the target resolution is obtained. Obtain the first segment information file adapted to the first video format based on the video stream link.

15. The apparatus according to claim 13, characterized in that, The file modification module is also used to add a file address tag for the initialization segment to the first segment information file; and to point the file address tag of the initialization segment to a virtual initialization segment file to obtain a second segment information file adapted to the second video format.

16. The apparatus according to claim 13, characterized in that, The file modification module is also used to determine the file compatibility version tag in the first segment information file; if the value of the file compatibility version tag is less than or equal to a preset version threshold, the value of the file compatibility version tag is modified to be greater than the preset version threshold, so as to obtain a second segment information file that is compatible with the second video format.

17. The apparatus according to claim 13, characterized in that, The video segment acquisition module is further configured to send the second segment information file to the video player; wherein, the sent second segment information file is used to trigger the video player to parse the second segment information file, obtain the address of the video segment to be played that matches the current video playback progress, and generate a video segment acquisition request based on the address of the video segment to be played; and obtain the video segment acquisition request generated by the video player to obtain the video segment to be played in the first video format that matches the current video playback progress from the address of the video segment to be played.

18. The apparatus according to claim 13, characterized in that, The format conversion module is also used to extract the first audio data from the video segment to be played, and convert the audio header in the first audio data to obtain the second audio data adapted to the second video format; allocate memory of a preset size, and write the second audio data and the video data in the video segment to be played into the memory; Determine the time data corresponding to the video data and the second audio data respectively; The time data is written into the memory, and the target video segment to be converted into the second video format is determined based on the data written into the memory.

19. The apparatus according to claim 18, characterized in that, The format conversion module is also used to call the bitstream filter in the video processing tool to extract the first audio header in the first audio data, and determine the audio features of the first audio data based on the first audio header; The bitstream filter is invoked to generate a second audio header adapted to the second video format based on the audio features. The second audio header is inserted into the first audio data, and the first audio header is deleted from the first audio data to obtain second audio data adapted to the second video format.

20. The apparatus according to claim 18, characterized in that, The target media data is either the second audio data or the video data in the video segment to be played; the time data corresponding to the target media data includes the update timestamp under the target time type. The format conversion module is further configured to: determine a first time base corresponding to the target media data according to the first video format; determine a second time base corresponding to the target media data according to the second video format; determine the original timestamp of the data frame in the target media data under the target time type; and determine the update timestamp of the data frame in the target media data under the target time type according to the first time base, the second time base and the original timestamp.

21. The apparatus according to claim 20, characterized in that, The format conversion module is further configured to, for each data frame in the target media data, multiply the first time base by the original timestamp of the data frame under the target time type to obtain a multiplied timestamp; and add the multiplied timestamp to half of the second time base to obtain a summed timestamp. The ratio of the summed timestamp to the second time base is used as the update timestamp of the data frame under the target time type.

22. The apparatus according to claim 18, characterized in that, The format conversion module is further configured to, upon reading the audio encoding format information of the second audio data, write the audio encoding format information into a preset first data unit; wherein, the first data unit is a data unit storing the metadata of the media; and store the first data unit in the memory.

23. The apparatus according to claim 18, characterized in that, The format conversion module is further configured to extract data from the memory, and parse the extracted data according to the data structure corresponding to the second video format to obtain at least one data unit; extract a first data unit and a second data unit from the at least one data unit; write the first data unit and the second data unit into the memory; wherein the first data unit is a data unit storing media metadata, and the second data unit is a data unit storing file type and compatibility information; delete the first data unit and the second data unit from the at least one data unit; modify the remaining data units, and write the modified data units into the memory to obtain a target video segment converted to the second video format.

24. The apparatus according to claim 23, characterized in that, The format conversion module is further configured to determine the time scaling factor corresponding to the second video format and the playback start time of the video segment to be played, and generate video display time and video decoding time according to the time scaling factor corresponding to the second video format and the playback start time of the video segment to be played; modify the third data unit in the remaining data unit according to the video display time; the third data unit is a unit that stores the video segment index; modify the fourth data unit in the remaining data unit according to the video decoding time; the fourth data unit is a unit that stores a time stream description table.

25. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 12.

26. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 12.

27. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 12.

Images