Video data transpackaging method, device, electronic device and storage medium

By decomposing streaming video data into Group of Pictures (GOPs) and determining the indexed structured data, the problem of low efficiency in transcoding streaming video data is solved, achieving efficient transcoding and saving storage resources in the event of transmission interruption.

CN118509651BActive Publication Date: 2024-11-12HANGZHOU HIKVISION SYST TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410975561.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-11-12
Estimated Expiration
2044-07-18

AI Technical Summary

Technical Problem

Existing technologies are prone to inefficiency when converting streaming video data into MP4 files due to transmission interruptions, making it impossible to achieve efficient index data generation.

Method used

By performing frame analysis on streaming video data, it is decomposed into multiple GOP data, and index structured data is determined in each GOP data until all data is transcapitalized to generate MP4 index data, thus avoiding the need to reread historical data due to transmission interruption.

Benefits of technology

It improves the efficiency of converting streaming video data into MP4 files, ensures a continuous conversion process even in the event of transmission interruption, and saves storage resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118509651B_ABST
    Figure CN118509651B_ABST
Patent Text Reader

Abstract

The embodiment of the present application provides a video data transpackaging method, device, electronic device and storage medium, which relates to the field of multimedia processing technology and is used to improve the efficiency of transpackaging streaming video data into MP4 files. The method includes: acquiring streaming video data; performing frame analysis on the streaming video data to determine multiple first picture group GOP data in the streaming video data; transpackaging each first GOP data into second GOP data in MP4 format, and determining the index structured data corresponding to each second GOP data during the transpackaging process; the index structured data includes frame information of each frame in the second GOP data; when it is determined that all the streaming video data are transpackaged into the second GOP data, determining the MP4 index data corresponding to the streaming video data based on all the index structured data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of multimedia processing technology, and in particular to a video data transpackaging method, device, electronic device and storage medium. Background Art

[0002] Since the display device may not support direct playback of the streaming video data in the original bitstream format collected by the acquisition device, it is necessary to encapsulate the streaming video data in the original bitstream format into video data in MP4 (MPEG-4 Part 14) format to facilitate playback by the display device.

[0003] In the related art, streaming video data is usually encapsulated into an MP4 file while being read, and index data of the MP4 data is generated during the transpackaging process. However, in the process of reading the streaming video data, storage interruption or intermittent recording may easily occur, that is, the streaming video data transmission is interrupted. This situation will cause the MP4 transpackaging to be interrupted. During the subsequent transpackaging, it is necessary to read the previously stored streaming video data and then generate a complete MP4 file for storage. In this case, the efficiency of transpackaging the streaming video data into an MP4 file is low. Summary of the invention

[0004] The present application provides a video data transpackaging method, device, electronic device and storage medium, which are used to generate corresponding index structured data during the process of transpackaging streaming video data into MP4 files. After all the streaming video data are transpackaged, corresponding MP4 index data are generated according to all the index structured data to improve the efficiency of transpackaging streaming video data into MP4 files.

[0005] A first aspect of an embodiment of the present application provides a method for transpackaging video data, comprising: acquiring streaming video data; performing frame analysis on the streaming video data to determine multiple first picture group GOP data in the streaming video data; transpackaging each first GOP data into second GOP data in MP4 format, and determining index structured data corresponding to each second GOP data during the transpackaging process; the index structured data includes frame information of each frame in the second GOP data; when it is determined that all the streaming video data is transpackaged into second GOP data, determining the MP4 index data corresponding to the streaming video data based on all the index structured data.

[0006] The video data transpackaging method provided in the embodiment of the present application does not directly generate MP4 index data during the transpackaging process, but determines the index structured data corresponding to each second GOP data, that is, the frame information of each frame in the second GOP data. Only after all the streaming video data are transpackaged into the second GOP data, the MP4 index data is generated according to all the index structured data, that is, the MP4 index data is generated according to the frame information of all frames. It can be seen that the video data transpackaging method provided in the embodiment of the present application generates MP4 index data according to the frame information of all frames. During the transpackaging process, regardless of whether the transmission of the streaming video data is interrupted, there is no need to re-read the historically stored streaming video data, which can avoid the problem of low efficiency in generating MP4 index data due to the interruption of the transmission of the streaming video data, that is, it can ensure the efficiency of transpackaging of the streaming video data.

[0007] In combination with the first implementation method of the first aspect, the method also includes: in the event that a streaming video data transmission interruption is detected, storing the first index structured data before the transmission interruption; in response to the continued transmission of the streaming video data, reading the first index structured data, and continuing to transpackaging according to the first index structured data, and determining the second index structured data during the transpackaging process; determining the MP4 index data corresponding to the streaming video data based on the index structured data, including: determining the MP4 index data corresponding to the streaming video data based on the first index structured data and the second index structured data.

[0008] In combination with the second implementation manner of the first aspect, after storing the MP4 index data corresponding to the streaming video data, the method further includes: deleting the first index structured data.

[0009] In combination with the third implementation manner of the first aspect, the method also includes: storing streaming video data and MP4 index data corresponding to the streaming video data; in response to receiving a play instruction sent by a display device, sending the MP4 index data corresponding to the streaming video data to the display device; transpackaging the streaming video data into MP4 data in MP4 format, and sending it to the display device, so that the display device displays the MP4 data based on the MP4 index data.

[0010] In combination with the fourth implementation manner of the first aspect, the method also includes: during the transpackaging process, determining the offset relationship between the first GOP data and the second GOP data; transpackaging the streaming video data into MP4 data in MP4 format, and sending it to the display device, including: in response to receiving a positioning playback instruction, determining the starting offset position indicated by the positioning playback instruction; the positioning playback instruction is used to indicate that the video should be played from the starting offset position relative to the MP4 video data; determining the target first GOP data corresponding to the starting offset position in the streaming video data according to the offset relationship; packaging the target first GOP data into target second GOP data, and sending the data after the starting offset position in the target second GOP data to the display device; transpackaging the first GOP data after the target first GOP data in the streaming video data into second GOP data in chronological order, and sending it to the display device.

[0011] In combination with the fifth implementation manner of the first aspect, the format of the streaming video data is a program stream PS format.

[0012] In combination with a sixth implementation of the first aspect, obtaining streaming video data includes: in response to receiving a video recording request, obtaining streaming video data according to a stream acquisition address carried in the video recording request.

[0013] The second aspect of an embodiment of the present application provides a video data transpackaging device, including: an acquisition module, used to acquire streaming video data; a determination module, used to perform frame analysis on the streaming video data to determine multiple first picture group GOP data in the streaming video data; a transpackaging module, used to transpack each first GOP data into second GOP data in MP4 format, and determine the index structured data corresponding to each second GOP data during the transpackaging process; the index structured data includes frame information of each frame in the second GOP data; the determination module is also used to determine the MP4 index data corresponding to the streaming video data based on all the index structured data when it is determined that all the streaming video data is transpackaged into second GOP data.

[0014] A third aspect of an embodiment of the present application provides an electronic device, comprising: one or more processors; one or more memories; wherein the one or more memories are used to store computer program codes, the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the video data transpackaging method provided by the first aspect and its possible implementation methods.

[0015] The fourth aspect of the embodiments of the present application provides a computer-readable storage medium, which stores computer execution instructions. When the computer execution instructions are executed on a computer, the computer executes the video data transpackaging method provided by the first aspect and its possible implementation methods.

[0016] A fifth aspect of an embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the video data transpackaging method provided in the first aspect and its possible implementation methods.

[0017] Among them, the beneficial effects described in the second to fifth aspects can refer to the beneficial effect analysis of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0019] Figure 1 A schematic diagram of a transpackaging MP4 data provided in an embodiment of the present application Figure 1 ;

[0020] Figure 2 A schematic diagram of the structure of an MP4 file provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of a transpackaging MP4 data provided in an embodiment of the present application Figure 2 ;

[0022] Figure 4 A schematic diagram of a video data transpackaging system provided in an embodiment of the present application;

[0023] Figure 5 A method flow of a video data transpackaging method provided in an embodiment of the present application Figure 1 ;

[0024] Figure 6 A method flow of a video data transpackaging method provided in an embodiment of the present application Figure 2 ;

[0025] Figure 7 A schematic diagram of determining MP4 index data provided by an embodiment of the present application;

[0026] Figure 8 A method flow of a video data transpackaging method provided in an embodiment of the present application Figure 3 ;

[0027] Fig. 9 A method flow of a video data transpackaging method provided in an embodiment of the present application Figure 4 ;

[0028] Fig.10A schematic diagram of positioning playback provided in an embodiment of the present application;

[0029] Fig.11 An overall flow chart of a video data transpackaging method provided in an embodiment of the present application;

[0030] Fig.12 A schematic diagram of the structure of a video data transpackaging device provided in an embodiment of the present application;

[0031] Fig.13 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0034] In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances. In addition, when describing a pipeline, the "connected" and "connection" used in this application have the meaning of conduction. The specific meaning needs to be understood in conjunction with the context.

[0035] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0036] In the related art, a capture device (such as a camera) is usually set up to capture video of an area, and then the user can browse the video captured by the capture device through a display device. However, the capture device usually encapsulates the captured streaming video data into a format that is convenient for data transmission, such as using a program stream (PS) encapsulation method or a transport stream (TS) encapsulation method. This encapsulation method is conducive to the efficient transmission of streaming video data.

[0037] However, the display device may not support direct playback of streaming video data in this type of encapsulation format, and it needs to be transcapsulated into video data in other encapsulation formats before display. Since MP4-encapsulated (MPEG-4 Part 14) files have the advantages of clear images and moderate file size, they are suitable for playback on a variety of devices and operating systems, so the acquired streaming video data needs to be transcapsulated into video data in MP4 format.

[0038] In the MP4 file format, index data (moov box) is a key component. The moov box does not contain specific video data, but contains macro-description information of all media data in the MP4 file, such as audio and video description information, timestamps, frame rates, and other data related to media playback. By reading the data in the moov box, the display device can quickly obtain the structure of the media file and the location of each data block, so as to play and parse it efficiently. In other words, before playing the MP4 file, the display device needs to obtain the moov box of the MP4 file before it can correctly parse and present the MP4 file. Therefore, when encapsulating the streaming video data into an MP4 file, it is necessary to generate the moov box in the MP4 file so that the display device can correctly parse and present the MP4 file according to the moov box when playing the video.

[0039] Therefore, when transpackaging streaming video data into an MP4 file, it is necessary to determine the index data of the MP4 file.

[0040] For example, see Figure 1 In the related art, when transpackaging streaming video data into an MP4 file, the following steps are included:

[0041] S101. Acquire streaming video data collected by a collection device.

[0042] S102: The streaming video data is stored in the storage space through the storage service, and the streaming video data is transcapsulated into MP4 data in real time.

[0043] S103: Generate index data of the MP4 data during the transpackaging process.

[0044] It is understandable that an MP4 file consists of multiple boxes (or atoms), each of which stores different information, such as file type, metadata, media data, etc. Figure 2 As shown, the main structure of an MP4 file includes: a file type box (FileType Box, ftyp), a metadata box (moov box, moov), and a media data box (Media Data Box, mdat).

[0045] Among them, ftyp is used to describe the specification version followed by the MP4 file; moov box is the index data in the embodiment of the present application, which contains metadata about the entire file, such as time scale, number of tracks, etc.; mdat contains actual media data, such as video frames, audio samples, etc.

[0046] like Figure 2 As shown, according to the position of the moov box in the MP4 file, the MP4 file can be divided into a pre-index MP4 file and a post-index MP4 file. The moov box in the pre-index MP4 file is at the head of the MP4 file. After the display device reads the moov box, it can read and play the MP4 file. The moov box in the post-index MP4 file is at the end of the MP4 file, which supports playback on general display devices without generating MP4 index data in advance. When the user requests to download the MP4 file, the PS stream can be read from the disk in real time, converted to MP4 encapsulation, and the index data can be placed at the end of the MP4 file and returned to the client. However, the online preview effect of the post-index MP4 file is poor. When displaying, the entire file needs to be downloaded before the index data can be obtained to present the media data screen.

[0047] Therefore, in order to ensure that the display device can preview the video screen online in real time, the relevant technology usually uses a pre-index MP4 file, that is, while reading the streaming video data, the streaming video data is encapsulated as a pre-index MP4 file to meet the real-time requirements of users when previewing the video screen online.

[0048] However, in the process of reading streaming video data, it is easy to have storage interruption or intermittent recording, that is, streaming video data transmission interruption. Figure 3 As shown, in the case of storing streaming video data 1 and streaming video data 2, this situation will cause the MP4 encapsulation to be interrupted. When the encapsulation is subsequently performed, the previously stored streaming video data 1 needs to be read and then a complete MP4 file is generated to obtain the corresponding MP4 index data. Since the amount of streaming video data is usually large, this situation will result in a low efficiency in encapsulating the streaming video data into an MP4 file.

[0049] Based on this, the embodiment of the present application provides a method for transpackaging video data. After acquiring the streaming video data, the streaming video data is first analyzed by frames, and the streaming video data is decomposed into a plurality of first group of pictures (Group of Pictures, GOP) data, and then each first GOP data is transpacked into second GOP data in MP4 format, and the frame information of each frame in each second GOP data is determined during the transpackaging process to obtain the index structured data corresponding to each second GOP data. When all the streaming video data are transpacked into the second GOP data, the index structured data corresponding to all the second GOP data, that is, the frame information of all frames, can be obtained, and then the MP4 index data can be generated according to all the index structured data.

[0050] It can be seen that the video data transpackaging method provided in the embodiment of the present application does not directly generate MP4 index data during the transpackaging process, but determines the index structured data corresponding to each second GOP data, that is, the frame information of each frame in the second GOP data. Only after all the streaming video data are transpackaged into the second GOP data, the MP4 index data is generated according to all the index structured data, that is, the MP4 index data is generated according to the frame information of all frames. It can be seen that the video data transpackaging method provided in the embodiment of the present application generates MP4 index data according to the frame information of all frames. During the transpackaging process, regardless of whether the transmission of the streaming video data is interrupted, there is no need to re-read the historically stored streaming video data, which can avoid the problem of low efficiency in generating MP4 index data due to the interruption of the transmission of the streaming video data, that is, it can ensure the efficiency of transpackaging of the streaming video data.

[0051] The video data transpackaging method provided in the embodiment of the present application can be applied to Figure 4 For the video data transpackaging system shown in Figure 4 The video data transpackaging system 10 includes: an acquisition device 11 and a transpackaging device 12.

[0052] The acquisition device 11 is used to acquire streaming video data of the area to be acquired, and send it to the transpackaging device 12 for storage.

[0053] After receiving the streaming video data, the transpackaging device 12 transpacks it into MP4 data in real time, and determines the index structured data during the transpackaging process, until the streaming video data is completely transpacked into MP4 data, and then generates MP4 index data according to all the index structured data.

[0054] In some embodiments, Figure 4As shown, the video data transpackaging system 10 may further include a display device 13, in which a browser or video playback software may be installed, and the video data stored in the transpackaging device 12 may be obtained for video display.

[0055] It should be understood that the acquisition device 11 in the embodiment of the present application can be a device with a video recording function such as a camera or a video recorder, and the embodiment of the present application does not limit this.

[0056] The transpackaging device 12 in the embodiment of the present application may be a personal computer (PC), a laptop computer, a mobile device, a tablet computer, a laptop computer, or other electronic device. The embodiment of the present application does not limit the specific form of the transpackaging device 12. Alternatively, the transpackaging device 12 may be a single server or a server cluster composed of multiple servers. In some implementations, the server cluster may be a distributed cluster server. The embodiment of the present application does not impose any limitation on this.

[0057] It should be pointed out that Figure 4 The structure shown does not constitute a limitation on the video data transpackaging system. In other embodiments, the system may include fewer or more components than shown in the figure, or a combination of certain components, or a different arrangement of components. The embodiments of the present application do not impose any limitations on this.

[0058] The video data transpackaging method provided in the embodiment of the present application can be applied to Figure 4 The transpackaging device 12 in the transpackaging system of the video data shown. Figure 5 , the video data transpackaging method comprises the following steps:

[0059] S501: Obtain streaming video data.

[0060] Streaming video data is a type of video data that is transmitted continuously and in real time. The transpackaging device 12 may obtain the streaming video data from an acquisition device or from other storage devices, and the embodiment of the present application does not impose any limitation on this.

[0061] As a feasible implementation method, when the acquisition device or other device needs the transpackaging device to save the video data, it will send a recording request to the transpackaging device. The recording request may include the stream address, the unique identifier of the required storage file, and the transpackaging requirements. This application does not limit this. After receiving the recording request, the transpackaging device obtains the streaming video data according to the stream address carried in the recording request. And save it in its own storage space. If the recording request specifies that it needs to be transpackaged into MP4 format video data, it will be transpackaged into MP4 video data while saving the streaming video data.

[0062] It should be understood that the streaming video data in the present application may be PS streaming video data or TS streaming video data, that is, streaming video data in PS encapsulation format or streaming video data in TS encapsulation format, and the embodiments of the present application do not impose any restrictions on this.

[0063] As a feasible implementation method, since streaming video data in the PS encapsulation format can generate data packets of non-fixed length according to the specific audio and video data content, it has advantages in storage applications, and PS encapsulation has high clarity in data management and transmission. Therefore, the streaming video data can be streaming video data in the PS encapsulation format.

[0064] S502: Perform frame analysis on the streaming video data to determine a plurality of first GOP data in the streaming video data.

[0065] It is understood that GOP is a term used in video coding to describe a group of continuous pictures. It is a complete set of video frames, which contains a key frame (Intra-coded frame, I frame) and may contain multiple forward prediction frames (Predictive-coded frame, P frame) and bidirectional prediction frames (Bidirectionally predictive-coded frame, B frame). Each GOP starts with an I frame, which is a complete image that can be decoded and displayed independently. Next are zero or more P frames, which represent the difference from the previous frame and depend on the I frame or the previous P frame for decoding. There may also be zero or more B frames, which depend on the previous I frame or P frame and the following P frame or I frame for decoding.

[0066] Therefore, when performing frame analysis on streaming video data, I frames can be identified in the streaming video data first. This can usually be done by checking the encoding type or metadata of the frame. Then the GOP length, that is, the number of frames contained in a GOP, is determined. The GOP length can vary according to the settings of the video encoder and can usually be found in the encoder parameters or the metadata of the video. After the I frame is identified and the GOP length is determined, starting from the I frame of the streaming video data, continuous frames can be extracted according to the GOP length as a first GOP data to obtain multiple first GOP data.

[0067] S503: transpackaging each first GOP data into second GOP data in MP4 format, and determining index structured data corresponding to each second GOP data during the transpackaging process.

[0068] It can be understood that the MP4 encapsulation process is to organize and store video, audio and other media data in accordance with the provisions of the MP4 format. That is, the first GOP data is reorganized according to the provisions of the MP4 format. The codec or data stream format in the MP4 format includes: Advanced Video Coding (H.264 / AVC), High Efficiency Video Coding (H.265 / HEVC), the eighth generation video coding (Video Codec8, VP8), VP9, ​​etc., that is, the first GOP data can be reorganized into data in H.264 / AVC, H.265 / HEVC or VP8 / 9 format, and the embodiments of the present application are not limited to this.

[0069] During the transpackaging process, frame information of frames included in each second GOP data may be determined, and then index structured data corresponding to the second GOP data may be obtained.

[0070] Exemplarily, the frame information may include the following aspects:

[0071] Frame type: I frame: intra-frame coding, does not rely on other frames for decoding, and completely saves a frame of image information; P frame: previous predictive coding frame, encoded by referring to the previous encoded frame (I frame or P frame); B frame: bidirectional predictive coding frame, referring to both the previous frame and the following frame; Instantaneous Decoding Refresh frame (IDR frame): exists in H.264 and is used for random access and synchronization. The decoder will clear the reference frame buffer after receiving the IDR frame.

[0072] Coding parameter information: Coding standards, such as H.264 / AVC, HEVC, etc., are used to specify the encoding format of the frame; Quantization parameter (QP), a key parameter that affects video quality and compression ratio. The larger the value, the higher the compression rate, but the worse the image quality; Macroblock type: In coding standards such as H.264, frames are divided into macroblocks for encoding. Macroblock types include I macroblocks, P macroblocks, and B macroblocks.

[0073] Time information: Presentation Time Stamp (PTS), used for video synchronization and output; Decoding Time Stamp (DTS), used for video decoding. In the presence of B frames, the output order of DTS and PTS may be different.

[0074] Frame header information: contains metadata such as frame type, encoding parameters, and time information.

[0075] Synchronization and delimitation information: Frame start and end flags: used to identify the start and end positions of the frame so that the decoder can correctly parse the frame data.

[0076] Additional information: may also include frame number, scene switching information, lens information, etc.

[0077] It should be understood that the embodiments of the present application do not specifically limit the content of the frame information. In actual applications, different coding standards and application scenarios may contain different frame information, which can be adjusted and expanded according to the needs of actual applications.

[0078] S504: When it is determined that all the streaming video data are transpacked into the second GOP data, determine the MP4 index data corresponding to the streaming video data based on all the index structured data.

[0079] When all the streaming video data are transcapsulated into the second GOP data, the MP4 index data corresponding to the streaming video data can be determined according to all the index structured data, that is, the index data of the MP4 video data can be determined according to the frame information of all the frames in the MP4 video data.

[0080] It should be understood that the MP4 index data contains the metadata information of the MP4 file, such as: the basic information of the MP4 file (such as duration, timestamp, etc.), the creation time, modification time, time measurement scale, playable duration of the MP4 file, the basic information of the specific media track (such as duration, position, etc.), track ID, type (audio or video), width, height, rotation angle, media information of the media stream (such as media type, codec information), etc. This application does not limit this.

[0081] After obtaining the MP4 index data corresponding to the streaming video data, if the display device needs to display the streaming video data later, the MP4 video data corresponding to the streaming video data can be correctly decoded and played according to the MP4 index data for user reference.

[0082] It can be seen that the video data transpackaging method provided in the embodiment of the present application does not directly generate MP4 index data during the transpackaging process, but determines the index structured data corresponding to each second GOP data, and generates MP4 index data based on all the index structured data after all the streaming video data are transpackaged into the second GOP data. Regardless of whether the transmission of the streaming video data is interrupted, there is no need to re-read the historically stored streaming video data, which can avoid the problem of low efficiency in generating MP4 index data due to the interruption of the transmission of the streaming video data, that is, it can ensure the efficiency of transpackaging the streaming video data.

[0083] In some embodiments, during the process of transmitting streaming video data from the acquisition device to the transcapsulation device, the streaming video data transmission may be interrupted due to network problems, server problems or intermittent recording problems. Therefore, during the process of transcapsulating the streaming video data into MP4 data, it is necessary to detect in real time whether the streaming video data transmission is interrupted.

[0084] As a possible implementation, see Figure 6 The video data transpackaging method provided in the embodiment of the present application further includes the following steps:

[0085] S601: When a streaming video data transmission interruption is detected, store the first index structured data before the transmission interruption.

[0086] During the normal transmission of streaming video data, the transpackaging device will continuously receive data. If the transpackaging device does not receive data within a preset time period, it can be determined that the streaming video data transmission is interrupted. At this time, the first index structured data before the transmission interruption can be stored in the storage space of the transpackaging device for subsequent use.

[0087] S602: In response to the continued transmission of the streaming video data, read the first index structured data, continue to transpackaging according to the first index structured data, and determine the second index structured data during the transpackaging process.

[0088] When the streaming video data continues to be transmitted, the first index structured data stored in the storage space is read and passed into the transpackaging context (or "session" or "stream"). This context usually contains information about the MP4 format, output settings, and possible other metadata. Then the streaming video data is continued to be transpackaged, and the parsed media data is passed into this context to obtain the second index structured data.

[0089] It can be understood that, in the case where the streaming video data transmission is interrupted, the above S604 can be specifically implemented as: determining the MP4 index data corresponding to the streaming video data based on the first index structured data and the second index structured data.

[0090] For example, see Figure 7 In the case of transmission interruption of streaming video data, that is, there is a certain time interval between time period 1 and time period 2 when no streaming video data is transmitted, the streaming video data can be divided into data 1 and data 2, the first index structured data corresponding to data 1 is transferred to the transpackaging context, and data 2 is continuously packaged to obtain the second index structured data. Then, the MP4 index data of the MP4 file is obtained according to the first index structured data and the second index structured data.

[0091] It can be seen that the video data transpackaging method provided by the embodiment of the present application does not need to read the streaming video data before the interruption when the streaming video data is interrupted in transmission. It only needs to read the first index structured data corresponding to the streaming video data before the interruption to continue transpackaging. The data volume of the first index structured data is relatively small. The streaming video data of 1GB only corresponds to about 20MB of index structured data. Therefore, compared with reading the streaming video data, the reading speed of reading the first index structured data is faster. Therefore, the video data transpackaging method provided by the embodiment of the present application can ensure the efficiency of transpackaging the streaming video data into MP4 data.

[0092] In some embodiments, since the data volumes of streaming video data and MP4 data are both relatively large, if both are stored in the storage space of the transpackaging device, it will inevitably cause a waste of storage resources.

[0093] Therefore, in order to save the storage resources of the transpackaging device, as a feasible implementation method, the transpackaging method of video data provided in the embodiment of the present application does not store the MP4 data during the process of transpackaging the streaming video data into MP4 data, but only stores the streaming video data and the index structured data generated during the transpackaging process. After generating the MP4 index data according to the index structured data, the index structured data stored in the transpackaging device can be deleted, and only the MP4 index data is stored.

[0094] It is understandable that the display content of streaming video data is consistent with that of MP4 data, and the data volume is similar. If the streaming video data and MP4 data are stored in the transpackaging device at the same time, it will inevitably cause a waste of storage resources of the transpackaging device. Therefore, in the process of transpackaging the streaming video data into MP4 data, the MP4 data may not be stored. If the display device needs to display MP4 data later, the streaming video data will be transpacked into MP4 data for display in real time.

[0095] After the MP4 index data is generated according to the index structured data, the index structured data is useless data. In order to avoid it occupying storage space, it can be deleted to avoid it occupying storage space.

[0096] As a possible implementation, see Figure 8 The video data transpackaging method provided in the embodiment of the present application further includes the following steps:

[0097] S801: In response to receiving a play instruction sent by a display device, sending MP4 index data corresponding to the streaming video data to the display device.

[0098] When the display device needs to play video data, it needs to send a play instruction to the transpackaging device to request the display of the MP4 video data. After receiving the play instruction sent by the display device, the transpackaging device determines the streaming video data corresponding to the play instruction, and then sends the MP4 index data corresponding to the streaming video data to the display device, so that the display device can parse and play the MP4 video data corresponding to the streaming video data according to the MP4 index data.

[0099] S802: Transpackaging the streaming video data into MP4 data in MP4 format, and sending the data to a display device, so that the display device displays the MP4 data based on the MP4 index data.

[0100] In order to save storage space, the transpackaging device does not store the MP4 video data corresponding to the streaming video data. Therefore, when the display device requests display, the streaming video data needs to be transpacked into MP4 data in MP4 format in real time, and appended to the MP4 index data and sent to the display device so that the display device can play it in real time.

[0101] It can be seen that the video data transpackaging method provided in this embodiment, after all the streaming video data are transpackaged into MP4 data, only the streaming video data and its corresponding MP4 index data are stored in the transpackaging device, which can save the storage space of the transpackaging device. When the display device requests to play the video data, the MP4 index data is first sent to the display device, and the streaming video data is transpackaged into MP4 data in real time and sent to the display device, so that the display device can display the MP4 video in real time. In other words, the video data transpackaging method provided in this embodiment can save the storage space of the transpackaging device without affecting the normal playback of the video data.

[0102] In some embodiments, when a user views video data, the user may not view the video data continuously in sequence, but may choose to play the video at a fixed point, that is, to selectively start playing the video from a certain point in time. In this case, the display device needs to send the time point selected by the user to the transpackaging device, so that the transpackaging device transpacks the streaming video data after the time point into MP4 data and sends it to the display device for playback.

[0103] Specifically, as a feasible implementation method, the video data transpackaging method provided in the embodiment of the present application further includes: during the transpackaging process, determining the offset relationship between the first GOP data and the second GOP data.

[0104] It can be understood that the offset relationship between the first GOP data and the second GOP data may include: the offset position and offset amount of the first GOP data relative to the streaming video data, and the offset position and offset amount of the second GOP data relative to the MP4 video data, that is, the starting position and GOP length of the first GOP data and the starting position and GOP length of the second GOP data.

[0105] For example, taking the encapsulation of PS stream into MP4 as an example, the offset relationship can be stored in the following structure:

[0106] The GOPn offset relationship structure is:

[0107] {

[0108] ps_offset = p2; / / PS stream GOP start position

[0109] ps_gop_size = p3-p2; / / GOP length of PS stream

[0110] mp4_offset=m1; / / GOP start position of MP4 data

[0111] mp4_gop_size=m2-m1; / / GOP length of MP4 data

[0112] };

[0113] The GOP(n+1) offset relationship structure is:

[0114] {

[0115] ps_offset = p4;

[0116] ps_gop_size = p5-p4;

[0117] mp4_offset=m3;

[0118] mp4_gop_size=m4-m3;

[0119] };

[0120] In this way, when the display device adopts the positioning playback method for MP4 online on-demand, that is, when it chooses to start playback at a certain time point or position in the MP4 video data, the display device will generate a positioning playback instruction based on the offset position of the time point in the MP4 video data, and send the positioning playback instruction to the transpackaging device, so that the transpackaging device sends the MP4 data after the time point to the display device for display.

[0121] For details, please refer to Fig. 9 , S702 can be specifically implemented as follows:

[0122] S901: In response to receiving a positioning playback instruction, determine a starting offset position indicated by the positioning playback instruction.

[0123] The positioning play instruction is used to instruct to start playing the video from the starting offset position relative to the MP4 video data. That is to say, when the display device positions and plays the MP4 video data, it needs to determine the position of the positioning play, the offset relative to the starting position of the entire MP4 video data, and generate the positioning play instruction after obtaining the starting offset position and send it to the transpackaging device.

[0124] After receiving the positioning playback instruction, the transpackaging device determines the starting offset position carried in the positioning playback instruction.

[0125] As a feasible implementation method, when the display device uses a browser to play MP4 online, the browser uses the range method in the HyperText Transfer Protocol (http) request header for positioning playback. The request contains the starting offset position of the MP4 file. For example, if the http request header contains Range: bytes=98, it means positioning to the relative offset position 98 of the MP4 file to start playback.

[0126] S902: Determine the target first GOP data corresponding to the starting offset position in the streaming video data according to the offset relationship.

[0127] The offset relationship between the first GOP data and the second GOP data pre-stores the starting position and GOP length of each first GOP data and the second GOP data. After obtaining the starting offset position, the transpackaging device can determine the target second GOP data corresponding to the starting offset position in the MP4 video data, and then obtain the corresponding target first GOP data in the streaming video data.

[0128] S903: Encapsulate the target first GOP data into target second GOP data, and send the data after the start offset position in the target second GOP data to the display device.

[0129] After determining the target first GOP data, the target first GOP data can be encapsulated as the target second GOP data. Since the data to be played by the display device is the data after the starting offset position, the data after the starting offset position in the target second GOP data needs to be sent to the display device for playback.

[0130] S904: The first GOP data following the target first GOP data in the streaming video data is sequentially transpacked into second GOP data in chronological order, and sent to the display device.

[0131] The first GOP data following the target first GOP data are sequentially transpacked into second GOP data and sent to the display device, so that the display device can play the second GOP data in time sequence.

[0132] For example, see Fig.10 , assuming that the starting offset position of the MP4 file requested by the user is X, according to the offset relationship between the first GOP data and the second GOP data, find the Nth GOP (GOPn) that meets the conditions, and obtain the starting position (PSGopMP4Map.ps_offset) and GOP length (PSGopMP4Map.ps_offset) of the GOP. Then the transcapsulation device reads the Nth GOP data of the streaming video data from the storage space, the starting position read is PSGopMP4Map.ps_offset, the length is PSGopMP4Map.ps_size, and performs the transcapsulation operation to obtain the MP4 encapsulation data of the Nth GOP. Then calculate the M position of the Nth GOP, and return the MP4 encapsulation data of the GOP with a length of H starting from the M position.

[0133] Among them, M = X - PSGopMP4Map.mp4_offset (Nth GOP), that is, the starting offset position minus the starting position of the Nth GOP, to obtain the offset position M of the starting offset position relative to the Nth GOP; H = PSGopMP4Map.mp4_size–M, that is, the GOP length of the Nth GOP minus M, to obtain the data length H required in the Nth GOP.

[0134] It is understandable that in some embodiments, when positioning and playing MP4 video data, the display device may instruct to play a video within a time interval, that is, the positioning and playing instruction may also include the end offset position. In this case, the transpackaging device needs to encapsulate all the first GOP data corresponding to the start offset position-the end offset position into the second GOP data, and send the MP4 data between the start offset position-the end offset position to the display device for playback. The specific transpackaging process and sending process can refer to the description in the above embodiment, and the embodiment of the present application will not be repeated here.

[0135] It can be seen that the video data transpackaging method provided in this embodiment stores the offset relationship between the first GOP data and the second GOP data during the transpackaging process. In this way, when the display device performs positioning playback, after receiving the starting offset position of the positioning playback, the corresponding position of the MP4 video data to be played in the streaming video data can be determined according to the offset relationship between the first GOP data and the second GOP data, and then the streaming video data after the position can be transpacked into MP4 data and sent to the display device, so that the display device can start playing the MP4 video data from after the starting offset position of the MP4 video data.

[0136] In some embodiments, the transpackaging device includes: a storage service, a transpackaging module and a storage space. Taking the transpackaging of streaming video data in PS format into an MP4 file as an example, in the case of interruption of streaming video data transmission, please refer to Fig.11 The video data transpackaging method provided in the embodiment of the present application, the data transmission process in the video data transpackaging system is as follows:

[0137] S1101. The transpackaging device obtains the first PS stream generated by the acquisition device through the storage service, and transpacks the PS stream into MP4 data in real time through the transpackaging module.

[0138] S1102: During the transpackaging process, the transpackaging module generates first index structured data and an offset relationship between the PS stream and the MP4 file.

[0139] S1103: Store the first PS stream in the storage space of the transcapsulation device.

[0140] S1104: When the PS stream transmission is interrupted, the transpackaging module stores the generated first index structured data and the offset relationship in a storage space.

[0141] S1105. The storage service continues to obtain the second PS stream generated by the acquisition device and sends it to the transcapsulation module.

[0142] S1106. The transpackaging module reads the first segment of index structured data stored in the storage space, and continues to transpack the second segment of PS stream based on the first segment of index structured data, to generate the second segment of index structured data and the offset relationship between the PS stream and the MP4 file.

[0143] S1107. Store the second PS stream in the storage space of the transcapsulation device.

[0144] S1108. After the PS stream transmission is completed, generate MP4 index data according to the first segment index structure data and the second segment index structure data, and store the MP4 index data and the offset relationship between the PS stream and the MP4 file in the storage space.

[0145] S1109. Delete the index structured data stored in the storage space.

[0146] S1110. The display device sends a play instruction to the transpackaging device.

[0147] S1111. After receiving the play instruction, the storage service of the transpackaging device reads the MP4 index data stored in the storage space, then reads the PS stream data, transpacks it into MP4 data in real time, and appends it to the MP4 index data and sends it to the display device.

[0148] S1112: The display device plays the MP4 data based on the MP4 index data.

[0149] The present application also provides a video data transpackaging device, such as Fig.12 As shown, the video data transpackaging device 120 includes: an acquisition module 121 , a determination module 122 and a transpackaging module 123 .

[0150] The acquisition module 121 is used to acquire streaming video data.

[0151] The determination module 122 is used to perform frame analysis on the streaming video data to determine a plurality of first group of pictures GOP data in the streaming video data.

[0152] The transpackaging module 123 is used to transpack each first GOP data into second GOP data in MP4 format, and determine the index structured data corresponding to each second GOP data during the transpackaging process; the index structured data includes frame information of each frame in the second GOP data.

[0153] The determination module 122 is further configured to determine the MP4 index data corresponding to the streaming video data based on all the index structured data when it is determined that all the streaming video data are transpacked into the second GOP data.

[0154] As a feasible implementation method, the video data transpackaging device 120 also includes: a storage module, which is used to store the first index structured data before the transmission interruption is detected when the streaming video data transmission is interrupted; a reading module, which is used to read the first index structured data in response to the continued transmission of the streaming video data, and continue to transpack according to the first index structured data, and determine the second index structured data during the transpackaging process; the determination module 122 is specifically used to determine the MP4 index data corresponding to the streaming video data based on the first index structured data and the second index structured data.

[0155] As a feasible implementation method, the video data transpackaging device 120 further includes: a deletion module, which is used to delete the first index structured data after storing the MP4 index data corresponding to the streaming video data.

[0156] As a feasible implementation method, the video data transpackaging device 120 also includes: a storage module, which is used to store streaming video data and MP4 index data corresponding to the streaming video data; a sending module, which is used to send the MP4 index data corresponding to the streaming video data to the display device after receiving a play instruction sent by the display device; transpackaging the streaming video data into MP4 data in MP4 format, and sending it to the display device, so that the display device displays the MP4 data based on the MP4 index data.

[0157] As a feasible implementation method, the determination module 122 is also used to determine the offset relationship between the first GOP data and the second GOP data during the transpackaging process; the sending module is specifically used to determine the starting offset position indicated by the positioning playback instruction in response to receiving the positioning playback instruction; the positioning playback instruction is used to indicate to start playing the video from the starting offset position relative to the MP4 video data; the determination module 122 is also used to determine the target first GOP data corresponding to the starting offset position in the streaming video data according to the offset relationship; the transpackaging module 123 is also used to package the target first GOP data into the target second GOP data. The sending module is also used to send the data after the starting offset position in the target second GOP data to the display device; the first GOP data after the target first GOP data in the streaming video data is transpacked into the second GOP data in chronological order and sent to the display device.

[0158] As a feasible implementation method, the format of the streaming video data is the program stream PS format.

[0159] As a feasible implementation, the acquisition module 121 is specifically configured to acquire streaming video data according to a stream acquisition address carried in the video recording request after receiving the video recording request.

[0160] The present application also provides an electronic device. Fig.13 The electronic device 201 includes: one or more memories 111, one or more processors 112, a communication bus 113 and a communication interface 114. The processor 112 is connected to the memory 111 via the bus 113; the one or more memories 111 are used to store computer program codes, and the computer program codes include computer instructions; when the one or more processors 112 execute the computer instructions, the loading control device 201 executes the video data transpackaging method provided in the above embodiment.

[0161] Optionally, the memory 111 may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc., and the embodiments of the present application do not impose any restrictions on this.

[0162] The processor 112 may be a central processing unit (CPU), a general-purpose processor, a network processor (NP), a digital signal processor (DSP), a microprocessor, a microcontroller, a programmable logic device (PLD), or any combination thereof, and the embodiments of the present application do not impose any limitation on this.

[0163] The communication bus 113 may be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus. The bus 113 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.13 Just one thick line is used, but it does not mean that there is only one bus or one type of communication bus.

[0164] The communication interface 114 uses any transceiver or other device for communicating with other devices or communication networks, such as control systems, radio access networks (RAN), wireless local area networks (WLAN), etc.

[0165] The embodiments of the present application also provide a computer program product including one or more instructions, where the one or more instructions are stored in a memory of a computer device and executed by a processor to complete the various processes of the above embodiments.

[0166] An embodiment of the present application further provides a computer-readable storage medium, which includes computer-executable instructions. When the computer-executable instructions are executed on a computer, the computer executes the video data transpackaging method provided in the above embodiment.

[0167] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0168] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0169] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0170] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0171] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a device or processor to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, ROM, RAM, disk or CD-ROM and other media that can store program code.

[0172] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto, and any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A video data transpackaging method, characterized in that: include: Get streaming video data; Performing frame analysis on the streaming video data to determine a plurality of first group of picture GOP data in the streaming video data; Transpackaging each first GOP data into second GOP data in MP4 format, and determining index structured data corresponding to each second GOP data during the transpackaging process; The index structured data includes frame information of each frame in the second GOP data; When it is determined that all the streaming video data are transpacked into the second GOP data, determining the MP4 index data corresponding to the streaming video data based on all the index structured data; Storing the streaming video data and MP4 index data corresponding to the streaming video data; In response to receiving a play instruction sent by a display device, sending the MP4 index data corresponding to the streaming video data to the display device; The streaming video data is transcapsulated into MP4 data in MP4 format and sent to the presentation device, so that the presentation device presents the MP4 data based on the MP4 index data.

2. The method according to claim 1, characterized in that The method further comprises: When the streaming video data transmission interruption is detected, storing the first index structured data before the transmission interruption; In response to the streaming video data continuing to be transmitted, reading the first index structured data, and continuing to transpackaging according to the first index structured data, and determining second index structured data during the transpackaging process; The determining the MP4 index data corresponding to the streaming video data based on the index structured data includes: MP4 index data corresponding to the streaming video data is determined based on the first index structured data and the second index structured data.

3. The method according to claim 2, characterized in that After storing the MP4 index data corresponding to the streaming video data, the method further includes: The first index structured data is deleted.

4. The method according to claim 1, characterized in that: The method further comprises: During the transpackaging process, determining an offset relationship between the first GOP data and the second GOP data; The step of transpackaging the streaming video data into MP4 data in MP4 format and sending the data to the display device includes: In response to receiving a positioning playback instruction, determining a starting offset position indicated by the positioning playback instruction; the positioning playback instruction is used to instruct to start playing the video from the starting offset position relative to the MP4 video data; Determine the target first GOP data corresponding to the starting offset position in the streaming video data according to the offset relationship; encapsulating the target first GOP data into target second GOP data, and sending the data after the starting offset position in the target second GOP data to the display device; The first GOP data after the target first GOP data in the streaming video data is sequentially transpacked into second GOP data in chronological order and sent to the display device.

5. The method according to any one of claims 1 to 4, characterized in that: The format of the streaming video data is program stream PS format.

6. The method according to any one of claims 1 to 4, characterized in that: The step of obtaining streaming video data comprises: In response to receiving a video recording request, the streaming video data is acquired according to a stream acquisition address carried in the video recording request.

7. A video data transpackaging device, characterized in that: include: An acquisition module, used for acquiring streaming video data; A determination module, configured to perform frame analysis on the streaming video data to determine a plurality of first group of pictures (GOP) data in the streaming video data; A transpackaging module, used for transpackaging each of the first GOP data into second GOP data in MP4 format, and determining index structured data corresponding to each of the second GOP data during the transpackaging process; the index structured data includes frame information of each frame in the second GOP data; The determining module is further configured to determine the MP4 index data corresponding to the streaming video data based on all the index structured data when it is determined that all the streaming video data are transcapsuled into the second GOP data; A storage module, used for storing the streaming video data and MP4 index data corresponding to the streaming video data; The sending module is used to send the MP4 index data corresponding to the streaming video data to the display device in response to receiving a play instruction sent by the display device; transpackaging the streaming video data into MP4 data in MP4 format and sending it to the display device, so that the display device displays the MP4 data based on the MP4 index data.

8. An electronic device, characterized in that: include: one or more processors; one or more memories; Among them, one or more of the memories are used to store computer program codes, and the computer program codes include computer instructions. When one or more of the processors execute the computer instructions, the electronic device executes the video data transpackaging method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed on a computer, the computer is enabled to execute the video data transpackaging method according to any one of claims 1 to 6.