Video bitstream packaging method, apparatus, electronic device, and program product

Abstract

The application discloses a video bit stream packaging method and device, electronic equipment and program product, and belongs to the technical field of audio and video. The video bit stream packaging method comprises the following steps: cutting the coded video bit stream to obtain a plurality of meta-codes; and packaging the meta-codes into real-time transport protocol (RTP) data packets according to the load type of each meta-code. Thus, the RTP packaging can be performed according to the load type of the meta-codes, so that the RTP data packets can be reasonably obtained, and the packet delay can be reduced.

Description

Technical Field

[0001] This application belongs to the field of audio and video technology, specifically relating to a video bitstream encapsulation method, apparatus, electronic device, and program product. Background Technology

[0002] In related technologies, when encapsulating encoded video bitstreams using the Real-time Transport Protocol (RTP), each meta-stream data or fragment is typically encapsulated directly into an RTP data packet. However, this method, where each meta-stream data or fragment is encapsulated into a single RTP data packet, can lead to significant packet delays. Summary of the Invention

[0003] The purpose of this application is to provide a video bitstream encapsulation method, apparatus, electronic device, and program product to solve the problem of large packet delay in related technologies.

[0004] To solve the above-mentioned technical problems, this application is implemented as follows:

[0005] Firstly, a video bitstream encapsulation method is provided, including:

[0006] The encoded video bitstream is segmented to obtain multiple meta-streams;

[0007] The metastreams are encapsulated into RTP packets according to the payload type of each metastream.

[0008] Secondly, a video bitstream encapsulation device is provided, comprising:

[0009] The segmentation module is used to segment the encoded video bitstream to obtain multiple meta-streams;

[0010] An encapsulation module is used to encapsulate the meta-stream into RTP data packets according to the payload type of each meta-stream.

[0011] Thirdly, an electronic device is provided, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0012] Fourthly, a readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0013] Fifthly, a computer program product is provided, including computer instructions that, when executed by a processor, implement the steps of the method described in the first aspect.

[0014] In this embodiment, after segmenting the encoded video bitstream to obtain multiple meta-streams, the meta-streams can be encapsulated into RTP data packets according to the payload type of each meta-stream. Therefore, compared to directly encapsulating each meta-stream data or meta-stream data fragment into a single RTP data packet, a more efficient method for obtaining RTP data packets can be used, reducing packet latency. Attached Figure Description

[0015] Figure 1 This is a flowchart of a video bitstream encapsulation method provided in an embodiment of this application;

[0016] Figure 2 This is a schematic diagram of the structure of a video bitstream encapsulation device provided in an embodiment of this application;

[0017] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0020] The video bitstream encapsulation method, apparatus, electronic device, and program product provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0021] Please see Figure 1 , Figure 1This is a flowchart illustrating a video bitstream encapsulation method provided in an embodiment of this application. This method is applied to electronic devices, such as encoding servers. Figure 1 As shown, the method includes the following steps:

[0022] Step 11: Segment the encoded video bitstream to obtain multiple meta-streams;

[0023] Step 12: Encapsulate the metastream into RTP data packets according to the payload type of each metastream.

[0024] In this embodiment, the video bitstream can be obtained by encoding a video sequence. The video bitstream can be an AVS (Audio Video Coding Standard) based video bitstream. For example, the scheme in this application implements an RTP packet method based on AVS3, and the corresponding meta-bitstream is an AVS3 video meta-bitstream.

[0025] When segmenting the encoded video bitstream, the corresponding segmentation rule can be segmentation according to the meta-bitstream. The meta-bitstream can also be called meta-bitstream data. The meta-bitstream can be understood as a video bitstream segment, which can be composed of encoded data (including the first start code) between two adjacent start codes or several adjacent start codes in the video bitstream.

[0026] Optionally, the start code of the meta-stream is a specific set of bit strings, meaning that these bit strings should not appear anywhere in the encoded video bitstream except at the start code of the meta-stream. The start code may include a start code prefix and a start code value. The start code prefix indicates the beginning of the start code, for example, a bit string '0000 0000 0000 0000 00000001'; the start code value is, for example, an 8-bit integer, used to represent the payload data type of the corresponding meta-stream, i.e., the payload type (or payload data type) of the corresponding meta-stream can be identified / determined based on the start code value. All start codes should be byte-aligned.

[0027] The solution in this application, after segmenting the encoded video bitstream to obtain multiple meta-streams, can encapsulate the meta-streams into RTP data packets according to the payload type of each meta-stream. Therefore, compared to directly encapsulating each meta-stream data or meta-stream data fragment into a single RTP data packet, a more efficient method for obtaining RTP data packets can be used, reducing packet latency.

[0028] In this embodiment, the payload type of the meta-bitstream may include, but is not limited to, sequence header, user data, video extended data, intra-frame (e.g., I-frame) / inter-frame (e.g., RL-frame, P-frame, or B-frame) predicted images, etc. For the intra-frame / inter-frame predicted image type meta-bitstream, it may consist of consecutive bytes from the start code of the intra-frame / inter-frame predicted image (e.g., start code value B3 or B6) to the start code of the next intra-frame / inter-frame predicted image (e.g., start code value B3 or B6), sequence start code (e.g., start code value B0), sequence end code (e.g., start code value B1), or video edit code (e.g., start code value B7).

[0029] Optionally, the payload type of the meta-stream in this embodiment may include at least one of the following:

[0030] (1) Sequence header meta-code stream, i.e., the meta-code stream of sequence header type, which contains sequence header data; for example, the start code prefix is ​​'0000 0000 0000 0000 0000 0001', and the start code value is B0;

[0031] (2) Video extended data metadata stream (or: video extended data metadata stream after sequence header), that is, the metadata stream of video extended data type, the video extended data metadata stream contains video extended data after sequence header; for example, the start code prefix is ​​'0000 0000 0000 0000 0000 0001', and the start code value is B5;

[0032] (3) User data metadata stream (or: user data metadata stream after sequence header), that is, the metadata stream of user data type, which includes user data after sequence header; for example, the start code prefix is ​​'0000 0000 00000000 0000 0001', and the start code value is B2;

[0033] (4) I-frame meta stream, i.e., I-frame type meta stream, the I-frame meta stream includes the image header of the I-frame and the video extended data, user data and slice data after the image header of the I-frame. The slice data can be all slice data; for example, the start code prefix is ​​'0000 0000 0000 0000 0000 0001', and the start code value is B3;

[0034] (5) RL frame meta stream, i.e., the meta stream of RL frame type, the RL frame meta stream includes the image header of RL frame and the video extended data, user data and slice data after the image header of RL frame. The slice data can be all slice data; for example, the start code prefix is ​​'0000 0000 0000 0000 0000 0001', and the start code value is B6;

[0035] (6) P-frame meta stream, i.e., P-frame type meta stream, the P-frame meta stream includes the P-frame image header and the video extended data, user data and slice data after the P-frame image header. The slice data can be all slice data; for example, the start code prefix is ​​'0000 0000 0000 0000 0000 0001', and the start code value is B6;

[0036] (7) B-frame meta stream, i.e., B-frame type meta stream, the B-frame meta stream includes the image header of the B-frame and the video extended data, user data and slice data after the image header of the B-frame. The slice data can be all slice data; for example, the start code prefix is ​​'0000 0000 0000 0000 0000 0001', and the start code value is B6;

[0037] (8) Sequence end code element stream, i.e., the sequence end code type element stream, the sequence end code element stream contains the end code of the video sequence; for example, the start code prefix is ​​'0000 0000 0000 0000 00000001', and the start code value is B1;

[0038] (9) Video editing code meta stream, i.e., video editing code type meta stream, the video editing code meta stream contains the editing code of the video sequence, based on which it can be determined that the video sequence before it can be independently edited; for example, the starting code prefix is ​​'0000 0000 0000 0000 0000 0001', and the starting code value is B7.

[0039] Understandably, for the aforementioned I-frame metastream, RL-frame metastream, P-frame metastream, and B-frame metastream, since they contain the image header and subsequent video extension data, user data, and all slice data, RTP packetization based on these metastreams can support the packetization of dynamic HDR (High Dynamic Range Imaging) data. By introducing video editing codes into the metastream types—that is, introducing video editing code metastreams—video editing codes can be added as markers for video sequences, indicating / signifying that the video sequence preceding the video editing code can be independently edited. This supports live streaming production and allows for secondary creation and editing of live video during the live stream.

[0040] In one optional implementation, taking AVS3 video meta stream as an example, the definition of the corresponding payload type can be shown in Table 1 below:

[0041] Table 1

[0042]

[0043]

[0044] In this embodiment of the application, when allocating RTP payload data for the segmented meta-stream, the RTP payload data can be encapsulated in the form of meta-streams. When the data volume of the meta-stream is small, such as less than the MTU (Maximum Transmission Unit) size of the IP layer, an RTP packet can contain one or more meta-streams; when the data volume of the meta-stream exceeds the limit of the MTU size of the IP layer, the meta-stream can be fragmented and RTP packets can be encapsulated based on the meta-stream fragments; the fragmentation rule is, for example, to divide the meta-stream into continuous segments with bytes as the smallest unit.

[0045] Optionally, the encapsulation rules for the RTP packets may include, but are not limited to, at least one of the following:

[0046] 1) Different types of meta streams are not allowed to be encapsulated / aggregated in the same RTP packet.

[0047] 2) Subject to the aggregate packet size limit, consecutive sequence header streams, video extended data element streams, and / or user data element streams may be encapsulated in the same RTP packet.

[0048] 1) and 2) above can be considered simultaneously. For example, different types of meta streams are not allowed to be encapsulated / aggregated in the same RTP packet, except in the following case: If the size of the aggregated packet meets the IP layer's MTU limit, consecutive sequence header meta streams, video extended data meta streams, and / or user data meta streams are allowed to be encapsulated in the same RTP packet. This allows consecutive, relatively small-volume sequence header meta streams, video extended data meta streams, and / or user data meta streams to be encapsulated in a single RTP packet, thus avoiding multiple RTP packets, reducing packet latency, and conserving network transmission resources. Furthermore, since these data are calculated together in the encoder, encapsulating them into a single RTP packet simplifies the processing flow.

[0049] 3) Multiple frames of images are not allowed to be encapsulated / aggregated in the same RTP data packet; this can prevent multiple frames of images from becoming dependent and avoid being unable to process after packet loss; for example, if an I-frame image is lost, it needs to be retransmitted, while if a P-frame image is lost, it is directly dropped. If the I-frame image and the P-frame image are encapsulated in the same RTP data packet, it will be impossible to process after a frame of image is lost.

[0050] 4) For a meta-stream whose encapsulation size is greater than the first value, the meta-stream must be segmented and the segmented meta-stream fragments must be encapsulated using a fragmented payload format; the first value can be set based on actual needs, such as being equal to the MTU size;

[0051] 5) The sequence end code or video editing code stream is encapsulated as a single RTP data packet. The sequence end code or video editing code indicates the end of the video sequence. Since the encapsulation logic for the sequence end code stream is either to be encapsulated with the last frame of image data into an aggregate packet or to be encapsulated independently as a single RTP data packet, if it's encapsulated with the last frame, the server must buffer one frame of image data each time it processes the video. When the next frame arrives, the server checks if it's the last frame. If it is, it's encapsulated with the sequence end code into an RTP data packet before being sent. This buffering causes latency. To avoid this, the sequence end code stream is encapsulated as a separate RTP data packet. The same logic applies to the video editing code stream; to avoid latency, the video editing code stream is also encapsulated as a separate RTP data packet.

[0052] 6) Aggregate packets and fragment packets are not allowed to be nested, that is, the aggregate payload is not allowed to contain meta-stream fragments.

[0053] 7) Metadata streams of non-independent bitstreams and knowledge bitstreams are not allowed to be encapsulated / aggregated in the same RTP packet. Knowledge bitstreams are similar to I-frames, and non-independent bitstreams are similar to P-frames. Decoding requires reference to the knowledge bitstream, so they cannot be aggregated in a single RTP packet. The reason is to prevent dependencies between multiple frames and to avoid being unable to process packets after loss.

[0054] 8) RTP packet payloads are not allowed to be nested; that is, an RTP payload cannot contain a complete RTP packet. This is because RTP is a transport layer technology, and network devices do not support its recognition. If a complete RTP packet is nested within the RTP payload, that RTP packet will not be recognized.

[0055] It should be noted that, taking AVS3 video streams as an example, the RTP payload format of an AVS3 video stream consists of two parts: an RTP payload header and an RTP payload that carries the AVS3 video bitstream data. The RTP payload can be categorized into single payloads, aggregated payloads, and fragmented payloads, depending on its type. All types of payloads contain a generic AVS payload header.

[0056] In this embodiment, RTP packetization can be performed based on the payload type of the meta-stream. The above-mentioned encapsulation of the meta-stream into RTP data packets based on the payload type of each meta-stream may include:

[0057] When the meta-stream is a first-type meta-stream, the meta-stream is encapsulated into a single RTP data packet; alternatively, the meta-stream and the K subsequent meta-streams are encapsulated into a single RTP data packet. Here, K is a positive integer. The sum of the sizes of the meta-stream and the K subsequent meta-streams is less than or equal to a second value, and satisfies at least one of the following: the (K+1)th meta-stream following the meta-stream is not a first-type meta-stream; the sum of the sizes of the meta-stream and the K+1 subsequent meta-streams is greater than the second value; each of the K meta-streams is a first-type meta-stream; the first-type meta-stream is any one of the following: sequence header meta-stream, video extended data meta-stream, and user data meta-stream. This allows consecutive and relatively small sequence header meta-streams, video extended data meta-streams, and / or user data meta-streams to be encapsulated in a single RTP data packet, thereby avoiding multiple RTP data packets and reducing packet latency.

[0058] Optionally, the second value can be set based on actual needs, such as being equal to the MTU size. When the first metastream following the metastream is not a first-type metastream, or when the sum of the sizes of the metastream and the first metastream following it is greater than the second value, the metastream can be directly encapsulated into an RTP data packet.

[0059] Optionally, the above-mentioned encapsulation of the meta-stream into RTP packets according to the payload type of each meta-stream may further include:

[0060] When the meta-stream is a second type of meta-stream, the meta-stream is encapsulated into an RTP data packet; wherein, the second type of meta-stream is any one of the following: I-frame meta-stream, RL-frame meta-stream, P-frame meta-stream, B-frame meta-stream, sequence end meta-stream, and video editing meta-stream.

[0061] For example, when the meta-stream is an I-frame meta-stream, RL-frame meta-stream, P-frame meta-stream, or B-frame meta-stream, it can be directly encapsulated into a single RTP data packet. This prevents dependencies between multiple frames and avoids the inability to process packets after loss.

[0062] For example, when the meta-stream is a sequence end-of-sequence meta-stream or a video editing meta-stream, it can be directly encapsulated into an RTP data packet. This avoids the latency caused by buffered image frames when aggregating with the last frame of image data, thereby reducing packet latency.

[0063] Optionally, before encapsulating the meta-stream into RTP packets according to the payload type of each meta-stream, the video bitstream encapsulation method in this embodiment may further include:

[0064] Determine whether the size of the meta-stream is less than a third value; the third value can be set based on actual needs, such as being equal to the MTU size, or equal to the difference between the MTU size and N, where N is the number of configurable reserved header bytes.

[0065] The above-mentioned encapsulation of the meta-stream into RTP packets based on the payload type of each meta-stream may include:

[0066] When the size of the first meta-stream among the plurality of meta-streams is less than the third value, the first meta-stream is encapsulated into an RTP data packet according to its payload type. The specific method for RTP packetization based on payload type can be found above and will not be repeated here. This allows RTP packetization to be performed while ensuring that the RTP data packet size limit is met.

[0067] Optionally, after determining whether the size of the meta-stream is less than the third value, the video bitstream encapsulation method in this embodiment may further include any of the following:

[0068] When the size of the second meta stream in the plurality of meta streams is greater than the third value, the second meta stream is segmented, and the segmented meta stream is fragmented and encapsulated into RTP data packets using a fragmented payload format.

[0069] When the size of the third-order stream among the plurality of meta-streams is equal to the third value, the third-order stream is encapsulated into an RTP data packet.

[0070] For example, when segmenting the second meta-stream and encapsulating the segmented meta-stream fragments into RTP data packets using a fragmented payload format, the second meta-stream can be divided into n meta-stream fragments, and each of the n meta-stream fragments can be encapsulated into an RTP data packet; wherein, K represents the size of the second meta-stream, and s represents the third value. This indicates rounding up; the size of the first n-1 meta-stream segments in the n meta-stream segments is equal to the third value, and the size of the nth meta-stream segment is equal to K%s.

[0071] In one optional implementation, the specific RTP packet encapsulation process may include:

[0072] S1: Segment the encoded video bitstream to obtain multiple meta-streams (i.e. multiple segments), and perform the following steps for each meta-stream.

[0073] S2: Calculate the size M(i) of the i-th meta-bit stream;

[0074] S3: When M(i) is less than the difference between the MTU size of the IP layer and N, where N is the number of configurable reserved header bytes, the following steps are performed:

[0075] S31: Determine the payload type T(i) of the i-th meta-stream;

[0076] If T(i) is a sequence header type, a video extended data type after the sequence header, or a user data type after the sequence header, that is, the i-th meta stream is a sequence header meta stream, a video extended data meta stream after the sequence header, or a user data meta stream after the sequence header, then execute S32 and S33.

[0077] If T(i) is a sequence end code type or a video editing code type, that is, the i-th meta code stream is a sequence end code meta code stream or a video editing code meta code stream, then execute S34;

[0078] If T(i) is of type I-frame, RL-frame, P-frame, or B-frame, that is, the i-th meta-stream is an I-frame meta-stream, RL-frame meta-stream, P-frame meta-stream, or B-frame meta-stream, then execute S34.

[0079] S32: Determine the payload type T(i+1) of the (i+1)th meta stream. If T(i+1) is a sequence header type, a video extended data type after the sequence header, or a user data type after the sequence header, then calculate the size M(i+1) of the (i+1)th meta stream.

[0080] S33: If M(i) + M(i+1) is less than the MTU size of the IP layer, then repeat S32 to process the (i+2)th meta-stream until M(i) + M(i+1) + ... + M(i+k) is greater than the MTU size of the IP layer, or T(i+k) is not a sequence header type, a video extended data type after the sequence header, or a user data type after the sequence header; at this point, the i-th meta-stream, the (i+1)-th meta-stream, ..., the (i+k-1)-th meta-stream are encapsulated into an aggregate package, that is: the consecutive sequence header, the video extended data after the sequence header, and the user data meta-stream after the sequence header are encapsulated into an aggregate package, and the size of the aggregate package is less than the MTU size of the IP layer;

[0081] S34: Encapsulate the i-th meta-stream into a single RTP data packet.

[0082] S4: When M(i) is greater than the difference between the MTU size of the IP layer and N, the i-th meta-stream is divided into M(i) / (MTU-N) meta-stream fragments, where the size of the first M(i) / (MTU-N)-1 meta-stream fragments is MTU-N bytes, and the size of the last meta-stream fragment is M(i)%(MTU-N); and the divided meta-stream fragments are encapsulated into RTP data packets.

[0083] S5: When M(i) equals the difference between the MTU size of the IP layer and N, the i-th meta stream is encapsulated into a single RTP packet.

[0084] It should be noted that the video bitstream encapsulation method provided in this application embodiment can be executed by a video bitstream encapsulation device or a control module within that device for executing the video bitstream encapsulation method. This application embodiment uses the execution of the video bitstream encapsulation method by a video bitstream encapsulation device as an example to illustrate the video bitstream encapsulation device provided in this application embodiment.

[0085] Please see Figure 2 , Figure 2 This is a schematic diagram of a video bitstream encapsulation device provided in an embodiment of this application. This device is applied to electronic devices, such as... Figure 2 As shown, the video bitstream encapsulation device 20 includes:

[0086] The cutting module 21 is used to cut the encoded video bitstream to obtain multiple meta-bitstreams;

[0087] The encapsulation module 22 is used to encapsulate the meta-stream into RTP data packets according to the payload type of each meta-stream.

[0088] Optionally, the payload type of the meta-stream includes at least one of the following:

[0089] A sequence header stream, wherein the sequence header stream contains sequence header data;

[0090] Video extended data metadata stream, wherein the video extended data metadata stream includes video extended data following the sequence header;

[0091] User data metadata stream, which includes user data following the sequence header;

[0092] The I-frame meta stream includes the image header of the I-frame and the video extended data, user data and slice data following the image header of the I-frame.

[0093] The RL frame meta stream includes the image header of the RL frame and the video extension data, user data, and slice data following the image header of the RL frame.

[0094] The P-frame meta stream includes the P-frame image header and the video extended data, user data and slice data following the P-frame image header;

[0095] B-frame meta stream, which includes the B-frame image header and the video extended data, user data and slice data following the B-frame image header;

[0096] The sequence end code stream contains the end code of the video sequence;

[0097] The video editing codebase stream contains the editing codes of the video sequence.

[0098] Optionally, the encapsulation rules for the RTP packets include at least one of the following:

[0099] Different types of meta-streams are not allowed to be encapsulated in the same RTP packet;

[0100] Provided that the aggregate packet size limit is met, consecutive sequence header streams, video extended data element streams, and / or user data element streams can be encapsulated in the same RTP packet;

[0101] Multiple frames of images are not allowed to be encapsulated in the same RTP packet;

[0102] For a meta-stream whose encapsulation size is greater than the first value, the meta-stream must be segmented and the segmented meta-stream fragments must be encapsulated using a fragmented payload format.

[0103] The sequence end symbol stream or video editing symbol stream is encapsulated into a single RTP data packet;

[0104] Aggregate packages and fragment packages are not allowed to be nested.

[0105] Optionally, the encapsulation module 22 is specifically used to: encapsulate the meta-stream into an RTP data packet when the meta-stream is a first type of meta-stream, or encapsulate the meta-stream and the K meta-streams following the meta-stream into an RTP data packet;

[0106] Wherein, K is a positive integer; the sum of the size of the meta-stream and the K meta-streams following the meta-stream is less than or equal to the second value, and satisfies at least one of the following: the (K+1)th meta-stream following the meta-stream is not a first-type meta-stream; the sum of the size of the meta-stream and the K+1 meta-streams following the meta-stream is greater than the second value; each of the K meta-streams is a first-type meta-stream; the first-type meta-stream is any one of the following: sequence header meta-stream, video extended data meta-stream, and user data meta-stream.

[0107] Optionally, the encapsulation module 22 is specifically used to: encapsulate the meta-stream into an RTP data packet when the meta-stream is a second type of meta-stream; wherein the second type of meta-stream is any one of the following: I-frame meta-stream, RL-frame meta-stream, P-frame meta-stream, B-frame meta-stream, sequence end meta-stream, and video editing meta-stream.

[0108] Optionally, the video bitstream encapsulation device 20 also includes

[0109] The judgment module is used to determine whether the size of the meta-stream is less than a third value before encapsulating the meta-stream into RTP data packets according to the payload type of each meta-stream.

[0110] The encapsulation module 22 is specifically used to: when the size of the first meta stream among the plurality of meta streams is less than the third value, encapsulate the first meta stream into an RTP data packet according to the payload type of the first meta stream.

[0111] Optionally, the encapsulation module 22 is also configured to perform any of the following:

[0112] When the size of the second meta stream in the plurality of meta streams is greater than the third value, the second meta stream is segmented, and the segmented meta stream is fragmented and encapsulated into RTP data packets using a fragmented payload format.

[0113] When the size of the third-order stream among the plurality of meta-streams is equal to the third value, the third-order stream is encapsulated into an RTP data packet.

[0114] The video bitstream encapsulation device 20 of this application embodiment can achieve the above-mentioned... Figure 1 The various processes of the method embodiments shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0115] Optional, such as Figure 3 As shown, this application embodiment also provides an electronic device 30, including a processor 31, a memory 32, and a program or instructions stored in the memory 32 and executable on the processor 31. When the program or instructions are executed by the processor 31, they implement the various processes of the above-described video bitstream encapsulation method embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0116] This application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, they can implement the various processes of the above-described video bitstream encapsulation method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0117] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they can implement the various processes of the above-described video bitstream encapsulation method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0118] Computer-readable media include both permanent and non-permanent, removable and non-removable media, which can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0119] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0120] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0121] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a service classification device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0122] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A video bitstream encapsulation method, characterized in that, include: The encoded video bitstream is segmented to obtain multiple meta-streams; Based on the payload type of each of the meta-streams, the meta-streams are encapsulated into Real-Time Transport Protocol (RTP) data packets; The step of encapsulating the meta-stream into RTP data packets according to the payload type of each meta-stream includes: When the meta-stream is a first-type meta-stream, the meta-stream is encapsulated into an RTP data packet, or the meta-stream and the K meta-streams following it are encapsulated into an RTP data packet; wherein, K is a positive integer; the sum of the size of the meta-stream and the K meta-streams following it is less than or equal to a second value, and satisfies at least one of the following: the (K+1)th meta-stream following the meta-stream is not a first-type meta-stream; the sum of the size of the meta-stream and the K+1 meta-streams following it is greater than the second value; each of the K meta-streams is a first-type meta-stream; the first-type meta-stream is any one of the following: sequence header meta-stream, video extended data meta-stream, and user data meta-stream; or, When the meta-stream is a second type of meta-stream, the meta-stream is encapsulated into an RTP data packet; wherein, the second type of meta-stream is any one of the following: I-frame meta-stream, RL-frame meta-stream, P-frame meta-stream, B-frame meta-stream, sequence end meta-stream, and video editing meta-stream.

2. The method according to claim 1, characterized in that, The payload type of the meta-stream includes at least one of the following: A sequence header stream, wherein the sequence header stream contains sequence header data; Video extended data metadata stream, wherein the video extended data metadata stream includes video extended data following the sequence header; User data metadata stream, which includes user data following the sequence header; The I-frame meta stream includes the image header of the I-frame and the video extended data, user data and slice data following the image header of the I-frame. The RL frame meta stream includes the image header of the RL frame and the video extension data, user data, and slice data following the image header of the RL frame. The P-frame meta stream includes the P-frame image header and the video extended data, user data and slice data following the P-frame image header; B-frame meta stream, which includes the B-frame image header and the video extended data, user data and slice data following the B-frame image header; The sequence end code stream contains the end code of the video sequence; The video editing codebase stream contains the editing codes of the video sequence.

3. The method according to claim 1, characterized in that, The encapsulation rules for the RTP data packets include at least one of the following: Different types of meta-streams are not allowed to be encapsulated in the same RTP packet; Provided that the aggregate packet size limit is met, consecutive sequence header bitstreams, video extended data bitstreams, and / or user data bitstreams can be encapsulated in the same RTP packet; Multiple frames of images are not allowed to be encapsulated in the same RTP packet; For a meta-stream whose encapsulation size is greater than the first value, the meta-stream must be segmented and the segmented meta-stream fragments must be encapsulated using a fragmented payload format. The sequence end symbol stream or video editing symbol stream is encapsulated into a single RTP data packet; Aggregate packages and fragment packages are not allowed to be nested.

4. The method according to claim 1, characterized in that, Before encapsulating the meta-stream into RTP packets according to the payload type of each meta-stream, the method further includes: Determine whether the size of the meta-stream is less than the third value; The step of encapsulating the meta-stream into RTP packets according to the payload type of each meta-stream includes: When the size of the first meta stream among the plurality of meta streams is less than the third value, the first meta stream is encapsulated into an RTP data packet according to the payload type of the first meta stream.

5. The method according to claim 4, characterized in that, After determining whether the size of the meta-code stream is less than the third value, the method further includes any one of the following: When the size of the second meta stream in the plurality of meta streams is greater than the third value, the second meta stream is segmented, and the segmented meta stream is fragmented and encapsulated into RTP data packets using a fragmented payload format. When the size of the third-order stream among the plurality of meta-streams is equal to the third value, the third-order stream is encapsulated into an RTP data packet.

6. A video bitstream encapsulation device, characterized in that, include: The segmentation module is used to segment the encoded video bitstream to obtain multiple meta-streams; An encapsulation module is used to encapsulate the meta-stream into RTP data packets according to the payload type of each meta-stream; Specifically, the encapsulation module 22 is used for one of the following: When the meta-stream is a first-type meta-stream, the meta-stream is encapsulated into an RTP data packet, or the meta-stream and the K meta-streams following it are encapsulated into an RTP data packet; wherein, K is a positive integer; the sum of the size of the meta-stream and the K meta-streams following it is less than or equal to a second value, and satisfies at least one of the following: the (K+1)th meta-stream following the meta-stream is not a first-type meta-stream; the sum of the size of the meta-stream and the K+1 meta-streams following it is greater than the second value; each of the K meta-streams is a first-type meta-stream; the first-type meta-stream is any one of the following: sequence header meta-stream, video extended data meta-stream, and user data meta-stream; When the meta-stream is a second type of meta-stream, the meta-stream is encapsulated into an RTP data packet; wherein, the second type of meta-stream is any one of the following: I-frame meta-stream, RL-frame meta-stream, P-frame meta-stream, B-frame meta-stream, sequence end meta-stream, and video editing meta-stream.

7. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method as described in any one of claims 1 to 5.

8. A computer program product, characterized in that, Includes computer instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1 to 5.

Images