Video bit stream packaging method and device, electronic equipment and program product

Abstract

The invention discloses a video bit stream packaging method and device, electronic equipment and a program product, and belongs to the technical field of audio and video. The video bit stream packaging method comprises the following steps: cutting a coded video bit stream to obtain a plurality of element code streams; and according to the load type of each element code stream, packaging the element code stream into a real-time transport protocol (RTP) data packet. Therefore, the RTP packaging can be performed according to the load type of the element code stream, so that the RTP data packet is reasonably obtained, and the packaging delay is reduced.

Description

Technical Field

[0001] The present application belongs to the field of audio and video technology, and specifically relates to a video bit stream encapsulation method, device, electronic equipment and program product. Background Art

[0002] In the related art, when the encoded video bit stream is encapsulated by the Real-time Transport Protocol (RTP), each meta-stream data or meta-stream data fragment is usually directly encapsulated by RTP to obtain an RTP data packet. In this case, since each meta-stream data or meta-stream data fragment is encapsulated as an RTP data packet, a large packet delay may be caused. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a video bitstream encapsulation method, device, electronic device and program product to solve the problem of large packet delay in the related art.

[0004] In order to solve the above technical problems, this application is implemented as follows:

[0005] In a first aspect, a video bitstream encapsulation method is provided, comprising:

[0006] Cutting the encoded video bit stream to obtain multiple meta-streams;

[0007] According to the payload type of each of the meta-streams, the meta-streams are encapsulated into RTP data packets.

[0008] In a second aspect, a video bitstream encapsulation device is provided, comprising:

[0009] A cutting module, used for cutting the encoded video bit stream to obtain multiple meta-streams;

[0010] The encapsulation module is used to encapsulate the meta-stream into an RTP data packet according to the load type of each meta-stream.

[0011] In a third aspect, an electronic device is provided, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0012] In a fourth aspect, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0013] In a fifth aspect, a computer program product is provided, comprising computer instructions, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0014] In the embodiment of the present application, after the encoded video bitstream is cut to obtain multiple metastreams, the metastreams can be encapsulated into RTP data packets according to the payload type of each metastream. Thus, compared with directly encapsulating each metastream data or metastream data fragment into one RTP data packet, the RTP data packet can be obtained more reasonably and the packet delay can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flow chart of a video bitstream encapsulation method provided by an embodiment of the present application;

[0016] Figure 2 It is a structural schematic diagram of a video bit stream encapsulation device provided in an embodiment of the present application;

[0017] Figure 3 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0019] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0020] The video bitstream encapsulation method, device, electronic device and program product provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0021] See also Figure 1 , Figure 11 is a flowchart of a video bitstream encapsulation method provided in an embodiment of the present application, and the method is applied to electronic devices, such as an encoding server, etc. Figure 1 As shown, the method comprises the following steps:

[0022] Step 11: Cut the encoded video bit stream to obtain multiple meta-streams;

[0023] Step 12: Encapsulate each meta-stream into an RTP data packet according to the payload type of each meta-stream.

[0024] In the embodiment of the present application, the video bitstream may be obtained by encoding a video sequence. The video bitstream may be a video bitstream based on AVS (Audio Video Coding Standard), for example, the solution implemented in the present application is an RTP packetization method based on AVS3, and the corresponding metastream is an AVS3 video metastream.

[0025] When the encoded video bitstream is cut, the corresponding cutting rule may be cutting according to the meta-stream. The meta-stream may also be referred to as meta-stream data. The meta-stream may be understood as a video bitstream segment, which may be composed of the encoded data (including the first start code) between two adjacent start codes or several adjacent start codes in the video bitstream.

[0026] Optionally, the start code of the meta-code stream is a set of specific bit strings, that is, in the encoded video bit stream, these bit strings should not appear at any position except the start code of the meta-code stream. The start code may include a start code prefix and a start code value. The start code prefix is ​​used to indicate the beginning of the start code, such as the bit string '0000 0000 0000 0000 00000001'; the start code value is, for example, an 8-bit integer, used to indicate the payload data type of the corresponding meta-code stream, that is, the payload type (or payload data type) of the corresponding meta-code stream can be identified / determined according to the start code value. All start codes should be byte aligned.

[0027] According to the solution of the embodiment of the present application, after the encoded video bitstream is cut to obtain multiple metastreams, the metastreams can be encapsulated into RTP data packets according to the payload type of each metastream. Thus, compared with directly encapsulating each metastream data or metastream data fragment into one RTP data packet, the RTP data packet can be obtained more reasonably and the packet delay can be reduced.

[0028] In the embodiment of the present application, the payload type of the meta-code stream may include but is not limited to a sequence header, user data, video extension data, intra-frame (such as I frame) / inter-frame (such as RL frame, P frame or B frame) predicted image, etc. For a meta-code stream of intra-frame / inter-frame predicted image type, it may be composed of consecutive bytes starting from an intra-frame / inter-frame predicted image start code (such as a start code value of B3 or B6) to the next intra-frame / inter-frame predicted image start code (such as a start code value of B3 or B6), a sequence start code (such as a start code value of B0), a sequence end code (such as a start code value of B1), or a video editing code (such as a start code value of B7).

[0029] Optionally, the payload type of the meta-stream in this embodiment may include at least one of the following:

[0030] (1) a sequence header meta-code stream, i.e., a meta-code stream of a sequence header type, wherein the sequence header meta-code stream includes sequence header data; for example, the start code prefix is ​​'0000 0000 0000 0000 0000 0001', and the start code value is B0;

[0031] (2) video extension data meta stream (or video extension data meta stream after sequence header), i.e., a meta stream of video extension data type, wherein the video extension data meta stream includes video extension data after the sequence header; for example, the start code prefix is ​​'0000 0000 0000 0000 0000 0001', and the start code value is B5;

[0032] (3) User data meta-code stream (or user data meta-code stream after sequence header), i.e., the meta-code stream of user data type, wherein the user data meta-code stream includes the user data after the sequence header; for example, the start code prefix is ​​'0000 0000 00000000 0000 0001', and the start code value is B2;

[0033] (4) I frame meta stream, i.e., a meta stream of an I frame type, wherein the I frame meta stream includes an image header of the I frame and video extension data, user data, and slice data after the image header of the I frame. The slice data may be all slice data; for example, the start code prefix is ​​'0000 0000 0000 0000 0000 0001', and the start code value is B3;

[0034] (5) RL frame meta-code stream, i.e., a meta-code stream of the RL frame type, wherein the RL frame meta-code stream includes an image header of the RL frame and video extension data, user data, and slice data after the image header of the RL frame. The slice data may be all slice data; for example, the start code prefix is ​​'0000 0000 0000 0000 0000 0001', and the start code value is B6;

[0035] (6) P frame meta stream, i.e., a meta stream of the P frame type, wherein the P frame meta stream includes a picture header of the P frame and video extension data, user data, and slice data following the picture header of the P frame. The slice data may be all slice data; for example, the start code prefix is ​​'0000 0000 0000 0000 0000 0001', and the start code value is B6;

[0036] (7) B frame meta stream, i.e., a meta stream of the B frame type, wherein the B frame meta stream includes a picture header of the B frame and video extension data, user data, and slice data following the picture header of the B frame. The slice data may be all slice data; for example, the start code prefix is ​​'0000 0000 0000 0000 0000 0001', and the start code value is B6;

[0037] (8) a sequence end symbol stream, i.e., a symbol stream of a sequence end code type, wherein the sequence end symbol stream includes an end code of a video sequence; for example, the start code prefix is ​​'0000 0000 0000 0000 00000001', and the start code value is B1;

[0038] (9) Video editing code element stream, i.e., a video editing code element stream, which contains the editing code of the video sequence. Based on the editing code, it can be determined that the previous video sequence can be edited independently. For example, the start code prefix is ​​'0000 0000 0000 0000 0000 0001', and the start code value is B7.

[0039] It is understandable that for the above-mentioned I frame meta stream, RL frame meta stream, P frame meta stream and B frame meta stream, since they contain the image header and the video extension data after the image header, user data and all slice data, when performing RTP packetization based on these meta streams, dynamic HDR (High Dynamic Range Imaging) data packetization can be supported. Since the type of video editing code is introduced in the meta stream type, that is, the video editing code meta stream is introduced, the video editing code can be added as a marker of the video sequence to represent / indicate that the video sequence before the video editing code can be edited independently, thereby supporting live production and supporting secondary creation and editing of live video during the live broadcast process.

[0040] In an optional implementation, taking the AVS3 video metastream as an example, the definition of the corresponding payload type may be as shown in Table 1 below:

[0041] Table 1

[0042]

[0043]

[0044] In the embodiment of the present application, when distributing RTP payload data for the cut meta-stream, the RTP payload data can be encapsulated in the form of meta-stream. When the data volume of the meta-stream is small, such as less than the MTU (Maximum Transmission Unit) size of the IP layer, one RTP data packet can contain one or more meta-streams; when the data volume of the meta-stream exceeds the MTU size limit of the IP layer, the meta-stream can be fragmented and RTP packets can be performed based on the meta-stream fragments; the fragmentation cutting rule is, for example, to divide the meta-stream into continuous segments with bytes as the smallest unit.

[0045] Optionally, the encapsulation rule of the RTP data packet may include but is not limited to at least one of the following:

[0046] 1) Different types of metastreams are not allowed to be encapsulated / aggregated in the same RTP packet.

[0047] 2) Under the condition that the aggregate packet size limit is met, continuous sequence header meta-streams, video extension data meta-streams and / or user data meta-streams are allowed to be encapsulated in the same RTP data packet.

[0048] The above 1) and 2) can be considered at the same time. For example, different types of meta-streams are not allowed to be encapsulated / aggregated in the same RTP data packet, but the following exceptions are made: when the size of the aggregated packet meets the MTU size limit of the IP layer, continuous sequence header meta-streams, video extension data meta-streams and / or user data meta-streams are allowed to be encapsulated in the same RTP data packet. In this way, continuous and small-data-volume sequence header meta-streams, video extension data meta-streams and / or user data meta-streams can be encapsulated in one RTP data packet, thereby avoiding multiple RTP data packets, reducing packet delays, and saving network transmission resources. In addition, these data are calculated together in the encoder and uniformly encapsulated in one RTP data packet, making the processing flow simpler.

[0049] 3) Multiple frames are not allowed to be encapsulated / aggregated in the same RTP data packet. This prevents multiple frames from becoming dependent on each other and prevents them from being unable to be processed after packet loss. For example, if an I frame is lost, it needs to be retransmitted, but if a P frame is lost, it is directly lost. If the I frame and the P frame are encapsulated in one RTP data packet, they will not be able to be processed after a certain frame is lost.

[0050] 4) For a meta-stream whose encapsulation size is greater than a first value, the meta-stream must be segmented and the segmented meta-stream segments are packaged using a segment payload format; the first value can be set based on actual needs, such as being equal to the MTU size;

[0051] 5) The sequence end symbol stream or video editing symbol stream is encapsulated into a single RTP data packet; the sequence end code or video editing code indicates the end of the video sequence; since the encapsulation logic of the sequence end symbol stream is to either encapsulate it into an aggregate packet with the last frame of image data or to encapsulate it into an RTP data packet independently, if it is encapsulated together with the last frame of image data, then each time it is processed, the server must cache a frame of image data and wait for the next frame of image data to arrive to determine whether it is the last frame of image data. If it is the last frame of image data, it is encapsulated into an RTP data packet with the sequence end code before sending it. The consequence of this logic is that one frame is cached, causing a delay; in order to avoid this situation, the sequence end symbol stream is encapsulated as an RTP data packet separately. The same logic also applies to the video editing symbol stream, that is, in order to avoid causing delays, the video editing symbol stream is encapsulated as an RTP data packet separately.

[0052] 6) Aggregation packets and fragmentation packets are not allowed to be nested with each other, that is, the aggregate payload is not allowed / cannot contain metastream fragments.

[0053] 7) Meta-stream data of non-independent main bitstream and knowledge bitstream are not allowed to be encapsulated / aggregated in the same RTP packet. Knowledge bitstream is similar to I frame, and non-independent main bitstream is similar to P frame. They need to refer to knowledge bitstream to be decoded, so they cannot be aggregated in one RTP packet. The reason is to prevent multiple frames from being dependent and to avoid being unable to process after packet loss.

[0054] 8) The payload data of RTP packets cannot be nested, that is, the RTP payload cannot contain a complete RTP packet. This is because RTP is at the transport level and network devices do not support identification. If a complete RTP packet is nested in the RTP payload, the RTP packet will not be recognized.

[0055] It should be pointed out that, taking the AVS3 video code stream as an example, the RTP payload format of the AVS3 video code stream includes two parts: the RTP payload header and the RTP payload carrying the AVS3 video bit stream data, wherein the RTP payload can be divided into a single payload, an aggregate payload, and a fragmented payload according to its payload type. Different types of payloads all contain the AVS general payload header.

[0056] In the embodiment of the present application, RTP packetization can be performed according to the payload type of the meta-stream. According to the payload type of each meta-stream, encapsulating the meta-stream into an RTP data packet may include:

[0057] When the meta-stream is a first-class meta-stream, the meta-stream is encapsulated into an RTP data packet, or the meta-stream and K meta-streams after the meta-stream are encapsulated into an RTP data packet; wherein K is a positive integer; the sum of the sizes of the meta-stream and the K meta-streams after the meta-stream is less than or equal to a second value, and at least one of the following is satisfied: the K+1th meta-stream after the meta-stream is not a first-class meta-stream, the sum of the sizes of the meta-stream and the K+1 meta-streams after the meta-stream is greater than the second value; each of the K meta-streams is a first-class meta-stream; the first-class meta-stream is any one of the following: a sequence header meta-stream, a video extension data meta-stream, and a user data meta-stream. In this way, a continuous sequence header meta-stream, a video extension data meta-stream, and / or a user data meta-stream with a small amount of data can be encapsulated in one RTP data packet, thereby avoiding multiple RTP data packets and reducing packet delay.

[0058] Optionally, the second value can be set based on actual needs, such as being equal to the MTU size. When the first meta-code stream after the meta-code stream is not a first-class meta-code stream, or the sum of the sizes of the meta-code stream and the first meta-code stream after it is greater than the second value, the meta-code stream can be directly encapsulated into an RTP data packet.

[0059] Optionally, encapsulating the meta-stream into an RTP data packet according to the payload type of each meta-stream may further include:

[0060] When the meta-code stream is a second-category meta-code stream, the meta-code stream is encapsulated into an RTP data packet; wherein the second-category meta-code stream is any one of the following: an I-frame meta-code stream, an RL-frame meta-code stream, a P-frame meta-code stream, a B-frame meta-code stream, a sequence end code stream, and a video editing code stream.

[0061] For example, when the meta-stream is an I-frame meta-stream, an RL-frame meta-stream, a P-frame meta-stream or a B-frame meta-stream, the meta-stream can be directly encapsulated into an RTP data packet, which can prevent multiple frames from being dependent on each other and avoid being unable to process after packet loss.

[0062] For another example, when the element stream is a sequence end element stream or a video editing element stream, the element stream can be directly encapsulated into an RTP data packet, so as to avoid the delay caused by caching the image frame when aggregating with the last frame of image data, thereby reducing the packet delay.

[0063] Optionally, before encapsulating the meta-stream into an RTP data packet according to the payload type of each meta-stream, the video bit stream encapsulation method in the embodiment of the present application may further include:

[0064] Determine whether the size of the meta-code stream is less than a third value; the third value can be set based on actual needs, such as being equal to the MTU size, or equal to the difference between the MTU size and N, where N is a configurable number of reserved header bytes.

[0065] According to the payload type of each meta-stream, encapsulating the meta-stream into an RTP data packet may include:

[0066] When the size of the first element code stream among the multiple element code streams is less than the third value, the first element code stream is encapsulated into an RTP data packet according to the payload type of the first element code stream. The specific manner of performing RTP encapsulation according to the payload type can be referred to the above content, which will not be repeated here. In this way, RTP encapsulation can be performed while ensuring that the RTP data packet meets the size limit.

[0067] Optionally, after determining whether the size of the elementary code stream is less than the third value, the video bit stream encapsulation method in the embodiment of the present application may further include any of the following items:

[0068] When the size of a second meta-stream among the multiple meta-streams is greater than a third value, segmenting the second meta-stream, and encapsulating the segmented meta-stream segments into RTP data packets using a segment payload format;

[0069] When the size of a third element code stream among the multiple element code streams is equal to a third value, the third element code stream is encapsulated into an RTP data packet.

[0070] For example, when the second meta-stream is segmented and the segmented meta-stream segments are encapsulated into RTP data packets using a segment payload format, the second meta-stream may be segmented into n meta-stream segments, and each of the n meta-stream segments may be encapsulated into one RTP data packet; wherein, K represents the size of the second bitstream, s represents the third value, represents rounding up; the sizes of the first n-1 meta-stream slices among the n meta-stream slices are equal to the third value, and the size of the nth meta-stream slice is equal to K%s.

[0071] In an optional implementation manner, the specific RTP packet process may include:

[0072] S1: Cut the encoded video bit stream to obtain multiple meta-streams (ie, multiple slices), and perform the following steps for each meta-stream.

[0073] S2: Calculate the size of the byte count M(i) of the i-th meta-code stream;

[0074] S3: When M(i) is less than the difference between the MTU size of the IP layer and N, where N is the configurable number of reserved header bytes, perform the following steps:

[0075] S31: Determine the payload type T(i) of the i-th element stream;

[0076] If T(i) is a sequence header type, a video extension data type after a sequence header, or a user data type after a sequence header, that is, the i-th meta stream is a sequence header meta stream, a video extension data meta stream after a sequence header, or a user data meta stream after a sequence header, then execute S32 and S33;

[0077] If T(i) is a sequence end code type or a video editing code type, that is, the i-th element code stream is a sequence end code element code stream or a video editing code element code stream, then execute S34;

[0078] If T(i) is an I frame, RL frame, P frame or B frame type, that is, the i-th meta-code stream is an I frame meta-code stream, an RL frame meta-code stream, a P frame meta-code stream or a B frame meta-code stream, then S34 is executed.

[0079] S32: Determine the payload type T(i+1) of the i+1th meta-stream. If T(i+1) is a sequence header type, a video extension data type after a sequence header, or a user data type after a sequence header, calculate the size byte number M(i+1) of the i+1th meta-stream.

[0080] S33: If M(i)+M(i+1) is smaller than the MTU size of the IP layer, S32 is executed in a loop to process the i+2th meta-code stream until M(i)+M(i+1)+...+M(i+k) is larger than the MTU size of the IP layer, or T(i+k) is not a sequence header type, a video extension data type after a sequence header, or a user data type after a sequence header; at this time, the i-th meta-code stream, the i+1-th meta-code stream, ..., the i+k-1-th meta-code stream are encapsulated into an aggregate packet, that is, the continuous sequence header, the video extension data after the sequence header, and the user data meta-code stream after the sequence header are encapsulated into one aggregate packet, and the aggregate packet data size is smaller than the MTU size of the IP layer;

[0081] S34: Encapsulate the i-th meta-stream into a single RTP data packet.

[0082] S4: When M(i) is larger than the difference between the MTU size of the IP layer and N, the i-th meta-stream is divided twice into M(i) / (MTU-N) meta-stream fragments, where the size of the first M(i) / (MTU-N)-1 meta-stream fragments is MTU-N bytes, and the size of the last meta-stream fragment is M(i)%(MTU-N); and the divided meta-stream fragments are encapsulated into RTP data packets.

[0083] S5: When M(i) is equal to the difference between the MTU size of the IP layer and N, encapsulate the i-th element stream into a single RTP data packet.

[0084] It should be noted that the video bitstream encapsulation method provided in the embodiment of the present application can be executed by a video bitstream encapsulation device, or a control module in the video bitstream encapsulation device for executing the video bitstream encapsulation method. In the embodiment of the present application, the video bitstream encapsulation device provided in the embodiment of the present application is described by taking the video bitstream encapsulation device executing the video bitstream encapsulation method as an example.

[0085] See also Figure 2 , Figure 2 is a schematic diagram of the structure of a video bit stream encapsulation device provided in an embodiment of the present application, and the device is applied to electronic devices such as Figure 2 As shown, the video bit stream encapsulation device 20 comprises:

[0086] A cutting module 21 is used to cut the encoded video bit stream to obtain multiple meta-streams;

[0087] The encapsulation module 22 is used to encapsulate each meta-stream into an RTP data packet according to the payload type of each meta-stream.

[0088] Optionally, the payload type of the meta-stream includes at least one of the following:

[0089] A sequence header meta-code stream, wherein the sequence header meta-code stream includes sequence header data;

[0090] A video extension data element code stream, wherein the video extension data element code stream includes video extension data after a sequence header;

[0091] A user data meta-code stream, wherein the user data meta-code stream includes user data after a sequence header;

[0092] An I-frame meta-code stream, wherein the I-frame meta-code stream includes an image header of the I-frame and video extension data, user data, and slice data after the image header of the I-frame;

[0093] An RL frame meta-code stream, wherein the RL frame meta-code stream includes an image header of the RL frame and video extension data, user data, and slice data following the image header of the RL frame;

[0094] A P frame meta stream, wherein the P frame meta stream includes a P frame image header and video extension data, user data, and slice data after the P frame image header;

[0095] A B frame meta stream, wherein the B frame meta stream includes a picture header of the B frame and video extension data, user data, and slice data after the picture header of the B frame;

[0096] A sequence end symbol stream, wherein the sequence end symbol stream includes an end code of a video sequence;

[0097] A video editing codeword stream includes editing codes of a video sequence.

[0098] Optionally, the encapsulation rule of the RTP data packet includes at least one of the following:

[0099] Different types of metastreams are not allowed to be encapsulated in the same RTP packet;

[0100] In the case where the aggregate packet size limit is met, continuous sequence header meta-streams, video extension data meta-streams and / or user data meta-streams are allowed to be encapsulated in the same RTP data packet;

[0101] Multiple frames of images are not allowed to be encapsulated in the same RTP packet;

[0102] For a meta-code stream whose encapsulation size is greater than the first value, the meta-code stream must be segmented, and the segmented meta-code stream segments are packaged using a segment payload format;

[0103] The sequence end symbol stream or the video editing symbol stream is encapsulated into a single RTP data packet;

[0104] Aggregation packages and fragmentation packages are not allowed to be nested within each other.

[0105] Optionally, the encapsulation module 22 is specifically used to: when the meta-code stream is a first-category meta-code stream, encapsulate the meta-code stream into an RTP data packet, or encapsulate the meta-code stream and K meta-code streams after the meta-code stream into an RTP data packet;

[0106] The K is a positive integer; the sum of the sizes of the meta stream and K meta streams after the meta stream is less than or equal to a second value, and at least one of the following is satisfied: the K+1th meta stream after the meta stream is not a first-category meta stream, and the sum of the sizes of the meta stream and K+1 meta streams after the meta stream is greater than the second value; each of the K meta streams is a first-category meta stream; and the first-category meta stream is any one of the following: a sequence header meta stream, a video extension data meta stream, and a user data meta stream.

[0107] Optionally, the encapsulation module 22 is specifically used to: when the meta-code stream is a second-category meta-code stream, encapsulate the meta-code stream into an RTP data packet; wherein the second-category meta-code stream is any one of the following: I frame meta-code stream, RL frame meta-code stream, P frame meta-code stream, B frame meta-code stream, sequence end code stream and video editing code stream.

[0108] Optionally, the video bitstream encapsulation device 20 further includes

[0109] A judging module, configured to judge whether the size of each meta-stream is less than a third value before encapsulating the meta-stream into an RTP data packet according to the payload type of each meta-stream;

[0110] The encapsulation module 22 is specifically configured to: when the size of a first element code stream among the multiple element code streams is smaller than the third value, encapsulate the first element code stream into an RTP data packet according to the payload type of the first element code stream.

[0111] Optionally, the encapsulation module 22 is further configured to perform any of the following:

[0112] When the size of a second meta-stream among the multiple meta-streams is greater than the third value, segmenting the second meta-stream, and encapsulating the segmented meta-stream segments into RTP data packets using a segment payload format;

[0113] When the size of a third element code stream among the multiple element code streams is equal to the third value, the third element code stream is encapsulated into an RTP data packet.

[0114] The video bitstream encapsulation device 20 of the embodiment of the present application can realize the above Figure 1 The various processes of the method embodiment shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0115] Optional, such as Figure 3 As shown, an embodiment of the present application also provides an electronic device 30, including a processor 31, a memory 32, and a program or instruction stored in the memory 32 and executable on the processor 31. When the program or instruction is executed by the processor 31, each process of the above-mentioned video bitstream encapsulation method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0116] An embodiment of the present application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of the above-mentioned video bitstream encapsulation method embodiment can be implemented and the same technical effect can be achieved. To avoid repetition, they will not be repeated here.

[0117] An embodiment of the present application also provides a readable storage medium on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned video bitstream encapsulation method embodiment can be implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0118] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0119] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0120] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0121] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a service classification device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0122] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A video bit stream encapsulation method, characterized in that: include: Cutting the encoded video bit stream to obtain multiple meta-streams; According to the load type of each of the metadata streams, the metadata streams are encapsulated into real-time transport protocol RTP data packets.

2. The method according to claim 1, characterized in that The payload type of the metastream includes at least one of the following: A sequence header meta-code stream, wherein the sequence header meta-code stream includes sequence header data; A video extension data element code stream, wherein the video extension data element code stream includes video extension data after a sequence header; A user data meta-code stream, wherein the user data meta-code stream includes user data after a sequence header; An I-frame meta-code stream, wherein the I-frame meta-code stream includes an image header of the I-frame and video extension data, user data, and slice data after the image header of the I-frame; An RL frame meta-code stream, wherein the RL frame meta-code stream includes an image header of the RL frame and video extension data, user data, and slice data following the image header of the RL frame; A P frame meta stream, wherein the P frame meta stream includes a P frame image header and video extension data, user data, and slice data after the P frame image header; A B frame meta stream, wherein the B frame meta stream includes a picture header of the B frame and video extension data, user data, and slice data after the picture header of the B frame; A sequence end symbol stream, wherein the sequence end symbol stream includes an end code of a video sequence; A video editing codeword stream includes editing codes of a video sequence.

3. The method according to claim 1, characterized in that: The encapsulation rule of the RTP data packet includes at least one of the following: Different types of metastreams are not allowed to be encapsulated in the same RTP packet; In the case where the aggregate packet size limit is met, continuous sequence header meta-streams, video extension data meta-streams and / or user data meta-streams are allowed to be encapsulated in the same RTP data packet; Multiple frames of images are not allowed to be encapsulated in the same RTP packet; For a meta-code stream whose encapsulation size is greater than the first value, the meta-code stream must be segmented, and the segmented meta-code stream segments are packaged using a segment payload format; The sequence end symbol stream or the video editing symbol stream is encapsulated into a single RTP data packet; Aggregation packages and fragmentation packages are not allowed to be nested within each other.

4. The method according to any one of claims 1 to 3, characterized in that: The step of encapsulating the meta-stream into an RTP data packet according to the payload type of each meta-stream comprises: When the meta-stream is a first-category meta-stream, encapsulating the meta-stream into an RTP data packet, or encapsulating the meta-stream and K meta-streams following the meta-stream into an RTP data packet; wherein K is a positive integer; the sum of the sizes of the meta-codestream and K meta-codestreams following the meta-codestream is less than or equal to a second value, and at least one of the following conditions is satisfied: the K+1th meta-codestream following the meta-codestream is not a first-category meta-codestream, and the sum of the sizes of the meta-codestream and K+1th meta-codestreams following the meta-codestream is greater than the second value; Each of the K meta-code streams is a first-category meta-code stream; The first type of meta-stream is any one of the following: a sequence header meta-stream, a video extension data meta-stream and a user data meta-stream.

5. The method according to any one of claims 1 to 3, characterized in that: The step of encapsulating the meta-stream into an RTP data packet according to the payload type of each meta-stream comprises: When the meta-code stream is a meta-code stream of the second type, encapsulating the meta-code stream into an RTP data packet; The second type of meta-code stream is any one of the following: an I frame meta-code stream, an RL frame meta-code stream, a P frame meta-code stream, a B frame meta-code stream, a sequence end code stream and a video editing code stream.

6. The method according to claim 1, characterized in that Before encapsulating the meta-stream into an RTP data packet according to the payload type of each meta-stream, the method further comprises: Determining whether the size of the elementary code stream is less than a third value; The step of encapsulating the meta-stream into an RTP data packet according to the payload type of each meta-stream comprises: When the size of a first element code stream among the multiple element code streams is smaller than the third value, the first element code stream is encapsulated into an RTP data packet according to a payload type of the first element code stream.

7. The method according to claim 6, characterized in that After determining whether the size of the elementary code stream is less than the third value, the method further includes any one of the following: When the size of a second meta-stream among the multiple meta-streams is greater than the third value, segmenting the second meta-stream, and encapsulating the segmented meta-stream segments into RTP data packets using a segment payload format; When the size of a third element code stream among the multiple element code streams is equal to the third value, the third element code stream is encapsulated into an RTP data packet.

8. A video bitstream encapsulation device, characterized in that: include: A cutting module, used for cutting the encoded video bit stream to obtain multiple meta-streams; The encapsulation module is used to encapsulate the meta-stream into an RTP data packet according to the load type of each meta-stream.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 7.

Images