Method and apparatus for transmitting AVS3 video code stream

Abstract

Provided in the embodiments of the present disclosure are a method and apparatus for transmitting an AVS3 video code stream. The method comprises: encapsulating an AVS3 video code stream into a plurality of real-time transport protocol (RTP) packets, wherein each RTP packet is provided therein with an AVS3 load head and an AVS3 load; and successively transmitting the plurality of RTP packets in the order of encapsulation.

Description

AVS3 video encoding stream transmission method and device Technical Field

[0001] The embodiments of the present disclosure relate to the field of communication technology, and in particular to a method and device for transmitting an AVS3 video encoding stream. Background Art

[0002] The video encoding stream of the third-generation audio and video codec technology standard (AVS3) is usually transmitted using TS (Transport Stream) and DASH (Dynamic Adaptive Streaming over HTTP). However, with the development of real-time interactive audio and video technology, AVS3 needs to support RTP (Real-time Transport Protocol) as soon as possible to meet the different transmission requirements of different services.

[0003] Summary of the Invention

[0004] The embodiments of the present disclosure provide a method and apparatus for transmitting an AVS3 video encoding stream, so as to at least solve the problem in the related art that the AVS3 video encoding stream needs to support the RTP transmission mode as soon as possible.

[0005] According to one embodiment of the present disclosure, a method for transmitting an AVS3 video encoding stream is provided, comprising: encapsulating the AVS3 video encoding stream into multiple real-time transport protocol RTP packets, wherein each of the RTP packets is provided with an AVS3 payload header and an AVS3 payload; and transmitting the multiple RTP packets in sequence in the order of encapsulation.

[0006] According to another embodiment of the present disclosure, a transmission device for an AVS3 video encoding stream is provided, including: an encapsulation module, configured to encapsulate the AVS3 video encoding stream into multiple RTP packets, wherein each of the RTP packets is provided with an AVS3 payload header and an AVS3 payload; a transmission module, configured to transmit the multiple RTP packets in sequence in the order of encapsulation.

[0007] According to another embodiment of the present disclosure, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is configured to execute the steps of any one of the above method embodiments when running.

[0008] According to another embodiment of the present disclosure, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any one of the above method embodiments.

[0009] Through the above-described embodiments of the present invention, the AVS3 video encoding stream is encapsulated into multiple Real-time Transport Protocol (RTP) packets, thereby enabling transmission of the AVS3 video encoding stream via RTP. This solves the problem of the AVS3 video encoding stream needing to support the RTP transmission mode as soon as possible. Furthermore, the AVS3 payload header is provided in the RTP packet to indicate relevant feature information and key information in the AVS3 video encoding stream, facilitating efficient and real-time transmission of the AVS3 video encoding stream. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] FIG1 is a hardware structure block diagram of a computer terminal for executing the AVS3 video encoding code stream transmission method according to an embodiment of the present disclosure;

[0011] FIG2 is a network architecture diagram of a transmission system for an AVS3 video encoding code stream according to an embodiment of the present disclosure;

[0012] FIG3 is a flowchart of a method for transmitting an AVS3 video encoding stream according to an embodiment of the present disclosure;

[0013] FIG4 is a structural block diagram of a device for transmitting an AVS3 video encoding stream according to an embodiment of the present disclosure;

[0014] FIG5 is a schematic diagram of the structure of an AVS3 video encoding stream according to an embodiment of the present disclosure;

[0015] FIG6 is a flowchart of an RTP-based AVS3 video encoding stream encapsulation and transmission method according to an embodiment of the present disclosure;

[0016] FIG7 is a schematic diagram of an RTP header according to an embodiment of the present disclosure;

[0017] FIG8 is a schematic diagram of an AVS3 load head according to an embodiment of the present disclosure;

[0018] FIG9 is a schematic diagram of an AVS3 universal header according to an embodiment of the present disclosure;

[0019] FIG10 is a schematic diagram of a simplified AVS3 universal header according to an embodiment of the present disclosure;

[0020] FIG11 is a schematic diagram of an RTP packet encapsulation method in a single packetization mode according to an embodiment of the present disclosure;

[0021] 12 is a schematic diagram of an RTP packet including an extension header in a single packetization mode according to an embodiment of the present disclosure;

[0022] 13 is a schematic diagram of an extended header of an RTP packet in a single packet manner according to an embodiment of the present disclosure;

[0023] FIG14 is a flow chart of encapsulating an AVS3 video encoding stream in a single packet according to an embodiment of the present disclosure;

[0024] FIG15 is a schematic diagram of an RTP packet encapsulation method according to an embodiment of the present disclosure;

[0025] FIG16 is a schematic diagram of an extension header of an RTP packet in a combined packet manner according to an embodiment of the present disclosure;

[0026] FIG17 is a flow chart of encapsulating an AVS3 video encoding stream in a combined packet manner according to an embodiment of the present disclosure;

[0027] FIG18 is a schematic diagram of adding data unit indication length information according to an embodiment of the present disclosure;

[0028] FIG19 is a schematic diagram of RTP packet encapsulation in a segmented packet manner according to an embodiment of the present disclosure;

[0029] FIG20 is a schematic diagram of an AVS3 extension header in a segmented packetization method according to an embodiment of the present disclosure;

[0030] FIG21 is a flow chart of encapsulating an AVS3 video encoding stream in a segmented packet manner according to an embodiment of the present disclosure;

[0031] FIG22 is a flow chart of encapsulating an AVS3 video encoding stream in a homogeneous packaging manner according to an embodiment of the present disclosure;

[0032] FIG23 is a flowchart of a method for decoding an RTP packet according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings and in conjunction with the embodiments.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0035] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking operation on a computer terminal as an example, FIG1 is a hardware structure block diagram of a computer terminal for running the transmission method of the AVS3 video encoding code stream in an embodiment of the present disclosure. As shown in FIG1 , the computer terminal may include one or more (only one is shown in FIG1 ) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 configured to store data, wherein the above-mentioned computer terminal may also include a transmission device 106 and an input and output device 108 configured to have a communication function. It can be understood by those skilled in the art that the structure shown in FIG1 is only for illustration and does not limit the structure of the above-mentioned computer terminal. For example, the computer terminal may also include more or fewer components than those shown in FIG1 , or have a configuration different from that shown in FIG1 .

[0036] The memory 104 can be configured to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the transmission method of the AVS3 video encoding code stream in the embodiment of the present disclosure. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above-mentioned method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0037] The transmission device 106 is configured to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by a computer terminal's communications provider. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module configured to communicate with the Internet wirelessly.

[0038] The embodiment of the present application can be run on the network architecture shown in Figure 2. As shown in Figure 2, the network architecture includes: an RTP transmitter 10 and an RTP receiver 20. RTP is a commonly used real-time multimedia transmission protocol that can provide end-to-end real-time transmission services for various multimedia data that require real-time transmission, such as voice and images, on IP (Internet Protocol) networks. At the same time, RTP is also widely used in streaming media-related communications and entertainment.

[0039] As shown in FIG. 2 , the RTP sending end 10 includes an AVS3 video encoding module 11 , an RTP packaging module 12 and an RTP transmission module 13 .

[0040] The AVS3 video encoding module 11, namely the AVS3 video encoder, is configured to encode the original video according to the AVS3 standard to generate an AVS3 video encoding stream. The original video includes but is not limited to one of the following: video captured by a camera, video generated by a computer.

[0041] The RTP packetization module 12 is configured to encapsulate the generated AVS3 video encoding stream into RTP packets, wherein the AVS3 video encoding stream needs to be split according to a certain length (i.e., data volume) to ensure that the length of each RTP packet (i.e., the size of the RTP packet) does not exceed the limit.

[0042] The RTP transmission module 13 is configured to transmit the RTP packet carrying the AVS3 video encoding stream to the RTP receiving end 20.

[0043] As shown in FIG. 2 , the RTP receiving end 20 includes an RTP receiving module 21 , an RTP decapsulating module 22 and an AVS3 video decoding module 23 .

[0044] The RTP receiving module 21 is configured to receive the RTP packet containing the AVS3 video encoding stream transmitted by the RTP transmitting module 13 .

[0045] The RTP decapsulation module 22 is configured to decapsulate the RTP packet received by the RTP receiving module 21 and extract the AVS3 video encoding stream data carried in the RTP packet.

[0046] The AVS3 video decoding module 23 , namely the AVS3 video decoder, is configured to decode the AVS3 video encoding stream data extracted by the RTP depacketizing module 22 .

[0047] In this embodiment, a method for transmitting an AVS3 video encoding stream running on the above-mentioned computer terminal or network architecture is provided. FIG3 is a flow chart of the method for transmitting an AVS3 video encoding stream according to an embodiment of the present disclosure. As shown in FIG3 , the process includes the following steps:

[0048] Step S302: encapsulate the AVS3 video encoding stream into multiple real-time transport protocol RTP packets, wherein each of the RTP packets is provided with an AVS3 payload header and an AVS3 payload;

[0049] In step S302 of this embodiment, the AVS3 video encoding stream is encapsulated into multiple RTP data packets, including: dividing the AVS3 video encoding stream into multiple data units according to the start code in the AVS3 video encoding stream; encapsulating the multiple data units into the multiple RTP packets, wherein each RTP packet carries one data unit, multiple data units and / or partial data of one or more data units.

[0050] In one embodiment, encapsulating the multiple data units into the multiple RTP packets includes at least one of the following: using a first packetization method to packetize a data unit into an RTP packet; using a second packetization method to packetize multiple data units of the same or different data types into an RTP packet; using a third packetization method to packetize part of the data of a data unit into an RTP packet; using a fourth packetization method to packetize all and / or part of the data of one or more data units of the same data type into an RTP packet.

[0051] In one embodiment, the AVS3 payload header includes an AVS3 general header, and the information indicated by the AVS3 general header includes at least one of the following: a discard indication, a time level, a payload type, a knowledge image indication, a knowledge image extension, an extension indication, and a packet type indication.

[0052] In one embodiment, a data unit is packaged into an RTP packet using a first packaging method, including: when the length of a data unit to be encapsulated (i.e., the size of the data unit) is less than or equal to the AVS3 video encoding data payload of the RTP packet, the data unit is individually encapsulated in one of the RTP packets.

[0053] The video encoding streams generated by different encoders in the RTP protocol (such as H.264, VP8, etc.) need to be extended to indicate some basic feature information and key information in the encoding stream, so as to meet the efficient and real-time transmission requirements of different encoding stream characteristics.

[0054] In one embodiment, in the RTP packet obtained by adopting the second packetization method, the AVS3 payload header of the RTP packet includes an AVS3 extension header; the information indicated by the AVS3 extension header includes at least one of the following: reference frame image indication, key parameter indication, heterogeneous data indication, slice data indication and number of image frames.

[0055] In one embodiment, a second packetization method is used to packetize multiple data units of the same or different data types into one RTP packet, including: when the sum of the lengths of the multiple adjacent data units to be encapsulated is less than or equal to the AVS3 video encoding data payload of the RTP packet, the multiple data units are encapsulated in one RTP packet.

[0056] In one embodiment, in the RTP packet obtained by adopting the third packetization method, the AVS3 payload header of the RTP packet includes an AVS3 extension header; the information indicated by the AVS3 extension header includes at least one of the following: the type of segment, the start of the segment, and the end of the segment.

[0057] In one embodiment, a third packetization method is used to packetize part of the data of a data unit into an RTP packet, including: when the length of a data unit to be encapsulated is greater than the AVS3 video encoding data payload of the RTP packet, encapsulating the data unit into multiple RTP packets, wherein each RTP packet contains part of the data of the data unit.

[0058] In one embodiment, in the RTP packet obtained by adopting the fourth packetization method, the information indicated by the AVS3 general header in the RTP packet includes at least one of the following: segment indication, the start of the segment, and the end of the segment; or, the AVS3 payload header in the RTP packet includes an AVS3 extension header, and the information indicated by the AVS3 extension header includes at least one of the following: segment indication, the start of the segment, and the end of the segment.

[0059] In one embodiment, a fourth packetization method is used to packetize all and / or part of the data of one or more data units of the same data type into an RTP packet, including: according to the AVS3 video encoding data payload of the RTP packet, encapsulating at least one data unit of the same data type whose length is smaller than the AVS3 video encoding data payload and / or part of the data of at least one data unit in one of the RTP packets.

[0060] In one embodiment, the multiple data units are distinguished based on the start code or based on the indication information of the data unit length in the AVS3 payload.

[0061] Step S304: transmitting the multiple RTP packets in the order of encapsulation, that is, transmitting them in the order of AVS3 video encoding streams.

[0062] Through the above steps, the AVS3 video encoding stream is encapsulated into multiple Real-time Transport Protocol (RTP) packets, allowing the AVS3 video encoding stream to be transmitted via RTP. This solves the problem of the AVS3 video encoding stream needing to support the RTP transmission mode as soon as possible. Furthermore, the AVS3 payload header is provided in the RTP packet to indicate relevant feature information and key information in the AVS3 video encoding stream, facilitating efficient and real-time transmission of the AVS3 video encoding stream.

[0063] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present disclosure.

[0064] In this embodiment, a device for transmitting an AVS3 video encoding code stream is also provided. The device is used to implement the above-mentioned embodiments and preferred implementation methods. The details that have been described will not be repeated here. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0065] FIG4 is a structural block diagram of a transmission device for an AVS3 video encoding stream according to an embodiment of the present disclosure. As shown in FIG4 , the device includes:

[0066] The encapsulation module 41 is configured to encapsulate the AVS3 video encoding stream into multiple RTP packets, wherein each of the RTP packets is provided with an AVS3 payload header; the encapsulation module 41 is functionally equivalent to the RTP packetization module 12 in the above embodiment.

[0067] The transmission module 42 is configured to transmit the multiple RTP packets in sequence according to the order of the encapsulated timestamps. The transmission module 42 is functionally equivalent to the RTP transmission module 13 in the above embodiment.

[0068] It should be noted that the above modules can be implemented through software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0069] The present disclosure provides a method for transmitting an AVS3 video encoding stream based on the RTP protocol. The method is applied to the RTP transmitter 10 in the network architecture of FIG2 . The method includes the following steps:

[0070] Step S1, processing the captured original video through the AVS3 encoder to obtain an AVS3 video encoding stream to be transmitted, that is, an AVS3 video encoding bit stream, wherein the original video can be captured by a camera.

[0071] Figure 5 is a structural diagram of the AVS3 video encoding stream according to an embodiment of the present disclosure. As shown in Figure 5, the AVS3 video encoding stream contains one or more (AVS3) video sequences; a video sequence contains an AVS3 video sequence header, one or more user / extended data, one or more intra-frame prediction images, and one or more inter-frame prediction images.

[0072] The intra-frame prediction image includes an intra-frame prediction image header, one or more user / extension data, and intra-frame prediction image data;

[0073] The inter-frame prediction picture includes an inter-frame prediction picture header, one or more user / extension data, and inter-frame prediction picture data.

[0074] The intra-frame prediction image data and the inter-frame prediction image data are respectively composed of one or more patches of data (FIG. 5 only uses the intra-frame prediction image data as an example for illustration);

[0075] In one embodiment, the slice data may be further divided, and the minimum division unit is the coding unit data (CU).

[0076] Step S2: split the AVS3 video encoding stream and encapsulate it into multiple RTP packets, and then transmit it through the network.

[0077] FIG6 is a flow chart of an RTP-based AVS3 video encoding stream encapsulation and transmission method according to an embodiment of the present disclosure. As shown in FIG6 , the process includes the following steps:

[0078] Step S602: The RTP transmitter splits the code stream into multiple data units according to the start code in the AVS3 video encoding code stream.

[0079] According to the AVS3-P2 standard, in an AVS3 video coding stream, a start code is provided for each video sequence header, user / extended data, intra-frame prediction image header, inter-frame prediction image header, each slice of intra-frame prediction image data, and each slice of inter-frame prediction image data. The start code can be used to separate data of different data types. That is, the data from one start code to the next can be regarded as a data unit of the AVS3 video coding stream. The start code of each data unit (or the start code value in the start code) can be used to identify the data type corresponding to the data unit. The start code consists of a start code prefix and a start code value. A data unit can contain a start code or only the start code value in the start code without the start code prefix.

[0080] In this embodiment, the coded images output by the AVS3 encoder are arranged in a code stream order, wherein the coded code stream order is the same as the decoded code stream order.

[0081] Step S604: The split AVS3 video encoding stream is packetized using the RTP protocol. The split data units are encapsulated into one or more RTP packets according to the length limit of the data carried by a single RTP packet. Depending on the length of each data unit, each RTP packet can carry one data unit, multiple data units, and / or one or more data unit segments (i.e., partial data of a data unit).

[0082] In this embodiment, when RTP transmits the AVS3 video encoding stream, the AVS3 video encoding stream structure must first be split into multiple data units, and then the data units must be encapsulated into multiple RTP packets based on the length of the data units and other requirements (such as media processing), and finally transmitted in the form of RTP packets over the network.

[0083] In a video transmission network system, the MTU (Maximum Transmission Unit) size is limited. For example, the MTU of Ethernet is 1500 bytes.

[0084] For media transmission based on IP / TCP / UDP, the length of the RTP packet plus the length of the IP header (at least 20 bytes) or the UDP header (8 bytes) must not be greater than the length of the MTU (i.e., the size of the MTU). An RTP packet contains an RTP header (12 bytes) and an RTP payload. The AVS3 video encoding stream data is actually encapsulated in the RTP payload, so the AVS3 video encoding stream data that can be loaded in an RTP packet has a length limit (i.e., a size limit). Generally, the maximum length of the RTP payload is 1460 bytes, and in normal transmission instances, the length limit is even less than 1460 bytes. In this embodiment, the AVS3 video encoding stream is transmitted based on the RTP protocol, and the split data unit is used as the minimum unit, and then one or more data units are encapsulated into an RTP packet for transmission. If the length of a single data unit is greater than the length limit of the RTP payload, it needs to be further split into segments of data units less than the length limit to ensure that each RTP packet is less than the length limit to meet the MTU size in network transmission.

[0085] Step S606: The RTP sending end establishes a transmission channel with the RTP receiving end, and transmits the RTP packets in the order of encapsulation.

[0086] In one embodiment, steps S601 and S602 can be performed simultaneously. That is, when the RTP transmitter sequentially splits the code stream according to the start code in the AVS3 video encoding code stream, each time one or more data units are split, they are encapsulated individually or combined into an RTP packet and then directly transmitted, and the code stream is further split and encapsulated into the next RTP packet. If the length of a data unit is greater than the length limit of the RTP packet payload data (i.e., the amount of data that the RTP packet can carry), the data unit is further split into two or more segments, and each segment is encapsulated into an RTP packet.

[0087] In one embodiment, when performing RTP packetization based on the data type and data length (i.e., data volume) corresponding to the data unit and the requirements of transmission processing, the following packetization principles must be followed:

[0088] 1. The code stream order (i.e., bit stream order) of AVS3 video encoding is the same as the decoding order. Packaging is performed according to the code stream order of AVS3 video encoding and transmission is performed according to the packet order. If there is no decoding order between multiple consecutive data units, such as user data, video data, and extended data, they are packaged according to the code stream order.

[0089] 2. Parameter-type data units such as sequence headers, user data, and extended data are usually small data units. If there is no special requirement, they should be combined into one RTP packet.

[0090] 3. Under the condition that the length of the data unit is less than the RTP packet length limit, the related data units should be encapsulated into one RTP packet. For example, the data units belonging to the same frame image (including the image header data) can be encapsulated into one RTP packet under the condition that the length of the data unit is less than the RTP packet length limit.

[0091] 4. Image frames with shorter data lengths, such as P frames and B frames, can be encapsulated into one RTP packet if the length is less than the RTP packet length limit, but it must be a complete frame of image data.

[0092] In one embodiment, adjacent RTP packets that are RTP encapsulated in the order of the AVS3 video encoding stream may adopt different packetization methods. The packetization method of each RTP packet is determined based on the data length, type or other necessary requirements of one or more data units to be encapsulated.

[0093] An RTP packet includes an RTP header and an RTP payload, and the RTP payload is used to carry AVS3 video encoding stream data. In the disclosed embodiment, basic extended information for RTP to support the transmission of AVS3 video encoding streams is also provided. As shown in Figure 7, in the RTP packet encapsulating the AVS3 video payload, the RTP payload after the RTP header includes two parts: the AVS3 payload header and the AVS3 payload.

[0094] In one embodiment, the length of the AVS3 payload header is an integer multiple of 8 bits, including an AVS3 common header and an AVS3 header extension, wherein the AVS3 common header is a fixed length, for example: 8 or 16 bytes, and the AVS3 extension header is an integer multiple of 8 bits. In order to ensure that the AVS3 video encoding stream loaded in the RTP packet carries as much data as possible, the AVS3 extension header is optional, that is, it is used when the extension indication information or other indication information (such as when Payload_Type is 12 or 13) indicates that extension is required.

[0095] The format of the RTP header is specified in IETF RFC 3550. In this embodiment, based on the characteristics of the AVS3 video encoding stream carried in the RTP packet, AVS3 payload header information is further provided on the basis of the general RTP header, as shown in Figure 7. This facilitates network transmission and processing by the RTP receiving module and the decapsulation module.

[0096] The RTP header of the RTP packet based on the AVS3 video payload is set as follows:

[0097] 1. Marker bit (M): 1 bit; used to specify the frame boundary. The marker bit of the last RTP packet of each video frame is set to 1, and the marker bits of other RTP packets are set to 0.

[0098] 2. Payload Type (PT): 7 bits. According to RFC 3551, upper-layer applications (such as SDP) should dynamically assign a PT value in each RTP session and maintain a mapping between the PT value and the payload format. The PT value range should comply with RFC 3551 (for example, 97-127).

[0099] 3. Timestamp: 32 bits. The RTP timestamp is set to the sampling timestamp of the encoded content, using a 90,000 Hz clock. This means the RTP timestamp unit is 1 / 90,000 seconds. Like the sequence number, the initial value of the timestamp should be random. The timestamps of different RTP packets within the same frame should be consistent. For payloads without time attributes, such as sequence headers, user extensions, and image headers, their timestamps should be consistent with the timestamps of the immediately following image data.

[0100] Other fields in the RTP header (V, P, X, CC, etc.) should comply with the provisions of IETF RFC 3550.

[0101] FIG8 is a schematic diagram of an AVS3 payload head according to an embodiment of the present disclosure, and FIG9 is a schematic diagram of an AVS3 universal head according to an embodiment of the present disclosure. As shown in FIG9 , the configuration of the AVS3 universal head in the AVS3 payload head includes:

[0102] Discard Indicator (N): 1 bit, used to indicate whether the AVS3 video codec stream carried by this RTP packet can be discarded during transmission. If discarding the RTP packet does not affect the decoder's decoding and content presentation, the RTP packet can be discarded. The decision to discard is made by the packetizer; if it is not clear, it is always considered non-discardable.

[0103] Temporal ID (TID): 3 bits, Temporal_id, used to indicate the temporal sublayer to which the current RTP payload belongs, with a value range of 0 to 7. The TID value indicates the relative importance of the RTP packet. For example, because image frames belonging to a higher temporal sublayer may not be used to decode image frames of a lower temporal sublayer, a lower TID value indicates greater importance, thus providing better protection against transmission losses.

[0104] Payload_Type: 5 bits, used to indicate the type of the AVS3 video encoding stream data carried in the RTP payload, that is, the data type of the data unit in the stream. Its value and type description are shown in Table 1:

[0105] Table 1

[0106] Knowledge image indication (L): 1 bit, used to indicate whether the current RTP packet contains a knowledge image. For example, when the value is 0, it indicates that the current RTP payload is an AVS3 non-knowledge image; when the value is 1, it indicates that the current RTP payload is an AVS3 knowledge image.

[0107] In one embodiment, when the payload data in the RTP packet is an AVS3 knowledge image, it may carry a knowledge image header, part or all of the knowledge image data, or a knowledge image header and knowledge image data, etc. The specific packaging method is determined by the size of the knowledge image data.

[0108] In one embodiment, the payload type (Payload_Type) can be used to indicate the knowledge image and related data, that is, the knowledge image and related data are one or more data types. If the payload type (Payload_Type) is defined as 14 to represent the knowledge image, then there is no need to use the knowledge image indication (L).

[0109] Knowledge image extension (Lx): 1 bit, used to indicate whether the AVS3 extension header needs to be used when L is 1, that is, whether to further indicate the relevant information of the AVS3 knowledge image of the payload.

[0110] Extension indication (X): 1 bit, used to indicate whether other AVS3 extension headers are used. For example, when the value is 0, it indicates that the AVS3 extension header is not used; when the value is 1, it indicates that other AVS3 extension headers are used.

[0111] The extension indication (X) of the AVS3 general header is different from the X field in the RTP header, that is, the required extension information content and purpose indicated by the two Xs are different.

[0112] In one embodiment, when the value of Payload_Type is 12 or 13, the AVS3 extension header is required to further indicate information, but if the value of X is 0, the AVS3 extension header may not be used. In other cases where the AVS3 extension header can be extended, if the value of X is 0, it also indicates that the AVS3 extension header may not be used.

[0113] In one embodiment, the knowledge image extension (Lx) and the extension indication (X) can be combined and the extension indication (X) can be directly used. In this usage scenario, the length of the extension indication is at least 2 bits. For example, when the value is set to 2, it means that the extended AVS3 extension header further indicates the knowledge image related format information.

[0114] Reserved bit (R): 4 bits. When this field is 0, this field is ignored.

[0115] In one embodiment, for different packet modes, such as when the payload type is 12 or 13, other indications can be used. For example, a new indication information, namely the packet type indication (P, 2 bits), is defined in the AVS3 general header (e.g., in a reserved bit). When the value is 0, it indicates a single packet mode; when the value is 1, it indicates a combined packet mode; when the value is 2, it indicates a segmented packet mode; and when the value is 3, it indicates a homogeneous packet mode. With this indication method, the length of the AVS3 general header can be further reduced. An example structure diagram of the streamlined AVS3 general header is shown in Figure 10.

[0116] In one embodiment, the RTP packet can select a variety of packetization modes according to the length of the data unit, media processing, and transmission requirements, including: single packetization mode, combined packetization mode, segmented packetization mode, and homogeneous packetization mode.

[0117] The embodiments of the present disclosure provide a single-packet method for encapsulating an AVS3 video encoding stream. The single-packet method is mainly used for situations where the data unit size of the AVS3 encoding stream is large or other situations where a separate packet must be used.

[0118] FIG11 is a schematic diagram of the encapsulation of an RTP packet in a single packet mode according to an embodiment of the present disclosure. In this embodiment, the AVS3 payload contains only one data unit, such as a sequence header, a user data, or slice data of an image frame. Due to the simple structure, if the AVS3 general header indicates the data type of the data unit, the AVS3 payload contains an AVS3 general header indicating the AVS3 video encoding information, and does not contain an AVS3 extension header. The Payload_Type value range of the AVS3 payload in the RTP packet is 1 to 11, that is, when the value of Payload_Type is within the range, it can be identified that the RTP packet adopts a single packet mode, as shown in FIG11 .

[0119] In one embodiment, if a single packetization method indicates the method of carrying AVS3 video encoding stream data in an RTP packet through a packet type indicator (P), the method still needs to include an AVS3 extension header, and an example structure is shown in FIG12 .

[0120] The AVS3 extension header further indicates the data characteristics in the AVS3 payload in a single packet mode. An example of the extension header is shown in Figure 13.

[0121] Among them, the payload type (Type): 5 bits, used to indicate the type of data carried in the AVS3 payload, which is the same as the type in Payload_Type, and the value range is all values ​​except 12 and 13.

[0122] Reserved bit (R): 3 bits. When this field is 0, this field is ignored.

[0123] The single packet method is mainly used for the case where the data unit of the AVS3 video encoding stream has a large amount of data. If an RTP packet carries two data units, it will exceed the maximum limit of the RTP packet. In this case, the single packet method can only be used to encapsulate the current data unit.

[0124] In the single packet method, packets are transmitted in the decoding (encoding) order.

[0125] FIG14 is a flow chart of encapsulating an AVS3 video encoding stream in a single packet according to an embodiment of the present disclosure. As shown in FIG14 , the flow includes the following steps:

[0126] Step S1401, the RTP packet module obtains the length of the data unit;

[0127] Step S1402, determining the packet type of the RTP packet and the data unit to be encapsulated according to the length of the data unit;

[0128] Step S1403: Encapsulate the data unit to be encapsulated in an RTP packet in a single packet manner, and fill / update the indication information of the AVS3 general header.

[0129] The single-packet mode includes any of the following:

[0130] When the difference between the AVS3 video encoding data payload of the RTP packet and the data volume of a data unit to be currently packetized is greater than or equal to zero and less than the minimum data unit length, the current data unit to be packetized is separately encapsulated in an RTP packet, and the indication information of the AVS3 general header is filled / updated; wherein, the AVS3 video encoding data payload is the data volume after deducting the RTP header length and the AVS3 general header length from the length limit of the RTP packet payload data.

[0131] When the sum of the data volume of a data unit to be encapsulated and the data volume of an adjacent data unit is greater than the AVS3 video encoding data payload of the current RTP packet, the data unit is encapsulated separately in an RTP packet and the indication information of the AVS3 general header is filled / updated.

[0132] For example, if the AVS3 video encoding data payload of the current RTP packet is K bytes, and the length of the current data unit to be encapsulated (any data unit in the codestream) is J bytes, if KJ is greater than or equal to 0 and less than 8 (or the minimum data unit length in the AVS3 video encoding codestream); or if KJ is greater than 8, the length (L bytes) of the next data unit to be encapsulated (n+1th) is obtained in the codestream order, and the sum of the lengths of the two data units to be encapsulated, J+L, is greater than K, then a single packet is used for this data unit. The AVS3 video encoding data payload in the RTP packet is the length limit minus the length of the RTP header and AVS3 general header, and K, J, and L are integers greater than 0.

[0133] In one embodiment, based on the requirements of media processing and transmission, when the length of the first data unit to be encapsulated is small, the data units following it in the code stream sequence may not be judged, and the data unit may be directly encapsulated in an RTP packet using a single packet method.

[0134] In one embodiment, if a data unit is required for transmission, media processing, etc., even if the length of the data unit is significantly smaller than the AVS3 video encoding data payload, it can be combined and encapsulated with other data units, but it can still be encapsulated separately in an RTP packet using a single packet method.

[0135] The embodiment of the present disclosure provides a combined packetization method for encapsulating an AVS3 video encoding stream. FIG15 is a schematic diagram of the encapsulation of an RTP packet according to the combined packetization method of the embodiment of the present disclosure. In this method, the AVS3 payload contains two or more data units. When the length of the RTP packet does not exceed the length limit, the combined packetization method can be applied to any data type that meets the encapsulation rules in the order of the AVS3 video encoding stream. For example: encapsulating multiple data units with data types as video data extensions in the RTP packet, or encapsulating an image frame's image header and one or more corresponding slice data of the image frame, or encapsulating multiple slice data containing an image frame, etc. The data type Payload_Type of the AVS3 payload in the RTP packet has a value range of 12, or the packet type indication (P) has a value of 1.

[0136] In the case of combined packets, the AVS3 payload header consists of an AVS3 general header and an AVS3 extended header. The AVS3 extended header further indicates the data characteristics of the AVS3 payload in the combined packet mode. The structure of the AVS3 general header is the same as that of the AVS3 general header in the above embodiment, as shown in Figure 9. An example diagram of the AVS3 extended header is shown in Figure 16. The extended header information includes:

[0137] Reference frame image indication (Re): 2 bits, used to indicate whether the data unit actually contained in the AVS3 payload is related to the reference image frame, such as the parameter header of the I frame, the slice data of the I frame, the knowledge image, etc.

[0138] Key parameter indication (K): 1 bit, indicating whether the data unit actually contained in the AVS3 payload contains the parameter set required for decoding, such as the sequence header parameter.

[0139] Heterogeneous Data Indicator (H): 1 bit, used to indicate whether the data units contained in the AVS3 payload are of different data types, for example, the aggregation of user data or video data extension data, image header data and slice data of an image frame. If the heterogeneous data value is 0, it indicates that the RTP payload aggregates multiple slice data of a frame.

[0140] Slice data indication (P): 1 bit, used to indicate whether the data unit actually contained in the AVS3 payload has slice data; if it is directly indicated by heterogeneous data indication (H), this indication information may not be used.

[0141] Image frame number (FN): 2 bits, used to indicate whether the data unit actually contained in the AVS3 payload contains the image header and complete image data of multiple frames.

[0142] In one embodiment, each frame of image has a timestamp, and multiple frames of image are packaged in an RTP packet. Based on the number of image frames, the time compensation of each frame of image can be calculated or expressed to obtain the timestamps of all image frames in the RTP packet. For example, in the RTP packet, the timestamp in the RTP header is the timestamp of the first image frame in the RTP packet, and then starting from the second frame of image in the RTP packet, time compensation information (usually 8 bits in length) can be added to the front of each frame of image header (except the first frame of the RTP packet); or, extended information of time compensation (8 bits in length, which is FN-1 times) can be added after the AVS3 extended header to describe the time compensation of each frame of image in all FN-1 frames after the first frame of image in the RTP packet in sequence.

[0143] Reserved bit (R), 2 bits. When this field is 0, this field is ignored.

[0144] In one embodiment, in this packetization method, if multiple image frames are packetized, the data of the packetized image frames are complete data, that is, the complete data of multiple image frames and the partial data of one or more image frames (which may contain multiple data units) are usually not placed in a combined packet.

[0145] FIG17 is a flow chart of encapsulating an AVS3 video encoding stream in a combined packet manner according to an embodiment of the present disclosure. As shown in FIG17 , the flow includes the following steps:

[0146] Step S1701: The RTP encapsulation module obtains the length of the data unit, and determines the packet type of the RTP packet and the data unit to be encapsulated according to the length of the data unit.

[0147] Step S1702: encapsulate multiple adjacent data units into the same RTP packet, and fill / update the indication information in the AVS3 general header and the extended header.

[0148] Among them, multiple data units are encapsulated into the same RTP packet when the following conditions are met:

[0149] If the difference between the AVS3 video encoding data payload of the RTP packet and the sum of the data amounts of multiple adjacent data units to be encapsulated is equal to 0 or less than the minimum data unit length, these data units are encapsulated into the same RTP packet. The AVS3 video encoding data payload of the RTP packet is the length limit of the RTP packet payload data minus the RTP header (length), AVS3 general header (length), and AVS3 extension header (length).

[0150] For example: the AVS3 video encoding data payload of the current RTP packet is H bytes. At this time, the AVS3 video encoding data payload of the current RTP packet is the length limit of the RTP packet payload data minus the RTP header and the AVS3 general header. The length of the current data unit to be packaged (nth) is W bytes. If W is at least greater than 8 or greater than the minimum data unit length in the AVS3 video encoding code stream, the length of the next (n+1th) data unit to be encapsulated is obtained in the code stream order. For example, the length of the n+1th data unit to be encapsulated is V bytes, then the lengths of the nth and n+1th data units to be packaged are added together W+V, and the AVS3 video encoding data payload of the RTP packet is H' bytes. At this time, the AVS3 video encoding data payload of the RTP packet is the length limit minus the RTP header, AVS3 general header and AVS3 extension header. If H'-(W+V) is less than 8 or less than the minimum data unit length in the AVS3 video encoding stream and greater than or equal to 0, it is determined that the RTP packet adopts the combined packetization method and encapsulates the above two data units into the same RTP packet.

[0151] If H'-(W+V) is still greater than (or the minimum data unit length in the AVS3 video encoding stream), it is determined that the RTP packet adopts the combined packetization mode, and the next i (greater than or equal to 1) data units to be encapsulated are continuously obtained in the order of the code stream until the sum of the lengths of all the obtained data units to be encapsulated (a total of i+2) from the AVS3 video encoding data payload of the RTP packet (H' bytes) minus the lengths of all the obtained data units to be encapsulated is less than 8 (or the minimum data unit length in the AVS3 video encoding stream) and greater than or equal to 0, then the above i+2 two data units are encapsulated into the same RTP packet; or, if the sum of the lengths of the i+2 data units to be encapsulated from the AVS3 video encoding data payload of the current RTP packet (H' bytes) minus the lengths of the i+2 data units to be encapsulated is still greater than 8 (or the minimum data unit length in the AVS3 video encoding stream), but the sum of the lengths after adding one data unit (the n+2+ith) is greater than H', then the above i+2 two data units are encapsulated into the same RTP packet;

[0152] In one embodiment, based on the requirements of media processing and transmission, when the sum of the lengths of multiple data units to be encapsulated is still small, the RTP packet may continue to encapsulate other data units, or may not determine the data units that follow in the code stream order and directly encapsulate multiple data units to be encapsulated into the same RTP packet.

[0153] In one embodiment, the association between data units is considered when combining packets. This improves media processing capabilities and reduces the overhead caused by packet loss of associated data units. If two adjacent data units are completely unrelated, or if the latter data unit has a stronger association with the following data unit, the latter data unit and the following data unit can be encapsulated into the next RTP packet.

[0154] In one embodiment, in order to further indicate the decoding order of each data unit in the combined packet, a decoding order indicator (DON) is added before each data unit start code in the AVS3 payload, and the length of the indicator information is at least 4 bits.

[0155] In one embodiment, in a combined packet, to further increase the AVS3 encoded stream data carried by the AVS3 payload, the prefix of the start code can be removed, retaining only the start code value. For example, if the intra-frame prediction picture header is "0x000001B3," the prefix is ​​"0x000001," and the start code value is "B3." If the start code prefix is ​​removed, to better distinguish each data unit in an RTP packet, a data unit length indicator (Size) must be added to the front of each data unit. This indicator must be at least 8 bits long, as shown in Figure 18.

[0156] An embodiment of the present disclosure provides a segmented packetization method for encapsulating an AVS3 video encoding stream. FIG19 is a schematic diagram of the encapsulation of an RTP packet using the segmented packetization method according to an embodiment of the present disclosure. In this method, the amount of data in a data unit in the AVS3 video encoding stream is greater than the length limit of the payload data of an RTP packet. In this case, the single data unit needs to be split into two or more segments, and then each segment needs to be individually packaged into two or more RTP packets.

[0157] According to the structure of the AVS3 video encoding code stream, the segmented packetization method is suitable for situations where the amount of data of a single slice in an image frame (such as an I frame) is large.

[0158] In the case of segmented packetization, the AVS3 payload header includes an AVS3 general header and an AVS3 extended header. The AVS3 extended header further indicates the data characteristics of the coding unit in the AVS3 payload in the segmented packetization mode. The structure of the AVS3 general header is the same as that in the above embodiment, as shown in FIG9 . An example diagram of the extended header is shown in FIG20 :

[0159] The extended header information includes:

[0160] Segment type (Type): 5 bits, used to indicate the type of segment data carried in the AVS3 payload, the same as the type in Payload_Type, and the value range is all values ​​except 12 and 13.

[0161] Segment start (S): 1 bit, used to indicate whether the segment data carried in the AVS3 payload contains the start segment in the slice data

[0162] End of segment (E): 1 bit, used to indicate whether the segment data carried in the AVS3 payload includes the end segment in the slice data.

[0163] Reserved (R): 1 bit. When this field is 0, this field is ignored.

[0164] In the segmented packetization method, the segmentation of the slice data is performed according to the code stream sequence of the specific coded data and then packaged. FIG21 is a flow chart of encapsulating the AVS3 video coding code stream in the segmented packetization method according to an embodiment of the present disclosure. As shown in FIG21, the packetization process includes the following steps:

[0165] Step S2101: The RTP encapsulation module obtains the length of the data unit to be encapsulated, and determines the packet type of the RTP packet and the data unit to be encapsulated according to the length of the data unit to be encapsulated.

[0166] Step S2102: Encapsulate the segmented data of the above data unit into an RTP packet, and fill / update the indication information in the AVS3 general header and the extended header.

[0167] When the length of the data unit to be encapsulated meets the following conditions, the segmented data of the data unit is encapsulated into an RTP packet:

[0168] When the AVS3 video-encoded data payload of an RTP packet is less than the length of the data unit to be encapsulated, the segmented data (partial data) of the encapsulated data unit is encapsulated into an RTP packet using segmented packetization. The AVS3 video-encoded data payload of an RTP packet is the length limit of the RTP packet payload data minus the RTP header, AVS3 general header, and AVS3 extension header.

[0169] For example: the AVS3 video encoding data payload of the current RTP packet is Q bytes, which is the length limit of the RTP packet payload data minus the RTP header and AVS3 general header. The length of the current data unit to be packaged is R bytes. If R is greater than Q, the segmented packetization method is adopted to split and select part of the data unit (i.e., segmented data) in the code stream order and encapsulate it into an RTP packet. The data length of this part of the data is less than or equal to Q minus the length of the AVS3 extension header; at this time, the AVS3 video encoding data payload of the RTP is Q' bytes, which is the length limit of the RTP packet payload data minus the data amount of the RTP header, AVS3 general header, and AVS3 extension header;

[0170] In one embodiment, if the minimum coding unit (CU) of slice data in an AVS3 video encoding stream can be distinguished based on the code stream order, the data units of the slice data type can be segmented according to the minimum coding unit (CU). The segmentation process is the same as step S2101 above, ensuring that the length of one or more minimum coding units (CUs) in the packetized slice data is less than the AVS3 video encoding data length limit of the RTP packet.

[0171] In one embodiment, in the segmented packetization method, segments of two different data units cannot be encapsulated into one RTP packet.

[0172] The embodiment of the present disclosure further provides a homogeneous packaging method, in which data units or segments of data units of the same type are encapsulated into one RTP packet.

[0173] In this packet mode, the value range of the payload type (Payload_Type) is 1 to 11. At the same time, the reserved bits in the AVS3 general header or the new AVS3 extension header are used to add the following indication information:

[0174] Fragmentation indicator (F): 1 bit, used to indicate whether the RTP packet contains fragments of fragment data. For example, a value of 0 indicates no fragments; a value of 1 indicates that fragments are included.

[0175] Start of segment (S): 1 bit, used to indicate whether the segment data carried in the AVS3 payload includes the start segment in the slice data.

[0176] End of segment (E): 1 bit, used to indicate whether the segment data carried in the AVS3 payload includes the end segment in the slice data.

[0177] When the segmentation indicator is 0, it means that there is no data unit segment in the RTP packet, and S and E can be ignored.

[0178] In one embodiment, the homogeneous packetization method can support encapsulating two or more data units and data unit segments of the same type in one RTP packet. For example, for data units A and B of a slice of I frame data, the RTP packet can include segmented data of data unit A (including the end of the segment) and segmented data of data unit B (including the start of the segment). In this case, the segment indicator F takes a value of 1, and the start of the segment S and the end of the segment E are both 1.

[0179] In one embodiment, if a homogeneous packetization method is used, there is no need to use both single and segmented packetization methods simultaneously. Furthermore, if a combined packetization method is used, only multiple data units of different types are packaged, such as when user data and video data extension data are packaged into a single RTP packet. Data units of the same type use the homogeneous packetization method. Furthermore, data units whose data type is video data extension data can be considered to be different types of data units due to the different video extension identifiers (extension_id) in the AVS3 video encoding stream. Therefore, the homogeneous packetization method is not used for RTP packetization, and the combined packetization method can be used instead.

[0180] FIG22 is a flow chart of encapsulating an AVS3 video encoding stream in a homogeneous packaging manner according to an embodiment of the present disclosure. As shown in FIG22 , the packaging process includes the following steps:

[0181] In step S2201, the RTP encapsulation module obtains the length of the data unit to be encapsulated, determines the encapsulation mode of the RTP packet as the same type of encapsulation according to the length of the data unit to be encapsulated, and determines the data unit to be encapsulated.

[0182] Step S2202: Encapsulate the one or more data units to be encapsulated and / or partial data of the one or more data units to be encapsulated into an RTP packet, and fill / update corresponding indication information in the AVS3 general header and / or the AVS3 extended header.

[0183] Encapsulate one or more data units to be encapsulated and / or part of one or more data units to be encapsulated into one RTP packet when the following conditions are met:

[0184] The AVS3 video encoding data payload (Z bytes) of the current RTP packet is the length limit of the RTP packet minus the RTP header and the AVS3 general header. The length of the current data unit to be packaged is R bytes. If R is greater than Z, the same type of packaging is used. If R is less than Z, and the next one or more data units to be packaged have the same data type as the current data unit, the same type of packaging is used.

[0185] If R is greater than Z, the data unit to be packaged is split into segments smaller than Z; or, if the sum of the lengths of one or more data units to be packaged is equal to or less than Z; or, if the sum of the lengths of one or more data units to be packaged and the lengths of partial data of one or more data units to be packaged is equal to or less than Z, go to step S2202.

[0186] The present disclosure also provides a method for an RTP receiving end to perform reverse decapsulation and decoding after receiving an RTP packet. The method is applied to the RTP receiving end 20 in the network architecture of FIG2 . FIG23 is a flow chart of the RTP packet decoding method according to the present disclosure embodiment. As shown in FIG23 , the basic process is as follows:

[0187] Step S2301: The RTP receiving end stores the received RTP packets in sequence into the buffer of the RTP decapsulation module;

[0188] Step S2302: The RTP decapsulation module decapsulates the data packets in the order of their RTP sequence numbers. If the RTP packet encapsulates one or more data units, it is directly passed to the AVS3 video decoding module. If the RTP packet encapsulates a fragment of a data unit, all fragments of the data unit are concatenated in the order of decapsulation and then passed to the AVS3 video decoding module.

[0189] Step S2303: The AVS3 video decoding module decodes the AVS3 video encoding data on the data units depacketized in the depacketization order to generate a playable AVS3 video sequence.

[0190] In the method and device for AVS3 video encoding transmission based on RTP protocol proposed in the above embodiments of the present disclosure, the RTP protocol can be extended with reference to the AVS3 video encoding features to provide a flexible packet format for the AVS3 video encoding stream. The data is segmented by combining the AVS3 general header and the optional extended header with different start codes in the AVS3 video encoding stream. According to the block / segment size of each frame of video, the size limit of the RTP packet payload data volume is fully considered to perform single packetization, combined packetization, segmented packetization and similar packetization, thereby improving the efficiency of packetization and realizing real-time transmission.

[0191] An embodiment of the present disclosure further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any one of the above method embodiments when run.

[0192] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0193] An embodiment of the present disclosure further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0194] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0195] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail here.

[0196] Obviously, those skilled in the art should understand that the modules or steps of the present disclosure described above can be implemented using a general-purpose computing device, they can be concentrated on a single computing device, or distributed across a network composed of multiple computing devices, they can be implemented using program code executable by the computing device, and thus, they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be performed in a different order than herein, or they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. Thus, the present disclosure is not limited to any particular combination of hardware and software.

[0197] The foregoing description is merely a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Those skilled in the art will readily appreciate that various modifications and variations of the present disclosure are possible. Any modifications, equivalent substitutions, or improvements made within the principles of the present disclosure shall be included within the scope of protection of the present disclosure.

Claims

1. A method for transmitting an AVS3 video encoding stream, comprising: Encapsulating the AVS3 video encoding stream into multiple real-time transport protocol RTP packets, wherein each of the RTP packets is provided with an AVS3 payload header and an AVS3 payload; The multiple RTP packets are transmitted in sequence in the order of encapsulation.

2. The method according to claim 1, wherein: Encapsulate the AVS3 video encoding stream into multiple RTP packets, including: According to the start code in the AVS3 video encoding code stream, the AVS3 video encoding code stream is divided into a plurality of data units; The multiple data units are encapsulated into the multiple RTP packets, wherein each RTP packet carries one data unit, multiple data units and / or partial data of one or more data units.

3. The method according to claim 2, wherein: Encapsulating the plurality of data units into the plurality of RTP packets comprises at least one of the following: A data unit is encapsulated into an RTP packet using the first encapsulation method; Using the second packetization method to packetize multiple data units of the same or different data types into one RTP packet; The third packetization method is used to encapsulate part of the data of a data unit into an RTP packet; The fourth packetization method is used to packetize all and / or part of the data of one or more data units of the same data type into one RTP packet.

4. The method according to claim 3, wherein: The AVS3 payload header includes an AVS3 general header, and the information indicated by the AVS3 general header includes at least one of the following: a discard indication, a time level, a payload type, a knowledge image indication, a knowledge image extension, an extension indication, and a packet type indication.

5. The method according to claim 3 or 4, wherein: In the RTP packet obtained by adopting the first packetization method, the AVS3 payload header of the RTP packet includes an AVS3 extension header; the information indicated by the AVS3 extension header is the payload type.

6. The method according to any one of claims 3 to 4, wherein: The first packetization method is used to encapsulate a data unit into an RTP packet, including: When the length of a data unit to be encapsulated is less than or equal to the AVS3 video encoding data payload of the RTP packet, the data unit is encapsulated separately in one of the RTP packets.

7. The method according to claim 3 or 4, wherein: In the RTP packet obtained by adopting the second packetization method, the AVS3 payload header of the RTP packet includes an AVS3 extension header; the information indicated by the AVS3 extension header includes at least one of the following: a reference frame image indication, a key parameter indication, a heterogeneous data indication, a slice data indication, and the number of image frames.

8. The method according to claim 7, wherein: The second packetization method is used to encapsulate multiple data units of the same or different data types into one RTP packet, including: When the sum of the lengths of the adjacent multiple data units to be encapsulated is less than or equal to the AVS3 video encoding of the RTP packet In the case of a large amount of data load, the multiple data units are encapsulated in one RTP packet.

9. The method according to claim 3 or 4, wherein: In the RTP packet obtained by adopting the third packetization method, the AVS3 payload header of the RTP packet includes an AVS3 extension header; the information indicated by the AVS3 extension header includes at least one of the following: the type of segment, the start of the segment, and the end of the segment.

10. The method according to claim 9, wherein: The third packetization method is used to encapsulate part of the data of a data unit into an RTP packet, including: When the length of a data unit to be encapsulated is greater than the AVS3 video encoding data payload of the RTP packet, the data unit is encapsulated into multiple RTP packets, wherein each RTP packet contains partial data of the data unit.

11. The method according to claim 3 or 4, wherein: In the RTP packet obtained by adopting the fourth packetization method, the information indicated by the AVS3 general header in the RTP packet includes at least one of the following: segment indication, the start of a segment, and the end of a segment; or, the AVS3 payload header in the RTP packet includes an AVS3 extension header, and the information indicated by the AVS3 extension header includes at least one of the following: segment indication, the start of a segment, and the end of a segment.

12. The method according to claim 3, wherein: The fourth packetization method is used to packetize all and / or part of one or more data units of the same data type into one RTP packet, including: According to the AVS3 video coding data payload of the RTP packet, at least one data unit of the same data type whose length is smaller than the AVS3 video coding data payload and / or partial data of at least one data unit are encapsulated in one of the RTP packets.

13. The method according to claim 2, wherein: The multiple data units are distinguished according to the start code or according to the indication information of the data unit length in the AVS3 load.

14. A transmission device for an AVS3 video encoding code stream, comprising: An encapsulation module, configured to encapsulate the AVS3 video encoding stream into multiple RTP packets, wherein each of the RTP packets is provided with an AVS3 payload header and an AVS3 payload; The transmission module is configured to transmit the multiple RTP packets in sequence in the order of encapsulation.

15. A computer-readable storage medium having a computer program stored therein, wherein: When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 13 are implemented.

16. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any one of claims 1 to 13 when executing the computer program.

Images