Method and device for transmitting AVS3 video coding code stream

Abstract

The embodiment of the invention provides a transmission method and device of an AVS3 video coding code stream. The method comprises the steps that an AVS3 video coding code stream is packaged into a plurality of real-time transport protocol (RTP) packets, and each RTP packet is internally provided with an AVS3 load head and an AVS3 load; and sequentially transmitting the plurality of RTP packets according to a packaging sequence. According to the invention, the AVS3 video coding code stream is packaged into a plurality of real-time transport protocol (RTP) packets, so that the AVS3 video coding code stream can be transmitted in an RTP transmission mode. Therefore, the problem that the AVS3 video coding code stream needs to support the RTP transmission mode as soon as possible can be solved, meanwhile, the AVS3 load head is arranged in the RTP packet and can be used for indicating related feature information and key information in the AVS3 video coding code stream, and efficient and real-time transmission of the AVS3 video coding code stream is facilitated.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of communication technology, and in particular, to a method and device for transmitting an AVS3 video encoding code stream. Background Art

[0002] The video encoding stream of the third-generation audio and video codec technology standard (AVS3) is usually transmitted using TS (Transport Stream) and DASH (Dynamic Adaptive Streaming over HTTP). However, with the development of real-time interactive audio and video technology, AVS3 needs to support RTP (Real-time Transport Protocol) transmission as soon as possible to meet the different transmission requirements of different services. Summary of the invention

[0003] The embodiment of the present invention provides a method and device for transmitting an AVS3 video encoding stream, so as to at least solve the problem in the related art that the AVS3 video encoding stream needs to support the RTP transmission mode as soon as possible.

[0004] According to one embodiment of the present invention, a method for transmitting an AVS3 video encoding stream is provided, comprising: encapsulating the AVS3 video encoding stream into multiple real-time transport protocol RTP packets, wherein each of the RTP packets is provided with an AVS3 payload header and an AVS3 payload; and transmitting the multiple RTP packets in sequence in the order of encapsulation.

[0005] According to another embodiment of the present invention, a transmission device for an AVS3 video encoding stream is provided, comprising: an encapsulation module, used to encapsulate the AVS3 video encoding stream into multiple RTP packets, wherein each of the RTP packets is provided with an AVS3 payload header and an AVS3 payload; and a transmission module, used to transmit the multiple RTP packets in sequence in the order of encapsulation.

[0006] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when run.

[0007] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0008] Through the above-mentioned embodiment of the present invention, the AVS3 video encoding stream is encapsulated into multiple real-time transport protocol RTP packets, so that the AVS3 video encoding stream can be transmitted in the RTP transmission mode. Therefore, the problem that the AVS3 video encoding stream needs to support the RTP transmission mode as soon as possible can be solved. At the same time, the AVS3 payload header is set in the RTP packet, which can be used to indicate the relevant feature information and key information in the AVS3 video encoding stream, which is helpful for the efficient and real-time transmission of the AVS3 video encoding stream. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 It is a hardware structure block diagram of a computer terminal for running the AVS3 video encoding code stream transmission method according to an embodiment of the present invention;

[0010] Figure 2 is a network architecture diagram of a transmission system of an AVS3 video encoding code stream according to an embodiment of the present invention;

[0011] Figure 3 is a flow chart of a method for transmitting an AVS3 video encoding code stream according to an embodiment of the present invention;

[0012] Figure 4 is a structural block diagram of a transmission device for an AVS3 video encoding code stream according to an embodiment of the present invention;

[0013] Figure 5 is a schematic diagram of the structure of an AVS3 video encoding code stream according to an embodiment of the present invention;

[0014] Figure 6 is a flow chart of a method for encapsulating and transmitting an AVS3 video encoding code stream based on RTP according to an embodiment of the present invention;

[0015] Figure 7 is a schematic diagram of an RTP header according to an embodiment of the present invention;

[0016] Figure 8 is a schematic diagram of an AVS3 load head according to an embodiment of the present invention;

[0017] Fig. 9 is a schematic diagram of an AVS3 universal header according to an embodiment of the present invention;

[0018] Fig.10 is a schematic diagram of a simplified AVS3 universal header according to an embodiment of the present invention;

[0019] Fig.11 is a schematic diagram of encapsulation of an RTP packet in a single packetization mode according to an embodiment of the present invention;

[0020] Fig.12 is a schematic diagram of an RTP packet including an extension header in a single packetization mode according to an embodiment of the present invention;

[0021] Fig.13 is a schematic diagram of an extended header of an RTP packet in a single packet mode according to an embodiment of the present invention;

[0022] Fig.14 is a flow chart of encapsulating an AVS3 video encoding stream in a single packet manner according to an embodiment of the present invention;

[0023] Fig.15 is a schematic diagram of encapsulation of an RTP packet in a combined packet manner according to an embodiment of the present invention;

[0024] Fig.16 is a schematic diagram of an extended header of an RTP packet in a combined packet manner according to an embodiment of the present invention;

[0025] Fig.17 is a flow chart of encapsulating an AVS3 video encoding stream in a combined packet manner according to an embodiment of the present invention;

[0026] Fig.18 is a schematic diagram of adding data unit indication length information according to an embodiment of the present invention;

[0027] Fig.19 is a schematic diagram of RTP packet encapsulation in a segmented packet manner according to an embodiment of the present invention;

[0028] Fig. 20 is a schematic diagram of an AVS3 extension header in a segmented packetization method according to an embodiment of the present invention;

[0029] Fig.21 is a flow chart of encapsulating an AVS3 video encoding stream in a segmented packet manner according to an embodiment of the present invention;

[0030] Fig. 22 is a flow chart of encapsulating an AVS3 video encoding stream in a homogeneous packaging manner according to an embodiment of the present invention;

[0031] Fig.23 is a flowchart of a method for decoding an RTP packet according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0034] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a computer terminal as an example, Figure 1 1 is a hardware structure block diagram of a computer terminal for running the AVS3 video encoding code stream transmission method according to an embodiment of the present invention. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the above-mentioned computer terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0035] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the transmission method of the AVS3 video encoding code stream in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0036] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0037] The present application embodiment can be run on Figure 2 In the network architecture shown in Figure 2As shown, the network architecture includes: an RTP transmitter 10 and an RTP receiver 20. RTP is a commonly used real-time multimedia transmission protocol, which can provide end-to-end real-time transmission services for various multimedia data such as voice and images that need to be transmitted in real time on the IP (Internet Protocol) network. At the same time, RTP is also widely used in streaming media related communications and entertainment.

[0038] like Figure 2 As shown, the RTP sending end 10 includes: an AVS3 video encoding module 11, an RTP packaging module 12 and an RTP transmission module 13.

[0039] The AVS3 video encoding module 11, namely the AVS3 video encoder, is used to encode the original video according to the AVS3 standard to generate an AVS3 video encoding stream. The original video includes but is not limited to one of the following: video captured by a camera and video generated by a computer.

[0040] The RTP packetization module 12 is used to encapsulate the generated AVS3 video encoding stream into RTP packets, wherein the AVS3 video encoding stream needs to be split according to a certain length (i.e., the data volume) to ensure that the length of each RTP packet (i.e., the size of the RTP packet) does not exceed the limit.

[0041] The RTP transmission module 13 is used to transmit the RTP packet carrying the AVS3 video encoding stream to the RTP receiving end 20.

[0042] like Figure 2 As shown, the RTP receiving end 20 includes: an RTP receiving module 21, an RTP decapsulation module 22 and an AVS3 video decoding module 23.

[0043] The RTP receiving module 21 is used to receive the RTP packet with the AVS3 video encoding stream transmitted by the RTP transmission module 13.

[0044] The RTP depacketizing module 22 is used to depacketize the RTP packets received by the RTP receiving module 21 and extract the AVS3 video encoding stream data loaded in the RTP packets.

[0045] The AVS3 video decoding module 23 , namely the AVS3 video decoder, is used to decode the AVS3 video encoding stream data extracted by the RTP depacketization module 22 .

[0046] In this embodiment, a method for transmitting an AVS3 video encoding code stream running on the above-mentioned computer terminal or network architecture is provided. Figure 3 is a flow chart of a method for transmitting an AVS3 video encoding code stream according to an embodiment of the present invention, such as Figure 3 As shown, the process includes the following steps:

[0047] Step S302, encapsulating the AVS3 video encoding code stream into multiple real-time transport protocol RTP packets, wherein each of the RTP packets is provided with an AVS3 payload header and an AVS3 payload;

[0048] In step S302 of this embodiment, the AVS3 video encoding stream is encapsulated into multiple RTP data packets, including: dividing the AVS3 video encoding stream into multiple data units according to the start code in the AVS3 video encoding stream; encapsulating the multiple data units into the multiple RTP packets, wherein each RTP packet carries one data unit, multiple data units and / or partial data of one or more data units.

[0049] In one embodiment, encapsulating the multiple data units into the multiple RTP packets includes at least one of the following: using a first packetization method to packetize a data unit into an RTP packet; using a second packetization method to packetize multiple data units of the same or different data types into an RTP packet; using a third packetization method to packetize part of the data of a data unit into an RTP packet; using a fourth packetization method to packetize all and / or part of the data of one or more data units of the same data type into an RTP packet.

[0050] In one embodiment, the AVS3 payload header includes an AVS3 general header, and the information indicated by the AVS3 general header includes at least one of the following: discard indication, time level, payload type, knowledge image indication, knowledge image extension, extension indication, and packet type indication.

[0051] In one embodiment, a data unit is packaged into an RTP packet using a first packetization method, including: when the length of a data unit to be encapsulated (i.e., the size of the data unit) is less than or equal to the AVS3 video encoding data payload of the RTP packet, the data unit is individually encapsulated in one of the RTP packets.

[0052] The video encoding streams generated by different encoders in the RTP protocol (such as H.264, VP8, etc.) need to be extended to indicate some basic feature information and key information in the encoding streams, so as to meet the efficient and real-time transmission requirements of different encoding stream characteristics.

[0053] In one embodiment, in the RTP packet obtained by adopting the second packetization method, the AVS3 payload header of the RTP packet includes an AVS3 extension header; the information indicated by the AVS3 extension header includes at least one of the following: reference frame image indication, key parameter indication, heterogeneous data indication, slice data indication and the number of image frames.

[0054] In one embodiment, a second packetization method is used to packetize multiple data units of the same or different data types into one RTP packet, including: when the sum of the lengths of the multiple adjacent data units to be encapsulated is less than or equal to the AVS3 video encoding data payload of the RTP packet, the multiple data units are encapsulated in one RTP packet.

[0055] In one embodiment, in the RTP packet obtained by adopting the third packetization method, the AVS3 payload header of the RTP packet includes an AVS3 extension header; the information indicated by the AVS3 extension header includes at least one of the following: the type of segment, the start of the segment, and the end of the segment.

[0056] In one embodiment, a third packetization method is used to packetize part of the data of a data unit into an RTP packet, including: when the length of a data unit to be encapsulated is greater than the AVS3 video encoding data payload of the RTP packet, encapsulating the data unit into multiple RTP packets, wherein each RTP packet contains part of the data of the data unit.

[0057] In one embodiment, in the RTP packet obtained by adopting the fourth packetization method, the information indicated by the AVS3 general header in the RTP packet includes at least one of the following: segment indication, the start of a segment, and the end of a segment; or, the AVS3 payload header in the RTP packet includes an AVS3 extension header, and the information indicated by the AVS3 extension header includes at least one of the following: segment indication, the start of a segment, and the end of a segment.

[0058] In one embodiment, a fourth packetization method is used to packetize all and / or part of the data of one or more data units of the same data type into one RTP packet, including: according to the AVS3 video encoding data payload of the RTP packet, encapsulating at least one data unit of the same data type whose length is smaller than the AVS3 video encoding data payload and / or part of the data of at least one data unit in one of the RTP packets.

[0059] In one embodiment, the multiple data units are distinguished based on the start code or based on the indication information of the data unit length in the AVS3 payload.

[0060] Step S304, transmitting the multiple RTP packets in the order of encapsulation, that is, transmitting them in the order of AVS3 video encoding stream.

[0061] Through the above steps, the AVS3 video encoding stream is encapsulated into multiple real-time transport protocol RTP packets, so that the AVS3 video encoding stream can be transmitted in RTP transmission mode. Therefore, the problem that the AVS3 video encoding stream needs to support the RTP transmission mode as soon as possible can be solved. At the same time, the AVS3 payload header is set in the RTP packet, which can be used to indicate the relevant feature information and key information in the AVS3 video encoding stream, which is helpful for the efficient and real-time transmission of the AVS3 video encoding stream.

[0062] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0063] In this embodiment, a transmission device for an AVS3 video encoding code stream is also provided, and the device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0064] Figure 4 is a structural block diagram of a transmission device for an AVS3 video encoding code stream according to an embodiment of the present invention, such as Figure 4 As shown, the device comprises:

[0065] The encapsulation module 41 is used to encapsulate the AVS3 video encoding code stream into multiple RTP packets, wherein each of the RTP packets is provided with an AVS3 payload header; the encapsulation module 41 is functionally equivalent to the RTP packetization module 12 in the above embodiment.

[0066] The transmission module 42 is used to transmit the multiple RTP packets in sequence according to the encapsulated timestamps. The transmission module 42 is functionally equivalent to the RTP transmission module 13 in the above embodiment.

[0067] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0068] The present invention proposes a method for transmitting AVS3 video encoding stream based on RTP protocol, which is applied to Figure 2 The RTP sending end 10 in the network architecture includes the following steps in the method:

[0069] Step S1, processing the captured original video through the AVS3 encoder to obtain an AVS3 video encoding stream to be transmitted, that is, an AVS3 video encoding bit stream, wherein the original video can be captured by a camera.

[0070] Figure 5 is a schematic diagram of the structure of an AVS3 video encoding code stream according to an embodiment of the present invention. Figure 5 As shown, the AVS3 video encoding code stream includes one or more (AVS3) video sequences; a video sequence includes an AVS3 video sequence header, one or more user / extension data, one or more intra-frame prediction images, and one or more inter-frame prediction images.

[0071] The intra-frame prediction image includes an intra-frame prediction image header, one or more user / extension data, and intra-frame prediction image data;

[0072] The inter-frame prediction picture includes an inter-frame prediction picture header, one or more user / extension data, and inter-frame prediction picture data.

[0073] The intra-frame prediction image data and the inter-frame prediction image data are respectively composed of one or more pieces of data (patch). Figure 5 Only intra-frame prediction image data is used as an example for display);

[0074] In one embodiment, the slice data may be further divided, and the minimum division unit is the coding unit data (CU).

[0075] Step S2, splitting the AVS3 video encoding code stream and encapsulating it into multiple RTP packets, and then transmitting it through the network.

[0076] Figure 6 : is a flow chart of an RTP-based AVS3 video encoding code stream encapsulation and transmission method according to an embodiment of the present invention, such as Figure 6 As shown, the process includes the following steps:

[0077] Step S602: The RTP transmitter splits the bitstream into multiple data units according to the start code in the AVS3 video encoding bitstream.

[0078] According to the AVS3-P2 standard, in the AVS3 video coding stream, each video sequence header, user / extended data, intra-frame prediction image header, inter-frame prediction image header, each slice data in the intra-frame prediction image data, and each slice data in the inter-frame prediction image data are respectively provided with a start code, and the data types of the data of different data types can be split by the start code, that is, the data from one start code to the next start code can be regarded as a data unit of an AVS3 video coding stream, and the data type corresponding to the data unit can be identified according to the start code of each data unit (or the start code value in the start code). The start code is composed of a start code prefix and a start code value, and the data unit can include the start code, or only include the start code value in the start code without the start code prefix.

[0079] In this embodiment, the coded images output by the AVS3 encoder are arranged in a code stream order, wherein the coded code stream order is the same as the decoded code stream order.

[0080] Step S604, using the RTP protocol to encapsulate the split AVS3 video encoding code stream, and encapsulate the split data units into one or more RTP packets according to the length limit of the data carried by a single RTP packet. According to the length of each data unit, each RTP packet can carry one data unit, multiple data units and / or one or more data unit segments (i.e., partial data of a data unit).

[0081] In this embodiment, when RTP transmits the AVS3 video encoding stream, the AVS3 video encoding stream structure must first be split into multiple data units, and then the data units must be encapsulated into multiple RTP packets based on the length of the data units and other requirements (such as media processing), and finally transmitted through the network in the form of RTP packets.

[0082] In a network system for video transmission, the MTU (Maximum Transmission Unit) size is limited, for example, the MTU of Ethernet is 1500 bytes.

[0083] For media transmission based on IP / TCP / UDP, the length of the RTP packet plus the length of the IP header (at least 20 bytes) or the UDP header (8 bytes) must not be greater than the length of the MTU (i.e., the size of the MTU), and an RTP packet contains an RTP header (12 bytes) and an RTP payload. The AVS3 video encoding stream data is actually encapsulated in the RTP payload, so the AVS3 video encoding stream data that can be loaded in an RTP packet has a length limit (i.e., a size limit). Generally, the maximum length of the RTP payload is 1460 bytes, and in a normal transmission instance, the length limit is even less than 1460 bytes. In this embodiment, the AVS3 video encoding stream is transmitted based on the RTP protocol, and the split data unit is used as the minimum unit, and then one or more data units are encapsulated into an RTP packet for transmission. If the length of a single data unit is greater than the length limit of the RTP payload, it needs to be further split into segments of data units less than the length limit to ensure that each RTP packet is less than the length limit to meet the MTU size in network transmission.

[0084] Step S606: The RTP sending end establishes a transmission channel with the RTP receiving end, and transmits the RTP packets in the order of encapsulation.

[0085] In one embodiment, step S601 and step S602 can be performed simultaneously, that is, when the RTP transmitter splits the code stream in sequence according to the start code in the AVS3 video encoding code stream, each time one or more data units are split, they are encapsulated into an RTP packet separately or combined and then directly transmitted, and the code stream is split and encapsulated into the next RTP packet. If the length of a data unit is greater than the length limit of the RTP packet payload data (i.e., the amount of data that the RTP packet can carry), the data unit is further split into two or more fragments, and each fragment is encapsulated into an RTP packet.

[0086] In one embodiment, when performing RTP packetization according to the data type and data length (i.e., data volume) corresponding to the data unit and the requirements of transmission processing, the following packetization principles shall be followed:

[0087] 1. The bitstream order (i.e., bitstream order) of AVS3 video coding is the same as the decoding order. Packaging is performed according to the bitstream order of AVS3 video coding and transmission is performed according to the packetization order. If there is no decoding order between multiple consecutive data units, they are packaged according to the bitstream order, such as user data, video data, and extended data;

[0088] 2. Data units of parameter types such as sequence header, user data, and extended data are usually small data units. If there is no special requirement, they should be combined into one RTP packet;

[0089] 3. Under the condition that the length of the data unit is less than the RTP packet length limit, the related data units should be encapsulated into one RTP packet. For example, data units belonging to the same frame of image (including image header data) can be encapsulated into one RTP packet under the condition that the length of the data unit is less than the RTP packet length limit.

[0090] 4. Image frames with smaller data length, such as P frames and B frames, can be encapsulated into one RTP packet if the length is less than the RTP packet length limit, but it must be a complete frame of image data.

[0091] In one embodiment, adjacent RTP packets that are RTP encapsulated in the order of the AVS3 video encoding stream may adopt different packaging methods. The packaging method of each RTP packet is determined based on the data length, type or other necessary requirements of one or more data units to be encapsulated.

[0092] An RTP packet includes an RTP header and an RTP payload, and the RTP payload is used to carry AVS3 video encoding stream data. In the embodiment of the present invention, basic extended information of RTP supporting AVS3 video encoding stream transmission is also provided. Figure 7 As shown, in the RTP packet encapsulating the AVS3 video payload, the RTP payload after the RTP header includes two parts, an AVS3 payload header (AVS3 payloadheader) and an AVS3 payload (AVS3 payload).

[0093] In one embodiment, the length of the AVS3 payload header is an integer multiple of 8 bits, including an AVS3 common header and an AVS3 header extension, wherein the AVS3 common header is a fixed length, for example: 8 or 16 bytes, and the AVS3 extension header is an integer multiple of 8 bits. In order to ensure that the AVS3 video encoding code stream loaded in the RTP packet carries as much data as possible, the AVS3 extension header is optional, that is, it is used when the extension indication information or other indication information (such as when Payload_Type is 12 or 13) indicates that extension is required.

[0094] The format of the RTP header is specified in IETF RFC 3550. In this embodiment, according to the characteristics of the AVS3 video encoding code stream in the RTP packet, the AVS3 payload header information is further set on the basis of the general RTP header, such as Figure 7 As shown in the figure, it is convenient for network transmission and processing by RTP receiving module and decapsulation module.

[0095] Among them, the RTP packet based on the AVS3 video payload has the following settings for the RTP header:

[0096] 1. Marker bit (M): 1 bit; used to specify the frame boundary. The marker bit of the last RTP data packet of each video frame is set to 1, and the marker bits of other RTP data packets are set to 0.

[0097] 2. Payload Type (PT): 7 bits; According to the definition of RFC 3551, the upper layer application should dynamically allocate a PT value in each RTP session (such as SDP) and maintain the mapping relationship between the PT value and the payload format. The value range of the PT value should follow the provisions of RFC 3551 (for example, 97-127).

[0098] 3. Timestamp: 32 bits; RTP timestamp is set to the sampling timestamp of the encoded content, and the clock used is 90000Hz, that is, the time unit of RTP timestamp is 1 / 90000s. Like the sequence number, the initial value of the timestamp should also be random, and the timestamps of different RTP data packets of the same frame image should be consistent. For the sequence header, user extension data, and image header that do not have time attributes, their timestamps should be consistent with the timestamps of the image data that follows them.

[0099] Other fields in the RTP header (V, P, X, CC, etc.) should comply with the provisions of IETF RFC 3550.

[0100] Figure 8 is a schematic diagram of an AVS3 load head according to an embodiment of the present invention, Fig. 9 is a schematic diagram of an AVS3 universal header according to an embodiment of the present invention, such as Fig. 9 As shown, the settings of the AVS3 general header in the AVS3 payload header include:

[0101] Discard indication (N): 1 bit, used to indicate whether the AVS3 video encoding stream data carried by the RTP packet can be discarded during transmission. If the RTP packet is discarded and does not affect the decoder decoding and content presentation, the RTP packet can be discarded. Among them, whether to discard is determined by the packetizer. If it cannot be clearly determined, it is always indicated that it cannot be discarded.

[0102] Time level (TID): 3 bits, Temporal_id, used to indicate which time domain sublayer the current RTP payload belongs to, with a value range of 0 to 7. The TID value indicates the relative importance of the RTP packet, for example, because the image frame belonging to a higher time sublayer may not be used for decoding the image frame of a lower time sublayer. The lower the TID value, the higher the importance, thus better preventing transmission losses.

[0103] Payload_Type: 5 bits, used to indicate the type of AVS3 video encoding stream data carried in the RTP payload, that is, the data type of the data unit in the stream. Its value and type description are shown in Table 1:

[0104] Table 1

[0105]

[0106] Knowledge image indication (L): 1 bit, used to indicate whether the current RTP packet contains a knowledge image. For example, when the value is 0, it indicates that the current RTP payload is an AVS3 non-knowledge image; when the value is 1, it indicates that the current RTP payload is an AVS3 knowledge image.

[0107] In one embodiment, when the payload data in the RTP packet is an AVS3 knowledge image, it may carry a knowledge image header, part or all of the knowledge image data, or a knowledge image header and knowledge image data, etc. The specific packaging method is determined by the size of the knowledge image data.

[0108] In one embodiment, the payload type (Payload_Type) can be used to indicate the knowledge image and related data, that is, the knowledge image and related data are one or more data types. For example, if the payload type (Payload_Type) is defined as 14 to represent the knowledge image, there is no need to use the knowledge image indication (L).

[0109] Knowledge image extension (Lx): 1 bit, used to indicate whether the AVS3 extension header needs to be used when L is 1, that is, whether to further indicate the relevant information of the AVS3 knowledge image of the load.

[0110] Extension indication (X): 1 bit, used to indicate whether to use other AVS3 extension headers. For example, when the value is 0, it indicates that the AVS3 extension header is not used; when the value is 1, it indicates that other AVS3 extension headers are used.

[0111] The extension indication (X) of the AVS3 general header is different from the X field in the RTP header, that is, the information content and purpose of the required extension indicated by the two Xs are different.

[0112] In one embodiment, when the value of Payload_Type is 12 or 13, the AVS3 extension header needs to be used to further indicate information, but if the value of X is 0, the AVS3 extension header may not be used. In other cases where the AVS3 extension header can be extended, if the value of X is 0, it also means that the AVS3 extension header may not be used.

[0113] In one embodiment, the knowledge image extension (Lx) and the extension indication (X) can be combined and the extension indication (X) can be directly used. In this usage scenario, the length of the extension indication is at least 2 bits. For example, when the value is set to 2, it means that the extended AVS3 extension header further indicates the knowledge image related format information.

[0114] Reserved bit (R): 4 bits. When this field is 0, this field is ignored.

[0115] In one embodiment, for different packet modes, such as when the payload type is 12 or 13, other indications can be used, for example: a new indication information is defined in the AVS3 general header (such as in the reserved bit), namely, the packet type indication (P, 2 bits), when the value is 0, it indicates a single packet mode; when the value is 1, it indicates a combined packet mode; when the value is 2, it indicates a segmented packet mode; when the value is 3, it indicates a similar packet mode. Under this indication mode, the length of the AVS3 general header can be further reduced, and the example structure diagram of the simplified AVS3 general header is shown in FIG. Fig.10 shown.

[0116] In one embodiment, according to the length of the data unit, media processing and transmission requirements, the RTP packet can select a variety of packetization methods, including: a single packetization method, a combined packetization method, a segmented packetization method, and a homogeneous packetization method.

[0117] The embodiment of the present invention provides a single-packet method for encapsulating an AVS3 video encoding stream. The single-packet method is mainly used for situations where the data unit size of the AVS3 encoding stream is large or other situations where a separate packet must be used.

[0118] Fig.11 It is a schematic diagram of the encapsulation of an RTP packet in a single packet mode according to an embodiment of the present invention. In this embodiment, the AVS3 payload contains only one data unit, such as a sequence header, a user data, or a slice data of an image frame. Due to the simple structure, if the AVS3 general header indicates the data type of the data unit, the AVS3 payload contains an AVS3 general header indicating the AVS3 video encoding information, and does not contain an AVS3 extension header. The Payload_Type value range of the AVS3 payload in the RTP packet is 1 to 11, that is, when the value of Payload_Type is within the range, it can be identified that the RTP packet adopts a single packet mode, such as Fig.11 shown.

[0119] In one embodiment, if a single packet mode indicates the mode of RTP packet carrying AVS3 video encoding stream data through packet type indication (P), then the mode still needs to include AVS3 extension header, and the example structure is as follows: Fig.12 shown.

[0120] The AVS3 extension header further indicates the data characteristics in the AVS3 payload in a single packet mode. Fig.13 As shown;

[0121] Among them, the payload type (Type): 5 bits, used to indicate the type of data carried in the payload of AVS3, which is the same as the type in Payload_Type, and the value range is all values ​​except 12 and 13.

[0122] Reserved bit (R): 3 bits. When this field is 0, this field is ignored.

[0123] The single packet method is mainly used for the case where the data unit of the AVS3 video encoding stream has a large amount of data. If an RTP packet carries two data units, it will exceed the maximum limit of the RTP packet, so only the single packet method can be used to package the current data unit.

[0124] In the single packet method, packets are transmitted in the decoding (encoding) order.

[0125] Fig.14 is a flow chart of encapsulating an AVS3 video encoding stream in a single packet according to an embodiment of the present invention, such as Fig.14 As shown, the process includes the following steps:

[0126] Step S1401, the RTP packet module obtains the length of the data unit;

[0127] Step S1402, determining the packet type of the RTP packet and the data unit to be encapsulated according to the length of the data unit;

[0128] Step S1403: Encapsulate the data unit to be encapsulated in an RTP packet separately using a single packetization method, and fill / update the indication information of the AVS3 general header.

[0129] The single-packet method includes one of the following:

[0130] When the difference between the AVS3 video encoding data payload of the RTP packet and the data amount of a data unit currently to be packetized is greater than or equal to zero and less than the minimum data unit length, the current data unit to be packetized is separately encapsulated in an RTP packet, and the indication information of the AVS3 general header is filled / updated; wherein the AVS3 video encoding data payload is the data amount after deducting the RTP header length and the AVS3 general header length from the length limit of the RTP packet payload data.

[0131] When the sum of the data volume of a data unit to be encapsulated and the data volume of an adjacent data unit is greater than the AVS3 video encoding data payload of the current RTP packet, the data unit is encapsulated separately in an RTP packet and the indication information of the AVS3 general header is filled / updated.

[0132] For example: the AVS3 video encoding data payload of the current RTP packet is K bytes, and the length of the current data unit to be encapsulated (any data unit in the code stream) is J bytes. If KJ is greater than or equal to 0 and less than 8 (or the minimum data unit length in the AVS3 video encoding code stream); or, if KJ is greater than 8, the length (L bytes) of the next data unit to be encapsulated (n+1th) is obtained in the code stream order, and the sum of the lengths of the two data units to be encapsulated J+L is greater than K, then a single packetization method is used for the data unit. Among them, the AVS3 video encoding data payload in the RTP packet is the length limit minus the length of the RTP header and the AVS3 general header, and K, J, and L are integers greater than 0.

[0133] In one embodiment, according to the requirements of media processing and transmission, when the length of the first data unit to be encapsulated is small, the data units following the code stream sequence may not be judged, and the data unit may be directly encapsulated in an RTP packet using a single packetization method.

[0134] In one embodiment, if a data unit is required for transmission, media processing, etc., even if the length of the data unit is significantly smaller than the AVS3 video encoding data payload, it can be packaged in combination with other data units, but it can still be packaged separately in an RTP packet using a single packetization method.

[0135] The embodiment of the present invention provides a combined packaging method to encapsulate AVS3 video encoding stream. Fig.15 This is a schematic diagram of the encapsulation of an RTP packet according to a combined packetization method according to an embodiment of the present invention, in which the AVS3 payload contains two or more data units. When the length of the RTP packet does not exceed the length limit, the combined packetization method can be applied to any data type that meets the encapsulation rules in the order of the AVS3 video encoding stream. For example: the RTP packet encapsulates multiple data units whose data type is video data extension, or encapsulates an image frame's image header and one or more corresponding slice data of the image frame, or encapsulates multiple slice data of an image frame, etc. The data type Payload_Type of the AVS3 payload in the RTP packet has a value range of 12, or the packet type indication (P) has a value of 1.

[0136] In the case of combined packet, the AVS3 payload header includes two parts: the AVS3 general header and the AVS3 extended header. The AVS3 extended header further indicates the data characteristics in the AVS3 payload in the combined packet mode. Among them, the structure of the AVS3 general header is the same as the AVS3 general header in the above embodiment, which can be seen in Fig. 9 ; AVS3 extension header example Fig.16 As shown, the extended header information includes:

[0137] Reference frame image indication (Re): 2 bits, used to indicate whether the data unit actually contained in the AVS3 payload is related to the reference image frame, such as the parameter header of the I frame, the slice data of the I frame, the knowledge image, etc.

[0138] Key parameter indication (K): 1 bit, indicating whether the data unit actually contained in the AVS3 payload contains the parameter set required for decoding, such as the sequence header parameter.

[0139] Heterogeneous data indication (H): 1 bit, used to indicate whether the data units contained in the AVS3 payload are of different data types, for example: user data or video data extension data, image header data and slice data of the image frame are aggregated at the same time. If the heterogeneous data value is 0, it means that the RTP payload aggregates multiple slice data of one frame.

[0140] Slice data indication (P): 1 bit, used to indicate whether the data unit actually contained in the AVS3 payload has slice data; if it is directly indicated by heterogeneous data indication (H), this indication information may not be used.

[0141] Image frame number (FN): 2 bits, used to indicate whether the data unit actually contained in the AVS3 payload contains the image header and complete image data of multiple frames of images.

[0142] In one embodiment, each frame of an image has a time stamp, and multiple frames of images are packaged in an RTP packet. Based on the number of image frames, the time compensation of each frame of the image can be calculated or expressed to obtain the timestamps of all image frames in the RTP packet. For example, in the RTP packet, the timestamp in the RTP header is the timestamp of the first image frame in the RTP packet, and then starting from the second frame of the image in the RTP packet, time compensation information (usually 8 bits in length) can be added in front of the header of each frame of the image (except the first frame of the RTP packet); or, extended information of time compensation (8 bits in length, FN-1 times) can be added after the AVS3 extended header to describe the time compensation of each frame of the image of all FN-1 frames after the first frame of the image in the RTP packet in sequence.

[0143] Reserved bit (R), 2 bits. When this field is 0, this field is ignored.

[0144] In one embodiment, in this packetization method, if multiple image frames are packetized, the data of the packetized image frames are complete data, that is, the complete data of multiple image frames and partial data of one or more image frames (which may contain multiple data units) are usually not placed in a combined packet.

[0145] Fig.17 is a flow chart of encapsulating an AVS3 video encoding stream in a combined packet manner according to an embodiment of the present invention, such as Fig.17 As shown, the process includes the following steps:

[0146] Step S1701: The RTP encapsulation module obtains the length of the data unit, and determines the encapsulation type of the RTP packet and the data unit to be encapsulated according to the length of the data unit.

[0147] Step S1702: encapsulate multiple adjacent data units into the same RTP packet, and fill / update the indication information in the AVS3 general header and the extended header.

[0148] Among them, multiple data units are encapsulated into the same RTP packet when the following conditions are met:

[0149] When the difference between the AVS3 video encoding data payload of the RTP packet and the sum of the data amounts of the adjacent multiple data units to be encapsulated is equal to 0 or less than the minimum data unit length, these data units are encapsulated into the same RTP packet. The AVS3 video encoding data payload of the RTP packet is the length limit of the RTP packet payload data minus the RTP header (length), the AVS3 general header (length), and the AVS3 extension header (length).

[0150] For example, the AVS3 video encoding data payload of the current RTP packet is H bytes. At this time, the AVS3 video encoding data payload of the current RTP packet is the length limit of the RTP packet payload data minus the RTP header and the AVS3 general header. The length of the current data unit to be packaged (nth) is W bytes. If W is at least greater than 8 or greater than the minimum data unit length in the AVS3 video encoding code stream, the length of the next (n+1th) data unit to be encapsulated is obtained in the code stream order. For example, the length of the n+1th data unit to be encapsulated is V bytes, then the lengths of the nth and n+1th data units to be packaged are added together W+V, and the AVS3 video encoding data payload of the RTP packet is H' bytes. At this time, the AVS3 video encoding data payload of the RTP packet is the length limit minus the RTP header, AVS3 general header and AVS3 extension header. If H'-(W+V) is less than 8 or less than the minimum data unit length in the AVS3 video encoding code stream and greater than or equal to 0, it is determined that the RTP packet adopts a combined packetization method and encapsulates the above two data units into the same RTP packet.

[0151] If H'-(W+V) is still greater than (or the minimum data unit length in the AVS3 video encoding code stream), determine that the RTP packet adopts the combined packetization mode, and continue to obtain the following i (greater than or equal to 1) data units to be encapsulated in the code stream order, until the sum of the lengths of all the acquired data units to be encapsulated (a total of i+2) in the AVS3 video encoding data payload (H' bytes) of the RTP packet is less than 8 (or the minimum data unit length in the AVS3 video encoding code stream) and greater than or equal to 0, then encapsulate the above i+2 two data units into the same RTP packet; or, if the sum of the lengths of the AVS3 video encoding data payload (H' bytes) of the current RTP packet minus the lengths of the i+2 data units to be encapsulated is still greater than 8 (or the minimum data unit length in the AVS3 video encoding code stream), but the sum of the lengths after adding one data unit (the n+2+ith) is greater than H', then encapsulate the above i+2 two data units into the same RTP packet;

[0152] In one embodiment, according to the requirements of media processing and transmission, when the sum of the lengths of multiple data units to be encapsulated is still small, the RTP packet can continue to encapsulate other data units, or the data units following the bit stream sequence can be ignored and multiple data units to be encapsulated can be directly encapsulated into the same RTP packet.

[0153] In one embodiment, in the combined packet, the association between data units needs to be considered. This can improve the media processing capability and reduce the overhead caused by the loss of associated data units. If two adjacent data units are completely unrelated, or the latter data unit in the adjacent data units has a stronger association with the data unit behind it, the latter data unit and the data unit behind it can be encapsulated into the next RTP packet.

[0154] In one embodiment, in the combined packet, in order to further indicate the decoding order of each data unit, a decoding order indication (DON) is added before each data unit start code in the AVS3 payload, and the indication information length is at least 4 bits.

[0155] In one embodiment, in the combined packet, in order to further increase the AVS3 encoded code stream data carried by the AVS3 payload part, the prefix in the start code can be removed and only the start code value can be retained. For example, the intra-frame prediction image header is "0x000001B3", the prefix is ​​"0x000001", and the start code value is "B3". If the start code prefix is ​​removed, in order to better distinguish each data unit in an RTP packet, a data unit length indication (Size) needs to be added in front of each data unit, and the indication length is at least 8 bits. Fig.18 shown.

[0156] The embodiment of the present invention provides a segmented packaging method to encapsulate AVS3 video encoding stream. Fig.19 This is a schematic diagram of the encapsulation of RTP packets in the segmented packetization method according to an embodiment of the present invention. In this method, the data volume of a data unit in the AVS3 video encoding stream is greater than the length limit of the payload data of an RTP packet. At this time, it is necessary to split the single data unit into 2 or more segments, and then separately packetize each segment into 2 or more RTP packets.

[0157] According to the structure of the AVS3 video coding code stream, the segmented packaging method is suitable for the case where the amount of data of a single slice in an image frame (such as an I frame) is large.

[0158] In the case of segmented packetization, the payload header of AVS3 includes two parts: AVS3 general header and AVS3 extended header. The AVS3 extended header further indicates the data characteristics of the coding unit in the AVS3 payload in the segmented packetization mode. The structure of the AVS3 general header is the same as the AVS3 general header in the above embodiment, which can be seen in Fig. 9 , the extended header example is as follows Fig. 20 As shown:

[0159] The extended header information includes:

[0160] Segment type (Type): 5 bits, used to indicate the type of segment data carried in the AVS3 payload, the same as the type in Payload_Type, and the value range is all values ​​except 12 and 13.

[0161] Segment start (S): 1 bit, used to indicate whether the segment data carried in the AVS3 payload contains the start segment in the slice data

[0162] End of segment (E): 1 bit, used to indicate whether the segment data carried in the AVS3 payload contains the end segment in the slice data.

[0163] Reserved (R): 1 bit. When this field is 0, this field is ignored.

[0164] In the segmented packetization method, the slice data is segmented and packaged according to the code stream sequence of the specific coded data. Fig.21 is a flow chart of encapsulating an AVS3 video encoding stream in a segmented packet manner according to an embodiment of the present invention, such as Fig.21 As shown, the packetization process includes the following steps:

[0165] Step S2101: The RTP packetization module obtains the length of the data unit to be encapsulated, and determines the packetization type of the RTP packet and the data unit to be encapsulated according to the length of the data unit to be encapsulated.

[0166] Step S2102: encapsulate the segmented data of the above data unit into an RTP packet, and fill / update the indication information in the AVS3 general header and the extended header.

[0167] When the length of the data unit to be encapsulated meets the following conditions, the segmented data of the data unit is encapsulated into one RTP packet:

[0168] When the AVS3 video encoding data payload of the RTP packet is less than the length of the data unit to be encapsulated, the segmented data (partial data) of the encapsulated data unit is encapsulated into one RTP packet by using the segmented packet method. Among them, the AVS3 video encoding data payload of the RTP packet is the length limit of the RTP packet payload data minus the amount of data after the RTP header, AVS3 general header and AVS3 extension header are subtracted.

[0169] For example: the AVS3 video encoding data payload of the current RTP packet is Q bytes, which is the length limit of the RTP packet payload data minus the RTP header and the AVS3 general header. The length of the current data unit to be packaged is R bytes. If R is greater than Q, the segmented packetization method is adopted to split and select part of the data in the data unit (i.e., segmented data) in the order of the bit stream and encapsulate it into an RTP packet. The data length of this part of the data is less than or equal to Q minus the length of the AVS3 extension header; at this time, the AVS3 video encoding data payload of the RTP is Q' bytes, that is, the length limit of the RTP packet payload data minus the data amount of the RTP header, the AVS3 general header, and the AVS3 extension header;

[0170] In one embodiment, if the minimum coding unit CU of the slice data in the AVS3 video coding stream can be distinguished according to the code stream order, the data unit of the slice data type can be segmented according to the minimum coding unit CU. The segmentation process is the same as the above step S2101, and it is necessary to ensure that the length of one or more minimum coding units CU in the packaged slice data is less than the AVS3 video coding data length limit of the RTP packet.

[0171] In one embodiment, in the segmented packetization method, segments of two different data units cannot be encapsulated into one RTP packet.

[0172] The embodiment of the present invention further provides a homogeneous packaging method, in which data units or segments of data units of the same type are encapsulated into one RTP packet.

[0173] In this packet mode, the value range of the payload type (Payload_Type) is 1 to 11. At the same time, the reserved bit in the AVS3 general header is used or the AVS3 extension header is newly added to add the following indication information:

[0174] Fragmentation indicator (F): 1 bit, used to indicate whether the RTP packet contains fragmentation of the fragment data. For example, a value of 0 indicates that it does not contain fragmentation; a value of 1 indicates that it contains fragmentation.

[0175] Start of segment (S): 1 bit, used to indicate whether the segment data carried in the AVS3 payload contains the start segment in the slice data.

[0176] End of segment (E): 1 bit, used to indicate whether the segment data carried in the AVS3 payload contains the end segment in the slice data.

[0177] When the segmentation indicator is 0, it means that there is no data unit segment in the RTP packet, and S and E can be ignored.

[0178] In one embodiment, the same type of packetization method can support encapsulating two or more data units and data unit segments of the same type in one RTP packet, such as data units A and B of the slice data of an I frame. The RTP packet can contain the segment data of data unit A (including the end of the segment) and the segment data of B (including the start of the segment). At this time, the segmentation indicator F takes a value of 1, and the start S and end E of the segment are both 1.

[0179] In one embodiment, if the same type of packetization is used, there is no need to use the single packetization and the segmented packetization at the same time. And if the combined packetization is used at the same time, only multiple data units of different types are packaged, such as packaging user data and video data extension data into one RTP packet, and the same type of data units use the same type of packetization. Further, the data unit whose data type is video data extension data can be considered as a different type of data unit because the video extension identifier (extension_id) in the AVS3 video encoding code stream is different, that is, the same type of packetization is not used for RTP packetization, but the combined packetization is used.

[0180] Fig. 22 is a flow chart of encapsulating an AVS3 video encoding stream in a similar packaging manner according to an embodiment of the present invention, such as Fig. 22 As shown, the packetization process includes the following steps:

[0181] Step S2201, the RTP encapsulation module obtains the length of the data unit to be encapsulated, determines the encapsulation mode of the RTP packet as the same type of packet according to the length of the data unit to be encapsulated, and determines the data unit to be encapsulated.

[0182] Step S2202, encapsulate the above-mentioned one or more data units to be encapsulated and / or partial data of one or more data units to be encapsulated into an RTP packet, and fill / update corresponding indication information in the AVS3 general header and / or the AVS3 extended header.

[0183] When the following conditions are met, one or more data units to be encapsulated and / or part of one or more data units to be encapsulated are encapsulated into one RTP packet:

[0184] The AVS3 video encoding data payload (Z bytes) of the current RTP packet is the length limit of the RTP packet minus the RTP header and the AVS3 general header. The length of the current data unit to be packaged is R bytes. If R is greater than Z, the same type of packaging is adopted; if R is less than Z, and the next one or more data units to be packaged are of the same data type as the current data unit, the same type of packaging is adopted;

[0185] If R is greater than Z, the data unit to be packaged is split into segments smaller than Z; or, if the sum of the lengths of one or more data units to be packaged is equal to or less than Z; or, if the sum of the lengths of one or more data units to be packaged and the lengths of partial data of one or more data units to be packaged is equal to or less than Z, go to step S2202.

[0186] The embodiment of the present invention also provides a method for an RTP receiving end to perform reverse decapsulation and decoding after receiving an RTP packet. The method is applied to Figure 2 RTP receiver 20 in the network architecture, Fig.23 is a flowchart of a method for decoding an RTP packet according to an embodiment of the present invention, such as Fig.23 As shown, the basic process is as follows:

[0187] Step S2301, the RTP receiving end stores the received RTP packets in sequence into the buffer of the RTP decapsulation module;

[0188] Step S2302, the RTP decapsulation module decapsulates the data packet in the order of the RTP sequence number. If the RTP packet encapsulates one or more data units, it is directly passed to the AVS3 video decoding module. If the RTP packet encapsulates a fragment of a data unit, all fragments of the data unit are concatenated in the order of decapsulation and then passed to the AVS3 video decoding module.

[0189] Step S2303, the AVS3 video decoding module decodes the AVS3 video encoding data of the data units depacketized in the depacketization order to generate a playable AVS3 video sequence.

[0190] In the method and device for AVS3 video coding transmission based on RTP protocol proposed in the above-mentioned embodiment of the present invention, the RTP protocol can be extended with reference to the AVS3 video coding features to provide a flexible packet format of the AVS3 video coding stream. The data is segmented by combining the AVS3 general header and the optional extended header with different start codes in the AVS3 video coding stream. According to the block / segment size of each frame of video, the size limit of the RTP packet payload data can be fully considered to perform single packetization, combined packetization, segmented packetization and similar packetization, thereby improving the efficiency of packetization and realizing real-time transmission.

[0191] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0192] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0193] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0194] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0195] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0196] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0197] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for transmitting an AVS3 video encoding stream, characterized in that: include: Encapsulating the AVS3 video encoding code stream into multiple real-time transport protocol RTP packets, wherein each of the RTP packets is provided with an AVS3 payload header and an AVS3 payload; The multiple RTP packets are transmitted in sequence in the order of encapsulation.

2. The method according to claim 1, characterized in that Encapsulate the AVS3 video encoding stream into multiple RTP packets, including: Dividing the AVS3 video encoding code stream into a plurality of data units according to a start code in the AVS3 video encoding code stream; The multiple data units are encapsulated into the multiple RTP packets, wherein each RTP packet carries one data unit, multiple data units and / or partial data of one or more data units.

3. The method according to claim 2, characterized in that Encapsulating the plurality of data units into the plurality of RTP packets comprises at least one of the following: A data unit is encapsulated into an RTP packet using the first encapsulation method; The second packetization method is used to encapsulate multiple data units of the same or different data types into one RTP packet; The third packetization method is used to encapsulate part of the data of a data unit into an RTP packet; The fourth packetization method is used to packetize all and / or part of the data of one or more data units of the same data type into one RTP packet.

4. The method according to claim 3, characterized in that in, The AVS3 payload header includes an AVS3 general header, and the information indicated by the AVS3 general header includes at least one of the following: a discard indication, a time level, a payload type, a knowledge image indication, a knowledge image extension, an extension indication, and a packet type indication.

5. The method according to claim 3 or 4, characterized in that: in, In the RTP packet obtained by adopting the first packetization method, the AVS3 payload header of the RTP packet includes an AVS3 extension header; the information indicated by the AVS3 extension header is the payload type.

6. The method according to claim 3 or 4, characterized in that: The first packetization method is used to encapsulate a data unit into an RTP packet, including: When the length of a data unit to be encapsulated is less than or equal to the AVS3 video encoding data payload of the RTP packet, the data unit is encapsulated separately in one of the RTP packets.

7. The method according to claim 3 or 4, characterized in that: in, In the RTP packet obtained by adopting the second packetization method, the AVS3 payload header of the RTP packet includes an AVS3 extension header; the information indicated by the AVS3 extension header includes at least one of the following: reference frame image indication, key parameter indication, heterogeneous data indication, slice data indication and number of image frames.

8. The method according to claim 7, characterized in that The second packetization method is used to encapsulate multiple data units of the same or different data types into one RTP packet, including: In a case where the sum of the lengths of the adjacent multiple data units to be encapsulated is less than or equal to the AVS3 video encoding data payload of the RTP packet, the multiple data units are encapsulated in one RTP packet.

9. The method according to claim 3 or 4, characterized in that: in, In the RTP packet obtained by adopting the third packetization method, the AVS3 payload header of the RTP packet includes an AVS3 extension header; the information indicated by the AVS3 extension header includes at least one of the following: the type of segment, the start of the segment, and the end of the segment.

10. The method according to claim 9, characterized in that The third packetization method is used to encapsulate part of the data of a data unit into an RTP packet, including: When the length of a data unit to be encapsulated is greater than the AVS3 video encoding data payload of the RTP packet, the data unit is encapsulated into multiple RTP packets, wherein each RTP packet contains partial data of the data unit.

11. The method according to claim 3 or 4, characterized in that: in, In the RTP packet obtained by adopting the fourth packetization method, the information indicated by the AVS3 general header in the RTP packet includes at least one of the following: segment indication, the start of a segment, and the end of a segment; or, the AVS3 payload header in the RTP packet includes an AVS3 extension header, and the information indicated by the AVS3 extension header includes at least one of the following: segment indication, the start of a segment, and the end of a segment.

12. The method according to claim 3, characterized in that The fourth packetization method is used to packetize all and / or part of one or more data units of the same data type into one RTP packet, including: According to the AVS3 video coding data payload of the RTP packet, at least one data unit of the same data type whose length is smaller than the AVS3 video coding data payload and / or partial data of at least one data unit are encapsulated in one of the RTP packets.

13. The method according to claim 2, characterized in that in, The multiple data units are distinguished according to the start code or according to the indication information of the data unit length in the AVS3 load.

14. A transmission device for AVS3 video encoding stream, characterized in that: include: An encapsulation module, used for encapsulating the AVS3 video encoding stream into multiple RTP packets, wherein each of the RTP packets is provided with an AVS3 payload header and an AVS3 payload; The transmission module is used to transmit the multiple RTP packets in sequence in the order of encapsulation.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 13 when executed by a processor.

16. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 13 are implemented.

Images