AVS3 video bit stream transmission method and system, terminal and storage medium

Abstract

The invention provides an AVS3 video bit stream transmission method and system, a terminal and a storage medium, and the method comprises the steps: employing a transmitting end to set transmission parameters for a video bit stream, the transmission parameters at least comprise an extended first parameter used for indicating whether a receiving end supports a slice recovery algorithm or not and the type of the used slice recovery algorithm, and then according to the transmission parameters, transmitting the video bit stream to a receiving end, packaging the video bit stream to obtain a plurality of data packets and sending the data packets to a receiving end; the method comprises the following steps: decoding received data packets by utilizing a receiving end to obtain a decoded video bit stream, judging whether packet loss occurs or not according to the number of the received data packets, if so, calling a preset slice recovery algorithm according to a first parameter of the decoded video bit stream to process the decoded video bit stream, and if not, processing the decoded video bit stream according to a second parameter of the decoded video bit stream. And obtaining the recovered video bit stream. According to the invention, when network transmission packet loss occurs, AVS3 bit stream data can be recovered without retransmission, so that transmission delay caused by retransmission is avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video transmission, in particular to an AVS3 video bitstream transmission method and system, a terminal and a storage medium. BACKGROUND

[0002] AVS3 is a part of the AVS (Audio Video Standard) series of audio and video coding standards independently developed by China, and its full name is "Third Generation Audio and Video Coding Standard". It is a new generation of video coding standard after AVS1 and AVS2, mainly used for efficient audio and video compression.

[0003] In the transmission of AVS3 video bitstream, considering the fluctuation of network bandwidth, the AVS video bitstream data cannot be correctly received and decoded by the receiving end many times. In view of this scenario, the receiving end can choose to re-request the unreceived AVS3 bitstream data, but the retransmission will introduce a large transmission delay.

[0004] Therefore, how to recover the AVS3 bitstream data without retransmission when network transmission packet loss occurs is a technical problem that technicians in the field need to solve. SUMMARY

[0005] To solve the above technical problems, the present application provides an AVS3 video bitstream transmission method, which can recover the AVS3 bitstream data without retransmission when network transmission packet loss occurs, thereby avoiding the transmission delay caused by retransmission. The present application also provides an AVS3 video bitstream transmission system, a terminal and a storage medium, which have the same technical effect.

[0006] The first object of the present application is to provide an AVS3 video bitstream transmission method.

[0007] The above application object of the present application is achieved by the following technical scheme: An AVS3 video bitstream transmission method, comprising: using a sending end to set transmission parameters for an AVS3 video bitstream to be transmitted, wherein the transmission parameters at least include an extended first parameter, and the first parameter is used to indicate whether the receiving end supports a slice recovery algorithm and the type of the slice recovery algorithm used; using the sending end to package the AVS3 video bitstream to be transmitted according to the transmission parameters, to obtain a plurality of data packets, and sending the plurality of data packets to the receiving end; using the receiving end to receive the data packets sent by the sending end, and decoding the received data packets to obtain a decoded AVS3 video bitstream; The receiving end is used to determine whether packet loss occurs according to the number of the received data packets, and if so, a preset slice recovery algorithm is called according to the first parameter of the decoded AVS3 video bitstream to process the decoded AVS3 video bitstream to obtain a recovered AVS3 video bitstream.

[0008] Preferably, in the AVS3 video bitstream transmission method, the transmission parameter further includes an extended second parameter, and the second parameter is used to indicate the order of the slice data in the image frame during transmission.

[0009] Preferably, in the AVS3 video bitstream transmission method, the sending end is used to package the to-be-transmitted AVS3 video bitstream according to the transmission parameter to obtain a plurality of data packets, including: The sending end is used to package the to-be-transmitted AVS3 video bitstream according to the transmission parameter and a preset first RTP packaging rule to obtain a plurality of data packets.

[0010] Preferably, in the AVS3 video bitstream transmission method, the first parameter and the second parameter are set in a sequence header, an intra prediction image header, or an inter prediction image header.

[0011] Preferably, in the AVS3 video bitstream transmission method, the transmission parameter further includes a slice priority parameter, and the slice priority parameter is used to define the transmission priority of the slice data in the image frame, and the slice data with high priority is transmitted before the slice data with low priority.

[0012] Preferably, in the AVS3 video bitstream transmission method, the first parameter is further used to indicate the order of the slice data in the image frame during transmission.

[0013] Preferably, in the AVS3 video bitstream transmission method, the sending end is used to package the to-be-transmitted AVS3 video bitstream according to the transmission parameter to obtain a plurality of data packets, including: The sending end is used to package the to-be-transmitted AVS3 video bitstream according to the transmission parameter and a preset second RTP packaging rule to obtain a plurality of data packets.

[0014] The second object of the present application is to provide an AVS3 video bitstream transmission system.

[0015] The above-mentioned second application object of the present application is achieved by the following technical scheme. An AVS3 video bitstream transmission system includes a sending end and a receiving end, wherein: The sending end is configured to set transmission parameters for the to-be-transmitted AVS3 video bitstream, wherein the transmission parameters at least include an extended first parameter, and the first parameter is used to indicate whether the receiving end supports a slice recovery algorithm and a type of the slice recovery algorithm used; The sending end is further configured to package the to-be-transmitted AVS3 video bitstream according to the transmission parameters to obtain a plurality of data packets, and send the plurality of data packets to the receiving end. The receiving end is configured to receive the data packets sent by the sending end, and decode the received data packets to obtain a decoded AVS3 video bitstream. The receiving end is further configured to determine whether packet loss occurs according to a number of the received data packets, and if so, call a preset slice recovery algorithm according to the first parameter of the decoded AVS3 video bitstream to process the decoded AVS3 video bitstream to obtain a recovered AVS3 video bitstream.

[0016] A third object of the present application is to provide an AVS3 video bitstream transmission terminal.

[0017] The third object of the present application is achieved by the following technical solution: An AVS3 video bitstream transmission terminal comprises a storage medium and a processor. The storage medium stores computer execution instructions. The processor executes the computer execution instructions stored in the storage medium to implement the AVS3 video bitstream transmission method as described above.

[0018] A fourth object of the present application is to provide a computer readable storage medium.

[0019] The fourth object of the present application is achieved by the following technical solution: A computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by a processor to implement the AVS3 video bitstream transmission method as described above.

[0020] The technical solution is characterized in that: the sending end sets transmission parameters for the to-be-transmitted AVS3 video bitstream, wherein the transmission parameters at least include an extended first parameter, and the first parameter is used to indicate whether the receiving end supports a slice recovery algorithm and a type of the slice recovery algorithm used; the sending end packs the to-be-transmitted AVS3 video bitstream according to the transmission parameters to obtain a plurality of data packets, and sends the plurality of data packets to the receiving end; the receiving end receives the data packets sent by the sending end, and decodes the received data packets to obtain a decoded AVS3 video bitstream; and the receiving end judges whether packet loss occurs according to the number of the received data packets, and if so, calls a preset slice recovery algorithm according to the first parameter of the decoded AVS3 video bitstream to process the decoded AVS3 video bitstream to obtain a recovered AVS3 video bitstream. The technical solution uses the extended first parameter as the transmission parameter, so that the receiving end can support obtaining slice data that cannot be correctly decoded through the slice recovery algorithm according to the first parameter when network transmission packet loss occurs, and the AVS3 bitstream data can be recovered without retransmission, thereby avoiding transmission delay caused by retransmission. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0022] Figure 1 A flowchart of an AVS3 video bitstream transmission method in an embodiment of the present application; Figure 2 A flowchart of another AVS3 video bitstream transmission method in an embodiment of the present application; Figure 3 A structural diagram of an AVS3 video bitstream transmission system in an embodiment of the present application; Figure 4 A structural diagram of an AVS3 video bitstream transmission terminal in an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make those skilled in the art better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0024] In the embodiments provided by the present application, it should be understood that the disclosed method and system can be implemented in other manners. The embodiments described are merely exemplary, and are not intended to limit the application. For example, the division of the modules is merely logical function division, and there can be another division manner in actual implementation. For example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the components shown or discussed can be indirect coupling or communication connection through some interfaces, devices, or modules, and can be electrical, mechanical, or in other forms.

[0025] It should be understood that, in the present application, if "system", "device", "unit" and / or "module" are used, it is merely a method for distinguishing different components, elements, parts, sections or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0026] In addition, the terms "first", "second", and the like are used only to describe different classes, and cannot be understood as indicating or implying relative importance or implying the number of the indicated technical features. Therefore, the features defined as "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "a plurality of" or "several" is two or more, unless otherwise explicitly and specifically limited.

[0027] If a flowchart is used in the present application, the flowchart is used to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or subsequent operations do not necessarily have to be performed in sequence. Instead, each step can be processed in reverse order or simultaneously. Other operations can also be added to these processes, or one or more steps can be removed from these processes.

[0028] It should also be noted that, in this document, terms such as "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusions, so that the articles or devices comprising a series of elements not only include those elements, but also include other elements not explicitly listed or inherent to such articles or devices. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of additional identical elements in the article or device comprising the above element.

[0029] Embodiments of the present application are written in a progressive manner.

[0030] As Figure 1 shown, the embodiments of the present application provide an AVS3 video bitstream transmission method, which comprises: S101. Utilize the sending end to set the transmission parameter for the AVS3 video bit stream to be transmitted, wherein the transmission parameter at least includes the first parameter of extension, and the first parameter is used to indicate whether the slice recovery algorithm is supported by the receiving end and the type of the slice recovery algorithm used; In S101, specifically, the sending end obtains the video data to be transmitted, processes according to the AVS3 coding standard, and obtains the AVS3 video bit stream data to be transmitted; then the sending end configures the transmission parameter for the AVS3 video bit stream data to be transmitted, wherein the transmission parameter at least includes the first parameter of extension, and the first parameter is used to indicate whether the slice recovery algorithm is supported by the receiving end and the type of the slice recovery algorithm used, and the slice (patch) refers to a plurality of adjacent maximum coding units arranged in the raster scan order, and the raster scan refers to mapping the two-dimensional rectangular raster to the one-dimensional raster, and the entry of the one-dimensional raster starts from the first row of the two-dimensional raster, and then scans the second row, the third row, and so on. The row in the raster is scanned from left to right. The slice recovery algorithm can be a known algorithm, and the present application does not make specific limitation thereto.

[0031] In some embodiments, the first parameter includes a patch recovery support flag (patch_recover_enable_flag) and a patch recovery mode (patch_recover_mode), wherein the patch recovery support flag is used to indicate whether the slice recovery algorithm is supported by the receiving end, and the patch recovery mode is used to indicate the type of the slice recovery algorithm used. Specifically, the patch recovery support flag (patch_recover_enable_flag) is defined as a binary variable, and the value of '1' indicates that the current bit stream supports the slice recovery algorithm, that is, the slice that cannot be correctly decoded can be recovered by other correctly decoded slices; and the value of '0' indicates that the current bit stream does not support the slice recovery algorithm. The patch recovery mode (patch_recover_mode) is defined as an 8-bit unsigned integer, indicating the type of the slice recovery algorithm supported by the current bit stream, and the value meaning is as follows.

[0032] Table 1

[0033] In other embodiments, the first parameter can also only include the patch recovery mode (patch_recover_mode), and the meaning of the flag bit value 0 is changed to the meaning of the corresponding mode value bit 0. The value meaning is as follows.

[0034] Table 2

[0035] In this way, the extension field can be simplified.

[0036] S102. Pack the AVS3 video bitstream to be transmitted into multiple data packets according to the transmission parameters, and send the multiple data packets to the receiving end by using the sending end; In S102, specifically, the sending end packs the AVS3 video bitstream to be transmitted into multiple data packets in combination with the transmission parameters, and sends them to the receiving end. For the bitstream indicating the first parameter, i.e., the tile recovery algorithm information, the tiles within an image frame are packed under the premise of meeting the tile recovery algorithm, such as N tiles (in order 0, 1, …, N-1) within an image frame. First, M tiles with odd order are packed, and then N-M tiles with even order are packed. Since the tile size varies, and the upper limit of the data packet load size is fixed, there may be a complete tile corresponding to multiple transmission packets, or a transmission packet corresponding to multiple complete tiles, which is not limited by the present application.

[0037] S103. Receive the data packets sent by the sending end by using the receiving end, and decode the received data packets to obtain the decoded AVS3 video bitstream; In S103, specifically, the receiving end decodes the received data packets, recovers the tiles to the decoding order, i.e., the raster scan order within an image frame, to obtain the decoded AVS3 video bitstream.

[0038] S104. Determine whether packet loss occurs according to the number of received data packets by using the receiving end. If so, call the preset tile recovery algorithm according to the first parameter of the decoded AVS3 video bitstream to process the decoded AVS3 video bitstream and obtain the recovered AVS3 video bitstream.

[0039] In S104, specifically, it is assumed that the number of data packets sent by the sending end is K, and the number of data packets received by the receiving end is L. The receiving end determines whether L is less than K. If so, it is considered that packet loss occurs, and then the preset tile recovery algorithm is called according to the first parameter of the decoded AVS3 video bitstream to process the decoded AVS3 video bitstream and obtain the recovered AVS3 video bitstream. For example, by using i tiles (such as the upper, lower, left, and right adjacent tiles of the missing tile) that have been received and correctly decoded, the missing tile j is recovered by the preset tile recovery algorithm to obtain the recovered AVS3 video bitstream.

[0040] In the transmission of the AVS3 video bitstream, considering the fluctuation of the network bandwidth, the AVS video bitstream data cannot be correctly received and decoded by the receiving end many times. For this scenario, the receiving end can choose to re-request the un-received AVS3 bitstream data, but the retransmission will introduce a large transmission delay.

[0041] The above embodiment sets the transmission parameter for the to-be-transmitted AVS3 video bitstream by using the sending end, wherein the transmission parameter at least includes the extended first parameter, and the first parameter is used to indicate whether the receiving end supports the patch recovery algorithm and the type of the patch recovery algorithm used; the sending end is used to package the to-be-transmitted AVS3 video bitstream according to the transmission parameter, to obtain a plurality of data packets, and send the plurality of data packets to the receiving end; the receiving end is used to receive the data packets sent by the sending end, and decode the received data packets to obtain the decoded AVS3 video bitstream; and the receiving end is used to determine whether packet loss occurs according to the number of received data packets, and if so, call the preset patch recovery algorithm according to the first parameter of the decoded AVS3 video bitstream to process the decoded AVS3 video bitstream to obtain the recovered AVS3 video bitstream. The above embodiment uses the extended first parameter as the transmission parameter, so that when network transmission packet loss occurs, the receiving end can support obtaining the patch data that cannot be correctly decoded through the patch recovery algorithm according to the first parameter, without the need to recover the AVS3 bitstream data through retransmission, thereby avoiding the transmission delay caused by retransmission.

[0042] In other embodiments of the present application, another AVS3 video bitstream transmission method is also provided, comprising: S201. The sending end is used to set the transmission parameter for the to-be-transmitted AVS3 video bitstream, wherein the transmission parameter at least includes the extended first parameter and the second parameter, the first parameter is used to indicate whether the receiving end supports the patch recovery algorithm and the type of the patch recovery algorithm used, and the second parameter is used to indicate the order of the patch data in the image frame during transmission.

[0043] In S201, specifically, the specific configuration of the first parameter can refer to the above S101. In some embodiments, the second parameter includes a patch transmission order flag (patch_trans_order_flag), a patch transmission order mode (patch_trans_order_mode), and a patch transmission order (patch_trans_order). The patch transmission order can be defined as an 8-bit unsigned integer indicating the transmission order of the current patch in the corresponding image frame. The patch transmission order flag can be defined as a binary variable, and the value of '1' indicates that the patch transmission order field patch_trans_order is indicated in the patch syntax; the value of '0' indicates that the patch order field patch_trans_order is not indicated in the patch syntax. The patch transmission order mode can be defined as an 8-bit unsigned integer, which is used to determine the parsing rule of the patch transmission order field when the patch transmission order field is indicated in the patch syntax, and the value meaning is as follows.

[0044] Table 3

[0045] The above embodiments can be implemented based on an existing AVS3 video bitstream and a real-time transport protocol (RTP). The first parameter and the second parameter can be set in a sequence header, an intra prediction picture header, or an inter prediction picture header. Taking the sequence header as an example, the first parameter includes a patch recovery support flag (patch_recover_enable_flag) and a patch recovery mode (patch_recover_mode), and the second parameter includes a patch transmission order flag (patch_trans_order_flag), a patch transmission order mode (patch_trans_order_mode), and a patch transmission order (patch_trans_order). The sequence header can be defined according to the following table.

[0046] Table 4

[0047] The patch can be defined according to the following table.

[0048] Table 5

[0049] The patch start code (patch_start_code) is defined as a 32-bit bit string, the first 24 bits are '0000 00000000 0000 0000 0001', and the last 8 bits are patch_index. patch_index is an 8-bit unsigned integer, and the value range is 0x00-0x7F. The row index and the column index of the patch are derived according to the following process: Table 6

[0050] S202. Utilize the sending end to package the AVS3 video bitstream to be transmitted according to the transmission parameter, obtain a plurality of data packets, and send the plurality of data packets to the receiving end; In S202, specifically, the sending end packages the AVS3 video bitstream to be transmitted into a plurality of data packets in combination with the transmission parameter, and sends the plurality of data packets to the receiving end. For a bitstream indicating the second parameter, i.e., patch transmission information, the patches in the image frame are packaged in compliance with the constraints of the patch transmission information.

[0051] In some embodiments, one of the implementation manners of the step of utilizing the sending end to package the AVS3 video bitstream to be transmitted according to the transmission parameter to obtain a plurality of data packets specifically includes: utilizing the sending end to package the AVS3 video bitstream to be transmitted according to the transmission parameter and a preset first RTP packaging rule to obtain a plurality of data packets.

[0052] Specifically, the RTP payload of the AVS3 video is encapsulated in the form of an AVS3 video elementary stream. One or more AVS3 video elementary streams or one AVS3 video elementary stream fragment can be included in the RTP payload of one AVS3 video. The AVS3 video elementary stream is a video coding bitstream segment conforming to T / AI 109.2-2021, composed of coding data between two adjacent start codes or a plurality of adjacent start codes (including the first start code), and the AVS3 video elementary stream data type, such as sequence header, user data, extension data, intra / inter prediction image, is identified according to the start code value. Among them, the AVS3 video elementary stream of the intra / inter prediction image type is composed of the intra / inter prediction image start code (including) to the next intra / inter prediction image start code, sequence start code, sequence end code, or the continuous bytes before the video editing code. The AVS3 video elementary stream fragment is a continuous segment formed by dividing the AVS3 video elementary stream into the smallest unit of bytes.

[0053] In particular, for the AVS3 video bitstream corresponding to the intra / inter prediction image, the part of the video bitstream can be further divided into smaller elementary streams, including the intra prediction image header, the inter prediction image header, the intra extension and user data, and the slice data.

[0054] The start code of the AVS3 elementary stream is a specific bit string. In the bitstream conforming to T / AI 109.2-2021, these bit strings should not appear in any case except the start code. The start code is composed of a start code prefix and a start code value. The start code prefix is the bit string '0000 0000 0000 0000 0000 0001'; the start code value is an 8-bit integer used to represent the payload data type. All start codes should be byte-aligned.

[0055] All integer fields defined in this embodiment follow the network byte order defined by RTP, i.e. high byte first. The AVS3 video payload data type definition is shown in the following table.

[0056] Table 7

[0057] According to the payload type of the AVS3 video RTP data packet, the AVS3 video coding bitstream data should satisfy the following first RTP packaging rule when encapsulated: A) Different types of meta-stream data should not be aggregated in the same RTP packet, except for the following cases: under the condition that the size of the aggregated packet meets the MTU (Maximum Transmission Unit) size limit of the IP layer, the continuous sequence header, video extension data after the sequence header, and user data meta-stream after the sequence header should be encapsulated into an aggregated packet. Under the condition that the size of the aggregated packet meets the MTU size limit of the IP layer, the continuous intra / inter prediction image header and video extension data after the image header, and user data after the image header should be encapsulated into an aggregated packet.

[0058] B) Multiple frames of images should not be aggregated in the same RTP packet.

[0059] C) When the IP packet size of an AVS3 video meta-stream exceeds the MTU size of the IP layer, the meta-stream must be segmented, and the segmented meta-stream slices are encapsulated and transmitted in a slice payload format.

[0060] D) Video sequence end code or video editing code type meta-stream should be encapsulated as a single packet.

[0061] E) Aggregated packets and slice packets should not be nested with each other, i.e., meta-stream slices should not be included in aggregated payloads.

[0062] F) Non-independent primary bitstream and knowledge bitstream meta-stream data should not be aggregated in the same RTP packet.

[0063] G) The payload data of the RTP packet should not be nested, i.e., a complete RTP packet should not be included in the RTP payload.

[0064] H) When patch-recover-enable-flag is set to 1, the patches required by the patch recovery algorithm should be transmitted through consecutive RTP packets as much as possible when encapsulating RTP packets. For example, N patches in an image frame (in order 0, 1, …, N-1), first package the M patches with odd order, and then package the N-M patches with even order I) When patch-trans-order-flag is set to 1, follow the constraints of patch-trans-order-mode, and encapsulate the patches in an image frame into RTP packets according to the patch_trans_order value indicated in the patch.

[0065] In the SDP (Session Description Protocol) declarative session description, the parameters profile-id and level-id are used to declare the profile and level of the transmitted video stream. The parameter sprop-sequence-header is used to indicate the characteristics of the transmitted bitstream; the bitstream_info_idc is used to indicate the bitstream type of the sending end when the main bitstream and the knowledge bitstream are transmitted through different RTP streams; the parameter sprop-max-don-diff is used to indicate the processing mode when the transmission order of the meta bitstream is inconsistent with the decoding order. The parameter patch-recover-enable-flag is used to indicate whether the patch recovery algorithm is supported during the decoding of the main bitstream; the parameter patch_recover_mode is used to indicate the type of the patch recovery algorithm used during the decoding of the main bitstream. The parameter patch-trans-order-flag is used to indicate whether the patch transmission order is indicated in the patches in the main bitstream; the parameter patch-trans-order-mode is used to indicate the transmission order mode of the intra-patch of the image frame in the main bitstream.

[0066] The value range of profile-id shall be consistent with T / AI 109.2-2021 B.2. When the parameter is not given, the parameter value is 0x20 by default, indicating the reference 8-bit profile (Main 8bit profile). The value range of level-id shall be consistent with T / AI 109.2-2021 B.3. When the parameter is not given, the parameter value is 0x20 by default, indicating the 4.0.30 level. sprop-sequence-header indicates the sequence header of the transmitted video stream, which describes the properties of the video stream, such as the width, height, frame rate, code rate, etc. of the transmitted video stream. Its value is the Base64 (see IETF RFC 4648) encoding of the video sequence header bitstream defined in T / AI 109.2-2021 7.1.2.2. bitstream_info_idc is used to indicate the type of the currently transmitted AVS3 video bitstream. The field takes the following values: 0: the transmitted video stream is an independent main bitstream; 1: the transmitted video stream is a non-independent main bitstream; 2: the transmitted video stream is a knowledge bitstream; 3: the transmitted video stream is a knowledge bitstream and a non-independent main bitstream; when the parameter is not given, the parameter value is 0 by default. sprop-max-don-diff: if the transmission order of the elementary bitstream is the same as the decoding order, the value of the parameter must be equal to 0. Otherwise, the parameter will specify the maximum absolute difference between the decode order index values of any two elementary bitstreams A and elementary bitstream B, where elementary bitstream A is located after elementary bitstream B in decoding order, and elementary bitstream A is located before elementary bitstream B in transmission order. The value of sprop-max-don-diff must be an integer in the range of 0 to 32767 (inclusive). When the parameter is not given, the parameter value is 0 by default. The specific definitions of the parameters patch-recover-enable-flag, patch-recover-mode, patch-trans-order-flag, and patch-trans-order-mode can be referred to the above embodiments, which will not be repeated here.

[0067] The sending end packs the AVS3 video bitstream to be transmitted into multiple data packets in combination with the above transmission parameters and RTP packing rules, and sends the data packets to the receiving end.

[0068] S203. Utilize the receiving end to receive the data packets sent by the sending end, and decode the received data packets to obtain the decoded AVS3 video bitstream. S204. With the receiving end, it is judged whether packet loss occurs according to the number of received data packets. If yes, a preset patch recovery algorithm is called according to the first parameter of the decoded AVS3 video bitstream, and the decoded AVS3 video bitstream is processed to obtain a recovered AVS3 video bitstream.

[0069] The specific implementation process of S203-S204 can refer to S103-S104 described above.

[0070] Since the retransmission scheme will introduce a large transmission delay, in some scenarios, the error recovery scheme will be more in line with the application requirements. Therefore, in order to support the above error recovery scheme, the above embodiment proposes a corresponding video bitstream and transmission protocol extension, so that the sending end and the receiving end of the AVS3 video bitstream can ensure the correct transmission and low delay transmission of the AVS3 video bitstream in the case of unstable network environment.

[0071] In other embodiments of the present application, in addition to directly defining the transmission order, the transmission order can also be defined in an indirect manner. Specifically, the transmission parameter further includes a patch priority parameter patch_priority, which is used to define the transmission priority of the patch data in the image frame. The patch data with high priority is transmitted before the patch data with low priority.

[0072] In other embodiments of the present application, in order to realize the minimum change, the first parameter is also used to indicate the order of the patch data in the image frame during transmission.

[0073] Specifically, the patch recovery algorithm is bound to the patch transmission information, that is, the patch transmission information is indicated when the patch recovery algorithm is opened. At the same time, the starting code value function is extended, and the patch starting code value is used to indicate the raster scan order and the transmission order at the same time. At this time, the AVS3 video bitstream syntax is as follows: Table 8

[0074] The first parameter includes a patch recovery support flag (patch_recover_enable_flag), which is defined as a binary variable. The value of '1' indicates that the current bitstream supports the patch recovery algorithm, that is, the patch that cannot be correctly decoded can be recovered by other correctly decoded patches. The value of '0' indicates that the current bitstream does not support the patch recovery algorithm.

[0075] The patch_start_code is defined as a 32-bit bit string, the first 24 bits are '0000 0000 0000 0000 0000 0001', and the last 8 bits are patch_index. The patch_index is an 8-bit unsigned integer, and the value range is 0x00~0x7F. When the patch_recover_enable_flag is 0, the row index and column index of the patch are derived according to the following process.

[0076] Table 9

[0077] When the patch_recover_enable_flag is 1, the row index and column index of the patch are derived according to the following process: Table 10

[0078] At the same time, the order of the patches in the image frame during transmission is determined according to the following rules: 1. PatchTransOrder = patch_index >> 5; 2. The patch with a smaller PatchTransOrder value should be transmitted before the patch with a larger PatchTransOrder value; 3. The patches with the same PatchTransOrder value follow the raster scan order (the smaller the PatchIndex, the earlier the raster scan order).

[0079] At this time, the extension in the RTP: the first part of the payload format remains unchanged, and the other parts are updated as follows: One of the implementation manners of the step of packaging the AVS3 video bitstream to be transmitted according to the transmission parameters by using the sending end to obtain a plurality of data packets comprises: packaging the AVS3 video bitstream to be transmitted according to the transmission parameters and a preset second RTP packaging rule by using the sending end to obtain a plurality of data packets.

[0080] According to the payload type of the RTP data packet of the AVS3 video, the following second RTP packaging rule should be met when the AVS3 video coding bitstream data is encapsulated: J) Different types of meta-stream data should not be aggregated in the same RTP packet, except for the following cases: under the condition that the size of the aggregated packet meets the MTU size limit of the IP layer, the continuous sequence header, video extension data after the sequence header, and user data meta-stream after the sequence header should be encapsulated into an aggregated packet. Under the condition that the size of the aggregated packet meets the MTU size limit of the IP layer, the continuous intra / inter prediction image header, video extension data after the image header, and user data after the image header should be encapsulated into an aggregated packet.

[0081] K) Multiple frames of images should not be aggregated in the same RTP packet.

[0082] L) When the size of an AVS3 video meta-stream encapsulated into an IP packet exceeds the MTU size of the IP layer, the meta-stream must be segmented, and the segmented meta-stream slices should be encapsulated and transmitted in a fragmentation payload format.

[0083] M) Video sequence end code or video editing code type meta-stream should be encapsulated as a single packet.

[0084] N) Aggregated packets and fragmented packets should not be nested with each other, i.e., meta-stream slices should not be included in the aggregated payload.

[0085] O) Non-independent primary bitstream and knowledge bitstream meta-stream data should not be aggregated in the same RTP packet.

[0086] P) The payload data of an RTP packet should not be nested, i.e., a complete RTP packet should not be included in the RTP payload.

[0087] Q) When patch-recover-enable-flag takes the value of 1, the transmission order of different patches is determined by the start code value of the patch, and the patches within an image frame are encapsulated into RTP packets in the transmission order. The transmission order determination rules are as follows: 1) patch_index is the last 8 bits of patch_start_code; 2) patches with smaller patch_index values should be transmitted before patches with larger patch_index values.

[0088] In the SDP declarative session description, the parameters profile-id and level-id are used to declare the profile and level of the transmitted video stream. The parameter sprop-sequence-header is used to indicate the characteristics of the transmission bitstream; when the main bitstream and the knowledge bitstream are transmitted through different RTP streams, the bitstream_info_idc is used to indicate the type of the bitstream of the sending end; the parameter sprop-max-don-diff is used to indicate the processing method when the transmission order of the meta bitstream is inconsistent with the decoding order. The parameter patch-recover-enable-flag is used to indicate whether the patch recovery algorithm is supported when the main bitstream is decoded.

[0089] Through the above embodiment, it is supported to obtain the patch that cannot be correctly decoded (such as the patch data missing due to packet loss) through the patch recovery algorithm, so as to avoid the transmission delay caused by retransmission.

[0090] As shown in the following table, in another embodiment of the present application, an AVS3 video bitstream transmission system is provided, which comprises a sending end 10 and a receiving end 11, wherein: Figure 3 The sending end 10 is configured to set transmission parameters for the AVS3 video bitstream to be transmitted, wherein the transmission parameters at least comprise an extended first parameter, and the first parameter is used to indicate whether the receiving end 11 supports the patch recovery algorithm and the type of the patch recovery algorithm used. The sending end 10 is further configured to package the AVS3 video bitstream to be transmitted according to the transmission parameters, to obtain a plurality of data packets, and to send the plurality of data packets to the receiving end 11. The receiving end 11 is configured to receive the data packets sent by the sending end 10, and to decode the received data packets to obtain a decoded AVS3 video bitstream. The receiving end 11 is further configured to determine whether packet loss occurs according to the number of the received data packets, and if so, to call a preset patch recovery algorithm according to the first parameter of the decoded AVS3 video bitstream, to process the decoded AVS3 video bitstream to obtain a recovered AVS3 video bitstream.

[0091] In other embodiments of the present application, the transmission parameters of the above-mentioned AVS3 video bitstream transmission system further comprise an extended second parameter, and the second parameter is used to indicate the order of the patch data in the image frame during transmission.

[0092] In other embodiments of the present application, when the sending end 10 performs packaging of the AVS3 video bitstream to be transmitted according to the transmission parameters to obtain a plurality of data packets, it is specifically configured to: ​Based on the transmission parameters and the preset first RTP packetization rule, the AVS3 video bitstream to be transmitted is packetized to obtain multiple data packets.

[0093] In other embodiments of this application, in the above-described AVS3 video bitstream transmission system, the first parameter and the second parameter are set in the sequence header, the intra-frame prediction image header, or the inter-frame prediction image header.

[0094] In other embodiments of this application, the transmission parameters in the AVS3 video bitstream transmission system described above also include slice priority parameters. Slice priority parameters are used to define the transmission priority of slice data within an image frame. Slice data with higher priority is transmitted before slice data with lower priority.

[0095] In other embodiments of this application, in the above-described AVS3 video bitstream transmission system, the first parameter is also used to indicate the order of slice data within an image frame during transmission.

[0096] In other embodiments of this application, in the above-described AVS3 video bitstream transmission system, when the transmitting end 10 performs the action of packaging the AVS3 video bitstream to be transmitted into multiple data packets according to the transmission parameters, it is specifically used for: Based on the transmission parameters and the preset second RTP packetization rules, the AVS3 video bitstream to be transmitted is packetized to obtain multiple data packets.

[0097] like Figure 3 As shown, in another embodiment of this application, an AVS3 video bitstream transmission terminal is also provided, including: a storage medium 20 and a processor 21; Storage medium 20 stores computer-executed instructions; The processor 21 executes computer execution instructions stored in the storage medium 20 to implement the AVS3 video bitstream transmission method as described above.

[0098] The processor 21 may include one or more processing cores. The processor 21 executes instructions, programs, code sets, or instruction sets stored in the storage medium 20, and calls data stored in the storage medium 20 to perform various functions and process data as described in this application. The processor 21 may be at least one of a specific application-specific integrated circuit, digital signal processor, digital signal processing device, programmable logic device, field-programmable gate array, central processing unit, controller, microcontroller, and microprocessor. It is understood that, for different devices, the electronic devices used to implement the functions of the processor 21 may also be other types.

[0099] The storage medium 20 can be used to store instructions, programs, codes, code sets or instruction sets. The storage medium 20 can include a storage program area and a storage data area, wherein the storage program area can store instructions for implementing an operating system, instructions for at least one function, and instructions for implementing the above-mentioned AVS3 video bitstream transmission method, etc.; and the storage data area can store data related to the above-mentioned AVS3 video bitstream transmission method, etc.

[0100] In another embodiment of the present application, a computer readable storage medium is also provided, and the computer readable storage medium stores computer execution instructions. When the computer execution instructions are executed by a processor, the computer execution instructions are used to implement the above-mentioned AVS3 video bitstream transmission method.

[0101] The computer readable storage medium can be a U disk, a mobile hard disk, a read-only memory, a random access memory, an optical disc or various media capable of storing program codes.

[0102] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An AVS3 video bitstream transmission method, characterized in that, include: Using the transmitting end, transmission parameters are set for the AVS3 video bitstream to be transmitted, wherein the transmission parameters include at least an extended first parameter, which is used to indicate whether the receiving end supports a slice recovery algorithm and the type of slice recovery algorithm used; Using the sending end, the AVS3 video bitstream to be transmitted is packaged according to the transmission parameters to obtain multiple data packets, and the multiple data packets are sent to the receiving end; Using the receiving end, the data packet sent by the sending end is received, and the received data packet is decoded to obtain the decoded AVS3 video bitstream; Using the receiving end, it is determined whether packet loss has occurred based on the number of data packets already received. If so, a preset slice recovery algorithm is invoked according to the first parameter of the decoded AVS3 video bitstream to process the decoded AVS3 video bitstream and obtain the recovered AVS3 video bitstream.

2. The method as described in claim 1, characterized in that, The transmission parameters also include an extended second parameter, which indicates the order of slice data within an image frame during transmission.

3. The method as described in claim 2, characterized in that, The method involves using the sending end to package the AVS3 video bitstream to be transmitted according to the transmission parameters, resulting in multiple data packets, including: Using the sending end, the AVS3 video bitstream to be transmitted is packaged according to the transmission parameters and the preset first RTP packetization rule to obtain multiple data packets.

4. The method as described in claim 2, characterized in that, The first parameter and the second parameter are set in the sequence header, the intra-frame prediction image header, or the inter-frame prediction image header.

5. The method as described in claim 1, characterized in that, The transmission parameters also include a slice priority parameter, which defines the transmission priority of slice data within an image frame. Slice data with higher priority is transmitted before slice data with lower priority.

6. The method as described in claim 1, characterized in that, The first parameter is also used to indicate the order of slice data within an image frame during transmission.

7. The method as described in claim 6, characterized in that, The method involves using the sending end to package the AVS3 video bitstream to be transmitted according to the transmission parameters, resulting in multiple data packets, including: Using the sending end, the AVS3 video bitstream to be transmitted is packaged according to the transmission parameters and the preset second RTP packetization rules to obtain multiple data packets.

8. An AVS3 video bitstream transmission system, characterized in that, include: The sender and receiver, where: The transmitting end is used to set transmission parameters for the AVS3 video bitstream to be transmitted, wherein the transmission parameters include at least an extended first parameter, which is used to indicate whether the receiving end supports a slice recovery algorithm and the type of slice recovery algorithm used. The sending end is further configured to package the AVS3 video bitstream to be transmitted according to the transmission parameters to obtain multiple data packets, and send the multiple data packets to the receiving end; The receiving end is used to receive the data packet sent by the sending end, and decode the received data packet to obtain the decoded AVS3 video bitstream; The receiving end is further configured to determine whether packet loss has occurred based on the number of data packets already received. If so, it calls a preset slice recovery algorithm based on the first parameter of the decoded AVS3 video bitstream to process the decoded AVS3 video bitstream and obtain the recovered AVS3 video bitstream.

9. An AVS3 video bitstream transmission terminal, characterized in that, include: Storage media and processor; The storage medium stores computer-executed instructions. The processor executes computer execution instructions stored in the storage medium to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 7.

Images