Coding and decoding method and device

Abstract

The invention provides a coding and decoding method and device. The method comprises the following steps: acquiring n CRR image fragments of a plurality of display images and CRR images; encoding the plurality of display images and the CRR image to obtain an image code stream; wherein the image code stream comprises n CRR image fragment code streams and a plurality of display image code streams. Moreover, each CRR image fragment code stream in the n CRR image fragment code streams comprises CRR image identification information, and the CRR image identification information is used for indicating that the CRR image fragment is carried in the corresponding code stream, so that the code stream receiving complexity of a decoding end is effectively reduced.

Description

[0001] This application is a divisional application. The application number of the original application is 202311133490.8, and the original application date is September 1, 2023. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The embodiments of the present application are directed to the field of data compression technology, and in particular, to a coding and decoding method and device. Background Art

[0003] Video codecs (video encoding and decoding) are widely used in digital video applications, such as broadcast digital television, Internet, photo albums, video transmission on mobile networks, real-time conversation applications such as video chat and video conferencing, Digital Versatile Disc (DVD) and Blu-ray Disc, video content acquisition and editing systems, and security applications for camcorders. Video encoding is to compress the original video stream through a specific compression algorithm, and the compressed video stream can be more conveniently stored and transmitted. After that, the video is decoded at the decoding end to obtain the original video stream.

[0004] The third generation audio and video coding standard (AVS3) proposes a cross random access point reference (CRR) coding and decoding scheme based on library pictures. This scheme obtains library pictures from existing videos. The system layer is provided with a long term buffer for storing reconstructed knowledge base pictures. The reconstructed knowledge base picture at the encoding end is obtained by encoding the knowledge base picture, and the reconstructed knowledge base picture at the decoding end is obtained by decoding the knowledge base picture bit stream. When encoding a video sequence, the reconstructed knowledge base picture corresponding to the current frame is obtained from the long term buffer, and the reconstructed knowledge base picture is used as a reference picture to perform inter-frame prediction coding on the current frame, thereby obtaining a video bit stream and a knowledge base picture bit stream corresponding to each reconstructed knowledge base picture used in encoding. The system layer of the decoder decodes each knowledge base picture bit stream and stores the obtained reconstructed knowledge base picture in the long term buffer. When decoding the current image bit stream, the decoder obtains the corresponding reconstructed knowledge base image from the long-term buffer, and uses the reconstructed knowledge base as a reference image to perform inter-frame prediction decoding on the current video bit stream. Summary of the invention

[0005] The present application provides a coding and decoding method and device. In the method, CRR image code stream is correctly identified by carrying CRR image identification information.

[0006] In a first aspect, the present application provides a coding method, comprising: obtaining multiple display images and n CRR image slices of a cross-random access point reference CRR image; n is an integer greater than 0. Encoding the multiple display images and the CRR image to obtain an image code stream; wherein the image code stream includes n CRR image slice code streams and multiple display image code streams; each CRR image slice code stream in the n CRR image slice code streams includes CRR image identification information, and the CRR image identification information is used to indicate that the corresponding code stream carries a CRR image slice.

[0007] In this way, during the encoding process, the present application adds CRR image identification information to distinguish different types of code streams, so that the decoding end can determine whether the received code stream is a CRR image fragmentation code stream based on the CRR image identification information, thereby improving the accuracy of receiving and identifying CRR image fragmentation code streams.

[0008] Exemplarily, the CRR image used as a reference image is transmitted in slices, so compared with the related art, it can effectively reduce bandwidth impact and decoder performance impact and improve encoding and decoding performance. Furthermore, each CRR image slice can be correctly identified through the CRR image identification information.

[0009] Exemplarily, the multiple display images may be display images in one RAS or display images in multiple RASs.

[0010] Exemplarily, the multiple display images may include at least one display image reference CRR image.

[0011] In a possible implementation, each CRR image slice code stream is in a network abstraction layer unit NALU format, and the NALU corresponding to each CRR image slice code stream includes a control field and a payload field, wherein the control field corresponding to each CRR image slice code stream includes CRR image identification information.

[0012] In this way, by encapsulating the CRR image identification information into the control field of the NALU, the decoder can determine whether the received code stream is a CRR image fragment code stream by reading the information in the control field, thereby effectively reducing the complexity of CRR image code stream recognition to improve the efficiency of the decoder in recognizing the integrity of CRR image reception. It can be understood that in the prior art, after decoding the CRR image fragments, when acquiring the CRR image, it can be determined whether the CRR image fragments are received completely. By adding CRR image identification information, the present application allows the decoder to determine whether the received code stream is a CRR image code stream after receiving the code stream, and can further determine whether it is received completely.

[0013] In a possible implementation, the CRR image is a non-displayable CRR image and when n is greater than 1, the CRR image identification information includes cross-random access point reference network abstraction layer unit type CRR NUT identification information and cross-random access point reference boundary network abstraction layer unit type CRR Delimiter NUT identification information; wherein, the first CRR image slice code stream and the last CRR image slice code stream among the n CRR image slice code streams include the CRR Delimiter NUT identification information, and the other CRR image slice code streams include the CRR NUT identification information.

[0014] In this way, by introducing CRR image identification information that can identify the boundary CRR image fragment code stream, the decoding end can determine the starting point and the ending point of the received CRR image fragment code stream based on the CRR identification information, and the complexity of CRR image code stream identification can be further reduced. In addition, the present application can effectively distinguish between non-displayable CRR images and displayable CRR images through different CRR image identification information, so that the decoder end can correctly identify the non-displayable CRR image code stream and the displayable CRR image code stream based on the CRR image identification information, so as to perform different decoding operations on different types of CRR images.

[0015] In a possible implementation, the CRR image is a non-displayable CRR image, and the CRR image identification information includes CRR NUT identification information. The present application can effectively distinguish between non-displayable CRR images and displayable CRR images through different CRR image identification information, so that the decoder end can correctly identify the non-displayable CRR image code stream and the displayable CRR image code stream based on the CRR image identification information, so as to perform different decoding operations on different types of CRR images, thereby further reducing the complexity of code stream identification on the decoder end.

[0016] In a possible implementation, the CRR image is a non-displayable CRR image, and when n is greater than 1, the CRR image identification information includes CRR NUT identification information, cross-random access point reference start network abstraction layer unit type CRR START NUT identification information, and cross-random access point reference end network abstraction layer unit type CRR END NUT identification information; wherein, the first CRR image slice code stream among the n CRR image slice code streams includes the CRR START NUT identification information, the last CRR image slice code stream includes the CRR END NUT identification information, and other CRR image slice code streams include the CRR NUT identification information.

[0017] In this way, by introducing CRR image identification information that can identify the boundary CRR image fragment code stream, the decoding end can determine the starting point and the ending point of the received CRR image fragment code stream based on the CRR identification information, which can further reduce the complexity of CRR image code stream identification. In addition, the present application can effectively distinguish between non-displayable CRR images and displayable CRR images through different CRR image identification information, so that the decoder end can correctly identify the non-displayable CRR image code stream and the displayable CRR image code stream based on the CRR image identification information, so as to perform different decoding operations on different types of CRR images. Thereby further reducing the complexity of code stream identification at the decoder end.

[0018] In a possible implementation, the CRR image is a displayable CRR image, and the CRR image identification information includes cross-random access point reference display network abstraction layer unit type CRR DP NUT identification information, wherein each CRR image slice code stream includes the CRR DP NUT identification information. The present application can effectively distinguish between non-displayable CRR images and displayable CRR images through different CRR image identification information, so that the decoder end can correctly identify the non-displayable CRR image code stream and the displayable CRR image code stream based on the CRR image identification information, so as to perform different decoding operations on different types of CRR images. Thereby further reducing the complexity of code stream identification at the decoder end.

[0019] In a possible implementation manner, n CRR image slice code streams in the image code stream are interleaved and arranged with a plurality of display image slice code streams.

[0020] In this way, the present application interweaves n CRR image bitstream segments and multiple display image bitstreams into one bitstream for transmission. In this way, all processes can be completed inside the encoder, and the entire encoding process can be completed without the cooperation of the system layer. It can be applied to the calling method of the existing encoder without modifying the system layer application that calls the encoder. Compared with the related technology, the design and deployment cost of the overall encoding and decoding system is reduced, and the applicability of the solution is improved.

[0021] Furthermore, in the present application, by adding CRR image identification information, the decoder end can correctly identify the CRR image code stream in the code stream transmitted in an interleaved order, so as to reduce the complexity of code stream identification.

[0022] In a possible implementation, each display image code stream is encapsulated in a NALU format, and the NALU corresponding to each display image code stream includes a control field and a load field; wherein the control field in the code stream corresponding to the random access point RAP image in the multiple display images includes a reference knowledge base image network abstraction layer unit type RL NUT identifier, the CRR image is a reference frame of the RAP image, and the RAP image is after n CRR image slice code streams.

[0023] In this way, the present application stipulates an RL image to mark the completion of CRR image code stream transmission, so that the decoder end can determine whether the CRR image code stream is received based on the RL NUT identification information.

[0024] Exemplarily, the encoder and the decoder agree that there is no other CRR image code stream between the RL image code stream and the CRR image code stream. In this way, the decoder can determine that the CRR image code stream has been received based on the identified RL image code stream.

[0025] In a possible implementation manner, the control field corresponding to each CRR image slice code stream also includes index information, and the control field in the code stream corresponding to the RAP image that references the CRR image includes the index information.

[0026] In a possible implementation, the image code stream includes a CRR image information code stream corresponding to the CRR image information, and the CRR image information is used to represent the CRR image. The CRR image information includes at least one of the following: a sequence parameter set SPS, a picture parameter set PPS, and a picture header PH of the CRR image.

[0027] In a possible implementation manner, the CRR image information code stream precedes the first CRR image slice code stream among the n CRR image slice code streams, and the CRR image information code stream is adjacent to the first CRR image slice code stream.

[0028] In this way, the decoder can determine the receiving start point of the CRR image slice stream based on the header information stream and the CRR image identification information, thereby realizing the counting of the CRR image slice stream and further performing packet loss detection on the CRR image slice stream.

[0029] In a possible implementation, the CRR image information code stream is encapsulated in NALU format.

[0030] In a possible implementation, the NALU corresponding to the CRR image information code stream includes a control field and a load field, wherein the control field corresponding to the CRR image information code stream includes image information identification information, and the image information identification information is used to indicate that the corresponding code stream carries CRR image information.

[0031] In this way, by carrying the image information in the header information, the complexity of bitstream reception can be further reduced, and the decoding end can determine that the received bitstream is the header information bitstream based on the NUT field of the bitstream.

[0032] In a second aspect, the present application provides a decoding method, comprising: receiving an image code stream; wherein the image code stream comprises: n CRR image slice code streams and multiple display image code streams; each of the n CRR image slice code streams comprises CRR image identification information, and the CRR image identification information is used to indicate that the corresponding code stream carries a CRR image slice; based on the CRR image identification information, decoding the n CRR image slice code streams in the image code stream; based on the decoded n CRR image slices, obtaining a CRR image; and, decoding the multiple display image code streams to obtain multiple display images.

[0033] In a possible implementation, each CRR image slice code stream is in the network abstraction layer unit NALU format, and the NALU corresponding to each CRR image slice code stream includes a control field and a load field, wherein the control field corresponding to each CRR image slice code stream includes CRR image identification information; based on the CRR image identification information, the n CRR image slice code streams in the image code stream are decoded, including: reading the CRR image identification information included in the control field of the code stream, and obtaining the CRR image slices carried in the code stream.

[0034] In a possible implementation, the CRR image is a non-displayable CRR image, and n is greater than 1, and the CRR image identification information includes cross-random access point reference network abstraction layer unit type CRR NUT identification information and cross-random access point reference boundary network abstraction layer unit type CRR Delimiter NUT identification information; wherein, the first CRR image slice code stream and the last CRR image slice code stream in the n CRR image slice code streams include the CRR Delimiter NUT identification information, and the other CRR image slice code streams include the CRR NUT identification information; based on the CRR image identification information, the n CRR image slice code streams in the image code stream are decoded, including: if the CRR Delimiter NUT identification information is read, determining that the code stream including the CRR Delimiter NUT identification information is the first CRR image slice code stream; if the CRR Delimiter NUT identification information is read again, determining that the code stream including the CRR Delimiter NUT identification information is the last CRR image slice code stream, and determining that the reception of the CRR image code stream is completed.

[0035] In a possible implementation manner, the CRR image is a non-displayable CRR image, and the CRR image identification information includes CRR NUT identification information.

[0036] In a possible implementation, the CRR image is a non-displayable CRR image, and n is greater than 1, and the CRR image identification information includes CRR NUT identification information, cross-random access point reference start network abstraction layer unit type CRR START NUT identification information, and cross-random access point reference end network abstraction layer unit type CRR END NUT identification information; wherein, the first CRR image slice code stream in the n CRR image slice code streams includes the CRR START NUT identification information, the last CRR image slice code stream includes the CRR END NUT identification information, and other CRR image slice code streams include the CRR NUT identification information; based on the CRR image identification information, the n CRR image slice code streams in the image code stream are decoded, including: reading the CRR START NUT identification information, determining that the code stream containing the CRR START NUT identification information is the first CRR image slice code stream; reading the CRR END NUT identification information, determining that the code stream containing the CRR END NUT identification information is the last CRR image slice code stream, and determining that the reception of the CRR image code stream is complete.

[0037] In one possible implementation, when n=1, the n CRR image slice code streams in the image code stream are decoded based on the CRR image identification information, including: if the CRR image identification information in the first received CRR image code stream carrying the CRR image identification information is the CRR NUT identification information, determining that the reception of the CRR image code stream is completed.

[0038] In a possible implementation, based on the CRR image identification information, n CRR image slice streams in the image stream are decoded, including: buffering each received CRR image slice stream; and acquiring the CRR image based on the buffered n CRR image slice streams.

[0039] In a possible implementation manner, n CRR image slice code streams in the image code stream are interleaved and arranged with a plurality of display image slice code streams.

[0040] In a possible implementation, the CRR image is a displayable CRR image, and the CRR image identification information includes cross-random access point reference display network abstraction layer unit type CRR DP NUT identification information, wherein each CRR image slice code stream includes the CRR DP NUT identification information.

[0041] In one possible implementation, based on the CRR image identification information, n CRR image slice code streams in the image code stream are decoded, including: reading the CRR DP NUT identification information, decoding the CRR image slice code stream containing the CRR DP NUT identification information to obtain CRR image slices; and obtaining the CRR image based on the decoded n CRR image slices.

[0042] In a possible implementation, each display image code stream is encapsulated in a NALU format, and the NALU corresponding to each display image code stream includes a control field and a load field; wherein the control field in the code stream corresponding to the random access point RAP image in the multiple display images includes a reference knowledge base image network abstraction layer unit type RL NUT identifier, the CRR image is a reference frame of the RAP image, and the RAP image is after n CRR image slice code streams.

[0043] In a possible implementation manner, the control field corresponding to each CRR image slice code stream also includes index information, and the control field in the code stream corresponding to the RAP image that references the CRR image includes the index information.

[0044] In a possible implementation, the image code stream includes a CRR image information code stream corresponding to the CRR image information, and the CRR image information is used to represent the CRR image. The CRR image information includes at least one of the following: a sequence parameter set SPS, a picture parameter set PPS, and a picture header PH of the CRR image.

[0045] In a possible implementation manner, the CRR image information code stream precedes the first CRR image slice code stream among the n CRR image slice code streams, and the CRR image information code stream is adjacent to the first CRR image slice code stream.

[0046] In a possible implementation, the CRR image information code stream is encapsulated in NALU format.

[0047] In a possible implementation, the NALU corresponding to the CRR image information code stream includes a control field and a load field, wherein the control field corresponding to the CRR image information code stream includes image information identification information, and the image information identification information is used to indicate that the corresponding code stream carries CRR image information.

[0048] The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect respectively. The technical effects corresponding to the second aspect and any implementation of the second aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here.

[0049] In a third aspect, the present application provides an encoding device, comprising:

[0050] An acquisition module, used to acquire n CRR image slices of a plurality of display images and a cross-random access point reference CRR image; n is an integer greater than 0;

[0051] The encoding module is used to encode multiple display images and CRR images to obtain an image code stream; wherein the image code stream includes n CRR image slice code streams and multiple display image code streams; each of the n CRR image slice code streams includes CRR image identification information, and the CRR image identification information is used to indicate that the corresponding code stream carries a CRR image slice.

[0052] In a possible implementation, each CRR image slice code stream is encapsulated into a network abstraction layer unit NALU format, and the NALU corresponding to each CRR image slice code stream includes a control field and a payload field, wherein the control field corresponding to each CRR image slice code stream includes CRR image identification information.

[0053] In a possible implementation manner, the CRR image is a non-displayable CRR image, and the CRR image identification information includes cross-random access point reference network abstraction layer unit type CRR NUT identification information and cross-random access point reference boundary network abstraction layer unit type CRR Delimiter NUT identification information;

[0054] The first CRR image slice code stream and the last CRR image slice code stream among the n CRR image slice code streams include CRR Delimiter NUT identification information, and the other CRR image slice code streams include CRR NUT identification information.

[0055] In a possible implementation manner, the CRR image is a non-displayable CRR image, and the CRR image identification information includes CRR NUT identification information.

[0056] In a possible implementation manner, the CRR image is a non-displayable CRR image, and the CRR image identification information includes CRR NUT identification information, cross-random access point reference start network abstraction layer unit type CRR START NUT identification information, and cross-random access point reference end network abstraction layer unit type CRR END NUT identification information;

[0057] The first CRR image slice code stream among the n CRR image slice code streams includes CRR START NUT identification information, the last CRR image slice code stream includes CRR END NUT identification information, and the other CRR image slice code streams include CRR NUT identification information.

[0058] In a possible implementation, the CRR image is a displayable CRR image, and the CRR image identification information includes cross-random access point reference display network abstraction layer unit type CRR DP NUT identification information, wherein each CRR image slice code stream includes the CRR DP NUT identification information.

[0059] In a possible implementation manner, n CRR image slice code streams in the image code stream are interleaved and arranged with a plurality of display image slice code streams.

[0060] In a possible implementation, each display image code stream is encapsulated in a NALU format, and the NALU corresponding to each display image code stream includes a control field and a load field;

[0061] Among them, the control field in the code stream corresponding to the random access point RAP image in the multiple display images includes the reference knowledge base image network abstraction layer unit type RL NUT identifier, the CRR image is the reference frame of the RAP image, and the RAP image is after the n CRR image slice code streams.

[0062] In a possible implementation manner, the control field corresponding to each CRR image slice code stream also includes index information, and the control field in the code stream corresponding to the RAP image that references the CRR image includes the index information.

[0063] In a possible implementation manner, the image code stream includes a CRR image information code stream corresponding to the CRR image information, and the CRR image information is used to represent the CRR image.

[0064] In a possible implementation manner, the CRR image information code stream precedes the first CRR image slice code stream among the n CRR image slice code streams, and the CRR image information code stream is adjacent to the first CRR image slice code stream.

[0065] In a possible implementation, the CRR image information code stream is encapsulated in NALU format.

[0066] In a possible implementation, the NALU corresponding to the CRR image information code stream includes a control field and a load field, wherein the control field corresponding to the CRR image information code stream includes image information identification information, and the image information identification information is used to indicate that the corresponding code stream carries CRR image information.

[0067] In a fourth aspect, the present application provides a decoding device, including:

[0068] A receiving module, configured to receive an image code stream; wherein the image code stream includes: n CRR image slice code streams and multiple display image code streams; each of the n CRR image slice code streams includes CRR image identification information, and the CRR image identification information is used to indicate that the corresponding code stream carries a CRR image slice;

[0069] A decoding module, used for decoding n CRR image slice code streams in the image code stream based on the CRR image identification information;

[0070] A decoding module, used to obtain a CRR image based on the decoded n CRR image slices; and

[0071] The decoding module is also used to decode multiple display image code streams to obtain multiple display images.

[0072] In a possible implementation, each CRR image slice code stream is encapsulated into a network abstraction layer unit NALU format, and the NALU corresponding to each CRR image slice code stream includes a control field and a load field, wherein the control field corresponding to each CRR image slice code stream includes CRR image identification information; a decoding module is specifically used to:

[0073] The control field of the code stream read includes CRR image identification information, and the CRR image slices carried in the code stream are obtained.

[0074] In a possible implementation, the CRR image is a non-displayable CRR image, and the CRR image identification information includes cross-random access point reference network abstraction layer unit type CRR NUT identification information and cross-random access point reference boundary network abstraction layer unit type CRR Delimiter NUT identification information; wherein the first CRR image slice code stream and the last CRR image slice code stream in the n CRR image slice code streams include the CRR Delimiter NUT identification information, and the other CRR image slice code streams include the CRR NUT identification information; the decoding module is specifically used to:

[0075] If the CRR Delimiter NUT identification information is read, it is determined that the code stream containing the CRR Delimiter NUT identification information is the first CRR image slice code stream;

[0076] If the CRR Delimiter NUT identification information is read again, it is determined that the code stream containing the CRR Delimiter NUT identification information is the last CRR image slice code stream, and it is determined that the reception of the CRR image code stream is completed.

[0077] In a possible implementation manner, the CRR image is a non-displayable CRR image, and the CRR image identification information includes CRR NUT identification information.

[0078] In a possible implementation, the CRR image is a non-displayable CRR image, and the CRR image identification information includes CRR NUT identification information, cross-random access point reference start network abstraction layer unit type CRR START NUT identification information, and cross-random access point reference end network abstraction layer unit type CRR END NUT identification information; wherein, the first CRR image slice code stream among n CRR image slice code streams includes CRR START NUT identification information, the last CRR image slice code stream includes CRR END NUT identification information, and other CRR image slice code streams include CRR NUT identification information; the decoding module is specifically used to: read the CRR START NUT identification information, determine that the code stream containing the CRR START NUT identification information is the first CRR image slice code stream; read the CRR END NUT identification information, determine that the code stream containing the CRR END NUT identification information is the last CRR image slice code stream, and determine that the reception of the CRR image code stream is completed.

[0079] In a possible implementation manner, the decoding module is specifically configured to: cache each received CRR image slice code stream; and acquire a CRR image based on the cached n CRR image slice code streams.

[0080] In a possible implementation manner, n CRR image slice code streams in the image code stream are interleaved and arranged with a plurality of display image slice code streams.

[0081] In a possible implementation, the CRR image is a displayable CRR image, and the CRR image identification information includes cross-random access point reference display network abstraction layer unit type CRR DP NUT identification information, wherein each CRR image slice code stream includes the CRR DP NUT identification information.

[0082] In one possible implementation, based on the CRR image identification information, n CRR image slice code streams in the image code stream are decoded, including: reading the CRR DP NUT identification information, decoding the CRR image slice code stream containing the CRR DP NUT identification information to obtain CRR image slices; and obtaining the CRR image based on the decoded n CRR image slices.

[0083] In a possible implementation, each display image code stream is encapsulated in a NALU format, and the NALU corresponding to each display image code stream includes a control field and a load field; wherein the control field in the code stream corresponding to the random access point RAP image in the multiple display images includes a reference knowledge base image network abstraction layer unit type RL NUT identifier, the CRR image is a reference frame of the RAP image, and the RAP image is after n CRR image slice code streams.

[0084] In a possible implementation manner, the control field corresponding to each CRR image slice code stream also includes index information, and the control field in the code stream corresponding to the RAP image that references the CRR image includes the index information.

[0085] In a possible implementation manner, the image code stream includes a CRR image information code stream corresponding to the CRR image information, and the CRR image information is used to represent the CRR image.

[0086] In a possible implementation manner, the CRR image information code stream precedes the first CRR image slice code stream among the n CRR image slice code streams, and the CRR image information code stream is adjacent to the first CRR image slice code stream.

[0087] In a possible implementation, the CRR image information code stream is encapsulated in NALU format.

[0088] In a possible implementation, the NALU corresponding to the CRR image information code stream includes a control field and a load field, wherein the control field corresponding to the CRR image information code stream includes image information identification information, and the image information identification information is used to indicate that the corresponding code stream carries CRR image information.

[0089] In a fifth aspect, the present application provides a coding device, comprising: one or more processors; a memory for storing one or more computer programs or instructions; when the one or more computer programs or instructions are executed by the one or more processors, the one or more processors implement a method as described in any one of the first aspects.

[0090] In a sixth aspect, the present application provides a decoding device, comprising: one or more processors; a memory for storing one or more computer programs or instructions; when the one or more computer programs or instructions are executed by the one or more processors, the one or more processors implement a method as described in any one of the second aspects.

[0091] In a seventh aspect, the present application provides a coding device, comprising a processor, configured to execute a method as described in any one of the first aspects.

[0092] In an eighth aspect, the present application provides a decoding device, comprising a processor, configured to execute a method as described in any one of the second aspects.

[0093] In a ninth aspect, the present application provides an electronic device comprising: a processing circuit and an interface circuit; wherein the interface circuit is used to couple with a memory outside the command transmission device and provide a communication interface for the processing circuit to access the memory; the processing circuit is used to execute program instructions in the memory to implement any method as described in the first aspect to the second aspect.

[0094] In the specific implementation process, the electronic device may be a chip, the input circuit may be an input pin, the output circuit may be an output pin, and the processing circuit may be a transistor, a gate circuit, a trigger, and various logic circuits. The input signal received by the input circuit may be received and input by, for example, but not limited to, a receiver, and the signal output by the output circuit may be, for example, but not limited to, output to a transmitter and transmitted by the transmitter, and the input circuit and the output circuit may be the same circuit, which is used as an input circuit and an output circuit at different times. The embodiments of the present application do not limit the specific implementation methods of the processor and various circuits.

[0095] In one implementation, the electronic device may be a wireless communication device, that is, a computer device that supports wireless communication functions. Specifically, the wireless communication device may be a terminal such as a smart phone, or a wireless access network device such as a base station. The system chip may also be referred to as a system on chip (SoC), or simply as an SoC chip. The communication chip may include a baseband processing chip and a radio frequency processing chip. The baseband processing chip is sometimes also referred to as a modem or a baseband chip. The radio frequency processing chip is sometimes also referred to as a radio frequency transceiver or a radio frequency chip. In a physical implementation, some or all of the chips in the communication chip may be integrated inside the SoC chip. For example, the baseband processing chip is integrated in the SoC chip, and the radio frequency processing chip is not integrated with the SoC chip. The interface circuit may be a radio frequency processing chip in a wireless communication device, and the processing circuit may be a baseband processing chip in a wireless communication device.

[0096] In another implementation, the electronic device may be a part of a wireless communication device, such as an integrated circuit product such as a system chip or a communication chip. The interface circuit may be an input / output interface, an interface circuit, an output circuit, an input circuit, a pin or a related circuit on the chip or the chip system. The processor may also be embodied as a processing circuit or a logic circuit.

[0097] In a tenth aspect, the present application provides a computer-readable storage medium, in which a program code is stored. When the program code is executed by a processor, the method described in any one of the first to second aspects is implemented.

[0098] In an eleventh aspect, the present application provides a chip, comprising: at least one processor. The at least one processor is configured to execute the method as described in any one of the first aspect to the second aspect.

[0099] Optionally, the chip further includes a memory. The at least one processor is used to execute the code in the memory, and when the at least one processor executes the code, the chip implements the method as described in any one of the first aspect to the second aspect.

[0100] Optionally, the above chip may also be an integrated circuit.

[0101] In a twelfth aspect, the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to implement a method as described in any one of the first to second aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] Figure 1 is a schematic diagram of the structure of an encoding and decoding system shown as an example;

[0103] Figure 2A A block diagram of a content providing system for implementing a content distribution service is shown as an example;

[0104] Figure 2B is a schematic diagram of the structure of an electronic device shown as an example;

[0105] Figure 3A The following is a schematic diagram of a streaming media system workflow shown as an example;

[0106] Figure 3B The following is a schematic diagram of a streaming media system workflow shown as an example;

[0107] Figure 4 is a schematic diagram of an exemplary video sequence;

[0108] Figure 5 Schematic diagram of the structure of NALU shown as an example;

[0109] Figure 6 is a schematic diagram of an exemplary CRR image;

[0110] Figure 7 A schematic diagram of a flow chart of an encoding method provided in an embodiment of the present application;

[0111] Figure 8is a schematic diagram of CRR image segmentation shown as an example;

[0112] Fig. 9 A flowchart of a decoding method provided in an embodiment of the present application;

[0113] Fig.10 A schematic diagram of a coding and decoding process provided in an embodiment of the present application;

[0114] Fig.11 A schematic diagram of an encoding and decoding process is shown as an example;

[0115] Figure 12a to Figure 12c is a schematic diagram of an exemplary interleaving sorting method;

[0116] Fig.13 A schematic diagram of a flow chart of an encoding method is shown as an example;

[0117] Fig.14 Schematic diagram of the image code stream structure of an undisplayable CRR image;

[0118] Fig.15 Schematic diagram of the image code stream structure of a displayable CRR image;

[0119] Fig.16 A flowchart of a decoding method is shown as an example;

[0120] Fig.17 is a schematic diagram of an image code stream shown as an example;

[0121] Fig.18 A schematic diagram of a flow chart of an encoding method is shown as an example;

[0122] Fig.19 Schematic diagram of the image code stream structure of an undisplayable CRR image;

[0123] Fig. 20 A flowchart of a decoding method is shown as an example;

[0124] Fig.21 A schematic diagram of a flow chart of an encoding method is shown as an example;

[0125] Fig. 22 Schematic diagram of the image code stream structure of an undisplayable CRR image;

[0126] Fig.23 A flowchart of a decoding method is shown as an example;

[0127] Fig.24 Schematic diagram of the structure of the header information NALU shown as an example;

[0128] Fig.25 is a schematic diagram of an image bit stream structure shown as an example;

[0129] Fig.26 is a schematic diagram of the structure of an encoding device shown as an example;

[0130] Fig. 27 is a schematic structural diagram of an exemplary decoding device;

[0131] Fig.28 The figure is a schematic diagram of the structure of an electronic device shown as an example. DETAILED DESCRIPTION

[0132] The term "at least one" in the embodiments of the present application refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where any of a, b, c, ab, ac, bc, or abc can include a single a, a single b, a single c, and can also include multiple a, multiple b, and multiple c.

[0133] Furthermore, unless otherwise stated, ordinal numbers such as "first" and "second" mentioned in the embodiments of the present application are used to distinguish between multiple objects, and are not used to limit the order, timing, priority or importance of multiple objects. For example, the first priority criterion and the second priority criterion are only used to distinguish different criteria, and do not indicate the difference in the content, priority or importance of the two criteria. In addition, the terms "including" and "having" in the embodiments of the present application, the claims and the drawings are not exclusive. For example, a process, method, system, product or device that includes a series of steps or modules / units is not limited to the listed steps or modules, and may also include steps or modules / units that are not listed.

[0134] The following combination Figure 1 Describe the coding and decoding system used in this application, Figure 1A schematic block diagram of a coding and decoding system used in an embodiment of the present application is shown, for example, a video coding and decoding system 10 (or simply referred to as coding and decoding system 10) that can utilize the technology of the present application. The video encoder 20 (or simply referred to as encoder 20) and the video decoder 30 (or simply referred to as decoder 30) in the video coding and decoding system 10 represent devices that can be used to perform various technologies according to various examples described in the present application.

[0135] like Figure 1 As shown, the encoding and decoding system 10 includes a source device 12, and the source device 12 is used to provide encoded data 21 such as encoded images to a destination device 14 for decoding the encoded data.

[0136] The source device 12 includes an encoder 20 , and optionally may further include an image source 16 , an image preprocessor 18 (or a preprocessing unit), and a communication interface or a communication unit 22 .

[0137] The image source 16 may include or may be any type of image capture device for capturing real-world images, etc., and / or any type of image generation device, such as a computer graphics processor for generating computer-animated images or any type of device for acquiring and / or providing real-world images, computer-generated images (e.g., screen content, virtual reality (VR) images, and / or any combination thereof (e.g., augmented reality (AR) images). The image source may be any type of memory or storage that stores any of the above images.

[0138] In order to distinguish the processing performed by the pre-processor 18 or the pre-processing unit 18 , the image or the image data 17 may also be referred to as a raw image or raw image data 17 .

[0139] The preprocessor 18 is used to receive (raw) image data 17 and preprocess the image data 17 to obtain a preprocessed image 19 or preprocessed image data 19. For example, the preprocessing performed by the preprocessor 18 may include trimming, color format conversion (e.g., from RGB to YCbCr), color adjustment, or denoising. It is understood that the preprocessing unit 18 may be an optional component.

[0140] The video encoder 20 is configured to receive the pre-processed image data 19 and provide encoded image data 21 .

[0141] The communication interface 22 in the source device 12 may be used to receive the encoded image data 21 and transmit the encoded image data 21 (or any other processed version thereof) to another device such as the destination device 14 or any other device via the communication channel 13 for storage or direct reconstruction.

[0142] Destination device 14 includes a decoder 30 (eg, video decoder 30 ) and, in addition or alternatively, may include a communication interface or communication unit 28 , a post-processor 32 (or post-processing unit 32 ), and a display device 34 .

[0143] The communication interface 28 in the destination device 14 is used to receive the encoded image data 21 (or any other processed version) directly from the source device 12 or from any other source device such as a storage device, for example, the storage device is a coded image data storage device, and provide the encoded image data 21 to the decoder 30.

[0144] The communication interface 22 and the communication interface 28 can be used to send or receive the encoded image data 21 or the encoded data through a direct communication link between the source device 12 and the destination device 14, such as a direct wired or wireless connection, or through any type of network, such as a wired network, a wireless network or any combination thereof, any type of private network and public network or any type of combination thereof.

[0145] For example, the communication interface 22 may be used to encapsulate the encoded image data 21 into a suitable format such as a message, and / or process the encoded image data using any type of transmission coding or processing for transmission over a communication link or network.

[0146] The communication interface 28 corresponds to the communication interface 22 , for example, and can be used to receive transmission data and process the transmission data using any type of corresponding transmission decoding or processing and / or decapsulation to obtain encoded image data 21 .

[0147] Both the communication interface 22 and the communication interface 28 can be configured as follows Figure 1 A unidirectional communication interface, or a bidirectional communication interface, is indicated by an arrow pointing from the source device 12 to the corresponding communication channel 13 of the destination device 14, and can be used to send and receive messages, etc. to establish a connection, confirm and exchange any other information related to the communication link and / or data transmission such as encoded image data transmission, etc.

[0148] The decoder 30 is configured to receive the encoded image data 21 and provide decoded image data 31 or a decoded image 31 .

[0149] The post-processor 32 in the destination device 14 is used to post-process the decoded image data 31 (also called reconstructed image data) such as the decoded image 31 to obtain the post-processed image data 33 such as the post-processed image 33. The post-processing performed by the post-processing unit 32 may include, for example, color format conversion (e.g., conversion from YCbCr to RGB), color adjustment, cropping or resampling, or any other processing for generating the decoded image data 31 for display by the display device 34 or the like.

[0150] The display device 34 in the destination device 14 is used to receive the post-processed image data 33 to display the image to a user or viewer, etc. The display device 34 can be or include any type of display for displaying the reconstructed image, such as an integrated or external display screen or display. For example, the display screen can include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display screen.

[0151] although Figure 1 The source device 12 and the destination device 14 are shown as independent devices, but the device embodiment may also include both the source device 12 and the destination device 14 or the functions of both the source device 12 and the destination device 14, that is, both the source device 12 or the corresponding functions and the destination device 14 or the corresponding functions. In these embodiments, the source device 12 or the corresponding functions and the destination device 14 or the corresponding functions may be implemented using the same hardware and / or software or by separate hardware and / or software or any combination thereof.

[0152] According to the description, Figure 1 The presence and (exact) division of different units or functionalities in the source device 12 and / or destination device 14 shown may vary depending on the actual device and application, as will be apparent to the skilled person.

[0153] Combine the following Figure 2A Describe the content providing system of the content distribution service applied by this application, Figure 2A A block diagram of a content providing system for implementing a content distribution service used in an embodiment of the present application. The content providing system 2100 includes a capture device 2102, a terminal device 2106, and an (optional) display 2126. The capture device 2102 communicates with the terminal device 2106 via a communication link 2104. The communication link may include the above-mentioned communication channel 13. The communication link 2104 includes, but is not limited to, WiFi, Ethernet, wired, wireless (3G / 4G / 5G), USB, or any combination thereof.

[0154] Capture device 2102 generates data, and can use the coding method shown in the above embodiment to encode the data. Alternatively, capture device 2102 can distribute data to a streaming media server (not shown in the figure), and the server encodes the data and transmits the encoded data to terminal device 2106. Capture device 2102 includes but is not limited to a camera, a smart phone or a tablet computer, a computer or a laptop computer, a video conferencing system, a PDA, a vehicle-mounted device or any combination thereof. For example, capture device 2102 may include the above-mentioned source device 12. When data includes video, the video encoder 20 in capture device 2102 can actually perform video encoding processing. When data includes audio (i.e., sound), the audio encoder 20 in capture device 2102 can actually perform audio encoding processing. In some actual scenes, capture device 2102 distributes encoded video data and encoded audio data by multiplexing the encoded video data and the encoded audio data together. In other actual scenes, for example, in a video conferencing system, the encoded audio data and the encoded video data are not multiplexed. Capture device 2102 distributes the encoded audio data and the encoded video data to terminal device 2106 respectively.

[0155] The terminal device 2106 in the content providing system 2100 receives and regenerates the encoded data. The terminal device 2106 can be a device with data receiving and recovery capabilities, such as a smart phone or tablet computer 2108, a computer or laptop computer 2110, a network video recorder (NVR) / digital video recorder (DVR) 2112, a television 2114, a set top box (STB) 2116, a video conferencing system 2118, a video surveillance system 2120, a personal digital assistant (PDA) 2122, a vehicle device 2124 or any combination thereof, or such a device capable of decoding the above-mentioned encoded data. For example, the terminal device 2106 can include the above-mentioned destination device 14. When the encoded data includes video, the video decoder 30 in the terminal device prioritizes video decoding. When the encoded data includes audio, the audio decoder in the terminal device prioritizes audio decoding processing. The terminal device 2106 may be a video playback application, a streaming media playback application, a streaming media playback platform, a live broadcast platform, etc. running on the terminal device.

[0156] For terminal devices with a display, such as a smartphone or tablet 2108, computer or laptop 2110, NVR / DVR 2112, TV 2114, PDA 2122, or vehicle-mounted device 2124, the terminal device can send the decoded data to its display. For terminal devices without a display, such as STB 2116, video conferencing system 3118, or video surveillance system 2120, connect the device to an external display 2126 to receive and display the decoded data.

[0157] When each device in the system performs encoding or decoding, the image encoding device or image decoding device shown in the above embodiments may be used.

[0158] Figure 2B for Figure 2A Schematic diagram of the example structure of the terminal device 2106 in FIG. After the terminal device 2106 receives the bit stream from the capture device 2102, the protocol processing unit 2202 analyzes the transmission protocol of the bit stream. The protocol includes but is not limited to Real Time Streaming Protocol (RTSP), Hyper Text Transfer Protocol (HTTP), HTTP Live streaming protocol (HLS), MPEG HTTP Dynamic Adaptive Streaming over HTTP (MPEG-DASH), Real-time Transport Protocol (RTP), Real Time Messaging Protocol (RTMP) or any combination thereof.

[0159] After the protocol processing unit 2202 processes the stream, a stream file is generated. The file is output to the demultiplexing unit 2204. The demultiplexing unit 2204 can separate the multiplexed data into coded audio data and coded video data. As mentioned above, in other actual scenarios, such as in a video conferencing system, the coded audio data and the coded video data are not multiplexed. In this case, the coded data is not transmitted to the video decoder 3206 and the audio decoder 2208 through the demultiplexing unit 2204.

[0160] Through the demultiplexing process, a video elementary stream (ES), an audio ES, and optional subtitles are generated. The video decoder 2206, including the video decoder 30 described in the above embodiment, decodes the video ES to generate a video frame by the decoding method shown in the above embodiment, and sends the data to the synchronization unit 2212. The audio decoder 2208 decodes the audio ES to generate an audio frame, and sends the data to the synchronization unit 2212. Alternatively, the video frame can be stored in a buffer (not shown in the figure) before being sent to the synchronization unit 2212. Similarly, the audio frame can be stored in a buffer (not shown in the figure) before being sent to the synchronization unit 2212.

[0161] The synchronization unit 2212 synchronizes the video frames and the audio frames and provides the video / audio to the video / audio display 3214. For example, the synchronization unit 2212 synchronizes the presentation of the video and audio information. The information can be encoded in the syntax using the timestamps associated with the encoded audio and visual data representations and the timestamps associated with the data stream transmission.

[0162] If subtitles are included in the bitstream, the subtitle decoder 2210 decodes the subtitles, synchronizes them with the video frames and audio frames, and provides the video / audio / subtitles to the video / audio / subtitle display 2216 .

[0163] The present invention is not limited to the above-mentioned system, and the image encoding device or the image decoding device in the above-mentioned embodiments can be used in other systems such as automobiles.

[0164] Combine the following Figure 3A A streaming media system to which the embodiments of the present application are applicable is described. Figure 3A A schematic diagram of a streaming media system workflow applicable to an embodiment of the present application.

[0165] The streaming media system includes a content creation module to generate the required content data, such as video or audio. The streaming media system also includes a video encoding module to encode the generated content through an encoder. The streaming media system also includes a video stream transmission module to transmit the encoded video in the form of a code stream. Optionally, the format of the video stream can be converted into a code stream format of a commonly used transmission protocol for OTT (over-the-top) devices, for example, the protocol includes but is not limited to Real Time Streaming Protocol (RTSP), Hyper Text Transfer Protocol (HTTP), HTTP Live streaming protocol (HLS), MPEG HTTP Dynamic Adaptive Streaming over HTTP (MPEG-DASH), Real-time Transport Protocol (RTP), Real-time Messaging Protocol (RTMP) or any combination thereof. Optionally, the video stream can be stored to store the original format of the video stream and / or the converted multiple code stream formats for easy use. Furthermore, the streaming media system also includes a video stream encapsulation module for encapsulating the video stream to generate an encapsulated video stream, which can be called a video streaming media package. Exemplarily, the video streaming media package can be generated according to the transcoded video stream or the stored video stream. Furthermore, the streaming media system also includes a content distribution network (CDN), which is used to distribute the video streaming media package to multiple OTT devices, such as mobile phones, computers, tablets, home projectors, etc.

[0166] It should be noted that video encoding, video streaming transmission, video stream transcoding, video stream storage, video streaming package generation and content distribution network can all be implemented on cloud servers.

[0167] Combine the following Figure 3B An exemplary streaming media system architecture of the present application is described. The streaming media system architecture includes: a client device, a content distribution network, and a cloud server.

[0168] The user on the client device sends a play or playback request to the cloud platform. Optionally, the content of the request sent may be the title of the movie or TV program being played.

[0169] The cloud platform makes a decision, replies to the client, and sends the address of the content requested by the client on the CDN to the client. Optionally, the content sent to the client can be a URL link (uniform resource locator). Specifically, the playback application service in the cloud platform checks user authorization and permission, and then considers the characteristics of each client and the current network conditions to determine which specific files are needed to process the playback request. It should be noted that the content delivery network (CDN) regularly reports the operating status, learned routes, and available content (files) to the cache control service in the cloud platform.

[0170] The client then requests the CDN to play the content based on the address, and the CDN provides the content to the client, ultimately completing the client's request.

[0171] Before describing the embodiments of the present application, the concepts and terms used in the embodiments of the present application are first explained.

[0172] 1) Reconstructed image: refers to the lossy image obtained by decoding the encoded image bit stream, which can be displayed and played. It exists at both the encoding end and the decoding end, and the reconstructed images in the encoding end and the decoding end are exactly the same. At the encoding end, the reconstructed image can be used as a reference image of the image to be encoded to perform inter-frame prediction encoding on the image to be encoded. At the decoding end, the reconstructed image can be used as a reference image of the image bit stream to be decoded to perform inter-frame prediction decoding on the image bit stream to be decoded.

[0173] 2) RAP (Random Access Point): A random access point in video encoding. The encoded video bitstream includes at least one RAP. When random access occurs, the video bitstream before the RAP can be skipped, and the subsequent image bitstream can be decoded from the RAP and the reconstructed video image output by the decoded output can be played. From the perspective of video playback, random access can be understood as dragging the progress bar of the player to achieve random video jumps when watching on-demand videos (or playing back videos locally); or when a black screen occurs due to packet loss when watching live broadcasts, the client will access the video stream and continue to play the video when the next RAP arrives. RAP is usually an intra-coded picture (I) frame. The I frame only uses the information inside the current frame for predictive coding to eliminate information redundancy in the spatial domain without relying on other frames.

[0174] 3) RAS (Random Access Segment): RAS includes all images from one RAP to the next RAP. The images in a RAS are all forward prediction (predictive, P) frames or bi-directional interpolated prediction frames (bi-directional interpolated prediction frames, B frames). P frames or B frames need to use the encoded images to perform time domain prediction to eliminate time domain redundancy.

[0175] For example, please refer to Figure 4 , Figure 4 A schematic diagram of a video sequence provided in an embodiment of the present application, Figure 4 11 video frames a1 to a11 are shown. Among them, a2, a6 and a10 are I frames and serve as RAP, all video frames between a2 and a6 are P frames and constitute RAS1, and all video frames between a6 and a10 are P frames and constitute RAS2. Figure 4 It can be seen that each P frame needs to be encoded with reference to the previous m video frames, where m is an integer greater than 1. For example, a3 can be encoded with reference to a2, and a4 can be encoded with reference to a2 and / or a3.

[0176] 4) CRR coding: When encoding a video frame in a RAS, the video frames in other RASs other than the RAS can be referenced for coding. Figure 4 The reference frame of the video frame a11 is the image frame a9.

[0177] In most videos (such as movies or TV programs), many scenes appear alternately, and the scenes displayed at different times may be similar. The interval between two adjacent similar scenes is usually long, so two adjacent similar scenes are in the same RAP but the interval is long, or the time correlation spans the RAP. For example, the video standard generally stipulates that the length of RAS is 1s or 2s, and the interval between two adjacent similar scenes is usually close to the length of RAS (for example, 0.7s) or greater than the length of RAS (for example, 3s). In addition, in video scenes such as security monitoring, the appearance of similar scenes usually lasts for a long time, so there may be two similar scenes in the same RAP but with a long interval, or the time correlation spans the RAP.

[0178] 5) SPS (Sequence Parameter Set): Contains a set of global parameters of the video encoding sequence, such as color space, width and height of each frame, encoding level, etc.

[0179] 6) PPS (Picture Parameter Set): contains encoding parameters for one or more pictures, such as quantization matrix, filter parameters, etc.

[0180] 7) Picture Header (PH): Contains the encoding parameters of a single image, such as the frame number.

[0181] 8) Image coding data: Contains prediction, residual and other data for encoding the content of a single image.

[0182] 9) Header information: refers to all encoding parameters used to encode and decode a single image. For example, SPS, PPS and PH are all header information.

[0183] 10)NALU (Network Abstraction Layer Unit): a grammatical structure. Figure 5 For an exemplary structural diagram of NALU, please refer to Figure 5 , each NALU includes a NALU Header field (also referred to as NALU header information) and a load field. Among them, the NALU Header field can also be referred to as a control field. The NALU Header field includes a NUT (Network Abstraction Layer Unit Type) field. In an embodiment of the present application, the NUT field is used to carry data type identification information and other related information, and the data type identification information is used to indicate the type of data carried in the load field.

[0184] Optionally, the information in the payload field may appear in the form of RBSP (Raw Byte Sequence Payload). In other embodiments, the payload field may also include interspersed anti-counterfeiting bytes. In an embodiment of the present application, the bit stream (also referred to as a code stream) generated by the encoder is encapsulated in NALU format. Optionally, in an embodiment of the present application, the NALU with "A" in the NUT field is also referred to as A NALU, which will not be repeated below.

[0185] 11) RL (Reference Library) image: The entire image can be encoded with inter-frame prediction and intra-frame prediction, and the reference image is only one CRR image. In other words, other display images may refer to one or more images, and the one or more reference images may include CRR images, or one or more display images before the display image. RL images only refer to CRR images.

[0186] 12) Display images: I images (I frames), P images (P frames), B images (B frames), RL images (RL frames), etc. that can be displayed in the coding sequence, excluding CRR images.

[0187] 13)CRR image: The entire image is encoded using intra-frame prediction, and its reconstructed image can be placed in the decoded image buffer for a long time, serving as a reference image for multiple RAS images. Figure 6 is a schematic diagram of an exemplary CRR image. Figure 6 , a CRR image (such as CRR image 1 and CRR image 2) needs to be referenced by one or more similar RAS, so the CRR image needs to be a common stable content that can represent these similar RAS segments. In one example, the CRR image is extracted or generated from an existing video, and the CRR image is only used for reference and will not be displayed for playback. This type of CRR image can be called a non-displayable CRR image. In another example, the CRR image can also be a displayable image, that is, the displayable CRR image is used for reference and can be displayed for playback. It can be understood that the displayable CRR image itself is an image frame to be displayed in the RAS.

[0188] In order to ensure the random access function of each RAP (P frame encoding), the corresponding CRR image needs to be provided when decoding each RAS. In order to avoid the need to re-encode and transmit the CRR image every time a RAS is decoded, AVS3 designs a long-term buffer at the system layer to store the reconstructed CRR image. When a similar scene occurs again, the corresponding CRR image in the long-term buffer can be reused. In addition, in order to smooth the impact of transmission bandwidth and decoding capacity brought by large I frames, AVS3 makes the encoding and transmission of CRR images independent of the encoding and transmission of video data to be displayed. Then the bit stream after CRR image encoding and the bit stream after video frame encoding are respectively called CRR image bit stream and main bit stream (also called display image bit stream, not limited in this application).

[0189] Each picture in the main bitstream can parse the index of the CRR image referenced by the current frame in the system layer long-term buffer, and then the decoder uses the corresponding CRR image as the reference image of the current frame for inter-frame prediction.

[0190] It should be noted that the "image" involved in the embodiments of the present application can be replaced by an image frame. The "bit stream" involved in the embodiments of the present application can be replaced by a "code stream", and the "image slice" in the embodiments of the present application can be replaced by a "sub-image", which will not be repeated below.

[0191] The embodiment of the present application provides a coding and decoding method, which can effectively reduce time domain redundancy and improve video coding performance when encoding a video with a high degree of scene repetition.

[0192] Please refer to Figure 7 , Figure 7 A schematic diagram of a flow chart of an encoding method provided in an embodiment of the present application, which can be applied to an encoder, specifically includes but is not limited to the following steps:

[0193] S701, obtaining n CRR image slices of a plurality of display images and CRR images.

[0194] The multiple display images may be multiple continuous display images in a video sequence. The encoder may be set with a preset duration, divide the video sequence into at least one video segment according to the preset duration, and obtain a CRR image corresponding to the video segment. Multiple display images may constitute a video segment. The preset duration may be greater than the length of the current RAS, for example, the preset duration may be 6s, 7s or 10s, and accordingly, the multiple display images include multiple RAPs, each of which may be a P frame or a B frame.

[0195] A reconstructed CRR image can be used as a reference image for a subsequent plurality of display images to be encoded. Optionally, any one of the plurality of display images can be used as a CRR image corresponding to the plurality of display images. Alternatively, a CRR image corresponding to the plurality of display images can be obtained based on the plurality of display images. The embodiment of the present application does not limit the method for obtaining the CRR image.

[0196] In the embodiment of the present application, the encoder divides the CRR image into n CRR image slices (which may also be called CRR image slices or CRR image segments, etc., which are not limited in the present application). Figure 8 For an exemplary CRR image segmentation diagram, please refer to Figure 8 , the encoder may segment CRR image 1 and CRR image 2 to obtain image slices corresponding to CRR image 1, including CRR image slice 1-1 (also referred to as CRR image slice 1-1, not limited in this application), CRR image slice 1-2, and CRR image slice 1-3. Image slices corresponding to CRR image 2 include CRR image slice 2-1 (also referred to as CRR image slice 2-1, not limited in this application), CRR image slice 2-2, and CRR image slice 2-3.

[0197] S702, encode multiple display images and CRR images to obtain image bit streams; wherein the image bit stream includes: n CRR image slice bit streams and multiple display image bit streams in an interleaved order; each of the n CRR image slice bit streams includes CRR image identification information, and the CRR image identification information is used to indicate that the corresponding bit stream carries a CRR image slice.

[0198] The coded image bit stream is a single-channel bit stream. The CRR image is used to generate a reconstructed image, which can then be used as a reference image for the subsequent to-be-encoded display image to perform inter-frame prediction encoding on the to-be-encoded display image. The CRR image does not need to be displayed. After encoding a display image in a video sequence, a display image bit stream and a reconstructed display image are obtained. It can be seen that in the embodiment of the present application, the reconstructed image includes two types: a reconstructed display image and a reconstructed CRR image.

[0199] When encoding each of the multiple display images, the reference image of the display image includes: a reconstructed display image and / or a currently effective reconstructed CRR image. The currently effective reconstructed CRR image can be stored in the encoder's cache space, for example, in the encoder's DPB. The currently effective reconstructed CRR image refers to the reconstructed CRR image that is in effect at the moment when the encoding operation is performed on each of the multiple display images.

[0200] When performing an encoding operation on each display image among a plurality of display images, at least one display image bit stream has been encoded, and correspondingly, there is also at least one reconstructed display image. Optionally, any number of reconstructed display images among all the currently obtained reconstructed display images can be used as reference images of the display image. For example, the reference image of the display image includes: at least one reconstructed display image obtained by the encoding operation performed on at least one display image that was previously output most recently. For example, when performing an encoding operation on the third display image among a plurality of display images, the current encoding obtains the first display image bit stream and the first reconstructed display image corresponding to the first display image, and the second display image bit stream and the second reconstructed display image corresponding to the second display image. Therefore, when performing an encoding operation on the third display image among a plurality of display images, the first reconstructed display image and the second reconstructed display image can be used as reference images of the third display image.

[0201] The currently effective reconstructed CRR image is obtained by encoding the CRR image. Optionally, it can be obtained when the CRR image slice group is encoded. The CRR image slice bitstream may include but is not limited to: the latest output n CRR image slice bitstreams, or the latest output n CRR image slice bitstreams corresponding to the same CRR image. The latest output here refers to the latest output at the moment of encoding the display image.

[0202] During the encoding process of the entire video sequence, before the nth CRR image slice bitstream is output, there is no currently valid reconstructed CRR image. Therefore, for the display image bitstream output before the nth CRR image bitstream fragment, the reference image used during encoding only includes the reconstructed display image.

[0203] In the embodiment of the present application, the encoder encapsulates each generated bit stream (including the CRR image slice bit stream and the display image bit stream) into the NALU format. Figure 7 As shown, NALU includes a NALU Header field and a payload field (i.e., RSBP). Among them, the NALU Header field includes but is not limited to the NUT field. In an embodiment of the present application, the NUT field corresponding to each CRR image slice generated by the encoder includes CRR image identification information, which is used to indicate that the data carried by the payload field in the bit stream is a CRR image slice and its corresponding data. Optionally, in other embodiments, other fields in the NALU control field may also carry CRR image identification information, which is not limited in this application. In some embodiments, each code stream may also be encapsulated into other formats, and the CRR image identification information and other identification information in this article may be set in the control field of other format code streams to reduce the complexity of code stream identification.

[0204] In a possible implementation, the RAS in the embodiment of the present application refers to only one CRR image. The referenced CRR image may be a displayable CRR image or a non-displayable CRR image.

[0205] In another possible implementation, multiple RASs may refer to one CRR image.

[0206] In another possible implementation, the non-displayable CRR image bitstream (i.e., all CRR image slice bitstreams corresponding to the CRR image) precedes the RAS bitstream that references the CRR image. The displayable CRR image bitstream is within the RAS bitstream that references the CRR image. Specific examples can be found below.

[0207] In summary, the encoding method provided in the embodiment of the present application first obtains the CRR images corresponding to multiple display images, and then performs encoding operations on the CRR images and the multiple display images to obtain a coded image bit stream, wherein the coded image bit stream includes: n CRR image slice bit streams and multiple display image bit streams, where n is an integer greater than 1, and each CRR image slice bit stream includes CRR image identification information to identify that the corresponding bit stream carries the CRR image slice. In the embodiment of the present application, n CRR image bit stream segments and multiple display image bit streams are encoded into one bit stream for transmission. This method effectively reduces the transmission bandwidth while realizing cross-RAP reference, and through the CRR image identification, the decoder end can correctly identify the CRR image slice.

[0208] Please refer to Fig. 9 , Fig. 9 A flowchart of a decoding method provided in an embodiment of the present application is provided. The method can be applied to a decoder, and specifically includes but is not limited to the following steps:

[0209] S901, receiving an image bit stream, wherein the image bit stream includes: n CRR image slice bit streams and multiple display image bit streams; each of the n CRR image slice bit streams includes CRR image identification information, and the CRR image identification information is used to indicate that the corresponding bit stream carries a CRR image slice.

[0210] S902: Decode the n CRR image slice bit streams in the image bit stream based on the CRR image identification information.

[0211] S903, obtaining a CRR image based on the decoded n CRR image slices.

[0212] S904, decoding a plurality of display image bit streams to obtain a plurality of display images.

[0213] Exemplarily, the decoder can identify that the corresponding bitstream carries the CRR image slice based on the CRR image identification information in the CRR image slice bitstream. The decoder splices the received n CRR image slice bitstreams to obtain the CRR image bitstream. The decoder decodes the CRR image bitstream to obtain a reconstructed image corresponding to the CRR image (also referred to as a reconstructed CRR image).

[0214] Exemplarily, each time the decoder decodes a display image bit stream, a reconstructed display image is obtained. When decoding the display image bit stream, the reference image of the display image bit stream includes but is not limited to: a reconstructed display image and / or a reconstructed CRR image corresponding to the previous image. The reconstructed CRR image can be stored in the buffer space of the decoder, for example, in the DPB of the decoder, which is not limited in this application.

[0215] The reference image used when decoding any display image bit stream in the coded image bit stream is consistent with the reference image used by the encoder when encoding the display image bit stream.

[0216] The reconstructed display image is obtained by decoding the previous display image bit stream. A display image bit stream is decoded to obtain a reconstructed display image.

[0217] In summary, the decoding method provided by the embodiment of the present application first receives a coded image bit stream, the coded image bit stream includes: n CRR image bit stream segments and multiple display image bit streams, n is an integer greater than 1, and then decodes the CRR image slice bit stream and the display image bit stream in the coded image bit stream in turn to obtain multiple reconstructed display images. When the display image bit stream is decoded, the reference image of the display image bit stream includes: a reconstructed display image and / or a reconstructed CRR image. The reconstructed display image is obtained by decoding the previous display image bit stream, and the reconstructed CRR image is obtained by decoding the previous CRR image bit stream segment. The reconstructed CRR image can be used as a reference image of the current display image bit stream to be decoded. The reconstructed CRR image is obtained based on the CRR image slice bit stream. Since the CRR image corresponds to multiple display images, and the multiple display images may include multiple RAPs, cross-RAP reference can be achieved. In addition, n CRR image bitstream segments and multiple display image bitstreams are encoded as one bitstream and transmitted to the decoder, which can effectively reduce the transmission bandwidth while achieving cross-RAP reference. In addition, through the CRR image identifier, the decoder end can correctly identify the CRR image segmentation.

[0218] In the embodiment of the present application, as described above, the CRR image can be divided into a non-displayable CRR image and a displayable CRR image. In the embodiment of the present application, the CRR image slice bit stream corresponding to the non-displayable CRR image and the display image bit stream can be interleaved and sorted into one image bit stream (also referred to as a coded image bit stream). The CRR image slice bit stream corresponding to the displayable CRR image is arranged according to its original order in the RAS.

[0219] The following is a detailed description of the interleaving ordering scenario between the non-displayable CRR image bitstream and the display image bitstream. For example, please refer to Fig.10 , Fig.10 A schematic diagram of a coding and decoding process provided by an embodiment of the present application: The encoder encodes the CRR image and multiple display images to obtain a coded image bit stream and sends the coded image bit stream to the decoder. Fig. 9In the figure, three CRR image slice bit streams b1 to b3 (n=3) and five display image bit streams c1 to c5 are taken as examples for description. In other embodiments, more or less CRR image slice bit streams and display image bit streams may be included, and this application does not limit this.

[0220] Please continue to refer to Fig.10 The decoder decodes the bit streams in the coded image bit stream in sequence to obtain five reconstructed display images c1′ to c5′ and a reconstructed CRR image. The five reconstructed display images are used for display, and the reconstructed CRR image is only used as a reference image when decoding the display image bit stream and is not used for display.

[0221] For example, Fig.11 For an exemplary schematic diagram of a coding and decoding process, please refer to Fig.11 , the encoder divides the CRR image e into three CRR image slices e1, e2 and e3 (wherein the CRR image slices are the CRR image slices described in the embodiment of the present application, and will not be repeated below). The CRR image slice e1 is interspersed between the display image d1 and the display image d2, the CRR image slice e2 is interspersed between the display image d2 and the display image d3, and the CRR image slice e3 is interspersed between the display image d3 and the display image d4, to obtain an image sequence to be encoded.

[0222] After that, the encoder encodes the sequence of images to be encoded. First, the encoder encodes the display image d1. Since the new CRR image slice bitstream has not been output at this time, only the first reconstructed CRR image is stored in the cache space. The currently effective reconstructed CRR image is the first reconstructed CRR image. Select the reference image to encode the display image d1 to obtain the display image bitstream d1′ and the reconstructed display image ( Fig.11 (not shown). Next, the encoder performs encoding operation on the CRR image slice e1 to obtain a CRR image slice bit stream e1' and a reconstructed CRR image slice e1", and the encoder buffers the reconstructed CRR image slice e1".

[0223] The encoder performs encoding operation on the display image d2. Since there are no three reconstructed CRR image slices cached at this time, the currently effective reconstructed CRR image is still the first reconstructed CRR image. Select the reference image to perform encoding operation on the display image d2, and obtain the display image bit stream d2′ and the reconstructed display image ( Fig.11 (not shown). The encoder performs encoding operation on the CRR image slice e2 to obtain a CRR image slice bit stream e2' and a reconstructed CRR image slice e2", and the encoder buffers the reconstructed CRR image slice e2".

[0224] The encoder performs encoding operation on the display image d3. Since there are no three reconstructed CRR image slices cached at this time, the currently effective reconstructed CRR image is still the first reconstructed CRR image. Select the reference image to encode d3 to obtain the display image bit stream d3′ and the reconstructed display image ( Figure 8 (not shown) Encoding operation is performed on e3 to obtain CRR image bit stream segment e3' and reconstructed CRR image slice e3", and e3" is cached.

[0225] At this time, three reconstructed CRR image slices e1″ to e3″ are cached in the cache space. The second reconstructed CRR image e″ is obtained by splicing e1″ to e3″, and the first reconstructed CRR image is removed from the cache space. The currently effective reconstructed CRR image is the reconstructed CRR image e″.

[0226] The encoder performs encoding operation on the display image d4, and the currently effective reconstructed CRR image is the reconstructed CRR image e″ (also referred to as the second CRR image). Select the reference image to perform encoding operation on the display image d4, and obtain the display image bit stream d4′ and the reconstructed display image ( Fig.11 The encoder performs encoding operation on the display image d5, and the currently effective reconstructed CRR image is the reconstructed CRR image e″. A reference image is selected to perform encoding operation on the display image d5, and a display image bit stream d5′ and a reconstructed display image ( Fig.11 (Not shown) At this time, a coded image bit stream is obtained.

[0227] Still refer to Fig.11 In this example, the decoder does not perform decoding operations when receiving the first n-1 CRR image bit stream segments. Specifically, first, the decoder performs a decoding operation on the display image bit stream d1′. Since the new CRR image bit stream segment has not been decoded at this time, only the first reconstructed CRR image is stored in the cache space, and the currently effective reconstructed CRR image is the first reconstructed CRR image. A reference image is selected to perform a decoding operation on the display image d1′ to obtain a reconstructed display image d1″. The reference image selected by the encoder for decoding d1′ is the same as the reference image selected for encoding d1. The decoder caches the received CRR image slice bit stream e1′.

[0228] The decoder performs a decoding operation on the display image bit stream d2′. Since three reconstructed CRR image slices have not been cached at this time, the currently effective reconstructed CRR image is still the first reconstructed CRR image. A reference image is selected to perform a decoding operation on the display image bit stream d2′ to obtain a reconstructed display image d2″. The reference image selected by the decoder for the decoding operation on d2′ is the same as the reference image selected for the encoding operation on d2. The decoder caches the received CRR image slice bit stream e2′.

[0229] The decoder performs a decoding operation on d3′. Since three reconstructed CRR image slices have not been cached at this time, the currently effective reconstructed CRR image is still the first reconstructed CRR image. A reference image is selected to perform a decoding operation on d3′ to obtain a reconstructed display image d3″. The reference image selected for the decoding operation on d3′ is the same as the reference image selected for the encoding operation on d3. The decoder caches the received CRR image slice bit stream e3′.

[0230] At this time, three CRR image slice bit streams e1′ to e3′ are cached in the cache space. The decoder splices e1′ to e3′ to obtain a CRR image bit stream. The decoder decodes the CRR image bit stream to obtain a second reconstructed CRR image e″. At the same time, the first reconstructed CRR image is removed from the cache space. The currently effective reconstructed CRR image is the reconstructed CRR image e″.

[0231] The decoder performs a decoding operation on the display image bit stream d4′, and the currently effective reconstructed CRR image is e″. A reference image is selected to perform a decoding operation on the display image d4′, and a reconstructed display image d4″ is obtained. The reference image selected by the decoder for the decoding operation on the display image bit stream d4′ is the same as the reference image selected for the encoding operation on the display image d4. Next, the decoder performs a decoding operation on the display image bit stream d5′, and the currently effective reconstructed CRR image is e″. A reference image is selected to perform a decoding operation on the display image bit stream d5′, and a reconstructed display image d5″ is obtained. The reference image selected for the decoding operation on the display image bit stream d5′ is the same as the reference image selected for the encoding operation on the display image d5.

[0232] As described above, in the embodiment of the present application, during the encoding and decoding process, the non-displayable CRR image bit stream is transmitted before the RAS that references the CRR image. Several possible CRR image bit stream interleaving and sorting methods are described below in conjunction with the accompanying drawings. Fig.12a For an exemplary diagram of an interleaving arrangement, please refer to Fig.12a , the reference image of at least one display image in RAS2 is CRR image 1. In the image bit stream generated by the encoder, the CRR image slices corresponding to CRR image 1 (for example, including but not limited to CRR image slice 1-1, CRR image slice 1-2, CRR image slice 1-3) and the display images in RAS1 (for example, including but not limited to frame 1, frame 2, frame 3 and frame 4) are interleaved (the corresponding bit streams are also interleaved). Figure 12b For an exemplary diagram of an interleaving arrangement, please refer to Figure 12b, the reference image of at least one display image in RAS2 is CRR image 1. The reference image of at least one display image in RAS3 is CRR image 2. Accordingly, in the image bit stream generated by encoding by the encoder, the CRR image slices corresponding to CRR image 1 (for example, including but not limited to CRR image slice 1-1, CRR image slice 1-2, CRR image slice 1-3) are interleaved and sorted with the display images in RAS1 (for example, including but not limited to frame 1, frame 2, frame 3 and frame 4) (the corresponding bit streams are also interleaved and sorted). The CRR image slices corresponding to CRR image 2 (for example, including but not limited to CRR image slice 2-1, CRR image slice 2-2, CRR image slice 2-3) are interleaved and sorted with the display images in RAS2 (for example, including but not limited to frame 5, frame 6, frame 7 and frame 8) (the corresponding bit streams are also interleaved and sorted). Fig.12c For an exemplary diagram of an interleaving arrangement, please refer to Fig.12c , the reference image of at least one display image in RAS2 is CRR image 2. The reference image of at least one display image in RAS3 is CRR image 2. Accordingly, in the image bitstream generated by encoding by the encoder, the CRR image slices corresponding to CRR image 2 (for example, including but not limited to CRR image slice 2-1, CRR image slice 2-2, CRR image slice 2-3) are arranged continuously. In other words, the displayable CRR image slice bitstream is not interleaved with the display image bitstream.

[0233] In an embodiment of the present application, the CRR image identification information may include different types of CRR image identification information, so as to distinguish between displayable CRR images and non-displayable CRR images, so that the decoder end can identify the displayable CRR images and non-displayable CRR images based on the CRR image identification information, and perform corresponding decoding operations on the displayable CRR images and perform corresponding decoding operations on the non-displayable CRR images. Optionally, the decoding operation of the displayable CRR images and the decoding operation of the non-displayable CRR images at the decoder end are different.

[0234] In a possible implementation, a display image can be used as a random access point in the RAS, and a non-displayable image cannot be used as a random access point.

[0235] In another possible implementation, the n CRR image slices of the CRR image may also be interleaved and sequenced with multiple RASs, wherein the multiple RASs are before the RAS that references the CRR image.

[0236] The application scenarios of different CRR image identification information are described in detail below with specific embodiments.

[0237] 1) CRR image identification information includes:

[0238] CRR NUT is used to identify the non-border CRR image slice bit stream corresponding to the non-displayable CRR image, and can also be understood as indicating that the corresponding code stream carries the non-border CRR image slice of the non-displayable CRR image.

[0239] CRR_Delimiter NUT is used to indicate the boundary CRR image slice bit stream of the non-displayable CRR image. It can also be understood as indicating that the corresponding code stream carries the boundary CRR image slice of the non-displayable CRR image. The boundary CRR image slice bit stream in the embodiment of the present application includes the first CRR image slice bit stream and the last CRR image slice bit stream in the CRR image slice. For example, Fig.12a As shown in , the slice bit stream corresponding to CRR image slice 1-1 is the first CRR image slice bit stream. The slice bit stream corresponding to CRR image slice 1-3 is the last CRR image slice bit stream. The slice bit stream corresponding to CRR image slice 1-2 is the non-boundary CRR image slice bit stream.

[0240] RL NUT is used to identify a P frame or B frame that only refers to CRR. This image frame is usually a random access point in RAS. It can be understood that before the RL image bitstream carrying this identification information, all CRR image slices of the CRR image referenced by the RL image have been encoded or transmitted. For example Fig.12a As shown, frame 5 is the first RAP image that references CRR image 1, and this frame is an RL image, so the bitstream corresponding to this image contains the RL NUT identifier. Optionally, the bitstream corresponding to other RAP images that reference this CRR image may also carry the RL NUT identifier, or may not, which is not limited in this application. Optionally, in other embodiments, if the agreed image frame is not an RL image, the display image of the first reference CRR image may be marked with other marks (such as NUT), which is not limited in this application.

[0241] CRR_DP NUT is used to identify the CRR image slice bit stream that can display the CRR image. It can also be understood as indicating that the corresponding code stream carries the image slice that can display the CRR image.

[0242] In the embodiment of the present application, the first CRR image slice bit stream of the CRR image is after the header information bit stream of the CRR image, and the first CRR image slice bit stream is adjacent to the header information bit stream of the CRR image. Also, there is no bit stream of other CRR images between the display image bit stream corresponding to the first display image (usually RAP) of the reference CRR image and the CRR image.

[0243] The following is a detailed description of the actual application scenario based on the above CRR image identification:

[0244] Fig.13 The flowchart of an encoding method is shown as an example. The method can be applied to an encoder, and specifically includes but is not limited to the following steps:

[0245] S1301, acquiring a CRR image and multiple display images.

[0246] Exemplarily, the encoder obtains a CRR image. As described above, the CRR images in the embodiments of the present application include non-displayable CRR images and displayable CRR images. The CRR image obtained by the encoder is a displayable CRR image or a non-displayable CRR image. Among them, the non-displayable CRR image can be a certain image to be encoded, or it can be an image constructed from multiple images to be encoded. The non-displayable CRR image does not need to be displayed, and is only used as a reference image for inter-frame prediction encoding of the image to be encoded. The displayable CRR image is a certain image to be encoded, and it needs to be displayed. For specific description, please refer to the above, which will not be repeated here.

[0247] Exemplarily, the encoder acquires multiple display images. The acquisition method can refer to the existing technology and is not limited in this application.

[0248] S1302: Divide the CRR image into n CRR image slices.

[0249] Exemplarily, the encoder segments the CRR image to obtain n CRR image slices (also referred to as CRR image slices, which will not be described again in the following). Figure 8 The relevant description will not be repeated here.

[0250] S1303: Acquire a sequence of images to be encoded based on n CRR image slices and multiple display images.

[0251] In the embodiment of the present application, for a non-displayable CRR image, the encoder may interleave and sort the n CRR image slices of the non-displayable CRR image with a plurality of display images. The implementation method may refer to Fig.11 , I will not go into details here.

[0252] For a displayable CRR image, the encoder arranges n CRR image slices in sequence.

[0253] S1304, encode the image sequence to be encoded to obtain an image bit stream. The boundary CRR image slice bit stream of the non-displayable CRR image includes a CRR_Delimiter NUT identifier, and the non-boundary CRR image slice bit stream includes a CRR NUT identifier. The CRR image slice bit stream of the displayable CRR image includes a CRR_DP NUT identifier. And the RL image bit stream of the reference CRR image includes an RL NUT identifier.

[0254] In the embodiment of the present application, the encoder encodes the image sequence to be encoded to obtain an image bit stream. The encoding method can be referred to Fig.11 The description in will not be repeated here.

[0255] In an embodiment of the present application, the encoder performs NALU encapsulation on each code stream (including CRR image slice bit stream and display image bit stream) to obtain a NALU bit stream. In addition, the encoder performs NALU encapsulation on the header information of the CRR image (including SPS, PPS, PH, etc.) to obtain a CRR image header information bit stream (including but not limited to CRR image SPS bit stream, CRR image PPS bit stream, and CRR image PH bit stream). In addition, the encoder performs NALU encapsulation on the header information corresponding to each display image to obtain a display image header information bit stream.

[0256] The image code stream corresponding to the non-displayable CRR image and the code stream corresponding to the displayable CRR image are described in detail below.

[0257] a. Unable to display CRR image stream:

[0258] Fig.14 : is a schematic diagram of the image code stream structure of an undisplayable CRR image. In this example, Fig.12a Please refer to Fig.14 The NUT field of the first RL picture bitstream includes the RLNUT identifier. Optionally, if the picture does not refer to the previous CRR picture, the RL NUT identifier may not be carried. The RBSP (ie, the payload field) of the first RL picture includes the RL picture coding data and the like.

[0259] Exemplarily, the NUT field of the CRR image slice 1-1 bit stream includes a CRR_Delimiter NUT identifier, which is used to indicate that the payload field of the bit stream carries the boundary slice image (corresponding coded data) of the CRR image. Fig.14 As shown, CRR image slice 1-1 is the first CRR image slice of the CRR image in the coding sequence. Accordingly, in the coded bit stream, before the first CRR image slice bit stream and adjacent to the first CRR image slice bit stream, the header information bit stream of the CRR image is also included. Optionally, different types of header information bit streams can be encapsulated into different NALUs respectively, or they can be encapsulated into the same NALU, which is not limited in this application.

[0260] Please continue to refer to Fig.14 ,according to Fig.12aIn the interleaved sorting method shown, the CRR image slice 1-1 bit stream is followed by the display image bit stream. The position before and adjacent to the display image bit stream may include the display image header information bit stream ( Fig.14 not shown).

[0261] Exemplarily, according to the order in the sequence of images to be encoded, the display image bitstream is followed by the CRR image slice 1-2 bitstream. The NUT field of the bitstream includes a CRR_NUT flag, which is used to indicate that the payload field of the codestream carries the encoded data of the non-border image slices of the CRR image.

[0262] The bit streams of other images (including display images and CRR image slices) are similar and will not be described again here.

[0263] Still refer to Fig.14 , CRR image slices 1-4 are the last CRR image slices of the CRR image, and accordingly, the NUT field of the CRR image slices 1-4 bitstream includes the CRR_Delimiter NUT identifier. Fig.12a In the interleaved sorting method shown, the CRR image slices 1-4 bit streams are followed by the display image bit stream.

[0264] Exemplarily, the second RAP image is a predetermined image (i.e., it is predetermined that the transmission of the CRR image is completed before this image), that is, the first RAP image that references CRR image 1. Accordingly, the NUT field of the RL image bitstream includes an RL NUT identifier, which is used to indicate that the codestream carries the coded data of the predetermined RL image.

[0265] b.Can display CRR image stream:

[0266] Fig.15 The schematic diagram of the image code stream structure of the CRR image to be displayed is shown as an example. In this example, Fig.12c Please refer to Fig.15 In this example, the bit stream corresponding to the RL image of the reference CRR image may include the RL NUT identifier or may not include it, which is not limited in this application.

[0267] Exemplarily, as described above, in the image sequence to be encoded, n CRR image slices are arranged continuously, and correspondingly, in the image code stream, n CRR image slice bit streams are also arranged continuously. Fig.15As shown, the CRR image slice 1-1 bit stream and the CRR image slice 1-2 bit stream are arranged continuously. Moreover, the NUT field of the CRR image slice 1-1 bit stream and the CRR image slice 1-2 bit stream includes a CRR_DP NUT identifier, which is used to indicate that the corresponding code stream payload field carries the coded data of the image slice corresponding to the CRR image. The other CRR image slice bit streams are the same as the CRR image slice 1-2 bit stream, and are not described one by one here.

[0268] For example, according to Fig.12c The coded image sequence, CRR image slices 1-4 bit stream is followed by the display image bit stream. For a detailed description, please refer to Fig.14 , I will not go into details here.

[0269] In a possible implementation manner, the SPS, PPS or PH information of the CRR image may include information on the number of image slices, which is used to indicate the number of image slices after the CRR image is segmented.

[0270] In another possible implementation, if the number of slices corresponding to the undisplayable CRR image is 1, the CRR image identifier in the CRR image slice code stream is a CRR NUT identifier.

[0271] Fig.16 The flowchart of a decoding method is shown as an example, and the method can be applied to a decoder, and specifically includes: the decoder receives an image code stream, and the decoder performs NALU decapsulation on the NALU to read the NUT field in the NALU.

[0272] Exemplarily, the decoder receives a display image bit stream that does not include CRR image identification information, and can decode the display image bit stream. Fig.11 The description in will not be repeated here.

[0273] In one example, if the decoder reads the NUT field in the NALU bitstream and finds that the NUT field includes CRR_Delimiter NUT identification information, the decoder determines that the NALU bitstream is a CRR_Delimiter NALU, that is, a CRR image slice bitstream that cannot display the boundary of the CRR image. The decoder can obtain and cache the CRR image slices.

[0274] Furthermore, the decoder switches the CRR image receiving state to a state of "receiving non-displayable CRR image slice bit stream". The CRR image receiving state may be a default state before switching, which is not limited in the present application.

[0275] Exemplarily, the decoder continues to receive the image code stream and decodes the received display code stream. Also, if the decoder reads the NUT field in the NALU bit stream including CRR NUT identification information, it determines that the NALU bit stream is CRRNALU, that is, a non-boundary CRR image slice bit stream that cannot display the CRR image. The decoder obtains and caches the CRR image slices.

[0276] Exemplarily, if the decoder reads the NUT field in the NALU bitstream again and includes the CRR_Delimiter NUT identification information, it can be determined that the CRR_Delimiter NALU is received again. The decoder obtains and caches the CRR image fragments. In addition, the decoder can determine that the CRR image reception is completed, and the decoder switches the CRR image reception state to the default state, that is, ends the "receiving non-displayable CRR image fragment bitstream" state.

[0277] In a possible implementation, if the decoder reads the RL NALU containing the RL NUT identification information in the "receiving non-displayable CRR image slice bitstream" state. It can also be understood that the RL NUT identification information is read before the CRR_DelimiterNUT identification information is read again. It can also be understood that the RL image that references the CRR image is received before the end CRR image slice of the CRR image is received. Then, the decoder determines that the CRR image reception has failed. That is to say, in this example, there is no other CRR image bitstream between the CRR image bitstream and the RL image bitstream.

[0278] For example, Fig.17 For an exemplary image code stream diagram, please refer to Fig.17 In this example, normally, the structure of the image bitstream should be as follows Fig.14 As shown. However, due to other reasons, the CRR image slice 1-2 bit stream is followed by the RL image bit stream. Accordingly, after receiving the CRR image slice 1-1 bit stream, the decoder switches the CRR image reception state to the "receiving non-displayable CRR image slice bit stream" state. After receiving the RL image bit stream, it is recognized that the NUT field of the bit stream includes the RL NUT identifier, that is, it is determined that the RL image that references the CRR image is received. Then, the decoder determines that the CRR image reception has failed, that is, the CRR image reception is incomplete. Optionally, the decoder can adopt a response method such as directly discarding, skipping the RAS bit stream that needs to reference this CRR image, and requesting retransmission. The specific method can be set according to actual needs, and this application is not limited.

[0279] Still refer to Fig.16In another example, when the decoder reads the NUT field in the NALU bitstream including the CRR DPNUT identification information, it determines that the NALU bitstream is a CRR DP NALU, that is, a boundary CRR image slice bitstream that can display the CRR image. The decoder decodes the CRR image slice. In addition, the decoder switches the CRR image receiving state to the "receiving displayable CRR image slice bitstream" state.

[0280] As described above, the image slice bit streams that can display the CRR image are arranged continuously. Accordingly, the decoder decodes the corresponding CRR image slice bit streams in sequence in the "receiving displayable CRR image slice bit stream" state. When the decoder recognizes a bit stream that does not include the CRR DP NUT identifier, it can be determined that the CRR image reception is complete (it may be complete or incomplete). The decoder switches the CRR image reception state to the default state, that is, ends the "receiving displayable CRR image slice bit stream" state.

[0281] Exemplarily, the decoder splices the decoded reconstructed CRR image slices to obtain the reconstructed CRR image.

[0282] That is to say, in an embodiment of the present application, the decoder can identify non-displayable CRR images and displayable CRR images based on different CRR image identification information. For different types of CRR images, the decoder can adopt different reception completion judgment conditions and different decoding methods. For example, for non-displayable CRR images, the decoder can first cache the image slice bit stream, splice the image slice bit stream, and then decode the CRR image bit stream. For displayable CRR images, the decoder decodes each CRR image slice bit stream it receives, and obtains the corresponding CRR image based on the decoded image slice.

[0283] In an embodiment of the present application, as described above, the header information bit stream of the CRR image includes the number of CRR image slices. After the decoder determines that the CRR image reception is complete, that is, the state is switched to the default state, it can detect whether the CRR image slices are received completely based on the number of CRR image slices received. Optionally, the decoder can start the counter after receiving the first CRR image slice bit stream, and if a CRR image slice bit stream is not received, the counter is increased by 1. In one example, if the number of received CRR image slice bit streams (i.e., the count result of the counter) is equal to the number of CRR image slices, it can be determined that the reception is complete. In another example, if the number of received image slice bit streams is not equal to the number of CRR image slices, it can be determined that the reception is incomplete. The decoder can take measures such as directly discarding, skipping the RAS bit stream that needs to refer to this CRR image, and requesting retransmission. The specific method can be set according to actual needs and is not limited by this application.

[0284] In a possible implementation, the process of decapsulating the NALU and identifying the identification information may also be performed by other modules of the electronic device, for example, by the system layer, and after the system layer determines that a complete CRR image has been received, the encoded data of the CRR image slices carried in the bitstream may be output to the decoder. The decoder may splice and decode the encoded data of the CRR image.

[0285] In another possible implementation, when the decoder is in the "receiving non-displayable CRR image slice bit stream" state, if it receives the header information bit stream and CRR_Delimiter NALU (i.e., the boundary CRR image slice bit stream of another non-displayable CRR image) (which may also be CRR DP NUT) of another CRR image again, the decoder may determine that the bit stream transmission of the currently received CRR image is finished, and the decoder may continue to receive the image slice bit stream corresponding to another CRR image in the "receiving non-displayable CRR image slice bit stream" state. It should be noted that in the embodiment of the present application, the fact that other CRR image bit streams are not included between the CRR image bit stream described above and the RL bit stream that references the CRR image can be understood as that other CRR image bit streams cannot be inserted before the CRR image transmission is completed (i.e., the CRR_Delimiter NUT identifier is recognized again).

[0286] In another possible implementation, the system layer processes the image sequence (including the CRR image and the display image) decoded by the decoder, including but not limited to:

[0287] 1. When RL NALU, CR_DP NALU or other display image NALU is received, it means that the received image needs to be output after being decoded into a reconstructed image. When encapsulated with the system layer protocol, operations such as timestamping can be performed. When storing, each image bit stream can be stored in the order in which the code stream is received.

[0288] 2. When a CRR NALU or CRR_Delimiter NALU is received, it indicates that the received bitstream does not need to be output after being decoded into a reconstructed image, and the bitstream of the image may be divided into multiple bitstream slices and transmitted interleaved with the codestream of the display image. The system layer may choose to perform some special processing on the image, such as building a table to index the offset of each bitstream slice in the file storage or the bitstream of the entire coding sequence, to facilitate subsequent indexing and searching of all bitstream slices of the image.

[0289] 3. When a random access occurs to a video bitstream with a CRR image bitstream, the system layer searches for the nearest random access point near the location of the requested random placement point. If an RL NALU or CRR_DP NALU is found, the location can be used as a random access point. If the searched random access point is an RL NALU, the system layer searches for CRR NALU and CRR_Delimiter NALU from the NALU of the RL image forward. When a CRR_Delimiter NALU is followed by a header information NALU, these header information NALUs and all CRR_Delimiter NALUs and CRR NALUs are collected in the order in the bitstream for splicing and reorganization and input into the decoder for decoding. The decoded CRR image is used as the reference image of the required random access point RL image.

[0290] 2) The CRR image identification information includes: CRR NUT, RL NUT and CRR_DP NUT. The specific description can be referred to above and will not be repeated here.

[0291] The following is a detailed description of the actual application scenario based on the above CRR image identification:

[0292] Fig.18 The flowchart of an encoding method is shown as an example. The method can be applied to an encoder, and specifically includes but is not limited to the following steps:

[0293] S1801, acquiring a CRR image and multiple display images.

[0294] S1802, dividing the CRR image into n CRR image slices.

[0295] S1803, obtaining a sequence of images to be encoded based on n CRR image slices and multiple display images.

[0296] S1801~S1803 can refer to S1301~S1303, and will not be repeated here.

[0297] S1804, encode the image sequence to be encoded to obtain an image bit stream. The CRR image slice bit stream of the non-displayable CRR image includes a CRR NUT identifier. The CRR image slice bit stream of the displayable CRR image includes a CRR_DP NUT identifier. And the RL image bit stream of the reference CRR image includes an RL NUT identifier.

[0298] In the embodiment of the present application, the encoder encodes the image sequence to be encoded to obtain an image bit stream. The encoding method can be referred to Fig.11 The description in will not be repeated here.

[0299] In an embodiment of the present application, the encoder performs NALU encapsulation on each code stream (including CRR image slice bit stream and display image bit stream) to obtain a NALU bit stream. In addition, the encoder performs NALU encapsulation on the header information of the CRR image (including SPS, PPS, PH, etc.) to obtain a CRR image header information bit stream (including but not limited to CRR image SPS bit stream, CRR image PPS bit stream, and CRR image PH bit stream). In addition, the encoder performs NALU encapsulation on the header information corresponding to each display image to obtain a display image header information bit stream.

[0300] The image code stream corresponding to the non-displayable CRR image and the code stream corresponding to the displayable CRR image are described in detail below.

[0301] a. Unable to display CRR image stream:

[0302] Fig.19 : is a schematic diagram of the image code stream structure of an undisplayable CRR image. In this example, Fig.12a Please refer to Fig.19 The NUT field of the first RL picture bitstream includes the RLNUT identifier. Optionally, if the picture does not refer to the previous CRR picture, the RL NUT identifier may not be carried. The RBSP (ie, the payload field) of the first RL picture includes the RL picture coding data and the like.

[0303] Exemplarily, the NUT field of the CRR image slice 1-1 bit stream includes a CRR NUT identifier, which is used to indicate that the payload field of the bit stream carries the image slice of the CRR image (corresponding coded data). Fig.19 As shown, CRR image slice 1-1 is the first CRR image slice of the CRR image in the coding sequence. Accordingly, in the coded bit stream, before the first CRR image slice bit stream and adjacent to the first CRR image slice bit stream, the header information bit stream of the CRR image is also included. Optionally, different types of header information bit streams can be encapsulated into different NALUs respectively, or they can be encapsulated into the same NALU, which is not limited in this application.

[0304] Please continue to refer to Fig.19 ,according to Fig.12a In the interleaved sorting method shown, the CRR image slice 1-1 bit stream is followed by the display image bit stream. The position before and adjacent to the display image bit stream may include the display image header information bit stream ( Fig.14 not shown).

[0305] Exemplarily, according to the order in the sequence of images to be encoded, the display image bitstream is followed by the CRR image slice 1-2 bitstream. The NUT field of the bitstream includes a CRR NUT identifier, which is used to indicate that the payload field of the codestream carries the encoded data of the image slice of the CRR image.

[0306] The bit streams of other images (including display images and CRR image slices) are similar and will not be described again here.

[0307] Exemplarily, the second RAP image is a predetermined image (i.e., it is predetermined that the transmission of the CRR image is completed before this image), that is, the first RAP image that references CRR image 1. Accordingly, the NUT field of the RL image bitstream includes an RL NUT identifier, which is used to indicate that the codestream carries the coded data of the predetermined RL image.

[0308] b.Can display CRR image stream:

[0309] For a detailed description of the CRR code stream that can be displayed, please refer to Fig.15 , I will not go into details here.

[0310] Fig. 20 The flowchart of a decoding method is shown as an example, and the method can be applied to a decoder, and specifically includes: the decoder receives an image code stream, and the decoder performs NALU decapsulation on the NALU to read the NUT field in the NALU.

[0311] Exemplarily, the decoder receives a display image bit stream that does not include CRR image identification information, and can decode the display image bit stream. Fig.11 The description in will not be repeated here.

[0312] In one example, if the decoder reads a NUT field in a NALU bitstream that includes a CRR NUT identifier, the decoder determines that the NALU bitstream is a CRR NALU, that is, a CRR image slice bitstream that cannot display a CRR image. The decoder can obtain and cache the CRR image slice. In addition, the decoder starts a counter to start counting.

[0313] Exemplarily, the decoder continues to receive the image code stream and decodes the received display code stream. Also, if the decoder reads the NUT field in the NALU bit stream and includes CRR NUT identification information, it determines that the NALU bit stream is CRRNALU, that is, a CRR image slice bit stream that cannot display the CRR image. The decoder obtains and caches the CRR image slice. Also, the decoder adds 1 to the counter.

[0314] Exemplarily, the decoder compares the count result of the counter with the number n of CRR image slices included in the header information bit stream of the CRR image. If it is less than n, it is determined that the CRR image is not received completely. If it is equal to n, it is determined that the CRR image is received completely. The decoder can splice the CRR image slice bit streams to obtain the CRR image bit stream, and decode the CRR image bit stream to obtain the reconstructed CRR image.

[0315] In one possible implementation, if the decoder reads the RL NALU containing the RL NUT identification information, and the count result of the counter is currently less than n, the decoder determines that the CRR image reception fails (i.e., the reception is incomplete). Optionally, the decoder can adopt a response method such as directly discarding, skipping the RAS bit stream that needs to refer to this CRR image, and requesting retransmission. The specific method can be set according to actual needs and is not limited in this application.

[0316] Still refer to Fig. 20 In another example, if the decoder reads the NUT field in the NALU bit stream including the CRR DPNUT identification information, the decoder determines that the NALU bit stream is a CRR DP NALU, which can display the boundary CRR image slice bit stream of the CRR image. The decoder decodes the CRR image slice. In addition, the decoder starts the counter to start counting.

[0317] As described above, the image slice bit streams that can display the CRR image are arranged continuously. Accordingly, the decoder decodes the corresponding CRR image slice bit streams in sequence, and the counter increases by 1 each time a CRR DP NALU is received.

[0318] When the decoder identifies a bit stream that does not include the CRR DP NUT identifier, it can be determined that the CRR image reception is complete (which may be complete or incomplete). Exemplarily, if the technical result of the counter is equal to n, the decoder determines that the CRR image reception is complete. The decoder image can splice the reconstructed CRR image slices to obtain a reconstructed CRR image.

[0319] In one possible implementation, if the decoder reads a bit stream that does not include the CRR DP NUT identifier, and the count result of the counter is currently less than n, the decoder determines that the CRR image reception has failed (i.e., the reception is incomplete). Optionally, the decoder can adopt a response method such as directly discarding, skipping the RAS bit stream that needs to refer to this CRR image, and requesting retransmission. The specific method can be set according to actual needs and is not limited in this application.

[0320] That is to say, in an embodiment of the present application, the decoder can identify non-displayable CRR images and displayable CRR images based on different CRR image identification information. For different types of CRR images, the decoder can adopt different reception completion judgment conditions and different decoding methods. For example, for non-displayable CRR images, the decoder can first cache the image slice bit stream, splice the image slice bit stream, and then decode the CRR image bit stream. For displayable CRR images, the decoder decodes each CRR image slice bit stream it receives, and obtains the corresponding CRR image based on the decoded image slice.

[0321] In one possible implementation, when the decoder receives the current CRR image, it receives the header information bit stream and CRR image identifier (which can be CRR NUT or CRR DP NUT) of another CRR image again. The decoder can determine that the bit stream transmission of the currently received CRR image is completed, and the decoder continues to receive the image slice bit stream corresponding to another CRR image, and the counter is reset and counted again.

[0322] In another possible implementation, the system layer processes the image sequence (including the CRR image and the display image) decoded by the decoder, including but not limited to:

[0323] 1. When a display image NALU or a displayable CRR image NALU is received, the operation is the same as in the above embodiment.

[0324] 2. When a CRR NALU is received, it means that the received image does not need to be output after being decoded into a reconstructed image, and the operation is the same as in the above embodiment.

[0325] 3. When a random access occurs to a video bitstream with a CRR image bitstream, the system layer searches for the nearest random access point near the location of the requested random placement point. If an RL NALU or CRR_DP NALU is found, the location can be used as a random access point. If the searched random access point is an RL NALU, the system layer searches for the CRR NALU from the NALU of the RL image forward. When a NUT is a CRR NUT NALU followed by a header information NALU, these header information NALUs and all CRR NALUs are collected in the order of the bitstream, spliced ​​and reorganized, and input into the decoder for decoding. The decoded CRR image is used as the reference image of the required random access point RL image.

[0326] 3)CRR image identification information includes:

[0327] CRR NUT, RL NUT and CRR_DP NUT. The specific description can be referred to above and will not be repeated here.

[0328] CRR_START NUT is used to indicate the first CRR picture slice bit stream of a non-displayable CRR picture.

[0329] CRR_END NUT is used to indicate the end CRR picture slice bit stream of a non-displayable CRR picture.

[0330] The following is a detailed description of the actual application scenario based on the above CRR image identification:

[0331] Fig.21 The flowchart of an encoding method is shown as an example. The method can be applied to an encoder, and specifically includes but is not limited to the following steps:

[0332] S2101, acquiring a CRR image and multiple display images.

[0333] S2102, dividing the CRR image into n CRR image slices.

[0334] S2103, obtaining a sequence of images to be encoded based on n CRR image slices and multiple display images.

[0335] S2101~S2103 can refer to S1301~S1303, and will not be repeated here.

[0336] S2104, encode the image sequence to be encoded to obtain an image bit stream. The first CRR image slice bit stream of the non-displayable CRR image includes a CRR_START NUT identifier, the last CRR image slice bit stream includes a CRR_END NUT identifier, and the non-border CRR image slice bit stream includes a CRR NUT identifier. The CRR image slice bit stream of the displayable CRR image includes a CRR_DP NUT identifier. In addition, the RL image bit stream of the reference CRR image includes an RL NUT identifier.

[0337] In the embodiment of the present application, the encoder encodes the image sequence to be encoded to obtain an image bit stream. The encoding method can be referred to Fig.11 The description in will not be repeated here.

[0338] In an embodiment of the present application, the encoder performs NALU encapsulation on each code stream (including CRR image slice bit stream and display image bit stream) to obtain a NALU bit stream. In addition, the encoder performs NALU encapsulation on the header information of the CRR image (including SPS, PPS, PH, etc.) to obtain a CRR image header information bit stream (including but not limited to CRR image SPS bit stream, CRR image PPS bit stream, and CRR image PH bit stream). In addition, the encoder performs NALU encapsulation on the header information corresponding to each display image to obtain a display image header information bit stream.

[0339] The image code stream corresponding to the non-displayable CRR image and the code stream corresponding to the displayable CRR image are described in detail below.

[0340] a. Unable to display CRR image stream:

[0341] Fig. 22 : is a schematic diagram of the image code stream structure of an undisplayable CRR image. In this example, Fig.12a Please refer to Fig. 22 The NUT field of the first RL picture bitstream includes the RLNUT identifier. Optionally, if the picture does not refer to the previous CRR picture, the RL NUT identifier may not be carried. The RBSP (ie, the payload field) of the first RL picture includes the RL picture coding data and the like.

[0342] Exemplarily, the NUT field of the CRR image slice 1-1 bit stream includes a CRR_START NUT flag, which is used to indicate that the payload field of the bit stream carries the first image slice (corresponding coded data) of the CRR image. Fig. 22 As shown, CRR image slice 1-1 is the first CRR image slice of the CRR image in the coding sequence. Accordingly, in the coded bit stream, before the first CRR image slice bit stream and adjacent to the first CRR image slice bit stream, the header information bit stream of the CRR image is also included. Optionally, different types of header information bit streams can be encapsulated into different NALUs respectively, or they can be encapsulated into the same NALU, which is not limited in this application.

[0343] Please continue to refer to Fig. 22 ,according to Fig.12a In the interleaved sorting method shown, the CRR image slice 1-1 bit stream is followed by the display image bit stream. The position before and adjacent to the display image bit stream may include the display image header information bit stream ( Fig. 22 not shown).

[0344] Exemplarily, according to the order in the sequence of images to be encoded, the display image bitstream is followed by the CRR image slice 1-2 bitstream. The NUT field of the bitstream includes a CRR_NUT flag, which is used to indicate that the payload field of the codestream carries the encoded data of the non-border image slices of the CRR image.

[0345] The bit streams of other images (including display images and CRR image slices) are similar and will not be described again here.

[0346] Still refer to Fig. 22, CRR image slices 1-4 are the end CRR image slices of the CRR image, and accordingly, the NUT field of the CRR image slices 1-4 bitstream includes the CRR_END NUT identifier. Fig.12a In the interleaved sorting method shown, the CRR image slices 1-4 bit streams are followed by the display image bit stream.

[0347] Exemplarily, the second RAP image is a predetermined image (i.e., it is predetermined that the transmission of the CRR image is completed before this image), that is, the first RAP image that references CRR image 1. Accordingly, the NUT field of the RL image bitstream includes an RL NUT identifier, which is used to indicate that the codestream carries the coded data of the predetermined RL image.

[0348] b.Can display CRR image stream:

[0349] For detailed description, please refer to Fig.15 , I will not go into details here.

[0350] Fig.23 The flowchart of a decoding method is shown as an example, and the method can be applied to a decoder, and specifically includes: the decoder receives an image code stream, and the decoder performs NALU decapsulation on the NALU to read the NUT field in the NALU.

[0351] Exemplarily, the decoder receives a display image bit stream that does not include CRR image identification information, and can decode the display image bit stream. Fig.11 The description in will not be repeated here.

[0352] In one example, if the decoder reads the NUT field in the NALU bitstream and includes CRR_START NUT identification information, the decoder determines that the NALU bitstream is a CRR_STAR NALU, that is, the first CRR image slice bitstream of the non-displayable CRR image. The decoder can obtain and cache the CRR image slice.

[0353] Furthermore, the decoder switches the CRR image receiving state to a state of "receiving non-displayable CRR image slice bit stream". The CRR image receiving state may be a default state before switching, which is not limited in the present application.

[0354] Exemplarily, the decoder continues to receive the image code stream and decodes the received display code stream. Also, if the decoder reads the NUT field in the NALU bit stream including CRR NUT identification information, it determines that the NALU bit stream is CRRNALU, that is, a non-boundary CRR image slice bit stream that cannot display the CRR image. The decoder obtains and caches the CRR image slices.

[0355] Exemplarily, if the decoder reads the NUT field in the NALU bitstream including CRR_END NUT identification information, it can be determined that the CRR_END NALU is received, that is, the end CRR image slice bitstream of the non-displayable CRR image. The decoder obtains and caches the CRR image slices. In addition, the decoder can determine that the CRR image reception is completed, and the decoder switches the CRR image reception state to the default state, that is, ends the "receiving non-displayable CRR image slice bitstream" state.

[0356] In a possible implementation, if the decoder reads the RL NALU containing the RL NUT identification information in the "receiving non-displayable CRR image segment bitstream" state, the decoder determines that the CRR image reception fails. That is, in this example, there is no other CRR image bitstream between the CRR image bitstream and the RL image bitstream.

[0357] The decoder can decode the CRR image for display. Fig.16 , I will not go into details here.

[0358] That is to say, in an embodiment of the present application, the decoder can identify non-displayable CRR images and displayable CRR images based on different CRR image identification information. For different types of CRR images, the decoder can adopt different reception completion judgment conditions and different decoding methods. For example, for non-displayable CRR images, the decoder can first cache the image slice bit stream, splice the image slice bit stream, and then decode the CRR image bit stream. For displayable CRR images, the decoder decodes each CRR image slice bit stream it receives, and obtains the corresponding CRR image based on the decoded image slice.

[0359] In an embodiment of the present application, as described above, the header information bit stream of the CRR image includes the number of CRR image slices. After the decoder determines that the CRR image reception is complete, that is, the state is switched to the default state, it can detect whether the CRR image slices are received completely based on the number of CRR image slices received. Optionally, the decoder can start the counter after receiving the first CRR image slice bit stream, and if a CRR image slice bit stream is not received, the counter is increased by 1. In one example, if the number of received CRR image slice bit streams (i.e., the count result of the counter) is equal to the number of CRR image slices, it can be determined that the reception is complete. In another example, if the number of received image slice bit streams is not equal to the number of CRR image slices, it can be determined that the reception is incomplete. The decoder can take measures such as directly discarding, skipping the RAS bit stream that needs to refer to this CRR image, and requesting retransmission. The specific method can be set according to actual needs and is not limited by this application.

[0360] In a possible implementation, the process of decapsulating the NALU and identifying the identification information may also be performed by other modules of the electronic device, for example, by the system layer, and after the system layer determines that a complete CRR image has been received, the encoded data of the CRR image slices carried in the bitstream may be output to the decoder. The decoder may splice and decode the encoded data of the CRR image.

[0361] In another possible implementation, when the decoder is in the "receiving non-displayable CRR image slice bit stream" state, if it receives the header information bit stream and CRR_START NALU (i.e., the boundary CRR image slice bit stream of another non-displayable CRR image) (which may also be CRR DP NUT) of another CRR image again, the decoder can determine that the bit stream transmission of the currently received CRR image is completed, and the decoder can continue to receive the image slice bit stream corresponding to another CRR image in the "receiving non-displayable CRR image slice bit stream" state.

[0362] In another possible implementation, the system layer processes the image sequence (including the CRR image and the display image) decoded by the decoder, including but not limited to:

[0363] 1. When a display image NALU or a displayable CRR image NALU is received, the operation is the same as in the above embodiment.

[0364] 2. When a CRR NALU, CRR_START NALU or CRR_END NALU is received, it means that the received image does not need to be output after being decoded into a reconstructed image, and the operation is the same as the above embodiment.

[0365] 3. When a random access occurs to a video bitstream with a CRR image bitstream, the system layer searches for the nearest random access point near the location of the requested random placement point. If the RL NALU or CRR_DP NALU is found, the location can be used as the random access point. If the searched random access point is RL NUT, the system layer searches for CRR NALU and CRR END NALU from the NALU of the RL image forward. When a CRR_START NALU is followed by a header information NALU, these header information NALUs are collected and reassembled with CRR_START NALU, CRR END NALU and CRR NALU in the order in the bitstream and input into the decoder for decoding. The decoded CRR image is used as the reference image of the required random access point RL image.

[0366] 3) The CRR image identification information includes: any one of 1), 2) and 3) above.

[0367] In this example, the encoder may set the header information NUT of the CRR image, including but not limited to:

[0368] CRR_SPS NUT is used to indicate that the SPS of the CRR image cannot be displayed.

[0369] CRR_PPS NUT is used to indicate that the PPS of the CRR picture cannot be displayed.

[0370] CRR_PH NUT is used to indicate the PH at which the CRR image cannot be displayed.

[0371] In this example, the header information NALU can be distinguished from the display image NALU by using different header information NUTs. That is, the decoder can identify the header information NALU by reading the NUT field.

[0372] Fig.24 For an exemplary structural diagram of the header information NALU, please refer to Fig.24 The NUT field in the CRC image SPS bit stream includes the CRR_SPS NUT identifier, the NUT field in the CRR image PPS bit stream includes the CRR_PPS NUT identifier, and the NUT field in the CRR image PH bit stream includes the CRR_PH NUT identifier.

[0373] Optionally, in this example, the SPS bit stream, the PPS bit stream, and the PH bit stream may be arranged continuously in order. Optionally, the SPS bit stream, the PPS bit stream, and the PH bit stream may also be arranged in order, but not continuously (i.e., there may be other image bit streams in between). Optionally, the SPS bit stream, the PPS bit stream, and the PH bit stream may also be arranged discontinuously and not in order, which is not limited in this application.

[0374] In this example, the SPS bitstream, the PPS bitstream and the PH bitstream are still located before the first CRR image slice bitstream. Optionally, the SPS bitstream, the PPS bitstream and the PH bitstream may be adjacent to the first CRR image slice bitstream.

[0375] In this example, the SPS bit stream, the PPS bit stream and the PH bit stream may not be adjacent to the first CRR picture slice bit stream, that is, the SPS bit stream, the PPS bit stream and the PH bit stream may be interleaved with the display picture bit stream.

[0376] In this example, the processing of the encoder and the decoder may refer to any of the above embodiments, and will not be repeated here.

[0377] In another possible implementation, the system layer processes the image sequence (including the CRR image and the display image) decoded by the decoder, including but not limited to:

[0378] 1. When a display image NALU or a displayable CRR image NALU is received, the operation is the same as in the above embodiment.

[0379] 2. When a CRR NALU or a CRR_Delimiter NALU is received, it means that the received image does not need to be output after being decoded into a reconstructed image, and the operation is the same as in the above embodiment.

[0380] 3. When a random access occurs to a video bitstream with a CRR image bitstream, the system layer searches for the nearest random access point near the location of the requested random placement point. If the RL NALU or CRR_DP NALU is found, the location can be used as the random access point. If the searched random access point is RL NUT, the system layer searches for CRR NALU and CRR_Delimiter NALU from the NALU of the RL image. When the second CRR_Delimiter NALU is found, all the image coding data bitstream segments that cannot display the CRR image have been found. Then, CRR_PHNALU, CRR_PPS NALU and CRR_PH NALU are found in sequence. These header information NALUs and all CRR_Delimiter NALUs and CRRNALUs are collected in the order in the bitstream, spliced ​​and reassembled, and input into the decoder for decoding. The decoded CRR image is used as the reference image of the required random access point RL image.

[0381] On the basis of the above embodiment, the embodiment of the present application further provides an identification method, when the encoder is encoding, the NUT field of the CRR image slice bitstream may include the CRR image identification and index information. The RAP bitstream in the RAS (which may be one or more) of the CRR image, or the NUT field of each display image bitstream in the RAS of the CRR image, also includes the index information. To indicate the reference relationship between the RAS and the CRR image.

[0382] Fig.25 For an exemplary diagram of the image bitstream structure, please refer to Fig.25 In this example, in the image bitstream generated by the encoder, the NUT field of each CRR image slice bitstream includes an index X. The NUT field of the RL image bitstream that references the CRR image also includes an index X.

[0383] In this example, since the CRR image and the RAS that references the CRR image can be associated through an index, correspondingly, other CRR images can be included between the CRR image and the RAS that references the CRR image. Accordingly, during the decoding process, if the decoder adopts the method in 1) or 3), each time the decoder receives a CRR image bit stream (that is, receives the header information bit stream and the first CRR image slice bit stream), the decoder sets the CRR image reception state corresponding to each CRR image. The decoder can determine whether the reception of the CRR image is completed based on the index information and CRR image identifier in the CRR image slice bit stream. For example, taking method 1) as an example, if the decoder reads the CRR_Delimiter NALU, and the corresponding index information read is index X, the decoder can set the CRR reception state corresponding to the CRR image corresponding to index X, and switch the state to "receiving non-displayable CRR image slice bit stream". Accordingly, if the decoder reads the CRR_Delimiter NALU again, and the corresponding index information read is index X, it can be determined that the CRR image bit stream corresponding to index X has been received, and the corresponding CRR image reception state is switched. Optionally, if the decoder receives a CRR_Delimiter NALU containing index Y before receiving a CRR_Delimiter NALU containing index X, the decoder can determine that the CRR image corresponding to index Y has been received, while the CRR image corresponding to index X has not been received.

[0384] In another possible implementation, the system layer processes the image sequence (including the CRR image and the display image) decoded by the decoder, including but not limited to:

[0385] 1. When a display image NALU or a displayable CRR image NALU is received, the operation is the same as in the above embodiment.

[0386] 2. When a CRR NALU or a CRR_Delimiter NALU is received, it means that the received image does not need to be output after being decoded into a reconstructed image, and the operation is the same as in the above embodiment.

[0387] 3. When a random access occurs to a video bitstream with a CRR image bitstream, the system layer searches for the nearest random access point near the location of the requested random placement point. If an RL NALU or CRR_DP NALU is found, the location can be used as the random access point. If the searched random access point is an RL NUT, the system layer identifies the index identifier of the CRR image referenced by the RL image from the NALU header of the RL image, and searches forward from the RL NALU for the CRR NALU and CRR_Delimiter NALU with the corresponding index identifier in the NALU header. When a NALU with a NUT of CRR_Delimiter NUT is followed by a header information NALU, these header information NALUs and all CRR_Delimiter NALUs and CRR NALUs are collected in the order in the bitstream for splicing and reorganization and input into the decoder for decoding. The decoded CRR image is used as the reference image of the desired random access point RL image.

[0388] It should be noted that the "before" and "after" involved in the embodiments of the present application refer to the time sequence, for example, before and after in the encoding order, or before and after in the transmission process, or before and after in the decoding order, and the present application does not limit this.

[0389] The present application also provides a coding device 2600, see Fig.26 , Fig.26 This is an example diagram of an image processing device provided in an embodiment of the present application. The device 2600 has the function of implementing the encoder in the above embodiment, for example, the function or unit or means can be implemented by software, or by hardware, or by hardware executing the corresponding software implementation.

[0390] For example, see Fig.26 , the apparatus 2600 may include:

[0391] The acquisition module 2601 is used to acquire n CRR image slices of a plurality of display images and a cross-random access point reference CRR image; n is an integer greater than 0;

[0392] The encoding module 2602 is used to encode multiple display images and CRR images to obtain an image code stream; wherein the image code stream includes n CRR image slice code streams and multiple display image code streams; each of the n CRR image slice code streams includes CRR image identification information, and the CRR image identification information is used to indicate that the corresponding code stream carries a CRR image slice.

[0393] The present application also provides a coding device 2700, see Fig. 27 , Fig. 27 This is an example diagram of an image processing device provided in an embodiment of the present application. The device 2700 has the function of implementing the encoder in the above embodiment, for example, the function or unit or means can be implemented by software, or by hardware, or by hardware executing the corresponding software implementation.

[0394] For example, see Fig. 27 , the apparatus 2700 may include:

[0395] The receiving module 2701 is used to receive an image code stream; wherein the image code stream includes: n CRR image slice code streams and multiple display image code streams; each of the n CRR image slice code streams includes CRR image identification information, and the CRR image identification information is used to indicate that the corresponding code stream carries a CRR image slice;

[0396] A decoding module 2702, configured to decode n CRR image slice code streams in the image code stream based on the CRR image identification information;

[0397] A decoding module 2702, configured to obtain a CRR image based on the decoded n CRR image slices; and

[0398] The decoding module 2702 is further configured to decode multiple display image code streams to obtain multiple display images.

[0399] The present application embodiment also provides an electronic device 2800, see Fig.28 , Fig.28 An example diagram of an electronic device provided in an embodiment of the present application. A method performed by an encoder or a decoder in the above embodiment.

[0400] like Fig.28 As shown, the electronic device 2800 may include a processor 2801 for executing a program or instruction stored in a memory 2802. When the program or instruction stored in the memory 2802 is executed, the processor is used to execute the method in the aforementioned embodiment of the present application.

[0401] Optionally, the electronic device 2800 may further include a communication interface 2803. Fig.28 The communication interface 2803 is indicated by a dotted line as being optional for the electronic device 2800 .

[0402] Among them, the number of processors 2801, memories 2802 and communication interfaces 2803 does not constitute a limitation on the embodiments of the present application. During specific implementation, they can be arbitrarily configured according to business requirements.

[0403] Optionally, the memory 2802 is located outside the electronic device 2800.

[0404] Optionally, the electronic device 2800 includes the memory 2802, the memory 2802 is connected to the at least one processor 2801, and the memory 2802 stores instructions that can be executed by the at least one processor 2801. Fig.28 The dashed lines indicate that the memory 282 is optional for the electronic device 2800 .

[0405] The processor 2801 and the memory 2802 may be coupled via an interface circuit or may be integrated together, which is not limited here.

[0406] The specific connection medium between the processor 2801, the memory 2802 and the communication interface 2803 is not limited in the embodiment of the present application. Fig.28 The processor 2801, the memory 2802 and the communication interface 2803 are connected via a bus 2804. Fig.28 The connections between the other components are shown in bold lines, which are only for illustration and are not intended to be limiting. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.28 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0407] It should be understood that the processor mentioned in the embodiments of the present application can be implemented by hardware or by software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented by software, the processor can be a general-purpose processor implemented by reading software code stored in a memory.

[0408] Exemplarily, the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0409] It should be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link DRAM (SLDRAM) and direct memory bus random access memory (directrambus RAM, DR RAM).

[0410] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, the memory (storage module) can be integrated into the processor.

[0411] It should be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0412] The embodiment of the present application also provides a computer-readable storage medium, including a program or an instruction. When the program or the instruction is run on a computer, the encoding and decoding method in the embodiment of the present application is executed.

[0413] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0414] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0415] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0416] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0417] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A coding method, characterized in that: include: Obtaining n reference image slices of a plurality of display images and a reference image; n is an integer greater than 0; The multiple display images and the reference image are encoded to obtain an image bit stream; wherein the image bit stream includes n reference image slice bit streams and multiple display image bit streams, the reference image slice bit stream is a network abstraction layer NAL unit structure, and the control field of the NAL unit of the reference image slice bit stream includes reference image identification information, and the reference image identification information is used to indicate that the load field in the NAL unit of the reference image bit stream carries the reference image slice.

2. The method according to claim 1, characterized in that The multiple display images include a reference knowledge base RL image, where the reference image is a reference frame when the RL image is subjected to inter-frame prediction.

3. The method according to claim 1, characterized in that The multiple display images include multiple random access segments RAS, and the reference image is a reference frame of the multiple RAS.

4. The method according to claim 1, characterized in that: The reference image is a cross-random access point reference CRR image.

5. The method according to claim 1, characterized in that The reference image is a non-displayable reference image and when n is greater than 1, the first reference image slice bitstream and the last reference image slice bitstream among the n reference image slice bitstreams include cross-random access point reference boundary network abstraction layer unit type CRR Delimiter NUT identification information, and other reference image slice bitstreams include cross-random access point reference network abstraction layer unit type CRR NUT identification information; wherein the CRR NUT identification information is used to indicate that the corresponding reference image slice is a non-displayable reference image slice, and the CRRDelimiter NUT identification information is used to indicate that the corresponding reference image slice is a non-displayable boundary reference image slice.

6. The method according to claim 1, characterized in that The reference image is a non-displayable reference image, and when n is equal to 1, the reference image identification information is CRR NUT identification information.

7. The method according to claim 1, characterized in that The reference image is a displayable reference image, and the reference image identification information includes cross-random access point reference display network abstraction layer unit type CRR DP NUT identification information, wherein the control field of the NAL unit corresponding to each of the reference image slice bitstreams includes the CRR DP NUT identification information.

8. The method according to any one of claims 1 to 7, characterized in that: The n reference image slice bit streams in the image bit stream are interleaved and arranged with the plurality of display image slice bit streams.

9. The method according to claim 2, characterized in that: The display image bit stream is a NAL unit structure, the multiple display image bit streams include a RL image bit stream, and the RL image bit stream is after the n reference image slice bit streams; The control field of the NAL unit corresponding to the RAP image bit stream includes a reference image network abstraction layer unit type RL NUT identifier.

10. The method according to any one of claims 1 to 9, characterized in that: The image bitstream includes a reference image information bitstream corresponding to reference image information, wherein the reference image information is used to represent the reference image; The reference image information includes at least one of the following: The sequence parameter set SPS, picture parameter set PPS, and picture header PH of the reference image.

11. The method according to claim 10, characterized in that The reference picture information bit stream precedes a first reference picture slice bit stream among the n reference picture slice bit streams, and the reference picture information bit stream is adjacent to the first reference picture slice bit stream.

12. The method according to claim 10, characterized in that The reference image information bit stream is a NAL unit structure.

13. A decoding method, characterized in that: include: Receive an image bitstream; wherein the image bitstream includes: n reference image slice bitstreams and multiple display image bitstreams, the reference image slice bitstream is a network abstraction layer NAL unit structure, the control field of the NAL unit of the reference image slice bitstream includes reference image identification information, and the reference image identification information is used to indicate that the load field in the NAL unit of the reference image bitstream carries the reference image slice; n is an integer greater than 0; Decoding the n reference image slice bit streams in the image bit stream based on the reference image identification information; Based on the decoded n reference image slices, a reference image is obtained; and, The plurality of display image bit streams are decoded to obtain a plurality of display images.

14. The method according to claim 13, characterized in that The multiple display images include a reference knowledge base RL image, and the reference image is a reference frame when the RL image is subjected to inter-frame prediction.

15. The method according to claim 13, characterized in that The multiple display images include multiple random access segments RAS, and the reference image is a reference frame of the multiple RAS.

16. The method according to claim 13, characterized in that The reference image is a cross-random access point reference CRR image.

17. The method according to claim 13, characterized in that The decoding of the n reference image slice bit streams in the image bit stream based on the reference image identification information includes: The control field of the bitstream is read to include the reference image identification information, and the reference image slice carried in the bitstream is obtained.

18. The method according to claim 13, characterized in that The reference image is a non-displayable reference image, and when n is greater than 1, the first reference image slice bitstream and the last reference image slice bitstream in the n reference image slice bitstreams include cross-random access point reference boundary network abstraction layer unit type CRR Delimiter NUT identification information, and other reference image slice bitstreams include cross-random access point reference network abstraction layer unit type CRR NUT identification information; wherein the CRR NUT identification information is used to indicate that the corresponding reference image slice is a non-displayable reference image slice, and the CRR Delimiter NUT identification information is used to indicate that the corresponding reference image slice is a non-displayable boundary reference image slice; the decoding of the n reference image slice bitstreams in the image bitstream based on the reference image identification information includes: If the CRR Delimiter NUT identification information is read, determining that the bit stream containing the CRR Delimiter NUT identification information is the first reference picture slice bit stream; If the CRR Delimiter NUT identification information is read again, it is determined that the bit stream containing the CRR Delimiter NUT identification information is the last reference picture slice bit stream, and it is determined that the reference picture bit stream is received completely.

19. The method according to claim 13, characterized in that The reference image is a non-displayable reference image, and when n is equal to 1, the reference image identification information is CRR NUT identification information.

20. The method according to claim 19, characterized in that The decoding of the n reference image slice bit streams in the image bit stream based on the reference image identification information includes: If the reference image identification information in the first received reference image bit stream carrying the reference image identification information is the CRR NUT identification information, it is determined that the reception of the reference image bit stream is completed.

21. The method according to any one of claims 13 to 20, characterized in that The decoding of the n reference image slice bit streams in the image bit stream based on the reference image identification information includes: Buffering each of the received reference picture slice bit streams; splicing the cached n reference image slice bit streams; Decode the concatenated n reference image slice bit streams.

22. The method according to any one of claims 13 to 21, characterized in that The n reference image slice bit streams in the image bit stream are interleaved and arranged with the multiple display image slice bit streams.

23. The method according to claim 13, characterized in that The reference image is a displayable reference image, and the reference image identification information includes cross-random access point reference display network abstraction layer unit type CRR DP NUT identification information, wherein each of the reference image slice bitstreams includes the CRR DP NUT identification information.

24. The method according to claim 23, characterized in that The decoding of the n reference image slice bit streams in the image bit stream based on the reference image identification information includes: The CRR DP NUT identification information is read, and a reference image slice bit stream containing the CRR DP NUT identification information is decoded to obtain a reference image slice.

25. The method according to claim 14, characterized in that The display image bit stream is a NAL unit structure, the multiple display image bit streams include a RL image bit stream, and the RL image bit stream is after the n reference image slice bit streams; The control field of the NAL unit corresponding to the RL image bit stream includes a reference image network abstraction layer unit type RL NUT identifier.

26. The method according to any one of claims 13 to 25, characterized in that: The image bitstream includes a reference image information bitstream corresponding to reference image information, wherein the reference image information is used to represent the reference image; The reference image information includes at least one of the following: The sequence parameter set SPS, picture parameter set PPS, and picture header PH of the reference image.

27. The method according to claim 26, characterized in that The reference picture information bit stream precedes a first reference picture slice bit stream among the n reference picture slice bit streams, and the reference picture information bit stream is adjacent to the first reference picture slice bit stream.

28. The method according to claim 26, characterized in that The reference image information bit stream is a NAL unit structure.

29. A coding device, characterized in that: include: one or more processors; a memory for storing one or more computer programs or instructions; When the one or more computer programs or instructions are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 12.

30. A decoding device, characterized in that: include: one or more processors; a memory for storing one or more computer programs or instructions; When the one or more computer programs or instructions are executed by the one or more processors, the one or more processors implement the method according to any one of claims 13 to 28.

Images