Coding method and apparatus, and decoding method and apparatus

Abstract

The present application provides a coding method and apparatus, and a decoding method and apparatus. The coding method comprises: acquiring a plurality of display images and n CRR image tiles of a CRR image; and coding the plurality of display images and the CRR image to obtain image code streams, wherein the image code streams include n CRR image tile code streams and a plurality of display image code streams, each of the n CRR image tile code streams comprises CRR image identification information, and the CRR image identification information is used for indicating that the CRR image tiles are carried in the corresponding code streams. Therefore, the code stream receiving complexity of a decoding end is effectively reduced.

Description

Coding and decoding method and device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on September 1, 2023, with application number 202311133490.8 and application name “Encoding and Decoding Method and Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application are directed to the field of data compression technology, and in particular, relate to a coding and decoding method and device. Background Art

[0003] Video encoding and decoding (video encoding and decoding) are widely used in digital video applications, such as broadcast digital television, video transmission over the internet, photo albums, and mobile networks; real-time conversational applications such as video chat and video conferencing; Digital Versatile Discs (DVDs) and Blu-ray Discs; video content acquisition and editing systems; and security applications in camcorders. Video encoding compresses the original video stream using a specific compression algorithm, making the compressed video stream more convenient for storage and transmission. The video is then decoded at the decoder to obtain the original video stream.

[0004] The third-generation audio and video coding standard (AVS3) proposes a cross-random access point reference (CRR) encoding and decoding scheme based on library pictures. This scheme obtains library pictures from existing videos. The system layer is equipped with a long-term buffer to store reconstructed knowledge base pictures. The reconstructed knowledge base pictures at the encoding end are obtained by encoding the knowledge base pictures, and the reconstructed knowledge base pictures at the decoding end are obtained by decoding the knowledge base picture bit stream. When encoding a video sequence, the reconstructed knowledge base picture corresponding to the current frame is obtained from the long-term buffer, and the reconstructed knowledge base picture is used as a reference picture to perform inter-frame prediction encoding on the current frame, thereby obtaining a video bit stream and a knowledge base picture bit stream corresponding to each reconstructed knowledge base picture used in the encoding. The system layer of the decoder decodes each knowledge base picture bit stream and stores the obtained reconstructed knowledge base picture in the long-term buffer. When decoding the current image bit stream, the decoder obtains the corresponding reconstructed knowledge base image from the long-term buffer and uses the reconstructed knowledge base as a reference image to perform inter-frame prediction decoding on the current video bit stream.

[0005] Summary of the Invention

[0006] The present application provides a coding and decoding method and apparatus, wherein the method carries CRR image identification information to correctly identify the CRR image code stream.

[0007] In a first aspect, the present application provides an encoding method, comprising: obtaining n CRR image slices of a plurality of display images and a cross-random access point reference CRR image; n is an integer greater than 0. The plurality of display images and the CRR image are encoded to obtain an image code stream; wherein the image code stream includes n CRR image slice code streams and a plurality of display image code streams; each of the n CRR image slice code streams includes CRR image identification information, the CRR image identification information being used to indicate that the corresponding code stream carries a CRR image slice.

[0008] In this way, during the encoding process, this application adds CRR image identification information to distinguish different types of code streams, so that the decoding end can determine whether the received code stream is a CRR image fragment code stream based on the CRR image identification information, thereby improving the accuracy of receiving and identifying CRR image fragment code streams.

[0009] For example, the CRR image used as the reference image is transmitted in slices, which can effectively reduce bandwidth impact and decoder performance impact compared to related technologies and improve encoding and decoding performance. Furthermore, each CRR image slice can be correctly identified through the CRR image identification information.

[0010] Exemplarily, the multiple display images may be display images in one RAS or display images in multiple RASs.

[0011] Exemplarily, the multiple display images may include at least one display image reference CRR image.

[0012] In one possible implementation, each CRR image slice code stream is in a network abstraction layer unit NALU format, and the NALU corresponding to each CRR image slice code stream includes a control field and a payload field, wherein the control field corresponding to each CRR image slice code stream includes CRR image identification information.

[0013] In this way, by encapsulating the CRR image identification information into the control field of the NALU, the decoder can determine whether the received code stream is a CRR image fragment code stream by reading the information in the control field, thereby effectively reducing the complexity of CRR image code stream recognition to improve the efficiency of the decoder in identifying the integrity of CRR image reception. It can be understood that in the existing technology, after decoding the CRR image fragment, it can be determined whether the CRR image fragment is received completely when the CRR image is obtained. By adding CRR image identification information, the present application can enable the decoder to determine whether the received code stream is a CRR image code stream after receiving the code stream, and can further determine whether it is received completely.

[0014] In a possible implementation, when the CRR image is a non-displayable CRR image and n is greater than 1, the CRR image identification information includes cross-random access point reference network abstraction layer unit type CRR NUT identification information and cross-random access point reference boundary network abstraction layer unit type CRR Delimiter NUT identification information; wherein, the first CRR image slice code stream and the last CRR image slice code stream in the n CRR image slice code streams include the CRR Delimiter NUT identification information, and the other CRR image slice code streams include the CRR NUT identification information.

[0015] In this way, by introducing CRR image identification information that can identify the boundary CRR image fragment code stream, the decoding end can determine the starting point and ending point of the received CRR image fragment code stream based on the CRR identification information, which can further reduce the complexity of CRR image code stream identification. In addition, the present application can effectively distinguish between non-displayable CRR images and displayable CRR images through different CRR image identification information, so that the decoder end can correctly identify non-displayable CRR image code streams and displayable CRR image code streams based on the CRR image identification information, so as to perform different decoding operations on different types of CRR images.

[0016] In one possible implementation, the CRR image is a non-displayable CRR image, and the CRR image identification information includes CRR NUT identification information. The present application can effectively distinguish between non-displayable CRR images and displayable CRR images through different CRR image identification information, so that the decoder end can correctly identify the non-displayable CRR image code stream and the displayable CRR image code stream based on the CRR image identification information, so as to perform different decoding operations on different types of CRR images, thereby further reducing the code stream identification complexity on the decoder end.

[0017] In a possible implementation, when the CRR image is a non-displayable CRR image and n is greater than 1, the CRR image identification information includes CRR NUT identification information, cross-random access point reference start network abstraction layer unit type CRR START NUT identification information, and cross-random access point reference end network abstraction layer unit type CRR END NUT identification information; wherein, the first CRR image slice code stream in the n CRR image slice code streams includes the CRR START NUT identification information, the last CRR image slice code stream includes the CRR END NUT identification information, and other CRR image slice code streams include the CRR NUT identification information.

[0018] In this way, by introducing CRR image identification information that can identify the boundary CRR image fragment code stream, the decoding end can determine the starting point and end point of the received CRR image fragment code stream based on the CRR identification information, which can further reduce the complexity of CRR image code stream identification. In addition, the present application can effectively distinguish between non-displayable CRR images and displayable CRR images through different CRR image identification information, so that the decoder end can correctly identify non-displayable CRR image code streams and displayable CRR image code streams based on the CRR image identification information, so as to perform different decoding operations on different types of CRR images. This further reduces the complexity of code stream identification on the decoder end.

[0019] In one possible implementation, the CRR image is a displayable CRR image, and the CRR image identification information includes cross-random access point reference display network abstraction layer unit type CRR DP NUT identification information, wherein each CRR image slice code stream includes CRR DP NUT identification information. The present application can effectively distinguish between non-displayable CRR images and displayable CRR images through different CRR image identification information, so that the decoder end can correctly identify the non-displayable CRR image code stream and the displayable CRR image code stream based on the CRR image identification information, so as to perform different decoding operations on different types of CRR images. This further reduces the complexity of code stream identification on the decoder end.

[0020] In a possible implementation, n CRR image slice streams in an image stream are interleaved and sequenced with a plurality of display image slice streams.

[0021] In this way, this application interweaves n CRR image bitstream segments and multiple display image bitstreams into one bitstream for transmission. In this way, all processes can be completed within the encoder, and the entire encoding process can be completed without the need for system-level collaboration. This can be applied to the calling method of existing encoders without modifying the system-level application that calls the encoder. Compared with related technologies, this reduces the design and deployment costs of the overall codec system and improves the applicability of the solution.

[0022] Moreover, in the present application, by adding CRR image identification information, the decoder end can correctly identify the CRR image code stream in the code stream transmitted in an interleaved order, thereby reducing the complexity of code stream identification.

[0023] In a possible implementation, each display image code stream is encapsulated in a NALU format, and the NALU corresponding to each display image code stream includes a control field and a payload field; wherein the control field in the code stream corresponding to the random access point RAP image in the multiple display images includes a reference knowledge base image network abstraction layer unit type RL NUT identifier, the CRR image is a reference frame of the RAP image, and the RAP image follows the n CRR image slice code streams.

[0024] In this way, the present application stipulates an RL image to mark the completion of CRR image code stream transmission, so that the decoder end can determine whether the CRR image code stream is received based on the RL NUT identification information.

[0025] For example, the encoder and the decoder agree that there is no other CRR image stream between the RL image stream and the CRR image stream. In this way, the decoder can determine that the CRR image stream has been received based on the identified RL image stream.

[0026] In a possible implementation, the control field corresponding to each CRR image slice code stream further includes index information, and the control field in the code stream corresponding to the RAP image that references the CRR image also includes index information.

[0027] In a possible implementation, the image code stream includes a CRR image information code stream corresponding to the CRR image information, and the CRR image information is used to represent the CRR image. The CRR image information includes at least one of the following: a sequence parameter set SPS, a picture parameter set PPS, and a picture header PH of the CRR image.

[0028] In a possible implementation, the CRR image information code stream precedes the first CRR image slice code stream among the n CRR image slice code streams, and the CRR image information code stream is adjacent to the first CRR image slice code stream.

[0029] In this way, the decoder end can determine the receiving starting point of the CRR image slice stream based on the header information stream and the CRR image identification information, thereby realizing the counting of the CRR image slice stream and further performing packet loss detection on the CRR image slice stream.

[0030] In a possible implementation, the CRR image information code stream is encapsulated in NALU format.

[0031] In a possible implementation, the NALU corresponding to the CRR image information code stream includes a control field and a payload field, wherein the control field corresponding to the CRR image information code stream includes image information identification information, and the image information identification information is used to indicate that the corresponding code stream carries CRR image information.

[0032] In this way, by carrying image information in the header information, the complexity of bitstream reception can be further reduced, and the decoding end can determine that the received bitstream is the header information bitstream based on the NUT field of the bitstream.

[0033] In a second aspect, the present application provides a decoding method, comprising: receiving an image code stream; wherein the image code stream comprises: n CRR image slice code streams and multiple display image code streams; each CRR image slice code stream in the n CRR image slice code streams comprises CRR image identification information, and the CRR image identification information is used to indicate that the corresponding code stream carries a CRR image slice; based on the CRR image identification information, decoding the n CRR image slice code streams in the image code stream; obtaining a CRR image based on the decoded n CRR image slices; and decoding the multiple display image code streams to obtain multiple display images.

[0034] In one possible implementation, each CRR image slice code stream is in the network abstraction layer unit NALU format, and the NALU corresponding to each CRR image slice code stream includes a control field and a payload field, wherein the control field corresponding to each CRR image slice code stream includes CRR image identification information; based on the CRR image identification information, the n CRR image slice code streams in the image code stream are decoded, including: reading the CRR image identification information included in the control field of the code stream, and obtaining the CRR image slice carried in the code stream.

[0035] In a possible implementation, the CRR image is a non-displayable CRR image, and n is greater than 1. The CRR image identification information includes cross-random access point reference network abstraction layer unit type CRR NUT identification information and cross-random access point reference boundary network abstraction layer unit type CRR Delimiter NUT identification information; wherein, the first CRR image slice code stream and the last CRR image slice code stream in the n CRR image slice code streams include CRR Delimiter NUT identification information, and the other CRR image slice code streams include CRR NUT identification information; based on the CRR image identification information, the n CRR image slice code streams in the image code stream are decoded, including: if the CRR Delimiter NUT identification information is read, determining that the code stream including the CRR Delimiter NUT identification information is the first CRR image slice code stream; if the CRR Delimiter NUT identification information is read again, determining that the code stream including the CRR Delimiter NUT identification information is the last CRR image slice code stream, and determining that the reception of the CRR image code stream is complete.

[0036] In a possible implementation, the CRR image is a non-displayable CRR image, and the CRR image identification information includes CRR NUT identification information.

[0037] In a possible implementation, the CRR image is a non-displayable CRR image, and n is greater than 1. The CRR image identification information includes CRR NUT identification information, cross-random access point reference start network abstraction layer unit type CRR START NUT identification information, and cross-random access point reference end network abstraction layer unit type CRR END NUT identification information; wherein, the first CRR image slice code stream in the n CRR image slice code streams includes CRR START NUT identification information, the last CRR image slice code stream includes CRR END NUT identification information, and other CRR image slice code streams include CRR NUT identification information; based on the CRR image identification information, the n CRR image slice code streams in the image code stream are decoded, including: reading the CRR START NUT identification information, determining that the code stream including the CRR START NUT identification information is the first CRR image slice code stream; reading the CRR END NUT identification information, determining that the code stream including the CRR END NUT identification information is the last CRR image slice code stream, and determining that the reception of the CRR image code stream is complete.

[0038] In one possible implementation, when n=1, the n CRR image slice code streams in the image code stream are decoded based on the CRR image identification information, including: if the CRR image identification information in the first received CRR image code stream carrying the CRR image identification information is the CRR NUT identification information, determining that the reception of the CRR image code stream is completed.

[0039] In a possible implementation, based on the CRR image identification information, n CRR image slice streams in the image stream are decoded, including: buffering each received CRR image slice stream; and acquiring a CRR image based on the buffered n CRR image slice streams.

[0040] In a possible implementation, n CRR image slice streams in an image stream are interleaved and sequenced with a plurality of display image slice streams.

[0041] In a possible implementation, the CRR image is a displayable CRR image, and the CRR image identification information includes cross-random access point reference display network abstraction layer unit type CRR DP NUT identification information, wherein each CRR image slice code stream includes the CRR DP NUT identification information.

[0042] In one possible implementation, based on the CRR image identification information, n CRR image slice code streams in the image code stream are decoded, including: reading the CRR DP NUT identification information, decoding the CRR image slice code stream containing the CRR DP NUT identification information to obtain CRR image slices; and obtaining the CRR image based on the n CRR image slices obtained by decoding.

[0043] In a possible implementation, each display image code stream is encapsulated in a NALU format, and the NALU corresponding to each display image code stream includes a control field and a payload field; wherein the control field in the code stream corresponding to the random access point RAP image in the multiple display images includes a reference knowledge base image network abstraction layer unit type RL NUT identifier, the CRR image is a reference frame of the RAP image, and the RAP image follows the n CRR image slice code streams.

[0044] In a possible implementation, the control field corresponding to each CRR image slice code stream further includes index information, and the control field in the code stream corresponding to the RAP image that references the CRR image also includes index information.

[0045] In a possible implementation, the image code stream includes a CRR image information code stream corresponding to the CRR image information, and the CRR image information is used to represent the CRR image. The CRR image information includes at least one of the following: a sequence parameter set SPS, a picture parameter set PPS, and a picture header PH of the CRR image.

[0046] In a possible implementation, the CRR image information code stream precedes the first CRR image slice code stream among the n CRR image slice code streams, and the CRR image information code stream is adjacent to the first CRR image slice code stream.

[0047] In a possible implementation, the CRR image information code stream is encapsulated in NALU format.

[0048] In a possible implementation, the NALU corresponding to the CRR image information code stream includes a control field and a payload field, wherein the control field corresponding to the CRR image information code stream includes image information identification information, and the image information identification information is used to indicate that the corresponding code stream carries CRR image information.

[0049] The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the second aspect and any implementation of the second aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0050] In a third aspect, the present application provides an encoding device, comprising:

[0051] An acquisition module, configured to acquire n CRR image slices of a plurality of display images and a cross-random access point reference CRR image; n is an integer greater than 0;

[0052] The encoding module is used to encode multiple display images and CRR images to obtain an image code stream; wherein the image code stream includes n CRR image slice code streams and multiple display image code streams; each CRR image slice code stream in the n CRR image slice code streams includes CRR image identification information, and the CRR image identification information is used to indicate that the corresponding code stream carries a CRR image slice.

[0053] In one possible implementation, each CRR image slice code stream is encapsulated into a network abstraction layer unit NALU format, and the NALU corresponding to each CRR image slice code stream includes a control field and a payload field, wherein the control field corresponding to each CRR image slice code stream includes CRR image identification information.

[0054] In a possible implementation, the CRR image is a non-displayable CRR image, and the CRR image identification information includes cross-random access point reference network abstraction layer unit type CRR NUT identification information and cross-random access point reference boundary network abstraction layer unit type CRR Delimiter NUT identification information;

[0055] The first CRR image slice code stream and the last CRR image slice code stream in the n CRR image slice code streams include CRR Delimiter NUT identification information, and the other CRR image slice code streams include CRR NUT identification information.

[0056] In a possible implementation, the CRR image is a non-displayable CRR image, and the CRR image identification information includes CRR NUT identification information.

[0057] In a possible implementation, the CRR image is a non-displayable CRR image, and the CRR image identification information includes CRR NUT identification information, cross-random access point reference start network abstraction layer unit type CRR START NUT identification information, and cross-random access point reference end network abstraction layer unit type CRR END NUT identification information;

[0058] The first CRR image slice code stream among the n CRR image slice code streams includes CRR START NUT identification information, the last CRR image slice code stream includes CRR END NUT identification information, and the other CRR image slice code streams include CRR NUT identification information.

[0059] In a possible implementation, the CRR image is a displayable CRR image, and the CRR image identification information includes cross-random access point reference display network abstraction layer unit type CRR DP NUT identification information, wherein each CRR image slice code stream includes the CRR DP NUT identification information.

[0060] In a possible implementation, n CRR image slice streams in an image stream are interleaved and sequenced with a plurality of display image slice streams.

[0061] In a possible implementation, each display image code stream is encapsulated in a NALU format, and the NALU corresponding to each display image code stream includes a control field and a payload field;

[0062] Among them, the control field in the code stream corresponding to the random access point RAP image in multiple display images includes the reference knowledge base image network abstraction layer unit type RL NUT identifier, the CRR image is the reference frame of the RAP image, and the RAP image is after the n CRR image slice code streams.

[0063] In a possible implementation, the control field corresponding to each CRR image slice code stream further includes index information, and the control field in the code stream corresponding to the RAP image that references the CRR image also includes index information.

[0064] In a possible implementation, the image code stream includes a CRR image information code stream corresponding to the CRR image information, and the CRR image information is used to represent the CRR image.

[0065] In a possible implementation, the CRR image information code stream precedes the first CRR image slice code stream among the n CRR image slice code streams, and the CRR image information code stream is adjacent to the first CRR image slice code stream.

[0066] In a possible implementation, the CRR image information code stream is encapsulated in NALU format.

[0067] In a possible implementation, the NALU corresponding to the CRR image information code stream includes a control field and a payload field, wherein the control field corresponding to the CRR image information code stream includes image information identification information, and the image information identification information is used to indicate that the corresponding code stream carries CRR image information.

[0068] In a fourth aspect, the present application provides a decoding device, comprising:

[0069] A receiving module, configured to receive an image code stream; wherein the image code stream includes: n CRR image slice code streams and multiple display image code streams; each of the n CRR image slice code streams includes CRR image identification information, and the CRR image identification information is used to indicate that the corresponding code stream carries a CRR image slice;

[0070] A decoding module, configured to decode n CRR image slice streams in an image stream based on CRR image identification information;

[0071] A decoding module, configured to obtain a CRR image based on the decoded n CRR image slices; and

[0072] The decoding module is further used to decode multiple display image code streams to obtain multiple display images.

[0073] In one possible implementation, each CRR image slice code stream is encapsulated into a network abstraction layer unit NALU format, and the NALU corresponding to each CRR image slice code stream includes a control field and a payload field, wherein the control field corresponding to each CRR image slice code stream includes CRR image identification information; the decoding module is specifically used to:

[0074] The control field of the code stream read includes CRR image identification information, and the CRR image slices carried in the code stream are obtained.

[0075] In a possible implementation, the CRR image is a non-displayable CRR image, and the CRR image identification information includes cross-random access point reference network abstraction layer unit type CRR NUT identification information and cross-random access point reference boundary network abstraction layer unit type CRR Delimiter NUT identification information; wherein, the first CRR image slice code stream and the last CRR image slice code stream in n CRR image slice code streams include the CRR Delimiter NUT identification information, and the other CRR image slice code streams include the CRR NUT identification information; the decoding module is specifically configured to:

[0076] If the CRR Delimiter NUT identification information is read, it is determined that the code stream containing the CRR Delimiter NUT identification information is the first CRR image slice code stream;

[0077] If the CRR Delimiter NUT identification information is read again, it is determined that the code stream containing the CRR Delimiter NUT identification information is the last CRR image slice code stream, and it is determined that the reception of the CRR image code stream is completed.

[0078] In a possible implementation, the CRR image is a non-displayable CRR image, and the CRR image identification information includes CRR NUT identification information.

[0079] In a possible implementation, the CRR image is a non-displayable CRR image, and the CRR image identification information includes CRR NUT identification information, cross-random access point reference start network abstraction layer unit type CRR START NUT identification information, and cross-random access point reference end network abstraction layer unit type CRR END NUT identification information; wherein, the first CRR image slice code stream in the n CRR image slice code streams includes CRR START NUT identification information, the last CRR image slice code stream includes CRR END NUT identification information, and other CRR image slice code streams include CRR NUT identification information; the decoding module is specifically used to: read the CRR START NUT identification information, determine that the code stream containing the CRR START NUT identification information is the first CRR image slice code stream; read the CRR END NUT identification information, determine that the code stream containing the CRR END NUT identification information is the last CRR image slice code stream, and determine that the reception of the CRR image code stream is complete.

[0080] In a possible implementation, the decoding module is specifically configured to: buffer each received CRR image slice code stream; and acquire a CRR image based on the buffered n CRR image slice code streams.

[0081] In a possible implementation, n CRR image slice streams in an image stream are interleaved and sequenced with a plurality of display image slice streams.

[0082] In a possible implementation, the CRR image is a displayable CRR image, and the CRR image identification information includes cross-random access point reference display network abstraction layer unit type CRR DP NUT identification information, wherein each CRR image slice code stream includes the CRR DP NUT identification information.

[0083] In one possible implementation, based on the CRR image identification information, n CRR image slice code streams in the image code stream are decoded, including: reading the CRR DP NUT identification information, decoding the CRR image slice code stream containing the CRR DP NUT identification information to obtain CRR image slices; and obtaining the CRR image based on the n CRR image slices obtained by decoding.

[0084] In a possible implementation, each display image code stream is encapsulated in a NALU format, and the NALU corresponding to each display image code stream includes a control field and a payload field; wherein the control field in the code stream corresponding to the random access point RAP image in the multiple display images includes a reference knowledge base image network abstraction layer unit type RL NUT identifier, the CRR image is a reference frame of the RAP image, and the RAP image follows the n CRR image slice code streams.

[0085] In a possible implementation, the control field corresponding to each CRR image slice code stream further includes index information, and the control field in the code stream corresponding to the RAP image that references the CRR image also includes index information.

[0086] In a possible implementation, the image code stream includes a CRR image information code stream corresponding to the CRR image information, and the CRR image information is used to represent the CRR image.

[0087] In a possible implementation, the CRR image information code stream precedes the first CRR image slice code stream among the n CRR image slice code streams, and the CRR image information code stream is adjacent to the first CRR image slice code stream.

[0088] In a possible implementation, the CRR image information code stream is encapsulated in NALU format.

[0089] In a possible implementation, the NALU corresponding to the CRR image information code stream includes a control field and a payload field, wherein the control field corresponding to the CRR image information code stream includes image information identification information, and the image information identification information is used to indicate that the corresponding code stream carries CRR image information.

[0090] In a fifth aspect, the present application provides a coding device, comprising: one or more processors; a memory for storing one or more computer programs or instructions; when the one or more computer programs or instructions are executed by the one or more processors, the one or more processors implement the method as described in any one of the first aspects.

[0091] In a sixth aspect, the present application provides a decoding device comprising: one or more processors; a memory for storing one or more computer programs or instructions; when the one or more computer programs or instructions are executed by the one or more processors, the one or more processors implement a method as described in any one of the second aspects.

[0092] In a seventh aspect, the present application provides an encoding device, comprising a processor for executing the method as described in any one of the first aspects.

[0093] In an eighth aspect, the present application provides a decoding device, comprising a processor for executing the method as described in any one of the second aspects.

[0094] In the ninth aspect, the present application provides an electronic device comprising: a processing circuit and an interface circuit; wherein the interface circuit is used to couple with a memory outside the command transmission device and provide a communication interface for the processing circuit to access the memory; the processing circuit is used to execute program instructions in the memory to implement any method described in any one of the first to second aspects.

[0095] In a specific implementation, the electronic device may be a chip, the input circuit may be an input pin, the output circuit may be an output pin, and the processing circuit may be a transistor, a gate circuit, a trigger, or various logic circuits. The input signal received by the input circuit may be, for example, but not limited to, received and input by a receiver, and the signal output by the output circuit may be, for example, but not limited to, output to and transmitted by a transmitter. The input circuit and the output circuit may be the same circuit, which functions as an input circuit and an output circuit at different times. The embodiments of the present application do not limit the specific implementation of the processor and various circuits.

[0096] In one implementation, the electronic device may be a wireless communication device, that is, a computer device that supports wireless communication functions. Specifically, the wireless communication device may be a terminal such as a smartphone, or a wireless access network device such as a base station. The system chip may also be referred to as a system on chip (SoC), or simply as an SoC chip. The communication chip may include a baseband processing chip and a radio frequency processing chip. The baseband processing chip is sometimes also referred to as a modem or baseband chip. The radio frequency processing chip is sometimes also referred to as a radio frequency transceiver or radio frequency chip. In a physical implementation, some or all of the chips in the communication chip may be integrated inside the SoC chip. For example, the baseband processing chip is integrated into the SoC chip, and the radio frequency processing chip is not integrated with the SoC chip. The interface circuit may be the radio frequency processing chip in the wireless communication device, and the processing circuit may be the baseband processing chip in the wireless communication device.

[0097] In another implementation, the electronic device may be a component of a wireless communication device, such as an integrated circuit product such as a system-on-chip (SoC) or a communication chip. The interface circuit may be an input / output interface, interface circuit, output circuit, input circuit, pin, or related circuit on the chip or chip system. The processor may also be embodied as a processing circuit or a logic circuit.

[0098] In a tenth aspect, the present application provides a computer-readable storage medium, in which a program code is stored. When the program code is executed by a processor, the method described in any one of the first to second aspects is implemented.

[0099] In an eleventh aspect, the present application provides a chip, comprising: at least one processor. The at least one processor is configured to execute the method according to any one of the first to second aspects.

[0100] Optionally, the chip further includes a memory. The at least one processor is configured to execute code in the memory, and when the at least one processor executes the code, the chip implements the method according to any one of the first aspect to the second aspect.

[0101] Optionally, the chip may also be an integrated circuit.

[0102] In a twelfth aspect, the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to implement the method as described in any one of the first to second aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0103] FIG1 is a schematic diagram illustrating the structure of an encoding and decoding system;

[0104] FIG2A is a block diagram of a content providing system for implementing a content distribution service;

[0105] FIG2B is a schematic structural diagram of an exemplary electronic device;

[0106] FIG3A is a schematic diagram illustrating an exemplary workflow of a streaming media system;

[0107] FIG3B is a schematic diagram illustrating an exemplary workflow of a streaming media system;

[0108] FIG4 is a schematic diagram showing an exemplary video sequence;

[0109] FIG5 is a schematic diagram showing the structure of a NALU;

[0110] FIG6 is a schematic diagram showing an exemplary CRR image;

[0111] FIG7 is a schematic diagram of a flow chart of an encoding method provided in an embodiment of the present application;

[0112] FIG8 is a schematic diagram showing an exemplary CRR image segmentation;

[0113] FIG9 is a schematic diagram of a flow chart of a decoding method provided in an embodiment of the present application;

[0114] FIG10 is a schematic diagram of an encoding and decoding process provided by an embodiment of the present application;

[0115] FIG11 is a schematic diagram showing an exemplary encoding and decoding process;

[0116] 12a to 12c are schematic diagrams illustrating exemplary interleaving sorting methods;

[0117] FIG13 is a schematic diagram illustrating a flow chart of an exemplary encoding method;

[0118] FIG14 is a schematic diagram showing an exemplary structure of an image code stream to which a non-displayable CRR image belongs;

[0119] FIG15 is a schematic diagram showing an exemplary structure of an image code stream to which a CRR image can be displayed;

[0120] FIG16 is a schematic flow chart illustrating an exemplary decoding method;

[0121] FIG17 is a schematic diagram of an exemplary image code stream;

[0122] FIG18 is a schematic diagram illustrating a flow chart of an exemplary encoding method;

[0123] FIG19 is a schematic diagram showing an exemplary structure of an image code stream to which a non-displayable CRR image belongs;

[0124] FIG20 is a schematic flow chart illustrating an exemplary decoding method;

[0125] FIG21 is a schematic flow chart illustrating an exemplary encoding method;

[0126] FIG22 is a schematic diagram showing an exemplary structure of an image code stream to which a non-displayable CRR image belongs;

[0127] FIG23 is a schematic flow chart illustrating an exemplary decoding method;

[0128] FIG24 is a schematic diagram showing the structure of a header information NALU;

[0129] FIG25 is a schematic diagram showing an exemplary image bit stream structure;

[0130] FIG26 is a schematic structural diagram illustrating an exemplary encoding device;

[0131] FIG27 is a schematic structural diagram illustrating an exemplary decoding device;

[0132] FIG28 is a schematic structural diagram of an exemplary electronic device. DETAILED DESCRIPTION

[0133] The term "at least one" in the embodiments of the present application refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where any of a, b, c, ab, ac, bc, or abc can include a single a, a single b, a single c, and can also include multiple a, multiple b, and multiple c.

[0134] Furthermore, unless otherwise indicated, ordinal numbers such as "first" and "second" in the embodiments of this application are used to distinguish between multiple objects and are not used to define the order, timing, priority, or importance of multiple objects. For example, the first priority criterion and the second priority criterion are only used to distinguish different criteria and do not indicate differences in the content, priority, or importance of the two criteria.

[0135] In addition, the terms "including" and "having" in the embodiments, claims, and drawings of this application are not exclusive. For example, a process, method, system, product, or device that includes a series of steps or modules / units is not limited to the listed steps or modules and may also include steps or modules / units that are not listed.

[0136] The following describes the codec system used in the present application in conjunction with FIG1 . FIG1 is a schematic block diagram of a codec system used in an embodiment of the present application, such as a video codec system 10 (or simply codec system 10) that can utilize the techniques of the present application. The video encoder 20 (or simply encoder 20) and video decoder 30 (or simply decoder 30) in the video codec system 10 represent devices that can be used to perform various techniques according to the various examples described in this application.

[0137] As shown in FIG1 , a coding and decoding system 10 includes a source device 12 , which is configured to provide coded data 21 such as coded images to a destination device 14 for decoding the coded data.

[0138] The source device 12 includes an encoder 20 and, optionally, may further include an image source 16 , an image preprocessor 18 (or a preprocessing unit), and a communication interface or communication unit 22 .

[0139] The image source 16 may include or may be any type of image capture device for capturing real-world images, etc., and / or any type of image generation device, such as a computer graphics processor for generating computer-animated images, or any type of device for acquiring and / or providing real-world images, computer-generated images (e.g., screen content, virtual reality (VR) images, and / or any combination thereof (e.g., augmented reality (AR) images). The image source may be any type of memory or storage that stores any of the above images.

[0140] In order to distinguish the processing performed by the pre-processor 18 or the pre-processing unit 18 , the image or image data 17 may also be referred to as a raw image or raw image data 17 .

[0141] The preprocessor 18 is configured to receive (raw) image data 17 and preprocess the image data 17 to obtain a preprocessed image 19 or preprocessed image data 19. For example, the preprocessing performed by the preprocessor 18 may include cropping, color format conversion (e.g., from RGB to YCbCr), color grading, or denoising. It will be appreciated that the preprocessor 18 may be an optional component.

[0142] The video encoder 20 is configured to receive the pre-processed image data 19 and provide encoded image data 21 .

[0143] Communication interface 22 in source device 12 may be used to receive encoded image data 21 and transmit the encoded image data 21 (or any other processed version thereof) to another device, such as destination device 14, or any other device, via communication channel 13 for storage or direct reconstruction.

[0144] Destination device 14 includes a decoder 30 (eg, video decoder 30 ) and, in addition and optionally, may include a communication interface or communication unit 28 , a post-processor 32 (or post-processing unit 32 ), and a display device 34 .

[0145] The communication interface 28 in the destination device 14 is used to receive the encoded image data 21 (or any other processed version) directly from the source device 12 or from any other source device such as a storage device, for example, a storage device that is an encoded image data storage device, and provide the encoded image data 21 to the decoder 30.

[0146] The communication interface 22 and the communication interface 28 may be used to send or receive the encoded image data 21 or the encoded data via a direct communication link between the source device 12 and the destination device 14, such as a direct wired or wireless connection, or via any type of network, such as a wired network, a wireless network, or any combination thereof, any type of private network and public network, or any combination thereof.

[0147] For example, the communication interface 22 may be used to encapsulate the encoded image data 21 into a suitable format such as a message, and / or process the encoded image data using any type of transmission coding or processing for transmission over a communication link or network.

[0148] The communication interface 28 corresponds to the communication interface 22 , for example, and can be used to receive transmission data and process the transmission data using any type of corresponding transmission decoding or processing and / or decapsulation to obtain encoded image data 21 .

[0149] Both the communication interface 22 and the communication interface 28 can be configured as a unidirectional communication interface as indicated by the arrow pointing from the source device 12 to the corresponding communication channel 13 of the destination device 14 in Figure 1, or a bidirectional communication interface, and can be used to send and receive messages, etc. to establish a connection, confirm and exchange any other information related to the communication link and / or data transmission such as encoded image data transmission, etc.

[0150] The decoder 30 is configured to receive the encoded image data 21 and provide decoded image data 31 or a decoded image 31 .

[0151] The post-processor 32 in the destination device 14 is configured to post-process the decoded image data 31 (also referred to as reconstructed image data), such as the decoded image 31, to obtain post-processed image data 33, such as the post-processed image 33. The post-processing performed by the post-processing unit 32 may include, for example, color format conversion (e.g., from YCbCr to RGB), color grading, cropping, or resampling, or any other processing for generating the decoded image data 31 for display on a display device 34, etc.

[0152] A display device 34 in the destination device 14 is configured to receive the post-processed image data 33 and display the image to a user or viewer. The display device 34 may be or include any type of display for displaying the reconstructed image, such as an integrated or external display screen or monitor. For example, the display screen may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro-LED display, a liquid crystal on silicon (LCoS) display, a digital light processor (DLP), or any other type of display screen.

[0153] Although FIG1 shows source device 12 and destination device 14 as separate devices, device embodiments may also include both source device 12 and destination device 14 or the functionality of both source device 12 and destination device 14, that is, both source device 12 or the corresponding functionality and destination device 14 or the corresponding functionality. In these embodiments, source device 12 or the corresponding functionality and destination device 14 or the corresponding functionality may be implemented using the same hardware and / or software or through separate hardware and / or software, or any combination thereof.

[0154] Based on the description, it will be apparent to those skilled in the art that the existence and (precise) division of different units or functions in the source device 12 and / or the destination device 14 shown in FIG. 1 may vary depending on actual devices and applications.

[0155] The content providing system of the content distribution service used in the present application is described below in conjunction with Figure 2A. Figure 2A is a block diagram of a content providing system for implementing the content distribution service used in an embodiment of the present application. The content providing system 2100 includes a capture device 2102, a terminal device 2106, and an (optional) display 2126. The capture device 2102 communicates with the terminal device 2106 via a communication link 2104. The communication link may include the above-mentioned communication channel 13. The communication link 2104 includes but is not limited to WiFi, Ethernet, wired, wireless (3G / 4G / 5G), USB, or any combination thereof.

[0156] Capture device 2102 generates data and can use the coding method shown in the above embodiment to encode the data. Alternatively, capture device 2102 can distribute data to a streaming media server (not shown), which encodes the data and transmits the encoded data to terminal device 2106. Capture device 2102 includes but is not limited to a camera, a smart phone or a tablet computer, a computer or a notebook computer, a video conferencing system, a PDA, a vehicle-mounted device, or any combination thereof. For example, capture device 2102 can include above-mentioned source device 12. When data includes video, the video encoder 20 in capture device 2102 can actually perform video encoding. When data includes audio (i.e., sound), the audio encoder 20 in capture device 2102 can actually perform audio encoding. In some actual scenarios, capture device 2102 distributes encoded video data and encoded audio data by multiplexing the encoded video data and the encoded audio data together. In other actual scenarios, for example, in a video conferencing system, the encoded audio data and the encoded video data are not multiplexed. Capture device 2102 distributes the encoded audio data and the encoded video data to terminal device 2106 respectively.

[0157] The terminal device 2106 in the content providing system 2100 receives and regenerates the encoded data. The terminal device 2106 can be a device with data reception and recovery capabilities, such as a smartphone or tablet computer 2108, a computer or laptop computer 2110, a network video recorder (NVR) / digital video recorder (DVR) 2112, a television 2114, a set-top box (STB) 2116, a video conferencing system 2118, a video surveillance system 2120, a personal digital assistant (PDA) 2122, an in-vehicle device 2124, or any combination thereof, or such devices capable of decoding the encoded data. For example, the terminal device 2106 can include the destination device 14 described above. When the encoded data includes video, the video decoder 30 in the terminal device prioritizes video decoding. When the encoded data includes audio, the audio decoder in the terminal device prioritizes audio decoding. The terminal device 2106 may be a video playback application, a streaming media playback application, a streaming media playback platform, a live broadcast platform, etc. running on the terminal device.

[0158] For end devices with displays, such as smartphones or tablets 2108, computers or laptops 2110, NVRs / DVRs 2112, televisions 2114, PDAs 2122, or in-vehicle devices 2124, the end device can send the decoded data to its display. For end devices without displays, such as STBs 2116, video conferencing systems 3118, or video surveillance systems 2120, the device can be connected to an external display 2126 to receive and display the decoded data.

[0159] When performing encoding or decoding, each device in this system may use the image encoding device or image decoding device shown in the above embodiments.

[0160] FIG2B is a schematic diagram of an example structure of the terminal device 2106 in FIG2A . After the terminal device 2106 receives the bitstream from the capture device 2102, the protocol processing unit 2202 analyzes the transmission protocol of the bitstream. The protocol includes, but is not limited to, Real Time Streaming Protocol (RTSP), Hyper Text Transfer Protocol (HTTP), HTTP Live Streaming Protocol (HLS), MPEG Dynamic Adaptive Streaming over HTTP (MPEG-DASH), Real-time Transport Protocol (RTP), Real Time Messaging Protocol (RTMP), or any combination thereof.

[0161] After processing the stream, protocol processing unit 2202 generates a stream file. This file is output to demultiplexing unit 2204. Demultiplexing unit 2204 can separate the multiplexed data into coded audio data and coded video data. As mentioned above, in other practical scenarios, such as video conferencing systems, coded audio data and coded video data are not multiplexed. In this case, the coded data is not transmitted to video decoder 3206 and audio decoder 2208 via demultiplexing unit 2204.

[0162] Through the demultiplexing process, a video elementary stream (ES), an audio ES, and optional subtitles are generated. The video decoder 2206, including the video decoder 30 described in the above embodiment, decodes the video ES by the decoding method shown in the above embodiment to generate video frames, and sends the data to the synchronization unit 2212. The audio decoder 2208 decodes the audio ES to generate audio frames, and sends the data to the synchronization unit 2212. Alternatively, the video frames can be stored in a buffer (not shown in the figure) before being sent to the synchronization unit 2212. Similarly, the audio frames can be stored in a buffer (not shown in the figure) before being sent to the synchronization unit 2212.

[0163] The synchronization unit 2212 synchronizes the video frames and audio frames and provides the video / audio to the video / audio display 3214. For example, the synchronization unit 2212 synchronizes the presentation of the video and audio information. The information can be encoded in the syntax using the timestamps associated with the representation of the coded audio and visual data and the timestamps associated with the transmission of the data stream.

[0164] If subtitles are included in the bitstream, the subtitle decoder 2210 decodes the subtitles, synchronizes them with the video frames and audio frames, and provides the video / audio / subtitles to the video / audio / subtitle display 2216.

[0165] The present invention is not limited to the above-mentioned system. The image encoding device or the image decoding device in the above-mentioned embodiments can be used in other systems such as automobiles.

[0166] The following describes a streaming media system applicable to an embodiment of the present application in conjunction with Figure 3A. Figure 3A is a schematic diagram of a workflow of a streaming media system applicable to an embodiment of the present application.

[0167] The streaming media system includes a content creation module that generates the required content data, such as video or audio. The streaming media system also includes a video encoding module that encodes the generated content through an encoder. The streaming media system also includes a video stream transmission module that transmits the encoded video in the form of a code stream. Optionally, the format of the video stream can be converted into a code stream format of a commonly used transmission protocol for OTT (over-the-top) devices, for example, the protocol includes but is not limited to Real Time Streaming Protocol (RTSP), Hyper Text Transfer Protocol (HTTP), HTTP Live streaming protocol (HLS), MPEG HTTP Dynamic Adaptive Streaming over HTTP (MPEG-DASH), Real-time Transport Protocol (RTP), Real Time Messaging Protocol (RTMP) or any combination thereof. Optionally, the video stream can be stored to store the original format of the video stream and / or the converted multiple code stream formats for easy use. Furthermore, the streaming system also includes a video stream encapsulation module for encapsulating the video stream to generate an encapsulated video stream. The encapsulated video stream can be referred to as a video streaming package. Exemplarily, the video streaming package can be generated based on a transcoded video stream or a stored video stream. Furthermore, the streaming system also includes a content distribution network (CDN) for distributing the video streaming package to multiple OTT devices, such as mobile phones, computers, tablets, and home projectors.

[0168] It should be noted that video encoding, video streaming transmission, video stream transcoding, video stream storage, video streaming package generation and content distribution network can all be implemented on cloud servers.

[0169] An exemplary streaming media system architecture of the present application is described below in conjunction with FIG3B . The streaming media system architecture includes: a client device, a content distribution network, and a cloud server.

[0170] The user on the client device sends a play or playback request to the cloud platform. Optionally, the content of the request can be the title of the movie or TV program to be played.

[0171] The cloud platform makes a decision and responds to the client, sending the client the address of the requested content on the CDN. Optionally, the content sent to the client can be a URL link (uniform resource locator). Specifically, the playback application service in the cloud platform checks user authorization and permissions, and then considers individual client characteristics and current network conditions to determine which specific files are needed to process the playback request. It should be noted that the content delivery network (CDN) regularly reports its operating status, learned routes, and available content (files) to the cache control service in the cloud platform.

[0172] The client then requests the CDN to play the content based on the address, and the CDN provides the content to the client, ultimately completing the client's request.

[0173] Before describing the embodiments of the present application, the terms and concepts used in the embodiments of the present application are first explained.

[0174] 1) Reconstructed Image: This refers to the lossy image obtained by decoding the encoded image bitstream, which can be displayed and played. It exists on both the encoder and decoder, and the reconstructed image on both ends is identical. On the encoder, the reconstructed image can be used as a reference image for the image to be encoded to perform inter-frame prediction encoding. On the decoder, the reconstructed image can be used as a reference image for the image to be decoded to perform inter-frame prediction decoding.

[0175] 2) RAP (Random Access Point): A random access point in video encoding. The encoded video bitstream includes at least one RAP. When random access occurs, the video bitstream before the RAP can be skipped, and the subsequent image bitstream can be decoded from the RAP and the reconstructed video image output by the decoder can be played. From the perspective of video playback, random access can be understood as dragging the progress bar of the player to achieve random jumps of the video when watching on-demand videos (or local playback videos); or when a black screen occurs due to packet loss when watching live broadcasts, the client will access the video stream and continue to play the video when the next RAP arrives. RAP is usually an intra-coded picture (I) frame. The I frame only uses the information within the current frame for predictive coding to eliminate information redundancy in the spatial domain without relying on other frames.

[0176] 3) Random Access Segment (RAS): A RAS includes all images from one RAP to the next. Images in a RAS are either forward-predicted (P) frames or bi-directionally interpolated prediction (B) frames. P and B frames use already coded images for temporal prediction to eliminate temporal redundancy.

[0177] For example, please refer to Figure 4, which is a schematic diagram of a video sequence provided in an embodiment of the present application. Figure 4 shows 11 video frames a1 to a11. Among them, a2, a6 and a10 are I frames and serve as RAP, all video frames between a2 and a6 are P frames and constitute RAS1, and all video frames between a6 and a10 are P frames and constitute RAS2. As can be seen from Figure 4, each P frame needs to be encoded with reference to the previous m video frames, where m is an integer greater than 1. For example, a3 can be encoded with reference to a2, and a4 can be encoded with reference to a2 and / or a3.

[0178] 4) CRR encoding: When encoding a video frame in a RAS, the video frames in other RASs other than the RAS can be referenced for encoding. For example, the reference frame of video frame a11 in FIG4 is image frame a9.

[0179] In most videos (such as movies or TV programs), many picture scenes appear alternately, and the picture scenes displayed at different times may be similar. The interval between two adjacent similar picture scenes is usually long, so the two adjacent similar picture scenes are in the same RAP but the interval is long, or the time correlation spans the RAP. For example, the video standard generally stipulates that the length of RAS is 1s or 2s, and the interval between two adjacent similar picture scenes is usually close to the length of RAS (for example, 0.7s) or greater than the length of RAS (for example, 3s). In addition, in video scenes such as security monitoring, the appearance of similar picture scenes usually lasts for a long time, so there will be two similar picture scenes in the same RAP but with a long interval, or the time correlation spans the RAP.

[0180] 5) SPS (Sequence Parameter Set): Contains a set of global parameters of the video coding sequence, such as color space, width and height of each frame image, encoding level, etc.

[0181] 6) PPS (Picture Parameter Set): Contains encoding parameters for one or more images, such as quantization matrix, filter parameters, etc.

[0182] 7) Picture Header (PH): Contains the encoding parameters of a single image, such as the frame number.

[0183] 8) Image coding data: Contains prediction, residual and other data for encoding the content of a single image.

[0184] 9) Header information: This refers to all encoding parameters used to encode or decode a single image. For example, SPS, PPS, and PH are all header information.

[0185] 10) NALU (Network Abstraction Layer Unit): a grammatical structure. Figure 5 is a schematic diagram of the structure of an exemplary NALU. Please refer to Figure 5. Each NALU includes a NALU Header field (also known as NALU header information) and a payload field. Among them, the NALU Header field can also be called a control field. The NALU Header field includes a NUT (Network Abstraction Layer Unit Type) field. In an embodiment of the present application, the NUT field is used to carry data type identification information and other related information, and the data type identification information is used to indicate the type of data carried in the payload field.

[0186] Optionally, the information in the payload field may appear in the form of RBSP (Raw Byte Sequence Payload). In other embodiments, the payload field may also include interspersed anti-counterfeiting bytes. In an embodiment of the present application, the bit stream (also referred to as a code stream) generated by the encoder is encapsulated in NALU format. Optionally, in an embodiment of the present application, a NALU with "A" in the NUT field is also referred to as an A NALU, which will not be repeated below.

[0187] 11) RL (Reference Library) images: The entire image can be encoded using both inter-frame and intra-frame prediction, and the reference image is a single CRR image. This means that other displayed images may reference one or more images, including the CRR image or one or more images preceding the displayed image. RL images only reference the CRR image.

[0188] 12) Display images: I images (I frames), P images (P frames), B images (B frames), RL images (RL frames), etc. that can be displayed in the coding sequence, excluding CRR images.

[0189] 13) CRR image: The entire image is encoded using intra-frame prediction, and its reconstructed image can be placed in the decoded image buffer for a long time, serving as a bit reference image for multiple RAS images. Figure 6 is a schematic diagram of an exemplary CRR image. Referring to Figure 6, a CRR image (such as CRR image 1 and CRR image 2) needs to be referenced by one or more similar RAS, so the CRR image needs to be a common stable content that can represent these similar RAS segments. In one example, the CRR image is extracted or generated from an existing video, and the CRR image is only used for reference and will not be displayed for playback. This type of CRR image can be called a non-displayable CRR image. In another example, the CRR image can also be a displayable image, that is, the displayable CRR image is used for reference and can be displayed for playback. It can be understood that the displayable CRR image itself is an image frame to be displayed in the RAS.

[0190] In order to ensure the random access function of each RAP (P frame encoding), the corresponding CRR image needs to be provided when decoding each RAS. In order to avoid the need to re-encode and transmit the CRR image every time a RAS is decoded, AVS3 designs a long-term buffer at the system layer to store the reconstructed CRR image. When a similar scene occurs again, the corresponding CRR image in the long-term buffer can be reused. In addition, in order to smooth the transmission bandwidth impact and decoding capability impact brought by large I frames, AVS3 makes the encoding and transmission of CRR images independent of the encoding and transmission of video data to be displayed. Then the bit stream after CRR image encoding and the bit stream after video frame encoding are respectively called CRR image bit stream and main bit stream (also called display image bit stream, which is not limited in this application).

[0191] Each picture in the main bitstream can parse the index of the CRR image referenced by the current frame in the system layer long-term buffer, and then the decoder uses the corresponding CRR image as the reference image of the current frame for inter-frame prediction.

[0192] It should be noted that the "image" involved in the embodiments of the present application can be replaced by an image frame. The "bitstream" involved in the embodiments of the present application can be replaced by a "code stream", and the "image slice" in the embodiments of the present application can be replaced by a "sub-image". The following description will not be repeated.

[0193] The embodiment of the present application provides a coding and decoding method, which can effectively reduce time domain redundancy and improve video coding performance when encoding videos with a high degree of scene repetition.

[0194] Please refer to FIG7 , which is a flowchart of an encoding method provided in an embodiment of the present application. The method can be applied to an encoder, specifically including but not limited to the following steps:

[0195] S701 : Acquire n CRR image slices of a plurality of display images and CRR images.

[0196] The multiple display images may be a plurality of continuous display images in a video sequence. The encoder may be provided with a preset duration, divide the video sequence into at least one video segment according to the preset duration, and obtain a CRR image corresponding to the video segment. The multiple display images may constitute a video segment. The preset duration may be greater than the length of the current RAS, for example, the preset duration may be 6 seconds, 7 seconds, or 10 seconds. Accordingly, the multiple display images include multiple RAPs, each of which may be a P frame or a B frame.

[0197] A reconstructed CRR image can be used as a reference image for multiple subsequent display images to be encoded. Optionally, any one of the multiple display images can be used as a CRR image corresponding to the multiple display images. Alternatively, CRR images corresponding to the multiple display images can be constructed based on the multiple display images. The embodiment of the present application does not limit the method for obtaining the CRR image.

[0198] In an embodiment of the present application, the encoder divides the CRR image into n CRR image slices (which may also be referred to as CRR image slices or CRR image segments, etc., which are not limited in this application). Figure 8 is an exemplary diagram of CRR image segmentation. Please refer to Figure 8. The encoder may segment CRR image 1 and CRR image 2 to obtain image slices corresponding to CRR image 1, including CRR image slice 1-1 (which may also be referred to as CRR image slice 1-1, which is not limited in this application), CRR image slice 1-2, and CRR image slice 1-3. The image slices corresponding to CRR image 2 include CRR image slice 2-1 (which may also be referred to as CRR image slice 2-1, which is not limited in this application), CRR image slice 2-2, and CRR image slice 2-3.

[0199] S702, encode multiple display images and CRR images to obtain an image bit stream; wherein the image bit stream includes: n CRR image slice bit streams and multiple display image bit streams in an interleaved order; each CRR image slice bit stream in the n CRR image slice bit streams includes CRR image identification information, and the CRR image identification information is used to indicate that the corresponding bit stream carries a CRR image slice.

[0200] The coded image bitstream is a single-channel bitstream. The CRR image is used to generate a reconstructed image, which can then be used as a reference image for subsequent coded display images to perform inter-frame predictive coding on the coded display images. The CRR image does not need to be displayed. After encoding a display image in a video sequence, a display image bitstream and a reconstructed display image are obtained. It can be seen that in the embodiments of the present application, the reconstructed images include two types: reconstructed display images and reconstructed CRR images.

[0201] When encoding each of the multiple display images, the reference image of the display image includes: a reconstructed display image and / or a currently effective reconstructed CRR image. The currently effective reconstructed CRR image can be stored in a buffer space of the encoder, for example, in the DPB of the encoder. The currently effective reconstructed CRR image refers to the reconstructed CRR image that is in effect at the moment when the encoding operation is performed on each of the multiple display images.

[0202] When performing an encoding operation on each of a plurality of display images, at least one display image bitstream has been encoded, and accordingly, at least one reconstructed display image also exists. Optionally, any number of reconstructed display images among all currently obtained reconstructed display images can be used as reference images for the display image. For example, the reference image of the display image includes: at least one reconstructed display image obtained by the encoding operation performed on at least one display image that was most recently output. For example, when performing an encoding operation on the third display image among a plurality of display images, the current encoding obtains the first display image bitstream and the first reconstructed display image corresponding to the first display image, and the second display image bitstream and the second reconstructed display image corresponding to the second display image. Therefore, when performing an encoding operation on the third display image among a plurality of display images, the first reconstructed display image and the second reconstructed display image can be used as reference images for the third display image.

[0203] The currently valid reconstructed CRR image is obtained by encoding the CRR image. Optionally, it can be obtained when encoding a CRR image slice group. The CRR image slice bit stream may include but is not limited to: the latest output n CRR image slice bit streams, or the latest output n CRR image slice bit streams corresponding to the same CRR image. The latest output here refers to the latest output at the moment of encoding the display image.

[0204] During the encoding process of the entire video sequence, before the nth CRR image slice bitstream is output, there is no currently valid reconstructed CRR image. Therefore, for the display image bitstream output before the nth CRR image bitstream segment, the reference image used during encoding only includes the reconstructed display image.

[0205] In an embodiment of the present application, the encoder encapsulates each generated bit stream (including the CRR image slice bit stream and the display image bit stream) into a NALU format. As shown in Figure 7, the NALU includes a NALU Header field and a payload field (i.e., RSBP). The NALU Header field includes, but is not limited to, a NUT field. In an embodiment of the present application, the NUT field corresponding to each CRR image slice generated by the encoder includes CRR image identification information, which is used to indicate that the data carried by the payload field in the bit stream is a CRR image slice and its corresponding data. Optionally, in other embodiments, other fields in the NALU control field may carry CRR image identification information, which is not limited in this application. In some embodiments, each code stream may also be encapsulated into other formats, and the CRR image identification information and other identification information in this article may be set in the control field of code streams in other formats to reduce the complexity of code stream identification.

[0206] In a possible implementation, the RAS in the embodiment of the present application refers to only one CRR image, and the referenced CRR image may be a displayable CRR image or a non-displayable CRR image.

[0207] In another possible implementation, multiple RASs may reference one CRR image.

[0208] In another possible implementation, the non-displayable CRR image bitstream (i.e., the bitstreams of all CRR image slices corresponding to the CRR image) precedes the RAS bitstream that references the CRR image. The displayable CRR image bitstream is within the RAS bitstream that references the CRR image. A specific example is provided below.

[0209] In summary, the encoding method provided in the embodiment of the present application first obtains CRR images corresponding to multiple display images, and then performs encoding operations on the CRR images and the multiple display images to obtain an encoded image bit stream, which includes: n CRR image slice bit streams and multiple display image bit streams, where n is an integer greater than 1, and each CRR image slice bit stream includes CRR image identification information to identify that the corresponding bit stream carries a CRR image slice. In the embodiment of the present application, n CRR image bit stream segments and multiple display image bit streams are encoded into one bit stream for transmission. This method effectively reduces the transmission bandwidth while realizing cross-RAP reference, and through the CRR image identification, the decoder end can correctly identify the CRR image slice.

[0210] Please refer to FIG9 , which is a flowchart of a decoding method provided in an embodiment of the present application. The method can be applied to a decoder, specifically including but not limited to the following steps:

[0211] S901, receiving an image bitstream, wherein the image bitstream includes: n CRR image slice bitstreams and multiple display image bitstreams; each of the n CRR image slice bitstreams includes CRR image identification information, and the CRR image identification information is used to indicate that the corresponding bitstream carries a CRR image slice.

[0212] S902: Decode the n CRR image slice bit streams in the image bit stream based on the CRR image identification information.

[0213] S903 , obtaining a CRR image based on the decoded n CRR image slices.

[0214] S904: Decode the multiple display image bit streams to obtain multiple display images.

[0215] For example, the decoder can identify that the corresponding bitstream carries a CRR image slice based on the CRR image identification information in the CRR image slice bitstream. The decoder concatenates the received n CRR image slice bitstreams to obtain a CRR image bitstream. The decoder decodes the CRR image bitstream to obtain a reconstructed image corresponding to the CRR image (also referred to as a reconstructed CRR image).

[0216] Exemplarily, each time the decoder decodes a display image bitstream, a reconstructed display image is obtained. When decoding the display image bitstream, the reference image of the display image bitstream includes, but is not limited to, a reconstructed display image and / or a reconstructed CRR image corresponding to the previous image. The reconstructed CRR image can be stored in the decoder's buffer space, for example, in the decoder's DPB, which is not limited in this application.

[0217] The reference image used when decoding any display image bit stream in the coded image bit stream is consistent with the reference image used by the encoder when encoding the display image bit stream.

[0218] The reconstructed display image is obtained by decoding a previous display image bit stream. A reconstructed display image is obtained by decoding a display image bit stream.

[0219] In summary, the decoding method provided by the embodiment of the present application first receives a coded image bitstream, which includes: n CRR image bitstream segments and multiple display image bitstreams, where n is an integer greater than 1. Then, the CRR image slice bitstream and the display image bitstream in the coded image bitstream are decoded in sequence to obtain multiple reconstructed display images. When the display image bitstream is decoded, the reference image of the display image bitstream includes: a reconstructed display image and / or a reconstructed CRR image. The reconstructed display image is obtained by decoding the previous display image bitstream, and the reconstructed CRR image is obtained by decoding the previous CRR image bitstream segment. The reconstructed CRR image can be used as a reference image for the current display image bitstream to be decoded. The reconstructed CRR image is obtained based on the CRR image slice bitstream. Since the CRR image corresponds to multiple display images, and the multiple display images may include multiple RAPs, cross-RAP reference can be achieved. In addition, n CRR image bitstream segments and multiple display image bitstreams are encoded into one bitstream and transmitted to the decoder, which can effectively reduce the transmission bandwidth while achieving cross-RAP reference. Moreover, through the CRR image identification, the decoder end can correctly identify the CRR image segmentation.

[0220] In the embodiment of the present application, as described above, CRR images can be divided into non-displayable CRR images and displayable CRR images. In the embodiment of the present application, the CRR image slice bit stream corresponding to the non-displayable CRR image and the display image bit stream can be interleaved and sorted into one image bit stream (also referred to as a coded image bit stream). The CRR image slice bit stream corresponding to the displayable CRR image is arranged according to its original order in the RAS.

[0221] The following describes in detail the interleaving ordering scenario between the non-displayable CRR image bitstream and the display image bitstream. For example, please refer to Figure 10, which is a schematic diagram of a coding and decoding process provided in an embodiment of the present application. The encoder encodes the CRR image and multiple display images to obtain a coded image bitstream and sends the coded image bitstream to the decoder. Figure 9 takes 3 CRR image slice bitstreams b1 to b3 (n=3) and 5 display image bitstreams c1 to c5 as an example for illustration. In other embodiments, more or fewer CRR image slice bitstreams and display image bitstreams may be included, and this application does not limit this.

[0222] Please continue to refer to Figure 10. The decoder decodes the bit streams in the encoded image bit stream in sequence to obtain 5 reconstructed display images c1′ to c5′ and a reconstructed CRR image. The 5 reconstructed display images are used for display, and the reconstructed CRR image is only used as a reference image when decoding the display image bit stream and is not used for display.

[0223] For example, FIG11 is a schematic diagram of an encoding and decoding process shown as an example. Referring to FIG11 , the encoder divides the CRR image e into three CRR image slices e1, e2, and e3 (wherein the CRR image slices are the CRR image slices described in the embodiment of the present application, and will not be repeated below). The CRR image slice e1 is interspersed between the display image d1 and the display image d2, the CRR image slice e2 is interspersed between the display image d2 and the display image d3, and the CRR image slice e3 is interspersed between the display image d3 and the display image d4 to obtain the image sequence to be encoded.

[0224] Afterwards, the encoder encodes the image sequence to be encoded. First, the encoder performs an encoding operation on the display image d1. Since the new CRR image slice bit stream has not been output at this time, only the first reconstructed CRR image is stored in the cache space, and the currently effective reconstructed CRR image is the first reconstructed CRR image. A reference image is selected to perform an encoding operation on the display image d1 to obtain the display image bit stream d1′ and the reconstructed display image (not shown in Figure 11). Next, the encoder performs an encoding operation on the CRR image slice e1 to obtain the CRR image slice bit stream e1′ and the reconstructed CRR image slice e1″. The encoder caches the reconstructed CRR image slice e1″.

[0225] The encoder performs encoding operation on the display image d2. Since there are no three reconstructed CRR image slices cached at this time, the currently effective reconstructed CRR image is still the first reconstructed CRR image. Select the reference image to perform encoding operation on the display image d2, and obtain the display image bit stream d2′ and the reconstructed display image (not shown in Figure 11). The encoder performs encoding operation on the CRR image slice e2, and obtains the CRR image slice bit stream e2′ and the reconstructed CRR image slice e2″. The encoder caches the reconstructed CRR image slice e2″.

[0226] The encoder encodes the display image d3. Since the three reconstructed CRR image slices are not cached at this time, the currently effective reconstructed CRR image is still the first reconstructed CRR image. The reference image d3 is selected and encoded to obtain the display image bitstream d3′ and the reconstructed display image (not shown in Figure 8). The encoding operation is performed on e3 to obtain the CRR image bitstream segment e3′ and the reconstructed CRR image slice e3″, and e3″ is cached.

[0227] At this time, three reconstructed CRR image slices e1″ to e3″ are cached in the cache space. The second reconstructed CRR image e″ is obtained by splicing e1″ to e3″, and the first reconstructed CRR image is removed from the cache space. The currently effective reconstructed CRR image is the reconstructed CRR image e″.

[0228] The encoder performs encoding operation on the display image d4, and the currently effective reconstructed CRR image is the reconstructed CRR image e″ (also called the second CRR image). A reference image is selected to perform encoding operation on the display image d4, and a display image bit stream d4′ and a reconstructed display image are obtained (not shown in Figure 11). The encoder performs encoding operation on the display image d5, and the currently effective reconstructed CRR image is the reconstructed CRR image e″. A reference image is selected to perform encoding operation on the display image d5, and a display image bit stream d5′ and a reconstructed display image are obtained (not shown in Figure 11). At this time, the encoded image bit stream is obtained.

[0229] Still referring to Figure 11, in this example, the decoder does not perform decoding operations when receiving the first n-1 CRR image bitstream segments. Specifically, first, the decoder performs a decoding operation on the display image bitstream d1′. Since the new CRR image bitstream segment has not been decoded at this time, only the first reconstructed CRR image is stored in the cache space, and the currently effective reconstructed CRR image is the first reconstructed CRR image. A reference image is selected to perform a decoding operation on the display image d1′ to obtain a reconstructed display image d1″. The reference image selected by the encoder for decoding d1′ is the same as the reference image selected for encoding d1. The decoder caches the received CRR image slice bitstream e1′.

[0230] The decoder decodes the display image bitstream d2′. Since three reconstructed CRR image slices have not yet been cached, the currently effective reconstructed CRR image is still the first reconstructed CRR image. A reference image is selected to decode the display image bitstream d2′, obtaining a reconstructed display image d2″. The reference image selected by the decoder for decoding d2′ is the same as the reference image selected for encoding d2. The decoder caches the received CRR image slice bitstream e2′.

[0231] The decoder decodes d3′. Since three reconstructed CRR image slices have not yet been cached, the currently valid reconstructed CRR image is still the first reconstructed CRR image. A reference image is selected to decode d3′ to obtain the reconstructed display image d3″. The reference image selected for decoding d3′ is the same as the reference image selected for encoding d3. The decoder caches the received CRR image slice bitstream e3′.

[0232] At this point, the cache space contains three CRR image slice bitstreams, e1′ to e3′. The decoder concatenates e1′ to e3′ to obtain a CRR image bitstream. The decoder decodes the CRR image bitstream to obtain a second reconstructed CRR image e″. Simultaneously, the first reconstructed CRR image is removed from the cache space. The currently active reconstructed CRR image is the reconstructed CRR image e″.

[0233] The decoder performs a decoding operation on the display image bit stream d4′, and the currently effective reconstructed CRR image is e″. A reference image is selected to perform a decoding operation on the display image d4′ to obtain the reconstructed display image d4″. The reference image selected by the decoder for the decoding operation on the display image bit stream d4′ is the same as the reference image selected for the encoding operation on the display image d4. Next, the decoder performs a decoding operation on the display image bit stream d5′, and the currently effective reconstructed CRR image is e″. A reference image is selected to perform a decoding operation on the display image bit stream d5′ to obtain the reconstructed display image d5″. The reference image selected for the decoding operation on the display image bit stream d5′ is the same as the reference image selected for the encoding operation on the display image d5.

[0234] As described above, in an embodiment of the present application, during the encoding and decoding process, the non-displayable CRR image bitstream is transmitted before the RAS that references the CRR image. Several possible interleaving and sorting methods of CRR image bitstreams are described below in conjunction with the accompanying drawings. Figure 12a is a schematic diagram of an exemplary interleaving and sorting method. Please refer to Figure 12a. The reference image of at least one display image in RAS2 is CRR image 1. In the image bitstream generated by the encoder, the CRR image slices corresponding to CRR image 1 (for example, including but not limited to CRR image slice 1-1, CRR image slice 1-2, CRR image slice 1-3) are interleaved and sorted with the display images in RAS1 (for example, including but not limited to frame 1, frame 2, frame 3 and frame 4) (the corresponding bitstreams are also interleaved and sorted). Figure 12b is a schematic diagram of an exemplary interleaving and sorting method. Please refer to Figure 12b. The reference image of at least one display image in RAS2 is CRR image 1. The reference image of at least one display image in RAS3 is CRR image 2. Accordingly, in the image bit stream generated by the encoder, the CRR image slices corresponding to CRR image 1 (for example, including but not limited to CRR image slice 1-1, CRR image slice 1-2, CRR image slice 1-3) are interleaved and sorted with the display images in RAS1 (for example, including but not limited to frame 1, frame 2, frame 3 and frame 4) (the corresponding bit streams are also interleaved and sorted). The CRR image slices corresponding to CRR image 2 (for example, including but not limited to CRR image slice 2-1, CRR image slice 2-2, CRR image slice 2-3) are interleaved and sorted with the display images in RAS2 (for example, including but not limited to frame 5, frame 6, frame 7 and frame 8) (the corresponding bit streams are also interleaved and sorted). Figure 12c is a schematic diagram of an exemplary interleaving sorting method. Please refer to Figure 12c. The reference image of at least one display image in RAS2 is CRR image 2. The reference image of at least one display image in RAS3 is CRR image 2. Accordingly, in the image bitstream generated by the encoder, the CRR image slices corresponding to CRR image 2 (for example, including but not limited to CRR image slice 2-1, CRR image slice 2-2, and CRR image slice 2-3) are sequentially ordered. In other words, the displayable CRR image slice bitstream is not interleaved with the display image bitstream.

[0235] In an embodiment of the present application, the CRR image identification information may include different types of CRR image identification information, thereby distinguishing between displayable CRR images and non-displayable CRR images, so that the decoder end can identify the displayable CRR images and non-displayable CRR images based on the CRR image identification information, and perform corresponding decoding operations on the displayable CRR images and perform corresponding decoding operations on the non-displayable CRR images. Optionally, the decoding operations on the displayable CRR images and the decoding operations on the non-displayable CRR images at the decoder end are different.

[0236] In a possible implementation, a display image can be used as a random access point in the RAS, while a non-displayable image cannot be used as a random access point.

[0237] In another possible implementation, the n CRR image slices of the CRR image may also be interleaved and sequenced with multiple RASs, wherein the multiple RASs precede the RAS that references the CRR image.

[0238] The following describes in detail the application scenarios of different CRR image identification information with specific embodiments.

[0239] 1) CRR image identification information includes:

[0240] CRR NUT is used to identify the non-border CRR image slice bitstream corresponding to the non-displayable CRR image, and can also be understood as indicating that the corresponding codestream carries the non-border CRR image slice of the non-displayable CRR image.

[0241] CRR_Delimiter NUT is used to indicate the boundary CRR image slice bit stream of the non-displayable CRR image. It can also be understood as being used to indicate that the corresponding code stream carries the boundary CRR image slice of the non-displayable CRR image. The boundary CRR image slice bit stream in the embodiment of the present application includes the first CRR image slice bit stream and the last CRR image slice bit stream in the CRR image slice. For example, as shown in Figure 12a, the slice bit stream corresponding to CRR image slice 1-1 is the first CRR image slice bit stream. The slice bit stream corresponding to CRR image slice 1-3 is the last CRR image slice bit stream. The slice bit stream corresponding to CRR image slice 1-2 is the non-boundary CRR image slice bit stream.

[0242] RL NUT is used to identify P frames or B frames that only refer to CRR. This image frame is usually a random access point in RAS. It can be understood that before the RL image bitstream carrying this identification information, all CRR image slices of the CRR image referenced by the RL image have been encoded or transmitted. For example, as shown in Figure 12a, frame 5 is the first RAP image that refers to CRR image 1. This frame is an RL image, and the bitstream corresponding to this image contains the RL NUT identifier. Optionally, the bitstreams corresponding to other RAP images that refer to the CRR image may also carry the RL NUT identifier or may not, which is not limited in this application. Optionally, in other embodiments, if the agreed image frame is not an RL image, the display image of the first reference CRR image may be marked with other marks (such as NUT), which is not limited in this application.

[0243] CRR_DP NUT is used to identify the CRR image slice bitstream that can display the CRR image. It can also be understood as indicating that the corresponding code stream carries the image slice that can display the CRR image.

[0244] In the embodiment of the present application, the first CRR image slice bit stream of the CRR image follows the header information bit stream of the CRR image, and the first CRR image slice bit stream is adjacent to the header information bit stream of the CRR image. In addition, there is no other CRR image bit stream between the display image bit stream corresponding to the first display image (usually RAP) that references the CRR image and the CRR image.

[0245] The following describes the actual application scenario in detail based on the above CRR image identification:

[0246] FIG13 is a flow chart showing an exemplary encoding method, which can be applied to an encoder and specifically includes but is not limited to the following steps:

[0247] S1301: Acquire a CRR image and multiple display images.

[0248] Exemplarily, the encoder obtains a CRR image. As described above, the CRR image in the embodiment of the present application includes a non-displayable CRR image and a displayable CRR image. The CRR image obtained by the encoder is a displayable CRR image or a non-displayable CRR image. Among them, the non-displayable CRR image can be a certain image to be encoded, or it can be an image constructed from multiple images to be encoded. The non-displayable CRR image does not need to be displayed and is only used as a reference image for inter-frame prediction encoding of the image to be encoded. The displayable CRR image is a certain image to be encoded and needs to be displayed. For specific description, please refer to the above and will not be repeated here.

[0249] Exemplarily, the encoder acquires multiple display images. The acquisition method can refer to existing technologies and is not limited in this application.

[0250] S1302: Divide the CRR image into n CRR image slices.

[0251] Exemplarily, the encoder segments the CRR image to obtain n CRR image slices (also referred to as CRR image slices, which will not be described in detail below). The segmentation method can be referred to the relevant description of FIG8 and will not be described in detail here.

[0252] S1303 : Acquire a sequence of images to be encoded based on n CRR image slices and multiple display images.

[0253] In an embodiment of the present application, for non-displayable CRR images, the encoder may interleave and sort n CRR image slices of the non-displayable CRR image with multiple display images. The implementation method can be referred to Figure 11 and will not be repeated here.

[0254] For a displayable CRR image, the encoder arranges n CRR image slices in sequence.

[0255] S1304: Encode the image sequence to be encoded to obtain an image bitstream. The bitstream of a non-displayable CRR image's boundary CRR image slice includes a CRR_Delimiter NUT identifier, and the bitstream of a non-border CRR image slice includes a CRR NUT identifier. The bitstream of a displayable CRR image's CRR image slice includes a CRR_DP NUT identifier. Furthermore, the bitstream of a reference CRR image's RL image includes an RL NUT identifier.

[0256] In the embodiment of the present application, the encoder encodes the image sequence to be encoded to obtain an image bit stream. The encoding method can be referred to the description in FIG11 and will not be repeated here.

[0257] In an embodiment of the present application, the encoder performs NALU encapsulation on each code stream (including the CRR image slice bit stream and the display image bit stream) to obtain a NALU bit stream. In addition, the encoder performs NALU encapsulation on the header information of the CRR image (including SPS, PPS, and PH, etc.) to obtain a CRR image header information bit stream (including but not limited to CRR image SPS bit stream, CRR image PPS bit stream, and CRR image PH bit stream). In addition, the encoder performs NALU encapsulation on the header information corresponding to each display image to obtain a display image header information bit stream.

[0258] The image code stream corresponding to the non-displayable CRR image and the code stream corresponding to the displayable CRR image are described in detail below.

[0259] a. Unable to display CRR image stream:

[0260] Figure 14 illustrates an exemplary image codestream structure for a non-displayable CRR image. In this example, the image sequence shown in Figure 12a is used for illustration. Referring to Figure 14 , the NUT field of the first RL image bitstream includes an RL NUT identifier. Optionally, if this image does not reference a previous CRR image, the RL NUT identifier may not be included. The RBSP (i.e., payload field) of the first RL image includes, among other things, RL image coding data.

[0261] Exemplarily, the NUT field of the CRR image slice 1-1 bit stream includes a CRR_Delimiter NUT identifier, which is used to indicate that the payload field of the bit stream carries the boundary slice image of the CRR image (corresponding encoded data). Exemplarily, as shown in Figure 14, CRR image slice 1-1 is the first CRR image slice of the CRR image in the encoding sequence. Accordingly, in the encoded bit stream, before the first CRR image slice bit stream and adjacent to the first CRR image slice bit stream, the header information bit stream of the CRR image is also included. Optionally, different types of header information bit streams can be encapsulated into different NALUs respectively, or they can be encapsulated into the same NALU, which is not limited in this application.

[0262] 14, according to the interleaving order shown in FIG12a, the CRR image slice 1-1 bit stream is followed by the display image bit stream, wherein the display image header information bit stream may be included before and adjacent to the display image bit stream (not shown in FIG14).

[0263] Exemplarily, according to the order of the image sequence to be encoded, the display image bitstream is followed by the CRR image slice 1-2 bitstream. The NUT field of the bitstream includes a CRR_NUT flag, which is used to indicate that the payload field of the bitstream carries the encoded data of the non-border image slices of the CRR image.

[0264] The bit streams of other images (including display images and CRR image slices) are similar and will not be described again here.

[0265] 14, CRR image slices 1-4 are the last CRR image slices of the CRR image. Accordingly, the NUT field of the CRR image slices 1-4 bitstream includes the CRR_Delimiter NUT flag. According to the interleaving order shown in FIG12a, the display image bitstream follows the CRR image slices 1-4 bitstream.

[0266] Exemplarily, the second RAP picture is a scheduled picture (i.e., the CRR picture is scheduled to be transmitted before this picture), that is, the first RAP picture that references CRR picture 1. Accordingly, the NUT field of the RL picture bitstream includes an RL NUT identifier, which is used to indicate that the codestream carries the coded data of the scheduled RL picture.

[0267] b.Can display CRR image stream:

[0268] Figure 15 is a schematic diagram illustrating an exemplary image codestream structure for a displayable CRR image. In this example, the image sequence shown in Figure 12c is used for illustration. Referring to Figure 15 , in this example, the bitstream corresponding to the RL image referenced by the CRR image may or may not include the RL NUT identifier, which is not a limitation in this application.

[0269] Exemplarily, as described above, in the image sequence to be encoded, n CRR image slices are arranged continuously, and correspondingly, in the image code stream, n CRR image slice bit streams are also arranged continuously. As shown in Figure 15, the CRR image slice 1-1 bit stream and the CRR image slice 1-2 bit stream are arranged continuously. Moreover, the NUT field of the CRR image slice 1-1 bit stream and the CRR image slice 1-2 bit stream includes a CRR_DP NUT identifier, which is used to indicate that the payload field of the corresponding code stream carries the encoded data of the image slice corresponding to the CRR image that can display the CRR image. The other CRR image slice bit streams are the same as the CRR image slice 1-2 bit stream and will not be described one by one here.

[0270] Exemplarily, according to the coded image sequence of FIG12c, the CRR image slices 1-4 bit stream is followed by the display image bit stream. For a detailed description, please refer to FIG14 and will not be repeated here.

[0271] In a possible implementation, the SPS, PPS, or PH information of the CRR image may include information on the number of image slices, which is used to indicate the number of image slices after the CRR image is segmented.

[0272] In another possible implementation, if the number of slices corresponding to the non-displayable CRR image is 1, the CRR image identifier in the CRR image slice code stream is a CRR NUT identifier.

[0273] FIG16 is a flow chart showing an exemplary decoding method, which can be applied to a decoder, specifically comprising: the decoder receives an image code stream, and the decoder decapsulates the NALU to read the NUT field in the NALU.

[0274] Exemplarily, the decoder receives a display image bit stream that does not include CRR image identification information and can decode the display image bit stream. Specific details can be found in the description of FIG11 and will not be repeated here.

[0275] In one example, if the decoder reads the NUT field in the NALU bitstream and finds the NUT field includes CRR_Delimiter NUT identification information, the decoder determines that the NALU bitstream is a CRR_Delimiter NALU, that is, a CRR image slice bitstream that cannot display the boundary of the CRR image. The decoder can obtain and cache the CRR image slice.

[0276] Furthermore, the decoder switches the CRR image receiving state to a "receiving non-displayable CRR image slice bit stream" state. The CRR image receiving state may be a default state before switching, which is not limited in this application.

[0277] Exemplarily, the decoder continues to receive the image codestream and decodes the received display codestream. Furthermore, if the decoder reads the NUT field in the NALU bitstream and finds that it includes CRR NUT identification information, it determines that the NALU bitstream is a CRR NALU, i.e., a non-border CRR image slice bitstream that cannot display a CRR image. The decoder obtains and caches the CRR image slices.

[0278] For example, if the decoder reads the NUT field in the NALU bitstream again and includes the CRR_Delimiter NUT identification information, it can be determined that the CRR_Delimiter NALU has been received again. The decoder obtains and caches the CRR image fragment. In addition, the decoder can determine that the CRR image reception is complete, and the decoder switches the CRR image reception state to the default state, that is, ends the "receiving non-displayable CRR image fragment bitstream" state.

[0279] In one possible implementation, if the decoder reads an RL NALU containing RL NUT identification information while in the "receiving a non-displayable CRR image slice bitstream" state, this can also be understood as the case where the RL NUT identification information is read before the CRR_Delimiter NUT identification information is read again. This can also be understood as the case where the RL image referencing the CRR image is received before the final CRR image slice of the CRR image is received. The decoder then determines that CRR image reception has failed. That is, in this example, no other CRR image bitstream exists between the CRR image bitstream and the RL image bitstream.

[0280] For example, Figure 17 is a schematic diagram of an exemplary image code stream. Please refer to Figure 17. In this example, under normal circumstances, the structure of the image bit stream should be as shown in Figure 14. However, due to other reasons, the CRR image slice 1-2 bit stream is followed by the RL image bit stream. Accordingly, after receiving the CRR image slice 1-1 bit stream, the decoder switches the CRR image reception state to the "receiving non-displayable CRR image slice bit stream" state. After receiving the RL image bit stream, it is recognized that the NUT field of the bit stream includes the RL NUT identifier, that is, it is determined that the RL image that references the CRR image is received. Then, the decoder determines that the CRR image reception has failed, that is, the CRR image reception is incomplete. Optionally, the decoder can take measures such as directly discarding, skipping the RAS bit stream that needs to reference this CRR image, and requesting retransmission. The specific method can be set according to actual needs and is not limited in this application.

[0281] Still referring to FIG. 16 , in another example, if the decoder reads the NUT field in the NALU bitstream and finds that the NUT field includes CRR DP NUT identification information, it determines that the NALU bitstream is a CRR DP NALU, which is a CRR image slice bitstream that can display a CRR image. The decoder decodes the CRR image slice. Furthermore, the decoder switches the CRR image reception state to the "receiving displayable CRR image slice bitstream" state.

[0282] As described above, the image slice bit streams that can display the CRR image are arranged continuously. Accordingly, the decoder decodes the corresponding CRR image slice bit streams in sequence in the "receiving displayable CRR image slice bit stream" state. When the decoder recognizes a bit stream that does not include the CRR DP NUT identifier, it can be determined that the CRR image reception is complete (which may be complete or incomplete). The decoder switches the CRR image reception state to the default state, that is, ends the "receiving displayable CRR image slice bit stream" state.

[0283] Exemplarily, the decoder splices the decoded reconstructed CRR image slices to obtain a reconstructed CRR image.

[0284] That is to say, in an embodiment of the present application, the decoder can identify non-displayable CRR images and displayable CRR images based on different CRR image identification information. For different types of CRR images, the decoder can adopt different reception completion judgment conditions and different decoding methods. For example, for non-displayable CRR images, the decoder can first cache the image slice bit stream, splice the image slice bit stream, and then decode the CRR image bit stream. For displayable CRR images, the decoder decodes each CRR image slice bit stream it receives, and obtains the corresponding CRR image based on the decoded image slice.

[0285] In an embodiment of the present application, as described above, the header information bit stream of the CRR image includes the number of CRR image slices. After the decoder determines that the CRR image reception is complete, that is, the state is switched to the default state, it can detect whether the CRR image slices are received completely based on the number of received CRR image slices. Optionally, the decoder can start a counter after receiving the first CRR image slice bit stream, and add 1 to the counter if no CRR image slice bit stream is received. In one example, if the number of received CRR image slice bit streams (i.e., the counter count result) is equal to the number of CRR image slices, it can be determined that the reception is complete. In another example, if the number of received image slice bit streams is not equal to the number of CRR image slices, it can be determined that the reception is incomplete. The decoder can take measures such as directly discarding, skipping the RAS bit stream that needs to refer to this CRR image, and requesting retransmission. The specific method can be set according to actual needs and is not limited by this application.

[0286] In a possible implementation, the process of decapsulating the NALU and identifying the identification information may also be performed by other modules of the electronic device, for example, by the system layer. After the system layer determines that a complete CRR image has been received, the encoded data of the CRR image fragments carried in the bitstream may be output to the decoder. The decoder may splice and decode the encoded data of the CRR image.

[0287] In another possible implementation, when the decoder is in the "receiving non-displayable CRR image slice bitstream" state, it receives the header information bitstream of another CRR image and the CRR_Delimiter NALU (i.e., the boundary CRR image slice bitstream of another non-displayable CRR image) (which can also be CRR DP NUT) again. The decoder can then determine that the bitstream transmission of the currently received CRR image is finished, and the decoder can continue to receive the image slice bitstream corresponding to another CRR image in the "receiving non-displayable CRR image slice bitstream" state. It should be noted that in the embodiment of the present application, the fact that other CRR image bitstreams are not included between the CRR image bitstream described above and the RL bitstream that references the CRR image can be understood as meaning that no other CRR image bitstreams can be inserted before the CRR image transmission is completed (i.e., the CRR_Delimiter NUT identifier is recognized again).

[0288] In another possible implementation, the system layer processes the decoded image sequence (including the CRR image and the display image) including but not limited to:

[0289] 1. When RL NALU, CR_DP NALU or other display image NALU is received, it means that the received image needs to be output after being decoded into a reconstructed image. When encapsulated with the system layer protocol, operations such as timestamp can be performed. When storing, each image bit stream can be stored in the order in which the code stream is received.

[0290] 2. When CRR NALU or CRR_Delimiter NALU is received, it means that the received bit stream does not need to be output after being decoded into a reconstructed image, and the bit stream of the image may be divided into multiple bit stream slices and interleaved with the code stream of the display image. The system layer can choose to perform some special processing on the image, such as building a table to index the offset of each bit stream slice in the file storage or the bit stream of the entire coding sequence, so as to facilitate subsequent indexing and searching of all bit stream slices of the image.

[0291] 3. When random access occurs to a video bitstream with a CRR image bitstream, the system layer searches for the nearest random access point near the location of the requested random placement point. If an RL NALU or CRR_DP NALU is found, the location can be used as the random access point. If the random access point found is an RL NALU, the system layer searches for CRR NALUs and CRR_Delimiter NALUs starting from the NALU of the RL image. When a CRR_Delimiter NALU is followed by a header information NALU, the header information NALUs and all CRR_Delimiter NALUs and CRR NALUs are collected in the order in the bitstream for splicing and reorganization and input into the decoder for decoding. The decoded CRR image is used as the reference image of the required random access point RL image.

[0292] 2) The CRR image identification information includes: CRR NUT, RL NUT and CRR_DP NUT. The detailed description can be found above and will not be repeated here.

[0293] The following describes the actual application scenario in detail based on the above CRR image identification:

[0294] FIG18 is a flow chart showing an exemplary encoding method, which may be applied to an encoder and specifically includes but is not limited to the following steps:

[0295] S1801, acquiring a CRR image and multiple display images.

[0296] S1802: Divide the CRR image into n CRR image slices.

[0297] S1803 : Acquire a sequence of images to be encoded based on n CRR image slices and multiple display images.

[0298] S1801 to S1803 may refer to S1301 to S1303 and will not be described in detail here.

[0299] S1804: Encode the image sequence to be encoded to obtain an image bitstream. The CRR image slice bitstream for non-displayable CRR images includes a CRR NUT identifier. The CRR image slice bitstream for displayable CRR images includes a CRR_DP NUT identifier. Furthermore, the RL image bitstream for reference CRR images includes an RL NUT identifier.

[0300] In the embodiment of the present application, the encoder encodes the image sequence to be encoded to obtain an image bit stream. The encoding method can be referred to the description in FIG11 and will not be repeated here.

[0301] In an embodiment of the present application, the encoder performs NALU encapsulation on each code stream (including the CRR image slice bit stream and the display image bit stream) to obtain a NALU bit stream. In addition, the encoder performs NALU encapsulation on the header information of the CRR image (including SPS, PPS, and PH, etc.) to obtain a CRR image header information bit stream (including but not limited to CRR image SPS bit stream, CRR image PPS bit stream, and CRR image PH bit stream). In addition, the encoder performs NALU encapsulation on the header information corresponding to each display image to obtain a display image header information bit stream.

[0302] The image code stream corresponding to the non-displayable CRR image and the code stream corresponding to the displayable CRR image are described in detail below.

[0303] a. Unable to display CRR image stream:

[0304] Figure 19 illustrates an exemplary image codestream structure for a non-displayable CRR image. In this example, the image sequence shown in Figure 12a is used for illustration. Referring to Figure 19 , the NUT field of the first RL image bitstream includes an RL NUT identifier. Optionally, if this image does not reference a previous CRR image, the RL NUT identifier may not be included. The RBSP (i.e., payload field) of the first RL image includes, among other things, RL image coding data.

[0305] Exemplarily, the NUT field of the CRR image slice 1-1 bit stream includes a CRR NUT identifier, which is used to indicate that the payload field of the bit stream carries the image slice of the CRR image (corresponding encoded data). Exemplarily, as shown in Figure 19, CRR image slice 1-1 is the first CRR image slice of the CRR image in the encoding sequence. Accordingly, in the encoded bit stream, before the first CRR image slice bit stream and adjacent to the first CRR image slice bit stream, the header information bit stream of the CRR image is also included. Optionally, different types of header information bit streams can be encapsulated into different NALUs respectively, or they can be encapsulated into the same NALU, which is not limited in this application.

[0306] 19, according to the interleaving order shown in FIG12a, the CRR image slice 1-1 bit stream is followed by the display image bit stream, wherein the display image header information bit stream may be included before and adjacent to the display image bit stream (not shown in FIG14).

[0307] Illustratively, according to the order of the image sequence to be coded, the display image bitstream is followed by the CRR image slice 1-2 bitstream. The NUT field of the bitstream includes a CRR NUT identifier, which is used to indicate that the payload field of the codestream carries coded data of the image slice of the CRR image.

[0308] The bit streams of other images (including display images and CRR image slices) are similar and will not be described again here.

[0309] Exemplarily, the second RAP picture is a scheduled picture (i.e., the CRR picture is scheduled to be transmitted before this picture), that is, the first RAP picture that references CRR picture 1. Accordingly, the NUT field of the RL picture bitstream includes an RL NUT identifier, which is used to indicate that the codestream carries the coded data of the scheduled RL picture.

[0310] b.Can display CRR image stream:

[0311] For a detailed description of the displayable CRR code stream, please refer to FIG15 , which will not be repeated here.

[0312] FIG20 is a flow chart showing an exemplary decoding method, which can be applied to a decoder, specifically comprising: the decoder receives an image code stream, and the decoder decapsulates the NALU to read the NUT field in the NALU.

[0313] Exemplarily, the decoder receives a display image bit stream that does not include CRR image identification information and can decode the display image bit stream. Specific details can be found in the description of FIG11 and will not be repeated here.

[0314] In one example, if the decoder reads a NALU bitstream containing a CRR NUT identifier in the NUT field, it determines that the NALU bitstream is a CRR NALU, i.e., a CRR image slice bitstream that cannot display a CRR image. The decoder can obtain and cache the CRR image slice. Furthermore, the decoder starts a counter to count.

[0315] Exemplarily, the decoder continues to receive the image codestream and decodes the received display codestream. Furthermore, if the decoder reads the NUT field in the NALU bitstream and finds that it includes CRR NUT identification information, it determines that the NALU bitstream is a CRR NALU, i.e., a CRR image slice bitstream that cannot display a CRR image. The decoder obtains and caches the CRR image slice. Furthermore, the decoder increments the counter by 1.

[0316] Exemplarily, the decoder compares the count result of the counter with the number n of CRR image slices included in the header information bitstream of the CRR image. If the count result is less than n, it is determined that the CRR image has not been completely received. If the count result is equal to n, it is determined that the CRR image has been completely received. The decoder may splice the CRR image slice bitstreams to obtain a CRR image bitstream, and decode the CRR image bitstream to obtain a reconstructed CRR image.

[0317] In one possible implementation, if the decoder reads an RL NALU containing RL NUT identification information and the count result of the counter is currently less than n, the decoder determines that the CRR image reception has failed (i.e., the reception is incomplete). Optionally, the decoder can adopt a response method such as directly discarding, skipping the RAS bit stream that needs to refer to this CRR image, and requesting retransmission. The specific method can be set according to actual needs and is not limited in this application.

[0318] Still referring to FIG. 20 , in another example, if the decoder reads the NUT field in the NALU bitstream and finds that the NUT field includes CRR DP NUT identification information, it determines that the NALU bitstream is a CRR DP NALU, and thus displays the boundary CRR image slice bitstream of the CRR image. The decoder decodes the CRR image slice and starts counting.

[0319] As described above, the image slice bit streams that can display the CRR image are arranged continuously. Accordingly, the decoder decodes the corresponding CRR image slice bit streams in sequence, and the counter is incremented by 1 each time a CRR DP NALU is received.

[0320] When the decoder identifies a bitstream that does not include the CRR DP NUT identifier, it can determine that the CRR image reception is complete (which may be complete or incomplete). Exemplarily, if the technical result of the counter is equal to n, the decoder determines that the CRR image reception is complete. The decoder image can splice the reconstructed CRR image slices to obtain a reconstructed CRR image.

[0321] In one possible implementation, if the decoder reads a bit stream that does not include the CRR DP NUT identifier and the count result of the counter is currently less than n, the decoder determines that the CRR image reception has failed (i.e., the reception is incomplete). Optionally, the decoder can adopt a response method such as directly discarding, skipping the RAS bit stream that needs to refer to this CRR image, and requesting retransmission. The specific method can be set according to actual needs and is not limited in this application.

[0322] That is to say, in an embodiment of the present application, the decoder can identify non-displayable CRR images and displayable CRR images based on different CRR image identification information. For different types of CRR images, the decoder can adopt different reception completion judgment conditions and different decoding methods. For example, for non-displayable CRR images, the decoder can first cache the image slice bit stream, splice the image slice bit stream, and then decode the CRR image bit stream. For displayable CRR images, the decoder decodes each CRR image slice bit stream it receives, and obtains the corresponding CRR image based on the decoded image slice.

[0323] In one possible implementation, when the decoder receives the current CRR image, it again receives the header information bit stream and CRR image identifier (which can be CRR NUT or CRR DP NUT) of another CRR image. The decoder can then determine that the bit stream transmission of the currently received CRR image has ended, and the decoder continues to receive the image slice bit stream corresponding to another CRR image, and the counter is reset and restarted.

[0324] In another possible implementation, the system layer processes the decoded image sequence (including the CRR image and the display image) including but not limited to:

[0325] 1. When a display image NALU or a displayable CRR image NALU is received, the operation is the same as in the above embodiment.

[0326] 2. When a CRR NALU is received, it indicates that the received image does not need to be output after being decoded into a reconstructed image. The operation is the same as in the above embodiment.

[0327] 3. When random access occurs to a video bitstream with a CRR image bitstream, the system layer searches for the nearest random access point near the location of the requested random point request point. If an RL NALU or CRR_DP NALU is found, the location can be used as the random access point. If the searched random access point is an RL NALU, the system layer searches for the CRR NALU from the NALU of the RL image forward. When a NUT is a CRR NUT NALU followed by a header information NALU, these header information NALUs and all CRR NALUs are collected in the order of the bitstream, spliced ​​and reassembled, and input into the decoder for decoding. The decoded CRR image is used as the reference image of the required random access point RL image.

[0328] 3) CRR image identification information includes:

[0329] The detailed description of CRR NUT, RL NUT and CRR_DP NUT can be found above and will not be repeated here.

[0330] CRR_START NUT is used to indicate the first CRR picture slice bit stream of a non-displayable CRR picture.

[0331] CRR_END NUT is used to indicate the end CRR picture slice bit stream of a non-displayable CRR picture.

[0332] The following describes the actual application scenario in detail based on the above CRR image identification:

[0333] FIG21 is a flow chart showing an exemplary encoding method, which can be applied to an encoder and specifically includes but is not limited to the following steps:

[0334] S2101, acquiring a CRR image and multiple display images.

[0335] S2102: Divide the CRR image into n CRR image slices.

[0336] S2103 : Acquire a sequence of images to be encoded based on n CRR image slices and multiple display images.

[0337] S2101 to S2103 can refer to S1301 to S1303 and will not be repeated here.

[0338] S2104: Encode the image sequence to be encoded to obtain an image bitstream. The first CRR image slice bitstream of a non-displayable CRR image includes a CRR_START NUT identifier, the last CRR image slice bitstream includes a CRR_END NUT identifier, and the non-border CRR image slice bitstream includes a CRR NUT identifier. The CRR image slice bitstream of a displayable CRR image includes a CRR_DP NUT identifier. Furthermore, the RL image bitstream of a reference CRR image includes an RL NUT identifier.

[0339] In the embodiment of the present application, the encoder encodes the image sequence to be encoded to obtain an image bit stream. The encoding method can be referred to the description in FIG11 and will not be repeated here.

[0340] In an embodiment of the present application, the encoder performs NALU encapsulation on each code stream (including the CRR image slice bit stream and the display image bit stream) to obtain a NALU bit stream. In addition, the encoder performs NALU encapsulation on the header information of the CRR image (including SPS, PPS, and PH, etc.) to obtain a CRR image header information bit stream (including but not limited to CRR image SPS bit stream, CRR image PPS bit stream, and CRR image PH bit stream). In addition, the encoder performs NALU encapsulation on the header information corresponding to each display image to obtain a display image header information bit stream.

[0341] The image code stream corresponding to the non-displayable CRR image and the code stream corresponding to the displayable CRR image are described in detail below.

[0342] a. Unable to display CRR image stream:

[0343] Figure 22 illustrates an exemplary image codestream structure for a non-displayable CRR image. In this example, the image sequence shown in Figure 12a is used for illustration. Referring to Figure 22 , the NUT field of the first RL image bitstream includes an RL NUT identifier. Optionally, if this image does not reference a previous CRR image, the RL NUT identifier may not be included. The RBSP (i.e., payload field) of the first RL image includes, among other things, RL image coding data.

[0344] Exemplarily, the NUT field of the CRR image slice 1-1 bit stream includes a CRR_START NUT identifier, which is used to indicate that the payload field of the bit stream carries the first image slice of the CRR image (corresponding encoded data). Exemplarily, as shown in FIG22 , CRR image slice 1-1 is the first CRR image slice of the CRR image in the encoding sequence. Accordingly, in the encoded bit stream, before the first CRR image slice bit stream and adjacent to the first CRR image slice bit stream, the header information bit stream of the CRR image is also included. Optionally, different types of header information bit streams can be encapsulated into different NALUs respectively, or they can be encapsulated into the same NALU, which is not limited in this application.

[0345] 22, according to the interleaving order shown in FIG12a, the CRR image slice 1-1 bit stream is followed by the display image bit stream, wherein the display image header information bit stream may be included before and adjacent to the display image bit stream (not shown in FIG22).

[0346] Exemplarily, according to the order of the image sequence to be encoded, the display image bitstream is followed by the CRR image slice 1-2 bitstream. The NUT field of the bitstream includes a CRR_NUT flag, which is used to indicate that the payload field of the bitstream carries the encoded data of the non-border image slices of the CRR image.

[0347] The bit streams of other images (including display images and CRR image slices) are similar and will not be described again here.

[0348] Still referring to Figure 22, CRR image slices 1-4 are the end CRR image slices. Accordingly, the NUT field of the CRR image slices 1-4 bitstream includes the CRR_END NUT identifier. According to the interleaved order shown in Figure 12a, the display image bitstream follows the CRR image slices 1-4 bitstream.

[0349] Exemplarily, the second RAP picture is a scheduled picture (i.e., the CRR picture is scheduled to be transmitted before this picture), that is, the first RAP picture that references CRR picture 1. Accordingly, the NUT field of the RL picture bitstream includes an RL NUT identifier, which is used to indicate that the codestream carries the coded data of the scheduled RL picture.

[0350] b.Can display CRR image stream:

[0351] For a detailed description, please refer to FIG15 , which will not be repeated here.

[0352] FIG23 is a flow chart showing an exemplary decoding method, which can be applied to a decoder, specifically comprising: the decoder receives an image code stream, and the decoder decapsulates the NALU to read the NUT field in the NALU.

[0353] Exemplarily, the decoder receives a display image bit stream that does not include CRR image identification information and can decode the display image bit stream. Specific details can be found in the description of FIG11 and will not be repeated here.

[0354] In one example, if the decoder reads a NALU bitstream containing the CRR_START NUT identifier in the NUT field, it determines that the NALU bitstream is a CRR_STAR NALU, i.e., the bitstream of the first CRR image slice that cannot display a CRR image. The decoder can obtain and cache the CRR image slice.

[0355] Furthermore, the decoder switches the CRR image receiving state to a "receiving non-displayable CRR image slice bit stream" state. The CRR image receiving state may be a default state before switching, which is not limited in this application.

[0356] Exemplarily, the decoder continues to receive the image code stream and decodes the received display code stream. Furthermore, if the decoder reads the NUT field in the NALU bit stream and finds that it includes CRR NUT identification information, the decoder determines that the NALU bit stream is a CRR NALU, i.e., a non-border CRR image slice bit stream that cannot display a CRR image. The decoder obtains and caches the CRR image slice.

[0357] For example, if the decoder reads the NUT field in the NALU bitstream and includes CRR_END NUT identification information, it can determine that the CRR_END NALU has been received, that is, the end CRR image slice bitstream of the non-displayable CRR image. The decoder obtains and buffers the CRR image slice. In addition, the decoder can determine that the CRR image reception is complete, and the decoder switches the CRR image reception state to the default state, that is, ends the "receiving non-displayable CRR image slice bitstream" state.

[0358] In one possible implementation, if the decoder reads an RL NALU containing RL NUT identification information while in the "receiving non-displayable CRR image slice bitstream" state, the decoder determines that CRR image reception has failed. That is, in this example, no other CRR image bitstream exists between the CRR image bitstream and the RL image bitstream.

[0359] The decoding of the displayable CRR image by the decoder can be referred to FIG16 , which will not be described in detail here.

[0360] That is to say, in an embodiment of the present application, the decoder can identify non-displayable CRR images and displayable CRR images based on different CRR image identification information. For different types of CRR images, the decoder can adopt different reception completion judgment conditions and different decoding methods. For example, for non-displayable CRR images, the decoder can first cache the image slice bit stream, splice the image slice bit stream, and then decode the CRR image bit stream. For displayable CRR images, the decoder decodes each CRR image slice bit stream it receives, and obtains the corresponding CRR image based on the decoded image slice.

[0361] In an embodiment of the present application, as described above, the header information bit stream of the CRR image includes the number of CRR image slices. After the decoder determines that the CRR image reception is complete, that is, the state is switched to the default state, it can detect whether the CRR image slices are received completely based on the number of received CRR image slices. Optionally, the decoder can start a counter after receiving the first CRR image slice bit stream, and add 1 to the counter if no CRR image slice bit stream is received. In one example, if the number of received CRR image slice bit streams (i.e., the counter count result) is equal to the number of CRR image slices, it can be determined that the reception is complete. In another example, if the number of received image slice bit streams is not equal to the number of CRR image slices, it can be determined that the reception is incomplete. The decoder can take measures such as directly discarding, skipping the RAS bit stream that needs to refer to this CRR image, and requesting retransmission. The specific method can be set according to actual needs and is not limited by this application.

[0362] In a possible implementation, the process of decapsulating the NALU and identifying the identification information may also be performed by other modules of the electronic device, for example, by the system layer. After the system layer determines that a complete CRR image has been received, the encoded data of the CRR image fragments carried in the bitstream may be output to the decoder. The decoder may splice and decode the encoded data of the CRR image.

[0363] In another possible implementation, when the decoder is in the "receiving non-displayable CRR image slice bit stream" state, it receives the header information bit stream of another CRR image and CRR_START NALU (i.e., the boundary CRR image slice bit stream of another non-displayable CRR image) (which can also be CRR DP NUT) again. Then, the decoder can determine that the bit stream transmission of the currently received CRR image is completed, and the decoder can continue to receive the image slice bit stream corresponding to another CRR image in the "receiving non-displayable CRR image slice bit stream" state.

[0364] In another possible implementation, the system layer processes the decoded image sequence (including the CRR image and the display image) including but not limited to:

[0365] 1. When a display image NALU or a displayable CRR image NALU is received, the operation is the same as in the above embodiment.

[0366] 2. When CRR NALU, CRR_START NALU or CRR_END NALU is received, it means that the received image does not need to be output after being decoded into a reconstructed image, and the operation is the same as the above embodiment.

[0367] 3. When random access occurs to a video bitstream with a CRR image bitstream, the system layer searches for the nearest random access point near the location of the requested random placement point. If an RL NALU or CRR_DP NALU is found, the location can be used as the random access point. If the random access point found is an RL NUT, the system layer searches for CRR NALU and CRR END NALU starting from the NALU of the RL image. When a CRR_START NALU is followed by a header information NALU, the header information NALUs, CRR_START NALU, CRR END NALU and CRR NALU are collected in the order in the bitstream, spliced ​​and reassembled, and input into the decoder for decoding. The decoded CRR image is used as the reference image of the required random access point RL image.

[0368] 3) The CRR image identification information includes: any one of 1), 2) and 3) above.

[0369] In this example, the encoder can set the header information NUT of the CRR image, including but not limited to:

[0370] CRR_SPS NUT is used to indicate that the SPS of the CRR image cannot be displayed.

[0371] CRR_PPS NUT is used to indicate that the PPS of the CRR image cannot be displayed.

[0372] CRR_PH NUT is used to indicate the PH where the CRR image cannot be displayed.

[0373] In this example, the header information NALU and the display image NALU can be distinguished by different header information NUT fields. That is, the decoder can identify the header information NALU by reading the NUT field.

[0374] FIG24 is a schematic diagram of the structure of the header information NALU. Referring to FIG24 , the NUT field in the CRC image SPS bit stream includes the CRR_SPS NUT identifier, the NUT field in the CRR image PPS bit stream includes the CRR_PPS NUT identifier, and the NUT field in the CRR image PH bit stream includes the CRR_PH NUT identifier.

[0375] Alternatively, in this example, the SPS bit stream, the PPS bit stream, and the PH bit stream may be arranged sequentially and continuously. Alternatively, the SPS bit stream, the PPS bit stream, and the PH bit stream may be arranged sequentially but discontinuously (i.e., there may be other image bit streams in between). Alternatively, the SPS bit stream, the PPS bit stream, and the PH bit stream may be arranged discontinuously and not in sequence, which is not limited in this application.

[0376] In this example, the SPS bitstream, the PPS bitstream, and the PH bitstream are still located before the first CRR picture slice bitstream. Alternatively, the SPS bitstream, the PPS bitstream, and the PH bitstream may be adjacent to the first CRR picture slice bitstream.

[0377] In this example, the SPS bit stream, the PPS bit stream, the PH bit stream and the first CRR picture slice bit stream may not be adjacent to each other, that is, the SPS bit stream, the PPS bit stream and the PH bit stream may be interleaved with the display picture bit stream.

[0378] In this example, the processing of the encoder and the decoder can refer to any of the above embodiments and will not be repeated here.

[0379] In another possible implementation, the system layer processes the decoded image sequence (including the CRR image and the display image) including but not limited to:

[0380] 1. When a display image NALU or a displayable CRR image NALU is received, the operation is the same as in the above embodiment.

[0381] 2. When a CRR NALU or CRR_Delimiter NALU is received, it indicates that the received image does not need to be output after being decoded into a reconstructed image. The operation is the same as in the above embodiment.

[0382] 3. When random access occurs to a video bitstream with a CRR image bitstream, the system layer searches for the nearest random access point near the location of the requested random placement point. If the RL NALU or CRR_DP NALU is found, the location can be used as the random access point. If the random access point found is RL NUT, the system layer searches for CRR NALU and CRR_Delimiter NALU from the NALU of the RL image. When the second CRR_Delimiter NALU is found, all image coding data bitstream segments that cannot display the CRR image have been found. Then, CRR_PH NALU, CRR_PPS NALU and CRR_PH NALU are found in sequence. These header information NALUs and all CRR_Delimiter NALUs and CRR NALUs are collected in the order in the bitstream, spliced ​​and reassembled, and input into the decoder for decoding. The decoded CRR image is used as the reference image of the required random access point RL image.

[0383] Based on the above embodiment, the embodiment of the present application further provides an identification method. When encoding, the encoder may include a CRR image identification and index information in the NUT field of the CRR image slice bitstream. The RAP bitstream in the RAS (which may be one or more) that references the CRR image, or the NUT field of each display image bitstream in the RAS that references the CRR image, also includes the index information. This indicates the reference relationship between the RAS and the CRR image.

[0384] FIG25 is a schematic diagram illustrating an exemplary image bitstream structure. Referring to FIG25 , in this example, in the image bitstream generated by the encoder, the NUT field of each CRR image slice bitstream includes an index X. The NUT field of the RL image bitstream that references the CRR image also includes an index X.

[0385] In this example, since the CRR image and the RAS that references the CRR image can be associated through an index, correspondingly, other CRR images can be included between the CRR image and the RAS that references the CRR image. Accordingly, during the decoding process, if the decoder adopts the method in 1) or 3), each time the decoder receives a CRR image bit stream (that is, receives the header information bit stream and the first CRR image slice bit stream), the decoder sets the CRR image reception state corresponding to each CRR image. The decoder can determine whether the reception of the CRR image is completed based on the index information and CRR image identifier in the CRR image slice bit stream. For example, taking method 1), if the decoder reads the CRR_Delimiter NALU and the corresponding index information read is index X, the decoder can set the CRR reception state corresponding to the CRR image corresponding to index X, and switch the state to "receiving non-displayable CRR image slice bit stream". Accordingly, if the decoder reads the CRR_Delimiter NALU again and the corresponding index information read is index X, it can be determined that the CRR image bit stream corresponding to index X has been received and the corresponding CRR image reception state is switched. Alternatively, if the decoder receives a CRR_Delimiter NALU containing index Y before receiving the CRR_Delimiter NALU containing index X, the decoder can determine that the CRR image corresponding to index Y has been received, but the CRR image corresponding to index X has not yet been received.

[0386] In another possible implementation, the system layer processes the decoded image sequence (including the CRR image and the display image) including but not limited to:

[0387] 1. When a display image NALU or a displayable CRR image NALU is received, the operation is the same as in the above embodiment.

[0388] 2. When a CRR NALU or CRR_Delimiter NALU is received, it indicates that the received image does not need to be output after being decoded into a reconstructed image. The operation is the same as in the above embodiment.

[0389] 3. When random access occurs to a video bitstream with a CRR image bitstream, the system layer searches for the nearest random access point near the location of the requested random placement point. If an RL NALU or CRR_DP NALU is found, this location can be used as the random access point. If the random access point found is an RL NUT, the system layer identifies the index identifier of the CRR image referenced by the RL image from the NALU header of the RL image, and searches forward from the RL NALU for the CRR NALU and CRR_Delimiter NALU with the corresponding index identifier in the NALU header. When a NALU with a CRR_Delimiter NUT immediately follows the header information NALU, these header information NALUs, all CRR_Delimiter NALUs, and CRR NALUs are collected in the order in the bitstream, spliced ​​and reassembled, and input into the decoder for decoding. The decoded CRR image is used as the reference image of the required random access point RL image.

[0390] It should be noted that the "before" and "after" involved in the embodiments of the present application refer to the time sequence, for example, before and after in the encoding order, or before and after in the transmission process, or before and after in the decoding order, and this application does not limit this.

[0391] The present application also provides an encoding device 2600, as shown in Figure 26, which is an example diagram of an image processing device provided in the present application. Device 2600 has the functions of implementing the encoder described in the above embodiments. For example, the functions, units, or means may be implemented in software or hardware, or the corresponding software implementation may be executed by hardware.

[0392] For example, referring to FIG. 26 , the apparatus 2600 may include:

[0393] An acquisition module 2601 is configured to acquire n CRR image slices of a plurality of display images and a cross-random access point reference CRR image; n is an integer greater than 0;

[0394] The encoding module 2602 is used to encode multiple display images and CRR images to obtain an image code stream; wherein the image code stream includes n CRR image slice code streams and multiple display image code streams; each CRR image slice code stream in the n CRR image slice code streams includes CRR image identification information, and the CRR image identification information is used to indicate that the corresponding code stream carries a CRR image slice.

[0395] The present application also provides an encoding device 2700, as shown in Figure 27, which is an example diagram of an image processing device provided in the present application. Device 2700 has the functions of implementing the encoder described in the above embodiments. For example, the functions, units, or means may be implemented in software or hardware, or the corresponding software implementation may be executed by hardware.

[0396] For example, referring to FIG. 27 , the apparatus 2700 may include:

[0397] Receiving module 2701 is configured to receive an image stream, wherein the image stream includes: n CRR image slice streams and multiple display image streams; each of the n CRR image slice streams includes CRR image identification information, and the CRR image identification information is used to indicate that the corresponding stream carries a CRR image slice;

[0398] A decoding module 2702 is configured to decode n CRR image slice streams in an image stream based on the CRR image identification information;

[0399] A decoding module 2702 is configured to obtain a CRR image based on the decoded n CRR image slices; and

[0400] The decoding module 2702 is further configured to decode multiple display image code streams to obtain multiple display images.

[0401] The present application also provides an electronic device 2800, as shown in Figure 28, which is an example diagram of an electronic device provided by the present application. The method is performed by the encoder or decoder in the above embodiments.

[0402] As shown in Figure 28, the electronic device 2800 may include a processor 2801 for executing programs or instructions stored in the memory 2802. When the programs or instructions stored in the memory 2802 are executed, the processor is used to execute the method in the aforementioned embodiment of the present application.

[0403] Optionally, the electronic device 2800 may further include a communication interface 2803. FIG28 uses dashed lines to indicate that the communication interface 2803 is optional for the electronic device 2800.

[0404] Among them, the number of processors 2801, memories 2802 and communication interfaces 2803 does not constitute a limitation on the embodiments of the present application. During specific implementation, they can be arbitrarily configured according to business requirements.

[0405] Optionally, the memory 2802 is located outside the electronic device 2800 .

[0406] Optionally, the electronic device 2800 includes the memory 2802, which is connected to the at least one processor 2801 and stores instructions executable by the at least one processor 2801. FIG28 uses dotted lines to indicate that the memory 282 is optional for the electronic device 2800.

[0407] The processor 2801 and the memory 2802 may be coupled via an interface circuit or may be integrated together, which is not limited here.

[0408] The specific connection medium between the processor 2801, memory 2802, and communication interface 2803 is not limited in the embodiments of the present application. In Figure 28, the processor 2801, memory 2802, and communication interface 2803 are connected via a bus 2804. The bus is represented by a bold line in Figure 28. The connection method between other components is only for schematic illustration and is not intended to be limiting. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one bold line is used in Figure 28, but this does not mean that there is only one bus or one type of bus.

[0409] It should be understood that the processors mentioned in the embodiments of the present application can be implemented by hardware or software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented by software, the processor can be a general-purpose processor that is implemented by reading software code stored in a memory.

[0410] Exemplarily, the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0411] It should be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0412] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, the memory (storage module) can be integrated into the processor.

[0413] It should be noted that the memory described herein is intended to include, but not be limited to, these and any other suitable types of memory.

[0414] An embodiment of the present application further provides a computer-readable storage medium, comprising a program or instructions. When the program or instructions are run on a computer, the encoding and decoding method in the embodiment of the present application is executed.

[0415] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0416] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each flow and / or box in the flow chart and / or block diagram, as well as the combination of the flow chart and / or box in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more flow charts and / or one or more boxes in the block diagram.

[0417] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0418] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0419] Obviously, those skilled in the art may make various modifications and variations to this application without departing from the spirit and scope of this application. Thus, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application is intended to include such modifications and variations.

Claims

1. A coding method, characterized in that: include: Obtaining n reference image slices of a plurality of display images and a reference image; n is an integer greater than 0; The multiple display images and the reference image are encoded to obtain an image bit stream; wherein the image bit stream includes n reference image slice bit streams and multiple display image bit streams, the reference image slice bit stream is a network abstraction layer NAL unit structure, and the control field of the NAL unit of the reference image slice bit stream includes reference image identification information, and the reference image identification information is used to indicate that the load field in the NAL unit of the reference image bit stream carries the reference image slice.

2. The method according to claim 1, characterized in that The multiple display images include a reference knowledge base RL image, where the reference image is a reference frame when the RL image is subjected to inter-frame prediction.

3. The method according to claim 1, characterized in that The multiple display images include multiple random access segments RAS, and the reference image is a reference frame of the multiple RAS.

4. The method according to claim 1, characterized in that: The reference image is a cross-random access point reference CRR image.

5. The method according to claim 1, characterized in that When the reference image is a non-displayable reference image and n is greater than 1, the first reference image slice bitstream and the last reference image slice bitstream among the n reference image slice bitstreams include cross-random access point reference boundary network abstraction layer unit type CRR Delimiter NUT identification information, and other reference image slice bitstreams include cross-random access point reference network abstraction layer unit type CRR NUT identification information; wherein the CRR NUT identification information is used to indicate that the corresponding reference image slice is a non-displayable reference image slice, and the CRR Delimiter NUT identification information is used to indicate that the corresponding reference image slice is a non-displayable boundary reference image slice.

6. The method according to claim 1, characterized in that The reference image is a non-displayable reference image, and when n is equal to 1, the reference image identification information is CRR NUT identification information.

7. The method according to claim 1, characterized in that The reference image is a displayable reference image, and the reference image identification information includes cross-random access point reference display network abstraction layer unit type CRR DP NUT identification information, wherein the control field of the NAL unit corresponding to each of the reference image slice bitstreams includes the CRR DP NUT identification information.

8. The method according to any one of claims 1 to 7, characterized in that: The n reference image slice bit streams in the image bit stream are interleaved and arranged with the plurality of display image slice bit streams.

9. The method according to claim 2, characterized in that: The display image bit stream is a NAL unit structure, the multiple display image bit streams include a RL image bit stream, and the RL image bit stream is after the n reference image slice bit streams; The control field of the NAL unit corresponding to the RAP image bit stream includes a reference image network abstraction layer unit type RL NUT identifier.

10. The method according to any one of claims 1 to 9, characterized in that: The image bitstream includes a reference image information bitstream corresponding to reference image information, wherein the reference image information is used to represent the reference image; The reference image information includes at least one of the following: The sequence parameter set SPS, picture parameter set PPS, and picture header PH of the reference image.

11. The method according to claim 10, characterized in that The reference picture information bit stream precedes a first reference picture slice bit stream among the n reference picture slice bit streams, and the reference picture information bit stream is adjacent to the first reference picture slice bit stream.

12. The method according to claim 10, characterized in that The reference image information bit stream is a NAL unit structure.

13. A decoding method, characterized in that: include: Receive an image bitstream; wherein the image bitstream includes: n reference image slice bitstreams and multiple display image bitstreams, the reference image slice bitstream is a network abstraction layer NAL unit structure, the control field of the NAL unit of the reference image slice bitstream includes reference image identification information, and the reference image identification information is used to indicate that the load field in the NAL unit of the reference image bitstream carries the reference image slice; n is an integer greater than 0; Decoding the n reference image slice bit streams in the image bit stream based on the reference image identification information; Based on the decoded n reference image slices, a reference image is obtained; and, The plurality of display image bit streams are decoded to obtain a plurality of display images.

14. The method according to claim 13, characterized in that The multiple display images include a reference knowledge base RL image, and the reference image is a reference frame when the RL image is subjected to inter-frame prediction.

15. The method according to claim 13, characterized in that The multiple display images include multiple random access segments RAS, and the reference image is a reference frame of the multiple RAS.

16. The method according to claim 13, characterized in that The reference image is a cross-random access point reference CRR image.

17. The method according to claim 13, characterized in that The decoding of the n reference image slice bit streams in the image bit stream based on the reference image identification information includes: The control field of the bitstream is read to include the reference image identification information, and the reference image slice carried in the bitstream is obtained.

18. The method according to claim 13, characterized in that The reference image is a non-displayable reference image, and when n is greater than 1, the first reference image slice bitstream and the last reference image slice bitstream in the n reference image slice bitstreams include cross-random access point reference boundary network abstraction layer unit type CRR Delimiter NUT identification information, and other reference image slice bitstreams include cross-random access point reference network abstraction layer unit type CRR NUT identification information; wherein the CRR NUT identification information is used to indicate that the corresponding reference image slice is a non-displayable reference image slice, and the CRR Delimiter NUT identification information is used to indicate that the corresponding reference image slice is a non-displayable boundary reference image slice; the decoding of the n reference image slice bitstreams in the image bitstream based on the reference image identification information includes: If the CRR Delimiter NUT identification information is read, determining that the bit stream containing the CRR Delimiter NUT identification information is the first reference picture slice bit stream; If the CRR Delimiter NUT identification information is read again, it is determined that the bit stream containing the CRR Delimiter NUT identification information is the last reference picture slice bit stream, and it is determined that the reference picture bit stream is received completely.

19. The method according to claim 13, characterized in that The reference image is a non-displayable reference image, and when n is equal to 1, the reference image identification information is CRR NUT identification information.

20. The method according to claim 19, characterized in that The decoding of the n reference image slice bit streams in the image bit stream based on the reference image identification information includes: If the reference image identification information in the first received reference image bit stream carrying the reference image identification information is the CRR NUT identification information, it is determined that the reception of the reference image bit stream is completed.

21. The method according to any one of claims 13 to 20, characterized in that The decoding of the n reference image slice bit streams in the image bit stream based on the reference image identification information includes: Buffering each of the received reference picture slice bit streams; splicing the cached n reference image slice bit streams; Decode the concatenated n reference image slice bit streams.

22. The method according to any one of claims 13 to 21, characterized in that The n reference image slice bit streams in the image bit stream are interleaved and arranged with the multiple display image slice bit streams.

23. The method according to claim 13, characterized in that The reference image is a displayable reference image, and the reference image identification information includes cross-random access point reference display network abstraction layer unit type CRR DP NUT identification information, wherein each of the reference image slice bitstreams includes the CRRDP NUT identification information.

24. The method according to claim 23, characterized in that The decoding of the n reference image slice bit streams in the image bit stream based on the reference image identification information includes: The CRR DP NUT identification information is read, and a reference image slice bit stream containing the CRR DP NUT identification information is decoded to obtain a reference image slice.

25. The method according to claim 14, characterized in that The display image bit stream is a NAL unit structure, the multiple display image bit streams include a RL image bit stream, and the RL image bit stream is after the n reference image slice bit streams; The control field of the NAL unit corresponding to the RL image bit stream includes a reference image network abstraction layer unit type RL NUT identifier.

26. The method according to any one of claims 13 to 25, characterized in that: The image bitstream includes a reference image information bitstream corresponding to reference image information, wherein the reference image information is used to represent the reference image; The reference image information includes at least one of the following: The sequence parameter set SPS, picture parameter set PPS, and picture header PH of the reference image.

27. The method according to claim 26, characterized in that The reference picture information bit stream precedes a first reference picture slice bit stream among the n reference picture slice bit streams, and the reference picture information bit stream is adjacent to the first reference picture slice bit stream.

28. The method according to claim 26, characterized in that The reference image information bit stream is a NAL unit structure.

29. A coding device, characterized in that: include: one or more processors; a memory for storing one or more computer programs or instructions; When the one or more computer programs or instructions are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 12.

30. A decoding device, characterized in that: include: one or more processors; a memory for storing one or more computer programs or instructions; When the one or more computer programs or instructions are executed by the one or more processors, the one or more processors implement the method according to any one of claims 13 to 28.

Images