Encoding and decoding method and device, encoder, decoder and storage medium

Abstract

The invention provides an encoding and decoding method and device, an encoder, a decoder and a storage medium, and aims at an application scene of visual media content comprising one or more expression formats, isomorphic blocks of different expression formats are spliced into a heterogeneous mixed spliced graph, isomorphic blocks of the same expression format are spliced into an isomorphic spliced graph, and the isomorphic mixed spliced graph is obtained. And writing the obtained spliced graph and the spliced graph information into a code stream. The isomorphic spliced graph (such as at least one of a multi-view spliced graph, a point cloud spliced graph and a grid spliced graph) and the heterogeneous mixed spliced graph exist in the code stream at the same time, so that the coding and decoding method is suitable for application scenes of visual media contents in various expression formats, and the application range is expanded. And the code stream contains the first syntax element, so that the decoding efficiency of a decoding end on the spliced graph can be improved. The isomorphic blocks with different expression formats are spliced in one heterogeneous mixed spliced graph for encoding and decoding, so that the number of called decoders can be reduced, the implementation cost is reduced, and the usability is improved.

Description

A coding and decoding method, device, encoder, decoder and storage medium Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a coding and decoding method, device, encoder, decoder and storage medium. Background Art

[0002] In 3D applications such as virtual reality (VR), augmented reality (AR), and mixed reality (MR), visual media objects with different expression formats may appear in the same scene. For example, in the same 3D scene, the scene background and some characters and objects may be expressed as video, while other characters may be expressed as 3D point clouds or 3D meshes.

[0003] Using multi-view video coding, point cloud coding, and grid coding respectively during compression coding can better maintain the effective information of the original expression format than projecting all into multi-view video coding, improve the quality of the rendered viewing window during viewing, and improve the overall efficiency of bit rate-quality.

[0004] However, the current coding and decoding technology encodes and decodes multi-viewpoint video, point cloud coding and grid grid separately. The number of codecs that need to be called in the coding and decoding process is large, which makes the coding and decoding cost high.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a coding and decoding method, apparatus, encoder, decoder, and storage medium.

[0007] In a first aspect, the present application provides a decoding method, applied to a decoder, comprising:

[0008] Decoding the bitstream to obtain a spliced ​​graph and spliced ​​graph information, wherein the spliced ​​graph information includes a first syntax element, and determining whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph according to the first syntax element;

[0009] When it is determined according to the first syntax element that the mosaic is a heterogeneous mixed mosaic, splitting the mosaic according to the mosaic information of the mosaic to obtain at least two isomorphic blocks, wherein the at least two isomorphic blocks correspond to different visual media content expression formats;

[0010] When it is determined according to the first syntax element that the spliced ​​graph is a homogeneous spliced ​​graph, splitting the spliced ​​graph according to the spliced ​​graph information of the spliced ​​graph to obtain homogeneous blocks, wherein the homogeneous blocks correspond to the same visual media content expression format;

[0011] The homogeneous blocks are decoded and reconstructed to obtain visual media content in at least one expression format.

[0012] In a second aspect, the present application provides an encoding method, applied to an encoder, comprising:

[0013] Processing visual media content in at least one expression format to obtain at least one isomorphic block, wherein different isomorphic blocks correspond to different visual media content expression formats;

[0014] Splicing the at least one isomorphic block to obtain at least one spliced ​​graph and spliced ​​graph information, wherein the spliced ​​graph information includes a first syntax element, and determining, based on the first syntax element, whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph, the heterogeneous mixed spliced ​​graph includes at least two isomorphic blocks, and the homogeneous spliced ​​graph includes one isomorphic block;

[0015] The at least one splicing graph and the splicing graph information are encoded to obtain a code stream.

[0016] In a third aspect, the present application provides a decoding device, applied to a decoder, comprising:

[0017] a decoding unit, configured to decode a bitstream to obtain a splice graph and splice graph information, wherein the splice graph information includes a first syntax element, and determine whether the splice graph is a heterogeneous mixed splice graph or a homogeneous splice graph according to the first syntax element;

[0018] a first splitting unit configured to, when determining according to the first syntax element that the mosaic is a heterogeneous mixed mosaic, split the mosaic according to mosaic information of the mosaic to obtain at least two isomorphic blocks, wherein the at least two isomorphic blocks correspond to different visual media content expression formats;

[0019] a second splitting unit, configured to, when determining that the spliced ​​graph is a homogeneous spliced ​​graph according to the first syntax element, split the spliced ​​graph according to the spliced ​​graph information of the spliced ​​graph to obtain homogeneous blocks, wherein the homogeneous blocks correspond to the same visual media content expression format;

[0020] The processing unit is configured to decode and reconstruct the homogeneous blocks to obtain visual media content in at least one expression format.

[0021] In a fourth aspect, the present application provides an encoding device, applied to an encoder, comprising:

[0022] a processing unit, configured to process visual media content in at least one expression format to obtain at least one isomorphic block, wherein different isomorphic blocks correspond to different visual media content expression formats;

[0023] a splicing unit, configured to splice the at least one isomorphic block to obtain at least one spliced ​​graph and spliced ​​graph information, wherein the spliced ​​graph information includes a first syntax element, and determining, based on the first syntax element, whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph, the heterogeneous mixed spliced ​​graph including at least two isomorphic blocks, and the homogeneous spliced ​​graph including one isomorphic block;

[0024] The encoding unit is configured to encode the at least one splicing graph and the splicing graph information to obtain a code stream.

[0025] In a fifth aspect, a decoder is provided, comprising a first memory and a first processor; the first memory stores a computer program that can be run on the first processor to execute the method in the above-mentioned first aspect or its various implementations.

[0026] In a sixth aspect, an encoder is provided, comprising a second memory and a second processor; the second memory stores a computer program that can be run on the second processor to execute the method in the above-mentioned second aspect or its various implementations.

[0027] In a seventh aspect, a coding and decoding system is provided, comprising an encoder and a decoder. The encoder is configured to execute the method of the second aspect or its respective implementations, and the decoder is configured to execute the method of the first aspect or its respective implementations.

[0028] In an eighth aspect, a chip is provided for implementing the method described in any one of the first and second aspects above, or their respective implementations. Specifically, the chip includes a processor configured to load and execute a computer program from a memory, causing a device equipped with the chip to perform the method described in any one of the first and second aspects above, or their respective implementations.

[0029] In a ninth aspect, a computer-readable storage medium is provided for storing a computer program, which enables a computer to execute the method of any one of the first to second aspects or their respective implementations.

[0030] In a tenth aspect, a computer program product is provided, comprising computer program instructions, which enable a computer to execute the method of any one of the first to second aspects or their respective implementations.

[0031] In an eleventh aspect, a computer program is provided, which, when executed on a computer, enables the computer to execute the method in any one of the first to second aspects or their respective implementations.

[0032] In a twelfth aspect, a code stream is provided, which is generated based on the encoding method of the second aspect.

[0033] Based on the above technical solution, for application scenarios that include visual media content in one or more expression formats, homogeneous blocks of different expression formats are spliced ​​into a heterogeneous mixed splice graph, and homogeneous blocks of the same expression format are spliced ​​into a homogeneous splice graph, and the resulting splice graph and splice graph information are written into the bitstream. The bitstream contains both homogeneous splices (e.g., at least one of a multi-view splice graph, a point cloud splice graph, and a mesh splice graph) and heterogeneous mixed splices, making the encoding and decoding method applicable to application scenarios of visual media content in multiple expression formats, thereby expanding the scope of application of the encoding and decoding method. Moreover, the splice graph information includes a first syntax element for indicating the splice graph type, thereby improving the decoding efficiency of the splice graph at the decoding end. Furthermore, since homogeneous blocks of different expression formats are spliced ​​into a heterogeneous mixed splice graph for encoding and decoding, the number of two-dimensional video codecs such as HEVC, VVC, AVC, and AVS that need to be called can be reduced, reducing implementation costs and improving ease of use. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] FIG1 is a schematic block diagram of a video encoding and decoding system according to an embodiment of the present application;

[0035] FIG2A is a schematic block diagram of a video encoder according to an embodiment of the present application;

[0036] FIG2B is a schematic block diagram of a video decoder according to an embodiment of the present application;

[0037] FIG3A is a diagram showing the organization and expression framework of multi-view video data;

[0038] FIG3B is a schematic diagram of generating a stitched image from multi-view video data;

[0039] FIG3C is a diagram showing the organization and expression framework of point cloud data;

[0040] 3D to 3F are schematic diagrams of different types of point cloud data;

[0041] FIG4 is a schematic diagram of encoding of a multi-view video;

[0042] FIG5 is a schematic diagram of decoding of multi-view video;

[0043] FIG6 is a schematic diagram of a coding method flow chart provided in an embodiment of the present application;

[0044] FIG7 is a schematic diagram of a heterogeneous mixed splicing graph provided in one embodiment of the present application;

[0045] FIG8 is a schematic diagram of an isomorphic splicing graph provided by an embodiment of the present application;

[0046] FIG9 is a schematic diagram of a V3C bitstream structure provided in an embodiment of the present application;

[0047] FIG10 is a schematic flow chart of a decoding method provided in an embodiment of the present application;

[0048] FIG11 is a schematic block diagram of an encoding device provided in an embodiment of the present application;

[0049] FIG12 is a schematic block diagram of a decoding device provided in an embodiment of the present application;

[0050] FIG13 is a schematic block diagram of an encoder provided in an embodiment of the present application;

[0051] FIG14 is a schematic block diagram of a decoder provided in accordance with an embodiment of the present application;

[0052] FIG15 is a schematic diagram of the composition structure of a coding and decoding system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The present application can be applied to the field of image coding and decoding, the field of video coding and decoding, the field of hardware video coding and decoding, the field of dedicated circuit video coding and decoding, the field of real-time video coding and decoding, etc. For example, the solution of the present application can be combined with an audio and video coding standard (AVS), such as the H.264 / audio and video coding (AVC) standard, the H.265 / high efficiency video coding (HEVC) standard, and the H.266 / versatile video coding (VVC) standard. Alternatively, the solution of the present application can be combined with other proprietary or industry standards and operated, and the standards include ITU-TH.261, ISO / IEC MPEG-1 Visual, ITU-TH.262 or ISO / IEC MPEG-2 Visual, ITU-TH.263, ISO / IEC MPEG-4 Visual, ITU-TH.264 (also known as ISO / IEC MPEG-4 AVC), including scalable video coding (SVC) and multi-view video coding (MVC) extensions. It should be understood that the technology of this application is not limited to any specific coding standard or technology.

[0054] The high-degree-of-freedom immersive coding system can be roughly divided into the following links according to the task line: data acquisition, data organization and expression, data encoding and compression, data decoding and reconstruction, data synthesis and rendering, and finally presenting the target data to the user.

[0055] The encoding involved in the embodiment of the present application is mainly video encoding and decoding. For ease of understanding, the video encoding and decoding system involved in the embodiment of the present application is first introduced in conjunction with Figure 1.

[0056] FIG1 is a schematic block diagram of a video encoding and decoding system involved in an embodiment of the present application. It should be noted that FIG1 is only an example, and the video encoding and decoding system of the embodiment of the present application includes but is not limited to that shown in FIG1. ​​As shown in FIG1, the video encoding and decoding system 100 includes an encoding device 110 and a decoding device 120. The encoding device is used to encode (which can be understood as compressing) the video data to generate a code stream, and transmit the code stream to the decoding device. The decoding device decodes the code stream generated by the encoding device to obtain decoded video data.

[0057] The encoding device 110 of the embodiment of the present application can be understood as a device with a video encoding function, and the decoding device 120 can be understood as a device with a video decoding function, that is, the embodiment of the present application includes a wider range of devices for the encoding device 110 and the decoding device 120, such as smartphones, desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, televisions, cameras, display devices, digital media players, video game consoles, car computers, etc.

[0058] In some embodiments, the encoding device 110 may transmit the encoded video data (eg, a code stream) to the decoding device 120 via a channel 130. The channel 130 may include one or more media and / or devices capable of transmitting the encoded video data from the encoding device 110 to the decoding device 120.

[0059] In one example, the channel 130 includes one or more communication media that enable the encoding device 110 to transmit the encoded video data directly to the decoding device 120 in real time. In this example, the encoding device 110 can modulate the encoded video data according to a communication standard and transmit the modulated video data to the decoding device 120. The communication media includes wireless communication media, such as radio frequency spectrum. Optionally, the communication media may also include wired communication media, such as one or more physical transmission lines.

[0060] In another example, channel 130 includes a storage medium that can store the video data encoded by encoding device 110. The storage medium includes various locally accessible data storage media, such as optical disks, DVDs, and flash memories. In this example, decoding device 120 can retrieve the encoded video data from the storage medium.

[0061] In another example, the channel 130 may include a storage server that can store the video data encoded by the encoding device 110. In this example, the decoding device 120 can download the stored encoded video data from the storage server. Alternatively, the storage server can store the encoded video data and transmit the encoded video data to the decoding device 120, such as a web server (e.g., for a website), a file transfer protocol (FTP) server, etc.

[0062] In some embodiments, the encoding device 110 includes a video encoder 112 and an output interface 113. The output interface 113 may include a modulator / demodulator (modem) and / or a transmitter.

[0063] In some embodiments, the encoding device 110 may further include a video source 111 in addition to the video encoder 112 and the input interface 113 .

[0064] The video source 111 may include at least one of a video acquisition device (eg, a video camera), a video archive, a video input interface, and a computer graphics system, wherein the video input interface is used to receive video data from a video content provider, and the computer graphics system is used to generate video data.

[0065] The video encoder 112 encodes the video data from the video source 111 to generate a bitstream. The video data may include one or more pictures or a sequence of pictures. The bitstream contains the coding information of the picture or picture sequence in the form of a bitstream. The coding information may include the coded picture data and associated data. The associated data may include a sequence parameter set (SPS), a picture parameter set (PPS), and other syntax structures. The SPS may contain parameters that apply to one or more sequences. The PPS may contain parameters that apply to one or more pictures. The syntax structure refers to a set of zero or more syntax elements arranged in a specified order in the bitstream.

[0066] The video encoder 112 transmits the encoded video data directly to the decoding device 120 via the output interface 113. The encoded video data may also be stored in a storage medium or a storage server for subsequent reading by the decoding device 120.

[0067] In some embodiments, the decoding device 120 includes an input interface 121 and a video decoder 122. In some embodiments, the decoding device 120 may include a display device 123 in addition to the input interface 121 and the video decoder 122.

[0068] The input interface 121 includes a receiver and / or a modem and can receive the encoded video data via the channel 130 .

[0069] The video decoder 122 is configured to decode the encoded video data to obtain decoded video data, and transmit the decoded video data to the display device 123 .

[0070] The decoded video data is displayed on the display device 123. The display device 123 may be integrated with the decoding device 120 or external to the decoding device 120. The display device 123 may include various display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or other types of display devices.

[0071] In addition, Figure 1 is only an example, and the technical solution of the embodiment of the present application is not limited to Figure 1. For example, the technology of the present application can also be applied to unilateral video encoding or unilateral video decoding.

[0072] The following is an introduction to the video encoding framework involved in the embodiments of the present application.

[0073] FIG2A is a schematic block diagram of a video encoder according to an embodiment of the present application. It should be understood that the video encoder 200 can be used to perform lossy compression or lossless compression on an image. The lossless compression can be visually lossless or mathematically lossless.

[0074] The video encoder 200 can be applied to image data in a luminance and chrominance (YCbCr, YUV) format. For example, the YUV ratio can be 4:2:0, 4:2:2, or 4:4:4, where Y represents brightness (Luma), Cb (U) represents blue chrominance, Cr (V) represents red chrominance, and U and V represent chrominance (Chroma) for describing color and saturation. For example, in terms of color format, 4:2:0 means that every 4 pixels have 4 luminance components and 2 chrominance components (YYYYCbCr), 4:2:2 means that every 4 pixels have 4 luminance components and 4 chrominance components (YYYYCbCrCbCr), and 4:4:4 means full pixel display (YYYYCbCrCbCrCbCrCbCr).

[0075] For example, the video encoder 200 reads video data, and for each frame of the video data, divides the frame into a number of coding tree units (CTUs). In some examples, CTB may be referred to as a "tree block", "largest coding unit" (LCU) or "coding tree block" (CTB). Each CTU may be associated with a pixel block of equal size within the image. Each pixel may correspond to a luminance (luminance or luma) sample and two chrominance (chroma) samples. Therefore, each CTU may be associated with a luminance sample block and two chroma sample blocks. The size of a CTU is, for example, 128×128, 64×64, 32×32, etc. A CTU can be further divided into a number of coding units (CUs) for encoding. The CU may be a rectangular block or a square block. A CU can be further divided into prediction units (PUs) and transform units (TUs), allowing for separation of coding, prediction, and transform, and greater flexibility in processing. In one example, a CTU is divided into CUs using a quadtree, and a CU is divided into TUs and PUs using a quadtree.

[0076] The video encoder and video decoder can support various PU sizes. Assuming that the size of a particular CU is 2N×2N, the video encoder and video decoder can support PU sizes of 2N×2N or N×N for intra-frame prediction, and support symmetric PUs of 2N×2N, 2N×N, N×2N, N×N, or similar sizes for inter-frame prediction. The video encoder and video decoder can also support asymmetric PUs of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter-frame prediction.

[0077] In some embodiments, as shown in FIG2A , the video encoder 200 may include a prediction unit 210, a residual unit 220, a transform / quantization unit 230, an inverse transform / quantization unit 240, a reconstruction unit 250, a loop filter unit 260, a decoded image buffer 270, and an entropy coding unit 280. It should be noted that the video encoder 200 may include more, fewer, or different functional components.

[0078] Optionally, in this application, the current block may be referred to as the current coding unit (CU) or the current prediction unit (PU), etc. The prediction block may also be referred to as a predicted image block or an image prediction block, and the reconstructed image block may also be referred to as a reconstructed block or an image reconstructed image block.

[0079] In some embodiments, the prediction unit 210 includes an inter-frame prediction unit 211 and an intra-frame estimation unit 212. Because there is a strong correlation between adjacent pixels in a video frame, intra-frame prediction is used in video coding and decoding technologies to eliminate spatial redundancy between adjacent pixels. Because there is a strong similarity between adjacent frames in a video, inter-frame prediction is used in video coding and decoding technologies to eliminate temporal redundancy between adjacent frames, thereby improving coding efficiency.

[0080] The inter-frame prediction unit 211 is used for inter-frame prediction. Inter-frame prediction includes motion estimation and motion compensation. It can refer to image information from different frames. Inter-frame prediction uses motion information to find a reference block from a reference frame and generates a prediction block based on the reference block to eliminate temporal redundancy. The frames used for inter-frame prediction can be P-frames and / or B-frames. P-frames refer to forward-predicted frames, while B-frames refer to bidirectionally predicted frames. Inter-frame prediction uses motion information to find a reference block from a reference frame and generates a prediction block based on the reference block. Motion information includes the reference frame list in which the reference frame is located, the reference frame index, and the motion vector. The motion vector can be integer pixel or fractional pixel. If the motion vector is fractional pixel, interpolation filtering is required to generate the required fractional pixel block in the reference frame. Here, the integer pixel or fractional pixel block in the reference frame found based on the motion vector is called a reference block. Some technologies directly use the reference block as the prediction block, while others further process the reference block to generate a prediction block. Reprocessing the reference block to generate a prediction block can also be understood as using the reference block as the prediction block and then processing the prediction block to generate a new prediction block.

[0081] The intra-frame estimation unit 212 only refers to the information of the same frame image to predict the pixel information in the current code image block to eliminate spatial redundancy. The frame used for intra-frame prediction can be an I frame.

[0082] Intra-frame prediction has multiple prediction modes. For example, the H-series international digital video coding standard H.264 / AVC has eight angular prediction modes and one non-angular prediction mode. H.265 / HEVC expands this to 33 angular prediction modes and two non-angular prediction modes. HEVC uses planar, DC, and 33 angular modes for a total of 35 intra-frame prediction modes. VVC uses planar, DC, and 65 angular modes for a total of 67 intra-frame prediction modes.

[0083] It should be noted that with the increase of angle modes, intra-frame prediction will be more accurate and more in line with the needs of high-definition and ultra-high-definition digital video development.

[0084] The residual unit 220 may generate a residual block for the CU based on the pixel blocks of the CU and the prediction blocks of the PUs of the CU. For example, the residual unit 220 may generate the residual block for the CU such that each sample in the residual block has a value equal to the difference between the sample in the pixel blocks of the CU and the corresponding sample in the prediction blocks of the PUs of the CU.

[0085] The transform / quantization unit 230 may quantize the transform coefficients. The transform / quantization unit 230 may quantize the transform coefficients associated with the TUs of a CU based on a quantization parameter (QP) value associated with the CU. The video encoder 200 may adjust the degree of quantization applied to the transform coefficients associated with the CU by adjusting the QP value associated with the CU.

[0086] The inverse transform / quantization unit 240 may apply inverse quantization and inverse transform, respectively, to the quantized transform coefficients to reconstruct a residual block from the quantized transform coefficients.

[0087] Reconstruction unit 250 may add samples of the reconstructed residual block to corresponding samples of one or more prediction blocks generated by prediction unit 210 to generate a reconstructed image block associated with the TU. By reconstructing the sample blocks of each TU of a CU in this manner, video encoder 200 can reconstruct the pixel blocks of the CU.

[0088] The loop filter unit 260 is used to process the inverse transformed and inverse quantized pixels to compensate for distortion information and provide a better reference for subsequent coded pixels. For example, it can perform a deblocking filtering operation to reduce the blocking effect of pixel blocks associated with the CU.

[0089] In some embodiments, the loop filtering unit 260 includes a deblocking filtering unit and a sample adaptive offset / adaptive loop filtering (SAO / ALF) unit, wherein the deblocking filtering unit is used to remove blocking effects, and the SAO / ALF unit is used to remove ringing effects.

[0090] The decoded image buffer 270 may store the reconstructed pixel blocks. The inter prediction unit 211 may use a reference image containing the reconstructed pixel blocks to perform inter prediction on PUs of other images. In addition, the intra estimation unit 212 may use the reconstructed pixel blocks in the decoded image buffer 270 to perform intra prediction on other PUs in the same image as the CU.

[0091] The entropy coding unit 280 may receive the quantized transform coefficients from the transform / quantization unit 230. The entropy coding unit 280 may perform one or more entropy coding operations on the quantized transform coefficients to generate entropy-coded data.

[0092] FIG2B is a schematic block diagram of a video decoder according to an embodiment of the present application.

[0093] 2B , the video decoder 300 includes an entropy decoding unit 310, a prediction unit 320, an inverse quantization / transformation unit 330, a reconstruction unit 340, a loop filter unit 350, and a decoded picture buffer 360. It should be noted that the video decoder 300 may include more, fewer, or different functional components.

[0094] The video decoder 300 may receive a bitstream. The entropy decoding unit 310 may parse the bitstream to extract syntax elements from the bitstream. As part of parsing the bitstream, the entropy decoding unit 310 may parse the entropy-encoded syntax elements in the bitstream. The prediction unit 320, the inverse quantization / transform unit 330, the reconstruction unit 340, and the loop filter unit 350 may decode the video data based on the syntax elements extracted from the bitstream, thereby generating decoded video data.

[0095] In some embodiments, the prediction unit 320 includes an inter-frame prediction unit 321 and an intra-frame estimation unit 322 .

[0096] Intra estimation unit 322 may perform intra prediction to generate a prediction block for a PU. Intra estimation unit 322 may use an intra prediction mode to generate a prediction block for the PU based on pixel blocks of spatially neighboring PUs. Intra estimation unit 322 may also determine the intra prediction mode for the PU based on one or more syntax elements parsed from a codestream.

[0097] The inter-frame prediction unit 321 may construct a first reference picture list (List 0) and a second reference picture list (List 1) based on syntax elements parsed from the codestream. In addition, if a PU is encoded using inter-frame prediction, the entropy decoding unit 310 may parse the motion information of the PU. The inter-frame prediction unit 321 may determine one or more reference blocks for the PU based on the motion information of the PU. The inter-frame prediction unit 321 may generate a prediction block for the PU based on the one or more reference blocks of the PU.

[0098] The inverse quantization / transform unit 330 may inversely quantize (ie, dequantize) the transform coefficients associated with the TU. The inverse quantization / transform unit 330 may use the QP value associated with the CU of the TU to determine the degree of quantization.

[0099] After inverse quantizing the transform coefficients, the inverse quantization / transform unit 330 may apply one or more inverse transforms to the inverse quantized transform coefficients in order to generate a residual block associated with the TU.

[0100] The reconstruction unit 340 uses the residual block associated with the TU of the CU and the prediction block of the PU of the CU to reconstruct the pixel block of the CU. For example, the reconstruction unit 340 can add samples of the residual block to corresponding samples of the prediction block to reconstruct the pixel block of the CU to obtain a reconstructed image block.

[0101] The loop filtering unit 350 may perform a deblocking filtering operation to reduce blocking artifacts of pixel blocks associated with a CU.

[0102] The video decoder 300 may store the reconstructed image of the CU in the decoded image buffer 360. The video decoder 300 may use the reconstructed image in the decoded image buffer 360 as a reference image for subsequent prediction, or transmit the reconstructed image to a display device for presentation.

[0103] The basic process of video encoding and decoding is as follows: At the encoder end, a frame of image is divided into blocks. For the current block, the prediction unit 210 uses intra-frame prediction or inter-frame prediction to generate a prediction block for the current block. The residual unit 220 calculates a residual block based on the predicted block and the original block of the current block. This residual block is the difference between the predicted block and the original block of the current block. This residual block is also called residual information. This residual block undergoes transformation and quantization by the transform / quantization unit 230, removing information that is insensitive to the human eye and eliminating visual redundancy. Optionally, the residual block before transformation and quantization by the transform / quantization unit 230 is called a time-domain residual block, and the time-domain residual block after transformation and quantization by the transform / quantization unit 230 is called a frequency residual block or a frequency-domain residual block. The entropy coding unit 280 receives the quantized change coefficients output by the change quantization unit 230 and performs entropy coding on these quantized change coefficients to output a bitstream. For example, the entropy coding unit 280 can eliminate character redundancy based on the target context model and probability information of the binary bitstream.

[0104] At the decoding end, the entropy decoding unit 310 can parse the code stream to obtain the prediction information, quantization coefficient matrix, etc. of the current block. The prediction unit 320 uses intra-frame prediction or inter-frame prediction for the current block based on the prediction information to generate a prediction block for the current block. The inverse quantization / transformation unit 330 uses the quantization coefficient matrix obtained from the code stream to inverse quantize and inverse transform the quantization coefficient matrix to obtain a residual block. The reconstruction unit 340 adds the prediction block and the residual block to obtain a reconstructed block. The reconstructed blocks constitute a reconstructed image, and the loop filtering unit 350 performs loop filtering on the reconstructed image based on the image or block to obtain a decoded image. The encoding end also requires similar operations as the decoding end to obtain a decoded image. The decoded image can also be called a reconstructed image, and the reconstructed image can be used as a reference frame for inter-frame prediction for subsequent frames.

[0105] It should be noted that the block division information determined by the encoder, as well as mode information or parameter information such as prediction, transform, quantization, entropy coding, and loop filtering, etc., are carried in the bitstream when necessary. The decoder parses the bitstream and analyzes the existing information to determine the same block division information, prediction, transform, quantization, entropy coding, loop filtering, etc. mode information or parameter information as the encoder, thereby ensuring that the decoded image obtained by the encoder and the decoder are identical.

[0106] The above is the basic process of the video codec under the block-based hybrid coding framework. With the development of technology, some modules or steps of the framework or process may be optimized. This application is applicable to the basic process of the video codec under the block-based hybrid coding framework, but is not limited to the framework and process.

[0107] In some application scenarios, multiple heterogeneous contents appear simultaneously in the same 3D scene, such as multi-view video and point cloud. For this situation, current encoding and decoding methods include at least the following two:

[0108] Method 1: MPEG (Moving Picture Experts Group) immersive video (MIV) technology is used for encoding and decoding multi-view videos, and point cloud video compression (VPCC) technology is used for encoding and decoding point clouds.

[0109] The following is an introduction to MIV technology and VPCC technology.

[0110] MIV technology: To reduce the transmission pixel rate while preserving as much scene information as possible to ensure sufficient information for rendering the target view, MPEG-I uses a solution, as shown in Figure 3A. A limited number of viewpoints are selected as base viewpoints, and the scene's visible range is expressed as much as possible. The base viewpoints are transmitted as complete images, and redundant pixels between the remaining non-base viewpoints and the base viewpoint are removed, retaining only the valid information that is not repeated. This valid information is then extracted into sub-block images, which are reorganized with the base viewpoint images to form a larger rectangular image, called a spliced ​​image. Figures 3A and 3B illustrate the generation process of the spliced ​​image. The spliced ​​image is fed into the codec for compression and reconstruction, and auxiliary data related to the splicing information of the sub-block images is also fed into the encoder to form the bitstream.

[0111] The VPCC encoding method is to project the point cloud into a two-dimensional image or video, converting the three-dimensional information into a two-dimensional information code. Figure 3C is the encoding block diagram of VPCC. The code stream is roughly divided into four parts. The geometry code stream is the code stream generated by the geometric depth map encoding, which is used to represent the geometric information of the point cloud; the attribute code stream is the code stream generated by the texture map encoding, which is used to represent the attribute information of the point cloud; the occupancy code stream is the code stream generated by the occupancy map encoding, which is used to indicate the valid area in the depth map and texture map. All three types of videos are encoded and decoded using a video encoder, as shown in Figures 3D to 3F. The auxiliary information code stream is the code stream generated by the encoding of the auxiliary information of the sub-block image, that is, the part related to the patch data unit in the V3C standard, which indicates information such as the position and size of each sub-block image.

[0112] In the second method, both multi-view videos and point clouds are encoded and decoded using the frame packing technology in Visual Volumetric Video-based Coding (V3C).

[0113] The following is an introduction to frame packing technology.

[0114] Taking multi-view video as an example, as shown in FIG4 , the encoding end includes the following steps:

[0115] In step 1, when encoding the acquired multi-view video, multi-view video sub-blocks (patches) are generated after some pre-processing. Then, the multi-view video sub-blocks are organized to generate a multi-view video mosaic.

[0116] For example, as shown in Figure 4, a multi-view video is input into TIMV for packaging, and a multi-view video mosaic is output. TIMV is a reference software for MIV. The packaging in the embodiment of the present application can be understood as splicing.

[0117] The multi-view video mosaic map includes a multi-view video texture mosaic map and a multi-view video geometry mosaic map, that is, it only contains multi-view video sub-blocks.

[0118] Step 2: Input the multi-view video mosaic image into the frame packer, and output the multi-view video mixed mosaic image.

[0119] Among them, the multi-view video mixed mosaic map includes a multi-view video texture mixed mosaic map, a multi-view video geometry mixed mosaic map, and a multi-view video texture and geometry mixed mosaic map.

[0120] Specifically, as shown in Figure 4, the multi-view video mosaic is frame-packed to generate a multi-view video hybrid mosaic. Each multi-view video mosaic occupies a region of the multi-view video hybrid mosaic. Accordingly, a flag, pin_region_type_id_minus2, is transmitted for each region in the bitstream. This flag records whether the current region belongs to a multi-view video texture mosaic or a multi-view video geometry mosaic. This information is required by the decoder.

[0121] Step 3: Use a video encoder to encode the multi-view video mixed mosaic image to obtain a bit stream.

[0122] Exemplarily, as shown in FIG5 , the decoding end includes the following steps:

[0123] Step 1: During multi-view video decoding, the acquired code stream is input into a video decoder for decoding to obtain a reconstructed multi-view video mixed splicing image.

[0124] Step 2: input the reconstructed multi-view video mixed mosaic image into the frame depacketizer, and output the reconstructed multi-view video mosaic image.

[0125] Specifically, first, obtain the flag pin_region_type_id_minus2 from the bitstream. If it is determined that pin_region_type_id_minus2 is V3C_AVD, it means that the current region is a multi-view video texture mosaic. Then, the current region is split and output as a reconstructed multi-view video texture mosaic.

[0126] If it is determined that the pin_region_type_id_minus2 is V3C_GVD, it means that the current region is a multi-view video geometric mosaic map, and the current region is split and output as a reconstructed multi-view video geometric mosaic map.

[0127] Step 3: decode the reconstructed multi-view video mosaic to obtain the reconstructed multi-view video.

[0128] Specifically, the multi-view video texture mosaic map and the multi-view video geometric mosaic map are decoded to obtain a reconstructed multi-view video.

[0129] The above uses multi-view video as an example to analyze and introduce the frame packing technology. The frame packing encoding and decoding method for point cloud is basically the same as that for the above multi-view video. For example, TMC (a reference software for VPCC) is used to pack the point cloud to obtain a point cloud mosaic map, and the point cloud mosaic map is input into the frame packer for frame packing to obtain a point cloud mixed mosaic map. The point cloud mixed mosaic map is stitched to obtain a point cloud code stream. I will not go into details here.

[0130] The following introduces the syntax related to frame packing in the standard.

[0131] The V3C unit header syntax is shown in Table 1:

[0132] Table 1

[0133]

[0134] The semantics of the V3C unit header are shown in Table 2:

[0135] Table 2: V3C unit types

[0136]

[0137]

[0138] At present, if multiple visual media contents in different expression formats appear simultaneously in the same three-dimensional scene, the visual media contents in the multiple expression formats are encoded and decoded separately. For example, in the case where point cloud and multi-view video appear simultaneously in the same three-dimensional scene, the current packaging technology is to compress the point cloud to form a point cloud compressed code stream (i.e., a V3C code stream), compress the multi-view video information to obtain a multi-view video compressed code stream (i.e., another V3C code stream), and then the system layer multiplexes the compressed code stream to obtain a fused three-dimensional scene multiplexed code stream. During decoding, the point cloud compressed code stream and the multi-view video compressed code stream are decoded separately. It can be seen from this that when the existing technology encodes and decodes visual media contents in multiple expression formats, many codecs are used, and the encoding and decoding cost is high.

[0139] To address the aforementioned technical issues, the present invention employs an embodiment that concatenates homogeneous blocks of different expression formats into a heterogeneous hybrid mosaic graph, and concatenates homogeneous blocks of the same expression format into a homogeneous mosaic graph. The resulting heterogeneous hybrid mosaic graph and / or homogeneous mosaic graph are then encoded and written into a bitstream. The bitstream can simultaneously contain homogeneous mosaic graphs (e.g., at least one of a multi-view mosaic graph, a point cloud mosaic graph, and a mesh mosaic graph) and heterogeneous hybrid mosaic graphs, thereby expanding the application scenarios of the encoding and decoding method. Furthermore, the mosaic graph information includes a first syntax element for indicating the mosaic graph type, which can improve the decoding efficiency of the mosaic graph at the decoding end.

[0140] 6 , the video encoding method provided in the embodiment of the present application is introduced by taking the encoding end as an example.

[0141] FIG6 is a flow chart of an encoding method provided in an embodiment of the present application. As shown in FIG6 , the encoding method includes:

[0142] Step 601: Processing visual media content in at least one expression format to obtain at least one isomorphic block, wherein different isomorphic blocks correspond to different visual media content expression formats;

[0143] In three-dimensional application scenarios, such as virtual reality (VR), augmented reality (AR), and mixed reality (MR), visual media objects with different expression formats may appear in the same scene. For example, in the same three-dimensional scene, the scene background and some characters and objects may be expressed as video, while other characters may be expressed as three-dimensional point clouds or three-dimensional meshes.

[0144] In some embodiments, the visual media content includes visual media content in at least one expression format, such as multi-view video, point cloud, and mesh. A special case of multi-view video is single-view video, i.e., the multi-view video may include multiple viewpoint videos and / or single viewpoint videos.

[0145] Each homogeneous block corresponds to one expression format. Exemplarily, the expression format corresponding to at least one homogeneous block includes at least one of the following: multi-view video, point cloud, and mesh. At least two homogeneous blocks correspond to at least two different expression formats. For example, in an embodiment of the present application, the at least two homogeneous blocks include homogeneous blocks in at least two different expression formats, such as multi-view video, point cloud, and mesh.

[0146] It should be noted that in the embodiments of the present application, each homogeneous block may include at least one homogeneous block with the same expression format. For example, a homogeneous block in a point cloud format may include one or more point cloud blocks, a homogeneous region in a multi-view video format may include one or more multi-view video blocks, and a homogeneous block in a grid format may include one or more grid blocks.

[0147] In some embodiments, step 601 may be: processing visual media content in one expression format to obtain a homogeneous block. In some embodiments, step 601 may be: processing visual media content in at least two expression formats to obtain at least two homogeneous blocks, wherein different visual media content corresponds to different expression formats. Specifically, processing visual media content in a first expression format to obtain a homogeneous block in the first expression format; processing visual multimedia content in a second expression format to obtain a homogeneous block in the second expression format. The first expression format is one of multi-view video, point cloud, and mesh, and the second expression format is one of multi-view video, point cloud, and mesh, and the first expression format and the second expression format are different expression formats.

[0148] In other words, the visual media content includes visual media content in at least one expression format, such as multi-view video, point cloud, and mesh. When visual media content is in one expression format, the visual media content is processed to obtain homogeneous blocks in that one expression format. When visual media content is in multiple expression formats, the visual media content is processed to obtain homogeneous blocks in multiple expression formats.

[0149] It should be noted that a block may also be referred to as a strip (tile), that is, a point cloud block may also be referred to as a point cloud strip, a multi-view video block may also be referred to as a multi-view video strip, and a grid block may also be referred to as a grid strip. A block may be a mosaic with a specific shape, for example, a mosaic of a rectangular area with a specific length and / or height. For example, at least one sub-tile may be spliced ​​in an orderly manner, such as from large to small according to the area of ​​the sub-tile, or from large to small according to the length and / or height of the sub-tile, to obtain a block corresponding to the visual media content. Optionally, a block may be accurately mapped to an atlas tile.

[0150] In some embodiments, each sub-tile in a block may have a patch ID (patch ID) to distinguish different sub-tiles in the same block. For example, the same block may include sub-tile 1 (patch 1), sub-tile 2 (patch 2), and sub-tile 3 (patch 3).

[0151] Furthermore, each sub-block in a homogeneous block has the same expression format. For example, each sub-block in a homogeneous block may be a multi-view video sub-block, or a point cloud sub-block, or other sub-blocks in the same expression format. The expression format corresponding to each sub-block in a homogeneous block is the expression format corresponding to the homogeneous block.

[0152] In some embodiments, homogeneous tiles may have tile identifiers (tileIDs) to distinguish different tiles of the same expression format. For example, a point cloud tile may include point cloud tile 1 or point cloud tile 2. For example, multiple visual media contents may include point clouds and multi-view videos. The point clouds may be processed to obtain point cloud tiles, where point cloud tile 1 includes point cloud sub-tiles 1 to 3. The multi-view videos may be processed to obtain multi-view video tiles, where the multi-view video tiles include multi-view video sub-tiles 1 to 4.

[0153] When it is necessary to process visual media content in an expression format, an isomorphic block in an expression format is obtained. When it is necessary to process at least two visual media contents, at least two isomorphic blocks in expression formats are obtained. In order to improve compression efficiency, the embodiment of the present application processes the at least two visual media contents, such as packaging (also known as splicing), to obtain blocks corresponding to each visual media content in the at least two visual media contents. For example, sub-patches corresponding to at least two visual media contents can be spliced ​​to obtain blocks. It should be noted that the embodiment of the present application processes at least two visual media contents separately, and the method of obtaining blocks is not limited.

[0154] In one possible implementation, the visual media content includes visual media content in two expression formats, namely, multi-view video and point cloud. The processing of the visual media content in at least one expression format to obtain at least one isomorphic block includes: projecting and de-redundant processing the acquired multi-view video, connecting non-repeated pixels into video sub-blocks, and splicing the video sub-blocks into multi-view video blocks; and parallel projecting the acquired point cloud, forming point cloud sub-blocks from connected points in the projection plane, and splicing the point cloud sub-blocks into point cloud blocks.

[0155] Specifically, for multi-viewpoint video, taking MPEG-I as an example, a limited number of viewpoints are selected as basic viewpoints and the visible range of the scene is expressed as much as possible. The basic viewpoints are transmitted as complete images, and the redundant pixels between the remaining non-basic viewpoints and the basic viewpoints are removed, that is, only the valid information expressed in a non-repeated manner is retained, and then the valid information is extracted into sub-block images and reorganized with the basic viewpoint images to form a larger strip-shaped image. The strip-shaped image is called a multi-viewpoint video block.

[0156] In some embodiments, the visual media content is media content presented simultaneously in the same three-dimensional space. In some embodiments, the visual media content is media content presented at different times in the same three-dimensional space. In some embodiments, the visual media content can also be media content in different three-dimensional spaces. That is, in the embodiments of the present application, there is no specific limitation on the at least two visual media contents.

[0157] Step 602: Splicing the at least one isomorphic block to obtain at least one spliced ​​graph and spliced ​​graph information, wherein the spliced ​​graph information includes a first syntax element, and determining whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph based on the first syntax element, wherein the heterogeneous mixed spliced ​​graph includes at least two isomorphic blocks, and the homogeneous spliced ​​graph includes one isomorphic block;

[0158] In some embodiments, the splicing of the at least one isomorphic block to obtain at least one splicing graph and splicing graph information includes: performing heterogeneous splicing on isomorphic blocks of at least two expression formats to generate a heterogeneous mixed splicing graph and splicing graph information; performing isomorphic splicing on isomorphic blocks of the same expression format to generate an isomorphic splicing graph and splicing graph information.

[0159] Exemplarily, the at least one isomorphic block includes a isomorphic block in a first expression format and a isomorphic block in a second expression format. The method specifically includes: performing isomorphic splicing on isomorphic blocks in a first expression format to obtain a first isomorphic splicing graph and splicing graph information, performing isomorphic splicing on isomorphic blocks in a second expression format to obtain a second isomorphic splicing graph and splicing graph information; or, performing heterogeneous splicing on isomorphic blocks in the first expression format and isomorphic blocks in the second expression format to obtain a heterogeneous mixed splicing graph and splicing graph information; or, performing isomorphic splicing on isomorphic blocks in the first expression format to obtain a first isomorphic splicing graph and splicing graph information, performing heterogeneous splicing on isomorphic blocks in the first expression format and isomorphic blocks in the second expression format to obtain a heterogeneous mixed splicing graph and splicing graph information; or, performing isomorphic splicing on isomorphic blocks in the second expression format to obtain a second isomorphic splicing graph and splicing graph information, performing heterogeneous splicing on isomorphic blocks in the first expression format and isomorphic blocks in the second expression format to obtain a heterogeneous mixed splicing graph and splicing graph information.

[0160] That is, a homogeneous mosaic graph can include one or more homogeneous blocks in the same expression format, and a heterogeneous mixed mosaic graph includes at least two homogeneous blocks in at least two expression formats. In this embodiment of the present application, the first expression format is one of multi-view video, point cloud, and mesh, and the second expression format is one of multi-view video, point cloud, and mesh, and the first expression format and the second expression format are different expression formats. As shown in Figure 7, multi-view video block 1, multi-view video block 2, and point cloud block 1 are spliced ​​together to form a heterogeneous mixed mosaic graph.

[0161] Exemplarily, the first expression format is a multi-view video, and the second expression format is a point cloud. The step of splicing the at least one homogeneous block to obtain at least one spliced ​​graph and spliced ​​graph information includes: splicing a portion of the multi-view video blocks and a portion of the point cloud blocks into a heterogeneous mixed spliced ​​graph; splicing another portion of the multi-view video blocks into a multi-view spliced ​​graph; and splicing another portion of the point cloud blocks into a point cloud spliced ​​graph.

[0162] The spliced ​​graph information includes a first syntax element, and the first syntax element is used to indicate whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph.

[0163] In some embodiments, determining whether the splice is a heterogeneous mixed splice or a homogeneous splice based on the first syntax element includes: if the first syntax element is a first preset value, determining that the splice is a heterogeneous mixed splice including homogeneous blocks in a first expression format and a second expression format, wherein the first expression format and the second expression format are different expression formats; if the first syntax element is a second preset value, determining that the splice is a homogeneous splice including homogeneous blocks in the first expression format; and if the first syntax element is a third preset value, determining that the splice is a homogeneous splice including homogeneous blocks in the second expression format. That is, by setting different values ​​for the first syntax element, it is used to indicate the splice type. Furthermore, the first syntax element can also be set to other values ​​to indicate that the splice is a homogeneous splice including homogeneous blocks in other expression formats, or to indicate that the splice is a heterogeneous splice including homogeneous blocks in at least two other expression formats.

[0164] In some embodiments, the first syntax element includes at least two sub-syntax elements. Exemplarily, the first syntax element includes: a first sub-syntax element and a second sub-syntax element, and determining whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph based on the first sub-syntax element and the second sub-syntax element; determining whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph based on the first syntax element includes: if the first sub-syntax element is a fourth preset value, then determining that the spliced ​​graph includes homogeneous blocks in a first expression format; if the second sub-syntax element is a fifth preset value, then determining that the spliced ​​graph includes homogeneous blocks in a second expression format.

[0165] It can be understood that when the first sub-syntax element is the fourth preset value, it is determined that the splicing graph includes isomorphic blocks in the first expression format, that is, the splicing graph is determined to be an isomorphic splicing graph including isomorphic blocks in the first expression format; when the second sub-syntax element is the fifth preset value, it is determined that the splicing graph includes isomorphic blocks in the second expression format, that is, the splicing graph is determined to be an isomorphic splicing graph including isomorphic blocks in the second expression format; when the first sub-syntax element is the fourth preset value and the second sub-syntax element is the fifth preset value, it is determined that the splicing graph includes isomorphic blocks in the first expression format and isomorphic blocks in the second expression format, that is, the splicing graph is determined to be a heterogeneous mixed splicing graph including isomorphic blocks in the first expression format and the second expression format.

[0166] In some embodiments, the method further includes: if the first sub-syntax element is a sixth preset value, determining that the splicing graph does not include isomorphic blocks in the first expression format; and if the second sub-syntax element is a seventh preset value, determining that the splicing graph does not include isomorphic blocks in the second expression format.

[0167] Specifically, the first sub-syntax element is the fourth preset value and the second sub-syntax element is the fifth preset value, determining that the splicing graph is a heterogeneous mixed splicing graph including isomorphic blocks in the first expression format and isomorphic blocks in the second expression format; the first sub-syntax element is the fourth preset value and the second sub-syntax element is the seventh preset value, determining that the splicing graph is an isomorphic splicing graph including isomorphic blocks in the first expression format; the first sub-syntax element is the sixth preset value and the second sub-syntax element is the fifth preset value, determining that the splicing graph is an isomorphic splicing graph including isomorphic blocks in the second expression format.

[0168] In other words, the expression format of the homogeneous blocks in the spliced ​​graph can also be determined based on the values ​​of the two sub-syntax elements. Furthermore, when the spliced ​​graph includes more expression formats, the expression format of the homogeneous blocks in the spliced ​​graph can be indicated by multiple syntax elements. For example, when including three expression formats, three syntax elements are set; when including four expression formats, four syntax elements are set. Alternatively, a single syntax element can be set with multiple values ​​to represent multiple expression formats.

[0169] In some embodiments, the first syntax element is located in a parameter set of the code stream. Exemplarily, the parameter set of the code stream may be V3C_VPS, and the first syntax element may be ptl_profile_toolset_idc in V3C_VPS.

[0170] In some embodiments, the mosaic sequence parameter set corresponding to the mosaic includes the first syntax element. Exemplarily, the mosaic sequence parameter set corresponding to the mosaic includes the first sub-syntax element and the second sub-syntax element. Exemplarily, the first sub-syntax element is asps_vpcc_extension_present_flag in the mosaic sequence parameter set, and the second sub-syntax element is asps_miv_extension_present_flag.

[0171] In other words, the first syntax element can be located in the parameter set of the bitstream, allowing the decoder to parse the mosaic type of each mosaic earlier. The first syntax element can also be located in the mosaic sequence parameter set corresponding to each mosaic, allowing the decoder to obtain and determine the mosaic type when parsing each mosaic.

[0172] In some embodiments, the heterogeneous mixed mosaic graph of the embodiments of the present application includes at least one of the following: a single-attribute heterogeneous mixed mosaic graph and a multi-attribute heterogeneous mixed mosaic graph.

[0173] A single-attribute heterogeneous mixed mosaic image refers to a heterogeneous mixed mosaic image in which all homogeneous blocks have the same attribute information. For example, a single-attribute heterogeneous mixed mosaic image only includes homogeneous blocks with attribute information, such as multi-view video texture blocks and point cloud texture blocks. Another example is a single-attribute heterogeneous mixed mosaic image only includes homogeneous blocks with geometric information, such as multi-view video geometry blocks and point cloud geometry blocks.

[0174] A multi-attribute heterogeneous hybrid mosaic image refers to a heterogeneous hybrid mosaic image comprising at least two homogeneous blocks with different attribute information. For example, a multi-attribute heterogeneous hybrid mosaic image includes homogeneous blocks with both attribute information and homogeneous blocks with geometric information. As an example, a heterogeneous hybrid mosaic image can be obtained by stitching blocks with any one attribute or any two attributes of at least two of point clouds, multi-view videos, and meshes into a single image. This application does not limit this.

[0175] In some embodiments, a homogeneous block with a single attribute in a first representation format is spliced ​​with a block with a single attribute in a second representation format to generate a heterogeneous hybrid spliced ​​image. The first and second representation formats are each any one of a multi-view video, a point cloud, and a mesh, and the first and second representation formats are different, while the first and second representation formats have the same attribute information.

[0176] The single attribute homogeneous block of the multi-view video includes at least one of a multi-view video texture block and a multi-view video geometry block.

[0177] The single attribute homogeneous block of the point cloud includes at least one of a point cloud texture block, a point cloud geometry block, and a point cloud occupancy status block.

[0178] The single-attribute homogeneous block of the mesh includes at least one of a mesh texture block and a mesh geometry block.

[0179] For example, by splicing at least two of the multi-view video geometry blocks, point cloud geometry blocks, and mesh geometry blocks into a single image, a heterogeneous mixed mosaic image is obtained. This heterogeneous mixed mosaic image is called a single-attribute heterogeneous mixed mosaic image. For another example, by splicing at least two of the multi-view video texture blocks, point cloud texture blocks, and mesh texture blocks into a single image, a heterogeneous mixed mosaic image is obtained. This heterogeneous mixed mosaic image is called a single-attribute heterogeneous mixed mosaic image.

[0180] In some embodiments, a heterogeneous hybrid mosaic is obtained by splicing multi-attribute homogeneous blocks in a first expression format and multi-attribute homogeneous blocks in a second expression format. The first expression format and the second expression format are each any one of a multi-view video, a point cloud, and a mesh, and the first expression format and the second expression format are different, and the attribute information of the first expression format and the second expression format are not completely identical.

[0181] For example, a multi-view video texture block is spliced ​​with at least one of a point cloud geometry block and a mesh geometry block into a single image to obtain a heterogeneous mixed spliced ​​image. For another example, a multi-view video geometry block is spliced ​​with at least one of a point cloud texture block and a mesh texture block into a single image to obtain a heterogeneous mixed spliced ​​image. For another example, a point cloud texture block is spliced ​​with at least one of a multi-view video geometry block and a mesh geometry block into a single image to obtain a heterogeneous mixed spliced ​​image. For another example, a point cloud geometry block is spliced ​​with at least one of a multi-view video texture block and a mesh texture block into a single image to obtain a heterogeneous mixed spliced ​​image. For another example, a point cloud geometry block, a multi-view video texture block, and a multi-view video texture block are spliced ​​into a single image to obtain a heterogeneous mixed spliced ​​image. For another example, point cloud geometry blocks, point cloud texture blocks, and multi-view video texture blocks are stitched together into one image to obtain a heterogeneous hybrid stitched image. Here, the obtained heterogeneous hybrid stitched image is called a multi-attribute heterogeneous hybrid stitched image.

[0182] The following describes the stitching method in detail by taking the first expression format of multi-view video and the second expression format of point cloud as examples.

[0183] Assuming that the multi-view video block includes the multi-view video texture block and the multi-view video geometry block, and the point cloud block includes the point cloud texture block, the point cloud geometry block, and the point cloud occupancy block, the above-mentioned heterogeneous stitching methods can include but are not limited to the following two:

[0184] Method 1: Multi-view video texture blocks, multi-view video geometry blocks, point cloud texture blocks, point cloud geometry blocks, and point cloud occupancy blocks are all stitched together into a heterogeneous hybrid stitching image.

[0185] Method 2: According to the preset heterogeneous stitching method, the multi-view video texture block, the multi-view video geometry block, the point cloud texture block, the point cloud geometry block and the point cloud occupancy block are stitched together to obtain M heterogeneous mixed stitching images, where M is a positive integer greater than or equal to 1.

[0186] Among them, method 2 may include at least the following examples: Example 1, splicing the multi-view video texture block and the point cloud texture block to obtain a heterogeneous mixed texture splicing image, splicing the multi-view video geometry block and the point cloud geometry block to obtain a heterogeneous mixed geometry splicing image, and treating the point cloud occupancy block as a separate mixed splicing image. Example 2, splicing the multi-view video texture block and the point cloud texture block to obtain a heterogeneous mixed texture splicing image, splicing the multi-view video geometry block, the point cloud geometry block, and the point cloud occupancy block to obtain a heterogeneous mixed geometry and occupancy splicing image. Example 3, splicing the multi-view video texture block, the point cloud texture block, and the point cloud occupancy block to obtain a sub-heterogeneous mixed splicing image, and splicing the multi-view video geometry block and the point cloud geometry block to obtain another sub-heterogeneous mixed splicing image. Furthermore, after obtaining M heterogeneous mixed mosaic images, the M heterogeneous mixed mosaic images can be respectively subjected to video encoding to obtain video compression sub-streams.

[0187] In some embodiments, the isomorphic mosaic graph of the present application includes at least one of the following: a single-attribute isomorphic mosaic graph and a multi-attribute isomorphic mosaic graph. In some embodiments, the isomorphic mosaic graph is obtained by splicing isomorphic blocks of a first attribute in a first expression format. Alternatively, the isomorphic mosaic graph is obtained by splicing isomorphic blocks of a first attribute and isomorphic blocks of a second attribute in a first expression format.

[0188] A single-attribute homogeneous mosaic is a homogeneous mosaic in which all homogeneous blocks have the same expression format and attribute information. For example, a single-attribute homogeneous mosaic includes only homogeneous blocks with attribute information in a certain expression format, such as a single-attribute homogeneous mosaic that includes only multi-view video texture blocks or only point cloud texture blocks. Another example is a single-attribute homogeneous mosaic that includes only homogeneous blocks with geometric information, such as only multi-view video geometry blocks or only point cloud geometry blocks.

[0189] A multi-attribute isomorphic mosaic is an isomorphic mosaic that includes at least two isomorphic blocks with the same expression format but different attribute information. For example, a multi-attribute isomorphic mosaic includes both isomorphic blocks with attribute information and isomorphic blocks with geometric information. As an example, a multi-attribute isomorphic mosaic includes a multi-view video texture block and a multi-view video collection block. For another example, a multi-attribute isomorphic mosaic includes a point cloud geometry block and a point cloud texture block. As shown in Figure 8, a multi-attribute isomorphic mosaic includes point cloud texture block 1, point cloud geometry block 1, and point cloud geometry block 2.

[0190] In some embodiments, the splicing graph information may further include syntax elements, and the splicing graph is determined to be a single-attribute heterogeneous mixed splicing graph, a multi-attribute heterogeneous mixed splicing graph, a single-attribute homogeneous splicing graph, or a multi-attribute homogeneous splicing graph based on the syntax elements.

[0191] Step 603: Encode the at least one mosaic image and the mosaic image information to obtain a bitstream.

[0192] In some embodiments, the bitstream includes a video compression substream and a mosaic information substream. Encoding the at least one mosaic and mosaic information to obtain the bitstream includes: encoding the at least one mosaic to obtain a video compression substream; encoding the mosaic information of the at least one mosaic to obtain a mosaic information substream; and combining the video compression substream and the mosaic information substream into the bitstream. This allows support for heterogeneous source formats such as video, point cloud, and mesh in the same compressed bitstream, enabling the simultaneous presence of multi-view video mosaics, point cloud video mosaics, mesh mosaics, and heterogeneous mixed mosaics in the compressed bitstream. This reduces the number of required two-dimensional video encoders, such as HEVC, VVC, AVC, and AVS, lowering implementation costs and improving usability.

[0193] In some embodiments, when the mosaic is determined to be a heterogeneous mixed mosaic based on the first syntax element, the mosaic information further includes a second syntax element, and the expression format of the i-th block in the mosaic is determined based on the second syntax element. The encoder's inclusion of the second syntax element in the bitstream can help improve decoding accuracy at the decoder and enable the V3C standard to support visual media content in different expression formats, such as multi-view video and point cloud, within the same compressed bitstream.

[0194] In some embodiments, determining the expression format of the i-th block in the spliced ​​image according to the second syntax element includes: if the second syntax element is an eighth preset value, determining the expression format of the i-th block to be the first expression format; if the second syntax element is a ninth preset value, determining the expression format of the i-th block to be the second expression format.

[0195] Specifically, the expression format type corresponding to the i-th block in the spliced ​​image can be indicated by setting different values ​​for the second syntax element. Taking the first expression format as point cloud and the second expression format as multi-view video as an example, if the i-th block is a point cloud block, the second syntax element is set to the eighth preset value; if the i-th block is a multi-view video block, the second syntax element is set to the ninth preset value. The embodiment of the present application does not limit the specific values ​​of the eighth preset value and the ninth preset value. Optionally, the eighth preset value is 0. Optionally, the ninth preset value is 1.

[0196] In some embodiments, encoding the at least one mosaic image and mosaic image information to obtain a code stream includes: if the expression format of the i-th block is a first expression format, determining that the sub-image block in the i-th block is encoded using the encoding standard corresponding to the first expression format, and obtaining a code stream corresponding to the visual media content in the first expression format; if the expression format of the i-th block is a second expression format, determining that the sub-image block in the i-th block is encoded using the encoding standard corresponding to the second expression format, and obtaining a code stream corresponding to the visual media content in the second expression format.

[0197] In some embodiments, the second syntax element is located in a patch data unit header for the i-th patch of the patch. In some embodiments, the second syntax element may also be located in a patch data unit (patch_data_unit). Exemplarily, if the second syntax element (ath_toolset_type) is 1, it is determined that the current sub-patch is encoded using a multi-view video coding standard. If the second syntax element (ath_toolset_type) is 0, it is determined that the current sub-patch is encoded using a point cloud coding standard.

[0198] In some embodiments, encoding the at least one mosaic image and the mosaic image information to obtain a code stream includes: calling a video encoder to encode the at least one mosaic image to obtain a video compression sub-code stream.

[0199] In an embodiment of the present application, in order to reduce the number of encoders and reduce the encoding cost, at least two visual media contents are first processed separately (i.e., packaged) during encoding to obtain multiple isomorphic blocks. Then, at least two isomorphic blocks with different expression formats are spliced ​​into a heterogeneous mixed splicing graph, and at least one isomorphic block with exactly the same expression format is spliced ​​into an isomorphic splicing graph, and the heterogeneous mixed splicing graph and the isomorphic splicing graph are encoded to obtain a video compression sub-code stream. This makes the encoding and decoding method applicable to application scenarios of visual media content in various expression formats, expands the scope of application, and by splicing isomorphic blocks of different expression formats into a heterogeneous mixed splicing graph for encoding, during encoding, the video encoder can be called only once for encoding, thereby reducing the number of two-dimensional video encoders such as HEVC, VVC, AVC, AVS that need to be called, reducing the encoding cost and improving ease of use.

[0200] In the embodiment of the present application, the video encoder used to perform video encoding on the heterogeneous mixed mosaic and the homogeneous mosaic to obtain the video compression substream can be the video encoder shown in Figure 2A above. That is, in the embodiment of the present application, the heterogeneous mixed mosaic or the homogeneous mosaic is treated as a frame of image, firstly block-divided, then intra-frame or inter-frame prediction is used to obtain predicted values ​​for the coding blocks, the predicted values ​​of the coding blocks are subtracted from the original values ​​to obtain residual values, and the residual values ​​are transformed and quantized to obtain the video compression substream.

[0201] In this embodiment of the present application, while generating at least one mosaic, mosaic information corresponding to each mosaic is generated. The mosaic information is encoded to obtain a mosaic information substream. The mosaic information includes a first syntax element indicating the mosaic type and a second syntax element representing the format of each isomorphic block in the mosaic. This embodiment of the present application does not limit the encoding method for the mosaic information; for example, conventional data compression encoding methods such as constant-length encoding or variable-length encoding may be used for compression.

[0202] Finally, the video compression sub-stream and the mosaic information sub-stream are written into the same stream to obtain the final stream. In other words, the embodiment of the present application not only supports heterogeneous source formats such as video, point cloud, and mesh, but also homogeneous source formats in the same compressed stream.

[0203] In some embodiments, the method further includes: encoding the parameter set of the code stream to obtain a code stream parameter set sub-code stream. Specifically, the encoding end combines the video compression sub-code stream, the splicing graph information sub-code stream and the parameter set sub-code stream into a code stream. The parameter set sub-code stream of the code stream includes a third syntax element, and the code stream corresponding to the visual media content in at least one expression format is determined according to the third syntax element. That is to say, the encoding end sends the third syntax element to indicate whether the code stream contains visual media content in at least two expression formats at the same time. Exemplarily, when the third syntax element indicates that the code stream includes a code stream corresponding to visual media content in one expression format, it can be understood that the encoding end processes the visual media content in one expression format to obtain a homogeneous block, and splices the homogeneous blocks to obtain a homogeneous splicing graph. When the third syntax element indicates that the code stream includes code streams corresponding to visual media contents in at least two expression formats, it can be understood that the encoding end obtains at least two isomorphic blocks for the visual media contents in at least two expression formats, and splices the at least two isomorphic blocks to obtain a homogeneous splicing graph and / or a heterogeneous mixed splicing graph.

[0204] Exemplarily, when the third syntax element indicates that the codestream includes codestreams corresponding to visual media content in at least two expression formats, the method includes: performing isomorphic splicing on isomorphic blocks of the first expression format to obtain a first isomorphic splicing graph, and performing isomorphic splicing on isomorphic blocks of the second expression format to obtain a second isomorphic splicing graph; or, performing heterogeneous splicing on isomorphic blocks of the first expression format and isomorphic blocks of the second expression format to obtain a heterogeneous mixed splicing graph; or, performing isomorphic splicing on isomorphic blocks of the first expression format to obtain a first isomorphic splicing graph, and performing heterogeneous splicing on isomorphic blocks of the first expression format and isomorphic blocks of the second expression format to obtain a heterogeneous mixed splicing graph; or, performing isomorphic splicing on isomorphic blocks of the second expression format to obtain a second isomorphic splicing graph, and performing heterogeneous splicing on isomorphic blocks of the first expression format and isomorphic blocks of the second expression format to obtain a heterogeneous mixed splicing graph.

[0205] In some embodiments, the third syntax element is set to different values ​​to indicate that the bitstream includes the visual media content in at least one expression format. In other words, certain preset values ​​of the third syntax element can indicate that the bitstream includes the visual media content in one or more expression formats.

[0206] Exemplarily, determining the codestream corresponding to the visual media content in at least one expression format in the codestream based on the third syntax element includes: the third syntax element is a first value, determining that the codestream includes both the codestream corresponding to the visual media content in the first expression format and the codestream corresponding to the visual media content in the second expression format; the third syntax element is a second value, determining that the codestream includes the visual media content in the first expression format; and the third syntax element is a third value, determining that the codestream includes the visual media content in the second expression format.

[0207] Exemplarily, the parameter set of the code stream may be V3C_VPS, and the third syntax element may be ptl_profile_toolset_idc in V3C_VPS.

[0208] Exemplarily, taking the case where the first expression format is multi-view video and the second expression format is point cloud, when the third syntax element is set to the first value, the first value is used to indicate that the code stream contains both multi-view video code stream and point cloud code stream. As a specific example, when ptl_profile_toolset_idc=X, X is 128 / 129 / 130 / 132 / 133 / 134, it means that the current code stream contains both point cloud and multi-view code streams. For another example, when the third syntax element is set to the second value, the second value is used to indicate that the code stream only contains point cloud code stream. As a specific example, when ptl_profile_toolset_idc=X, X is 0 / 1, it means that the current code stream only contains point cloud code stream. For another example, when the third syntax element is set to the third value, the third value is used to indicate that the code stream only contains multi-view video code stream. As a specific example, when ptl_profile_toolset_idc=X, X is 64 / 65 / 66, which means that the current stream only contains multi-view video streams. It should be understood that the values ​​of the first value, the second value, and the third value are only examples, and the embodiments of the present application are not limited thereto.

[0209] In this example, the V3C_VPS in the existing V3C standard can be reused, and ptl_profile_toolset_idc is preconfigured with values ​​such as 0 / 1, 64 / 65 / 66, 128 / 129 / 130 / 132 / 133 / 134 to indicate the type of codestream contained in the current codestream. When encoding visual media content, this embodiment of the present application adds the value of a third syntax element to the parameter set to indicate the codestream corresponding to the visual media content in which the codestream is expressed. This helps improve decoding accuracy at the decoder end and enables the V3C standard to support visual media content in one or more expression formats, such as multi-view video, point cloud, and mesh, within the same compressed codestream.

[0210] Table 3 shows an example of available toolset profile components. Table 3 provides a list of toolset profile components defined for V3C and their corresponding identification syntax element values, such as ptl_profile_toolset_idc and ptc_one_v3c_frame_only_flag, which can be used only in this document. The syntax element ptl_profile_toolset_idc provides the main definition of the toolset profile. Additional syntax elements such as ptc_one_v3c_frame_only_flag can specify additional features or restrictions of the defined profile. ptc_one_v3c_frame_only_flag can be used to support only a single V3C frame. It should be noted that 2..63, 67..127, 131, 135..255 in ptl_profile_toolset_idc are reserved and temporarily undefined. Standards organizations may make further provisions in future standards. The profile types defined in Table 3 can include dynamic or static.

[0211] Table 3 Available toolset profile components

[0212]

[0213]

[0214] In some embodiments, the parameter set of the code stream further includes a first syntax element, wherein the first syntax element is used to indicate the type of each mosaic, specifically to indicate whether the mosaic is the heterogeneous mixed mosaic or the homogeneous mosaic; the first syntax element is written into the parameter set of the code stream. Exemplarily, a first syntax element (vps_toolset_type) is added to V3C_VPS, and vps_toolset_type is used to distinguish whether each mosaic and its corresponding V3C unit should belong to a point cloud mosaic / multi-view mosaic / heterogeneous mixed mosaic of point cloud + multi-view. At the same time, in order to be compatible with previous standards, the following new syntax and semantics, as well as constraints on old semantics, are implemented.

[0215] Exemplarily, the first syntax element is a first preset value, determining that the splicing graph is a heterogeneous mixed splicing graph including homogeneous blocks in a first expression format and a second expression format, wherein the first expression format and the second expression format are different expression formats; the first syntax element is a second preset value, determining that the splicing graph is a homogeneous splicing graph including homogeneous blocks in the first expression format; the first syntax element is a third preset value, determining that the splicing graph is a homogeneous splicing graph including homogeneous blocks in the second expression format.

[0216] Exemplarily, taking the case where the first expression format is multi-view video and the second expression format is point cloud as an example, the first syntax element is a first preset value, and the first preset value is used to indicate that the mosaic graph includes a heterogeneous mixed mosaic graph of point cloud blocks and multi-view video blocks. The first syntax element is a second preset value, and the second preset value is used to indicate that the mosaic graph includes a homogeneous mosaic graph of multi-view video blocks (which can be called a multi-view video mosaic graph). The first syntax element is a third preset value, and the third preset value is used to indicate that the mosaic graph includes a homogeneous mosaic graph of point cloud blocks (which can be called a point cloud mosaic graph).

[0217] For example, after obtaining the third syntax element ptl_profile_toolset_idc=128 / 129 / 130 / 132 / 133 / 134, the VPS needs to parse each mosaic to obtain the first syntax element (vps_toolset_type) and determine vps_toolset_type=X. X is 1, indicating that the mosaic contains only multi-view video blocks and should meet the requirements of the multi-view coding method; X is 2, indicating that the mosaic contains only point cloud blocks and should meet the requirements of the point cloud coding method; X is 3, indicating that the mosaic contains both multi-view video blocks and point cloud blocks and should meet the requirements of both multi-view and point cloud coding methods. It should be understood that the value of the first syntax element above is only for example and the embodiments of the present application are not limited thereto.

[0218] Table 4 shows the general V3C parameter set syntax. The V3C parameter set adds a new syntax element, vps_toolset_type. Specifically, vps_toolset_type[j] can be used to represent the type of the mosaic with index j. By adding the syntax element vps_toolset_type to the V3C parameter set, the decoder can obtain the vps_toolset_type from the V3C parameter set. Based on the vps_toolset_type, the decoder can quickly distinguish each mosaic and its corresponding V3C unit from the point cloud, multi-view, or point cloud + multi-view classification, thereby determining which coding method the mosaic should meet.

[0219] Table 4 General V3C parameter set syntax

[0220]

[0221]

[0222]

[0223] In some embodiments, the mosaic sequence parameter set corresponding to the mosaic includes the first syntax element. Exemplarily, the mosaic sequence parameter set corresponding to the mosaic includes the first sub-syntax element and the second sub-syntax element. The first sub-syntax element and the second sub-syntax element are used to indicate a mosaic type, wherein the mosaic is the heterogeneous mixed mosaic or the homogeneous mosaic.

[0224] Exemplarily, the first sub-syntax element is a fourth preset value and the second sub-syntax element is a fifth preset value, determining that the splicing graph is a heterogeneous mixed splicing graph including isomorphic blocks in the first expression format and isomorphic blocks in the second expression format; the first sub-syntax element is a fourth preset value and the second sub-syntax element is a seventh preset value, determining that the splicing graph is an isomorphic splicing graph including isomorphic blocks in the first expression format; the first sub-syntax element is a sixth preset value and the second sub-syntax element is a fifth preset value, determining that the splicing graph is an isomorphic splicing graph including isomorphic blocks in the second expression format.

[0225] Optionally, the first sub-syntax element is asps_vpcc_extension_present_flag in the splicing map sequence parameter set, and the second sub-syntax element is asps_miv_extension_present_flag.

[0226] For example, the NAL-ASPS of the V3C_AD code stream may include asps_miv_extension_present_flag and asps_vpcc_extension_present_flag.

[0227] In the embodiment of the present application, the first sub-syntax element and the second sub-syntax element are set to specific values ​​to indicate whether the spliced ​​graph is the heterogeneous mixed spliced ​​graph or the homogeneous spliced ​​graph.

[0228] Exemplarily, after obtaining the third syntax element ptl_profile_toolset_idc=128 / 129 / 130 / 132 / 133 / 134, asps_vpcc_extension_present_flag=X and asps_miv_extension_present_flag=Y are obtained from the mosaic information of the mosaic. X=0 and Y=1 indicate that the mosaic contains only multi-view video blocks and should meet the requirements of the multi-view coding method; X=1 and Y=0 indicate that the mosaic contains only point cloud blocks and should meet the requirements of the point cloud coding method; X=1 and Y=1 indicate that the mosaic contains both multi-view video blocks and point cloud blocks and should meet the requirements of both the multi-view and point cloud coding methods. It should be understood that the values ​​of the seventh and eighth preset values ​​above are merely examples and are not limited to these values ​​in the present application.

[0229] Table 5 shows the syntax of the general atlas sequence parameter set (RBSP syntax). The atlas sequence parameter set can be understood as the atlas information. The encoder uses the syntax elements asps_vpcc_extension_present_flag and asps_miv_extension_present_flag in the atlas sequence parameter set to indicate the atlas type. The encoder parses the bitstream to obtain these two syntax elements from the atlas parameter set. According to the values ​​of these two syntax elements, it is determined whether the atlas should belong to point cloud / multi-view / point cloud + multi-view, thereby determining which coding method the atlas should meet.

[0230] Table 5 General atlas sequence parameter set RBSP syntax

[0231]

[0232]

[0233] After implementing the above syntax, a mosaic image with multiple viewpoints and point clouds can be created on a single VPS. Furthermore, it is necessary to implement the situation where multiple homogeneous blocks exist within a mosaic image, each of which is a collection of multi-view sub-blocks or a collection of point cloud sub-blocks. Because existing technologies can only support the existence of one type of homogeneous block within a mosaic image, this embodiment of the present application adds a second syntax element, which determines the representation format of a homogeneous block within a mosaic image, such as multi-view video, point cloud, or mesh, based on this second syntax element.

[0234] In some embodiments, the mosaic block data unit header of the i-th block includes a second syntax element. When the mosaic is determined to be a heterogeneous mixed mosaic based on the first syntax element, the mosaic information further includes a second syntax element. The expression format of the i-th block in the mosaic is determined based on the second syntax element.

[0235] When encoding a heterogeneous hybrid mosaic, embodiments of the present application set a second syntax element to indicate the expression format of the i-th tile in the heterogeneous hybrid mosaic. This helps improve decoding accuracy at the decoder end and enables the V3C standard to support visual media content in different expression formats, such as multi-view video and point cloud, within the same compressed bitstream. Exemplarily, the second syntax element may be ath_toolset_type in the atlas tile data unit header (atlas_tile_header).

[0236] Exemplarily, the second syntax element is the eighth preset value, determining that the expression format of the i-th block is the first expression format; the second syntax element is the ninth preset value, determining that the expression format of the i-th block is the second expression format.

[0237] For example, when the mosaic is a heterogeneous mixed mosaic, the atlas_tile_header in the ACL NAL unit type code stream is parsed in the AD unit to obtain ath_toolset_type, and ath_toolset_type is determined to be X. If X is 0, it indicates that the current block is a point cloud block; if X is 1, it indicates that the current block is a multi-view video block.

[0238] Table 6 shows the Atlas tile header syntax. The encoder adds a new syntax element ath_toolset_type to the Atlas tile header syntax to indicate the tile type. The decoded bitstream can obtain ath_toolset_type from the Atlas tile header syntax to determine whether the current tile belongs to multi-view video decoding or point cloud decoding.

[0239] Table 6 Atlas tile header syntax

[0240]

[0241]

[0242] Optionally, the second syntax element may also be located in the patch data unit (patch_data_unit). Exemplarily, under the premise that the second syntax element (ath_toolset_type) is known to be 1, it is determined that the current sub-block is encoded using the multi-view video encoding method. Under the premise that the second syntax element (ath_toolset_type) is known to be 0, it is determined that the current sub-block is encoded using the point cloud encoding method. The patch data unit syntax (Patch data unit syntax) can be shown in Table 7:

[0243] Table 7 Patch data unit syntax

[0244]

[0245]

[0246] In some embodiments, the value of vps_toolset_type[j] is 1, indicating that the values ​​of the syntax elements of the toolset-level component of the atlas with index j shall conform to the values ​​specified in Table A-1-1 (i.e., Table 8) of ISO / IEC 23090-12;

[0247] The value of vps_toolset_type[j] is 2, indicating that the values ​​of the syntax elements of the toolset profile component of the atlas with index j shall conform to the values ​​specified in Table H-3 of ISO / IEC 23090-5, except for the values ​​of vps_extension_present_flag, vps_packing_information_present_flag, vps_miv_extension_present_flag, vuh_unit_type, and vps_atlas_count_minus1, which shall conform to the values ​​specified in Table A-1-1 of ISO / IEC 23090-12;

[0248] The value of vps_toolset_type[j] is 3, which indicates that the values ​​of the syntax elements of the toolset-level component of the atlas with index j shall conform to the values ​​specified in the extended ISO / IEC 23090-12 Table A-1-2 (i.e., Table 9-1 and Table 9-2); Table A-1-1 and Table A-1-2 respectively indicate the restrictions on the syntax of the toolbox-level component for multi-viewpoints and the restrictions on the syntax of the toolbox-level component for heterogeneous data under the integrated codestream.

[0249] A vps_toolset_type[j] value of 0 or any value from 4 to 7 indicates that the value is reserved for future use by ISO / IEC and shall not appear in bitstreams conforming to this version of this document. Decoders conforming to this version of this document shall ignore such reserved unit types. Allowed values ​​for syntax element values ​​of the MIV toolset profile.

[0250] The value of the syntax element ath_toolset_type, which represents the toolset profile component of the current tile, should conform to the values ​​specified in Table A-1 of the ISO / IEC 23090-12 extension. The value range of ath_toolset_type should be between 0 and 1.

[0251] Table 8 Allowable values ​​of syntax element values ​​for the MIV toolset profile

[0252]

[0253]

[0254]

[0255] Table 9-1 Allowable values ​​of syntax element values ​​for the MIV toolset profile (Extended)

[0256]

[0257]

[0258]

[0259] Table 9-2 Allowable values ​​of syntax element values ​​for the MIV toolset profile (Extended)

[0260]

[0261]

[0262]

[0263] Figure 9 is a schematic diagram of the V3C bitstream structure provided by an embodiment of the present application. The V3C parameter set (V3C_parameter_set()) of the V3C_VPS may include ptl_profile_toolset_idc. A ptl_profile_toolset_idc value of 128 / 129 / 130 / 132 / 133 / 134 indicates that the current bitstream contains both a point cloud bitstream (such as VPCC Basic or VPCC Extended) and a multi-view video bitstream (such as MIV Main, MIV Extended, or MIV Geometry Absent).

[0264] The V3C parameter set () (v3C_parameter_set()) of V3C_VPS can include a first syntax element (vps_toolset_type). When ptl_profile_toolset_idc is 128 / 129 / 130 / 132 / 133 / 134, vps_toolset_type is 1, indicating that the current spliced ​​image only has multi-view video blocks; 2, indicating that the current spliced ​​image only has point cloud blocks; and 3, indicating that the current spliced ​​image has both multi-view blocks and point cloud blocks.

[0265] Alternatively, the atlas_sequence_parameter_set_rbsp() in the NAL_ASPS of the atlas sub-bitstream() of V3C_AD can include asps_vpcc_extension_present_flag and asps_miv_extension_present_flag. When ptl_profile_toolset_idc is 128 / 129 / 130 / 132 / 133 / 134, asps_vpcc_extension_present_flag = X and asps_miv_extension_present_flag = Y. If X is 0 and Y is 1, the mosaic contains only multi-view video blocks; if X is 1 and Y is 0, the mosaic contains only point cloud blocks; if X is 1 and Y is 1, the mosaic contains both multi-view video blocks and point cloud blocks.

[0266] The ACL NAL unit type (ACL_NAL_unit_type) in the V3C_AD atlas_sub_bitstream() includes mosaic information. For example, the mosaic tile data unit (atlas_tile_data_unit()) may include ath_toolset_type. If ath_toolset_type is false (i.e., 0), it indicates that the current tile is a point cloud tile. If atdu_type_flag is true (i.e., 1), it indicates that the current tile is a multi-view video tile.

[0267] Furthermore, the patch information data (patch_information_data) includes a patch data unit (patch_data_unit). If ath_toolset_type is false (i.e., 0), it indicates that the current patch is implemented using a point cloud video encoding method. If ath_toolset_type is true (i.e., 1), it indicates that the current patch is implemented using a multi-view video encoding method.

[0268] By obtaining the first syntax element of each mosaic image, it is determined based on the value of the first syntax element whether the mosaic image includes both point cloud blocks and multi-view video blocks. When it is determined that the mosaic image contains both point cloud blocks and multi-view video blocks, it is necessary to obtain the ath_toolset_type of each block in the mosaic image to determine the block type.

[0269] The above describes the encoding method of the present application using the encoding end as an example. The following describes the video decoding method provided in the embodiment of the present application using the decoding end as an example.

[0270] FIG10 is a schematic flow chart of a decoding method provided in an embodiment of the present application. As shown in FIG10 , the decoding method in an embodiment of the present application includes:

[0271] Step 1001: Decode a bitstream to obtain a mosaic graph and mosaic graph information, wherein the mosaic graph information includes a first syntax element, and determine whether the mosaic graph is a heterogeneous mixed mosaic graph or a homogeneous mosaic graph according to the first syntax element;

[0272] In some embodiments, determining whether the splicing graph is a heterogeneous mixed splicing graph or a homogeneous splicing graph based on the first syntax element includes: if the first syntax element is a first preset value, determining the splicing graph as a heterogeneous mixed splicing graph including homogeneous blocks in a first expression format and a second expression format, wherein the first expression format and the second expression format are different expression formats; if the first syntax element is a second preset value, determining the splicing graph as an isomorphic splicing graph including homogeneous blocks in the first expression format; if the first syntax element is a third preset value, determining the splicing graph as an isomorphic splicing graph including homogeneous blocks in the second expression format.

[0273] In some embodiments, the first syntax element includes: a first sub-syntax element and a second sub-syntax element, and the splicing graph is determined to be a heterogeneous mixed splicing graph or a homogeneous splicing graph based on the first sub-syntax element and the second sub-syntax element; accordingly, the determination of whether the splicing graph is a heterogeneous mixed splicing graph or a homogeneous splicing graph based on the first syntax element includes: if the first sub-syntax element is a fourth preset value, then it is determined that the splicing graph includes homogeneous blocks in a first expression format; and / or, if the second sub-syntax element is a fifth preset value, then it is determined that the splicing graph includes homogeneous blocks in a second expression format; wherein the first expression format and the second expression format are different expression formats.

[0274] In some embodiments, determining whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph based on the first syntax element further includes: if the first sub-syntax element is a sixth preset value, determining that the spliced ​​graph does not include homogeneous blocks in the first expression format; if the second sub-syntax element is a seventh preset value, determining that the spliced ​​graph does not include homogeneous blocks in the second expression format.

[0275] Specifically, the determining of whether the splicing graph is a heterogeneous mixed splicing graph or a homogeneous splicing graph based on the first syntax element includes: the first sub-syntax element is a fourth preset value and the second sub-syntax element is a fifth preset value, determining that the splicing graph is a heterogeneous mixed splicing graph including homogeneous blocks in the first expression format and homogeneous blocks in the second expression format, the first sub-syntax element is a fourth preset value and the second sub-syntax element is a seventh preset value, determining that the splicing graph is an isomorphic splicing graph including homogeneous blocks in the first expression format; the first sub-syntax element is a sixth preset value and the second sub-syntax element is a fifth preset value, determining that the splicing graph is an isomorphic splicing graph including homogeneous blocks in the second expression format.

[0276] In some embodiments, the first syntax element is located in a parameter set sub-codestream of the codestream.

[0277] In some other embodiments, the mosaic sequence parameter set corresponding to the mosaic includes the first syntax element.

[0278] In some embodiments, the at least one expression format includes at least one of multi-view video, point cloud, and mesh. Specifically, the first expression format is one of multi-view video, point cloud, and mesh, and the second expression format is one of multi-view video, point cloud, and mesh, and the first expression format and the second expression format are different.

[0279] In some embodiments, the bitstream further includes a parameter set sub-bitstream of the bitstream, the parameter set sub-bitstream of the bitstream including a third syntax element, and the bitstream corresponding to the visual media content including at least one expression format in the bitstream is determined based on the third syntax element. The method further includes decoding the parameter set sub-bitstream of the bitstream to obtain a parameter set of the bitstream, and obtaining the third syntax element from the parameter set of the bitstream.

[0280] In some embodiments, the method further includes: the third syntax element is a first value, determining that the code stream includes both the code stream corresponding to the visual media content in the first expression format and the code stream corresponding to the visual media content in the second expression format; the third syntax element is a second value, determining that the code stream includes the code stream corresponding to the visual media content in the first expression format; the third syntax element is a third value, determining that the code stream includes the code stream corresponding to the visual media content in the second expression format.

[0281] In some embodiments, decoding the bitstream to obtain at least one spliced ​​graph includes: determining, based on the third syntax element, that the bitstream includes bitstreams corresponding to visual media content in at least two expression formats, and decoding the bitstream to obtain a heterogeneous mixed spliced ​​graph.

[0282] In some embodiments, decoding the bitstream to obtain at least one splicing graph includes: determining, based on the third syntax element, that the bitstream includes bitstreams corresponding to visual media content in at least two expression formats, and decoding the bitstream to obtain isomorphic splicing graphs in the at least two expression formats. That is, when the bitstream includes bitstreams corresponding to visual media content in at least two expression formats, each expression format corresponds to a isomorphic splicing graph.

[0283] In some embodiments, decoding the bitstream to obtain at least one splice graph includes: determining, based on the third syntax element, that the bitstream includes bitstreams corresponding to visual media content in at least two expression formats, and decoding the bitstream to obtain a heterogeneous mixed splice graph and a homogeneous splice graph in the at least two expression formats. That is, when the bitstream includes bitstreams corresponding to visual media content in at least two expression formats, a portion of homogeneous blocks in the expression formats are used to construct a heterogeneous mixed splice graph, while another portion of homogeneous blocks in the expression formats are used to construct a homogeneous splice graph.

[0284] In some embodiments, the heterogeneous mixed mosaic graph includes at least one of the following: a single-attribute heterogeneous mixed mosaic graph and a multi-attribute heterogeneous mixed mosaic graph; the homogeneous mosaic graph includes at least one of the following: a single-attribute homogeneous mosaic graph and a multi-attribute homogeneous mosaic graph.

[0285] In some embodiments, the bitstream includes a video compression substream and a mosaic graph information substream, and decoding the bitstream to obtain at least one mosaic graph and mosaic graph information includes: decoding the video compression substream to obtain the at least one mosaic graph; decoding the mosaic graph information substream to obtain mosaic graph information of the at least one mosaic graph. Exemplarily, based on the third syntax element, determining that the bitstream includes bitstreams corresponding to visual media content in at least two expression formats, decoding the video compression substream, and decoding the bitstream to obtain a heterogeneous mixed mosaic graph. Alternatively, based on the third syntax element, determining that the bitstream includes bitstreams corresponding to visual media content in at least two expression formats, decoding the video compression substream, and decoding the bitstream to obtain a heterogeneous mixed mosaic graph and a homogeneous mosaic graph; or alternatively, based on the third syntax element, determining that the bitstream includes bitstreams corresponding to visual media content in at least two expression formats, decoding the video compression substream, and obtaining homogeneous mosaic graphs in at least two expression formats.

[0286] Step 1002: When it is determined according to the first syntax element that the mosaic is a heterogeneous mixed mosaic, the mosaic is split according to the mosaic information of the mosaic to obtain at least two isomorphic blocks, wherein the at least two isomorphic blocks correspond to different visual media content expression formats;

[0287] Step 1003: when it is determined according to the first syntax element that the mosaic is a homogeneous mosaic, the mosaic is split according to the mosaic information of the mosaic to obtain homogeneous blocks, wherein the homogeneous blocks correspond to the same visual media content expression format;

[0288] Step 1004: Decode and reconstruct the homogeneous blocks to obtain visual media content in at least one expression format.

[0289] In some embodiments, the method further includes: when determining that the spliced ​​graph is a heterogeneous mixed spliced ​​graph according to the first syntax element, the spliced ​​graph information further includes a second syntax element, and determining the expression format of the i-th block in the spliced ​​graph according to the second syntax element.

[0290] Exemplarily, determining the expression format of the i-th block in the spliced ​​graph according to the second syntax element includes: when the second syntax element is an eighth preset value, determining the expression format of the i-th block to be the first expression format; when the second syntax element is a ninth preset value, determining the expression format of the i-th block to be the second expression format.

[0291] In some embodiments, the second syntax element is located in a mosaic tile data unit header of the i-th tile of the mosaic tile.

[0292] In some embodiments, the decoding and reconstruction of the homogeneous blocks to obtain visual media content in at least one expression format includes: if the expression format of the i-th block is a first expression format, determining that the sub-blocks in the i-th block are decoded and reconstructed using a decoding method corresponding to the first expression format to obtain visual media content in the first expression format; if the expression format of the i-th block is a second expression format, determining that the sub-blocks in the i-th block are decoded and reconstructed using a decoding method corresponding to the second expression format to obtain visual media content in the second expression format.

[0293] Exemplarily, the decoded bitstream obtains a multi-view video mosaic, a point cloud mosaic, and a heterogeneous mixed mosaic. Based on the mosaic information of the heterogeneous mixed mosaic, the heterogeneous mixed mosaic is split, and reconstructed multi-view video blocks and point cloud blocks are output; based on the mosaic information corresponding to the multi-view video mosaic, the multi-view video mosaic is split, and reconstructed multi-view video blocks are output; based on the mosaic information corresponding to the point cloud mosaic, the point cloud mosaic is split, and reconstructed point cloud blocks are output; all obtained multi-view video blocks are decoded through multi-view video to generate a reconstructed multi-view video; and all obtained point cloud blocks are decoded through point cloud to generate a reconstructed point cloud.

[0294] Using the above technical solution, for application scenarios that include visual media content in one or more expression formats, homogeneous blocks of different expression formats are spliced ​​into a heterogeneous mixed splice graph, and homogeneous blocks of the same expression format are spliced ​​into a homogeneous splice graph, and the resulting splice graph and splice graph information are written into the bitstream. The bitstream contains both homogeneous splices (e.g., at least one of a multi-view splice graph, a point cloud splice graph, and a mesh splice graph) and heterogeneous mixed splices, making the encoding and decoding method applicable to application scenarios of visual media content in multiple expression formats, thereby expanding the scope of application of the encoding and decoding method. In addition, the splice graph information includes a first syntax element for indicating the splice graph type, thereby improving the decoding efficiency of the splice graph at the decoding end. Furthermore, since homogeneous blocks of different expression formats are spliced ​​into a heterogeneous mixed splice graph for encoding and decoding, the number of two-dimensional video codecs such as HEVC, VVC, AVC, and AVS that need to be called can be reduced, reducing implementation costs and improving ease of use.

[0295] The present application also provides an encoding device. FIG11 is a schematic block diagram of an encoding device provided in an embodiment of the present application. The encoding device 110 is applied to an encoder. As shown in FIG11 , the encoding device 110 includes:

[0296] The processing unit 1101 is configured to process visual media content in at least one expression format to obtain at least one isomorphic block, wherein different isomorphic blocks correspond to different visual media content expression formats;

[0297] a splicing unit 1102 configured to splice the at least one isomorphic block to obtain at least one spliced ​​graph and spliced ​​graph information, wherein the spliced ​​graph information includes a first syntax element, and determining, based on the first syntax element, whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph, the heterogeneous mixed spliced ​​graph including at least two isomorphic blocks, and the homogeneous spliced ​​graph including one isomorphic block;

[0298] The encoding unit 1103 is configured to encode the at least one mosaic image and the mosaic image information to obtain a code stream.

[0299] In some embodiments, determining whether the splicing graph is a heterogeneous mixed splicing graph or a homogeneous splicing graph based on the first syntax element includes: if the first syntax element is a first preset value, determining the splicing graph as a heterogeneous mixed splicing graph including homogeneous blocks in a first expression format and a second expression format, wherein the first expression format and the second expression format are different expression formats; if the first syntax element is a second preset value, determining the splicing graph as an isomorphic splicing graph including homogeneous blocks in the first expression format; if the first syntax element is a third preset value, determining the splicing graph as an isomorphic splicing graph including homogeneous blocks in the second expression format.

[0300] In some embodiments, the first syntax element includes: a first sub-syntax element and a second sub-syntax element, and determining whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph is based on the first sub-syntax element and the second sub-syntax element;

[0301] The determining, based on the first syntax element, whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph includes: if the first sub-syntax element is a fourth preset value, determining that the spliced ​​graph includes homogeneous blocks in a first expression format; and / or, if the second sub-syntax element is a fifth preset value, determining that the spliced ​​graph includes homogeneous blocks in a second expression format; wherein the first expression format and the second expression format are different expression formats.

[0302] In some embodiments, determining whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph based on the first syntax element further includes: if the first sub-syntax element is a sixth preset value, determining that the spliced ​​graph does not include homogeneous blocks in the first expression format; if the second sub-syntax element is a seventh preset value, determining that the spliced ​​graph does not include homogeneous blocks in the second expression format.

[0303] In some embodiments, the first syntax element is located in a parameter set sub-codestream of the codestream.

[0304] In some embodiments, the mosaic sequence parameter set corresponding to the mosaic includes the first syntax element.

[0305] In some embodiments, when the spliced ​​graph is determined to be a heterogeneous mixed spliced ​​graph according to the first syntax element, the spliced ​​graph information further includes a second syntax element, and an expression format of the i-th block in the spliced ​​graph is determined according to the second syntax element.

[0306] In some embodiments, determining the expression format of the i-th block in the spliced ​​image according to the second syntax element includes: the second syntax element is an eighth preset value, determining the expression format of the i-th block to be the first expression format; the second syntax element is a ninth preset value, determining the expression format of the i-th block to be the second expression format.

[0307] In some embodiments, the second syntax element is located in a mosaic tile data unit header of the i-th tile of the mosaic tile.

[0308] In some embodiments, the encoding unit 1103 is used to, if the expression format of the i-th block is a first expression format, determine that the sub-block in the i-th block is encoded using the encoding method corresponding to the first expression format, so as to obtain a code stream corresponding to the visual media content in the first expression format; if the expression format of the i-th block is a second expression format, determine that the sub-block in the i-th block is encoded using the encoding method corresponding to the second expression format, so as to obtain a code stream corresponding to the visual media content in the second expression format.

[0309] In some embodiments, the parameter set sub-codestream of the codestream includes a third syntax element, and the codestream corresponding to the visual media content including at least one expression format in the codestream is determined according to the third syntax element.

[0310] In some embodiments, determining the codestream corresponding to the visual media content in at least one expression format in the codestream based on the third syntax element includes: the third syntax element is a first value, determining that the codestream includes both the codestream corresponding to the visual media content in the first expression format and the codestream corresponding to the visual media content in the second expression format; the third syntax element is a second value, determining that the codestream includes the visual media content in the first expression format; and the third syntax element is a third value, determining that the codestream includes the visual media content in the second expression format.

[0311] In some embodiments, when the at least one mosaic graph includes a heterogeneous mixed mosaic graph, the third syntax element is used to indicate a codestream corresponding to visual media content in at least two expression formats included in the codestream.

[0312] In some embodiments, the encoding unit 1103 is configured to encode the at least one mosaic to obtain a video compression substream; encode the mosaic information of the at least one mosaic to obtain a mosaic information substream; and combine the video compression substream and the mosaic information substream into the codestream.

[0313] In some embodiments, the at least one expression format includes at least one of multi-view video, point cloud, and mesh.

[0314] In some embodiments, the heterogeneous mixed mosaic graph includes at least one of the following: a single-attribute heterogeneous mixed mosaic graph and a multi-attribute heterogeneous mixed mosaic graph; the homogeneous mosaic graph includes at least one of the following: a single-attribute homogeneous mosaic graph and a multi-attribute homogeneous mosaic graph.

[0315] The present application also provides a decoding device. FIG12 is a schematic block diagram of a decoding device provided by an embodiment of the present application. The decoding device 120 is applied to a decoder. As shown in FIG12 , the decoding device 120 includes:

[0316] The decoding unit 1201 is configured to decode a bitstream to obtain a mosaic graph and mosaic graph information, wherein the mosaic graph information includes a first syntax element, and determine whether the mosaic graph is a heterogeneous mixed mosaic graph or a homogeneous mosaic graph according to the first syntax element;

[0317] a splitting unit 1202 configured to, when determining that the mosaic is a heterogeneous mixed mosaic based on the first syntax element, split the mosaic based on the mosaic information of the mosaic to obtain at least two isomorphic blocks, wherein the at least two isomorphic blocks correspond to different visual media content expression formats;

[0318] The splitting unit 1202 is configured to, when determining that the mosaic is a homogeneous mosaic according to the first syntax element, split the mosaic according to the mosaic information of the mosaic to obtain homogeneous blocks, wherein the homogeneous blocks correspond to the same visual media content expression format;

[0319] The processing unit 1203 is configured to decode and reconstruct the homogeneous blocks to obtain visual media content in at least one expression format.

[0320] In some embodiments, determining whether the splicing graph is a heterogeneous mixed splicing graph or a homogeneous splicing graph based on the first syntax element includes: if the first syntax element is a first preset value, determining the splicing graph as a heterogeneous mixed splicing graph including homogeneous blocks in a first expression format and a second expression format, wherein the first expression format and the second expression format are different expression formats; if the first syntax element is a second preset value, determining the splicing graph as an isomorphic splicing graph including homogeneous blocks in the first expression format; if the first syntax element is a third preset value, determining the splicing graph as an isomorphic splicing graph including homogeneous blocks in the second expression format.

[0321] In some embodiments, the first syntax element includes: a first sub-syntax element and a second sub-syntax element, and determining whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph is based on the first sub-syntax element and the second sub-syntax element;

[0322] The determining, based on the first syntax element, whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph includes: if the first sub-syntax element is a fourth preset value, determining that the spliced ​​graph includes homogeneous blocks in a first expression format; and / or, if the second sub-syntax element is a fifth preset value, determining that the spliced ​​graph includes homogeneous blocks in a second expression format; wherein the first expression format and the second expression format are different expression formats.

[0323] In some embodiments, determining whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph based on the first syntax element further includes: if the first sub-syntax element is a sixth preset value, determining that the spliced ​​graph does not include homogeneous blocks in the first expression format; if the second sub-syntax element is a seventh preset value, determining that the spliced ​​graph does not include homogeneous blocks in the second expression format.

[0324] In some embodiments, the first syntax element is located in a parameter set sub-codestream of the codestream.

[0325] In some embodiments, the mosaic sequence parameter set corresponding to the mosaic includes the first syntax element.

[0326] In some embodiments, when the spliced ​​graph is determined to be a heterogeneous mixed spliced ​​graph according to the first syntax element, the spliced ​​graph information further includes a second syntax element, and an expression format of the i-th block in the spliced ​​graph is determined according to the second syntax element.

[0327] In some embodiments, determining the expression format of the i-th block in the spliced ​​image according to the second syntax element includes: the second syntax element is an eighth preset value, determining the expression format of the i-th block to be the first expression format; the second syntax element is a ninth preset value, determining the expression format of the i-th block to be the second expression format.

[0328] In some embodiments, the second syntax element is located in a mosaic tile data unit header of the i-th tile of the mosaic tile.

[0329] In some embodiments, the processing unit 1203 is used to, if the expression format of the i-th block is a first expression format, determine that the sub-image block in the i-th block is decoded and reconstructed using a decoding method corresponding to the first expression format to obtain visual media content in the first expression format; if the expression format of the i-th block is a second expression format, determine that the sub-image block in the i-th block is decoded and reconstructed using a decoding method corresponding to the second expression format to obtain visual media content in the second expression format.

[0330] In some embodiments, the parameter set sub-codestream of the codestream includes a third syntax element, and the codestream corresponding to the visual media content including at least one expression format in the codestream is determined according to the third syntax element.

[0331] In some embodiments, determining the codestream corresponding to the visual media content in at least one expression format in the codestream based on the third syntax element includes: the third syntax element is a first value, determining that the codestream includes both the codestream corresponding to the visual media content in the first expression format and the codestream corresponding to the visual media content in the second expression format; the third syntax element is a second value, determining that the codestream includes the visual media content in the first expression format; and the third syntax element is a third value, determining that the codestream includes the visual media content in the second expression format.

[0332] In some embodiments, the decoding unit 1201 is configured to determine, based on the third syntax element, that the bitstream includes bitstreams corresponding to visual media content in at least two expression formats, and decode the bitstream to obtain a heterogeneous hybrid spliced ​​graph.

[0333] In some embodiments, the code stream includes a video compression sub-code stream and a mosaic image information sub-code stream, and the decoding unit 1201 is configured to decode the video compression sub-code stream to obtain the at least one mosaic image; and decode the mosaic image information sub-code stream to obtain mosaic image information of the at least one mosaic image.

[0334] In some embodiments, the at least one expression format includes at least one of multi-view video, point cloud, and mesh.

[0335] In some embodiments, the heterogeneous mixed mosaic graph includes at least one of the following: a single-attribute heterogeneous mixed mosaic graph and a multi-attribute heterogeneous mixed mosaic graph; the homogeneous mosaic graph includes at least one of the following: a single-attribute homogeneous mosaic graph and a multi-attribute homogeneous mosaic graph.

[0336] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here.

[0337] The above describes the apparatus and system of the embodiment of the present application from the perspective of functional units in conjunction with the accompanying drawings. It should be understood that the functional unit can be implemented in the form of hardware, can be implemented by instructions in the form of software, or can be implemented by a combination of hardware and software units. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software units in the decoding processor. Optionally, the software unit can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps in the above method embodiment in conjunction with its hardware.

[0338] In practical applications, an embodiment of the present application further provides an encoder. FIG13 is a schematic block diagram of an encoder provided in an embodiment of the present application. As shown in FIG13 , the encoder 1310 includes:

[0339] The second memory 1320 and the second processor 1330; the second memory 1320 stores a computer program that can be run on the second processor 1330, and the second processor 1330 executes the program when the encoding method on the encoder side is executed.

[0340] In practical applications, an embodiment of the present application further provides a decoder. FIG14 is a schematic block diagram of a decoder provided in an embodiment of the present application. As shown in FIG14 , a decoder 1410 includes:

[0341] The first memory 1420 and the first processor 1430; the first memory 1420 stores a computer program that can be run on the first processor 1430, and the first processor 1430 executes the decoding method on the decoder side when executing the program.

[0342] In some embodiments of the present application, the processor may include but is not limited to:

[0343] General-purpose processor, Digital Signal Processor (DSP), Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0344] In some embodiments of the present application, the memory includes but is not limited to:

[0345] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0346] In addition, the functional modules in this embodiment may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or software functional modules.

[0347] In yet another embodiment of the present application, see Figure 15 , which shows a schematic diagram of the structure of a coding and decoding system provided in an embodiment of the present application. As shown in Figure 15 , the coding and decoding system 150 may include an encoder 1501 and a decoder 1502. The encoder 1501 may be a device integrated with the encoding device described in the previous embodiment; the decoder 1502 may be a device integrated with the decoding device described in the previous embodiment.

[0348] In an embodiment of the present application, in the encoding and decoding system 150, both the encoder 1501 and the decoder 1502 can use the color component information of adjacent reference pixels and the pixels to be predicted to calculate the corresponding weighting coefficients of the pixels to be predicted; and different reference pixels can have different weighting coefficients. Applying this weighting coefficient to the chrominance prediction of the pixels to be predicted in the current block can not only improve the accuracy of the chrominance prediction and save bit rate, but also improve the encoding and decoding performance.

[0349] The present application also provides a chip for implementing the above encoding and decoding method. Specifically, the chip includes a processor for calling and running a computer program from a memory, so that an electronic device equipped with the chip executes the above encoding and decoding method.

[0350] The present application also provides a computer storage medium storing a computer program, which, when executed by a second processor, implements the encoding method of the encoder; or, when executed by a first processor, implements the decoding method of the decoder. Alternatively, the present application also provides a computer program product comprising instructions, which, when executed by a computer, causes the computer to perform the method of the aforementioned method embodiment.

[0351] The present application also provides a code stream, which is generated according to the above encoding method. Optionally, the code stream includes the above first syntax element, or includes the second syntax element and the third syntax element.

[0352] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid state drive (SSD)).

[0353] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0354] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the unit is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0355] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment. For example, the functional units in the various embodiments of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0356] The above content is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A decoding method, wherein: include: Decoding the bitstream to obtain a spliced ​​graph and spliced ​​graph information, wherein the spliced ​​graph information includes a first syntax element, and determining whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph according to the first syntax element; When it is determined according to the first syntax element that the mosaic is a heterogeneous mixed mosaic, splitting the mosaic according to the mosaic information of the mosaic to obtain at least two isomorphic blocks, wherein the at least two isomorphic blocks correspond to different visual media content expression formats; When it is determined according to the first syntax element that the spliced ​​graph is a homogeneous spliced ​​graph, splitting the spliced ​​graph according to the spliced ​​graph information of the spliced ​​graph to obtain homogeneous blocks, wherein the homogeneous blocks correspond to the same visual media content expression format; The homogeneous blocks are decoded and reconstructed to obtain visual media content in at least one expression format.

2. The method according to claim 1, wherein The determining, according to the first syntax element, whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph includes: If the first syntax element is a first preset value, it is determined that the spliced ​​graph is a heterogeneous mixed spliced ​​graph including homogeneous blocks in a first expression format and a second expression format, wherein the first expression format and the second expression format are different expression formats; If the first syntax element is a second preset value, determining that the spliced ​​graph is an isomorphic spliced ​​graph including isomorphic blocks in the first expression format; If the first syntax element is a third preset value, it is determined that the splicing graph is an isomorphic splicing graph including isomorphic blocks in the second expression format.

3. The method according to claim 1, wherein The first syntax element includes: a first sub-syntax element and a second sub-syntax element, and determining whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph according to the first sub-syntax element and the second sub-syntax element; The determining, according to the first syntax element, whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph includes: If the first sub-syntax element is a fourth preset value, it is determined that the spliced ​​graph includes isomorphic blocks in the first expression format; If the second sub-syntax element is a fifth preset value, it is determined that the spliced ​​graph includes isomorphic blocks in a second expression format; The first expression format and the second expression format are different expression formats.

4. The method according to claim 3, wherein: The determining, according to the first syntax element, whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph further includes: If the first sub-syntax element is a sixth preset value, it is determined that the splicing graph does not include isomorphic blocks in the first expression format; If the second sub-syntax element is a seventh preset value, it is determined that the splicing graph does not include isomorphic blocks in the second expression format.

5. The method according to any one of claims 1 to 4, wherein: The first syntax element is located in a parameter set sub-codestream of the codestream.

6. The method according to any one of claims 1 to 4, wherein: The mosaic sequence parameter set corresponding to the mosaic includes the first syntax element.

7. The method according to any one of claims 1 to 6, wherein: The method further comprises: When it is determined according to the first syntax element that the spliced ​​graph is a heterogeneous mixed spliced ​​graph, the spliced ​​graph information further includes a second syntax element, and an expression format of the i-th block in the spliced ​​graph is determined according to the second syntax element.

8. The method according to claim 7, wherein: The determining of the expression format of the i-th block in the spliced ​​graph according to the second syntax element includes: The second syntax element is an eighth preset value, determining that the expression format of the i-th block is the first expression format; If the second syntax element is a ninth preset value, it is determined that the expression format of the i-th block is the second expression format.

9. The method according to claim 7, wherein: The second syntax element is located in a mosaic map block data unit header of the i-th block of the mosaic map.

10. The method according to any one of claims 7 to 9, wherein: The decoding and reconstruction of the homogeneous blocks to obtain visual media content in at least one expression format includes: If the expression format of the i-th block is a first expression format, determining to decode and reconstruct a sub-block in the i-th block using a decoding method corresponding to the first expression format to obtain visual media content in the first expression format; If the expression format of the i-th block is the second expression format, it is determined that the sub-block in the i-th block is decoded and reconstructed using a decoding method corresponding to the second expression format to obtain visual media content in the second expression format.

11. The method according to any one of claims 1 to 10, wherein: The parameter set sub-codestream of the codestream includes a third syntax element, and the codestream corresponding to the visual media content including at least one expression format in the codestream is determined according to the third syntax element.

12. The method according to claim 11, wherein The determining, according to the third syntax element, a codestream corresponding to the visual media content including at least one expression format in the codestream includes: The third syntax element is a first value, and determines that the code stream includes both a code stream corresponding to the visual media content in the first expression format and a code stream corresponding to the visual media content in the second expression format; The third syntax element is a second value, which determines that the code stream includes a code stream corresponding to the visual media content in the first expression format; The third syntax element is a third value, which determines the code stream corresponding to the visual media content in the second expression format included in the code stream.

13. The method according to any one of claims 11 to 12, wherein: The decoding bitstream to obtain at least one splicing graph includes: Determine, according to the third syntax element, that the bitstream includes bitstreams corresponding to visual media content in at least two expression formats, and decode the bitstream to obtain a heterogeneous mixed splicing graph.

14. The method according to any one of claims 1 to 13, wherein: The code stream includes a video compression sub-code stream and a splicing graph information sub-code stream, and the decoding code stream obtains at least one splicing graph and splicing graph information, including: Decoding the video compression sub-stream to obtain the at least one spliced ​​graph; The mosaic information sub-code stream is decoded to obtain mosaic information of the at least one mosaic.

15. The method according to any one of claims 1 to 14, wherein: The at least one expression format includes at least one of multi-view video, point cloud, and mesh.

16. The method according to any one of claims 1 to 15, wherein: The heterogeneous mixed mosaic graph is at least one of the following: a single-attribute heterogeneous mixed mosaic graph and a multi-attribute heterogeneous mixed mosaic graph; The isomorphic mosaic graph includes at least one of the following: a single-attribute isomorphic mosaic graph and a multi-attribute isomorphic mosaic graph.

17. A coding method, wherein: include: Processing visual media content in at least one expression format to obtain at least one isomorphic block, wherein different isomorphic blocks correspond to different visual media content expression formats; Splicing the at least one isomorphic block to obtain at least one spliced ​​graph and spliced ​​graph information, wherein the spliced ​​graph information includes a first syntax element, and determining, based on the first syntax element, whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph, the heterogeneous mixed spliced ​​graph includes at least two isomorphic blocks, and the homogeneous spliced ​​graph includes one isomorphic block; The at least one mosaic image and the mosaic image information are encoded to obtain a code stream.

18. The method according to claim 17, wherein The determining, according to the first syntax element, whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph includes: If the first syntax element is a first preset value, it is determined that the spliced ​​graph is a heterogeneous mixed spliced ​​graph including homogeneous blocks in a first expression format and a second expression format, wherein the first expression format and the second expression format are different expression formats; If the first syntax element is a second preset value, determining that the spliced ​​graph is an isomorphic spliced ​​graph including isomorphic blocks in the first expression format; If the first syntax element is a third preset value, it is determined that the splicing graph is an isomorphic splicing graph including isomorphic blocks in the second expression format.

19. The method according to claim 17, wherein The first syntax element includes: a first sub-syntax element and a second sub-syntax element, and determining whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph according to the first sub-syntax element and the second sub-syntax element; The determining, according to the first syntax element, whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph includes: If the first sub-syntax element is a fourth preset value, it is determined that the spliced ​​graph includes isomorphic blocks in the first expression format; If the second sub-syntax element is a fifth preset value, it is determined that the spliced ​​graph includes isomorphic blocks in a second expression format; The first expression format and the second expression format are different expression formats.

20. The method according to claim 19, wherein The determining, according to the first syntax element, whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph further includes: If the first sub-syntax element is a sixth preset value, it is determined that the splicing graph does not include isomorphic blocks in the first expression format; If the second sub-syntax element is a seventh preset value, it is determined that the splicing graph does not include isomorphic blocks in the second expression format.

21. The method according to any one of claims 17 to 20, wherein: The first syntax element is located in a parameter set sub-codestream of the codestream.

22. The method according to any one of claims 17 to 20, wherein: The mosaic sequence parameter set corresponding to the mosaic includes the first syntax element.

23. The method according to any one of claims 17 to 22, wherein: The method further comprises: When it is determined according to the first syntax element that the spliced ​​graph is a heterogeneous mixed spliced ​​graph, the spliced ​​graph information further includes a second syntax element, and an expression format of the i-th block in the spliced ​​graph is determined according to the second syntax element.

24. The method according to claim 23, wherein The determining of the expression format of the i-th block in the spliced ​​graph according to the second syntax element includes: The second syntax element is an eighth preset value, determining that the expression format of the i-th block is the first expression format; If the second syntax element is a ninth preset value, it is determined that the expression format of the i-th block is the second expression format.

25. The method according to claim 24, wherein The second syntax element is located in a mosaic map block data unit header of the i-th block of the mosaic map.

26. The method according to any one of claims 23 to 25, wherein: The encoding of the at least one mosaic image and the mosaic image information to obtain a code stream includes: If the expression format of the i-th block is a first expression format, determining that a sub-block in the i-th block is encoded using an encoding method corresponding to the first expression format to obtain a code stream corresponding to the visual media content in the first expression format; If the expression format of the i-th block is the second expression format, it is determined that the sub-block in the i-th block is encoded using the encoding method corresponding to the second expression format to obtain a code stream corresponding to the visual media content in the second expression format.

27. The method according to any one of claims 17 to 26, wherein: The parameter set sub-codestream of the codestream includes a third syntax element, and the codestream corresponding to the visual media content including at least one expression format in the codestream is determined according to the third syntax element.

28. The method according to claim 27, wherein The determining, according to the third syntax element, a codestream corresponding to the visual media content including at least one expression format in the codestream includes: The third syntax element is a first value, and determines that the code stream includes both a code stream corresponding to the visual media content in the first expression format and a code stream corresponding to the visual media content in the second expression format; The third syntax element is a second value, which determines that the code stream includes a code stream corresponding to the visual media content in the first expression format; The third syntax element is a third value, which determines the code stream corresponding to the visual media content in the second expression format included in the code stream.

29. The method according to any one of claims 27-28, wherein: When the at least one mosaic graph includes a heterogeneous mixed mosaic graph, a code stream corresponding to the visual media content in at least two expression formats in the code stream is determined according to the third syntax element.

30. The method according to any one of claims 17 to 29, wherein: The encoding of the at least one mosaic image and the mosaic image information to obtain a code stream includes: Encoding the at least one spliced ​​image to obtain a video compression sub-stream; Encoding the mosaic information of the at least one mosaic to obtain a mosaic information substream; The video compression sub-stream and the splicing graph information sub-stream are synthesized into the stream.

31. The method according to any one of claims 17 to 30, wherein: The at least one expression format includes at least one of multi-view video, point cloud, and mesh.

32. The method according to any one of claims 17 to 31, wherein: The heterogeneous mixed mosaic graph is at least one of the following: a single-attribute heterogeneous mixed mosaic graph and a multi-attribute heterogeneous mixed mosaic graph; The isomorphic mosaic graph includes at least one of the following: a single-attribute isomorphic mosaic graph and a multi-attribute isomorphic mosaic graph.

33. A decoding device, wherein: include: a decoding unit, configured to decode a bitstream to obtain a splice graph and splice graph information, wherein the splice graph information includes a first syntax element, and determine whether the splice graph is a heterogeneous mixed splice graph or a homogeneous splice graph according to the first syntax element; a splitting unit, configured to, when determining according to the first syntax element that the mosaic is a heterogeneous mixed mosaic, split the mosaic according to the mosaic information of the mosaic to obtain at least two isomorphic blocks, wherein the at least two isomorphic blocks correspond to different visual media content expression formats; The splitting unit is configured to, when determining that the spliced ​​graph is a homogeneous spliced ​​graph according to the first syntax element, split the spliced ​​graph according to the spliced ​​graph information of the spliced ​​graph to obtain homogeneous blocks, wherein the homogeneous blocks correspond to the same visual media content expression format; The processing unit is configured to decode and reconstruct the homogeneous blocks to obtain visual media content in at least one expression format.

34. An encoding device, wherein: include: a processing unit, configured to process visual media content in at least one expression format to obtain at least one isomorphic block, wherein different isomorphic blocks correspond to different visual media content expression formats; a splicing unit, configured to splice the at least one isomorphic block to obtain at least one spliced ​​graph and spliced ​​graph information, wherein the spliced ​​graph information includes a first syntax element, and determining, based on the first syntax element, whether the spliced ​​graph is a heterogeneous mixed spliced ​​graph or a homogeneous spliced ​​graph, the heterogeneous mixed spliced ​​graph including at least two isomorphic blocks, and the homogeneous spliced ​​graph including one isomorphic block; The encoding unit is configured to encode the at least one splicing graph and the splicing graph information to obtain a code stream.

35. A decoder, wherein The decoder comprises: a first memory and a first processor; The first memory stores a computer program that can be run on the first processor, and the first processor implements the decoding method according to any one of claims 1 to 16 when executing the program.

36. An encoder, wherein The encoder comprises: a second memory and a second processor; The second memory stores a computer program that can be run on the second processor, and when the second processor executes the program, the encoding method according to any one of claims 17 to 32 is implemented.

37. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, and when the computer program is executed by a first processor, it implements the decoding method described in any one of claims 1 to 16; or, when the computer program is executed by a second processor, it implements the encoding method described in any one of claims 17 to 32.

38. A code stream, wherein The code stream is generated based on the method according to any one of claims 17 to 32.

Images