Video encoding and decoding methods and apparatuses

Abstract

The present application provides video encoding and decoding methods and apparatuses. A video encoding method of the present application comprises: segmenting an image to be encoded, to obtain multiple sub-images; acquiring multiple pieces of sub-image description information, the multiple pieces of sub-image description information corresponding to the multiple sub-images, and the sub-image description information being used for describing sizes and positions of the corresponding sub-images; separately encoding the multiple sub-images, to obtain multiple sub-image compressed bit streams; combining and transcribing the multiple sub-image compressed bit streams, to obtain a full-image compressed bit stream, the full-image compressed bit stream comprising the multiple pieces of sub-image description information. The present application can be suitable for use in a multi-core architecture system, and can reduce system level complexity and increase compatibility.

Description

Video encoding and decoding methods and apparatus Technical Field

[0001] This application relates to video processing technology, and more particularly to a video encoding and decoding method and apparatus. Background Technology

[0002] In video encoding and decoding systems, sometimes the high resolution and frame rate of the video prevent real-time encoding and decoding operations from being performed in a single encoder or decoder. In such cases, each frame of the video can be divided into multiple sub-images, and then multiple sets of encoders and decoders can process each sub-image separately. Specifically, each encoder compresses the input sub-image to obtain a corresponding compressed bitstream, and each decoder decodes the compressed bitstream of the input sub-image to output the sub-image. Then, spatial stitching is performed on the corresponding sub-images at the same time to obtain the decoded image sequence.

[0003] However, multiple codecs require multi-core collaboration, which not only increases the complexity of the system solution but also makes the solution unusable in systems that do not have multi-core collaborative control. Summary of the Invention

[0004] This application provides a video encoding and decoding method and apparatus suitable for use in multi-core architecture systems, which can reduce system-level complexity and enhance compatibility.

[0005] In a first aspect, this application provides a video encoding method, comprising: segmenting an image to be encoded into multiple sub-images; acquiring multiple sub-image description information, wherein the multiple sub-image description information corresponds to the multiple sub-images, and the sub-image description information is used to describe the size and position of the corresponding sub-image; encoding the multiple sub-images respectively to obtain multiple sub-image compressed bitstreams; and merging and transcribing the multiple sub-image compressed bitstreams to obtain a full image compressed bitstream, wherein the full image compressed bitstream includes the multiple sub-image description information.

[0006] This application splits the full image into multiple sub-images, then encodes each sub-image independently using multiple encoders, and merges and transcribs the resulting compressed bitstreams of the multiple sub-images into a single compressed bitstream of the full image. The multiple encoders do not need to coordinate with each other, resulting in low system complexity and making it more suitable for use in multi-core architecture systems. Furthermore, the merging and transcribing of the compressed bitstream does not require changes to the transmission and storage modules in the existing video system, resulting in low system-level complexity and stronger compatibility.

[0007] This application segments the image to be encoded (the whole image) into multiple sub-images. The segmentation direction includes horizontal and / or vertical, and the number of segments in the horizontal and / or vertical directions includes one or more. Optionally, the segmentation width can be aligned to the coding tree unit (CTU). For example, in the H.265 standard, the size of the smallest coding block is 16×16, and the width of the sub-images can be aligned to a multiple of 16.

[0008] In one possible implementation, the image to be encoded is segmented by overlapping segmentation: any two adjacent sub-images obtained by segmentation contain overlapping regions at their adjacent boundaries, and the corresponding pixel values ​​in the overlapping regions of any two adjacent sub-images are the same.

[0009] In one possible implementation, the image to be encoded is segmented in a non-overlapping manner: any two adjacent sub-images obtained from the segmentation do not contain overlapping regions at their adjacent boundaries.

[0010] It should be noted that in this application, whether in the horizontal or vertical direction, the segmentation can be equal or unequal, and the width of the overlapping area of ​​different sub-images can be equal or unequal. No specific limitation is made on the foregoing.

[0011] Multiple sub-image description information corresponds to multiple sub-images. After segmenting the whole image, the sub-image description information of each sub-image can be obtained, which is used to describe the size and position of the corresponding sub-image.

[0012] In one possible implementation, the sub-image description information includes the segmentation information and position information of the corresponding sub-image. The sub-image segmentation information may include the width and height of the corresponding sub-image. The position information is used to describe the position of the corresponding sub-image in the image to be encoded. For example, the position information may be the index or coordinates of the first CTU in the corresponding sub-image in the whole image, or the position information may be the coordinates of the top-left pixel of the corresponding sub-image in the whole image.

[0013] It should be noted that this application may also describe the size and position of the sub-image in other ways, without making specific limitations.

[0014] In one possible implementation, when adjacent sub-images contain overlapping regions at their adjacent boundaries, the sub-image description information can also be used to describe the size of the overlapping regions of the corresponding sub-images.

[0015] Optionally, the width and height of the overlapping area of ​​the corresponding sub-images can be directly included in the sub-image segmentation information.

[0016] Optionally, the size of the overlapping area of ​​sub-images can be determined indirectly through positional information. For example, the coordinates of the upper right pixel of sub-image 1 are (x1, y1), and the coordinates of the upper left pixel of sub-image 2 are (x2, y2). x1 > x2 and y1 = y2, indicating that sub-image 1 and sub-image 2 have an overlapping area. The width of the overlapping area is x1 - x2, and the height is y1.

[0017] It should be noted that this application may also use other methods to describe the size of the overlapping area of ​​the sub-images, without making specific limitations.

[0018] In one possible implementation, multiple sub-images (sub-image 1, sub-image 2, sub-image 3) can be encoded by multiple first encoders (encoder 1, encoder 2, encoder 3) to obtain multiple sub-image compressed bitstreams (compressed bitstream 1, compressed bitstream 2, compressed bitstream 3).

[0019] In one possible implementation, multiple first sub-image compressed bitstreams can be obtained by encoding multiple sub-images (sub-image 1, sub-image 2, sub-image 3) using multiple second encoders (protocol 1 encoder 1, protocol 1 encoder 2, protocol 1 encoder 3); multiple second sub-images can be obtained by decoding multiple first sub-image compressed bitstreams using multiple second decoders (protocol 1 decoder 1, protocol 1 decoder 2, protocol 1 decoder 3), with the second encoders and second decoders corresponding to the same video encoding protocol; and multiple second sub-images can be obtained by encoding multiple sub-images using multiple third encoders (protocol 2 encoder 1, protocol 2 encoder 2, protocol 2 encoder 3), with the second encoders and third encoders corresponding to different video encoding protocols.

[0020] In one possible implementation, a first full-image compressed bitstream can be obtained by encoding multiple sub-images separately using a second encoder (protocol 1 encoder); the first full-image compressed bitstream can be split and transcribed to obtain multiple first sub-image compressed bitstreams (protocol 1 compressed bitstream 1, protocol 1 compressed bitstream 2, protocol 1 compressed bitstream 3); multiple second decoders (protocol 1 decoder 1, protocol 1 decoder 2, protocol 1 decoder 3) can be used to decode the multiple first sub-image compressed bitstreams to obtain multiple second sub-images, with the second encoder and the second decoder corresponding to the same video encoding protocol; multiple third encoders (protocol 2 encoder 1, protocol 2 encoder 2, protocol 2 encoder 3) can be used to encode the multiple second sub-images to obtain multiple sub-image compressed bitstreams (protocol 2 compressed bitstream 1, protocol 2 compressed bitstream 2, protocol 2 compressed bitstream 3), with the second encoder and the third encoder corresponding to different video encoding protocols.

[0021] In one possible implementation, when inter-frame predictive coding is used to encode multiple sub-images, the search region of the reference block of any sub-image is smaller than that of the reference sub-image. This search process can be referred to as motion-constrained sub-image coding.

[0022] In one possible implementation, multiple sub-image compressed bitstreams can be written into the encoded data unit of the whole image in a preset order to obtain the whole image compressed bitstream.

[0023] This method can refer to the tile / patch syntax mechanism to merge and transcribe multiple sub-image bitstreams into a single full-image compressed bitstream.

[0024] In one possible implementation, multiple sub-image compressed bitstreams can be written into the coded data units of multiple image layers respectively; the coded data of the coded data units of multiple image layers can be merged to obtain the full image compressed bitstream.

[0025] This approach can refer to the hierarchical coding mechanism specified in the standard, write multiple sub-images into the coding data units of different image layers respectively, and then merge the coding data of these multiple image layer coding data units to obtain a single full-image compressed bitstream.

[0026] In one possible implementation, the compressed bitstream of the first sub-image can be written into the coding data unit of the whole image to obtain the compressed bitstream of the whole image; the compressed bitstreams of the sub-images of the other sub-images besides the first sub-image can be written into the supplementary enhancement information (SEI) information coding unit of the compressed bitstream of the whole image, wherein the first sub-image is one of the multiple sub-images.

[0027] This method involves selecting one sub-image compressed bitstream as the target bitstream, encapsulating the other sub-image compressed bitstreams in a user-defined SEI information encoding unit, embedding it into the target bitstream, and thus obtaining a single full-image compressed bitstream.

[0028] It should be noted that this application may also use other methods to merge and transcribe multiple sub-image compressed bitstreams, without making specific limitations.

[0029] Secondly, this application provides a video decoding method, comprising: acquiring a full image compressed bitstream; splitting and transcribing the full image compressed bitstream to obtain multiple sub-image compressed bitstreams, the multiple sub-image compressed bitstreams corresponding to multiple decoded sub-images; acquiring multiple sub-image description information based on the full image compressed bitstream, the multiple sub-image description information corresponding to the multiple decoded sub-images, the sub-image description information being used to describe the size and position of the corresponding decoded sub-image; decoding the multiple sub-image compressed bitstreams respectively to obtain the multiple decoded sub-images; and stitching the multiple decoded sub-images together based on the multiple sub-image description information to obtain a decoded full image.

[0030] This application splits and transcribes a single full-image compressed bitstream into multiple sub-image compressed bitstreams. Then, multiple decoders independently decode the multiple sub-image compressed bitstreams, and the decoded sub-images are then stitched together to obtain the decoded full image. The multiple decoders do not need to coordinate with each other, which reduces the complexity of the system and makes it more suitable for use in multi-core architecture systems. Furthermore, filtering is performed at the boundary between two adjacent sub-images when stitching them together, which can eliminate stitching defects between adjacent sub-images and improve image quality.

[0031] Based on the encoding system's method of merging and transcribing multiple sub-image compressed bitstreams, the decoding system can use a corresponding split transcribing method to split a single full-image compressed bitstream into multiple sub-image compressed bitstreams.

[0032] In one possible implementation, encoded data is extracted from the full image encoded data units of the full image compressed bitstream in a preset order to obtain multiple sub-image compressed bitstreams.

[0033] This method can refer to the tile / patch syntax mechanism to split and rewrite a single full-image compressed bitstream into multiple sub-image bitstreams.

[0034] In one possible implementation, coded data is extracted from the coded data units of multiple image layers of the full image compressed bitstream to obtain multiple sub-image compressed bitstreams.

[0035] This approach can refer to the layered decoding mechanism specified in the standard, splitting a single full-image compressed bitstream into encoded data of three image layer encoded data units, and then mapping the encoded data of the three image layers to three sub-image compressed bitstreams.

[0036] In one possible implementation, encoded data is extracted from the full image coding data unit of the full image compressed bitstream to obtain a sub-image compressed bitstream of the third decoded sub-image, where the third decoded sub-image is one of multiple decoded sub-images; encoded data is extracted from the SEI information coding unit of the full image compressed bitstream to obtain sub-image compressed bitstreams of other sub-images among the multiple decoded sub-images besides the third decoded sub-image.

[0037] In this approach, the full image coding data unit of the full image compressed bitstream contains one sub-image compressed bitstream, while the SEI information coding unit of the full image compressed bitstream contains other sub-image compressed bitstreams.

[0038] The above method splits and rewrites a single full-image compressed bitstream into multiple sub-image compressed bitstreams without changing the transmission and storage modules in the existing video system, resulting in lower system complexity and stronger compatibility.

[0039] It should be noted that this application may also use other methods to split and transwrite a single full-image compressed bitstream, and no specific limitations are made in this regard.

[0040] Optionally, multiple sub-image description information can be encapsulated in the SEI information encoding unit of the full image compressed bitstream.

[0041] Optionally, multiple sub-image description information can be encapsulated in the sub-image compressed bitstream of the corresponding decoded sub-image.

[0042] The aforementioned sub-image description information can be extracted from the corresponding positions in the full image compressed bitstream.

[0043] In one possible implementation, the sub-image description information includes the segmentation information and position information of the corresponding sub-image. The sub-image segmentation information may include the width and height of the corresponding sub-image. The position information is used to describe the position of the corresponding sub-image in the image to be encoded. For example, the position information may be the index or coordinates of the first CTU in the corresponding sub-image in the whole image, or the position information may be the coordinates of the top-left pixel of the corresponding sub-image in the whole image.

[0044] It should be noted that this application may also describe the size and position of the sub-image in other ways, without making specific limitations.

[0045] In one possible implementation, when adjacent sub-images contain overlapping regions at their adjacent boundaries, the sub-image description information can also be used to describe the size of the overlapping regions of the corresponding sub-images.

[0046] Optionally, the width and height of the overlapping area of ​​the corresponding sub-images can be directly included in the sub-image segmentation information.

[0047] Optionally, the size of the overlapping area of ​​sub-images can be determined indirectly through positional information. For example, the coordinates of the upper right pixel of sub-image 1 are (x1, y1), and the coordinates of the upper left pixel of sub-image 2 are (x2, y2). x1 > x2 and y1 = y2, indicating that sub-image 1 and sub-image 2 have an overlapping area. The width of the overlapping area is x1 - x2, and the height is y1.

[0048] It should be noted that this application may also use other methods to describe the size of the overlapping area of ​​the sub-images, without making specific limitations.

[0049] In one possible implementation, multiple first decoders (decoder 1, decoder 2, decoder 3) can be used to decode multiple sub-image compressed bitstreams (compressed bitstream 1, compressed bitstream 2, compressed bitstream 3) to obtain multiple decoded sub-images (decoded sub-image 1, decoded sub-image 2, decoded sub-image 3).

[0050] In one possible implementation, a first decoder (decoder) can be used to decode multiple sub-image compressed bitstreams to obtain a reconstructed decoded full image; the reconstructed decoded full image can then be split to obtain multiple decoded sub-images.

[0051] In one possible implementation, multiple fourth decoded sub-images can be obtained by decoding multiple sub-image compressed bitstreams (Protocol 2 compressed bitstream 1, Protocol 2 compressed bitstream 2, Protocol 2 compressed bitstream 3) using multiple second decoders (Protocol 2 decoder 1, Protocol 2 decoder 2, Protocol 2 decoder 3); multiple first sub-image compressed bitstreams can be obtained by encoding multiple fourth decoded sub-images using multiple third encoders (Protocol 1 encoder 1, Protocol 1 encoder 2, Protocol 1 encoder 3), with the third encoders and second decoders corresponding to different video encoding protocols; and multiple decoded sub-images (decoded sub-image 1, decoded sub-image 2, decoded sub-image 3) can be obtained by decoding multiple first sub-image compressed bitstreams using multiple third decoders (Protocol 1 decoder 1, Protocol 1 decoder 2, Protocol 1 decoder 3), with the third decoders and third encoders corresponding to the same video encoding protocol.

[0052] In one possible implementation, multiple fourth decoded sub-images can be obtained by decoding multiple sub-image compressed bitstreams (Protocol 2 compressed bitstream 1, Protocol 2 compressed bitstream 2, Protocol 2 compressed bitstream 3) using multiple second decoders (Protocol 2 decoder 1, Protocol 2 decoder 2, Protocol 2 decoder 3); multiple first sub-image compressed bitstreams (Protocol 1 compressed bitstream 1, Protocol 1 compressed bitstream 2, Protocol 1 compressed bitstream 3) can be obtained by encoding multiple fourth decoded sub-images using multiple third encoders (Protocol 1 encoder 1, Protocol 1 encoder 2, Protocol 1 encoder 3), where the third encoders and second decoders correspond to different video encoding protocols; the multiple first sub-image compressed bitstreams are merged and transcribed to obtain a first full-image compressed bitstream; and the first full-image compressed bitstream is decoded using a third decoder (Protocol 1 decoder) to obtain multiple decoded sub-images, where the third decoder and third encoder correspond to the same video encoding protocol.

[0053] In one possible implementation, when decoding multiple sub-images using inter-frame predictive coding, the search region of the reference block for any one sub-image is smaller than that of the reference sub-image. This search process can be termed motion-constrained sub-image coding.

[0054] In one possible implementation, the first decoded sub-image and the second decoded sub-image can be concatenated based on the sub-image description information of the first decoded sub-image and the second decoded sub-image. The first decoded sub-image and the second decoded sub-image are two of a plurality of decoded sub-images and are adjacent to each other. Then, a preset region at the adjacent boundary of the first decoded sub-image and the second decoded sub-image is filtered to obtain the decoded full image.

[0055] In this application, the filtering process may include the following processing methods:

[0056] 1. When the first decoded sub-image and the second decoded sub-image contain overlapping sub-image regions at adjacent boundaries, the overlapping sub-image regions of the first decoded sub-image and the second decoded sub-image are weighted to obtain the decoded full image. The corresponding pixels in the overlapping sub-image regions of the first decoded sub-image and the second decoded sub-image are the same.

[0057] The first and second decoded sub-images each contain overlapping regions at their adjacent boundaries. Therefore, the decoding system can perform weighted processing on the overlapping regions of the two sub-images. This weighted processing can be weighted summation, weighted averaging, etc., without any specific limitation.

[0058] 2. When the first decoded sub-image and the second decoded sub-image contain overlapping sub-image regions at adjacent boundaries, the first decoded sub-image overlapping region and / or the second decoded sub-image overlapping region are cropped to obtain the decoded full image, and the corresponding pixels in the first decoded sub-image overlapping region and the second decoded sub-image overlapping region are the same.

[0059] The first and second decoded sub-images each contain overlapping areas at their adjacent boundaries. Therefore, the decoding system can crop (discard) the overlapping areas of the two sub-images. This cropping process can be either discarding the overlapping area of ​​one of the sub-images or discarding a portion of the overlapping areas of the two adjacent sub-images. No specific limitation is made in this regard.

[0060] 3. When neither the first decoded sub-image nor the second decoded sub-image contains overlapping areas at their adjacent boundaries, a smoothing filter is applied to the adjacent boundaries of the first decoded sub-image and the second decoded sub-image to obtain the decoded full image.

[0061] The first and second decoded sub-images do not contain overlapping areas at their adjacent boundaries. Therefore, the decoding system can perform smoothing filtering on the boundary between them. This smoothing filtering can be boundary smoothing, low-pass filtering, neural networks, weighted averaging, etc., without any specific limitations.

[0062] The above filtering processes can eliminate stitching defects between adjacent sub-images and improve image quality. It should be noted that other filtering methods can also be used besides those mentioned above; this application does not specifically limit their application.

[0063] Thirdly, this application provides a video encoding apparatus, comprising: a segmentation module for segmenting an image to be encoded into multiple sub-images; acquiring multiple sub-image description information, wherein the multiple sub-image description information corresponds to the multiple sub-images, and the sub-image description information is used to describe the size and position of the corresponding sub-image; an encoding module for encoding the multiple sub-images respectively to obtain multiple sub-image compressed bitstreams; and a merging and transcribing module for merging and transcribing the multiple sub-image compressed bitstreams to obtain a full image compressed bitstream, wherein the full image compressed bitstream includes the multiple sub-image description information.

[0064] In one possible implementation, any two adjacent sub-images among the plurality of sub-images each contain an overlapping region at their adjacent boundaries, and the corresponding pixel values ​​in the overlapping regions of the two adjacent sub-images are the same.

[0065] In one possible implementation, the sub-image description information is further used to describe the size of the overlapping region of the corresponding sub-images.

[0066] In one possible implementation, the merging and transcribing module is specifically used to write the multiple sub-image compressed bitstreams into the encoded data unit of the whole image in a preset order to obtain the whole image compressed bitstream.

[0067] In one possible implementation, the merging and transcribing module is specifically used to write the multiple sub-image compressed bitstreams into the encoded data units of multiple image layers respectively; and merge the encoded data of the encoded data units of the multiple image layers to obtain the full image compressed bitstream.

[0068] In one possible implementation, the merging and transcribing module is specifically used to write the sub-image compressed bitstream of the first sub-image into the encoding data unit of the whole image to obtain the whole image compressed bitstream; and to write the sub-image compressed bitstreams of the other sub-images among the plurality of sub-images (excluding the first sub-image) into the supplementary enhancement information (SEI) information encoding unit of the whole image compressed bitstream, wherein the first sub-image is one of the plurality of sub-images.

[0069] In one possible implementation, the plurality of sub-image description information is encapsulated in the SEI information encoding unit of the full image compressed bitstream.

[0070] In one possible implementation, the multiple sub-image description information are respectively encapsulated in the sub-image compressed bitstream of the corresponding sub-image.

[0071] In one possible implementation, the sub-image description information includes segmentation information and position information of the corresponding sub-image. The sub-image segmentation information includes the width and height of the corresponding sub-image, and the position information is used to describe the position of the corresponding sub-image in the image to be encoded.

[0072] In one possible implementation, the sub-image segmentation information also includes the width and height of the overlapping region of the corresponding sub-images.

[0073] In one possible implementation, the location information includes the index or position coordinates of the first coding tree unit in the corresponding sub-image in the image to be encoded; or, the location information includes the position coordinates of the top-left vertex pixel of the corresponding sub-image in the image to be encoded.

[0074] In one possible implementation, the encoding module is specifically used to encode the plurality of sub-images respectively through a plurality of first encoders to obtain the plurality of sub-image compressed bitstreams.

[0075] In one possible implementation, the encoding module is specifically used to encode the plurality of sub-images respectively using a plurality of second encoders to obtain a plurality of first sub-image compressed bitstreams; to decode the plurality of first sub-image compressed bitstreams respectively using a plurality of second decoders to obtain a plurality of second sub-images, wherein the second encoders and the second decoders correspond to the same video encoding protocol; and to encode the plurality of second sub-images respectively using a plurality of third encoders to obtain the plurality of sub-image compressed bitstreams, wherein the second encoders and the third encoders correspond to different video encoding protocols.

[0076] In one possible implementation, the encoding module is specifically configured to encode the plurality of sub-images respectively using a second encoder to obtain a first full-image compressed bitstream; split and transcribe the first full-image compressed bitstream to obtain a plurality of first sub-image compressed bitstreams; decode the plurality of first sub-image compressed bitstreams respectively using a plurality of second decoders to obtain a plurality of second sub-images, wherein the second encoder and the second decoder correspond to the same video encoding protocol; and encode the plurality of second sub-images respectively using a plurality of third encoders to obtain the plurality of sub-image compressed bitstreams, wherein the second encoder and the third encoder correspond to different video encoding protocols.

[0077] In one possible implementation, when the encoding of the plurality of sub-images adopts an inter-frame predictive coding mode, the search region of the reference block of any one of the sub-images is smaller than that of the reference sub-image.

[0078] Fourthly, this application provides a video decoding apparatus, comprising: a splitting and transcribing module for acquiring a full image compressed bitstream; splitting and transcribing the full image compressed bitstream to obtain multiple sub-image compressed bitstreams, the multiple sub-image compressed bitstreams corresponding to multiple decoded sub-images; acquiring multiple sub-image description information based on the full image compressed bitstream, the multiple sub-image description information corresponding to the multiple decoded sub-images, the sub-image description information describing the size and position of the corresponding decoded sub-image; a decoding module for decoding the multiple sub-image compressed bitstreams respectively to obtain the multiple decoded sub-images; and a stitching module for stitching the multiple decoded sub-images together based on the multiple sub-image description information to obtain a decoded full image.

[0079] In one possible implementation, the stitching module is specifically used to stitch the first decoded sub-image and the second decoded sub-image together according to the sub-image description information of the first decoded sub-image and the second decoded sub-image, wherein the first decoded sub-image and the second decoded sub-image are two of the plurality of decoded sub-images and are adjacent to each other; and to perform filtering processing on a preset area at the adjacent boundary of the first decoded sub-image and the second decoded sub-image to obtain the decoded full image.

[0080] In one possible implementation, the stitching module is specifically used to perform weighted processing on the overlapping regions of the first and second decoded sub-images at the adjacent boundaries to obtain the decoded full image when the first decoded sub-image and the second decoded sub-image respectively contain overlapping regions of sub-images. The pixels at corresponding positions in the overlapping regions of the first and second decoded sub-images are the same.

[0081] In one possible implementation, the stitching module is specifically used to crop the overlapping regions of the first and second decoded sub-images at their adjacent boundaries to obtain the full decoded image when the first decoded sub-image and the second decoded sub-image respectively contain overlapping regions of sub-images. The corresponding pixels in the overlapping regions of the first and second decoded sub-images are the same.

[0082] In one possible implementation, the sub-image description information is further used to describe the size of the overlapping region of the corresponding decoded sub-image.

[0083] In one possible implementation, the stitching module is specifically used to perform smoothing filtering on the adjacent boundaries of the first decoded sub-image and the second decoded sub-image to obtain the decoded full image when neither the first decoded sub-image nor the second decoded sub-image contains an overlapping area at the adjacent boundary.

[0084] In one possible implementation, the plurality of sub-image description information is encapsulated in the supplementary enhancement information (SEI) information encoding unit of the full image compressed bitstream.

[0085] In one possible implementation, the plurality of sub-image description information are respectively encapsulated in the sub-image compressed bitstream of the corresponding decoded sub-image.

[0086] In one possible implementation, the split transcribing module is specifically used to extract encoded data from the full image encoded data units of the full image compressed bitstream in a preset order to obtain the plurality of sub-image compressed bitstreams.

[0087] In one possible implementation, the split transcribing module is specifically used to extract encoded data from the encoded data units of multiple image layers of the full image compressed bitstream to obtain the multiple sub-image compressed bitstreams.

[0088] In one possible implementation, the split transcribing module is specifically used to extract encoded data from the full image encoding data unit of the full image compressed bitstream to obtain a sub-image compressed bitstream of a third decoded sub-image, wherein the third decoded sub-image is one of the plurality of decoded sub-images; and to extract encoded data from the SEI information encoding unit of the full image compressed bitstream to obtain sub-image compressed bitstreams of other sub-images among the plurality of decoded sub-images except for the third decoded sub-image.

[0089] In one possible implementation, the split transcribing module is specifically used to decode the multiple sub-image compressed bitstreams using multiple first decoders to obtain the multiple decoded sub-images.

[0090] In one possible implementation, the decoding module is specifically used to decode the compressed bitstreams of the plurality of sub-images respectively through a first decoder to obtain a reconstructed decoded full image; and to split the reconstructed decoded full image to obtain the plurality of decoded sub-images.

[0091] In one possible implementation, the decoding module is specifically configured to decode the plurality of sub-image compressed bitstreams using a plurality of second decoders to obtain a plurality of fourth decoded sub-images; encode the plurality of fourth decoded sub-images using a plurality of third encoders to obtain a plurality of first sub-image compressed bitstreams, wherein the third encoders and the second decoders correspond to different video encoding protocols; and decode the plurality of first sub-image compressed bitstreams using a plurality of third decoders to obtain the plurality of decoded sub-images, wherein the third decoders and the third encoders correspond to the same video encoding protocol.

[0092] In one possible implementation, the decoding module is specifically configured to decode the plurality of sub-image compressed bitstreams using a plurality of second decoders to obtain a plurality of fourth decoded sub-images; encode the plurality of fourth decoded sub-images using a plurality of third encoders to obtain a plurality of first sub-image compressed bitstreams, wherein the third encoders and the second decoders correspond to different video encoding protocols; merge and transcribe the plurality of first sub-image compressed bitstreams to obtain a first full-image compressed bitstream; and decode the first full-image compressed bitstream using a third decoder to obtain the plurality of decoded sub-images, wherein the third decoder and the third encoder correspond to the same video encoding protocol.

[0093] In one possible implementation, when the decoding of the plurality of sub-image compressed bitstreams adopts an inter-frame prediction decoding mode, the search region of the reference block of any one of the decoded sub-images is smaller than that of the reference decoded sub-image.

[0094] Fifthly, this application provides an encoding system comprising: one or more processors; a memory for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the method as described in any one of the first aspects above.

[0095] In a sixth aspect, this application provides a decoding system, comprising: one or more processors; a memory for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the method as described in any one of the second aspects above.

[0096] In a seventh aspect, this application provides a computer-readable storage medium including a computer program that, when executed on a computer, causes the computer to perform the method described in any one of the first to second aspects above.

[0097] Eighthly, this application provides a computer program product comprising computer program code, which, when run on a computer, causes the computer to perform the method described in any one of the first to second aspects. Attached Figure Description

[0098] Figure 1a is an architecture diagram of the video encoding system of this application;

[0099] Figure 1b is an architecture diagram of the video decoding system of this application;

[0100] Figure 2 is an architecture diagram of the video decoding system of this application;

[0101] Figure 3a is an architecture diagram of the video encoding system of this application;

[0102] Figure 3b is an architecture diagram of the video encoding system of this application;

[0103] Figure 4a is an architecture diagram of the video decoding system of this application;

[0104] Figure 4b is an architecture diagram of the video decoding system of this application;

[0105] Figure 5 is a flowchart of process 500 of the video encoding method provided in an embodiment of this application;

[0106] Figure 6a is a schematic diagram of the sub-images obtained by overlapping segmentation in this application;

[0107] Figure 6b is a schematic diagram of the sub-images obtained by non-overlapping segmentation in this application;

[0108] Figure 7 is a schematic diagram of the motion constraint sub-image encoding of this application;

[0109] Figure 8 is a schematic diagram of slice address writing in the sub-image bitstream merging and transcribing process of this application;

[0110] Figure 9 is a flowchart of process 900 of the video decoding method provided in an embodiment of this application;

[0111] Figure 10a is a schematic diagram of the weighted average of pixel values ​​in the overlapping region of this application;

[0112] Figure 10b is a schematic diagram of pixel value clipping in the overlapping area of ​​this application;

[0113] Figure 10c is a schematic diagram of post-processing filtering in this application;

[0114] Figure 11 is a schematic diagram of the decoded full image of this application;

[0115] Figure 12 is a schematic diagram of the sub-image boundary post-processing filtering of this application;

[0116] Figure 13 is a structural schematic diagram of the video encoding device 1300 of this application;

[0117] Figure 14 is a schematic diagram of the structure of the video decoding device 1400 of this application. Detailed Implementation

[0118] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0119] The terms "first," "second," etc., used in the specification, embodiments, claims, and drawings of this application are for distinguishing purposes only and should not be construed as indicating or implying relative importance or order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as including a series of steps or units. A method, system, product, or apparatus is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0120] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0121] Before introducing the technical solution of this application, the key terms of this application will be explained.

[0122] An image is a two-dimensional matrix composed of pixels (image elements), each pixel having specific chroma and luminance samples. YUV420 is a common image color format, frequently used in video compression and digital image processing. In YUV420 format, the sampling frequency of chroma information is half that of luminance information. A YUV420 image consists of three component matrices: Y, U, and V. The horizontal and vertical dimensions of the U / V component matrices are half that of the Y component matrix. The resolution of the image is usually represented by the dimensions of the Y component matrix.

[0123] Image encoding: Compresses the input raw image and outputs a compressed image bitstream.

[0124] Image decoding: Decompresses the input image compressed bitstream and outputs a decoded image. The decoded image differs from the original image and contains compression distortion.

[0125] Video: A series of consecutive images.

[0126] Video encoding: Each image in the input raw video is sequentially compressed, and the output is a compressed video bitstream. Images in the video can be compressed independently, or temporal predictive coding can be performed with reference to the decoded images of previously compressed images.

[0127] Video decoding: Decompresses the input compressed video bitstream and outputs a decoded video. The decoded video differs from the original video and contains compression distortion.

[0128] A video consists of a series of images. In video encoding and decoding operations, each image needs to be encoded and decoded. Therefore, in this application, the terms "video" and "image sequence" will be used indiscriminately, as will the terms "video" and "image".

[0129] Network Abstraction Layer (NAL) unit: A syntax structure that contains a type indicator for subsequent data and the number of bytes it contains. The data appears in the form of a raw byte sequence payload (RBSP), which may also include scattered anti-spoofing bytes if necessary.

[0130] Access unit (AU): A group of NAL units that are interconnected according to specified rules and are sequential in the decoding order.

[0131] Bitstream: A binary data stream formed by encoding an image.

[0132] Inter-frame prediction: The process of generating predicted values ​​for current image samples using previously decoded images.

[0133] Output picture: The decoder will reconstruct the image output as an RL image, IDR image, P image, B image, or RAPI image after decoding.

[0134] Sequence parameter set (SPS): A syntactic structure containing syntactic elements that operate on a encoded video sequence. The encoded video sequence begins with an SPS NAL unit.

[0135] Picture parameter set (PPS): A syntactic structure containing syntactic elements that operate on an encoded picture. The encoded picture begins with a PPS NAL unit.

[0136] The video encoding and decoding method provided in this application can be combined with any encoding and decoding standard that supports sub-image mechanism. It can be used for single video encoding and decoding or for secondary concatenated encoding and decoding in standard protocol conversion.

[0137] Figure 1a is an architecture diagram of the video encoding system of this application. As shown in Figure 1a, the front-end camera captures high-resolution video, and each image (full image) in the video is sequentially sent to the encoding system. The sub-image segmentation module of this encoding system segments the full image into three sub-images (sub-image 1, sub-image 2, and sub-image 3), which are then sent to three encoders (encoder 1, encoder 2, and encoder 3) for independent encoding. Encoder 1 outputs compressed bitstream 1, encoder 2 outputs compressed bitstream 2, and encoder 3 outputs compressed bitstream 3. The aforementioned three compressed bitstreams (sub-image compressed bitstreams) are merged, transcribed, and encapsulated into a single compressed bitstream (full image compressed bitstream). The obtained single compressed bitstream is directly stored or transmitted to the decoding system.

[0138] Figure 1b is an architecture diagram of the video decoding system of this application. As shown in Figure 1b, the decoding system splits and rewrites the input single compressed bitstream (full image compressed bitstream) into three compressed bitstreams (compressed bitstream 1, compressed bitstream 2, and compressed bitstream 3), which are then sent to three decoders (decoder 1, decoder 2, and decoder 3) for independent decoding. Decoder 1 outputs decoded sub-image 1, decoder 2 outputs decoded sub-image 2, and decoder 3 outputs decoded sub-image 3. The aforementioned three decoded sub-images are then stitched together to obtain the decoded full image (decoded display image).

[0139] Figure 2 is an architecture diagram of the video decoding system of this application. As shown in Figure 2, unlike the decoding system with multiple independent decoders shown in Figure 1b, the decoding system of this embodiment splits and transcribes the input single compressed bitstream (full image compressed bitstream) to obtain full image / sub-image overlap segmentation information, and then sends the single compressed bitstream into a decoder for decoding, outputting a decoded image. Based on the full image / sub-image overlap segmentation information, the aforementioned decoded image is split, de-overlapped, and stitched together to obtain the decoded full image.

[0140] Figure 3a is an architecture diagram of the video encoding system of this application. As shown in Figure 3a, this encoding system is a protocol conversion encoding system. The front-end camera acquires high-resolution video and sends each image (full image) in the video sequentially to chip / device 1. The sub-image segmentation module of chip / device 1 segments the full image into 3 sub-images (sub-image 1, sub-image 2, and sub-image 3), which are then sent to 3 Protocol 1 encoders (Protocol 1 encoder 1, Protocol 1 encoder 2, and Protocol 1 encoder 3) for independent encoding. Protocol 1 encoder 1 outputs Protocol 1 compressed bitstream 1, Protocol 1 encoder 2 outputs Protocol 1 compressed bitstream 2, and Protocol 1 encoder 3 outputs Protocol 1 compressed bitstream 3.

[0141] The aforementioned three Protocol 1 compressed bitstreams (sub-image compressed bitstreams) are transmitted to chip / device 2 via the bus. They are then independently decoded by three Protocol 1 decoders (Protocol 1 decoder 1, Protocol 1 decoder 2, and Protocol 1 decoder 3). Protocol 1 decoder 1 outputs Protocol 1 decoded sub-image 1, Protocol 1 decoder 2 outputs Protocol 1 decoded sub-image 2, and Protocol 1 decoder 3 outputs Protocol 1 decoded sub-image 3. These three Protocol 1 decoded sub-images are then independently encoded by three Protocol 2 encoders (Protocol 2 encoder 1, Protocol 2 encoder 2, and Protocol 2 encoder 3). Protocol 2 encoder 1 outputs Protocol 2 compressed bitstream 1, Protocol 2 encoder 2 outputs Protocol 2 compressed bitstream 2, and Protocol 2 encoder 3 outputs Protocol 2 compressed bitstream 3. These three Protocol 2 compressed bitstreams (sub-image compressed bitstreams) are then merged, transcribed, and encapsulated into a single Protocol 2 compressed bitstream (full image compressed bitstream). The resulting single Protocol 2 compressed bitstream is directly stored or transmitted to the decoding system.

[0142] Figure 3b is an architecture diagram of the video encoding system of this application. As shown in Figure 3b, this encoding system is a protocol conversion encoding system. The front-end camera acquires high-resolution video and sends each image (full image) in the video sequentially to chip / device 1. The sub-image segmentation module of chip / device 1 segments the full image into 3 sub-images (sub-image 1, sub-image 2, and sub-image 3), which are then sent to a Protocol 1 encoder for encoding. The Protocol 1 encoder outputs a single Protocol 1 compressed bitstream.

[0143] The aforementioned single Protocol 1 compressed bitstream (full image compressed bitstream) is transmitted to chip / device 2 via the bus. It is then split and transcribed into three Protocol 1 compressed bitstreams (Protocol 1 compressed bitstream 1, Protocol 1 compressed bitstream 2, and Protocol 1 compressed bitstream 3), which are then fed into three Protocol 1 decoders (Protocol 1 decoder 1, Protocol 1 decoder 2, and Protocol 1 decoder 3) for independent decoding. Protocol 1 decoder 1 outputs Protocol 1 decoded sub-image 1, Protocol 1 decoder 2 outputs Protocol 1 decoded sub-image 2, and Protocol 1 decoder 3 outputs Protocol 1 decoded sub-image 3. These three Protocol 1 decoded sub-images are then fed into three Protocol 2 encoders (Protocol 2 encoder 1, Protocol 2 encoder 2, and Protocol 2 encoder 3) for independent encoding. Protocol 2 encoder 1 outputs Protocol 2 compressed bitstream 1, Protocol 2 encoder 2 outputs Protocol 2 compressed bitstream 2, and Protocol 2 encoder 3 outputs Protocol 2 compressed bitstream 3. These three Protocol 2 compressed bitstreams (sub-image compressed bitstreams) are then merged, transcribed, and encapsulated into a single Protocol 2 compressed bitstream (full image compressed bitstream). The obtained single Protocol 2 compressed bitstream is directly stored or transmitted to the decoding system.

[0144] Figure 4a is an architecture diagram of the video decoding system of this application. As shown in Figure 4a, this decoding system is a protocol conversion and decoding system. Chip / device 1 receives a single Protocol 2 compressed bitstream (full image compressed bitstream) output by the encoding system, splits and transcodes it into three Protocol 2 compressed bitstreams (Protocol 2 compressed bitstream 1, Protocol 2 compressed bitstream 2, and Protocol 2 compressed bitstream 3), and sends them to three Protocol 2 decoders (Protocol 2 decoder 1, Protocol 2 decoder 2, and Protocol 2 decoder 3) for independent decoding. Protocol 2 decoder 1 outputs Protocol 2 decoded sub-image 1, Protocol 2 decoder 2 outputs Protocol 2 decoded sub-image 2, and Protocol 2 decoder 3 outputs Protocol 2 decoded sub-image 3. The aforementioned three Protocol 2 decoded sub-images are then sent to three Protocol 1 encoders (Protocol 1 encoder 1, Protocol 1 encoder 2, and Protocol 1 encoder 3) for independent encoding. Protocol 1 encoder 1 outputs Protocol 1 compressed bitstream 1, Protocol 1 encoder 2 outputs Protocol 1 compressed bitstream 2, and Protocol 1 encoder 3 outputs Protocol 1 compressed bitstream 3.

[0145] The aforementioned three Protocol 1 compressed bitstreams (sub-image compressed bitstreams) are transmitted to chip / device 2 via the bus. They are then independently decoded by three Protocol 1 decoders (Protocol 1 decoder 1, Protocol 1 decoder 2, and Protocol 1 decoder 3). Protocol 1 decoder 1 outputs Protocol 1 decoded sub-image 1, Protocol 1 decoder 2 outputs Protocol 1 decoded sub-image 2, and Protocol 1 decoder 3 outputs Protocol 1 decoded sub-image 3. These three Protocol 1 decoded sub-images are then stitched together to obtain the decoded full image.

[0146] Figure 4b is an architecture diagram of the video decoding system of this application. As shown in Figure 4b, this decoding system is a protocol conversion and decoding system. Chip / device 1 receives a single Protocol 2 compressed bitstream (full image compressed bitstream) output by the encoding system, splits and transcodes it into three Protocol 2 compressed bitstreams (Protocol 2 compressed bitstream 1, Protocol 2 compressed bitstream 2, and Protocol 2 compressed bitstream 3), and sends them to three Protocol 2 decoders (Protocol 2 decoder 1, Protocol 2 decoder 2, and Protocol 2 decoder 3) for independent decoding. Protocol 2 decoder 1 outputs Protocol 2 decoded sub-image 1, Protocol 2 decoder 2 outputs Protocol 2 decoded sub-image 2, and Protocol 2 decoder 3 outputs Protocol 2 decoded sub-image 3. The aforementioned three Protocol 2 decoded sub-images are then sent to three Protocol 1 encoders (Protocol 1 encoder 1, Protocol 1 encoder 2, and Protocol 1 encoder 3) for independent encoding. Protocol 1 encoder 1 outputs Protocol 1 compressed bitstream 1, Protocol 1 encoder 2 outputs Protocol 1 compressed bitstream 2, and Protocol 1 encoder 3 outputs Protocol 1 compressed bitstream 3. The aforementioned three Protocol 1 compressed bitstreams (sub-image compressed bitstreams) are merged and transcribed into a single Protocol 1 compressed bitstream (full image compressed bitstream).

[0147] The aforementioned single Protocol 1 compressed bitstream (full image compressed bitstream) is transmitted to chip / device 2 via the bus. It is then sent to a Protocol 1 decoder for decoding, and the Protocol 1 decoder outputs a Protocol 1 decoded image. The decoded image is then subjected to sub-image boundary cropping (e.g., splitting, de-overlapping, and stitching) to obtain the decoded full image.

[0148] It should be noted that (1) the encoding system shown in Figure 3a can be combined with the decoding system shown in Figure 4a to form a video encoding and decoding system, and the encoding system shown in Figure 3a can also be combined with the decoding system shown in Figure 4b to form a video encoding and decoding system; and the encoding system shown in Figure 3b can be combined with the decoding system shown in Figure 4a to form a video encoding and decoding system, and the encoding system shown in Figure 3b can also be combined with the decoding system shown in Figure 4b to form a video encoding and decoding system. The combination of multiple encoding and decoding systems can be compatible with more diverse video encoding and decoding scenarios. (2) The above multiple encoders / decoders can be encoders / decoders with independent hardware resources, or multiple encoders / decoders can be virtualized based on a standard encoder hardware resource and independently encoded / decoded in a time-division multiplexing manner. Multiple encoders / decoders can realize independent parallel encoding / decoding operations of multiple sub-images, and the encoders / decoders do not need to coordinate with each other, which makes the system solution less complex and more suitable for use in multi-core architecture systems. (3) The encoder / decoder described above can be a video encoder / decoder conforming to any standard, such as H.263, H.264, H.265, H.266, AVS series standards, SVAC series standards, etc., or a video encoder / decoder supporting neural networks, such as end-to-end video encoding and decoding. Conformity to any standard can enhance the compatibility of the encoding and decoding systems. (4) Figures 1a-4b respectively illustrate the architecture of the encoding / decoding system, but this does not limit the encoding / decoding system. For example, there can be 3 encoders / decoders, or other numbers (e.g., 2, 4, etc.). For example, protocol conversion can include two or more protocol conversions. This application does not specifically limit the foregoing content.

[0149] Based on the architecture of the encoding / decoding system described above, the technical solution provided in this application will be explained below.

[0150] Figure 5 is a flowchart of process 500 of the video encoding method provided in an embodiment of this application. Process 500 can be executed by the encoding system described above. Process 500 is described as a series of steps or operations. It should be understood that process 500 can be executed in various orders and / or occur simultaneously, and is not limited to the execution order shown in Figure 5. Process 500 may include:

[0151] Step 501: Segment the image to be encoded to obtain multiple sub-images.

[0152] This application segments the image to be encoded (the whole image) into multiple sub-images. The segmentation direction includes horizontal and / or vertical, and the number of segments in the horizontal and / or vertical directions includes one or more. Optionally, the segmentation width can be aligned to the coding tree unit (CTU). For example, in the H.265 standard, the size of the smallest coding block is 16×16, and the width of the sub-images can be aligned to a multiple of 16.

[0153] In one possible implementation, the image to be encoded is segmented by overlapping segmentation: any two adjacent sub-images obtained by segmentation contain overlapping regions at their adjacent boundaries, and the corresponding pixel values ​​in the overlapping regions of any two adjacent sub-images are the same.

[0154] For example, Figure 6a is a schematic diagram of the sub-images obtained by overlapping segmentation in this application. As shown in Figure 6a, the entire image is divided into three sub-images in the horizontal direction, denoted as sub-image 1, sub-image 2, and sub-image 3. Sub-image 1 and sub-image 2 are adjacent. The right column of pixels of sub-image 1 and the left column of pixels of sub-image 2 constitute their respective sub-image overlapping areas, and the corresponding pixel values ​​in the overlapping areas of these two sub-images are the same. Sub-image 2 and sub-image 3 are adjacent. The right column of pixels of sub-image 2 and the left column of pixels of sub-image 3 constitute their respective sub-image overlapping areas, and the corresponding pixel values ​​in the overlapping areas of these two sub-images are the same.

[0155] Taking equal horizontal segmentation as an example, let the width of the entire image be w, the height be h, the width of the three sub-images be ws, the width of the overlapping area between sub-images 1 and 2 be d1, and the width of the overlapping area between sub-images 2 and 3 be d2. Then, w = 3 × ws - d1 - d2. Since there is no vertical segmentation, the height of the three sub-images is equal to the height of the entire image, which is h. If the width of the overlapping area of ​​all sub-images is equal, then d1 = d2. This overlapping segmentation method, where adjacent sub-images contain regions with the same number of pixels, can help the decoding system eliminate stitching defects between sub-images when stitching them together to obtain the entire image, thus improving image quality.

[0156] In one possible implementation, the image to be encoded is segmented in a non-overlapping manner: any two adjacent sub-images obtained from the segmentation do not contain overlapping regions at their adjacent boundaries.

[0157] For example, Figure 6b is a schematic diagram of the non-overlapping segmentation of the sub-images obtained in this application. As shown in Figure 6b, the entire image is divided into three sub-images in the horizontal direction, denoted as sub-image 1, sub-image 2, and sub-image 3. Sub-image 1 and sub-image 2 are adjacent, and there is no overlap between them. Sub-image 2 and sub-image 3 are adjacent, and there is no overlap between them either.

[0158] Taking a horizontally equally divided image as an example, let the width of the entire image be w, the height be h, and the width of the three sub-images be ws. Then w = 3 × ws. Since there is no vertical division, the height of the three sub-images is equal to the height of the entire image, which is h.

[0159] It should be noted that in this application, whether in the horizontal or vertical direction, the segmentation can be equal or unequal, and the width of the overlapping area of ​​different sub-images can be equal or unequal. No specific limitation is made on the foregoing.

[0160] Step 502: Obtain description information for multiple sub-images.

[0161] Multiple sub-image description information corresponds to multiple sub-images. After segmenting the whole image, the sub-image description information of each sub-image can be obtained, which is used to describe the size and position of the corresponding sub-image.

[0162] In one possible implementation, the sub-image description information includes the segmentation information and position information of the corresponding sub-image. The sub-image segmentation information may include the width and height of the corresponding sub-image, such as ws and h mentioned above. The position information is used to describe the position of the corresponding sub-image in the image to be encoded. For example, the position information may be the index or coordinates of the first CTU in the corresponding sub-image in the whole image, or the position information may be the coordinates of the top-left pixel of the corresponding sub-image in the whole image.

[0163] It should be noted that this application may also describe the size and position of the sub-image in other ways, without making specific limitations.

[0164] In one possible implementation, when adjacent sub-images contain overlapping regions at their adjacent boundaries, the sub-image description information can also be used to describe the size of the overlapping regions of the corresponding sub-images.

[0165] Optionally, the width and height of the overlapping regions of the corresponding sub-images can be directly included in the sub-image segmentation information, such as d1, d2, and h mentioned above.

[0166] Optionally, the size of the overlapping area of ​​sub-images can be determined indirectly through positional information. For example, the coordinates of the upper right pixel of sub-image 1 are (x1, y1), and the coordinates of the upper left pixel of sub-image 2 are (x2, y2). x1 > x2 and y1 = y2, indicating that sub-image 1 and sub-image 2 have an overlapping area. The width of the overlapping area is x1 - x2, and the height is y1.

[0167] It should be noted that this application may also use other methods to describe the size of the overlapping area of ​​the sub-images, without making specific limitations.

[0168] Step 503: Encode the multiple sub-images to obtain multiple sub-image compressed bitstreams.

[0169] In one possible implementation, referring to the encoding system shown in FIG1a, multiple sub-images (sub-image 1, sub-image 2, sub-image 3) can be encoded by multiple first encoders (encoder 1, encoder 2, encoder 3) to obtain multiple sub-image compressed bitstreams (compressed bitstream 1, compressed bitstream 2, compressed bitstream 3).

[0170] In one possible implementation, referring to the encoding system shown in FIG3a, multiple sub-images (sub-image 1, sub-image 2, sub-image 3) can be encoded by multiple second encoders (protocol 1 encoder 1, protocol 1 encoder 2, protocol 1 encoder 3) to obtain multiple first sub-image compressed bitstreams; multiple second decoders (protocol 1 decoder 1, protocol 1 decoder 2, protocol 1 decoder 3) can be decoded by multiple first sub-image compressed bitstreams to obtain multiple second sub-images, wherein the second encoders and second decoders correspond to the same video encoding protocol; multiple third encoders (protocol 2 encoder 1, protocol 2 encoder 2, protocol 2 encoder 3) can be encoded by multiple second sub-images to obtain multiple sub-image compressed bitstreams (protocol 2 compressed bitstream 1, protocol 2 compressed bitstream 2, protocol 2 compressed bitstream 3), wherein the second encoders and third encoders correspond to different video encoding protocols.

[0171] In one possible implementation, referring to the encoding system shown in FIG3b, a first full-image compressed bitstream is obtained by encoding multiple sub-images separately using a second encoder (protocol 1 encoder); the first full-image compressed bitstream is split and transcribed to obtain multiple first sub-image compressed bitstreams (protocol 1 compressed bitstream 1, protocol 1 compressed bitstream 2, protocol 1 compressed bitstream 3); multiple second decoders (protocol 1 decoder 1, protocol 1 decoder 2, protocol 1 decoder 3) are used to decode the multiple first sub-image compressed bitstreams to obtain multiple second sub-images, wherein the second encoder and the second decoder correspond to the same video encoding protocol; multiple third encoders (protocol 2 encoder 1, protocol 2 encoder 2, protocol 2 encoder 3) are used to encode the multiple second sub-images to obtain multiple sub-image compressed bitstreams (protocol 2 compressed bitstream 1, protocol 2 compressed bitstream 2, protocol 2 compressed bitstream 3), wherein the second encoder and the third encoder correspond to different video encoding protocols.

[0172] In one possible implementation, when inter-frame predictive coding is used to encode multiple sub-images, the search region of the reference block of any sub-image is smaller than that of the reference sub-image. This search process can be referred to as motion-constrained sub-image coding.

[0173] For example, Figure 7 is a schematic diagram of motion-constrained sub-image encoding according to this application. As shown in Figure 7, the three sub-images (image 1, sub-image 2, and sub-image 3) are fed into three encoders for independent encoding. For example, international, national, industry, or group standards such as H.265, AVS3, and SVAC3.0 can be used for sub-image encoding. Taking H.265 as an example, when encoding each sub-image, the encoding block can adopt intra-frame predictive coding or inter-frame predictive coding mode. When using inter-frame predictive coding mode, the reference sub-image can be a previously encoded reconstructed sub-image, and all reference pixels used in the encoding block during motion compensation interpolation filtering must be located within the reference sub-image. This is because interpolation filtering requires the use of pixel values ​​outside the reference block for interpolation operations, so the search area of ​​the reference block is smaller than that of the reference sub-image, as shown by the dashed area in Figure 7. The distance between the dashed area and the boundary of the reference sub-image is related to the interpolation filter used. Therefore, when performing motion search on the coded block, the search area of ​​the reference block needs to be constrained within the dashed area of ​​the reference sub-image.

[0174] Step 504: Merge and transcribe the compressed bitstreams of multiple sub-images to obtain a full image compressed bitstream, which includes description information of multiple sub-images.

[0175] In one possible implementation, multiple sub-image compressed bitstreams can be written into the encoded data unit of the whole image in a preset order to obtain the whole image compressed bitstream.

[0176] This method, referencing the tile / patch syntax mechanism, merges and transcribs multiple sub-image bitstreams into a single full-image compressed bitstream. The merging and transcribing process includes the following steps:

[0177] 1. Remove redundant non-coded NAL units. For example, retain only one sequence parameter set and one image parameter set to ultimately form a single full-image compressed bitstream that conforms to the standard.

[0178] 2. Set the size-related parameters for the image and its sub-images. For example, set the image width and height in the sequence parameter set of the full image bitstream to 3×ws and h, respectively. Rewrite the size-related syntax of the tiles in the image parameter set, setting the width of the three tiles to ws (fixed according to the H.265 standard; the ws in pixels needs to be converted to a value in coding tree unit size).

[0179] 3. Set the slice address syntax in the slice header. For example, set the slice_segment_address of the first slice in each sub-image to the index value of the first CTU in the (3×ws)×h size full image, and set the slice_segment_address of the remaining slices in the same way if the current sub-image includes multiple slices. For example, Figure 8 is a schematic diagram of slice address writing in the sub-image bitstream merging and transcribing of this application. As shown in Figure 8, when each sub-image contains only one slice, the address slice_segment_address of the first coding tree unit is 0. If three sub-images are stitched together into a full image, each sub-image becomes one slice. The addresses of the first coding tree units of the left, middle and right slices in the full image are 0, X1 and X2, respectively. Therefore, the slice_segment_address of these three slices is set to 0, X1 and X2, respectively.

[0180] 4. Merge NAL units and construct a full image access unit. For example, merge the coded slice NAL units in the three sub-image compressed bitstreams, adjust the order of the coded slice NAL units and the non-coded slice NAL units, and construct an access unit (AU) that conforms to the H.265 standard and contains coded slice NAL units of three sub-images (SVAC 7.2.2.2.2.3).

[0181] 5. Write sub-image description information. For example, the three sub-image description information can be encapsulated in the corresponding sub-image compressed bitstream. That is, the Supplemental Enhancement Information (SEI) NAL units containing sub-image segmentation and position information from the three sub-image compressed bitstreams are directly written into the full image compressed bitstream. Alternatively, multiple sub-image description information can be encapsulated in the SEI information encoding unit of the full image compressed bitstream. That is, the sub-image segmentation and position information are written into the full image compressed bitstream in the form of user-defined SEI information NAL units.

[0182] It should be noted that this application does not specify the order of the five steps involved in the merging and transcribing process.

[0183] In one possible implementation, the H.265 standard tile syntax is used to record the sub-image segmentation method and write it to the full image bitstream. That is, the tile syntax element values ​​in the image parameter set are set according to the sub-image segmentation method. Corresponding to the non-overlapping segmentation method shown in Figure 6b, the number of tile columns is set to 3, the number of tile rows is set to 0, the equal segmentation flag is set to 1, the tile width is set to ws (converting ws in pixels to a value in coding tree unit size), and the tile height is set to h (converting ws in pixels to a value in coding tree unit size). Because the H.265 standard tile syntax has already been used to record the sub-image segmentation method, it is optional not to write the sub-image segmentation information.

[0184] In one possible implementation, multiple sub-image compressed bitstreams can be written into the coded data units of multiple image layers respectively; the coded data of the coded data units of multiple image layers can be merged to obtain the full image compressed bitstream.

[0185] This method references the hierarchical coding mechanism specified in the standard, writing multiple sub-images into coded data units of different image layers, and then merging the coded data of these multiple image layer coded data units to obtain a single full-image compressed bitstream. Taking the H.265 standard as an example, this merging and transcribing process includes the following steps:

[0186] 1. Add a Video Parameter Set (VPS) NAL unit conforming to the H.265 standard Multi-layer Profile specification. Set the number of layers, vps_max_layers_minus1, to the number of sub-images minus 1. Set the number of rep_format structures in the VPS parameter set, vps_num_rep_formats_minus1, to the number of sub-images minus 1. Set the image width, pic_width_vps_in_luma_samples, and height, pic_height_vps_in_luma_samples, in each rep_format structure to ws and h, respectively. Set the inter-layer dependency flag, direct_dependency_flag, in the VPS parameter set to 0.

[0187] 2. Rewrite the nuh_layer_id of all NAL cell headers in each sub-image bitstream, setting a different nuh_layer_id for each sub-image. For example, set the nuh_layer_id of the cell headers of all NAL cells in the compressed bitstreams of sub-image 1, sub-image 2, and sub-image 3 to 0, 1, and 2 respectively.

[0188] 3. Rewrite the sequence parameter set NAL unit, image parameter set NAL unit, and coded slice NAL unit in each sub-image bitstream to make them conform to the H.265 standard Multi-layer Profile specification.

[0189] For example, if the nal_layer_id of the sequence parameter set NAL cell is not equal to 0, assign the value of the sps_max_sub_layers_minus1 syntax in the sequence parameter set to the sps_ext_or_max_sub_layers_minus1 syntax. Modify the value of sps_seq_parameter_set_id in the sequence parameter set NAL cell to the value of nuh_layer_id.

[0190] Modify the values ​​of pps_pic_parameter_set_id and pps_seq_parameter_set_id in the NAL unit of the image parameter set to the value of nuh_layer_id.

[0191] Modify the slice_pic_parameter_set_id in the slice_segment_header structure of the NAL unit of the encoded slice to the value of nuh_layer_id.

[0192] 4. Merge NAL units and construct layered access units (AUs). Interleave and merge the non-coded and coded NAL units from the three sub-image compressed bitstreams according to their nuh_layer_id values ​​in ascending order. This constructs an access unit (AU) that conforms to the H.265 standard Multi-layer Profile and contains all NAL units from the three sub-images. For example, interleaving is performed in the following order:

[0193] AU0: VPS NAL unit, SPS NAL unit (nuh_layer_id=0), SPS NAL unit (nuh_layer_id=1), SPS NAL unit (nuh_layer_id=2), PPS NAL unit (nuh_layer_id=0), PPS NAL unit (nuh_layer_id=1), PPS NAL unit (nuh_layer_id=2), coded slice NAL unit (nuh_layer_id=0), coded slice NAL unit (nuh_layer_id=1), coded slice NAL unit (nuh_layer_id=2)

[0194] AU1: NAL unit of coded slice (nuh_layer_id=0), NAL unit of coded slice (nuh_layer_id=1), NAL unit of coded slice (nuh_layer_id=2)

[0195] AU2: NAL unit of coded slice (nuh_layer_id=0), NAL unit of coded slice (nuh_layer_id=1), NAL unit of coded slice (nuh_layer_id=2)

[0196] 5. Write sub-image description information. For example, the description information of three sub-images can be encapsulated in the corresponding sub-image compressed bitstream. That is, the SEI information NAL units containing sub-image segmentation information and position information in the three sub-image compressed bitstreams are directly written into the full image compressed bitstream. Alternatively, the description information of multiple sub-images can be encapsulated in the SEI information encoding unit of the full image compressed bitstream. That is, the sub-image segmentation information and position information are written into the full image compressed bitstream in the form of user-defined SEI information NAL units.

[0197] It should be noted that this application does not specify the order of the five steps involved in the merging and transcribing process.

[0198] In one possible implementation, the compressed bitstream of the first sub-image can be written into the coding data unit of the whole image to obtain the compressed bitstream of the whole image; the compressed bitstreams of the sub-images of the other sub-images besides the first sub-image can be written into the supplementary enhancement information (SEI) information coding unit of the compressed bitstream of the whole image, wherein the first sub-image is one of the multiple sub-images.

[0199] This method selects a sub-image compressed bitstream as the target bitstream, encapsulates the other sub-image compressed bitstreams within user-defined SEI information encoding units, and embeds them into the target bitstream to obtain a single full-image compressed bitstream. Taking the H.265 standard as an example, this merging and transcribing process includes the following steps:

[0200] 1. Select a sub-image bitstream and write it into the encoded data unit of the whole image to obtain the compressed bitstream of the whole image.

[0201] 2. Write the compressed bitstream of other sub-images into the user-defined SEI information encoding unit in the full image compressed bitstream.

[0202] 3. Write sub-image description information. For example, multiple sub-image description information can be encapsulated in the SEI information encoding unit of the full image compressed bitstream. That is, the sub-image segmentation information and position information are written into the full image compressed bitstream in the form of user-defined SEI information NAL units.

[0203] The above method merges and rewrites the compressed bitstreams generated by encoding multiple sub-images into a single full-image compressed bitstream without changing the transmission and storage modules in the existing video system, resulting in lower system complexity and stronger compatibility.

[0204] It should be noted that this application may also use other methods to merge and transcribe multiple sub-image compressed bitstreams, without making specific limitations.

[0205] This application splits the full image into multiple sub-images, then encodes each sub-image independently using multiple encoders, and merges and transcribs the resulting compressed bitstreams of the multiple sub-images into a single compressed bitstream of the full image. The multiple encoders do not need to coordinate with each other, resulting in low system complexity and making it more suitable for use in multi-core architecture systems. Furthermore, the merging and transcribing of the compressed bitstream does not require changes to the transmission and storage modules in the existing video system, resulting in low system-level complexity and stronger compatibility.

[0206] Figure 9 is a flowchart of process 900 of the video decoding method provided in an embodiment of this application. Process 900 can be executed by the decoding system described above. Process 900 is described as a series of steps or operations. It should be understood that process 900 can be executed in various orders and / or occur simultaneously, and is not limited to the execution order shown in Figure 9. Process 900 may include:

[0207] Step 901: Obtain the full image compressed bitstream.

[0208] The full image compression bitstream comes from the output of the encoding system, and its acquisition process can be referred to the embodiment shown in Figure 5.

[0209] Step 902: Split and transcribe the full image compressed bitstream to obtain multiple sub-image compressed bitstreams.

[0210] Based on the encoding system's method of merging and transcribing multiple sub-image compressed bitstreams, the decoding system can use a corresponding split transcribing method to split a single full-image compressed bitstream into multiple sub-image compressed bitstreams.

[0211] In one possible implementation, encoded data is extracted from the full image encoded data units of the full image compressed bitstream in a preset order to obtain multiple sub-image compressed bitstreams.

[0212] This method can refer to the tile / patch syntax mechanism to split and transpose a single full-image compressed bitstream into multiple sub-image bitstreams. This splitting and transposing process includes the following steps:

[0213] 1. Extract sub-image description information from the full image compressed bitstream. For example, three sub-image description information can be extracted from the SEI information encoding unit of the full image compressed bitstream. That is, the user-defined SEI information NAL unit of the full image compressed bitstream contains three sub-image segmentation information and position information, which are then extracted.

[0214] The sub-image description information is used to guide the sub-image stitching operation on the one hand, and to guide other operations in the compressed bitstream splitting and transcribing on the other hand.

[0215] 2. Configure the non-coded slice NAL units of the sub-image compressed bitstream. For example, set a sequence parameter set and an image parameter set for each sub-image compressed bitstream to form a sub-image compressed bitstream that conforms to the standard.

[0216] 3. Set the slice address syntax in the slice header. For example, set the slice_segment_address of the first slice in each sub-image to the first CTU in that sub-image, and set the slice_segment_address of the remaining slices accordingly if the current sub-image includes multiple slices.

[0217] 4. Set the size-related parameters for the sub-image. For example, set the image width and height in the sequence parameter set of the sub-image bitstream to ws and h, respectively. Rewrite the size-related syntax of the tiles in the image parameter set to divide the image into individual tiles.

[0218] 5. Split NAL units and construct sub-image access units (AUs). For example, extract the coded slice NAL units of three sub-images from the full image access unit to construct access units for the three sub-images. The access units for each sub-image conform to the standard specifications.

[0219] In one possible implementation, coded data is extracted from the coded data units of multiple image layers of the full image compressed bitstream to obtain multiple sub-image compressed bitstreams.

[0220] This method references the layered decoding mechanism specified in the standard, splitting a single full-image compressed bitstream into encoded data for three image layers, and then mapping the encoded data of the three image layers to three sub-image compressed bitstreams. Taking the H.265 standard as an example, this splitting and transcribing process includes the following steps:

[0221] 1. Extract sub-image description information from the full image compressed bitstream. For example, the corresponding sub-image description information can be extracted from the sub-image compressed bitstreams of three sub-images respectively. That is, the SEI information NAL units containing sub-image segmentation information and position information in the three sub-image compressed bitstreams can be extracted. Alternatively, the three sub-image description information can be extracted from the SEI information encoding units of the full image compressed bitstream. That is, the user-defined SEI information NAL units of the full image compressed bitstream contain three sub-image segmentation information and position information, which can be extracted.

[0222] The sub-image description information is used to guide the sub-image stitching operation on the one hand, and to guide other operations in the compressed bitstream splitting and transcribing on the other hand.

[0223] 2. Rewrite the VPS parameter set NAL unit, setting the number of layers (vps_max_layers_minus1) to 0 to conform to the H.265 standard Main Profile specification. Set the rewritten VPS parameter set for each sub-image compressed bitstream to form a compressed bitstream that conforms to the standard.

[0224] 3. Based on the nuh_layer_id in the NAL unit header, split the NAL unit and construct sub-image access units (AUs). Extract NAL units with the same nuh_layer_id from the layered access units to construct access units for 3 sub-images.

[0225] 4. Modify the nuh_layer_id of the NAL unit header of each NAL unit in the sub-image access unit to 0.

[0226] 5. Rewrite the sequence parameter set NAL unit to conform to the H.265 standard Main Profile specification.

[0227] In one possible implementation, encoded data is extracted from the full image coding data unit of the full image compressed bitstream to obtain a sub-image compressed bitstream of the third decoded sub-image, where the third decoded sub-image is one of multiple decoded sub-images; encoded data is extracted from the SEI information coding unit of the full image compressed bitstream to obtain sub-image compressed bitstreams of other sub-images among the multiple decoded sub-images besides the third decoded sub-image.

[0228] This method involves a sub-image compressed bitstream within the full image coded data unit of the full image compressed bitstream, while other sub-image compressed bitstreams are contained within the SEI information coded unit of the full image compressed bitstream. Taking the H.265 standard as an example, this split-transfer process includes the following steps:

[0229] 1. Extract sub-image description information from the full image compressed bitstream. For example, three sub-image description information can be extracted from the SEI information encoding unit of the full image compressed bitstream. That is, the user-defined SEI information NAL unit of the full image compressed bitstream contains three sub-image segmentation information and position information, which are then extracted.

[0230] The sub-image description information is used to guide the sub-image stitching operation on the one hand, and to guide other operations in the compressed bitstream splitting and transcribing on the other hand.

[0231] 2. Extract a sub-image bitstream from the full image compressed bitstream. This sub-image bitstream does not contain user-defined SEI information NAL units that are present in other sub-image bitstreams within the full image compressed bitstream.

[0232] 3. Extract user-defined SEI information NAL units containing other sub-image bitstreams from the full image compressed bitstream, extract other sub-image bitstreams from them and output them.

[0233] The above method splits and rewrites a single full-image compressed bitstream into multiple sub-image compressed bitstreams without changing the transmission and storage modules in the existing video system, resulting in lower system complexity and stronger compatibility.

[0234] It should be noted that this application may also use other methods to split and transwrite a single full-image compressed bitstream, and no specific limitations are made in this regard.

[0235] Step 903: Obtain description information for multiple sub-images based on the full image compressed bitstream.

[0236] Optionally, multiple sub-image description information can be encapsulated in the SEI information encoding unit of the full image compressed bitstream.

[0237] Optionally, multiple sub-image description information can be encapsulated in the sub-image compressed bitstream of the corresponding decoded sub-image.

[0238] Referring to the description in step 902, the aforementioned sub-image description information can be extracted from the corresponding position in the full image compressed bitstream.

[0239] In one possible implementation, the sub-image description information includes the segmentation information and position information of the corresponding sub-image. The sub-image segmentation information may include the width and height of the corresponding sub-image, such as ws and h mentioned above. The position information is used to describe the position of the corresponding sub-image in the image to be encoded. For example, the position information may be the index or coordinates of the first CTU in the corresponding sub-image in the whole image, or the position information may be the coordinates of the top-left pixel of the corresponding sub-image in the whole image.

[0240] It should be noted that this application may also describe the size and position of the sub-image in other ways, without making specific limitations.

[0241] In one possible implementation, when adjacent sub-images contain overlapping regions at their adjacent boundaries, the sub-image description information can also be used to describe the size of the overlapping regions of the corresponding sub-images.

[0242] Optionally, the width and height of the overlapping regions of the corresponding sub-images can be directly included in the sub-image segmentation information, such as d1, d2, and h mentioned above.

[0243] Optionally, the size of the overlapping area of ​​sub-images can be determined indirectly through positional information. For example, the coordinates of the upper right pixel of sub-image 1 are (x1, y1), and the coordinates of the upper left pixel of sub-image 2 are (x2, y2). x1 > x2 and y1 = y2, indicating that sub-image 1 and sub-image 2 have an overlapping area. The width of the overlapping area is x1 - x2, and the height is y1.

[0244] It should be noted that this application may also use other methods to describe the size of the overlapping area of ​​the sub-images, without making specific limitations.

[0245] Step 904: Decode the compressed bitstreams of multiple sub-images to obtain multiple decoded sub-images.

[0246] In one possible implementation, referring to the decoding system shown in FIG1b, multiple first decoders (decoder 1, decoder 2, decoder 3) can respectively decode multiple sub-image compressed bitstreams (compressed bitstream 1, compressed bitstream 2, compressed bitstream 3) to obtain multiple decoded sub-images (decoded sub-image 1, decoded sub-image 2, decoded sub-image 3).

[0247] In one possible implementation, referring to the decoding system shown in FIG2, a first decoder (decoder) can decode multiple sub-image compressed bitstreams to obtain a reconstructed decoded full image; the reconstructed decoded full image is then split to obtain multiple decoded sub-images.

[0248] In one possible implementation, referring to the decoding system shown in FIG4a, multiple fourth decoded sub-images can be obtained by decoding multiple sub-image compressed bitstreams (Protocol 2 compressed bitstream 1, Protocol 2 compressed bitstream 2, Protocol 2 compressed bitstream 3) using multiple second decoders (Protocol 2 decoder 1, Protocol 2 decoder 2, Protocol 2 decoder 3); multiple first sub-image compressed bitstreams can be obtained by encoding multiple fourth decoded sub-images using multiple third encoders (Protocol 1 encoder 1, Protocol 1 encoder 2, Protocol 1 encoder 3), with the third encoders and second decoders corresponding to different video encoding protocols; and multiple decoded sub-images (decoded sub-image 1, decoded sub-image 2, decoded sub-image 3) can be obtained by decoding multiple first sub-image compressed bitstreams using multiple third decoders (Protocol 1 decoder 1, Protocol 1 decoder 2, Protocol 1 decoder 3), with the third decoders and third encoders corresponding to the same video encoding protocol.

[0249] In one possible implementation, referring to the decoding system shown in FIG4b, multiple fourth decoded sub-images can be obtained by decoding multiple sub-image compressed bitstreams (Protocol 2 compressed bitstream 1, Protocol 2 compressed bitstream 2, Protocol 2 compressed bitstream 3) using multiple second decoders (Protocol 2 decoder 1, Protocol 2 decoder 2, Protocol 2 decoder 3); multiple first sub-image compressed bitstreams (Protocol 1 compressed bitstream 1, Protocol 1 compressed bitstream 2, Protocol 1 compressed bitstream 3) can be obtained by encoding multiple fourth decoded sub-images using multiple third encoders (Protocol 1 encoder 1, Protocol 1 encoder 2, Protocol 1 encoder 3), where the third encoders and second decoders correspond to different video encoding protocols; multiple first sub-image compressed bitstreams are merged and transcribed to obtain a first full-image compressed bitstream; and multiple decoded sub-images are obtained by decoding the first full-image compressed bitstream using a third decoder (Protocol 1 decoder), where the third decoder and third encoder correspond to the same video encoding protocol.

[0250] In one possible implementation, when decoding multiple sub-images using inter-frame predictive coding, the search region of the reference block for any one sub-image is smaller than that of the reference sub-image. This search process can be termed motion-constrained sub-image coding.

[0251] For example, the embodiment shown in Figure 7 can be referred to, which will not be described in detail here.

[0252] Step 905: Based on the description information of multiple sub-images, stitch together multiple decoded sub-images to obtain the decoded full image.

[0253] In one possible implementation, the first decoded sub-image and the second decoded sub-image can be concatenated based on the sub-image description information of the first decoded sub-image and the second decoded sub-image. The first decoded sub-image and the second decoded sub-image are two of a plurality of decoded sub-images and are adjacent to each other. Then, a preset region at the adjacent boundary of the first decoded sub-image and the second decoded sub-image is filtered to obtain the decoded full image.

[0254] In this application, the filtering process may include the following processing methods:

[0255] 1. When the first decoded sub-image and the second decoded sub-image contain overlapping sub-image regions at adjacent boundaries, the overlapping sub-image regions of the first decoded sub-image and the second decoded sub-image are weighted to obtain the decoded full image. The corresponding pixels in the overlapping sub-image regions of the first decoded sub-image and the second decoded sub-image are the same.

[0256] Corresponding to the overlapping segmentation method shown in Figure 6a, the first decoded sub-image and the second decoded sub-image each contain overlapping sub-image regions at adjacent boundaries. Therefore, the decoding system can perform weighted processing on the overlapping sub-image regions of the two, which can be weighted addition, weighted averaging, etc., without specific limitations.

[0257] For example, Figure 10a is a schematic diagram of the weighted average of pixel values ​​in the overlapping region of this application. As shown in Figure 10a, the pixel values ​​in the overlapping region of decoded sub-image 1 and the pixel values ​​at the same position in the overlapping region of decoded sub-image 2 are weighted and averaged, and the result is placed in the corresponding position of the overlapping region of sub-images 1 and 2 in the decoded full image. The pixel values ​​in the overlapping region of decoded sub-image 2 and the pixel values ​​at the same position in the overlapping region of decoded sub-image 3 are weighted and averaged, and the result is placed in the corresponding position of the overlapping region of sub-images 2 and 3 in the decoded full image. This application can use various weights to perform the weighted average of pixel values ​​in the overlapping region, as long as the sum of the weights of the two pixel values ​​at the same position participating in the weighting is 1. For example, the weights of the two pixel values ​​at the same position participating in the weighting are both set to 0.5, or the weight of each pixel column is set according to the pixel position, giving a larger weight to the pixel column located inside the decoded sub-image.

[0258] 2. When the first decoded sub-image and the second decoded sub-image contain overlapping sub-image regions at adjacent boundaries, the first decoded sub-image overlapping region and / or the second decoded sub-image overlapping region are cropped to obtain the decoded full image, and the corresponding pixels in the first decoded sub-image overlapping region and the second decoded sub-image overlapping region are the same.

[0259] Corresponding to the overlapping segmentation method shown in Figure 6a, the first decoded sub-image and the second decoded sub-image each contain overlapping areas at adjacent boundaries. Therefore, the decoding system can perform cropping (discarding) processing on the overlapping areas of the two sub-images. This cropping processing can be to discard the overlapping area of ​​one of the sub-images, or to discard part of the overlapping area of ​​each of the two adjacent sub-images. No specific limitation is made in this regard.

[0260] For example, Figure 10b is a schematic diagram of pixel value cropping in the overlapping region of this application. As shown in Figure 10b, the right column of pixels in the overlapping region of decoded sub-image 1 is discarded, and the left column of pixels in the overlapping region of decoded sub-image 2 is discarded. The sum of the number of discarded pixel columns is the width d1 of the overlapping region. The right column of pixels in the overlapping region of decoded sub-image 2 is discarded, and the left column of pixels in the overlapping region of decoded sub-image 3 is discarded. The sum of the number of discarded pixel columns is the width d2 of the overlapping region. The cropped decoded sub-image 1, decoded sub-image 2, and decoded sub-image 3 are stitched together horizontally to obtain the decoded full image.

[0261] 3. When neither the first decoded sub-image nor the second decoded sub-image contains overlapping areas at their adjacent boundaries, a smoothing filter is applied to the adjacent boundaries of the first decoded sub-image and the second decoded sub-image to obtain the decoded full image.

[0262] Corresponding to the non-overlapping segmentation method shown in Figure 6b, the first decoded sub-image and the second decoded sub-image do not contain overlapping areas at their adjacent boundaries. Therefore, the decoding system can perform smoothing filtering on the boundary between the two. This smoothing filtering can be boundary smoothing, low-pass filtering, neural network, weighted averaging, etc., without specific limitations.

[0263] For example, Figure 10c is a schematic diagram of the post-processing filtering of this application. As shown in Figure 10c, the right-hand column of pixels at the boundary of decoded sub-image 1 and the left-hand column of pixels at the boundary of decoded sub-image 2 are filtered by a low-pass filter. The right-hand column of pixels at the boundary of decoded sub-image 2 and the left-hand column of pixels at the boundary of decoded sub-image 3 are also filtered by a low-pass filter. The filtered decoded sub-image 1, decoded sub-image 2, and decoded sub-image 3 are then stitched together horizontally to obtain the decoded full image.

[0264] The above filtering processes can eliminate stitching defects between adjacent sub-images and improve image quality. It should be noted that other filtering methods can also be used besides those mentioned above; this application does not specifically limit their application.

[0265] This application splits and transcribes a single full-image compressed bitstream into multiple sub-image compressed bitstreams. Then, multiple decoders independently decode the multiple sub-image compressed bitstreams, and the decoded sub-images are then stitched together to obtain the decoded full image. The multiple decoders do not need to coordinate with each other, which reduces the complexity of the system and makes it more suitable for use in multi-core architecture systems. Furthermore, filtering is performed at the boundary between two adjacent sub-images when stitching them together, which can eliminate stitching defects between adjacent sub-images and improve image quality.

[0266] The technical solutions of the method embodiments shown in Figures 5 and 9 will be described in detail below using several specific examples.

[0267] Based on the encoding system shown in Figure 1a and the decoding system shown in Figure 1b, the following steps are included:

[0268] Encoding system:

[0269] 1. Molecular image segmentation: Overlapping molecular image segmentation

[0270] As shown in Figure 6a, the entire image is horizontally divided into three sub-images, denoted as sub-image 1, sub-image 2, and sub-image 3. Let the width of the entire image be w, the height be h, the width of each sub-image be ws, the width of the overlapping region between sub-images 1 and 2 be d1, and the width of the overlapping region between sub-images 2 and 3 be d2. Therefore, w = 3 × ws - d1 - d2. Since there is no vertical division, the height of each of the three sub-images is equal to the height of the entire image, which is h. If the width of the overlapping region of all sub-images is equal, then d1 = d2.

[0271] The encoding system can write sub-image segmentation information into the sub-image compressed bitstream or into the compressed bitstream of the entire image obtained after merging and transcribing. Optionally, user-defined SEI information NAL units can be constructed to write sub-image segmentation information and position information. Sub-image segmentation information includes the width and height of the sub-image, and the width and height of the overlapping area of ​​the sub-images. Sub-image position information can be the index or coordinate position of the first CTU in the sub-image in the entire image, or the coordinates of the top-left pixel in the sub-image in the entire image.

[0272] 2. Sub-image coding: Motion-constrained sub-image coding

[0273] As shown in Figure 7, the three sub-images (Image 1, Sub-image 2, and Sub-image 3) are fed into three encoders for independent encoding. Taking H.265 as an example, when encoding each sub-image, the coding block can use either intra-frame predictive coding or inter-frame predictive coding mode. When using inter-frame predictive coding mode, because interpolation filtering requires using pixel values ​​outside the reference block for interpolation operations, the search area of ​​the reference block is smaller than that of the reference sub-image, as shown by the dashed line area in Figure 7. The distance between the dashed line area and the boundary of the reference sub-image is related to the interpolation filter used. Based on this, when the coding block performs motion search, the search area of ​​the reference block needs to be constrained within the dashed line area of ​​the reference sub-image.

[0274] The sub-image compressed bitstream output by the encoder conforms to the H.265 standard. In the sequence parameter set of the sub-image bitstream, the image width `pic_width_in_luma_samples` has a value of `ws`, and the image height `pic_height_in_luma_samples` has a value of `h`. When the sub-image is encoded as a P-frame, the output compressed bitstream segments are encapsulated into NAL units of type `TRAIL_N` or `TRAIL_R`. When the sub-image is encoded as an I-frame, the output compressed bitstream segments can be encapsulated into NAL units of type `IDR_W_RADL` or `IDR_N_LP`.

[0275] 3. Sub-image compressed bitstream merging and transcribing: Using the tile / patch syntax mechanism, three sub-image compressed bitstreams are merged and transcribed into a single full-image compressed bitstream.

[0276] The process of merging and transcribing using the tile / patch syntax mechanism, as described in step 504 above, will not be repeated here.

[0277] Decoding system:

[0278] 1. Full image compression bitstream splitting and transcribing

[0279] The process of splitting and transcribing based on the tile / patch syntax mechanism described in step 902 above will not be repeated here.

[0280] 2. Sub-image decoding

[0281] Three standard decoders are used to independently and in parallel decode the three sub-image compressed bitstreams, and three decoded sub-images are output.

[0282] 3. Stitching sub-images

[0283] The splicing of sub-images can be done using the methods shown in Figures 10a-10c, which will not be described in detail here.

[0284] In this embodiment, the decoding system performs filtering at the boundaries of adjacent sub-images, which can eliminate stitching defects between sub-images and improve the image quality of the encoded image. The encoding system and decoding system perform independent and parallel encoding / decoding operations without coordination, resulting in low system complexity and ease of use in multi-core architecture systems. Furthermore, merging and transcribing multiple sub-image compressed bitstreams into a single full-image compressed bitstream does not require changes to the transmission and storage modules in the existing video system, resulting in low system-level complexity.

[0285] Based on the decoding system shown in Figure 2, this embodiment can use the encoding system shown in Figure 1a. Therefore, the steps of the encoding system can be referred to above, and the decoding system includes the following steps:

[0286] 1. Compressed bitstream splitting and transcribing

[0287] Extract the full image / sub-image overlap segmentation information from the full image compressed bitstream.

[0288] 2. Full image decoding

[0289] The decoder outputs three sub-images, which are then stitched together to obtain the decoded full image. This decoded full image differs from the full image of the input encoding system, with a width of 3×ws. This is because the overlapping area of ​​two sub-images in the decoded full image appears twice, as shown in Figure 11 (Figure 11 is a schematic diagram of the decoded full image of this application). The boundaries of decoded sub-image 1 and decoded sub-image 2 each have an overlapping area with a width of d1, and the boundaries of decoded sub-image 2 and decoded sub-image 3 each have an overlapping area with a width of d2.

[0290] 3. Stitching sub-images

[0291] Optionally, the decoder output image is a fully decoded image obtained by stitching together three sub-images from left to right. The decoding system can divide the fully decoded image into decoded sub-image 1, decoded sub-image 2, and decoded sub-image 3. Then, decoded sub-image 1, decoded sub-image 2, and decoded sub-image 3 are stitched together to obtain the final output fully decoded image.

[0292] Optionally, the decoder output image is a fully decoded image obtained by stitching the three sub-images together. There is no need to split and stitch the sub-images together. Instead, filtering is performed directly at the boundaries of the corresponding sub-images in the fully decoded image, as shown in Figure 12 (Figure 12 is a schematic diagram of the sub-image boundary post-processing filtering in this application).

[0293] Based on the decoding system shown in Figure 4b, the decoder outputs a fully decoded image obtained by stitching together the three sub-images from left to right, as shown in Figure 11.

[0294] Optionally, the decoding system can segment the decoded full image into decoded sub-image 1, decoded sub-image 2, and decoded sub-image 3. Then, decoded sub-image 1, decoded sub-image 2, and decoded sub-image 3 are stitched together to obtain the final output decoded full image.

[0295] Optionally, the decoding system does not need to perform sub-image splitting and stitching operations, but instead performs filtering operations directly at the corresponding sub-image boundaries in the decoded full image, as shown in Figure 12.

[0296] Figure 13 is a schematic diagram of the structure of the video encoding device 1300 of this application. As shown in Figure 13, the video encoding device 1300 of this embodiment can be applied to the encoding system described above. The video encoding device 1300 may include: a segmentation module 1301, an encoding module 1302, and a merging and transcribing module 1303.

[0297] The segmentation module 1301 is used to segment the image to be encoded into multiple sub-images; acquire multiple sub-image description information, wherein the multiple sub-image description information corresponds to the multiple sub-images, and the sub-image description information is used to describe the size and position of the corresponding sub-image; the encoding module 1302 is used to encode the multiple sub-images respectively to obtain multiple sub-image compressed bitstreams; the merging and transcribing module 1303 is used to merge and transcribe the multiple sub-image compressed bitstreams to obtain a full image compressed bitstream, wherein the full image compressed bitstream includes the multiple sub-image description information.

[0298] In one possible implementation, any two adjacent sub-images among the plurality of sub-images each contain an overlapping region at their adjacent boundaries, and the corresponding pixel values ​​in the overlapping regions of the two adjacent sub-images are the same.

[0299] In one possible implementation, the sub-image description information is further used to describe the size of the overlapping region of the corresponding sub-images.

[0300] In one possible implementation, the merging and transcribing module 1303 is specifically used to write the multiple sub-image compressed bitstreams into the encoded data unit of the whole image in a preset order to obtain the whole image compressed bitstream.

[0301] In one possible implementation, the merging and transcribing module 1303 is specifically used to write the multiple sub-image compressed bitstreams into the coded data units of multiple image layers respectively; and merge the coded data of the coded data units of the multiple image layers to obtain the full image compressed bitstream.

[0302] In one possible implementation, the merging and transcribing module 1303 is specifically used to write the sub-image compressed bitstream of the first sub-image into the encoding data unit of the whole image to obtain the whole image compressed bitstream; and to write the sub-image compressed bitstreams of the other sub-images among the plurality of sub-images, excluding the first sub-image, into the supplementary enhancement information (SEI) information encoding unit of the whole image compressed bitstream, wherein the first sub-image is one of the plurality of sub-images.

[0303] In one possible implementation, the plurality of sub-image description information is encapsulated in the SEI information encoding unit of the full image compressed bitstream.

[0304] In one possible implementation, the multiple sub-image description information are respectively encapsulated in the sub-image compressed bitstream of the corresponding sub-image.

[0305] In one possible implementation, the sub-image description information includes segmentation information and position information of the corresponding sub-image. The sub-image segmentation information includes the width and height of the corresponding sub-image, and the position information is used to describe the position of the corresponding sub-image in the image to be encoded.

[0306] In one possible implementation, the sub-image segmentation information also includes the width and height of the overlapping region of the corresponding sub-images.

[0307] In one possible implementation, the location information includes the index or position coordinates of the first coding tree unit in the corresponding sub-image in the image to be encoded; or, the location information includes the position coordinates of the top-left vertex pixel of the corresponding sub-image in the image to be encoded.

[0308] In one possible implementation, the encoding module 1302 is specifically used to encode the plurality of sub-images respectively through a plurality of first encoders to obtain the plurality of sub-image compressed bitstreams.

[0309] In one possible implementation, the encoding module 1302 is specifically used to encode the plurality of sub-images respectively using a plurality of second encoders to obtain a plurality of first sub-image compressed bitstreams; to decode the plurality of first sub-image compressed bitstreams respectively using a plurality of second decoders to obtain a plurality of second sub-images, wherein the second encoders and the second decoders correspond to the same video encoding protocol; and to encode the plurality of second sub-images respectively using a plurality of third encoders to obtain the plurality of sub-image compressed bitstreams, wherein the second encoders and the third encoders correspond to different video encoding protocols.

[0310] In one possible implementation, the encoding module 1302 is specifically configured to encode the plurality of sub-images respectively using a second encoder to obtain a first full-image compressed bitstream; split and transcribe the first full-image compressed bitstream to obtain a plurality of first sub-image compressed bitstreams; decode the plurality of first sub-image compressed bitstreams respectively using a plurality of second decoders to obtain a plurality of second sub-images, wherein the second encoder and the second decoder correspond to the same video encoding protocol; and encode the plurality of second sub-images respectively using a plurality of third encoders to obtain the plurality of sub-image compressed bitstreams, wherein the second encoder and the third encoder correspond to different video encoding protocols.

[0311] In one possible implementation, when the encoding of the plurality of sub-images adopts an inter-frame predictive coding mode, the search region of the reference block of any one of the sub-images is smaller than that of the reference sub-image.

[0312] The apparatus in this embodiment can be used to execute the technical solution of the method embodiment shown in FIG5. Its implementation principle and technical effect are similar, and will not be described again here.

[0313] Figure 14 is a schematic diagram of the structure of the video decoding device 1400 of this application. As shown in Figure 14, the video decoding device 1400 of this embodiment can be applied to the decoding system described above. The video decoding device 1400 may include: a split-transfer module 1401, a decoding module 1402, and a splicing module 1403.

[0314] The splitting and transcribing module 1401 is used to acquire the full image compressed bitstream; split and transcribe the full image compressed bitstream to obtain multiple sub-image compressed bitstreams, the multiple sub-image compressed bitstreams corresponding to multiple decoded sub-images; acquire multiple sub-image description information based on the full image compressed bitstream, the multiple sub-image description information corresponding to the multiple decoded sub-images, the sub-image description information being used to describe the size and position of the corresponding decoded sub-image; the decoding module 1402 is used to decode the multiple sub-image compressed bitstreams respectively to obtain the multiple decoded sub-images; the stitching module 1403 is used to stitch the multiple decoded sub-images based on the multiple sub-image description information to obtain the decoded full image.

[0315] In one possible implementation, the stitching module 1403 is specifically used to stitch the first decoded sub-image and the second decoded sub-image according to the sub-image description information of the first decoded sub-image and the second decoded sub-image, wherein the first decoded sub-image and the second decoded sub-image are two of the plurality of decoded sub-images and are adjacent to each other; and to perform filtering processing on a preset area at the adjacent boundary of the first decoded sub-image and the second decoded sub-image to obtain the decoded full image.

[0316] In one possible implementation, the stitching module 1403 is specifically used to perform weighted processing on the overlapping regions of the first and second decoded sub-images at the adjacent boundaries to obtain the decoded full image when the first decoded sub-image and the second decoded sub-image respectively contain overlapping regions of sub-images at the adjacent boundaries. The pixels at corresponding positions in the overlapping regions of the first and second decoded sub-images are the same.

[0317] In one possible implementation, the stitching module 1403 is specifically used to crop the overlapping regions of the first and second decoded sub-images at the adjacent boundaries to obtain the decoded full image when the first decoded sub-image and the second decoded sub-image respectively contain overlapping regions of sub-images. The corresponding pixels in the overlapping regions of the first and second decoded sub-images are the same.

[0318] In one possible implementation, the sub-image description information is further used to describe the size of the overlapping region of the corresponding decoded sub-image.

[0319] In one possible implementation, the stitching module 1403 is specifically used to perform smoothing filtering on the adjacent boundaries of the first decoded sub-image and the second decoded sub-image to obtain the decoded full image when neither the first decoded sub-image nor the second decoded sub-image contains an overlapping area at the adjacent boundary.

[0320] In one possible implementation, the plurality of sub-image description information is encapsulated in the supplementary enhancement information (SEI) information encoding unit of the full image compressed bitstream.

[0321] In one possible implementation, the plurality of sub-image description information are respectively encapsulated in the sub-image compressed bitstream of the corresponding decoded sub-image.

[0322] In one possible implementation, the splitting and transcribing module 1401 is specifically used to extract encoded data from the full image encoded data unit of the full image compressed bitstream in a preset order to obtain the plurality of sub-image compressed bitstreams.

[0323] In one possible implementation, the split transcribing module 1401 is specifically used to extract encoded data from the encoded data units of multiple image layers of the full image compressed bitstream to obtain the multiple sub-image compressed bitstreams.

[0324] In one possible implementation, the split transcribing module 1401 is specifically used to extract encoded data from the full image encoding data unit of the full image compressed bitstream to obtain a sub-image compressed bitstream of a third decoded sub-image, wherein the third decoded sub-image is one of the plurality of decoded sub-images; and to extract encoded data from the SEI information encoding unit of the full image compressed bitstream to obtain sub-image compressed bitstreams of other sub-images among the plurality of decoded sub-images except for the third decoded sub-image.

[0325] In one possible implementation, the split transcribing module 1401 is specifically used to decode the multiple sub-image compressed bitstreams through multiple first decoders to obtain the multiple decoded sub-images.

[0326] In one possible implementation, the decoding module 1402 is specifically used to decode the compressed bitstreams of the plurality of sub-images respectively through a first decoder to obtain a reconstructed decoded full image; and to split the reconstructed decoded full image to obtain the plurality of decoded sub-images.

[0327] In one possible implementation, the decoding module 1402 is specifically used to decode the multiple sub-image compressed bitstreams using multiple second decoders to obtain multiple fourth decoded sub-images; to encode the multiple fourth decoded sub-images using multiple third encoders to obtain multiple first sub-image compressed bitstreams, wherein the third encoders and the second decoders correspond to different video encoding protocols; and to decode the multiple first sub-image compressed bitstreams using multiple third decoders to obtain the multiple decoded sub-images, wherein the third decoders and the third encoders correspond to the same video encoding protocol.

[0328] In one possible implementation, the decoding module 1402 is specifically configured to: decode the plurality of sub-image compressed bitstreams using a plurality of second decoders to obtain a plurality of fourth decoded sub-images; encode the plurality of fourth decoded sub-images using a plurality of third encoders to obtain a plurality of first sub-image compressed bitstreams, wherein the third encoders and the second decoders correspond to different video encoding protocols; merge and transcribe the plurality of first sub-image compressed bitstreams to obtain a first full-image compressed bitstream; and decode the first full-image compressed bitstream using a third decoder to obtain the plurality of decoded sub-images, wherein the third decoder and the third encoder correspond to the same video encoding protocol.

[0329] In one possible implementation, when the decoding of the plurality of sub-image compressed bitstreams adopts an inter-frame prediction decoding mode, the search region of the reference block of any one of the decoded sub-images is smaller than that of the reference decoded sub-image.

[0330] The apparatus in this embodiment can be used to execute the technical solution of the method embodiment shown in FIG9. Its implementation principle and technical effect are similar, and will not be described again here.

[0331] In implementation, each step of the above method embodiments can be completed by integrated logic circuits in the processor hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this application can be directly implemented by a hardware encoding processor, or by a combination of hardware and software modules in the encoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0332] The memory mentioned in the above embodiments can be volatile memory or non-volatile memory, or may include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0333] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0334] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0335] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0336] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0337] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0338] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0339] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A video encoding method, characterized in that, include: The image to be encoded is segmented to obtain multiple sub-images; Obtain multiple sub-image description information, wherein the multiple sub-image description information corresponds to the multiple sub-images, and the sub-image description information is used to describe the size and position of the corresponding sub-image; The multiple sub-images are encoded to obtain multiple sub-image compressed bitstreams; The compressed bitstreams of the multiple sub-images are merged and transcribed to obtain a full image compressed bitstream, which includes the description information of the multiple sub-images.

2. The method according to claim 1, characterized in that, Any two adjacent sub-images in the plurality of sub-images contain overlapping regions at their adjacent boundaries, and the corresponding pixel values ​​in the overlapping regions of the two adjacent sub-images are the same.

3. The method according to claim 2, characterized in that, The sub-image description information is also used to describe the size of the overlapping area of ​​the corresponding sub-images.

4. The method according to any one of claims 1-3, characterized in that, The process of merging and transcribing the multiple sub-image compressed bitstreams to obtain the full image compressed bitstream includes: The compressed bitstreams of the multiple sub-images are written into the encoded data unit of the whole image in a preset order to obtain the compressed bitstream of the whole image.

5. The method according to any one of claims 1-3, characterized in that, The process of merging and transcribing the multiple sub-image compressed bitstreams to obtain the full image compressed bitstream includes: The compressed bitstreams of the multiple sub-images are respectively written into the encoded data units of multiple image layers; The encoded data of the encoded data units of the multiple image layers are merged to obtain the full image compressed bitstream.

6. The method according to any one of claims 1-3, characterized in that, The process of merging and transcribing the multiple sub-image compressed bitstreams to obtain the full image compressed bitstream includes: The compressed bitstream of the first sub-image is written into the encoded data unit of the whole image to obtain the compressed bitstream of the whole image; The sub-image compressed bitstreams of the other sub-images (excluding the first sub-image) are written into the supplementary enhancement information (SEI) information encoding unit of the full image compressed bitstream, wherein the first sub-image is one of the multiple sub-images.

7. The method according to any one of claims 1-6, characterized in that, The multiple sub-image description information is encapsulated in the SEI information encoding unit of the full image compressed bitstream.

8. The method according to any one of claims 1-6, characterized in that, The description information of the multiple sub-images is encapsulated in the sub-image compressed bitstream of the corresponding sub-image.

9. The method according to any one of claims 1-8, characterized in that, The sub-image description information includes the segmentation information and position information of the corresponding sub-image. The sub-image segmentation information includes the width and height of the corresponding sub-image, and the position information is used to describe the position of the corresponding sub-image in the image to be encoded.

10. The method according to claim 9, characterized in that, The sub-image segmentation information also includes the width and height of the overlapping region of the corresponding sub-images.

11. The method according to claim 9 or 10, characterized in that, The location information includes the index or position coordinates of the first coding tree unit in the corresponding sub-image in the image to be encoded; or, the location information includes the position coordinates of the top-left vertex pixel of the corresponding sub-image in the image to be encoded.

12. The method according to any one of claims 1-11, characterized in that, The step of encoding the plurality of sub-images to obtain a plurality of compressed bitstreams of sub-images includes: The multiple sub-images are encoded by multiple first encoders to obtain the compressed bitstream of the multiple sub-images.

13. The method according to any one of claims 1-11, characterized in that, The step of encoding the plurality of sub-images to obtain a plurality of compressed bitstreams of sub-images includes: Multiple compressed bitstreams of the first sub-image are obtained by encoding the multiple sub-images separately using multiple second encoders; Multiple second sub-images are obtained by decoding the multiple first sub-image compressed bitstreams using multiple second decoders, and the second encoder and the second decoder correspond to the same video encoding protocol; The multiple sub-images are encoded by multiple third encoders to obtain the compressed bitstream of the multiple sub-images. The second encoders and the third encoders correspond to different video encoding protocols.

14. The method according to any one of claims 1-11, characterized in that, The step of encoding the plurality of sub-images to obtain a plurality of compressed bitstreams of sub-images includes: A first full-image compressed bitstream is obtained by encoding the plurality of sub-images separately using a second encoder; The first full-image compressed bitstream is split and transcribed to obtain multiple first sub-image compressed bitstreams; Multiple second sub-images are obtained by decoding the multiple first sub-image compressed bitstreams using multiple second decoders, and the second encoder and the second decoder correspond to the same video encoding protocol; The multiple sub-images are encoded by multiple third encoders to obtain the compressed bitstream of the multiple sub-images. The second encoders and the third encoders correspond to different video encoding protocols.

15. The method according to any one of claims 1-14, characterized in that, When the encoding of the plurality of sub-images adopts the inter-frame predictive coding mode, the search region of the reference block of any one of the sub-images is smaller than that of the reference sub-image.

16. A video decoding method, characterized in that, include: Obtain the compressed bitstream of the entire image; The full image compressed bitstream is split and transcribed to obtain multiple sub-image compressed bitstreams, and the multiple sub-image compressed bitstreams correspond to multiple decoded sub-images; Multiple sub-image description information is obtained based on the full image compressed bitstream. The multiple sub-image description information corresponds to the multiple decoded sub-images. The sub-image description information is used to describe the size and position of the corresponding decoded sub-image. The plurality of sub-image compressed bitstreams are decoded respectively to obtain the plurality of decoded sub-images; The decoded full image is obtained by stitching together the multiple decoded sub-images based on the description information of the multiple sub-images.

17. The method according to claim 16, characterized in that, The step of stitching together the multiple decoded sub-images according to the multiple sub-image description information to obtain the decoded full image includes: The first decoded sub-image and the second decoded sub-image are concatenated according to the sub-image description information of the first decoded sub-image and the sub-image description information of the second decoded sub-image. The first decoded sub-image and the second decoded sub-image are two of the plurality of decoded sub-images, and the first decoded sub-image and the second decoded sub-image are adjacent to each other. The decoded full image is obtained by filtering the preset regions at the adjacent boundaries of the first decoded sub-image and the second decoded sub-image.

18. The method according to claim 17, characterized in that, The step of filtering the preset regions at the adjacent boundaries of the first decoded sub-image and the second decoded sub-image to obtain the decoded full image includes: When the first decoded sub-image and the second decoded sub-image respectively contain overlapping sub-image regions at the adjacent boundaries, the overlapping sub-image regions of the first decoded sub-image and the second decoded sub-image are weighted to obtain the decoded full image, wherein the corresponding pixels in the overlapping sub-image regions of the first decoded sub-image and the second decoded sub-image are the same.

19. The method according to claim 17, characterized in that, The step of filtering the preset regions at the adjacent boundaries of the first decoded sub-image and the second decoded sub-image to obtain the decoded full image includes: When the first decoded sub-image and the second decoded sub-image respectively contain overlapping sub-image regions at the adjacent boundaries, the overlapping sub-image regions of the first decoded sub-image and / or the overlapping sub-image regions of the second decoded sub-image are cropped to obtain the decoded full image, wherein the corresponding pixels in the overlapping sub-image regions of the first decoded sub-image and the overlapping sub-image regions of the second decoded sub-image are the same.

20. The method according to claim 18 or 19, characterized in that, The sub-image description information is also used to describe the size of the overlapping region of the corresponding decoded sub-image.

21. The method according to claim 17, characterized in that, The step of filtering the preset regions at the adjacent boundaries of the first decoded sub-image and the second decoded sub-image to obtain the decoded full image includes: When neither the first decoded sub-image nor the second decoded sub-image contains overlapping areas at their adjacent boundaries, a smoothing filter is applied to the adjacent boundaries of the first decoded sub-image and the second decoded sub-image to obtain the decoded full image.

22. The method according to any one of claims 16-21, characterized in that, The multiple sub-image description information is encapsulated in the supplementary enhancement information (SEI) information encoding unit of the full image compressed bitstream.

23. The method of any one of claims 16-21, wherein, The description information of the multiple sub-images is encapsulated in the sub-image compressed bitstream of the corresponding decoded sub-image.

24. The method of any one of claims 16-23, wherein, The process of splitting and transcribing the full image compressed bitstream to obtain multiple sub-image compressed bitstreams includes: The plurality of sub-image compressed bitstreams are obtained by extracting encoded data from the full image encoded data units of the full image compressed bitstream in a preset order.

25. The method according to any one of claims 16-23, characterized in that, The process of splitting and transcribing the full image compressed bitstream to obtain multiple sub-image compressed bitstreams includes: The multiple sub-image compressed bitstreams are obtained by extracting encoded data from the encoded data units of multiple image layers of the full image compressed bitstream.

26. The method according to any one of claims 16-23, characterized in that, The process of splitting and transcribing the full image compressed bitstream to obtain multiple sub-image compressed bitstreams includes: Encoded data is extracted from the full image encoded data unit of the full image compressed bitstream to obtain the sub-image compressed bitstream of the third decoded sub-image, wherein the third decoded sub-image is one of the plurality of decoded sub-images; Encoded data is extracted from the SEI information encoding unit of the full image compressed bitstream to obtain the sub-image compressed bitstreams of the other sub-images among the plurality of decoded sub-images, excluding the third decoded sub-image.

27. The method according to any one of claims 16-26, characterized in that, The step of decoding the compressed bitstreams of the plurality of sub-images to obtain the plurality of decoded sub-images includes: The multiple decoded sub-images are obtained by decoding the multiple sub-image compressed bitstreams using multiple first decoders.

28. The method according to any one of claims 16-26, characterized in that, The step of decoding the compressed bitstreams of the plurality of sub-images to obtain the plurality of decoded sub-images includes: The reconstructed decoded full image is obtained by decoding the compressed bitstreams of the multiple sub-images respectively through a first decoder; The reconstructed decoded full image is split into multiple decoded sub-images.

29. The method according to any one of claims 16-26, characterized in that, The step of decoding the compressed bitstreams of the plurality of sub-images to obtain the plurality of decoded sub-images includes: Multiple fourth decoded sub-images are obtained by decoding the compressed bitstreams of the multiple sub-images using multiple second decoders; Multiple first sub-image compressed bitstreams are obtained by encoding the multiple fourth decoded sub-images using multiple third encoders, wherein the third encoders and the second decoders correspond to different video encoding protocols; The multiple decoded sub-images are obtained by decoding the multiple first sub-image compressed bitstreams using multiple third decoders, and the third decoders and the third encoders correspond to the same video encoding protocol.

30. The method according to any one of claims 16-26, characterized in that, The step of decoding the compressed bitstreams of the plurality of sub-images to obtain the plurality of decoded sub-images includes: Multiple fourth decoded sub-images are obtained by decoding the compressed bitstreams of the multiple sub-images using multiple second decoders; Multiple first sub-image compressed bitstreams are obtained by encoding the multiple fourth decoded sub-images using multiple third encoders, wherein the third encoders and the second decoders correspond to different video encoding protocols; The multiple first sub-image compressed bitstreams are merged and transcribed to obtain a first full-image compressed bitstream; The first full-image compressed bitstream is decoded by a third decoder to obtain the plurality of decoded sub-images. The third decoder and the third encoder correspond to the same video encoding protocol.

31. The method according to any one of claims 16-30, characterized in that, When the decoding of the compressed bitstream of the plurality of sub-images adopts the inter-frame prediction decoding mode, the search area of ​​the reference block of any one of the decoded sub-images is smaller than that of the reference decoded sub-image.

32. A video encoding device, characterized in that, include: The segmentation module is used to segment the image to be encoded into multiple sub-images; and to obtain description information for multiple sub-images, wherein the description information for multiple sub-images corresponds to the multiple sub-images, and the description information for multiple sub-images is used to describe the size and position of the corresponding sub-image. The encoding module is used to encode the multiple sub-images respectively to obtain multiple sub-image compressed bitstreams; The merging and transcribing module is used to merge and transcribe the multiple sub-image compressed bitstreams to obtain a full image compressed bitstream, wherein the full image compressed bitstream includes the description information of the multiple sub-images.

33. The apparatus according to claim 32, characterized in that, Any two adjacent sub-images in the plurality of sub-images contain overlapping regions at their adjacent boundaries, and the corresponding pixel values ​​in the overlapping regions of the two adjacent sub-images are the same.

34. The apparatus according to claim 33, characterized in that, The sub-image description information is also used to describe the size of the overlapping area of ​​the corresponding sub-images.

35. The apparatus according to any one of claims 32-34, characterized in that, The merging and transcribing module is specifically used to write the multiple sub-image compressed bitstreams into the encoded data unit of the whole image in a preset order to obtain the whole image compressed bitstream.

36. The apparatus according to any one of claims 32-34, characterized in that, The merging and transcribing module is specifically used to write the multiple sub-image compressed bitstreams into the encoded data units of multiple image layers respectively; and merge the encoded data of the encoded data units of the multiple image layers to obtain the full image compressed bitstream.

37. The apparatus according to any one of claims 32-34, characterized in that, The merging and transcribing module is specifically used to write the sub-image compressed bitstream of the first sub-image into the encoding data unit of the whole image to obtain the whole image compressed bitstream; and to write the sub-image compressed bitstreams of the other sub-images among the plurality of sub-images (excluding the first sub-image) into the supplementary enhancement information (SEI) information encoding unit of the whole image compressed bitstream, wherein the first sub-image is one of the plurality of sub-images.

38. The apparatus according to any one of claims 32-37, characterized in that, The multiple sub-image description information is encapsulated in the SEI information encoding unit of the full image compressed bitstream.

39. The apparatus according to any one of claims 32-37, characterized in that, The description information of the multiple sub-images is encapsulated in the sub-image compressed bitstream of the corresponding sub-image.

40. The apparatus according to any one of claims 32-39, characterized in that, The sub-image description information includes the segmentation information and position information of the corresponding sub-image. The sub-image segmentation information includes the width and height of the corresponding sub-image, and the position information is used to describe the position of the corresponding sub-image in the image to be encoded.

41. The apparatus according to claim 40, characterized in that, The sub-image segmentation information also includes the width and height of the overlapping region of the corresponding sub-images.

42. The apparatus according to claim 40 or 41, characterized in that, The location information includes the index or position coordinates of the first coding tree unit in the corresponding sub-image in the image to be encoded; or, the location information includes the position coordinates of the top-left vertex pixel of the corresponding sub-image in the image to be encoded.

43. The apparatus according to any one of claims 32-42, characterized in that, The encoding module is specifically used to encode the multiple sub-images using multiple first encoders to obtain the compressed bitstream of the multiple sub-images.

44. The apparatus according to any one of claims 32-42, characterized in that, The encoding module is specifically configured to encode the multiple sub-images using multiple second encoders to obtain multiple first sub-image compressed bitstreams; decode the multiple first sub-image compressed bitstreams using multiple second decoders to obtain multiple second sub-images, wherein the second encoders and second decoders correspond to the same video encoding protocol; and encode the multiple second sub-images using multiple third encoders to obtain the multiple sub-image compressed bitstreams, wherein the second encoders and third encoders correspond to different video encoding protocols.

45. The apparatus according to any one of claims 32-42, characterized in that, The encoding module is specifically used to encode the plurality of sub-images respectively through a second encoder to obtain a first full-image compressed bitstream; and to split and transcribe the first full-image compressed bitstream to obtain a plurality of first sub-image compressed bitstreams; Multiple second sub-images are obtained by decoding the multiple first sub-image compressed bitstreams using multiple second decoders, with the second encoder and the second decoder corresponding to the same video encoding protocol; the multiple sub-images are obtained by encoding the multiple second sub-images using multiple third encoders, with the second encoder and the third encoder corresponding to different video encoding protocols.

46. ​​The apparatus according to any one of claims 32-45, characterized in that, When the encoding of the plurality of sub-images adopts the inter-frame predictive coding mode, the search region of the reference block of any one of the sub-images is smaller than that of the reference sub-image.

47. A video decoding device, characterized in that, include: The split transcription module is used to obtain the compressed bitstream of the entire image; The full image compressed bitstream is split and transcribed to obtain multiple sub-image compressed bitstreams, and the multiple sub-image compressed bitstreams correspond to multiple decoded sub-images; Multiple sub-image description information is obtained based on the full image compressed bitstream. The multiple sub-image description information corresponds to the multiple decoded sub-images. The sub-image description information is used to describe the size and position of the corresponding decoded sub-image. The decoding module is used to decode the compressed bitstreams of the plurality of sub-images respectively to obtain the plurality of decoded sub-images; The stitching module is used to stitch together the multiple decoded sub-images according to the description information of the multiple sub-images to obtain the decoded full image.

48. The apparatus according to claim 47, characterized in that, The stitching module is specifically used to stitch the first decoded sub-image and the second decoded sub-image together according to the sub-image description information of the first decoded sub-image and the sub-image description information of the second decoded sub-image. The first decoded sub-image and the second decoded sub-image are two of the plurality of decoded sub-images and are adjacent to each other. The module also performs filtering processing on a preset area at the adjacent boundary of the first decoded sub-image and the second decoded sub-image to obtain the decoded full image.

49. The apparatus according to claim 48, characterized in that, The stitching module is specifically used to perform weighted processing on the overlapping regions of the first and second decoded sub-images at the adjacent boundaries to obtain the decoded full image when the first decoded sub-image and the second decoded sub-image respectively contain overlapping regions of sub-images. The corresponding pixels in the overlapping regions of the first and second decoded sub-images are the same.

50. The apparatus according to claim 48, characterized in that, The stitching module is specifically used to crop the overlapping areas of the first and second decoded sub-images at their adjacent boundaries to obtain the full decoded image when the first decoded sub-image and the second decoded sub-image respectively contain overlapping areas of sub-images. The corresponding pixels in the overlapping areas of the first and second decoded sub-images are the same.

51. The apparatus according to claim 49 or 50, characterized in that, The sub-image description information is also used to describe the size of the overlapping region of the corresponding decoded sub-image.

52. The apparatus according to claim 48, characterized in that, The stitching module is specifically used to perform smoothing filtering on the adjacent boundaries of the first decoded sub-image and the second decoded sub-image to obtain the decoded full image when neither the first decoded sub-image nor the second decoded sub-image contains an overlapping area at the adjacent boundary.

53. The apparatus according to any one of claims 47-52, characterized in that, The multiple sub-image description information is encapsulated in the supplementary enhancement information (SEI) information encoding unit of the full image compressed bitstream.

54. The apparatus according to any one of claims 47-52, characterized in that, The description information of the multiple sub-images is encapsulated in the sub-image compressed bitstream of the corresponding decoded sub-image.

55. The apparatus according to any one of claims 47-54, characterized in that, The splitting and transcribing module is specifically used to extract encoded data from the full image encoded data unit of the full image compressed bitstream in a preset order to obtain the multiple sub-image compressed bitstreams.

56. The apparatus according to any one of claims 47-54, characterized in that, The split transcribing module is specifically used to extract encoded data from the encoded data units of multiple image layers of the full image compressed bitstream to obtain the multiple sub-image compressed bitstreams.

57. The apparatus according to any one of claims 47-54, characterized in that, The splitting and transcribing module is specifically used to extract encoded data from the full image encoding data unit of the full image compressed bitstream to obtain a sub-image compressed bitstream of the third decoded sub-image, wherein the third decoded sub-image is one of the plurality of decoded sub-images; and to extract encoded data from the SEI information encoding unit of the full image compressed bitstream to obtain sub-image compressed bitstreams of the other sub-images among the plurality of decoded sub-images except for the third decoded sub-image.

58. The apparatus according to any one of claims 47-57, characterized in that, The split-transfer module is specifically used to decode the multiple sub-image compressed bitstreams using multiple first decoders to obtain the multiple decoded sub-images.

59. The apparatus according to any one of claims 47-57, characterized in that, The decoding module is specifically used to decode the compressed bitstreams of the plurality of sub-images respectively through a first decoder to obtain the reconstructed decoded full image; and to split the reconstructed decoded full image to obtain the plurality of decoded sub-images.

60. The apparatus according to any one of claims 47-57, characterized in that, The decoding module is specifically used to decode the multiple sub-image compressed bitstreams using multiple second decoders to obtain multiple fourth decoded sub-images; to encode the multiple fourth decoded sub-images using multiple third encoders to obtain multiple first sub-image compressed bitstreams, wherein the third encoders and the second decoders correspond to different video encoding protocols; and to decode the multiple first sub-image compressed bitstreams using multiple third decoders to obtain the multiple decoded sub-images, wherein the third decoders and the third encoders correspond to the same video encoding protocol.

61. The apparatus according to any one of claims 47-57, characterized in that, The decoding module is specifically used to decode the multiple sub-image compressed bitstreams through multiple second decoders to obtain multiple fourth decoded sub-images; and to encode the multiple fourth decoded sub-images through multiple third encoders to obtain multiple first sub-image compressed bitstreams, wherein the third encoders and the second decoders correspond to different video encoding protocols; The multiple first sub-image compressed bitstreams are merged and transcribed to obtain a first full-image compressed bitstream; The first full-image compressed bitstream is decoded by a third decoder to obtain the plurality of decoded sub-images. The third decoder and the third encoder correspond to the same video encoding protocol.

62. The apparatus according to any one of claims 47-61, characterized in that, When the decoding of the compressed bitstream of the plurality of sub-images adopts the inter-frame prediction decoding mode, the search area of ​​the reference block of any one of the decoded sub-images is smaller than that of the reference decoded sub-image.

63. An encoding system, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-15.

64. A decoding system, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 16-31.

65. A computer-readable storage medium, characterized in that, Includes a computer program, which, when executed on a computer, causes the computer to perform the method of any one of claims 1-31.

66. A computer program product, characterized in that, The computer program product includes computer program code that, when run on a computer, causes the computer to perform the method according to any one of claims 1-31.

Images