Flexible band offset mode in HEVC's adaptive sampling offset

Abstract

This disclosure relates to a flexible band offset mode in sample adaptive offset in HEVC, and to flexible band offset (FBO) devices and methods such as performing sample adaptive offset (SAO) filtering within the encoder and decoder according to the High Efficiency Video Coding (HEVC) standard, as well as similarly configured encoding means. The number of band offset (BO) modes and the number of required offsets are reduced. This invention advantageously provides simpler encoding, reduced temporary buffer size requirements, and small performance gains over existing SAO techniques in the HEVC test model HM5.

Description

[0001] This application is a divisional application of patent application No. 201310005027.5, filed on January 8, 2013, entitled "Flexible Band Offset Mode in Adaptive Sampling Offset in HEVC". More specifically, this divisional application is a further divisional application of the divisional application No. 201810881770.X, filed on January 8, 2013, entitled "Flexible Band Offset Mode in Adaptive Sampling Offset in HEVC".

[0002] Cross-references to related applications

[0003] This application is a non-provisional application filed on January 20, 2012, with U.S. Provisional Patent Application Serial No. 61 / 589,127, the entire contents of which are incorporated herein by reference.

[0004] Statement regarding federally sponsored research or development

[0005] not applicable

[0006] The references to the materials submitted on the compressed disk will be merged.

[0007] not applicable

[0008] Notice of copyright-protected topics

[0009] Certain materials in this patent document are protected by copyright laws in the United States and other countries. The copyright holder does not object to any reproduction of this patent document or patent disclosure as it appears in any document or record publicly available in the United States Patent and Trademark Office, but otherwise reserves all copyright rights. Accordingly, the copyright holder does not waive any right to keep this patent document confidential, including but not limited to the right under 37 CFR § 1.14. Technical Field

[0010] This invention relates generally to video coding and decoding, and more specifically to sample adaptive offset (SAO) within efficient video coding (HEVC) systems. Background Technology

[0011] While scalable across the entire target resolution range, the need for improved video resolution and optimized compression systems for these videos has been constant. Recent efforts by the Joint Collaboration on Video Coding (JCT-VC) of ITU-T and ISO / IEC MPEG are dedicated to supporting ultra-high-definition resolution and mobile applications in the High Efficiency Video Coding (HEVC) standard, as well as reducing the bitrate of compression in H.264 / AVC coding systems.

[0012] Unlike many previous coding systems, HEVC divides frames into Coded Tree Blocks (CTBs), which can be further subdivided into Coded Units (CUs) using methods such as quadtree partitioning. To achieve optimal coding efficiency, CUs can have various sizes depending on the video content. A CU typically includes a luma component (Y) and two chroma components (U and V). The sizes of the U and V components are related to the number of samples and can be the same as or different from the size of the Y component, depending on the video sampling format. CUs can also be further subdivided into Prediction Unit (PU) partitions during intra-frame and inter-frame prediction, and Transform Units (TUs) defined for transform and quantization. A Transform Unit (TU) typically refers to the residual data block from which the transform was applied when generating transform coefficients.

[0013] HEVC introduces new elements on top of current video coding systems, such as H.264 / AVC, and similar codecs. For example, although HEVC still involves motion-compensated inter-frame prediction, transform, and entropy coding, it also employs arithmetic coding or variable-length coding. Additionally, following the deblocking filter is a new Sample Adaptive Offset (SAO) filter. The SAO unit treats the entire frame as a hierarchical quadtree. Quadrants in this quadtree are activated by SAO transmit syntax values, each representing an intensity band of a pixel value called a band offset (BO) or a difference in intensity compared to neighboring pixels called an edge offset (EO). For each type (BO and EO) in the SAO, a transmit offset value called the SAO offset is added to the corresponding pixel.

[0014] In HEVC's HM 5.0 version, there are four (4) EO (edge ​​offset) modes and two (2) BO (with offset) modes for Sample Adaptive Offset (SAO). The EO mode sends four (4) offsets, while the BO mode sends sixteen (16) offsets. Therefore, a temporary buffer is needed at the decoder to store the SAO offsets for each partition until the image (or a portion of the image) is decoded. Summary of the Invention

[0015] This invention addresses an inventive Sample Adaptive Offset (SAO) method within HEVC and similar encoders and / or decoders. Specifically, a Flexible Band Offset (FBO) method for SAO is taught, which reduces the number of band offset (BO) modes from two (2) to one (1), and the number of SAO offsets used for BO from sixteen (16) to four (4). Thus, embodiments of the invention provide four (4) SAO offsets regardless of the SAO mode (EO or BO). Due to the reduction in the maximum number of SAO offsets (from 16 to 4), the temporary buffers required to maintain the SAO parameters at the decoder are reduced by approximately 75%. In response to the inventive approach, simulation results show a small performance gain, particularly for the chroma component.

[0016] Further aspects of the invention will be set forth in the following portions of this specification, wherein the detailed description is intended to fully disclose preferred embodiments of the invention and not to set limitations thereon. Attached Figure Description

[0017] The invention will be more fully understood by referring to the following figures, which are for illustrative purposes only:

[0018] Figure 1 This is a schematic diagram of a video encoder according to an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of a video decoder according to an embodiment of the present invention.

[0020] Figure 3A This is a data graph of the standard SAO BO mode used in the existing HEVC encoding under the test model HM 5.0.

[0021] Figure 3B This is a data graph of SAO using flexible band offset (FBO) SAO filtering according to an embodiment of the present invention.

[0022] Figure 4A and Figure 4B In the encoder ( Figure 4A ) and decoder ( Figure 4B The flowchart below illustrates a flexible band offset (FBO) method for performing sample adaptive offset within an HEVC encoding system according to an embodiment of the present invention. Detailed Implementation

[0023] As described in the following paragraphs, the inventive flexible band offset (FBO) method of the present invention can be implemented in both encoder and decoder devices.

[0024] Figure 1An example embodiment of an encoding device comprising an encoder 10 according to the invention is shown, which is used to perform FBO mode within a sample adaptive offset (SAO) after a deblocking filter.

[0025] The encoder 10 is shown having an encoding element 12, which is executed by a computing unit 46, executed by one or more processors 48 and one or more memory devices 50. It will be appreciated that elements of the invention can be implemented as a program stored on a medium, which can be accessed by a CPU for execution of the encoder and / or decoder.

[0026] In this example, video frame input 14 is shown together with reference frame 16 and frame output 18. Inter-frame prediction 20 is depicted using motion estimation (ME) 22 and motion compensation (MC) 24. Intra-frame prediction 26 is shown together with a switch 25 depicted between inter-frame and intra-frame prediction. A sum junction 28 is shown as the output to forward transform 30, which is performed based on the prediction to generate transform coefficients for the residual data. Quantization of the transform coefficients is performed at quantization stage 32, followed by entropy coding 34. Inverse quantization 36 and inverse transform 38 are shown coupled with summing junction 40.

[0027] The output of summation 40 is the decoded video signal, which is received by deblocking filter 42 (a sample adaptive offset (SAO) filter with inventive FBO mode 44 according to the invention) to produce output 18.

[0028] Figure 2 An example embodiment 70 of a decoder is shown, which is illustrated with a processing block 72 and associated processing components 102. Note that this decoder is essentially... Figure 1 A subset of elements comprising encoder 10 operates on reference frame 74 and outputs a video signal. Decoder block receives encoded video signal 76, which is processed by entropy decoder 78, inverse quantization 80, inverse transform 82, and a summation 84 between the output of inverse transform 82 and selection 92, wherein selection 92 is between inter-frame prediction 86 (shown together with motion compensation 88) and separate intra-frame prediction blocks 90. The output from summation 84 is received by deblocking filter 94, followed by a SAO module with an inventive flexible band offset (FBO) mode 96 according to the invention, to produce video output 98.

[0029] It should be recognized that the decoder can be implemented using a processing unit 100, which includes at least one processing means 102 for executing a program associated with decoding and at least one memory 104. Additionally, it should be noted that elements of the invention can be implemented as a program stored on a medium, wherein the medium can be accessed by the processing means (CPU) 102 for execution.

[0030] It will be appreciated that elements 10 and 70 of the present invention are implemented for execution by processing units 46 and 100, such as in response to programs residing in memories 50 and 104 that can be executed on computer processors (CPUs) 48 and 102. Furthermore, it will be appreciated that elements of the present invention can be implemented as programs stored on a medium accessible to CPUs 48 and / or 102 for execution.

[0031] It should also be recognized that the above-described program can be executed from a tangible (physical) computer-readable medium that is non-transitory, because it not only constitutes a transient propagation signal but is also practically capable of retaining the program, such as in any ideal form and number of static or dynamic memory devices. These memory devices need not be implemented to retain data under all conditions (e.g., power outages) as considered non-transitory media herein.

[0032] Figure 3A and Figure 3B The current HEVC SAO filter operation ( Figure 3A This is compared to the inventive flexible band-offset SAO filter of this invention. Figure 3A The diagram illustrates two SAO types based on band offsets (BOs), where the first type (first group BO_0) has a middle band, and the second type (second group BO_1) has side bands visible at each end of the band structure. To reduce the side information to the 16 offsets of a conventional SAO, the initial 32 uniform bands are divided into two groups as shown in the diagram, with the middle 16 bands assigned to group 1 and the 16 bands on each side assigned to group 2. Therefore, one set of SAO offsets is sent to the middle 16 bands (group 1), and one set of SAO offsets is sent to the outer 16 bands (group 2).

[0033] There has been some discussion about increasing the number of BO modes by varying numbers of offsets. However, this further complicates the design. The present invention addresses the opposite of such offset increases and aims to simplify the design, minimize temporary buffers at the decoder, and unify the number of SAO offsets used for EO and BO modes. The present invention teaches a method that utilizes only one BO mode with four (4) SAO offsets, thus unifying the number of SAO offsets across all SAO modes.

[0034] In the apparatus and method of the present invention, only a single band offset type is required. The encoder determines the four (4) consecutive bands from which the SAO offset will be sent. It should be understood that, without departing from the teachings of the present invention, the encoder may select the four consecutive bands based on the maximum distortion effect, or may employ other desired criteria. It is assumed that the other bands have zero SAO offsets. Since there are 32 possible BO bands in the current design, the first band (i.e., first_band) from which the SAO offset is sent is between 0 and 31. Thus, the encoder uses a 5-bit fixed-length (FL) code to indicate the first_band to the decoder. These offsets are indicated in the first non-zero offset band. Figure 3B The diagram is depicted in which the four SAO offsets, shown in dashed lines, begin from the first non-zero offset band.

[0035] After the first non-zero offset band, in response to adding offset i to the first band and using the number N based on the BO bands. Bands The modulo remainder (e.g., in This determines four consecutive bands with SAO offset. The number of bands, N, is exemplified and without limitation. Bands The number is 32. In this way, the number of possible offsets for the remaining band is essentially reduced, for example, from sixteen (16) SAO offsets to four (4) SAO offsets in the example above.

[0036] Figure 4A This is a summary of an example embodiment of a flexible band offset method for SAO filters operating in an encoder. Four consecutive bands with SAO offsets are determined (selected) in step 110, while in step 112 it is assumed that the remaining bands have zero SAO offsets. Next, a fixed-length code is encoded in step 114 to indicate the exact location of the first band to the decoder. The locations of the consecutive bands with SAO offsets are indicated, and each band is encoded with its SAO offset according to step 116. As a result of the invention, the number of BO modes is reduced from two to one, and the number of SAO offsets required for BO is reduced from sixteen to four.

[0037] Figure 4BThis is a summary of an example embodiment of the flexible band offset method for a SAO filter operating in the decoder. In step 130, a fixed-length code received from the encoder is decoded to indicate a first non-zero offset band. In step 132, SAO offsets for four consecutive bands are decoded. Then, in step 134, in response to adding a SAO offset containing offset i (preferably, values ​​from 0 to 3 may be assumed only) to the first band and determining the positions of the four consecutive bands using a modulus based on the number of BO bands, the decoded SAO offsets are added to the corresponding bands. In step 136, the remaining bands, excluding the four consecutive bands, are added with a zero SAO offset.

[0038] Embodiments of the present invention may be described with reference to the following: flowcharts of methods and systems according to embodiments of the present invention, and / or algorithms, formulas, or other computational descriptions that may also be implemented as computer program products. In this regard, each block or step of the flowchart, and combinations of blocks (and / or steps) in the flowchart, algorithms, formulas, or computational descriptions may be implemented by various means, such as hardware, firmware, and / or software including one or more computer program instructions implemented in computer-readable program code logic. It will be appreciated that any such computer program instructions may be loaded into a computer, including but not limited to a general-purpose computer or a special-purpose computer, or other programmable processing apparatus that produces a machine, such that the computer program instructions, which execute on the computer or other programmable processing apparatus, create components for implementing the functions specified in the blocks (multiple blocks) of the flowchart (multiple flowcharts).

[0039] Therefore, the blocks, algorithms, formulas, or calculations depicted in a flowchart support combinations of components for performing a specific function, combinations of steps for performing a specific function, and computer program instructions such as those implemented in computer-readable program code logic components for performing a specific function. It will also be understood that each block, algorithm, formula, or calculation depiction of the flowcharts described herein, and combinations thereof, can be implemented by a dedicated hardware-based computer system, or a combination of dedicated hardware and computer-readable program code logic components, to perform a specific function or step.

[0040] Furthermore, these computer program instructions (such as those implemented in computer-readable program code logic) may also be stored in a computer-readable storage medium that can instruct a computer or other programmable processing device to operate in a particular manner, thereby causing the instructions stored in the computer-readable storage medium to produce manufactured articles, including indicative components that implement the functions specified in the boxes (multiple boxes) of flowcharts (multiple flowcharts). The computer program instructions may also be loaded onto a computer or other programmable processing device to cause a series of operational steps to be performed on the computer or other programmable processing device to produce a computer-implemented process, thereby causing the instructions executing on the computer or other programmable processing device to provide steps for implementing the functions specified in the boxes (multiple boxes) of flowcharts (multiple flowcharts), algorithms (multiple algorithms), formulas (multiple formulas), or computational descriptions (multiple computational descriptions).

[0041] From the above discussion, it will be apparent that the present invention can be implemented in various ways, including the following:

[0042] 1. An apparatus for transmitting a Sample Adaptive Offset (SAO) signal during entropy encoding and decoding of video, comprising: (a) a video encoder having a computer processor; and (b) a program executable on the computer processor of the video encoder for: (i) receiving a decoded video signal from a deblocking filter; and (ii) setting a fixed-length code within the encoder for transmission to a decoder in response to the steps of: (A) selecting four consecutive bands, each band having an SAO offset to be transmitted; (B) assuming the remaining bands have zero SAO offset; (C) encoding a fixed-length code indicating the degree of offset for a first non-zero band offset (BO) of the four consecutive bands; and (D) adding the offset to the first non-zero band offset (BO) and using a modulus based on the number of BO bands. (c) A video decoder having a computer processor; (b) a program executable on the computer processor of the video decoder for: (i) performing sample adaptive offset filtering on the encoded video signal based on receiving the fixed-length code within the video decoder in response to the following steps, the steps including: (A) determining the position of the first band based on decoding the fixed-length code; (B) decoding the four SAO offsets of the four consecutive bands determined during encoding; (C) determining the position of the remaining bands in response to adding the offset to the first band and using a modulo remainder based on the number of BO bands, and adding the decoded SAO offset to the remaining bands; and (D) adding a zero SAO offset to all the remaining bands.

[0043] 2. The device of any of the foregoing embodiments, wherein the system for video encoding and decoding operates according to the High Efficiency Video Coding (HEVC) standard.

[0044] 3. The device of any of the foregoing embodiments, wherein the system requires only one type with offset (BO).

[0045] 4. The device of any of the foregoing embodiments, wherein the number of SAO offsets for the BO band is reduced from 16 to 4.

[0046] 5. The device of any of the foregoing embodiments, wherein the reduction of the possible SAO offsets from 16 to 4 results in a reduction of the required temporary buffer by approximately 75%.

[0047] 6. The device of any of the foregoing embodiments, wherein a program executable on the video encoder computer processor is configured to send thirty-two possible BO tapes for the first non-zero tape.

[0048] 7. The device of any of the foregoing embodiments, wherein a program executable on a computer processor of the video encoder is configured to add an offset i to the first non-zero band and use a number based on the BO band. The modulus is used to generate the positions for the four continuous bands; this modulus is... ,in .

[0049] 8. An apparatus for performing sampled adaptive offset filtering during video entropy encoding, comprising: (a) a video encoder having a computer processor; and (b) a program executable on the computer processor of the video encoder for: (i) receiving a decoded video signal from a deblocking filter; and (ii) setting a fixed-length code within the encoder for transmission to a decoder in response to the steps of: (A) selecting four consecutive bands, each of which will have its SAO offset transmitted; (B) assuming the remaining bands have zero SAO offset; (C) encoding a fixed-length code indicating the degree of offset for a first non-zero band offset (BO); and (D) generating positions for the four consecutive bands by adding the offset to the first non-zero band offset (BO) and using a modulo remainder based on the number of BO bands, and encoding the SAO offset for each band.

[0050] 9. The device of any of the foregoing embodiments, wherein the program executable on the computer processor of the video encoder utilizes a single band offset (BO) type.

[0051] 10. The device of any of the foregoing embodiments, wherein a program executable on a computer processor of the video encoder generates four possible SAO offsets for the continuous band, which is a reduction compared to the 16 possible SAO offsets on a conventional HEVC device.

[0052] 11. The device of any of the foregoing embodiments, wherein the reduction in SAO offset results in a reduction of the required temporary buffer by approximately 75%.

[0053] 12. The device of any of the foregoing embodiments, wherein the program executable on the computer processor of the video encoder is configured to send thirty-two possible BO tapes for the first non-zero tape.

[0054] 13. The device of any of the foregoing embodiments, wherein the program executable on the computer processor of the video encoder determines the number of BO-based bands by adding an offset i to the first non-zero band. The modulus remainder is used to generate the positions for the four continuous bands; this modulus remainder is... ,in .

[0055] 14. An apparatus for performing sample adaptive offset filtering during video entropy decoding, comprising: (a) a video decoder having a computer processor; and (b) a program executable on the computer processor of the video decoder for: (i) performing sample adaptive offset filtering on video from a received encoded video signal containing a fixed-length code within the video decoder, having the steps of: (A) determining the position of a first non-zero band offset (BO) based on decoding the fixed-length code; (B) decoding four SAO offsets of four consecutive bands determined during encoding; (C) in response to adding an offset to the first non-zero band and determining the position of the remaining bands using a modulo remainder based on the number of BO bands, and adding the decoded SAO offsets to the remaining bands; and (D) adding a zero SAO offset to all remaining bands.

[0056] 15. The device of any of the foregoing embodiments, wherein the program execution, which can be executed on the computer processor of the video decoder, requires only a single sample adaptive offset filter of the offset (BO) type.

[0057] 16. The device of any of the foregoing embodiments, wherein a program executable on the computer processor of the video decoder utilizes four possible SAO offsets of the consecutive bands, which is a reduction compared to the 16 possible SAO offsets on a conventional HEVC device.

[0058] 17. The device of any of the foregoing embodiments, wherein the reduction of the possible SAO offsets from 16 to 4 results in a reduction of the required temporary buffer by approximately 75%.

[0059] 18. The device of any of the foregoing embodiments, wherein the program executable on the computer processor of the video decoder is configured to utilize thirty-two possible BO bands for the first non-zero band.

[0060] 19. The device of any of the foregoing embodiments, wherein the program executable on the computer processor of the video decoder determines the number of BO-based bands by adding an offset i to the first non-zero band. The modulo remainder is used to decode the position information of the encoded video signal from the remaining band; this modulo remainder is... ,in .

[0061] 20. The device of any of the foregoing embodiments, wherein the device for sampling adaptive offset filtering during video entropy decoding operates according to the High Efficiency Video Coding (HEVC) standard.

[0062] While the foregoing description contains numerous details, these should not be construed as limiting the scope of the invention, but merely as illustrative of certain currently preferred embodiments of the invention. Therefore, it will be appreciated that the scope of the invention fully encompasses other embodiments that may be apparent to those skilled in the art, and that the scope of the invention is limited only by the appended claims, wherein, unless expressly stated otherwise, reference to an element in the singular should not mean "one and only one," but rather "one or more." All structural and functional equivalents of the elements of the foregoing preferred embodiments known to those skilled in the art are expressly incorporated herein by reference and should be encompassed by the claims. Furthermore, the apparatus or method need not solve every problem sought to be addressed by the invention, as such will be encompassed by the claims. Moreover, elements, components, or method steps in this disclosure should not be intended to be disclosed, regardless of whether such elements, components, or method steps are expressly recited in the claims. The claim elements herein are not interpreted according to paragraph 6 of 35 USC 112 unless the element is explicitly stated using the phrase "component for...".

Claims

1. An image processing apparatus for encoding images, the image processing apparatus comprising: The setting unit is configured to, in a band offset mode that applies an offset to each band representing the value range to which a pixel value belongs, select the position of the starting band located at the beginning of a continuous band that is the beginning of four consecutive bands to which an offset is applied, dividing the bands into 32 bands, set an offset for each band of the set continuous band; and The transmitting unit is configured to transmit a bit stream including an offset set by the setting unit.

2. An image processing method for encoding an image, the image processing method comprising: In the offset setting step, in the band offset mode that applies offset to each band representing the value range to which the pixel value belongs, the continuous band including the starting band is set by selecting the position of the starting band located at the beginning of the continuous band of the four consecutive bands that are divided into 32 bands and apply offset to each band of the set continuous band. as well as The transmission step involves transmitting a bit stream including the offset set in the offset setting step.

Images