Entropy-Inspired Directional Filtering for Image Coding

Abstract

Entropy-inspired directional filtering is used to encode image blocks. During encoding, intensity differences are determined for at least some pixels of the image block based on neighboring pixels of corresponding pixels in the at least some pixels. An angle is estimated for each of these pixels based on the intensity difference. A main filtering direction for the image block is then determined based on the estimated angle. The image block is filtered according to the main filtering direction to remove the slanted edges associated with the image block. The filtered image block is then encoded into an encoded image. During decoding, an angle map indicating the angles estimated for the pixels of the encoded image block is received, and the angle map is used to determine the main filtering direction of the image block. The image block can then be filtered according to the main filtering direction and then the filtered image block is output for display or storage.

Description

Background Art

[0001] Image content represents a large amount of online content. A web page may include multiple images, and much of the time and resources spent rendering a web page is dedicated to rendering those images for display. The amount of time and resources required to receive and render an image for display depends in part on the way the image is encoded. Thus, by using encoding and decoding techniques to reduce the overall data size of an image, the image can be rendered faster, and therefore the web page including the image can be rendered faster. Summary of the invention

[0002] This article discloses, among other things, systems and techniques for image coding using entropy-inspired directional filtering.

[0003] A method for encoding an image block according to an implementation of the present disclosure includes initializing the value at the index of an angle candidate array to zero. The number of indices of the array is equal to the number of angle candidates that can be used to encode the image block. Each index in the index corresponds to an angle candidate in the angle candidates. For each current pixel in a plurality of pixels of the image block: determine the intensity difference based on the difference in intensity values ​​of adjacent pixels of the current block; estimate the angle of the current pixel by calculating a function of the intensity difference of the current pixel; and increase the value at the index corresponding to the estimated angle within the angle candidate array. Determine the main filtering direction of the image block based on the angle candidate with the maximum value in the array. Filter the image block according to the main filtering direction to remove one or more inclined edges associated with the image block. Encode the filtered image block into an encoded image.

[0004] A device for encoding an image block according to an implementation of the present invention includes a memory and a processor. The memory stores instructions and an angle candidate array, the number of indexes of the angle candidate array being based on the number of angle candidates that can be used to encode the image block. The processor is configured to execute instructions stored in the memory to initialize the value at the index of the angle candidate array to zero. For each current pixel of a plurality of pixels of the image block, the processor further executes instructions to: determine the intensity difference based on the difference in intensity values ​​of neighboring pixels of the current pixel; estimate the angle of the current pixel by calculating a function of the intensity difference of the current pixel; and increase the value at the index corresponding to the estimated angle within the angle candidate array. The processor further executes instructions to determine the main filtering direction of the image block based on the angle candidate with the maximum value in the array. The processor further executes instructions to filter the image block according to the main filtering direction to remove one or more inclined edges associated with the image block. The processor further executes instructions to encode the filtered image block into an encoded image.

[0005] A method for encoding an image block according to an implementation of the present invention comprises determining corresponding intensity differences of at least some pixels of the image block based on neighboring pixels of corresponding pixels of the at least some pixels. Estimating corresponding angles of each pixel of the at least some pixels based on the corresponding intensity differences. Determining a main filtering direction of the image block based on the estimated angles. Filtering the image block according to the main filtering direction to remove one or more inclined edges associated with the image block. Encoding the filtered image block into an encoded image.

[0006] A method for decoding a coded image block according to an implementation of the present invention includes receiving an angle map indicating angle candidates for pixels of the coded image block. The coded image block is decoded to generate a decoded image block. The value at the index of the angle candidate array is initialized to zero. The number of indexes of the array is equal to the number of angle candidates that can be used to filter the decoded image block. Each index corresponds to an angle candidate. For each current pixel of a plurality of pixels of the decoded image block, the value at the index within the angle candidate array corresponding to the current pixel indicated in the angle map is increased. The main filtering direction of the decoded image block is determined based on the angle candidate with the maximum value in the array. The decoded image block is filtered according to the main filtering direction to remove one or more inclined edges associated with the decoded image block. The filtered image block is output for display or storage.

[0007] According to an implementation of the present invention, a device for decoding a coded image block includes a memory and a processor. The memory stores instructions and an angle candidate array, the number of indexes of the angle candidate array being based on the number of angle candidates that can be used to filter the coded image block. The processor is configured to execute instructions stored in the memory to receive an angle map indicating angle candidates for pixels of the coded image block. The processor further executes instructions to decode the coded image block to generate a decoded image block. The processor further executes instructions to initialize the value at the index of the angle candidate array to zero. For each current pixel of a plurality of pixels of the decoded image block, the processor further executes instructions to increase the value at the index within the angle candidate array corresponding to the current pixel as indicated in the angle map. The processor further executes instructions to determine the main filtering direction of the decoded image block based on the angle candidate with the maximum value in the array. The processor further executes instructions to filter the decoded image block according to the main filtering direction to remove one or more inclined edges associated with the decoded image block. The processor further executes instructions to output the filtered image block for display or storage.

[0008] A method for decoding a coded image block according to an implementation of the present invention includes decoding the coded image block to generate a decoded image block, and receiving an angle map indicating angle candidates of pixels of the coded image block. Determining a main filtering direction of the decoded image block based on the angle map. Filtering the decoded image block according to the main filtering direction to remove one or more inclined edges associated with the decoded image block. Outputting the filtered image block for display or storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The present disclosure can be best understood from the following detailed description when read in conjunction with the accompanying drawings. It is emphasized that, according to common practice, the various features of the drawings are not drawn to scale. Instead, the sizes of the various features are arbitrarily expanded or reduced for clarity.

[0010] Figure 1 is a block diagram of an example of an image coding system.

[0011] Figure 2 is a block diagram of an example of an internal configuration of a computing device that may be used in an image coding system.

[0012] Figure 3 is a block diagram of an example of an image encoder.

[0013] Figure 4 is a block diagram of an example of an image decoder.

[0014] Figure 5 is an illustration of an example of a portion of an image.

[0015] Figure 6 is a flow diagram of an example of a technique for encoding an image block using entropy inspired directional filtering.

[0016] Figure 7 is a flow chart of an example of a technique for decoding a coded image block using entropy inspired directional filtering. DETAILED DESCRIPTION

[0017] Lossy image coding involves reducing the amount of data within an image to be encoded, such as by using quantization. In exchange for a reduction in the bit cost of the resulting encoded image, the image suffers a certain loss in quality. The extent of the quality loss depends largely on the way in which the image data is quantized during encoding. In particular, quantization of the image data may result in discontinuities or artifacts throughout the image. These artifacts may exist along block boundaries, for example due to differences in the encoding of adjacent blocks within the image. However, in some cases, artifacts may exist on oblique lines, such as on non-vertical edges of objects shown within the image itself.

[0018] These artifacts can be reduced by applying filters to the coefficients of the image blocks. For example, filters can be applied to the quantized transform coefficients before they are entropy encoded into the encoded image during encoding. In another example, filters can be applied to those coefficients after the coefficients are entropy decoded, dequantized, and inverse transformed during decoding. Filtering can remove artifacts from an image so that the image can be reproduced in its pre-encoded form. However, conventional filtering methods are typically designed to remove artifacts from block boundaries. As such, conventional filtering methods may not be effective in removing certain artifacts, such as artifacts near the slanted edges of objects within an image.

[0019] A solution to filtering artifacts of slanted edges in an image uses a directional filter of a given tap size to filter along the edge in question. Based on the tap size used, multiple pixels on each side of the edge are compared and used to calculate a filtered value, which may be, for example, a simple average of the corresponding pixel values. However, this method fails to take into account other pixel values ​​within the block, but rather limits its processing to a certain number of pixels directly adjacent to the edge. In many cases, this may limit the efficacy of the filtering itself, for example by ignoring other pixel information within the block.

[0020] The implementation of the present disclosure solves problems such as those using entropy-inspired directional filtering for image coding. During encoding, intensity differences are determined for at least some pixels of an image block based on neighboring pixels of corresponding pixels in the at least some pixels. An angle is estimated based on the intensity difference for each of those pixels. For example, an array of angle candidates can be used. For each of those pixels, it can be determined that the intensity difference corresponds to an angle candidate in the angle candidates, and the value at the index of the angle candidate within the array can be increased.

[0021] A primary filtering direction for the image block is then determined based on the estimated angle. The image block is filtered according to the primary filtering direction to remove one or more inclined edges associated with the image block. The filtered image block is then encoded as an encoded image. During decoding, an angle map indicating the angles estimated for pixels of the encoded image block, such as during encoding, may be received and used to determine the primary filtering direction for the image block. The image block may then be filtered according to the primary filtering direction and then output for display or storage.

[0022] Further details of these techniques are described herein initially with reference to a system in which techniques for image coding using entropy inspired directional filtering may be implemented. Figure 1 is a diagram of an example of an image coding system 100. The image coding system 100 includes a sending station 102, a receiving station 104, and a network 106. The image coding system 100 may be used, for example, to encode and decode images.

[0023] The sending station 102 is a computing device that encodes and transmits an image. Alternatively, the sending station 102 may include two or more distributed devices for encoding and transmitting an image. The receiving station 104 is a computing device that receives and decodes the encoded image. Alternatively, the receiving station 104 may include two or more distributed devices for receiving and decoding the encoded image. Figure 2 Examples of computing devices for implementing one or both of the transmitting station 102 or the receiving station 104 are described.

[0024] The network 106 connects the sending station 102 and the receiving station 104 for encoding, transmitting, receiving and decoding the image. The network 106 can be, for example, the Internet. The network 106 can also be a local area network (LAN), a wide area network (WAN), a virtual private network (VPN), a cellular phone network, or other means for transmitting the image from the sending station 102 to the receiving station 104.

[0025] The implementation of the coding system 100 may be different from the reference Figure 1 In some implementations, the coding system 100 may omit the network 106. In some implementations, the image may be encoded and then stored for later transmission to the receiving station 104 or another device with memory. In some implementations, the receiving station 104 may receive the encoded image (e.g., via the network 106, a computer bus, and / or some communication path) and store the encoded image for later decoding.

[0026] In some implementations, the functionality of the sending station 102 and the receiving station 104 may change based on the specific operations being performed. For example, during an operation for encoding an image, the sending station 102 may be a computing device for uploading an image for encoding to a server, and the receiving station 104 may be a server that receives the image from the sending station 102 and encodes the image for later use (e.g., when rendering a web page). In another example, during an operation for decoding an encoded image, the sending station 102 may be a server that decodes the encoded image, and the receiving station 104 may be a computing device that receives the decoded image from the sending station 102 and renders the decoded image (e.g., as part of a web page).

[0027] Figure 2 It can be used in image encoding and decoding systems (such as Figure 11 is a block diagram of an example of the internal configuration of a computing device 200 used in the image coding system 100 shown in FIG. The computing device 200 may, for example, implement one or both of the sending station 102 or the receiving station 104. The computing device 200 may be in the form of a computing system including a plurality of computing devices, or in the form of a computing device, such as a mobile phone, a tablet computer, a laptop computer, a notebook computer, a desktop computer, etc.

[0028] The processor 202 in the computing device 200 may be a conventional central processing unit. Alternatively, the processor 202 may be one or more devices of another type that are now existing or later developed and are capable of manipulating or processing information. For example, although the disclosed implementations may be practiced with one processor (e.g., processor 202) as shown, advantages in speed and efficiency may be achieved by using more than one processor.

[0029] In one implementation, the memory 204 in the computing device 200 may be a read-only memory (ROM) device or a random access memory (RAM) device. However, other suitable types of storage devices may also be used as the memory 204. The memory 204 may include code and data 206 accessed by the processor 202 using the bus 212. The memory 204 may also include an operating system 208 and an application 210, which includes at least one program that allows the processor 202 to perform the techniques described herein. For example, the application 210 may include applications 1 to N, which further include image encoding and / or decoding software that performs some or all of the techniques described herein. The computing device 200 may also include an auxiliary storage 214, which may be, for example, a memory card used with a mobile computing device. For example, an image may be stored in whole or in part in the auxiliary storage 214 and loaded into the memory 204 as needed for processing.

[0030] The computing device 200 may also include one or more output devices, such as a display 218. In one example, the display 218 may be a touch-sensitive display that combines a display with a touch-sensitive element operable to sense touch input. The display 218 may be coupled to the processor 202 via the bus 212. In addition to or in lieu of the display 218, other output devices may be provided that allow a user to program the computing device 200 or otherwise use the computing device. When the output device is or includes a display, the display may be implemented in various ways, including as a liquid crystal display (LCD), a cathode ray tube (CRT) display, or a light emitting diode (LED) display, such as an organic LED (OLED) display.

[0031] The computing device 200 may also include or be in communication with an image sensing device 220, such as a camera or another image sensing device now existing or later developed, which may sense images, such as images of a user operating the computing device 200. The image sensing device 220 may be positioned so that it is pointed at the user operating the computing device 200. For example, the position and optical axis of the image sensing device 220 may be configured so that the field of view includes an area directly adjacent to the display 218 and from which the display 218 is visible.

[0032] The computing device 200 may also include or be in communication with a sound sensing device 222, such as a microphone or another sound sensing device now existing or later developed, which can sense sounds near the computing device 200. The sound sensing device 222 may be positioned so that it is directed toward a user operating the computing device 200 and may be configured to receive sounds, such as voice or other utterances, uttered by the user while operating the computing device 200.

[0033] The implementation of computing device 200 may differ from that described in detail with respect to Figure 2 Implementations shown and described. In some implementations, the operations of processor 202 may be distributed across multiple machines (where each machine may have one or more processors), which may be coupled directly or across a local area network or other network. In some implementations, memory 204 may be distributed across multiple machines, such as network-based memory or memory in multiple machines that perform operations of computing device 200. In some implementations, bus 212 of computing device 200 may be comprised of multiple buses. In some implementations, auxiliary storage 214 may be directly coupled to other components of computing device 200, or may be accessible via a network, and may include an integrated unit, such as a memory card, or multiple units, such as multiple memory cards.

[0034] Figure 3 is a block diagram of an example of an image encoder 300 . Figure 3 300 is a block diagram of an example of an image encoder 300. The image encoder 300 may be, for example, a transmitting station (eg, Figure 1 The image encoder 300 receives an input image 302 and encodes it to produce an encoded image 304, which can be output to a decoder (e.g., by a receiving station (such as Figure 1 The receiving station 104 is shown to implement or be used for storage.

[0035] The image encoder 300 includes an inverse filtering stage 306, a transform stage 308, a quantization stage 310, and an entropy coding stage 312. The inverse filtering stage 306 performs filtering on the input image 302 to directly or indirectly reduce artifacts resulting from the transform and / or quantization performed by the transform stage 308 and / or the quantization stage 310, respectively, for example by modifying the pixel values ​​of the input image 302 prior to its transformation and quantization. In particular, the filtering stage 306 may perform the same filtering as that performed later in the image decoder (e.g., as described below with reference to FIG. 1 ). Figure 4 The filtering is the opposite of the filtering performed at the image decoder 400 described above, such as to compensate for blurring caused by the filtering of the image decoder. Figure 6 Implementations and examples for filtering image blocks during encoding are described.

[0036] The transform stage 308 transforms the filtered blocks of the input image 302 to the frequency domain. For example, the transform stage 308 may use a discrete cosine transform (DCT) to transform the filtered blocks of the input image 302 from the spatial domain to the frequency domain. Alternatively, the transform stage 308 may use another Fourier-related transform or a discrete Fourier transform to transform the filtered blocks of the input image 302 from the spatial domain to the frequency domain. As another alternative, the transform stage 308 may use another block-based transform to transform the filtered blocks of the input image 302 from the spatial domain to the frequency domain.

[0037] The quantization stage 310 quantizes the transform coefficients produced as output by the transform stage 308. The quantization stage 310 converts the transform coefficients into discrete values, referred to as quantized transform coefficients, using a quantization factor. For example, the transform coefficients may be divided by the quantization factor and truncated.

[0038] The entropy coding stage 312 entropy encodes the quantized transform coefficients output from the quantization stage 310 using a lossless coding technique. For example, the lossless coding technique used by the entropy coding stage 312 to entropy encode the quantized transform coefficients may be or include Huffman coding, arithmetic operations, variable length coding, or another coding technique. An encoded image 304 is generated based on the output of the entropy coding stage 312. The encoded image 304 may be stored at a server (e.g., in a database or similar data storage) for later retrieval and decoding. For example, the encoded image 304 may be an image hosted on a website or provided for display on a web page.

[0039] The implementation of the image encoder 300 may be different from that described in detail. Figure 3In some implementations, the image encoder 300 may be used to encode frames of a video sequence, such as by encoding an input video stream thereof into a bitstream. For example, the input image 302 may be an input frame to be encoded, and the encoded image 304 may be an encoded frame, which may be encoded into a bitstream.

[0040] In such an implementation, the image encoder 300 may include a prediction stage for predicting motion within a frame. For example, the prediction stage may include functionality for generating a prediction residual for a current block of a frame using either inter-frame prediction or intra-frame prediction. In the case of intra-frame prediction, the prediction block may be formed from previously encoded and reconstructed samples in the frame itself. In the case of inter-frame prediction, the prediction block may be formed from samples in one or more previously constructed reference frames. The prediction block may be subtracted from the current block at the prediction stage to generate a prediction residual. The transform stage 306 may then receive and process the prediction residual, for example to generate transform coefficients for the current block.

[0041] In another example of an implementation in which the image encoder 300 may be used to encode frames of a video sequence, the image encoder 300 may include a reconstruction path to ensure that the image encoder 300 and a decoder (e.g., a decoder implemented at a receiving station of an image coding system (such as the receiving station 104 of the image coding system 100)) use the same reference frames to decode the bitstream produced by the image encoder 300 to represent the input video stream.

[0042] For example, the reconstruction path may perform functions similar to those that occur during the decoding process (see below). Figure 4 The invention also provides a similar function to that described in the previous embodiment, including dequantizing the quantized transform coefficients at the dequantization stage and inversely transforming the dequantized transform coefficients at the inverse transform stage to produce a derivative residual. The prediction block predicted at the prediction stage can then be added to the derivative residual to create a reconstructed block.

[0043] In some implementations, the image encoder 300 may omit the filtering stage 306. For example, instead of performing filtering on the input image 302 or image data processed therefrom, the image encoder 300 may estimate the angles used to filter the pixels of the blocks of the input image 302. The estimated angles may be recorded as an angle map of the input image 302. The angle map may be encoded into the encoded image 304, for example, into a header or other portion of the encoded image 304. Alternatively, the angle map may be encoded as a file separate from the encoded image 304, but it may be processed during decoding with the encoded image 304 in order to decode the encoded image 304. In some implementations, the angle map as described above may be generated by the filtering stage 306 or using the output of the filtering stage 306.

[0044] Figure 44 is a block diagram of an example of an image decoder 400. The image decoder 400 may be, for example, a receiving station (eg, Figure 1 The image decoder 400 is implemented at a receiving station 104 of the image coding system 100 shown in FIG. 1 . The image decoder 400 receives the encoded image 402 (e.g., from storage or memory) and decodes it to produce an output image 404, which can be output for display or storage. The output image 404 is perceptually similar to the input image encoded using the encoder (e.g., Figure 3 The input image 302 and image encoder 300 shown in FIG. 4 are the same or similar. However, assuming that the encoding that produces the encoded image 402 is lossy, the output image 404 may appear substantially the same as the input image, but is not necessarily identical to the input image.

[0045] The image decoder 400 includes an entropy decoding stage 406, a dequantization stage 408, an inverse transform stage 410, and a filtering stage 412. The entropy decoding stage 406 entropy decodes the encoded image data from the encoded image 402 using a lossless coding technique. For example, the lossless coding technique used by the entropy decoding stage 406 to entropy decode the encoded image data from the encoded image 402 may be or include Huffman coding, arithmetic coding, variable length coding, or another coding technique.

[0046] The entropy decoding stage 406 entropy decodes the encoded image data to produce quantized transform coefficients. The dequantization stage 408 dequantizes the quantized transform coefficients output from the entropy decoding stage 406, for example, by multiplying the quantized transform coefficients by a quantization factor used to produce the encoded image 402. The inverse transform stage 410 inverse transforms the dequantized transform coefficients, for example, by inverse transforming the dequantized transform coefficients from the frequency domain to the spatial domain.

[0047] The filtering stage 412 performs filtering to remove artifacts resulting from the encoding of the encoded image 402. For example, the filtering stage 412 may filter the coefficients for a block of the encoded image 402 output from the inverse transform stage 410 according to the dominant filtering direction of the block. Figure 7 Implementations and examples are described for filtering blocks of an image during decoding.

[0048] The implementation of the image decoder 400 may be different from that described in detail. Figure 4 In some implementations, the image decoder 400 can be used to decode coded frames of a video sequence, such as by decoding the coded frames into an output video stream from a bitstream. For example, the coded image 402 can be a coded frame received from a bitstream, and the output image 404 can be a decoded frame to be output for display or storage, such as within an output video stream.

[0049] In such an implementation, the image decoder 400 may include a prediction stage for predicting motion within a frame. For example, the prediction stage may include functionality for generating a prediction residual for a current block of a frame based on the output of the inverse transform stage 410 and / or based on the output of the entropy decoding stage using inter-frame prediction or intra-frame prediction. For example, using header information decoded from a bitstream and a prediction residual output from an inverse transform, the prediction stage may create a prediction block identical to a prediction block created at a prediction stage for encoding a frame.

[0050] In the case of intra-frame prediction, the prediction block can be formed from previously decoded and reconstructed samples in the frame itself. In the case of inter-frame prediction, the prediction block can be formed from samples in one or more previously reconstructed reference frames. The prediction block and the prediction residual output from the inverse transform stage 410 can be used to reconstruct the block. The filtering stage 412 can then perform filtering on the reconstructed block. In addition, the reconstructed frame generated based on the reconstruction of its block can be stored as a reference frame for use in reconstructing a frame to be decoded later.

[0051] Figure 5 5 is an illustration of an example of a portion of an image 500. As shown, the image 500 includes four 64×64 blocks 510 in two rows and two columns in a matrix or Cartesian plane. In some implementations, a 64×64 block may be the largest coding unit, N=64. Each 64×64 block may include four 32×32 blocks 520. Each 32×32 block may include four 16×16 blocks 530. Each 16×16 block may include four 8×8 blocks 540. Each of the 8×8 blocks 540 may include four 4×4 blocks 550. Each 4×4 block 550 may include 16 pixels, which may be represented in four rows and four columns in each corresponding block in a Cartesian plane or matrix.

[0052] Pixels may include information representing an image captured in the image 500, such as brightness information, color information, and position information. In some implementations, a block (e.g., a 16×16 pixel block as shown) may include a luma block 560, which may include luma pixels 562; and two chroma blocks 570, 580, such as a U or Cb chroma block 570 and a V or Cr chroma block 580. The chroma blocks 570, 580 may include chroma pixels 590. For example, the luma block 560 may include 16×16 luma pixels 562, and each chroma block 570, 580 may include 8×8 chroma pixels 590, as shown. Although one arrangement of blocks is shown, any arrangement may be used. Although Figure 7N×N blocks are shown, but in some implementations, N×M blocks may be used, where N and M are different numbers. For example, 32×64 blocks, 64×32 blocks, 16×32 blocks, 32×16 blocks, or any other size blocks may be used. In some implementations, N×2N blocks, 2N×N blocks, or combinations thereof may be used.

[0053] In some implementations, coding the image 500 may include ordered block-level coding. Ordered block-level coding may include coding blocks of the image 500 in a sequence, such as a raster scan order, where blocks may be identified and processed starting with a block in the upper left corner of the image 500 or a portion of the image 500 and continuing along rows from left to right and from the top row to the bottom row, thereby identifying each block in turn for processing. For example, the 64×64 block in the top row and left column of the image 500 may be the first block to be coded, and the 64×64 block immediately to the right of the first block may be the second block to be coded. The second row from the top may be the second row to be coded, such that the 64×64 block in the left column of the second row may be coded after the 64×64 block in the rightmost column of the first row.

[0054] In some implementations, coding a block of the image 500 may include using quadtree coding, which may include coding smaller block units within the block in a raster scan order. For example, the 64×64 block shown in the lower left corner of the portion of the image 500 may be coded using quadtree coding, where the upper left 32×32 block may be coded, then the upper right 32×32 block may be coded, then the lower left 32×32 block may be coded, and then the lower right 32×32 block may be coded. Each 32×32 block may be coded using quadtree coding, where the upper left 16×16 block may be coded, then the upper right 16×16 block may be coded, then the lower left 16×16 block may be coded, and then the lower right 16×16 block may be coded.

[0055] Each 16×16 block may be coded using quadtree coding, where the top left 8×8 block may be coded, then the top right 8×8 block may be coded, then the bottom left 8×8 block may be coded, and then the bottom right 8×8 block may be coded. Each 8×8 block may be coded using quadtree coding, where the top left 4×4 block may be coded, then the top right 4×4 block may be coded, then the bottom left 4×4 block may be coded, and then the bottom right 4×4 block may be coded. In some implementations, for 16×16 blocks, the 8×8 block may be omitted, and the 16×16 block may be coded using quadtree coding, where the top left 4×4 block may be coded, and then the other 4×4 blocks in the 16×16 block may be coded in raster scan order.

[0056] In some implementations, encoding the image 500 may include encoding the image 500 by, for example, omitting from the corresponding encoded image an original version of the image (e.g., an input image such as Figure 3 The coding may include reducing spectral redundancy, reducing spatial redundancy, or a combination thereof.

[0057] Reducing spectral redundancy may include using a color model based on a luma component (Y) and two chroma components (U and V or Cb and Cr), which may be referred to as a YUV or YCbCr color model or color space. Using the YUV color model may include using a relatively large amount of information to represent the luma component of a portion of the image 500, and using a relatively small amount of information to represent each corresponding chroma component of the portion of the image 500. For example, a portion of the image 500 may be represented by a high-resolution luma component and two lower-resolution chroma components, wherein the high-resolution luma component may include a 16×16 pixel block, and each lower-resolution chroma component represents the portion of the image as an 8×8 pixel block. A pixel may indicate a value, for example, a value in a range from 0 to 255, and the value may be stored or transmitted using, for example, eight bits. Although the present disclosure is described with reference to the YUV color model, another color model may be used.

[0058] Reducing spatial redundancy may include transforming the block to the frequency domain using, for example, a discrete cosine transform. For example, a unit of an encoder, such as Figure 3 The transform stage 306 shown in FIG. 1 may perform a discrete cosine transform using transform coefficient values ​​based on spatial frequencies.

[0059] Although described herein with reference to a matrix or Cartesian representation of image 500 for clarity, image 500 may be stored, transmitted, processed, or any combination thereof in any data structure such that pixel values ​​may be efficiently represented for image 500. For example, image 500 may be stored, transmitted, processed, or any combination thereof in a two-dimensional data structure such as the matrix shown, or in a one-dimensional data structure such as a vector array.

[0060] Furthermore, although described herein as showing a chroma subsampled image where U and V have half the resolution of Y, the image 500 may have a different configuration for its color channels. For example, still referring to the YUV color space, full resolution may be used for all color channels of the image 500. In another example, a color space other than the YUV color space may be used to represent the resolution of the color channels of the image 500. Implementations of the present disclosure describe filtering that may be used on images of varying color channel resolutions and / or varying color spaces.

[0061] The blocks shown and described with respect to image 500 are square or rectangular. However, objects displayed within image 500 may not be square or rectangular. For example, a generally circular object may be included within or otherwise intersect multiple blocks of image 500. In addition, such non-square or non-rectangular objects will have slanted edges that are not aligned with the boundaries of the blocks of image 500. Encoding blocks that include slanted edges may result in block artifacts generated along the slanted edges. Filtering tools that use pixel information of a block to determine the primary filtering direction of the block can be used to reduce artifacts along the slanted edges.

[0062] Reference now Figure 6 to Figure 7 An image coding technique using entropy-inspired directional filtering is described. Figure 6 is a flow diagram of an example of a technique 600 for encoding an image block using entropy inspired directional filtering. Figure 7 is a flow chart of an example of a technique 700 for decoding an encoded image block using entropy inspired directional filtering.

[0063] One or more of the techniques 600 or 700 may be implemented, for example, as a software program that may be executed by a computing device such as the sending station 102 or the receiving station 104. For example, the software program may include machine-readable instructions that may be stored in a memory (such as the memory 204 or the secondary storage 214) and that, when executed by a processor (such as the processor 202), may cause the computing device to perform one or more of the techniques 600 or 700. One or more of the techniques 600 or 700 may be implemented using dedicated hardware or firmware. As described above, some computing devices may have multiple memories or processors, and the operations described in one or more of the techniques 600 or 700 may be distributed using multiple processors, memories, or both.

[0064] For simplicity of explanation, technology 600 and technology 700 are each depicted and described as a series of steps or operations. However, the steps or operations according to the present disclosure may occur in various orders and / or simultaneously. In addition, other steps or operations not presented and described herein may be used. In addition, not all steps or operations shown are necessary to implement the technology according to the disclosed subject matter.

[0065] First reference Figure 6, a flowchart of an example of a technique 600 for encoding an image block using entropy-inspired directional filtering is shown. An image block is a block of size M×N, where M and N can be the same number or different numbers. For example, an image block can have a size between 2x2 and 32x32. At 602, an angle candidate array is initialized. The angle candidate array includes a number of indices equal to the number of angle candidates that can be used to encode the image block. The angle candidates represent edge angles of varying degrees. Each of the indices of the angle candidate array corresponds to an angle candidate among the angle candidates. For example, considering 64 angle candidates, the array includes 64 indices. Each of the 64 angle candidates will correspond to a different edge angle that can be conceivably represented within the image block. The value at each index of the array is initially set to zero.

[0066] At 604, the intensity difference of pixel I is determined. Pixel I is the current pixel of the image block. For example, according to the scanning order or other processing order, pixel I can be the first pixel of the image block or the last pixel of the image block. The intensity difference is the difference between the intensity value of pixel I and the intensity value of one or more pixels selected from a group of pixels located in the image block that do not include pixel I. For example, the intensity difference of pixel I can be determined based on the difference in intensity values ​​of adjacent pixels of pixel I, for example, a first delta value can be determined as a difference in intensity values ​​between the left adjacent pixel of pixel I and the right adjacent pixel of pixel I. The second delta value can be determined as a difference in intensity values ​​between the upper adjacent pixel of pixel I and the lower adjacent pixel of pixel I, the first delta value representing the change in pixel intensity of pixel I along the X-axis. The second delta value represents the change in pixel intensity of pixel I along the Y-axis.

[0067] The intensity difference of pixel I can be determined based on the first incremental value, the second incremental value, or both. For example, when the intensity difference of pixel I is determined based on the first incremental value, the intensity difference of pixel I can be equal to the difference in intensity values ​​between the left and right adjacent pixels of pixel I. In another example, when the intensity difference of pixel I is determined based on the second incremental value, the intensity difference of pixel I can be equal to the difference in intensity values ​​between the upper and lower adjacent pixels of pixel I. In yet another example in which both the first incremental value and the second incremental value are used to determine the intensity difference of pixel I, the first incremental value and the second incremental value can be combined. For example, the intensity value of pixel I can be determined as the average of the first incremental value and the second incremental value. The average value can be weighted or unweighted.

[0068] At 606, the angle of pixel I is estimated. The angle represents the angle of the edge of the object where pixel I is located. The angle is estimated by calculating a function of the intensity difference of pixel I. The calculated function can be an inverse tangent^2 (squared) function or another suitable function. The estimated angle can correspond to an angle candidate in the angle candidate array.

[0069] At 608, the value at the index within the angle candidate array corresponding to the estimated angle is increased. In the case where the estimated angle corresponds to an angle candidate in the angle candidate array, the index associated with the angle candidate in the array is identified. The value at the index is then increased. The value at the index can be increased by a floating point value, which is calculated as the output of a function of the edge strength applied to pixel I along the estimated angle. The function can be a logarithmic function. For example, the logarithmic function can be expressed as log(x+1), where x represents the angle estimated for pixel I and f represents a floating point value. Alternatively, the function can be an exponential function. For example, the exponential function can be expressed as f=1-exp(-x), where x represents the angle estimated for pixel I and f represents a floating point value. In some implementations, the value of the index can be increased by one or incremented according to a fixed ratio.

[0070] At 610, it is determined whether pixel I is the last pixel in the image block. For example, a scan order or other processing tool can be used to check whether another pixel of the image block is still to be processed. At 612, in response to determining that pixel I is not the last pixel in the image block, the value of I is increased by 1. Then, the technique 600 returns to 604 to repeat the operations at 604, 606, and 608 for the new pixel I.

[0071] At 614, in response to determining that pixel 1 is not the last pixel in the image block, a primary filtering direction of the image block is determined based on the values ​​of the array. The primary filtering direction is determined based on the angle candidate with the maximum value in the array. Thus, determining the primary filtering direction includes identifying the index of the array with the maximum value after the pixels of the image block are incremented, and selecting the angle candidate associated with the identified index as the primary filtering direction.

[0072] At 616, the image block is filtered according to a main filtering direction. Filtering the image block according to the main filtering direction includes using a filter of a filter type to change at least one pixel value along the main filtering direction within the image block. The filter type is one of a directional filter, a semi-directional filter, or a circular filter. The filter type can be selected according to the ratio of the main filtering direction to the average filtering direction of the image block. For example, the average value of the values ​​at one or more indices of the array can be determined. Then, the filter type for filtering along the main filtering direction can be selected based on the ratio of the maximum value to the average value. The average filtering direction represents the following angle, which corresponds to the angle candidate having a value equal to the average value of the value. In the case where no angle candidate has a value equal to the average value of the value, the average filtering direction represents the following angle, which corresponds to the angle candidate having a value closest to the average value of the value.

[0073] The selection of the filter type is based on a measure of confidence in the main filtering direction determined for the image block. That is, the closer the main filtering direction is to the average filtering direction, the greater the possibility that the image block includes edges at multiple different angles, and therefore the lower the confidence in the main filtering direction. On the contrary, the farther the main filtering direction is from the average filtering direction, the greater the possibility that the main filtering direction represents the angle of the main (or in some cases, the only) edge of the image block, and therefore the higher the confidence in the main filtering direction. A threshold value and / or a threshold value range can be established to indicate which filter type to use based on the relationship between the main filtering direction and the average filtering direction. For example, when the relationship between the main filtering direction and the average filtering direction indicates a high confidence, a first filter type can be used, when the relationship between the main filtering direction and the average filtering direction indicates a medium confidence, a second filter type can be used, and when the relationship between the main filtering direction and the average filtering direction indicates a low confidence, a third filter type can be used. Therefore, filtering the image block can include using a filter of the selected filter type. At 618, the filtered image block is encoded into an encoded image.

[0074] In some implementations, for each pixel I, the values ​​at more than one index of the angle candidate array may be increased, e.g., a Gaussian distribution of angle candidates may be determined based on the estimated angle of pixel I, and the index of the array associated with the angle candidate corresponding to the estimated angle of pixel I may be increased by a first increment value (e.g., 50). Each adjacent index of the index incremented by the first increment value may be increased by a second increment value (e.g., 25) that is less than the first increment value. In some such implementations, other adjacent indices (e.g., each adjacent index of the index incremented by the second increment value) may be increased by increasingly smaller increment values ​​according to a Gaussian distribution.

[0075] In some implementations, technique 600 includes generating an angle map. For example, the angle map can be an array, matrix, table, or other variable or object in which positions correspond to pixels of an image block. The estimated angle of each pixel of the image block can be recorded at a corresponding position in the angle map. The angle map can be encoded into the encoded image, such as into a header or other portion of the encoded image. Alternatively, the angle map can be encoded into a file separate from the encoded image, but it can be processed together with the encoded image during decoding in order to decode the encoded image.

[0076] In some implementations, the angle candidate array may not be used. For example, a different structure may be used that can be updated based on the angles estimated for individual pixels or groups of pixels of the image block. For example, the different structure may be a matrix, a table, or another variable or object. In another example, a data structure may not be used to monitor the estimated angles. For example, a heuristic for directional calculations that is not inspired by the entropy of the image block may consider various directionalities of the image block.

[0077] In some implementations, the angle candidate array may be reset after the image block is encoded into the encoded image. For example, after the image block is encoded, the value at the index of the angle candidate array may be reset to zero.

[0078] Next reference Figure 7 , a flow chart showing an example of a technique 700 for decoding a coded image block using entropy inspired directional filtering. At 702, an angle map is received. The angle map can be an array, matrix, table, or other variable or object in which positions correspond to pixels of the image block. The estimated angle of each pixel of the image block can be recorded at a corresponding position in the angle map. The angle map can be encoded into the coded image, such as into a header or other portion of the coded image. Alternatively, the angle map can be encoded into a file separate from the coded image. The angle map indicates the angles estimated for the pixels of the coded image.

[0079] At 704, an image block is decoded from the encoded image. Decoding the image block from the encoded image may include entropy decoding syntax elements from the encoded image to produce quantized transform coefficients, dequantizing the quantized transform coefficients to produce transform coefficients, and inverse transforming the transform coefficients to produce pixel values ​​for the image block.

[0080] At 706, an angle candidate array is initialized. The angle candidate array includes a plurality of indices equal to the number of angle candidates available for encoding the image block. The angle candidates represent edge angles of varying degrees. Each of the indices of the angle candidate array corresponds to one of the angle candidates. For example, in the case of considering 64 angle candidates, the array includes 64 indices. Each of the 64 angle candidates will correspond to a different edge angle that can be conceivably represented within the image block. The value at each index of the array is initially set to zero.

[0081] At 708, an index of an array corresponding to the estimated angle of pixel I of the image block is incremented. Increasing the index of the array includes determining the estimated angle of pixel I based on the angle map. For example, a location within the angle map corresponding to pixel I may be identified. The angle indicated at the location may be identified as the estimated angle of pixel I. Then, a value at the index corresponding to the angle candidate associated with the estimated angle may be incremented.

[0082] At 710, it is determined whether pixel I is the last pixel in the image block. For example, a scan order or other processing tool can be used to check whether another pixel of the image block is still to be processed. At 712, in response to determining that pixel I is not the last pixel in the image block, the value of I is increased by 1. Then, the technique 700 returns to 708 to repeat the operation at 708 for the new pixel I.

[0083] At 714, in response to determining that pixel 1 is not the last pixel in the image block, a primary filtering direction of the image block is determined based on the values ​​of the array. The primary filtering direction is determined based on the angle candidate with the maximum value in the array. Thus, determining the primary filtering direction includes identifying the index of the array with the maximum value after the pixels of the image block are incremented, and selecting the angle candidate associated with the identified index as the primary filtering direction.

[0084] At 716, the image block is filtered according to the main filtering direction. Filtering the image block according to the main filtering direction includes using a filter of a filter type to change at least one pixel value along the main filtering direction within the image block. The filter type is one of a directional filter, a semi-directional filter, or a circular filter. The filter type can be selected according to the ratio of the main filtering direction to the average filtering direction of the image block. For example, the average value of the values ​​at one or more indexes of the array can be determined. Then, the filter type for filtering along the main filtering direction can be selected based on the ratio of the maximum value to the average value. Therefore, filtering the image block can include using a filter of the selected filter type. At 718, the filtered image block is output for display or storage.

[0085] In some implementations, for each pixel I, the values ​​at more than one index of the angle candidate array may be increased, for example, a Gaussian distribution of angle candidates may be determined based on the estimated angle of pixel I, and the index of the array associated with the angle candidate corresponding to the estimated angle of pixel I may be increased by a first increment value (e.g., 50). Each neighboring index of the index increased by the first increment value may be increased by a second increment value (e.g., 25) that is less than the first increment value. In some such implementations, other neighboring indices (e.g., each neighboring index of the index increased by the second increment value) may be increased by increasingly smaller increment values ​​according to a Gaussian distribution.

[0086] In some implementations, the angle candidate array may not be used. For example, the following different structures may be used, which can be updated based on the angles estimated for each pixel or pixel group of the image block. For example, the different structure can be a matrix, a table, or another variable or object. In another example, the data structure may not be used to monitor the estimated angle. For example, the inspiration for directional calculation that is not inspired by the entropy of the image block may consider various directionalities of the image block. In another example, the angle candidate array can be simply omitted. For example, the angle map can be used to identify the main filtering direction of the image block without using an array or other structure for processing or other verification.

[0087] In some implementations, the angle candidate array may be reset after the image block is encoded into the encoded image. For example, after the image block is encoded, the value at the index of the angle candidate array may be reset to zero.

[0088] The encoding and decoding aspects described above illustrate some examples of encoding and decoding techniques and hardware components configured to perform all or part of those examples of encoding and / or decoding techniques. However, it should be understood that, as used in the claims, encoding and decoding can mean encoding, decoding, transformation or another processing or change of data.

[0089] The word "example" is used here to mean used as an example, instance or illustration. Any aspect or design described herein as an "example" is not necessarily to be interpreted as being preferred or advantageous over other aspects or designs. On the contrary, the use of the word "example" is intended to present the concept in a concrete way. As used in this application, the term "or" is intended to represent an inclusive "or" rather than an exclusive "or". That is, unless the context otherwise specifies or clearly indicates otherwise, the expression "X includes A or B" is intended to represent any natural inclusive arrangement thereof. That is, if X includes A; X includes B; or X includes A and B, then "X includes A or B" satisfies in any of the foregoing cases. In addition, unless otherwise specified or the context clearly indicates that a singular form is involved, the articles "one" and "an" used in this application and the appended claims should generally be interpreted as meaning "one or more". In addition, the use of the term "implementation" or the term "an implementation" in this disclosure is not intended to require the same implementation unless so described.

[0090] All or part of the implementation of the present disclosure may take the form of a computer program product accessible from, for example, a computer-usable or computer-readable medium. A computer-usable or computer-readable medium may be, for example, any device capable of tangibly containing, storing, transmitting, or transporting a program for use by or in conjunction with any processor. The medium may be, for example, an electronic, magnetic, optical, electromagnetic, or semiconductor device. Other suitable media are also available.

[0091] The above implementations, examples, and aspects have been described to facilitate easy understanding of the present disclosure, and do not limit the present disclosure. On the contrary, the present disclosure is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims, which is consistent with the broadest interpretation allowed by law, so as to cover all such modifications and equivalent arrangements.

Claims

1. A method for encoding an image block, the method comprising: Initializing values ​​at indices of an array of angle candidates to zero, wherein the number of indices of the array is equal to the number of angle candidates available for encoding the image block, wherein each of the indices corresponds to one of the angle candidates; For each current pixel of the plurality of pixels of the image block: determining an intensity difference based on a difference in intensity values ​​of neighboring pixels of the current pixel; estimating the angle of the current pixel by calculating a function of the intensity difference of the current pixel; and incrementing a value at an index corresponding to the estimated angle within the array of angle candidates; determining a main filtering direction of the image block based on the angle candidate having the maximum value in the array; filtering the image block according to the primary filtering direction to remove one or more artifacts along a slanted edge associated with the image block; and The filtered image blocks are encoded into a coded image.

2. The method according to claim 1, wherein: Determining the intensity difference based on the difference in intensity values ​​of neighboring pixels of the current pixel comprises: determining a first delta value representing a difference in intensity values ​​between a left neighboring pixel of the current pixel and a right neighboring pixel of the current pixel; determining a second delta value representing a difference in intensity values ​​between an upper neighboring pixel of the current pixel and a lower neighboring pixel of the current pixel; and The intensity difference of the current pixel is determined based on the first delta value and the second delta value.

3. The method according to claim 1, wherein: Increasing a value at an index corresponding to the estimated angle within the array of angle candidates comprises: calculating a function of the edge strength of the current pixel along the estimated angle to determine a floating point value of the current pixel; and A value at the index of the angle candidate is incremented based on the floating point value.

4. The method according to claim 3, wherein: The function calculated for the edge strength of the current pixel is one of a logarithmic function or an exponential function.

5. The method according to claim 1, wherein: Increasing the value at the index of the angle candidate within the array for a current pixel comprises: increasing the value at the index of the angle candidate in the array by a first increment value; and The values ​​at the neighboring indices of the index of the angle candidate in the array are increased by a second increasing value, the second increasing value being smaller than the first increasing value.

6. The method according to any one of claims 1 to 5, wherein: Filtering the image block according to the primary filtering direction to remove the one or more artifacts along the slanted edge associated with the image block comprises: determining an average of the values ​​at one or more of the indices of the array; and selecting a filter type based on a ratio of the maximum value to the average value, Therein, the image block is filtered using a filter of the selected filter type.

7. The method according to any one of claims 1 to 5, further comprising: An angle map is generated, the angle map being used to indicate a dominant filtering direction of the image block to a decoder for decoding the encoded image.

8. A device for encoding an image block, the device comprising: a memory storing an array of angle candidates, the array of angle candidates having a number of indices based on a number of angle candidates available for encoding the image block; as well as A processor, the processor being configured to: Initializing the value at the index of the array of angle candidates to zero; For each current pixel of the plurality of pixels of the image block: determining an intensity difference based on a difference in intensity values ​​of neighboring pixels of the current pixel; estimating the angle of the current pixel by calculating a function of the intensity difference of the current pixel; as well as incrementing a value at an index corresponding to the estimated angle within the array of angle candidates; determining a main filtering direction of the image block based on the angle candidate having the maximum value in the array; filtering the image block according to the primary filtering direction to remove one or more artifacts along a slanted edge associated with the image block; as well as The filtered image blocks are encoded into a coded image.

9. The device according to claim 8, wherein: Determining the intensity difference based on the difference in intensity values ​​of neighboring pixels of the current pixel comprises: determining a first delta value representing a difference in intensity values ​​between a left neighboring pixel of the current pixel and a right neighboring pixel of the current pixel; determining a second delta value representing a difference in intensity values ​​between an upper neighboring pixel of the current pixel and a lower neighboring pixel of the current pixel; and The intensity difference of the current pixel is determined based on the first delta value and the second delta value.

10. The device according to claim 8, wherein: Increasing a value at an index corresponding to the estimated angle within the array of angle candidates comprises: calculating a function of the edge strength of the current pixel along the estimated angle to determine a floating point value of the current pixel; and A value at the index of the angle candidate is incremented based on the floating point value.

11. The device according to claim 10, wherein: The function calculated for the edge strength of the current pixel is one of a logarithmic function and an exponential function.

12. The device according to claim 8, wherein: Increasing the value at the index of the angle candidate within the array for a current pixel comprises: increasing the value at the index of the angle candidate in the array by a first increment value; and The values ​​at the neighboring indices of the index of the angle candidate in the array are increased by a second increasing value, the second increasing value being smaller than the first increasing value.

13. The device according to any one of claims 8 to 12, wherein: Filtering the image block according to the primary filtering direction to remove the one or more artifacts along the slanted edge associated with the image block comprises: determining an average of the values ​​at one or more of the indices of the array; and selecting a filter type based on a ratio of the maximum value to the average value, Therein, the image block is filtered using a filter of the selected filter type.

14. The device according to any one of claims 8 to 12, wherein: The processor is configured to: An angle map is generated, the angle map being used to indicate a dominant filtering direction of the image block to a decoder for decoding the encoded image.

15. A method for encoding an image block, the method comprising: determining respective intensity differences of at least some pixels of the image block based on neighboring pixels of the respective pixels of the at least some pixels; estimating a respective angle of each of the at least some pixels based on the respective intensity differences; determining a main filtering direction of the image block based on the estimated angle; filtering the image block according to the primary filtering direction to remove one or more artifacts along a slanted edge associated with the image block; as well as The filtered image blocks are encoded into a coded image.

16. The method according to claim 15, wherein: Determining the intensity difference of a current pixel among the at least some pixels based on neighboring pixels of the current pixel among the at least some pixels comprises: determining a first delta value representing a difference in intensity between a left neighboring pixel of the current pixel and a right neighboring pixel of the current pixel; determining a second delta value representing a difference in intensity between an upper neighboring pixel of the current pixel and a lower neighboring pixel of the current pixel; and The intensity difference of the current pixel is determined based on the first delta value and the second delta value.

17. The method according to claim 15, wherein: The array of angle candidates for encoding the image block has a number of indices based on the number of angle candidates available for encoding the image block, wherein each of the indices corresponds to an angle candidate among the angle candidates, wherein the method further comprises: For the current estimated angle among the estimated angles: determining a current angle candidate among the number of angle candidates corresponding to a currently estimated angle; and Increment the value at the index of the current angle candidate within the array.

18. The method according to claim 17, wherein: Determining a main filtering direction of the image block based on the estimated angle comprises: An angle candidate corresponding to an index of the array having a maximum value among the number of angle candidates is selected as the main filtering direction.

19. The method according to claim 17 or 18, further comprising: Before encoding the image block, initializing the value at the index of the array of angle candidates to zero; as well as After encoding the image block, the value at the index of the array of angle candidates is reset to zero.

20. The method according to claim 18, wherein: Filtering the image block according to the primary filtering direction to remove the one or more artifacts along the slanted edge associated with the image block comprises: determining an average of the values ​​at one or more of the indices of the array of angle candidates; and selecting a filter type based on a ratio of the maximum value to the average value, Therein, the image block is filtered using a filter of the selected filter type.

21. An apparatus for encoding an image block, comprising: a memory storing instructions; as well as A processor configured to execute the instructions to perform the method according to any one of claims 15 to 20.

Images