Non-linear inverse transformation for video compression

Abstract

Decoding using rate distortion optimization including an inverse transform including transform size adaptive directional nonlinear filtering includes generating decoded block data by decoding a current block of a current frame. Decoding the current block comprises: obtaining quantized transform coefficients of the current block from the encoded bitstream; obtaining transform data of the current block from the encoded bitstream, wherein the transform data indicates a transform type and a transform size; obtaining dequantized transform block data by dequantizing the quantized transform coefficient; and obtaining decoded residual block data by inversely transforming the dequantized transform block data from the transform data. Inverse transform of the dequantized transform block data includes: obtaining intermediate decoded block data by combining prediction block data of a current block and decoded residual block data; and obtaining decoded block data by filtering the intermediate decoded block data using a transform size adaptive directional nonlinear filter.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority and benefit to U.S. Provisional Patent Application Serial No. 63 / 531,397, filed August 8, 2023, the entire disclosure of which is hereby incorporated by reference. Background Technology

[0003] Digital images and videos can be used, for example, over the Internet for remote business meetings via video conferencing, high-definition video entertainment, video advertising, or the sharing of user-generated content. Due to the large amounts of data involved in the transmission and processing of image and video data, high-performance compression can facilitate transmission and storage. Therefore, providing high-resolution images and videos for transmission over communication channels with limited bandwidth is advantageous. Summary of the Invention

[0004] This application relates to encoding and decoding image data, video stream data, or both, for transmission, storage, or both. This document discloses aspects of systems, methods, and apparatuses for rate-distortion optimized encoding and decoding, including inverse transforms incorporating transform-sized adaptive directional nonlinear filtering.

[0005] The changes in these and other aspects will be described in more detail below.

[0006] One aspect is a method for rate-distortion optimized coding including an inverse transform incorporating a transform-size adaptive directional nonlinear filter. Rate-distortion optimized coding including an inverse transform incorporating a transform-size adaptive directional nonlinear filter can include generating encoded block data by encoding a current block from a current frame of an input video. Encoding the current block can include: obtaining residual block data indicating the difference between the current block and its predicted block data; and identifying the optimal transform from two or more available transforms for transforming the residual block data, wherein identifying the optimal transform includes identifying the optimal transform type and the optimal transform size. Evaluating the corresponding available transforms can include: obtaining candidate transform block data by transforming the residual block data using the corresponding available transform; obtaining candidate quantized block data by quantizing the candidate transform block data; obtaining candidate dequantized transform block data by dequantizing the candidate quantized block data; and obtaining candidate decoded residual block data by performing an inverse transform on the candidate dequantized transform block data. Performing an inverse transform on the candidate dequantized transform block data may include: obtaining intermediate decoded block data by combining the predicted block data and the candidate decoded residual block data; and obtaining candidate decoded block data by filtering the intermediate decoded block data using an optimal transform-size adaptive directional nonlinear filter. Encoding the current block may include including the candidate quantized block data in the encoded bitstream and outputting the encoded bitstream.

[0007] One aspect is a method for rate-distortion optimized decoding using an inverse transform incorporating a transform-size adaptive directional nonlinear filter. Rate-distortion optimized decoding using an inverse transform incorporating a transform-size adaptive directional nonlinear filter can include generating decoded block data by decoding the current block of the current frame and outputting the decoded block data. Decoding the current block can include: obtaining quantized transform coefficients of the current block from an encoded bitstream; obtaining transform data of the current block from the encoded bitstream, wherein the transform data indicates transform type and transform size; obtaining dequantized transform block data by dequantizing the quantized transform coefficients; and obtaining decoded residual block data by performing an inverse transform on the dequantized transform block data based on the transform data. Performing an inverse transform on the dequantized transform block data can include: obtaining intermediate decoded block data by combining the prediction block data of the current block and the decoded residual block data; and obtaining the decoded block data by filtering the intermediate decoded block data using a transform-size adaptive directional nonlinear filter.

[0008] On one hand, there is a non-transitory computer-readable storage medium storing an encoded bitstream comprising quantized transform coefficients and transform data of a current block from a current frame of a video. The transform data indicates the transform type and transform size for an inverse transform of the dequantized transform block data, corresponding to the dequantization of the quantized transform coefficients to obtain decoded residual block data. The transform data also indicates a transform-size adaptive directional nonlinear filter corresponding to the transform used to obtain the decoded block data, which corresponds to filtering a combination of the decoded residual block data and the predicted block data of the current block using this transform-size adaptive directional nonlinear filter.

[0009] One aspect is an apparatus for rate-distortion optimized coding including an inverse transform comprising a transform-size adaptive directional nonlinear filter. The apparatus includes a non-transitory computer-readable medium and a processor configured to execute instructions stored on the non-transitory computer-readable medium to generate encoded block data. To generate the encoded block data, the processor is configured to execute the instructions to encode a current block from a current frame of an input video. To encode the current block, the processor is configured to execute the instructions to obtain residual block data indicating the difference between the current block and its predicted block data, and to identify an optimal transform from available transforms for transforming the residual block data. To identify the optimal transform, the processor is configured to execute the instructions to evaluate two or more of the available transforms, wherein the optimal transform includes an optimal transform type and an optimal transform size. To evaluate the corresponding available transforms, the processor is configured to execute these instructions to obtain candidate transform block data, wherein, in order to obtain the candidate transform block data, the processor is configured to execute these instructions to transform the residual block data using the corresponding available transforms; obtain candidate quantized block data, wherein, in order to obtain the candidate quantized block data, the processor is configured to execute these instructions to quantize the candidate transform block data; obtain candidate dequantized transform block data, wherein, in order to obtain the candidate dequantized transform block data, the processor is configured to execute these instructions to dequantize the candidate quantized block data; and obtain candidate decoded residual block data, wherein, in order to obtain the candidate decoded residual block data, the processor is configured to execute these instructions to perform an inverse transform on the candidate dequantized transform block data. To perform an inverse transform on the candidate dequantized transform block data, the processor is configured to execute instructions to obtain intermediate decoded block data, wherein, to obtain the intermediate decoded block data, the processor is configured to execute instructions to combine the prediction block data and the candidate decoded residual block data; and to obtain the candidate decoded block data, wherein, to obtain the candidate decoded block data, the processor is configured to execute instructions to filter the intermediate decoded block data using an optimal transform-size adaptive directional nonlinear filter. The processor is configured to include the candidate quantized block data in an encoded bitstream and output the encoded bitstream.

[0010] One aspect is an apparatus for rate-distortion optimized decoding using an inverse transform incorporating transform-sized adaptive directional nonlinear filtering. The apparatus includes a non-transitory computer-readable medium and a processor configured to execute instructions stored on the non-transitory computer-readable medium to generate decoded block data. To generate the decoded block data, the processor is configured to execute instructions stored on the non-transitory computer-readable medium to decode the current block of the current frame and output the decoded block data. To decode the current block, the processor is configured to execute instructions stored on the non-transitory computer-readable medium to obtain quantized transform coefficients of the current block from the encoded bitstream; obtain transform data of the current block from the encoded bitstream, wherein the transform data indicates transform type and transform size; obtain dequantized transform block data, wherein, to obtain the dequantized transform block data, the processor is configured to execute instructions stored on the non-transitory computer-readable medium to dequantize the quantized transform coefficients; and obtain decoded residual block data, wherein, to obtain the decoded residual block data, the processor is configured to execute instructions stored on the non-transitory computer-readable medium to perform an inverse transform on the dequantized transform block data according to the transform data. To perform the inverse transform on the dequantized transform block data, the processor is configured to execute instructions stored on the non-transitory computer-readable medium to obtain intermediate decoded block data. To obtain the intermediate decoded block data, the processor is configured to execute instructions stored on the non-transitory computer-readable medium to combine the prediction block data of the current block and the decoded residual block data. To perform an inverse transform on the dequantized transform block data, the processor is configured to execute instructions stored on the non-transitory computer-readable medium to obtain the decoded block data. To obtain the decoded block data, the processor is configured to execute instructions stored on the non-transitory computer-readable medium to filter the intermediate decoded block data using a transform-size adaptive oriented nonlinear filter. Attached Figure Description

[0011] The description herein is taken with reference to the accompanying drawings, wherein, unless otherwise stated or otherwise made clear from the context, the same reference numerals throughout the various views refer to the same parts.

[0012] Figure 1 This is a diagram of a computing device according to an implementation of the present disclosure.

[0013] Figure 2 It is a diagram of a computing and communication system according to an implementation of this disclosure.

[0014] Figure 3It is a diagram of a video stream used in encoding and decoding according to an implementation of this disclosure.

[0015] Figure 4 This is a block diagram of an encoder according to the implementation of this disclosure.

[0016] Figure 5 This is a block diagram of a decoder implemented according to this disclosure.

[0017] Figure 6 It is a block diagram representing a portion of a frame according to an implementation of this disclosure.

[0018] Figure 7 This is a block diagram of the encoder selected according to the implementation method of this disclosure.

[0019] Figure 8 This is a flowchart illustrating an example of rate-distortion optimized encoding that incorporates an inverse transform of an adaptive directional nonlinear filter, according to an implementation of this disclosure.

[0020] Figure 9 This is a flowchart illustrating an example of rate-distortion optimized decoding using an implementation of this disclosure, including an inverse transform incorporating a transform-sized adaptive directional nonlinear filter.

[0021] Figure 10 This is a block diagram of an encoder that implements transform selection through nonlinear inverse transform according to the implementation of this disclosure.

[0022] Figure 11 This is a block diagram of a decoder that implements transform selection through nonlinear inverse transform according to the implementation of this disclosure. Detailed Implementation

[0023] Image and video compression schemes may include breaking down an image or frame into smaller parts, such as blocks, and generating an output bitstream using techniques to minimize bandwidth utilization of the information included for each block in the output. In some implementations, the information included for each block in the output can be limited by reducing spatial redundancy, reducing temporal redundancy, or a combination thereof. For example, temporal or spatial redundancy can be reduced by predicting a frame or a portion thereof based on information available to both the encoder and decoder and including information representing the difference or residual between the predicted frame and the original frame in the encoded bitstream. The residual information can be further compressed by transforming the residual information into transform coefficients (e.g., energy compression), quantizing the transform coefficients, and entropy-coding the quantized transform coefficients. For example, transforming the residual information may include using block-based transforms such as the Karhunen-Loève transform (KLT), discrete cosine transform (DCT), singular value decomposition transform (SVD), wavelet transform, or asymmetric discrete sine transform (ADST). Other code processing information, such as motion information, can be included in the encoded bitstream. This encoded bitstream may include differential information transmitted based on predictions of the encoded information, which can be subjected to entropy coding to further reduce the corresponding bandwidth utilization. The encoded bitstream can be decoded to reconstruct blocks and the source image from the limited information. In some implementations, the accuracy, efficiency, or both of the code processing of the blocks may be limited.

[0024] To improve code processing efficiency, block-based hybrid video codecs or codecs perform rate-distortion optimization to identify optimized or best coding parameters relative to available coding parameters and resource constraints. This may include identifying the transform optimized by rate-distortion optimization. The reconstructed frame, or a portion thereof, may contain quantization artifacts. The decoder may perform post-processing on the reconstructed frame data to remove or minimize artifacts or other distortions, such as quantization artifacts. In some block-based hybrid video codecs or codecs, rate-distortion optimization omits or excludes filtering, such as post-processing filtering for removing or reducing quantization artifacts. Rate-distortion optimization that omits or excludes filtering (such as post-processing filtering for removing or reducing quantization artifacts) may not accurately identify the best transform (type, size, or both). For example, rate-distortion optimization that omits or excludes filtering (such as post-processing filtering for removing or reducing quantization artifacts) may determine that a relatively large transform corresponds to a relatively large number of quantization artifacts in an image portion that includes edges, and may use a relatively small transform to encode that image portion to reduce quantization artifacts, although compression efficiency is correspondingly reduced compared to using a relatively large transform and post-processing filtering to reduce or eliminate quantization artifacts.

[0025] This paper describes encoding and decoding using rate-distortion optimization, which includes an inverse transform incorporating a transform-size adaptive directional nonlinear filter. By performing rate-distortion optimization using an inverse transform incorporating a transform-size adaptive directional nonlinear filter, this improves video code processing techniques or codecs. Compared to rate-distortion optimization that omits or excludes filtering, this can reduce encoding costs, improve accuracy, or both by identifying relatively large transforms.

[0026] Figure 1 This is a diagram of a computing device 100 according to an implementation of the present disclosure. The computing device 100 shown includes a memory 110, a processor 120, a user interface (UI) 130, an electronic communication unit 140, a sensor 150, a power supply 160, and a bus 170. As used herein, the term "computing device" includes any unit or combination of units capable of performing any method disclosed herein or any one or more portions thereof.

[0027] The computing device 100 may be a fixed computing device, such as a personal computer (PC), server, workstation, minicomputer, or mainframe computer; or a mobile computing device, such as a mobile phone, personal digital assistant (PDA), laptop computer, or tablet PC. Although shown as a single unit, any one or more components of the computing device 100 may be integrated into any number of separate physical units. For example, the user interface 130 and processor 120 may be integrated into a first physical unit, and the memory 110 may be integrated into a second physical unit.

[0028] Memory 110 may include any non-transitory computer-usable or computer-readable medium, such as any tangible means that can, for example, contain, store, transmit, or transfer data 112, instructions 114, operating system 116, or any information associated therewith, for use by or in conjunction with other components of computing device 100. Non-transitory computer-usable or computer-readable media may be, for example, solid-state drives, memory cards, removable media, read-only memory (ROM), random access memory (RAM), any type of disk (including hard disks, floppy disks, optical disks, magnetic cards, or optical cards), application-specific integrated circuits (ASICs), or any type of non-transitory medium suitable for storing electronic information, or any combination thereof.

[0029] Although shown as a single unit, memory 110 may include multiple physical units, such as one or more main memory units (e.g., random access memory units), one or more secondary data storage units (e.g., disks), or combinations thereof. For example, data 112 or a portion thereof, instructions 114 or a portion thereof, or both, may be stored in secondary storage units and may be loaded or otherwise transferred to main storage units in conjunction with processing the corresponding data 112, executing the corresponding instructions 114, or both. In some implementations, memory 110 or a portion thereof may be removable memory.

[0030] Data 112 may include information such as input audio data, encoded audio data, decoded audio data, etc. Instructions 114 may include instructions, such as code, for performing any method disclosed herein or any one or more portions thereof. Instructions 114 may be implemented in hardware, software, or any combination thereof. For example, instructions 114 may be implemented as information stored in memory 110, such as a computer program, which may be executed by processor 120 to perform any of the corresponding methods, algorithms, aspects, or combinations thereof as described herein.

[0031] Although shown as being included in memory 110, in some implementations, instructions 114 or portions thereof may be implemented as a dedicated processor or circuit system, which may include dedicated hardware for performing any of the methods, algorithms, aspects, or combinations thereof described herein. Portions of instructions 114 may be distributed across multiple processors on the same or different machines, or across networks such as local area networks, wide area networks, the Internet, or combinations thereof.

[0032] Processor 120 may include any device or system capable of manipulating or processing existing or subsequently developed digital signals or other electronic information, including optical processors, quantum processors, molecular processors, or combinations thereof. For example, processor 120 may include a dedicated processor, a central processing unit (CPU), a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a programmable logic array, a programmable logic controller, microcode, firmware, any type of integrated circuit (IC), a state machine, or any combination thereof. As used herein, the term "processor" includes a single processor or multiple processors.

[0033] User interface 130 may include any unit capable of interfacing with a user, such as a virtual or physical keyboard, touchpad, display, touch display, speaker, microphone, video camera, sensor, or any combination thereof. For example, user interface 130 may be an audiovisual display device, and computing device 100 may use user interface 130 or the audiovisual display device to present audio (such as decoded audio) in conjunction with displayed video (such as decoded video). Although shown as a single unit, user interface 130 may include one or more physical units. For example, user interface 130 may include an audio interface for performing audio communication with the user and a touch display for performing visual and touch-based communication with the user.

[0034] Electronic communication unit 140 can transmit, receive, or transmit and receive signals via wired or wireless electronic communication medium 180 (such as radio frequency (RF) communication medium, ultraviolet (UV) communication medium, visible light communication medium, fiber optic communication medium, cable communication medium, or combinations thereof). For example, as shown, electronic communication unit 140 is operatively connected to electronic communication interface 142 (such as an antenna) configured to communicate via wireless signals.

[0035] Although the electronic communication interface 142 is in Figure 1 While shown as a wireless antenna, the electronic communication interface 142 can be a wireless antenna as shown, a wired communication port (such as an Ethernet port), an infrared port, a serial port, or any other wired or wireless unit capable of interfacing with wired or wireless electronic communication medium 180. Although Figure 1 A single electronic communication unit 140 and a single electronic communication interface 142 are shown, but any number of electronic communication units and any number of electronic communication interfaces can be used.

[0036] Sensor 150 may include, for example, an audio sensing device, a visible light sensing device, a motion sensing device, or a combination thereof. For example, sensor 150 may include a sound sensing device, such as a microphone, or any other existing or later-developed sound sensing device capable of sensing sounds near computing device 100, such as speech or other utterances made by a user operating computing device 100. In another example, sensor 150 may include a camera or any other existing or later-developed image sensing device capable of sensing images, such as images of a user operating computing device. Although a single sensor 150 is shown, computing device 100 may include multiple sensors 150. For example, computing device 100 may include a first camera oriented toward the user's field of view towards computing device 100 and a second camera oriented toward the user's field of view away from computing device 100.

[0037] The power supply 160 can be any suitable device for powering the computing device 100. For example, the power supply 160 may include a wired external power interface; one or more dry cell batteries, such as nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel hydride (NiMH), lithium-ion (Li-ion); a solar cell; a fuel cell; or any other device capable of powering the computing device 100. Although Figure 1 A single power source 160 is shown, but the computing device 100 may include multiple power sources 160, such as batteries and wired external power interfaces.

[0038] Although shown as separate units, the electronic communication unit 140, electronic communication interface 142, user interface 130, power supply 160, or portions thereof, can be configured as a combined unit. For example, the electronic communication unit 140, electronic communication interface 142, user interface 130, and power supply 160 can be implemented as a communication port capable of interfacing with an external display device and providing communication, power, or both.

[0039] One or more of the following components—memory 110, processor 120, user interface 130, electronic communication unit 140, sensor 150, or power supply 160—may be operatively coupled via bus 170. Although Figure 1 A single bus 170 is shown, but the computing device 100 may include multiple buses. For example, memory 110, processor 120, user interface 130, electronic communication unit 140, sensor 150, and bus 170 may receive power from power source 160 via bus 170. In another example, memory 110, processor 120, user interface 130, electronic communication unit 140, sensor 150, power source 160, or combinations thereof, may transmit data via bus 170, such as by sending and receiving electronic signals.

[0040] although Figure 1 Although not shown separately, one or more of the processor 120, user interface 130, electronic communication unit 140, sensor 150, or power supply 160 may include internal memory, such as internal buffers or registers. For example, processor 120 may include internal memory (not shown) and may read data 112 from memory 110 into internal memory (not shown) for processing.

[0041] Although shown as separate components, memory 110, processor 120, user interface 130, electronic communication unit 140, sensor 150, power supply 160 and bus 170, or any combination thereof, may be integrated into one or more electronic units, circuits or chips.

[0042] Figure 2This is a diagram of a computing and communication system 200 according to an implementation of this disclosure. The computing and communication system 200 shown includes computing and communication devices 100A, 100B, 100C, access points 210A, 210B, and a network 220. For example, the computing and communication system 200 may be a multiple access system that provides communications (such as voice, audio, data, video, messaging, broadcasting, or combinations thereof) to one or more wired or wireless communication devices, such as computing and communication devices 100A, 100B, 100C. Although for simplicity... Figure 2 Three computing and communication devices 100A, 100B, and 100C, two access points 210A and 210B, and a network 220 are shown, but any number of computing and communication devices, access points, and networks can be used.

[0043] The computing and communication devices 100A, 100B, and 100C can be, for example, computing devices, such as... Figure 1 The computing device 100 is shown in the diagram. For example, computing and communication devices 100A and 100B can be user devices, such as mobile computing devices, laptop computers, thin clients, or smartphones, and computing and communication device 100C can be a server, such as a mainframe or cluster. Although computing and communication devices 100A and 100B are described as user devices, and computing and communication device 100C is described as a server, any computing and communication device can perform some or all of the functions of a server, some or all of the functions of a user device, or some or all of the functions of a server and a user device. For example, server computing and communication device 100C can receive, encode, process, store, transmit, or a combination thereof of audio data, and one or both of computing and communication devices 100A and 100B can receive, decode, process, store, present, or a combination thereof of audio data.

[0044] Each computing and communication device 100A, 100B, 100C (which may include user equipment (UE), mobile station, fixed or mobile subscriber unit, cellular phone, personal computer, tablet computer, server, consumer electronics product, or any similar device) may be configured to perform wired or wireless communication, such as via network 220. For example, computing and communication devices 100A, 100B, 100C may be configured to transmit or receive wired or wireless communication signals. Although each computing and communication device 100A, 100B, 100C is shown as a single unit, the computing and communication device may include any number of interconnecting elements.

[0045] Each access point 210A, 210B can be any type of device configured to communicate with computing and communication devices 100A, 100B, 100C, network 220, or both, via wired or wireless communication links 180A, 180B, 180C. For example, access points 210A, 210B can include base stations, base transceivers (BTS), node Bs, enhanced node Bs (eNode-Bs), home node Bs (HNode-Bs), wireless routers, wired routers, hubs, repeaters, switches, or any similar wired or wireless devices. Although each access point 210A, 210B is shown as a single unit, the access points can include any number of interconnecting elements.

[0046] Network 220 can be any type of network configured to provide services such as voice, data, applications, Internet Voice Protocol (VoIP), or any other communication protocol or combination of communication protocols via wired or wireless communication links. For example, network 220 can be a local area network (LAN), a wide area network (WAN), a virtual private network (VPN), a mobile or cellular telephone network, the Internet, or any other electronic communication method. The network can use communication protocols such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Internet Protocol (IP), Real-Time Transfer Protocol (RTP), Hypertext Transfer Protocol (HTTP), or combinations thereof.

[0047] Computing and communication devices 100A, 100B, and 100C can communicate with each other via network 220 using one or more wired or wireless communication links, or a combination of wired and wireless communication links. For example, as shown, computing and communication devices 100A and 100B can communicate via wireless communication links 180A and 180B, and computing and communication device 100C can communicate via wired communication link 180C. Any one of computing and communication devices 100A, 100B, and 100C can communicate using any one or more wired or wireless communication links. For example, the first computing and communication device 100A can communicate via a first access point 210A using a first type of communication link, the second computing and communication device 100B can communicate via a second access point 210B using a second type of communication link, and the third computing and communication device 100C can communicate via a third access point (not shown) using a third type of communication link. Similarly, access points 210A and 210B can communicate with network 220 via one or more types of wired or wireless communication links 230A and 230B. Although Figure 2The computing and communication devices 100A, 100B, and 100C are shown communicating via network 220, but the computing and communication devices 100A, 100B, and 100C can communicate with each other via any number of communication links (such as direct wired or wireless communication links).

[0048] In some implementations, communication between one or more of the computing and communication devices 100A, 100B, and 100C may omit communication via network 220 and may include data transfer via another medium (not shown), such as a data storage device. For example, server computing and communication device 100C may store audio data (such as encoded audio data) in a data storage device (such as a portable data storage unit), and one or both of computing and communication devices 100A or 100B may access, read, or retrieve the stored audio data from the data storage unit, such as by physically disconnecting the data storage unit from server computing and communication device 100C and physically connecting the data storage unit to computing and communication device 100A or computing and communication device 100B.

[0049] Other implementations of the computing and communication system 200 are possible. For example, in one implementation, network 220 may be an ad hoc network, and one or more access points 210A and 210B may be omitted. The computing and communication system 200 may include... Figure 2 Devices, units, or elements not shown. For example, computing and communication system 200 may include more communication devices, networks, and access points.

[0050] Figure 3 This is a diagram of a video stream 300 used in encoding and decoding according to an implementation of this disclosure. The video stream 300—such as a video stream captured by a video camera or a video stream generated by a computing device—may include a video sequence 310. The video sequence 310 may include a sequence of adjacent frames 320. Although three adjacent frames 320 are shown, the video sequence 310 may include any number of adjacent frames 320.

[0051] Each frame 330 from adjacent frames 320 can represent a single image from the video stream. Although Figure 3 Not shown, but frame 330 may include one or more segments, tiles, or planes that can be independently (e.g., in parallel) coded or otherwise processed. Frame 330 may include one or more tiles 340. Each tile in tile 340 may be a rectangular region of the frame that can be independently coded. Each tile in tile 340 may include a corresponding block 350. Although Figure 3Not shown, but a block can include pixels. For example, a block can include a 16×16 pixel group, an 8×8 pixel group, an 8×16 pixel group, or any other pixel group. Unless otherwise indicated herein, the term 'block' can include a superblock, macroblock, segment, slice, or any other portion of a frame. Frames, blocks, pixels, or combinations thereof can include display information, such as luminance information, chrominance information, or any other information that can be used to store, modify, transmit, or display a video stream or a portion thereof.

[0052] Figure 4 This is a block diagram of an encoder 400 according to an implementation of this disclosure. The encoder 400 can be used in, for example... Figure 1 The computing device 100 shown in the figure or Figure 2 The computing and communication devices 100A, 100B, and 100C shown are implemented, for example, stored in a storage device such as... Figure 1 The computer software program in the data storage unit of the memory 110 shown herein. This computer software program may include programs that can be generated by, for example... Figure 1 The processor 120 shown executes machine instructions and can cause the device to encode video data as described herein. The encoder 400 can be implemented as dedicated hardware included, for example, in computing device 100.

[0053] Encoder 400 can handle, for example, Figure 3 The input video stream 402 of the video stream 300 shown is encoded to generate an encoded (compressed) bitstream 404. In some implementations, the encoder 400 may include a forward path for generating the compressed bitstream 404. The forward path may include an intra / inter-frame prediction unit 410, a transform unit 420, a quantization unit 430, an entropy coding unit 440, or any combination thereof. In some implementations, the encoder 400 may include a reconstruction path (indicated by a dotted connection line) for reconstructing frames for encoding further blocks. The reconstruction path may include a dequantization unit 450, an inverse transform unit 460, a reconstruction unit 470, a filtering unit 480, or any combination thereof. Other structural variations of the encoder 400 may be used to encode the video stream 402.

[0054] To encode video stream 402, each frame within video stream 402 can be processed in blocks. Therefore, the current block can be identified from the blocks within a frame, and the current block can be encoded.

[0055] At intra / inter-frame prediction unit 410, the current block can be encoded using either intra-frame prediction (which can occur within a single frame) or inter-frame prediction (which can occur from frame to frame). Intra-frame prediction may include generating a prediction block based on previously encoded and reconstructed samples in the current frame. Inter-frame prediction may include generating a prediction block based on samples from one or more previously constructed reference frames. Generating a prediction block for the current block in the current frame may include performing motion estimation to generate motion vectors indicating appropriate reference portions of the reference frames.

[0056] Intra / inter-frame prediction unit 410 can subtract a prediction block from the current block (original block) to produce a residual block. Transform unit 420 can perform a block-based transform that may include transforming the residual block into transform coefficients in, for example, the frequency domain. Examples of block-based transforms include the Karhunen-Loève transform (KLT), discrete cosine transform (DCT), singular value decomposition transform (SVD), and asymmetric discrete sine transform (ADST). In the example, DCT may include transforming the block to the frequency domain. DCT may include using transform coefficient values ​​based on spatial frequency, where the lowest frequency (i.e., DC) coefficients are located in the upper left of the matrix and the highest frequency coefficients are located in the lower right of the matrix.

[0057] Quantization unit 430 can convert transform coefficients into discrete quantum values, which may be referred to as quantized transform coefficients or quantization levels. The quantized transform coefficients can be entropy-encoded by entropy coding unit 440 to produce entropy-encoded coefficients. Entropy coding may include the use of probability distribution indices. The entropy-encoded coefficients and information for decoding blocks can be output to a compressed bitstream 404, which may include the prediction type used, motion vectors, and quantizer values. Various techniques such as run-length encoding (RLE) and zero-run coding can be used to format the compressed bitstream 404.

[0058] The reconstruction path can be used to maintain encoder 400 and such Figure 5The decoder 500 shown in the diagram synchronizes reference frames between corresponding decoders. The reconstruction path can be similar to the decoding process discussed below and can include decoding an encoded frame or a portion thereof. Decoding an encoded frame or a portion thereof can include decoding an encoded block. Decoding an encoded block can include dequantizing the quantized transform coefficients at dequantization unit 450 and performing an inverse transform on the dequantized transform coefficients at inverse transform unit 460 to produce a derived residual block. Reconstruction unit 470 can add the prediction block generated by intra / inter-frame prediction unit 410 to the derived residual block to create a decoded block. Filtering unit 480 can be applied to the decoded block to generate a reconstructed block that can reduce distortion such as blocking artifacts. Although Figure 4 A filtering unit 480 is shown, but filtering the decoded block can include loop filtering, deblocking filtering, or other types of filtering or combinations of multiple filtering types. The reconstructed block can be stored or otherwise made accessible as a reconstructed block, which can be part of a reference frame, used to encode another part of the current frame, another frame, or both, as indicated by the dashed line at 482. Code processing information of the frame (such as the deblocking threshold index value) can be encoded, included in the compressed bitstream 404, or both, as indicated by the dashed line at 484.

[0059] Other variations of encoder 400 can be used to encode the compressed bitstream 404. For example, a non-transform-based encoder 400 can directly quantize the residual block without transform unit 420. In some implementations, quantization unit 430 and dequantization unit 450 can be combined into a single unit.

[0060] Figure 5 This is a block diagram of a decoder 500 according to an implementation of this disclosure. The decoder 500 can be configured in, for example... Figure 1 The computing device 100 shown in the figure or Figure 2 The computing and communication devices 100A, 100B, and 100C shown are implemented, for example, stored in a storage device such as... Figure 1 The computer software program is located in the data storage unit of the memory 110 shown in the figure. The computer software program may include programs that can be generated by, for example, Figure 1 The processor 120 shown executes machine instructions and can cause the device to decode video data as described herein. The decoder 500 can be implemented as dedicated hardware included in, for example, computing device 100.

[0061] Decoder 500 can receive compressed bitstream 502, such as Figure 4The compressed bitstream 404 is shown, and the compressed bitstream 502 can be decoded to generate the output video stream 504. The decoder 500 may include an entropy decoding unit 510, a dequantization unit 520, an inverse transform unit 530, an intra / inter-frame prediction unit 540, a reconstruction unit 550, a filtering unit 560, or any combination thereof. Other structural variations of the decoder 500 can be used to decode the compressed bitstream 502.

[0062] Entropy decoding unit 510 can use, for example, context-adaptive binary arithmetic decoding to decode data elements within the compressed bitstream 502 to produce a quantized set of transform coefficients. Dequantization unit 520 can dequantize the quantized transform coefficients, and inverse transform unit 530 can perform an inverse transform on the dequantized transform coefficients to produce a derived residual block, which can be combined with... Figure 4 The inverse transform unit 460 shown corresponds to the derived residual block generated. Using header information decoded from the compressed bitstream 502, the intra / inter-frame prediction unit 540 can generate a prediction block corresponding to the prediction block created in the encoder 400. At the reconstruction unit 550, the prediction block can be added to the derived residual block to create a decoded block. The filtering unit 560 can be applied to the decoded block to reduce artifacts such as blockiness artifacts. This can include loop filtering, deblocking filtering, or other types of filtering or combinations of multiple filtering types, and can include generating a reconstructed block that can be output as the output video stream 504.

[0063] Other variations of decoder 500 can be used to decode the compressed bitstream 502. For example, decoder 500 can produce output video stream 504 without deblocking filter unit 560.

[0064] Figure 6 A frame (such as) according to an implementation of this disclosure. Figure 3The diagram shows a block representation of a portion 600 of frame 330. As shown, this portion 600 of the frame includes four 64×64 blocks 610, arranged in two rows and two columns in a matrix or Cartesian plane. In some implementations, the 64×64 block may be a maximum code processing unit, N=64. Each 64×64 block may include four 32×32 blocks 620. Each 32×32 block may include four 16×16 blocks 630. Each 16×16 block may include four 8×8 blocks 640. Each 8×8 block 640 may include four 4×4 blocks 650. Each 4×4 block 650 may include 16 pixels, which may be represented in four rows and four columns in each corresponding block in the Cartesian plane or matrix. Pixels may include information representing the image captured in the frame, such as luminance information, color information, and positional information. In some implementations, a 16×16 pixel block, as shown, may include: a luminance block 660, which may include luminance pixels 662; and two chrominance blocks 670 and 680, such as a U or Cb chrominance block 670 and a V or Cr chrominance block 680. Chroma blocks 670 and 680 may include chrominance pixels 690. For example, luminance block 660 may include 16×16 luminance pixels 662, and each chrominance block 670 and 680 may include 8×8 chrominance pixels 690, as shown. Although one arrangement of blocks is shown, any arrangement can be used. Figure 6 An N×N block is shown, but in some implementations, an N×M block can be used. For example, 32×64 blocks, 64×32 blocks, 16×32 blocks, 32×16 blocks, or any other block size can be used. In some implementations, N×2N blocks, 2N×N blocks, or combinations thereof can be used.

[0065] In some implementations, video bitcoding may include ordered block-level bitcoding. Ordered block-level bitcoding may involve bitcoding blocks of a frame in an order such as raster scan order, where blocks can be identified and processed starting with the top-left block of the frame or a portion of the frame, and proceeding along rows from left to right and from top to bottom, thus identifying each block sequentially for processing. For example, a 64×64 block in the top left column of the frame may be the first bitcoded block, and a 64×64 block immediately to the right of the first block may be the second bitcoded block. The second row from the top may be the second bitcoded row, such that a 64×64 block in the left column of the second row can be bitcoded after the 64×64 block in the rightmost column of the first row.

[0066] In some implementations, code processing of blocks may include using quadtree code processing, which may involve code processing smaller block units within the block in raster scan order. For example, quadtree code processing can be used to... Figure 6The 64×64 block shown in the lower left corner of the frame portion is processed by encoding. This can be done by encoding the top-left 32×32 block, then the top-right 32×32 block, then the bottom-left 32×32 block, and then the bottom-right 32×32 block. Quadtree encoding can be used to encode each 32×32 block, where the top-left 16×16 block, then the top-right 16×16 block, then the bottom-left 16×16 block, and then the bottom-right 16×16 block. Quadtree encoding can also be used to encode each 16×16 block, where the top-left 8×8 block, then the top-right 8×8 block, then the bottom-left 8×8 block, and then the bottom-right 8×8 block. Quadtree coding can be used to code each 8×8 block, where the top-left 4×4 block can be coded, followed by the top-right 4×4 block, then the bottom-left 4×4 block, and finally the bottom-right 4×4 block. In some implementations, the 8×8 blocks can be omitted for a 16×16 block, and quadtree coding can be used to code the 16×16 block, where the top-left 4×4 block can be coded, and then the other 4×4 blocks in the 16×16 block can be coded according to the raster scan order.

[0067] In some implementations, video bitcoding may include compressing the information included in the original or input frame by, for example, omitting some information from the original frame in the corresponding encoded frame. For example, bitcoding may include reducing spectral redundancy, reducing spatial redundancy, reducing temporal redundancy, or a combination thereof.

[0068] In some implementations, reducing spectral redundancy may include using a color model based on a luminance component (Y) and two chrominance components (U and V or Cb and Cr), which may be referred to as the YUV or YCbCr color model or color space. Using the YUV color model may involve using a relatively large amount of information to represent the luminance component of a portion of a frame and using a relatively small amount of information to represent each corresponding chrominance component of that portion of the frame. For example, a portion of a frame may be represented by a high-resolution luminance component that may include a 16×16 pixel block and two lower-resolution chrominance components, where each chrominance component represents that portion of the frame as an 8×8 pixel block. Pixels may indicate values, for example, values ​​in the range of 0 to 255, and may be stored or transmitted using, for example, eight bits. Although this disclosure is described with reference to the YUV color model, any color model may be used.

[0069] In some implementations, reducing spatial redundancy may include using, for example, a discrete cosine transform (DCT) to transform the block into the frequency domain. For example, encoder units (such as...) Figure 4 The transform unit 420 shown can perform DCT using transform coefficient values ​​based on spatial frequency.

[0070] In some implementations, reducing temporal redundancy may include using the similarity between frames to encode frames using a relatively small amount of data based on one or more reference frames, which may be previously encoded, decoded, and reconstructed frames of the video stream. For example, a block or pixel in the current frame may resemble a spatially corresponding block or pixel in a reference frame. In some implementations, a block or pixel in the current frame may resemble a block or pixel in a reference frame at a different spatial location, and reducing temporal redundancy may include generating motion information indicating the spatial difference or translation between the position of a block or pixel in the current frame and its corresponding position in a reference frame.

[0071] In some implementations, reducing temporal redundancy may include identifying a portion of a reference frame corresponding to the current block or pixel of the current frame. For example, a reference frame, or a portion thereof, that may be stored in memory, may be searched to identify the portion used to generate a prediction for encoding the current block or pixel of the current frame with maximum efficiency. For example, the search may identify a portion of the reference frame for which the difference in pixel values ​​between the current block and the prediction block generated based on that portion of the reference frame is minimized—and may be referred to as a motion search. In some implementations, the portion of the reference frame being searched may be limited. For example, the portion of the reference frame being searched may include a finite number of rows of the reference frame, which may be referred to as a search region. In the example, identifying the portion of the reference frame used to generate the prediction may include calculating a cost function, such as the sum of absolute differences (SAD), between the pixels of each portion of the search region and the pixels of the current block.

[0072] In some implementations, the spatial difference between the position of the portion of the reference frame used to generate the prediction within the reference frame and the position of the current block in the current frame can be represented as a motion vector. The difference in pixel values ​​between the predicted block and the current block can be referred to as differential data, residual data, prediction error, or simply a residual block. In some implementations, generating the motion vector can be referred to as motion estimation, and Cartesian coordinates can be used to indicate the pixels of the current block based on their position. f x, y。 Similarly, Cartesian coordinates can be used to indicate the pixels of the search area in the reference frame based on their position. r x, y。 The motion vector (MV) for the current block can be determined based on, for example, the SAD between the pixels of the current frame and the corresponding pixels of the reference frame.

[0073] Although this document describes frames using a matrix or Cartesian representation for clarity, frames can be stored, transmitted, processed, or any combination thereof using any data structure, making it possible to efficiently represent pixel values ​​for frames or images. For example, frames can be stored, transmitted, processed, or any combination thereof using a two-dimensional data structure (such as the matrix shown) or a one-dimensional data structure (such as a vector array). In implementations, the representation of a frame (such as the two-dimensional representation shown) can correspond to its physical location when the frame is rendered as an image. For example, the top-left corner position of a block at the top-left corner of a frame can correspond to the physical location of the top-left corner when the frame is rendered as an image.

[0074] In some implementations, block-based code processing efficiency can be improved by partitioning the input block into one or more prediction partitions, which can be rectangular (including square) partitions for predictive code processing. In some implementations, video code processing using prediction partitions can include selecting a prediction partitioning scheme from multiple candidate prediction partitioning schemes. For example, in some implementations, candidate prediction partitioning schemes for a 64×64 code processing unit can include rectangular prediction partitions ranging in size from 4×4 to 64×64—such as 4×4, 4×8, 8×4, 8×8, 8×16, 16×8, 16×16, 16×32, 32×16, 32×32, 32×64, 64×32, or 64×64. In some implementations, video code processing using prediction partitions can include a complete prediction partition search, which can include encoding the code processing unit using each available candidate prediction partitioning scheme and selecting the optimal scheme—such as the one that produces the minimum rate distortion error—to select the prediction partitioning scheme.

[0075] In some implementations, encoding a video frame may include identifying a prediction partitioning scheme for encoding the current block (such as block 610). In some implementations, identifying the prediction partitioning scheme may include determining whether to encode the block as a single prediction partition of the maximum code processing unit size (which could be 64×64 as shown) or to partition the block into multiple prediction partitions (which may correspond to sub-blocks, such as 32×32 block 620, 16×16 block 630, or 8×8 block 640 as shown), and may include determining whether to partition into one or more smaller prediction partitions. For example, a 64×64 block may be partitioned into four 32×32 prediction partitions. Three of the four 32×32 prediction partitions may be encoded as 32×32 prediction partitions, and the fourth 32×32 prediction partition may be further partitioned into four 16×16 prediction partitions. Three of the four 16×16 prediction partitions can be encoded as 16×16 prediction partitions, and the fourth 16×16 prediction partition can be further partitioned into four 8×8 prediction partitions, each of which can be encoded as an 8×8 prediction partition. In some implementations, identifying the prediction partitioning scheme may include using a prediction partitioning decision tree.

[0076] In some implementations, video code processing for the current block may include identifying the optimal predictive code processing mode from multiple candidate predictive code processing modes. This provides flexibility in handling video signals with various statistical properties and can improve compression efficiency. For example, the video code processor may evaluate each candidate predictive code processing mode to identify the optimal one, which could be, for example, the predictive code processing mode that minimizes an error metric such as rate-distortion cost for the current block. In some implementations, the complexity of searching for candidate predictive code processing modes can be reduced by limiting the set of available candidate predictive code processing modes based on the similarity between the current block and the corresponding predictive block. In some implementations, the complexity of searching for each candidate predictive code processing mode can be reduced by performing a targeted refinement mode search. For example, metrics can be generated for a finite set of candidate block sizes such as 16×16, 8×8, and 4×4, with the error metrics associated with each block size in descending order, and additional candidate block sizes such as 4×8 and 8×4 can be evaluated.

[0077] In some implementations, block-based code processing efficiency can be improved by partitioning the current residual block into one or more transform partitions, where the transform partitions can be rectangular (including square) partitions used for transform code processing. In some implementations, video code processing—such as video code processing using transform partitions—can include selecting a uniform transform partitioning scheme. For example, the current residual block (such as block 610) can be a 64×64 block, and this current residual block can be transformed using a 64×64 transform without partitioning.

[0078] although Figure 6 While not explicitly stated, a unified transform partitioning scheme can be used to transform partition the residual blocks. For example, a unified transform partitioning scheme comprising four 32×32 transform blocks, a unified transform partitioning scheme comprising sixteen 16×16 transform blocks, a unified transform partitioning scheme comprising sixty-four 8×8 transform blocks, or a unified transform partitioning scheme comprising 256 4×4 transform blocks can be used to transform partition the 64×64 residual blocks.

[0079] In some implementations, video bitcoding—such as video bitcoding using transform partitioning—may include using multi-form transform partitioning to identify multiple transform block sizes for the residual block. In some implementations, multi-form transform partitioning may include recursively determining whether to transform the current block using the current block size transform, or by partitioning the current block and performing multi-form transform partitioning on each partition. For example, Figure 6 The lower left block 610 shown can be a 64×64 residual block, and the multiform transform partitioning code processing can include determining whether to use a 64×64 transform to code the current 64×64 residual block, or to code the 64×64 residual block by partitioning it into partitions such as four 32×32 blocks 620 and performing multiform transform partitioning code processing on each partition. In some implementations, determining whether to transform partition the current block can be based on a comparison of the cost of encoding the current block using a current block size transform with the sum of the costs of encoding each partition using a partition size transform.

[0080] Figure 7 This is a block diagram of a transform-selective encoder 700 implemented according to the present disclosure. The transform-selective encoder 700 can be implemented in, for example... Figure 1 The computing device 100 shown in the figure or Figure 2 The computing and communication devices 100A, 100B, and 100C shown are implemented, for example, stored in a storage device such as... Figure 1 The computer software program in the data storage unit of the memory 110 shown herein. This computer software program may include programs that can be generated by, for example... Figure 1 The processor 120 shown executes machine instructions and can cause the device to encode video data as described herein. The encoder 700 implementing transform selection can be implemented as dedicated hardware, for example, included in the computing device 100. The encoder 700 implementing transform selection and... Figure 4 The encoder 400 shown is similar, unless otherwise described herein or otherwise clearly understood from the context.

[0081] The encoder 700, which implements transform selection, pairs such as... Figure 3 The input video stream 702 of the video stream 300 shown is encoded to generate an encoded (compressed) bitstream 704. In some implementations, the encoder 700 implementing transform selection includes a forward path for generating the compressed bitstream 704. The forward path includes an intra / inter-frame prediction unit 710, a transform unit 720, a quantization unit 730, an entropy coding unit 740, or any combination thereof. In some implementations, the encoder 700 implementing transform selection includes a reconstruction path (indicated by a dotted connection line) for reconstructing frames for encoding further blocks. The reconstruction path includes a dequantization unit 750, an inverse transform unit 760, a filtering unit 770, or any combination thereof. Other structural variations of the encoder 700 implementing transform selection can be used to encode the video stream 702.

[0082] To encode video stream 702, each frame within video stream 702 can be processed in blocks. Therefore, the current block can be identified from the blocks within a frame, and the current block can be encoded.

[0083] Intra / Inter-frame Prediction Unit 710 and Figure 4 The intra / inter-frame prediction unit 410 shown is similar, unless otherwise described herein or otherwise clear from the context. Intra-frame prediction involves generating a prediction block based on previously encoded and reconstructed samples in the current frame. Inter-frame prediction involves generating a prediction block (prediction block data) based on samples from one or more previously constructed reference frames. The intra / inter-frame prediction unit 710 subtracts the prediction block from the current block (original block) to produce a residual block (residual block data).

[0084] Transformer 720 and Figure 4 The transform unit 420 shown is similar, unless otherwise described herein or otherwise apparent from the context. Transform unit 720 performs a block-based transform, which includes transforming the residual block into transform coefficients, for example, in the frequency domain.

[0085] Transformation unit 720 selects, determines, identifies, or obtains the optimal transform from the available transforms for transforming the residual block data. Identifying the optimal transform includes identifying the optimal transform type from the available transform types and the optimal transform size from the available transform sizes. Examples of available transform types include Karhunen-Loève transform (KLT), discrete cosine transform (DCT), singular value decomposition transform (SVD), and asymmetric discrete sine transform (ADST). The transform type can be indicated by a transform type identifier. Examples of available transform sizes include 4×4, 4×8, 8×4, 4×16, 16×4, 8×8, 8×16, 16×8, 8×32, 32×8, 16×16, 16×32, 32×16, 16×64, 64×16, 32×32, 32×64, 64×32, or 64×64. Available transforms can be combinations or tuples of available transform types and available transform sizes.

[0086] Transformation unit 720 selects, determines, identifies, or obtains the optimal transformation by evaluating available transformations (such as two or more available transformations), including the optimal transformation type, the optimal transformation size, or both, wherein evaluating the corresponding available transformations includes evaluating the available transformation type and the available transformation size (n×m).

[0087] Evaluating the available transformations involves transforming the residual block data using the available transformations to obtain candidate transform block data.

[0088] Evaluating the available transforms involves obtaining candidate quantized block data by quantizing the candidate transform block data using quantization unit 730. Quantization unit 730 and... Figure 4 The quantization unit 430 shown is similar, unless otherwise described herein or otherwise clear from the context.

[0089] Evaluating the available transforms involves obtaining candidate dequantized transform block data (c) by dequantizing the candidate quantized block data using the dequantization unit 750. The dequantization unit 750 and... Figure 4 The dequantization unit 450 shown is similar, unless otherwise described herein or otherwise clear from the context.

[0090] The evaluation of the corresponding available transforms includes obtaining the candidate decoded residual block data (r) by performing an inverse transform on the candidate dequantized transform block data using an inverse transform (T) corresponding to the corresponding available transform (r=Tc) by the inverse transform unit 760. The inverse transform unit 760 and... Figure 4 The inverse transform unit 460 shown is similar, unless otherwise described herein or otherwise clear from the context.

[0091] The evaluation of the corresponding available transforms includes obtaining candidate decoded block data (q) (q = p + r) by combining the predicted block data (p) and the candidate decoded residual block data (r) by the inverse transform unit 760.

[0092] Evaluating the available transforms involves obtaining one or more rate-distortion optimization metrics, such as distortion metrics, coding cost metrics (bitrate), or both, based on candidate decoded block data (q = p + r) obtained using the available transforms.

[0093] The evaluation of available transforms includes transform unit 720 comparing one or more rate-distortion optimization metrics obtained for a first available transform with one or more equivalent rate-distortion optimization metrics obtained for a second available transform, as indicated by the directed dashed line at 762. The selection, determination, identification, or acquisition of the optimal transform omits, skips, excludes, or otherwise avoids processes such as filtering of the decoded block data (q = p + r) by filtering unit 770.

[0094] Transformation unit 720 selects, determines, identifies, or obtains the available transformations corresponding to one or more minimum rate distortion optimization indices as the optimal transformation.

[0095] Entropy coding unit 740 performs entropy coding on the quantized transform coefficients obtained using the optimal transform to produce entropy-coded coefficients. Entropy coding unit 740 and... Figure 4 The entropy coding unit 440 shown is similar, unless otherwise described herein or clearly understood from the context. The entropy-coded coefficients and information used for decoding the block can be output to a compressed bitstream 704, which may include the prediction type used, motion vectors, and quantizer values. The compressed bitstream 704 is similar to... Figure 4 The compressed bitstream shown is similar to 404, unless otherwise described herein or otherwise clearly understood from the context.

[0096] Applying filter unit 770 to the decoded block data (q = p + r) obtained using optimal transform to generate reconstructed blocks reduces distortion such as block artifacts. Filter unit 770 and Figure 4 The filter unit 480 shown is similar, unless otherwise described herein or otherwise clear from the context. Although Figure 7A filtering unit 770 is shown, but filtering the decoded block includes loop filtering, deblocking filtering, or other types of filtering or combinations of multiple filtering types. The reconstructed block can be stored or otherwise made accessible as a reconstructed block, which can be part of a reference frame, used to encode another part of the current frame, another frame, or both, as indicated by the dashed line at 772. Code processing information of the frame (such as the deblocking threshold index value) can be encoded, included in the compressed bitstream 704, or both, as indicated by the dashed line at 774.

[0097] Other variations of the encoder 700 that implements transform selection can be used to encode the compressed bitstream 704.

[0098] Figure 8 This is a flowchart illustrating an example of rate-distortion optimized encoding 800, including an inverse transform incorporating transform-size adaptive directional nonlinear filtering, according to an implementation of this disclosure. The rate-distortion optimized encoding 800, including an inverse transform incorporating transform-size adaptive directional nonlinear filtering, can be used by, for example... Figure 4 The encoder implementation of encoder 400 is shown in the figure.

[0099] The code 800, which uses rate-distortion optimization including an inverse transform incorporating transform-sized adaptive oriented nonlinear filtering, includes features such as... Figure 4 The input video stream 402 shown is encoded, or one or more portions thereof, to generate an encoded (compressed) output bitstream, such as... Figure 4 The encoded (compressed) bitstream 404 is shown in the figure.

[0100] In block-based hybrid video bitcoding, to reduce or minimize resource utilization (such as bandwidth utilization) of compressed or encoded video data in signal transmission, storage, or both, redundant data, such as spatial redundant data, temporal redundant data, or both, is omitted or eliminated from the compressed or encoded data. The amount of encoded data (such as the number of bits, count, or radix) of a portion of video (such as a frame sequence, frame, or block) is the bit rate or bitrate (R) used to encode that portion. The difference between the reconstructed portion of the encoded or compressed video and the input or source video portion can be used as an indicator or measure of distortion (D) caused by the bitcoding process, which corresponds to the quality loss of the reconstructed video. Depending on the bitrate (R) constraints or objectives, optimal video bitcoding minimizes distortion (D). The combination of bitrate (R) and distortion (D) can be used as an indicator or measure (cost) of bitcoding optimization.

[0101] To maximize coding optimization, such as minimizing distortion (D) based on bitrate (R) constraints or objectives, the encoder performs rate-distortion optimization (RDO) on a portion of a video or frame (such as a block), where the encoder identifies or determines coding parameters from a pool of candidate coding parameters that maximize coding optimization. For example, rate-distortion optimization may include determining the size of a prediction block or sub-block, a transform block or sub-block, or both. In another example, rate-distortion optimization may include mode decisions, which include determining, identifying, or selecting a prediction mode, such as an intra-frame prediction mode or an inter-frame prediction mode, for coding a portion of a video or frame (such as a block). Rate-distortion optimization includes: determining a corresponding cost for a set of one or more code processing parameters, such as based on a combination of bitrate (R) or bitrate values ​​and distortion (D) or approximations thereof; and identifying or selecting a set of code processing parameters from the set of code processing parameters that correspond to the minimum cost as parameters for code processing the current portion of the video or frame (such as the current block).

[0102] For example, in block-based pattern decision-making, the encoder selects the candidate pattern's rate-distortion cost with the minimum rate-distortion cost or the rate-distortion optimization cost value. The candidate mode identifier is used as the encoding mode for the current block. For a given candidate mode, the encoder obtains (e.g., through computation or otherwise access) a bitrate metric corresponding to encoding the current block using the candidate mode. The encoder obtains the distortion index value (). ), which represents or measures the source image ( ) or the portion corresponding to the current block and the candidate reconstructed image ( The distortion can be determined as the difference between the source image (or the portion corresponding to the current block). In some implementations, the distortion can be determined as the difference between the source image (or the portion corresponding to the current block). ) and reconstructed image ( The square between ) )error( Other distortion metrics can be used. In some implementations, rate-distortion optimization of the cost function can be used to obtain candidate reconstructed blocks. Rate distortion cost or rate distortion cost value () The rate-distortion optimization cost function obtains (such as calculating) distortion index values. ) and the bitrate metric value ( Multiplied by the Lagrange multiplier ( The sum of the results can be expressed as follows: .

[0103] [Equation 1]

[0104] In coding 800 using rate-distortion optimization, which includes an inverse transform incorporating transform-size adaptive directional nonlinear filtering, rate-distortion optimization includes identifying one or more transforms and corresponding inverse transforms used to encode the current block.

[0105] The rate-distortion optimized encoding 800, which includes an inverse transform incorporating a transform-sized adaptive directional nonlinear filter, includes obtaining the input block (at 810), obtaining the encoded block data (at 820), and outputting the encoded bitstream (at 830). Although Figure 8 Not explicitly shown, but using rate-distortion optimized coding 800, which includes an inverse transform incorporating transform-sized adaptive directional nonlinear filtering, includes other aspects of video code processing.

[0106] Get the input, current, or source block (at 810) (such as...) Figure 3 One of the blocks in block 350 shown or Figure 6 The block shown includes data from one or more frames (such as...). Figure 3 The frame 320 shown, such as from a frame sequence (such as...) Figure 3 The video sequence 310 shown, such as from a video (such as...) Figure 3 The video stream shown is 300 or Figure 4 The input video stream 402 shown obtains the input, current, or source frame (such as a frame, such as...). Figure 3 (Frame 330 shown).

[0107] In some implementations, input video data is obtained (at 810). The input video data comprises a sequence of frames (input frames). For example, this can be obtained from an encoder or its components (such as the encoder's intra / inter-frame prediction unit, etc.). Figure 4 The intra / inter-frame prediction unit 410 shown can obtain the input video stream.

[0108] The current frame for encoding is obtained from a sequence of frames, such as those from the input video data (at 810). The current frame (at 810) may be obtained after encoding one or more other frames, such as frames in the input video stream that are sequentially preceding the current frame, and generating or otherwise obtaining one or more corresponding reconstructed frames or one or more portions thereof to be used as one or more reference frames for encoding the current frame.

[0109] The current block for encoding is obtained from the current frame (at 810). The current block (at 810) may be obtained after one or more other blocks (such as blocks in the current frame that are sequentially preceding the current block in the current frame) have been encoded according to the block code processing order used for coding the current frame, and the corresponding reconstructed block or one or more portions thereof has been generated or otherwise obtained.

[0110] Obtaining the encoded block data (at 820) includes rate-distortion optimization (at 840) for encoding the current block (obtained at 810). Obtaining the encoded block data (at 820) may include... Figure 8 Other aspects of the encoding not explicitly shown in the text.

[0111] Rate-distortion optimization (at 840) involves identifying or selecting coding parameters or a set thereof from a plurality of available parameters or a set thereof for encoding the current block. A directed dashed line (at 845) is shown to indicate that rate-distortion optimization (at 840) involves evaluating two or more coding parameters or a set thereof from a plurality of available parameters or a set thereof for encoding the current block.

[0112] Rate-distortion optimization (at 840) includes: for a set of available parameters or a corresponding set of current candidate coding parameters (such as the current candidate transform), encoding the current block (the current input block) using the current candidate coding parameter set to obtain an encoded block of the current candidate; decoding and reconstructing the encoded block of the current candidate to obtain a reconstructed block of the current candidate; and obtaining a rate-distortion cost or score for encoding the current block using the current candidate coding parameter set. The rate-distortion cost, value, or score for encoding the current block using the corresponding set of candidate coding parameters is obtained using a rate-distortion optimization cost function. The candidate coding parameters corresponding to the minimum cost or score are identified as the code processing parameters of the current block.

[0113] Rate-distortion optimization (at 840) involves obtaining prediction block data (predicted block or predicted block) for encoding the current block (not in...). Figure 8 (as clearly shown in the text).

[0114] Rate-distortion optimization (at 840) involves obtaining residual block data (not in the current block) that indicates the difference between the current block and the predicted block data. Figure 8 (as clearly shown in the text).

[0115] Rate-distortion optimization (at 840) includes identifying the current transformation or candidate transformation for transforming the residual block data (at 850).

[0116] Identifying the current or candidate transform (at 850) involves identifying the transform type from one or more available transform types. Available transform types can include Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), and Asymmetric Discrete Sine Transform (ADST). Other transform types can also be used.

[0117] Identifying the current transformation or candidate transformation (at 850) includes identifying the transformation size from one or more available transformation sizes. Available transformation sizes can include rectangular transformations ranging from 4×4 to 64×64—such as 4×4, 4×8, 8×4, 8×8, 8×16, 16×8, 16×16, 16×32, 32×16, 32×32, 32×64, 64×32, or 64×64.

[0118] Rate-distortion optimization (at 840) includes identifying the corresponding inverse transform (at 860) of the adaptive directional nonlinear filter with transform size.

[0119] In some implementations, rate-distortion optimization (at 840) includes determining that the code processing mode of the current block is intra-prediction mode, and in response, determining that the corresponding inverse transform, including transform-size adaptive directional nonlinear filtering, is available. In some implementations, rate-distortion optimization includes determining that the code processing mode of the current block is inter-prediction mode, and in response, determining that the corresponding inverse transform, including transform-size adaptive directional nonlinear filtering, is unavailable. In some implementations, the corresponding inverse transform, including transform-size adaptive directional nonlinear filtering, is available regardless of the prediction mode.

[0120] In some implementations, rate-distortion optimization includes determining that the corresponding inverse transform, including transform-size adaptive oriented nonlinear filtering, is unavailable in response to determining that the number, count, or cardinality of zero-value transform coefficients in the current or candidate transform block data of the current block is below (e.g., less than) a defined threshold (e.g., four (4) or zero (0)). In some implementations, the corresponding inverse transform, including transform-size adaptive oriented nonlinear filtering, is available for the transform block regardless of the number, count, or cardinality of zero-value transform coefficients.

[0121] In some implementations, rate distortion optimization includes determining that the corresponding inverse transform, including transform-size adaptive directional nonlinear filtering, is unavailable in response to determining that the transform size is below (e.g., less than) a defined threshold. For example, rate distortion optimization may include determining that the corresponding inverse transform, including transform-size adaptive directional nonlinear filtering, is unavailable in response to determining that the maximum or highest size of the current transform size is less than a defined threshold (e.g., sixteen (16)). In another example, rate distortion optimization may include determining that the corresponding inverse transform, including transform-size adaptive directional nonlinear filtering, is unavailable in response to determining that the minimum or lowest size of the current transform size is less than a defined threshold (e.g., eight (8)). In yet another example, rate distortion optimization may include determining that the corresponding inverse transform, including transform-size adaptive directional nonlinear filtering, is unavailable in response to determining that the maximum or highest size of the current transform size is less than a defined maximum transform size threshold (e.g., sixteen (16)), or that the minimum or lowest size of the current transform size is less than a defined minimum transform size threshold (e.g., eight (8)).

[0122] Rate-distortion optimization (at 840) involves identifying the optimal transformation or rate-distortion optimal transformation from among two or more available transformations for transforming the residual block data. Identifying the optimal transformation includes identifying the optimal transformation type or rate-distortion optimal transformation type, as well as the optimal transformation size or rate-distortion optimal transformation size.

[0123] As indicated by the dashed line (at 845), rate-distortion optimization (at 840) involves evaluating two or more available transformations and identifying the transformation corresponding to the rate-distortion optimization metric or cost as the best transformation for code processing the current block.

[0124] Evaluating the current, candidate, or corresponding available transformations (at 840) involves obtaining candidate transform block data by transforming the residual block data using candidate transformations. Figure 8 (Not explicitly shown in the text).

[0125] Evaluating the current, candidate, or corresponding available transformations (at 840) involves obtaining quantized block data of the candidates by quantizing the candidate transformation block data. Figure 8 (Not explicitly shown in the text).

[0126] Evaluating the current, candidate, or corresponding available transforms (at 840) involves obtaining the dequantized transform block data of the candidates by dequantizing the quantized block data of the candidates. c ),in c(n × 1) ( Figure 8 (Not explicitly shown in the text).

[0127] Evaluating the current, candidate, or corresponding available transforms (at 840) involves obtaining the candidate decoded residual block data by performing an inverse transform on the candidate dequantized transform block data using the corresponding inverse transform, which includes an adaptive directional nonlinear filter for transform size. Figure 8 (Not explicitly shown in the text).

[0128] The candidate dequantized transform block data is inversely transformed using a corresponding inverse transform including transform-size adaptive oriented nonlinear filtering to obtain the candidate decoded residual block data. r (The decoded blocks in the middle), such as transforming block data by dequantizing the candidate blocks ( c Multiply by the transformation matrix corresponding to the inverse transformation ( T ),in T (n×n), which can be expressed as the following formula:

[0129] Inverse transformation of candidate dequantized transform block data includes combining prediction block data ( p ) and candidate decoded residual block data ( r To obtain candidate decoded block data ( q ()( Figure 8 (Not explicitly shown in the text), this can be expressed as the following formula:

[0130] Inverse transform of candidate dequantized transform block data includes using an adaptive directional nonlinear filter with optimal transform size. N (.)) for candidate decoded block data ( q ) is filtered to obtain candidate reconstructed block data. d ()( Figure 8 (Not explicitly shown in the text), this can be expressed as the following formula:

[0131] Optimal transform size adaptive directional nonlinear filter ( N (.) represents optimal, transform-sized, adaptive, directional, and nonlinear filters. For example, an optimal transform-sized adaptive directional nonlinear filter ( N (.)) is optimal because the optimal transform-size adaptive directional nonlinear filter ( N (.) is identified, selected, or determined via rate-distortion optimization. In another example, the optimal transform-size adaptive directional nonlinear filter ( N (.)) is the transform size, because the optimal transform size adaptive directional nonlinear filter ( NThe size of (.) is the size of the transform block. In another example, the optimal transform-sized adaptive directional nonlinear filter ( N (.)) is an adaptive filter, such as a pixel-adaptive filter, where the filter is derived on a per-pixel basis using an artificial intelligence model or machine learning model (such as an artificial neural network). To obtain, determine, or identify the optimal transform size adaptive oriented nonlinear filter (... N Artificial neural networks can derive vectors of combined scalars on a per-pixel basis. These vectors can be combined with a filter bank to derive a filter on a per-pixel basis. This is achieved using an optimal transform-size adaptive directional nonlinear filter. N (.)) can include applying a per-pixel filter to the image or pixel values ​​in the neighborhood of the corresponding pixel (such as within a defined distance) to obtain the filtered value of the corresponding pixel. In some implementations, an optimal transform-size adaptive oriented nonlinear filter can be indicated in the encoded bitstream. N (.)) or one or more aspects thereof (such as identifiers). In some implementations, the optimal transform-size adaptive directional nonlinear filter ( N (.)) can be a non-separable filter.

[0132] Optimal transform size adaptive directional nonlinear filter ( N The size of (.) is the size of the optimal transform or the corresponding inverse transform. For example, the inverse transform can be a 16×16 transform, and the optimal transform size is an adaptive directional nonlinear filter (.). N (.)) is a 16×16 filter. In some implementations, the optimal transform-sized adaptive directional nonlinear filter ( N (.)) represents a denoising filter, such as a weighted average denoising filter with an overcomplete dictionary, or a weighted overcomplete denoising filter. In some implementations, the optimal transform-sized adaptive directional nonlinear filter ( N (.)) is a filter designed for context.

[0133] Obtaining encoded block data (at 820) involves including candidate quantized block data obtained using the optimal transform in the encoded bitstream.

[0134] although Figure 8 While not explicitly stated, in some implementations, the reconstructed block data (at 820) can be processed after obtaining the encoded block data, such as during the encoder's reconstruction or decoding path, using reconstruction filtering (such as loop filtering) or other post-processing filtering. d Further processing is required. Reconstruction filtering and... Figure 4 The filter shown (at 480) Figure 5 The filter shown (at 560) or Figure 7 The filtering shown (at 770) is similar, unless otherwise described herein or otherwise clearly understood from the context. Reconstruction filtering is separate and distinct from transform-size adaptive directional nonlinear filtering. Identifying the transform corresponding to the rate-distortion optimization metric or cost as the optimal transform for code processing of the current block is performed before and independently of reconstruction filtering.

[0135] although Figure 8 While not explicitly shown, in some implementations, candidate decoded block data can be processed after obtaining the encoded block data (at 820), such as during the encoder's reconstruction or decoding path, using reconstruction filtering (such as loop filtering) or other post-processing filtering. q Further processing is required. Reconstruction filtering and... Figure 4 The filter shown (at 480) Figure 5 The filter shown (at 560) or Figure 7 The filtering shown (at 770) is similar, unless otherwise described herein or otherwise clearly understood from the context. Reconstruction filtering is separate and distinct from transform-size adaptive directional nonlinear filtering. Identifying the transform corresponding to the rate-distortion optimization metric or cost as the optimal transform for code processing of the current block is performed before and independently of reconstruction filtering.

[0136] The output, compressed or encoded bitstream is output (such as stored or transmitted) to a decoder (at 830).

[0137] In some implementations, the encoder includes data such as bits, flags, or other symbols that indicate whether nonlinear transform filtering (such as using a transform-size adaptive directional nonlinear filter) is enabled.

[0138] In some implementations, the encoder includes data such as bits, flags, or other symbols that indicate the disabling of nonlinear transform filtering (such as using a transform-size adaptive directional nonlinear filter).

[0139] Figure 9 This is a flowchart illustrating an example of a rate-distortion optimized decoder 900, including an inverse transform incorporating a transform-size adaptive directional nonlinear filter, according to an implementation of this disclosure. The rate-distortion optimized decoder 900, including an inverse transform incorporating a transform-size adaptive directional nonlinear filter, can be used by, for example... Figure 5 The decoder implementation of decoder 500 is shown in the figure. Decoder 900, which uses rate-distortion optimization including an inverse transform incorporating transform-sized adaptive directional nonlinear filtering, includes block-based hybrid video code processing as described herein.

[0140] Decoding 900, which uses rate-distortion optimization including an inverse transform incorporating transform-sized adaptive oriented nonlinear filtering, includes methods such as... Figure 5 The compressed bitstream 502 shown in the figure is decoded, along with one or more portions thereof, to generate reconstructed video data, such as... Figure 5 The output video stream 504 shown is a reconstructed video or a portion thereof.

[0141] Decoding an encoded bitstream or one or more portions thereof using a rate-distortion optimized decoder 900, which includes an inverse transform incorporating transform-size adaptive directional nonlinear filtering, includes obtaining the encoded bitstream (at 910), obtaining the transform data (at 920), performing the inverse transform (at 930), and outputting the reconstructed video data (at 940). For simplicity and brevity, one or more aspects of using the rate-distortion optimized decoder 900, which includes an inverse transform incorporating transform-size adaptive directional nonlinear filtering, may be omitted from the description herein.

[0142] Obtain the encoded bitstream (at 910). For example, the decoder or its components (such as the decoder's intra / inter prediction unit, etc.) Figure 5 The entropy decoding unit 510 shown can obtain an encoded bitstream. Obtaining the encoded bitstream (at 910) includes identifying the current frame from the current frame sequence to decode from the encoded bitstream to generate the current reconstructed frame. Obtaining the encoded bitstream (at 910) includes identifying the current block from the current frame. Obtaining the encoded bitstream (at 910) includes obtaining the encoded block data of the current block from the encoded bitstream.

[0143] Transform data for inverse transform obtained from the encoded bitstream (at 920) such as one or more transform parameters, such as transform type data indicating the type of transform, transform size data indicating the size of transform, or a combination thereof.

[0144] Dequantizing the encoded block data to obtain dequantized transform coefficients (c), such as a dequantized transform block or dequantized transform block data. Obtaining dequantized transform block data may include entropy decoding the encoded block data to obtain quantized transform coefficients (such as a quantized transform block or quantized transform block data) and dequantizing the quantized transform block data.

[0145] although Figure 9 Not explicitly shown, but using rate-distortion optimized decoding 900, which includes an inverse transform incorporating adaptive directional nonlinear filtering with transform size, includes obtaining the prediction block data for the current block. p ).

[0146] Decoded residual block data ( r ) is obtained by performing an inverse transform on the dequantized transform block data (at 930) based on the transform data (obtained at 920), such as by performing an inverse transform as identified by the transform data (obtained at 920). T ) Applied to dequantized transform block data ( r=Tc ).

[0147] The inverse transform of the dequantized transform block data (at 930) involves combining the decoded residual block data with the prediction block data of the current block. q = p + r To obtain the intermediate decoded block data ( q ).

[0148] Inverse transform of the dequantized transform block data (at 930) includes obtaining a transform-size adaptive directional nonlinear filter, such as in Figure 8 The optimal transform size adaptive directional nonlinear filter (marked at 840) N (.)). The decoder obtains a transform-size adaptive directional nonlinear filter based on the transform data (obtained at 920). In some implementations, the transform data (obtained at 920) may include data, such as bits, flags, or other symbols, indicating whether nonlinear transform filtering (such as using a transform-size adaptive directional nonlinear filter) is enabled. In response to determining whether the data indicating whether nonlinear transform filtering (such as using a transform-size adaptive directional nonlinear filter) is enabled is disabled, the inverse transform of the dequantized transform block data (at 930) omits, skips, avoids, or excludes filtering using the transform-size adaptive directional nonlinear filter.

[0149] Inverse transformation of the dequantized transform block data (at 930) involves using a transform-size adaptive directional nonlinear filter. N (.)) for the decoded block data in the middle ( q ) is filtered to obtain the decoded block data. d This can be represented as ( d = N(q) ).

[0150] although Figure 9 It is not explicitly shown, but the reconstructed block data is obtained by analyzing the decoded block data. d The reconstruction is achieved through reconstructive filtering, which may include loop filtering, deblocking filtering, or other types of filtering, or a combination of filter types. Reconstructive filtering and... Figure 4 The filter shown (at 480) Figure 5The filter shown (at 560) or Figure 7 The filtering shown (at 770) is similar, unless otherwise described herein or otherwise clear from the context. Reconstruction filtering is separate and distinct from transform-sized adaptive directional nonlinear filtering.

[0151] The decoder obtains the reconstructed frame data for the current frame. To obtain the reconstructed frame data, the decoder includes the reconstructed block data in the reconstructed frame data for the current frame.

[0152] The decoder outputs the reconstructed frame data (at 940).

[0153] Figure 10 This is a block diagram of an encoder 1000 that implements transform selection via nonlinear inverse transform according to the implementation of this disclosure. The encoder 1000 that implements transform selection via nonlinear inverse transform can be used in, for example... Figure 1 The computing device 100 shown in the figure or Figure 2 The computing and communication devices 100A, 100B, and 100C shown are implemented, for example, stored in a storage device such as... Figure 1 The computer software program in the data storage unit of the memory 110 shown herein. This computer software program may include programs that can be generated by, for example... Figure 1 The processor 120 shown executes machine instructions and can cause the device to encode video data as described herein. The encoder 1000, which performs transform selection via a nonlinear inverse transform, can be implemented as dedicated hardware, for example, included in the computing device 100. The encoder 1000, which performs transform selection via a nonlinear inverse transform, and... Figure 4 The encoder 400 shown is similar, unless otherwise described herein or clearly understood from the context. The encoder 1000, which achieves transform selection via a nonlinear inverse transform, is similar to... Figure 7 The encoder 700 shown is similar to the one that implements transform selection, unless otherwise described herein or otherwise clearly understood from the context.

[0154] The encoder 1000, which implements transform selection through nonlinear inverse transform, achieves the following: Figure 8 The usage shown includes rate-distortion optimized coding 800 that incorporates an inverse transform of a transform-sized adaptive directional nonlinear filter.

[0155] Encoder 1000, which achieves transform selection through nonlinear inverse transform, pairs such as... Figure 3The input video stream 1002 of the video stream 300 shown is encoded to generate an encoded (compressed) bitstream 1004. In some implementations, the encoder 1000, which implements transform selection via a nonlinear inverse transform, includes a forward path for generating the compressed bitstream 1004. The forward path includes an intra / inter-frame prediction unit 1010, a transform unit 1020, a quantization unit 1030, an entropy coding unit 1040, or any combination thereof. In some implementations, the encoder 1000, which implements transform selection via a nonlinear inverse transform, includes a reconstruction path (indicated by a dotted connection line) for reconstructing frames for encoding further blocks. The reconstruction path includes a dequantization unit 1050, a nonlinear inverse transform unit 1060, a filtering unit 1070, or any combination thereof. Other structural variations of the encoder 1000, which implements transform selection via a nonlinear inverse transform, can be used to encode the video stream 1002.

[0156] To encode video stream 1002, each frame within video stream 1002 can be processed in blocks. Therefore, the current block can be identified from the blocks within a frame, and the current block can be encoded. Identifying the current block can be similar to... Figure 8 The input block is shown in Figure 810.

[0157] Intra / Inter-frame Prediction Unit 1010 and Figure 7 The intra / inter-frame prediction unit 710 shown is similar, unless otherwise described herein or otherwise clearly understood from the context. Intra-frame prediction involves generating a prediction block based on previously encoded and reconstructed samples in the current frame. Inter-frame prediction involves generating a prediction block (prediction block data) based on samples from one or more previously constructed reference frames. The intra / inter-frame prediction unit 1010 subtracts the prediction block from the current block (original block) to produce a residual block (residual block data).

[0158] Transformation unit 1020 and Figure 7 The transform unit 720 shown is similar, unless otherwise described herein or otherwise apparent from the context. Transform unit 1020 performs a block-based transform, which includes transforming the residual block into transform coefficients, for example, in the frequency domain.

[0159] Transformation unit 1020 selects, determines, identifies, or obtains the optimal transform from the available transforms for transforming residual block data. Identifying the optimal transform includes identifying the optimal transform type from the available transform types and the optimal transform size from the available transform sizes. Examples of available transform types include Karhunen-Loève transform (KLT), discrete cosine transform (DCT), singular value decomposition transform (SVD), and asymmetric discrete sine transform (ADST). The transform type can be indicated by a transform type identifier. Examples of available transform sizes include 4×4, 4×8, 8×4, 4×16, 16×4, 8×8, 8×16, 16×8, 8×32, 32×8, 16×16, 16×32, 32×16, 16×64, 64×16, 32×32, 32×64, 64×32, or 64×64. Available transforms can be combinations or tuples of available transform types and available transform sizes.

[0160] Transformation unit 1020 selects, determines, identifies, or obtains the optimal transformation by evaluating available transformations (such as two or more available transformations), including the optimal transformation type, the optimal transformation size, or both. Evaluating the corresponding available transformations includes evaluating the available transformation type and the available transformation size (n×m). Obtaining the optimal transformation can be similar to... Figure 8 Rate distortion optimization is shown (at 840).

[0161] Evaluating the available transformations involves transforming the residual block data using the available transformations to obtain candidate transform block data.

[0162] Evaluating the available transforms involves obtaining candidate quantized block data by quantizing the candidate transform block data using quantization unit 1030. Quantization unit 1030 and... Figure 7 The quantization unit 730 shown is similar, unless otherwise described herein or otherwise clearly understood from the context.

[0163] Evaluating the available transforms involves obtaining candidate dequantized transform block data by dequantizing the candidate quantized block data using the dequantization unit 1050. c ). Dequantization unit 1050 and Figure 7 The dequantization unit 750 shown is similar, unless otherwise described herein or otherwise clear from the context.

[0164] The evaluation of the corresponding available transformations includes using the corresponding available transformations via the nonlinear inverse transformation unit 1060. r= Tc The corresponding inverse transform ( T The candidate dequantized transform block data is inversely transformed to obtain the candidate decoded residual block data. rNonlinear inverse transform unit 1060 and Figure 7 The inverse transform unit 760 shown is similar, unless otherwise described herein or clearly understood from the context. The inverse transform can be similar to... Figure 8 The inverse transform shown (at 860).

[0165] The evaluation of the available transformations includes combining the prediction block data through the nonlinear inverse transformation unit 1060. p ) and candidate decoded residual block data ( r To obtain the intermediate decoded block data ( q ) ( q = p + r ).

[0166] Evaluating the available transforms includes using an adaptive directional nonlinear filter with candidate transform size ( N (.) For the decoded block data in the middle ( q ) is filtered to obtain candidate decoded block data ( d ), where the candidate transform-size adaptive directional nonlinear filter is an adaptive directional nonlinear filter having a transform size equal to or matching that of the corresponding available transform, which can be expressed as ( d = N(q) ).

[0167] The evaluation of the appropriate available transformations includes evaluating candidate decoded block data obtained using the appropriate available transformations. d = N(q) This involves using adaptive directional nonlinear filters of corresponding candidate transform sizes to obtain one or more rate-distortion optimization metrics, such as distortion metrics, coding cost metrics (code rate), or both, for encoding the current block using the appropriate available transforms. For example, obtaining the distortion metric could include determining candidate decoded block data (...). d The difference between the current block of data and the data in the current block, such as the sum of absolute differences (SAD).

[0168] Evaluating available transformations (such as evaluating two or more available transformations) includes transformation unit 1020 obtaining the rate-distortion optimization cost of each available transformation on a per-transformation basis. Evaluating available transformations (such as evaluating two or more available transformations) includes transformation unit 1020 comparing one or more rate-distortion optimization metrics obtained for a first available transformation (such as a corresponding available transformation) with one or more equivalent rate-distortion optimization metrics obtained for a second available transformation, as indicated by the directed dashed line at 1062.

[0169] Transformation unit 1020 selects, determines, identifies, or obtains the available transforms corresponding to one or more minimum rate-distortion optimization metrics as the optimal transform. For example, in response to determining that a corresponding available transform corresponds to the minimum rate-distortion optimization cost in the rate-distortion optimization cost, transformation unit 1020 selects, determines, identifies, or obtains that corresponding available transform as the optimal transform for code processing of the current block.

[0170] Entropy coding unit 1040 performs entropy coding on the quantized transform coefficients obtained using the optimal transform to produce entropy-coded coefficients. Entropy coding unit 1040 and... Figure 4 The entropy coding unit 440 shown is similar, unless otherwise described herein or clearly understood from the context. The entropy-coded coefficients and information used for decoding the block can be output to the compressed bitstream 1004, which may include the prediction type used, motion vectors, and quantizer values. The compressed bitstream 1004 is similar to... Figure 4 The compressed bitstream shown is similar to 404, unless otherwise described herein or otherwise clearly understood from the context.

[0171] Filtering unit 1070 processes the decoded block data obtained using an adaptive directional nonlinear filter with the optimal transform and the corresponding candidate transform size. d = N(q) Reconstruction filtering is performed to generate reconstructed blocks, which reduces distortion such as blockage artifacts. Filtering unit 1070 and... Figure 7 The filter unit 770 shown is similar, unless otherwise described herein or otherwise clear from the context. Although Figure 10 A filtering unit 1070 is shown, but filtering the decoded block includes loop filtering, deblocking filtering, or other types of filtering or combinations of multiple filtering types. The reconstructed block can be stored or otherwise made accessible as a reconstructed block, which can be part of a reference frame for encoding another part of the current frame, another frame, or both, as indicated by the dashed line at 1072. Code processing information of the frame (such as the deblocking threshold index value) can be encoded, included in the compressed bitstream 1004, or both, as indicated by the dashed line at 1074. The reconstruction filtering applied by filtering unit 1070 is separate and distinct from transform-size adaptive directional nonlinear filtering. The transform corresponding to the minimum rate-distortion optimization metric or cost is identified as the optimal transform for code processing the current block and is performed before and independently of reconstruction filtering. In some implementations, filtering unit 1070 omits, skips, avoids, or excludes the decoded block data ( d = N(q) The filtering unit 1070 performs reconstruction filtering on the intermediate decoded block data ( qReconstruction filtering is performed to generate reconstructed blocks, which can reduce distortions such as block artifacts.

[0172] The compressed bitstream 1004 can be encoded using other variations of the encoder 1000 that achieves transform selection through nonlinear inverse transform.

[0173] Figure 11 This is a block diagram of a decoder 1100 that implements transform selection via nonlinear inverse transform according to the implementation of this disclosure. The decoder 1100 that implements transform selection via nonlinear inverse transform can be used in, for example... Figure 1 The computing device 100 shown in the figure or Figure 2 The computing and communication devices 100A, 100B, and 100C shown are implemented, for example, stored in a storage device such as... Figure 1 The computer software program in the data storage unit of the memory 110 shown herein. This computer software program may include programs that can be generated by, for example... Figure 1 The processor 120 shown executes machine instructions and can cause the device to decode video data as described herein. The decoder 1100, which performs transform selection via a nonlinear inverse transform, can be implemented as dedicated hardware, for example, included in the computing device 100. The decoder 1100, which performs transform selection via a nonlinear inverse transform, can be used with… Figure 5 The decoder shown is similar to the 500, unless otherwise described herein or otherwise clearly understood from the context.

[0174] The decoder 1100 implements transform selection via nonlinear inverse transform, as shown in the example... Figure 9 The usage shown includes rate-distortion optimized decoding 900 that incorporates an inverse transform with adaptive directional nonlinear filtering of transform size.

[0175] Decoder 1100, which performs transform selection via nonlinear inverse transform, receives compressed bitstream 1102, such as... Figure 4 The compressed bitstream 404 is shown. The decoder 1100, which performs transform selection via nonlinear inverse transform, decodes the compressed bitstream 1102 to generate the output video stream 1104. This includes decoding the encoded block data from the compressed bitstream 1102 to generate the decoded block data of the current block of the current frame.

[0176] The decoder 1100, which implements transform selection via nonlinear inverse transform, includes an entropy decoding unit 1110, a dequantization unit 1120, an intra / inter-frame prediction unit 1130, a nonlinear inverse transform unit 1140, a filtering unit 1150, or any combination thereof. Other structural variations of the decoder 1100, which implements transform selection via nonlinear inverse transform, can be used to decode the compressed bitstream 1102.

[0177] Entropy decoding unit 1110 uses, for example, context-adaptive binary arithmetic decoding to decode data elements (such as encoded block data of the current block) within the compressed bitstream 1102 to obtain quantized transform coefficients (quantized transform block data).

[0178] The decoder 1100 or its components (such as the entropy decoding unit 1110) that performs transform selection via nonlinear inverse transform obtains the transform data of the current block from the encoded bitstream 1102. The transform data indicates the transform (or transform tuple), which indicates the transform type and transform size.

[0179] Dequantization unit 1120 dequantizes the quantized transform coefficients to obtain dequantized transform coefficients (dequantized transform block data). Dequantization unit 1120 and... Figure 5 The dequantization unit 520 shown is or Figure 10 The dequantization unit 1050 shown is similar, unless otherwise described herein or otherwise clear from the context.

[0180] Using the header information decoded from the compressed bitstream 1102, the intra / inter-frame prediction unit 1130 generates a sequence of data that is in the format specified in the original text. Figure 10 The prediction block (prediction block data) shown corresponds to the prediction block created in the encoder 1000, which implements transform selection via nonlinear inverse transform. The intra / inter-frame prediction unit 1130 and... Figure 10 The intra / inter-frame prediction unit 1040 shown is similar, unless otherwise described herein or otherwise clearly understood from the context.

[0181] The nonlinear inverse transform unit 1140 performs an inverse transform on the dequantized transform coefficients according to transform data (such as using an inverse transform (transform type and transform size) indicated by the transform data) to generate or obtain a derived residual block (decoded residual block data), which may correspond to the dequantized transform coefficients. Figure 10 The decoded residual block data generated by the nonlinear inverse transform unit 1060 shown.

[0182] The nonlinear inverse transform unit 1140 obtains intermediate decoded block data by combining (such as adding) the predicted block data of the current block and the decoded residual block data.

[0183] The nonlinear inverse transform unit 1140 obtains a transform-sized adaptive directional nonlinear filter with a size corresponding to, matching, or equal to the transform size, such as a filter corresponding to the transform size. Figure 10 The transformation unit 1020 shown is an adaptive directional nonlinear filter for the candidate transform size corresponding to the best transform of the current block identifier. N(.) ).

[0184] The nonlinear inverse transform unit 1140 obtains the decoded block data by filtering the intermediate decoded block data using a transform-size adaptive directional nonlinear filter. d ).

[0185] The reconstruction filtering unit 1150 can process the decoded block data ( d One or more reconstruction filters are applied to reduce artifacts such as blockiness. This may include loop filtering, deblocking filtering, or other types of reconstruction filtering, or a combination of multiple reconstruction filtering types. It may also include generating reconstructed blocks (reconstructed block data) that can be output as or included in the output video stream 1104. For example, a decoder 1100 that implements transform selection via a nonlinear inverse transform can include the reconstructed block data in the reconstructed frame data of the current frame, and can also include the reconstructed frame data in the output video stream 1104. The reconstruction filtering performed by the reconstruction filtering unit 1150 is separate from and different from the transform-size adaptive directional nonlinear filtering implemented by the nonlinear inverse transform unit 1140.

[0186] Other variations of the decoder 1100, which implements transform selection through nonlinear inverse transform, can be used to decode the compressed bit stream 1102.

[0187] As used herein, the terms “best,” “optimized,” “optimized,” or other forms thereof are relative to the appropriate context and do not indicate absolute theoretical optimization unless expressly stated herein.

[0188] As used herein, the term “set” refers to a distinguishable collection or grouping of zero or more distinct elements or members, which may be represented as a one-dimensional array or vector, unless explicitly described herein or otherwise clear from the context.

[0189] The terms “example” or “exemplary” are used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “example” or “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs. Rather, the use of the terms “example” or “exemplary” is intended to present concepts in a specific manner. As used herein, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or clearly apparent from the context, “X comprises A or B” is intended to mean any of the natural inclusive arrangements. In other words, “X comprises A or B” is satisfied under any of the foregoing examples if X comprises A; X comprises B; or X comprises both A and B. Additionally, the articles “a” and “an” as used herein and in the appended claims should generally be interpreted as meaning “one or more” unless otherwise specified or clearly apparent from the context. Furthermore, the use of the terms “embodiment” or “an embodiment” or “an implementation” throughout the document is not intended to represent the same embodiment or implementation unless so described. As used herein, the terms “identify” and “identify”, or any variations thereof, include the use of… Figure 1 One or more of the devices shown may in any way select, detect, calculate, search, receive, determine, establish, obtain, or otherwise identify or determine.

[0190] Furthermore, for the sake of simplicity, although the figures and descriptions herein may include a series of steps or stages, the elements of the methods disclosed herein may occur in various orders and / or simultaneously. Additionally, elements of the methods disclosed herein may appear together with other elements not explicitly presented or described herein. Furthermore, one or more elements of the methods described herein may be omitted from the implementation of the methods according to the disclosed subject matter.

[0191] The implementation of the transmitting computing and communication device 100A and / or the receiving computing and communication device 100B (as well as the algorithms, methods, instructions, etc. stored on and / or executed by these devices) can be implemented in hardware, software, or any combination thereof. Hardware may include, for example, a computer, intellectual property (IP) core, application-specific integrated circuit (ASIC), programmable logic array, optical processor, programmable logic controller, microcode, microcontroller, server, microprocessor, digital signal processor, or any other suitable circuitry. In the claims, the term "processor" should be understood to cover any of the foregoing hardware individually or in combination. The terms "signal" and "data" are used interchangeably. Furthermore, the parts of the transmitting computing and communication device 100A and the receiving computing and communication device 100B need not be implemented in the same manner.

[0192] Furthermore, in one implementation, for example, a computer program may be used to implement the transmitting computing and communication device 100A or the receiving computing and communication device 100B, which, when executed, performs any of the corresponding methods, algorithms, and / or instructions described herein. Alternatively or alternatively, for example, a dedicated computer / processor may be utilized, which may include dedicated hardware for performing any of the methods, algorithms, or instructions described herein.

[0193] The transmitting computing and communication device 100A and the receiving computing and communication device 100B can be implemented, for example, on a computer in a real-time video system. Alternatively, the transmitting computing and communication device 100A can be implemented on a server, and the receiving computing and communication device 100B can be implemented on a device separate from the server (such as a handheld communication device). In this example, the transmitting computing and communication device 100A can use an encoder 400 to encode content into an encoded video signal and transmit the encoded video signal to the communication device. The communication device can then use a decoder 500 to decode the encoded video signal. Alternatively, the communication device can decode content stored locally on the communication device (e.g., content not transmitted by the transmitting computing and communication device 100A). Other suitable implementations of the transmitting computing and communication device 100A and the receiving computing and communication device 100B are available. For example, the receiving computing and communication device 100B can be a generally fixed personal computer instead of a portable communication device, and / or the device including the encoder 400 can also include the decoder 500.

[0194] Furthermore, the implementation may take, in whole or in part, the form of a computer program product accessible from, for example, a tangible computer-usable or computer-readable medium. A computer-usable or computer-readable medium may be any means capable of, for example, tangibly containing, storing, transmitting, or transporting the program for use by or in conjunction with any processor. The medium may be, for example, an electronic, magnetic, optical, electromagnetic, or semiconductor device. Other suitable media may also be used.

[0195] It should be understood that the aspects can be implemented in any convenient form. For example, the aspects can be implemented by a suitable computer program that can be carried on a suitable carrier medium, which can be a tangible carrier medium (e.g., a disk) or an intangible carrier medium (e.g., a communication signal). The aspects can also be implemented using suitable devices, which can take the form of a programmable computer running a computer program arranged to implement the methods and / or techniques disclosed herein. The aspects can be combined such that features described in the context of one aspect can be implemented in another aspect.

[0196] The above implementations have been described for ease of understanding of this application, but are not restrictive. Rather, this application covers various modifications and equivalent arrangements included within the scope of the appended claims, which should be interpreted in the broadest possible sense to cover all such modifications and equivalent structures permitted by law.

Claims

1. A method comprising: Encoded block data is generated by encoding the current block from the current frame of the input video, wherein encoding the current block includes: Obtain residual block data indicating the difference between the current block and the predicted block data of the current block; and The optimal transformation is identified from the available transformations by evaluating two or more available transformations for transforming the residual block data, wherein identifying the optimal transformation includes identifying the optimal transformation type and the optimal transformation size, and wherein evaluating the corresponding available transformations includes: Candidate transform block data are obtained by transforming the residual block data using the corresponding available transforms; Candidate quantized block data is obtained by quantizing the candidate transform block data; Candidate dequantized transform block data is obtained by dequantizing the candidate quantized block data; and Candidate decoded residual block data is obtained by performing an inverse transform on the candidate dequantized transform block data, wherein performing an inverse transform on the candidate dequantized transform block data includes: Intermediate decoded block data is obtained by combining the predicted block data and the candidate decoded residual block data; and Candidate decoded block data are obtained by filtering the intermediate decoded block data using an optimal transform-size adaptive directional nonlinear filter. Include the candidate quantized block data in the encoded bitstream; and Output the encoded bit stream.

2. The method of claim 1, wherein evaluating two or more of the available transformations comprises: The rate-distortion optimization cost of each available transformation is obtained based on that available transformation; as well as In response to determining that the corresponding available transformation corresponds to the minimum rate-distortion optimization cost in the rate-distortion optimization cost, the corresponding available transformation is identified as the optimal transformation for coding the current block.

3. The method of claim 1, wherein identifying the optimal transformation comprises: Identify the optimal transformation type; as well as Indicate the optimal transformation size.

4. The method of claim 1, wherein performing an inverse transform on the candidate dequantized transform block data comprises: The optimal transform-size adaptive directional nonlinear filter is obtained.

5. The method of claim 1, further comprising: Reconstructed block data is obtained by performing reconstruction filtering on the candidate decoded block data; The reconstructed block data is included in the reconstructed frame data of the current frame; as well as The reconstructed frame data is stored so that another frame from the input video can be encoded subsequently.

6. The method of claim 1, further comprising: Reconstructed block data is obtained by performing reconstruction filtering on the intermediate decoded block data; The reconstructed block data is included in the reconstructed frame data of the current frame; as well as The reconstructed frame data is stored so that another frame from the input video can be encoded subsequently.

7. The method of claim 1, wherein including the candidate quantized block data in the encoded bitstream comprises: The data indicating the use of a transform-size adaptive directional nonlinear filter will be included in the encoded bitstream.

8. A method comprising: Decoded block data is generated by decoding the current block of the current frame, wherein decoding the current block includes: The quantized transform coefficients of the current block are obtained from the encoded bitstream; The transform data of the current block is obtained from the encoded bitstream, wherein the transform data indicates the transform type and transform size; Dequantized transform block data is obtained by dequantizing the quantized transform coefficients; and Decoded residual block data is obtained by performing an inverse transformation on the dequantized transform block data based on the transformed data, wherein the inverse transformation on the dequantized transform block data includes: Intermediate decoded block data is obtained by combining the predicted block data of the current block and the decoded residual block data; and The decoded block data is obtained by filtering the intermediate decoded block data using a transform-size adaptive directional nonlinear filter; and Output the decoded block data.

9. The method of claim 8, wherein outputting the decoded block data comprises: Reconstructed block data is obtained by performing reconstruction filtering on the decoded block data; The reconstructed block data is included in the reconstructed frame data of the current frame; and Output the reconstructed frame data.

10. The method of claim 8, wherein obtaining the transformed data comprises: Obtain transformation type data indicating the transformation type; as well as Obtain transformation size data that indicates the size of the transformation.

11. The method of claim 8, wherein performing an inverse transform on the dequantized transform block data comprises: The transform-size adaptive directional nonlinear filter is obtained based on the transform data.

12. The method of claim 8, wherein performing an inverse transform on the dequantized transform block data includes... Accessing data from the encoded bitstream indicates the use of a transform-size adaptive directional nonlinear filter.

13. The method according to claim 8, wherein: In response to accessing data from the encoded bitstream that indicates disabling the use of a transform-sized adaptive directional nonlinear filter: Performing an inverse transform on the dequantized transform block data omits filtering the intermediate decoded block data using a transform-size adaptive directional nonlinear filter; and The output of the decoded block data includes: Reconstructed block data is obtained by performing reconstruction filtering on the intermediate decoded block data; The reconstructed block data is included in the reconstructed frame data of the current frame; and Output the reconstructed frame data.

14. An apparatus comprising: Non-transitory computer-readable medium; as well as A processor, configured to execute instructions stored on the non-transitory computer-readable medium, to: Generate encoded block data, wherein, in order to generate the encoded block data, the processor is configured to execute the instructions to encode the current block from the current frame of the input video, wherein, in order to encode the current block, the processor is configured to execute the instructions to: Obtain residual block data indicating the difference between the current block and the predicted block data of the current block; and Identify the optimal transformation from the available transformations for transforming the residual block data, wherein, in order to identify the optimal transformation, the processor is configured to execute the instructions to evaluate two or more available transformations, wherein the optimal transformation includes an optimal transformation type and an optimal transformation size, wherein, in order to evaluate the corresponding available transformation, the processor is configured to execute the instructions to: To obtain candidate transform block data, the processor is configured to execute the instructions to transform the residual block data using the corresponding available transforms. To obtain candidate quantized block data, the processor is configured to execute the instructions to quantize the candidate transform block data. Obtaining candidate dequantized transform block data, wherein, in order to obtain the candidate dequantized transform block data, the processor is configured to execute the instructions to dequantize the candidate quantized block data; and Obtain candidate decoded residual block data, wherein, in order to obtain the candidate decoded residual block data, the processor is configured to execute the instructions to perform an inverse transform on the candidate dequantized transform block data, wherein, in order to perform an inverse transform on the candidate dequantized transform block data, the processor is configured to execute the instructions to: Obtaining intermediate decoded block data, wherein, in order to obtain the intermediate decoded block data, the processor is configured to execute the instructions to combine the predicted block data and the candidate decoded residual block data; and To obtain candidate decoded block data, the processor is configured to execute the instructions to filter the intermediate decoded block data using an optimal transform-size adaptive oriented nonlinear filter. Include the candidate quantized block data in the encoded bitstream; and Output the encoded bit stream.

15. The device as claimed in claim 14, wherein, In order to evaluate two or more of the available transformations, the processor is configured to execute the instructions to: The rate-distortion optimization cost of each available transformation is obtained based on that available transformation; as well as In response to determining that the corresponding available transformation corresponds to the minimum rate-distortion optimization cost in the rate-distortion optimization cost, the corresponding available transformation is identified as the optimal transformation for coding the current block.

16. The apparatus of claim 14, wherein, in order to identify the optimal transformation, the processor is configured to execute the instructions to: Identify the optimal transformation type; and Indicate the optimal transformation size.

17. The device as claimed in claim 14, wherein, In order to perform an inverse transform on the candidate dequantized transform block data, the processor is configured to execute the instructions to: The optimal transform-size adaptive directional nonlinear filter is obtained.

18. The apparatus of claim 14, wherein the processor is configured to execute the instructions to obtain reconstructed block data, wherein, The processor is configured to execute the instructions to: The candidate decoded block data is reconstructed and filtered to obtain the reconstructed block data; The reconstructed block data is included in the reconstructed frame data of the current frame; as well as The reconstructed frame data is stored so that another frame from the input video can be encoded subsequently.

19. The apparatus of claim 14, wherein the processor is configured to execute the instructions to obtain reconstructed block data, wherein, The processor is configured to execute the instructions to: The intermediate decoded block data is reconstructed and filtered to obtain the reconstructed block data; The reconstructed block data is included in the reconstructed frame data of the current frame; as well as The reconstructed frame data is stored so that another frame from the input video can be encoded subsequently.

20. The device of claim 14, wherein the processor is configured to execute the instructions to: The data indicating the use of a transform-size adaptive directional nonlinear filter will be included in the encoded bitstream.

Images