Motion search for reference block selection

Abstract

A method of encoding video data, comprising: determining integer samples in a reference picture of the video data; determining at least a first fractional sample and a second fractional sample based on the integer sample, where the first fractional sample has a first fractional pixel resolution and the second fractional sample has a second fractional pixel resolution different from the first fractional pixel resolution; after determining both the first fractional sample and the second fractional sample, determining a first cost metric associated with the first fractional sample and a second cost metric associated with the second fractional sample; determining a reference block of the current block based on at least one of the first cost metric or the second cost metric; and encoding the current block based on the reference block.

Description

[0001] This application claims priority to U.S. patent application serial number 17 / 934,508, filed on September 22, 2022, the entire contents of which are hereby incorporated by reference. Technical Field

[0002] The present disclosure relates to video encoding and video decoding. Background Art

[0003] Digital video capabilities may be incorporated into a variety of devices, including digital televisions, digital live broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-book readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite wireless telephones, so-called "smart phones", video teleconferencing devices, video streaming devices, etc. Digital video devices implement video codec technologies, such as those described in the standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4, Part 10, Advanced Video Codec (AVC), ITU-T H.265 / High Efficiency Video Codec (HEVC), ITU-T H.266 / Versatile Video Codec (VVC), and extensions of such standards, as well as proprietary video codecs / formats such as AOMedia Video 1 (AV1) developed by the Alliance for Open Media. Video devices may more efficiently transmit, receive, encode, decode, and / or store digital video information by implementing such video codec technologies.

[0004] Video coding techniques include spatial (intra-picture) prediction and / or temporal (inter-picture) prediction to reduce or eliminate redundancy inherent in video sequences. For block-based video coding, a video slice (e.g., a video picture or a portion of a video picture) may be divided into video blocks, which may also be referred to as codec tree units (CTUs), codec units (CUs), and / or codec nodes. For video blocks in an intra-frame codec (I) slice of a picture, spatial prediction relative to reference samples in neighboring blocks in the same picture is used for encoding. For video blocks in an inter-frame codec (P or B) slice of a picture, spatial prediction relative to reference samples in neighboring blocks in the same picture or temporal prediction relative to reference samples in other reference pictures may be used. Pictures may be referred to as frames, and reference pictures may be referred to as reference frames. Summary of the invention

[0005] In general, the present disclosure describes techniques for fractional search to identify samples to be used to generate a reference block. For example, a video encoder may identify integer samples. The video encoder may determine which fractional samples of a plurality of fractional samples to determine a corresponding cost metric for. An example of a cost metric is a distortion value (e.g., indicating a difference between a reference block containing the fractional samples and a current block). For example, a video encoder may determine at least a first fractional sample and a second fractional sample based on the integer samples (e.g., which are used to determine a corresponding cost metric). For example, a video encoder may identify a set of fractional samples of a plurality of fractional samples for which a corresponding cost metric is to be determined, wherein the set of fractional samples includes a first fractional sample and a second fractional sample.

[0006] The determined fractional samples (e.g., a set of fractional samples) include samples having different fractional pixel resolutions. For example, a first fractional sample has a first fractional pixel resolution, and a second fractional sample has a second fractional pixel resolution different from the first fractional pixel resolution. For example, the first fractional sample has a half-pixel resolution, and the second fractional sample has a quarter-pixel resolution.

[0007] After determining all fractional samples for which corresponding cost metrics are to be determined, the video encoder can determine the corresponding cost metrics (e.g., in one stage). For example, after determining a first fractional sample and a second fractional sample, the video encoder can determine a first cost metric associated with the first fractional sample and a second cost metric associated with the second fractional sample. That is, compared to other techniques, the video encoder can determine the corresponding cost metrics for both half-pixel samples and quarter-pixel samples in the same stage, while in other techniques, the video encoder generates a cost metric for a half-pixel sample, followed by a cost metric for quarter-pixel samples surrounding the best half-pixel sample. Based on the evaluation, the video encoder can determine a motion vector for a current block and a reference block for the current block.

[0008] In one or more examples, the example technique can be considered a single (i.e., one) stage fractional search because the fractional search includes samples at different resolutions, as compared to iterative techniques that iteratively search for samples at different resolutions. By determining all fractional samples for which corresponding cost metrics are to be determined, including fractional samples having different resolutions (e.g., identifying first and second fractional samples having different fractional pixel resolutions), and then determining the cost metric, the processing time required to determine a motion vector for a current block and a reference block can be reduced compared to techniques that iteratively determine the cost metric.

[0009] In one example, a method for encoding video data includes: determining integer samples in a reference picture of the video data; determining at least a first fractional sample and a second fractional sample based on the integer samples, wherein the first fractional sample has a first fractional pixel resolution and the second fractional sample has a second fractional pixel resolution different from the first fractional pixel resolution; after determining both the first fractional sample and the second fractional sample, determining a first cost metric associated with the first fractional sample and a second cost metric associated with the second fractional sample; determining a reference block of a current block based on at least one of the first cost metric or the second cost metric; and encoding the current block based on the reference block.

[0010] In another example, a device for encoding video data includes: a memory configured to store video data; and one or more processors implemented in a circuit, coupled to the memory and configured to: determine integer samples in a reference picture of the video data; determine at least a first fractional sample and a second fractional sample based on the integer samples, wherein the first fractional sample has a first fractional pixel resolution and the second fractional sample has a second fractional pixel resolution different from the first fractional pixel resolution; after determining both the first fractional sample and the second fractional sample, determine a first cost metric associated with the first fractional sample and a second cost metric associated with the second fractional sample; determine a reference block for a current block based on at least one of the first cost metric or the second cost metric; and encode the current block based on the reference block.

[0011] In another example, a computer-readable storage medium having instructions stored thereon, which when executed causes one or more processors to: determine integer samples in a reference picture of video data; determine at least a first fractional sample and a second fractional sample based on the integer samples, wherein the first fractional sample has a first fractional pixel resolution and the second fractional sample has a second fractional pixel resolution different from the first fractional pixel resolution; after determining both the first fractional sample and the second fractional sample, determine a first cost metric associated with the first fractional sample and a second cost metric associated with the second fractional sample; determine a reference block for a current block based on at least one of the first cost metric or the second cost metric; and encode the current block based on the reference block.

[0012] In another example, an apparatus for encoding video data includes: a component for determining integer samples in a reference picture of the video data; a component for determining at least a first fractional sample and a second fractional sample based on the integer samples, wherein the first fractional sample has a first fractional pixel resolution and the second fractional sample has a second fractional pixel resolution different from the first fractional pixel resolution; a component for determining a first cost metric associated with the first fractional sample and a second cost metric associated with the second fractional sample after determining both the first fractional sample and the second fractional sample; a component for determining a reference block of a current block based on at least one of the first cost metric or the second cost metric; and a component for encoding the current block based on the reference block.

[0013] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a block diagram illustrating an example video encoding and decoding system that may perform the techniques of this disclosure.

[0015] Figure 2 is a conceptual diagram showing an example of a search area for integer samples.

[0016] Figure 3A-3C is a conceptual diagram illustrating an example of score samples evaluated for selecting a reference block.

[0017] Figure 4A-4C is an additional conceptual diagram illustrating an example of score samples evaluated for selecting a reference block.

[0018] Figure 5 is a block diagram illustrating an example video encoder that may perform the techniques of this disclosure.

[0019] Figure 6 is a block diagram illustrating an example video decoder that may perform the techniques of this disclosure.

[0020] Figure 7 is a flowchart illustrating an example method for encoding a current block according to the techniques of this disclosure. DETAILED DESCRIPTION

[0021] One example video coding technique is inter-frame prediction. In inter-frame prediction, the video encoder determines a motion vector for the current block being encoded. The motion vector identifies a set of samples in a reference picture. The video encoder uses the set of samples to generate a reference block for the current block and a prediction block from the reference block. The video encoder subtracts the prediction block from the current block to generate a residual value. The video encoder signals information indicating the residual value and information used to determine the motion vector that the video decoder receives. The video decoder generates a prediction block and a reference block based on the motion vector and adds the residual value to the prediction block to reconstruct the current block.

[0022] This disclosure describes example techniques for determining which sample set can be used to generate a reference block for a current block. The technique of determining which sample set to use to generate a reference block is called motion search. Generally speaking, motion search refers to identifying a sample set that forms a block that is closest in value to the current block.

[0023] The samples used to generate the reference block may be integer samples or fractional samples. Integer samples may be samples that already exist in the reference picture. Fractional samples may be samples that do not exist in the reference picture, but are generated by the video encoder using samples that exist in the reference picture. As an example, a half-pixel sample is a sample between two samples that already exist in the reference picture that the video encoder uses to interpolate the half-pixel sample. Fractional samples may be referred to as being at different fractional pixel resolutions.

[0024] In the present disclosure, a fractional sample is referred to as having a particular fractional pixel resolution if the sample grid having the fractional pixel resolution is the coarsest grid on which the fractional sample can be found. For example, a quarter-pixel sample can be found on a quarter-pixel grid, but may not be found on a half-pixel grid. In other words, fractional pixel grids according to the present disclosure may be mutually disjoint. A fractional sample may belong to exactly one fractional pixel grid. The set of all fractional samples at a particular fractional pixel resolution does not include any samples at a different fractional pixel resolution. Likewise, no set of fractional samples includes any integer samples.

[0025] According to one or more examples described in the present disclosure, a video encoder can generate samples with different pixel resolutions. For example, a video encoder can generate samples with half-pixel resolution, quarter-pixel resolution, and / or samples at other resolutions. The video encoder can then evaluate samples with different resolutions. For example, the video encoder can determine which fractional samples in a plurality of fractional samples to determine a corresponding cost metric for. In other words, the video encoder can identify a set of fractional samples in a plurality of fractional samples for which a corresponding cost metric is to be determined.

[0026] An example of a cost metric includes a distortion value. For convenience, the present disclosure describes examples with respect to a "distortion value." However, a distortion value is an example of a cost metric, and the technique should not be considered limited to a "distortion" value. That is, a cost metric may indicate distortion, but the example technique should not be considered limited to requiring that the cost metric indicate distortion.

[0027] The video encoder may determine at least a first fractional sample and a second fractional sample (e.g., the fractional samples are used to determine a corresponding cost metric, such as a distortion value). In this example, the first fractional sample and the second fractional sample are samples from a set of fractional samples from a plurality of fractional samples identified by the video encoder. In this example, the determined fractional samples (e.g., the set of fractional samples) include at least a first fractional sample having a first fractional pixel resolution and a second fractional sample having a second fractional pixel resolution, wherein the first fractional pixel resolution is different from the second fractional pixel resolution.

[0028] After determining all fractional samples for which corresponding cost metrics are to be determined, the video encoder may determine a corresponding cost metric associated with each determined fractional sample (e.g., a set of fractional samples). For example, after determining both a first fractional sample and a second fractional sample, the video encoder may determine a first cost metric associated with the first fractional sample (e.g., a first distortion value) and a second cost metric associated with the second fractional sample (e.g., a second distortion value).

[0029] Evaluating samples with different resolutions (e.g., determining a cost metric for the samples) may mean that the video encoder generates a candidate reference block based on each sample. For example, a first candidate reference block is based on a first sample, a second candidate reference block is based on a second sample, and so on. The first sample and the second sample may be samples with different resolutions (e.g., the first sample is an integer sample and the second sample is a half-pixel sample, or the first sample is a half-pixel sample and the second sample is a quarter-pixel sample). The video encoder may determine a corresponding cost metric indicating the difference between each of the candidate reference blocks and a block (e.g., a current block being encoded or some other block, such as a previously decoded block or a block based on decoded video data). For example, the cost metric (e.g., a distortion value) may be a sum of absolute differences (SAD) value or a sum of absolute transform differences (SATD) value between the block and the samples in the candidate reference block. The video encoder may generate a reference block from a candidate reference block with minimal distortion (e.g., a candidate reference block with minimal cost metric relative to the block).

[0030] For example, the video encoder may determine a cost metric (e.g., a distortion value) from the corresponding cost metrics, the cost metric indicating the lowest amount of distortion relative to the block among the determined cost metrics. The video encoder may determine a candidate reference block associated with the determined cost metric, and determine the reference block based on the determined candidate reference block. For example, the video encoder may set the reference block equal to the determined candidate reference block associated with the cost metric indicating the lowest amount of distortion relative to the current block.

[0031] Thus, in one or more examples, the video encoder may be configured to perform motion search refinement in a single stage of sample generation. As used in the present disclosure, as part of motion search, a sample generation stage (or simply stage) refers to the video encoder determining (e.g., identifying, calculating, selecting, deriving, etc.) for which fractional samples to determine corresponding distortion values. For example, as part of motion search, the determined fractional samples may define candidate reference blocks (e.g., the upper left samples of the candidate reference blocks) that the video encoder evaluates. The distortion values ​​generated in a single (i.e., one) stage may be for corresponding candidate reference blocks with samples of different fractional resolutions, which are identified by corresponding motion vectors of the candidate reference blocks. In other words, according to the present disclosure, distortion values ​​associated with candidate reference blocks with different fractional resolutions may be determined by the video encoder simultaneously (e.g., in parallel (e.g., simultaneously or overlapping in time)). The above and below are described with respect to distortion values, which are examples of cost metrics.

[0032] For example, in some prior art techniques, a video encoder selects a motion vector by first evaluating integer samples in a reference picture (e.g., samples that actually exist without interpolation) to identify integer samples that result in the smallest error (e.g., lowest distortion) when used to generate a reference block. In a first stage, the video encoder determines distortion values ​​of half-pixel samples (e.g., interpolated samples between integer samples) surrounding the identified integer samples to identify the best half-pixel sample (e.g., the best half-pixel sample that results in the lowest distortion if used to generate the reference block). In a second stage, the video encoder determines distortion values ​​of quarter-pixel samples (e.g., interpolated samples between half-pixel samples) surrounding the identified half-pixel sample to identify the best quarter-pixel sample. The video encoder then generates a reference block based on the identified best quarter-pixel sample.

[0033] Existing techniques for motion search (refinement) are iterative and difficult for hardware to perform with a certain degree of parallelism. For example, the video encoder does not start the quarter-pixel search until all half-pixel samples have been evaluated. That is, the video encoder first determines the distortion values ​​for the half-pixel samples, and cannot determine which quarter-pixel samples to determine the distortion values ​​for until the distortion values ​​for the half-pixel samples have been evaluated.

[0034] Some example prior art techniques are multi-stage techniques for fractional sample evaluation, where the video encoder evaluates only half-pixel samples (e.g., determines and compares distortion values) in a first stage, and then evaluates quarter-pixel samples in a next stage. For example, in a first stage, the video encoder determines distortion values ​​for half-pixel samples, and does not determine distortion values ​​for quarter-pixel samples until a second stage. Then, in these prior art techniques, the video encoder performs another stage, where the video encoder determines distortion values ​​for more samples (e.g., quarter-pixel samples).

[0035] The present disclosure describes examples of determining which fractional samples of a plurality of fractional samples to determine corresponding distortion values ​​for based on integer samples. For example, the example technique includes determining a set of fractional samples of a plurality of fractional samples for which corresponding distortion values ​​are to be determined. The determined fractional samples (e.g., the set of fractional samples) include at least a first fractional sample having a first fractional pixel resolution and a second fractional sample having a second fractional pixel resolution, wherein the first fractional pixel resolution is different from the second fractional pixel resolution. Thus, the example technique includes determining at least a first fractional sample and a second fractional sample (e.g., of the set of fractional samples) for determining corresponding distortion values ​​based on the integer samples, wherein the first fractional sample and the second fractional sample have different fractional pixel resolutions.

[0036] After determining all fractional samples for which corresponding distortion values ​​are to be determined, the technique includes determining corresponding distortion values ​​based on the determined fractional samples (e.g., a set of fractional samples). For example, the example technique includes determining corresponding distortion values ​​associated with the first fractional sample and the second fractional sample after determining both the first fractional sample and the second fractional sample.

[0037] In some examples, the example techniques may facilitate single-stage motion search refinement to identify samples to be used to generate a reference block by first determining all fractional samples for which corresponding distortion values ​​are to be determined, and then determining corresponding distortion values. As an example, a video encoder may determine in one stage which fractional samples of a plurality of fractional samples for which corresponding distortion values ​​are to be determined (e.g., identifying in one stage a first fractional sample and a second fractional sample having different fractional pixel resolutions). In other words, the video encoder may identify in one stage all fractional samples for which distortion values ​​are to be determined. In other techniques, the video encoder determines in a first stage some fractional samples for which distortion values ​​are to be determined, determines and evaluates distortion values, and then in a second stage identifies which additional fractional samples for which distortion values ​​are to be determined.

[0038] For example, a video encoder may identify the best integer sample (e.g., having the lowest distortion) and generate fractional samples at different resolutions around the identified integer sample (e.g., generating both half-pixel samples and quarter-pixel samples). The video encoder may then determine in a stage which of the fractional samples to determine the distortion value for and evaluate (e.g., determine the distortion value). The stage may refer to the video encoder determining which of the multiple fractional samples to determine the corresponding distortion value for. In some examples, the stage may include determining the corresponding distortion value for the determined fractional sample. The determined fractional sample for which the video encoder may determine the distortion value may form the upper left sample of a candidate reference block for distortion value calculation and may be identified by a motion vector. In this way, the video encoder may determine (e.g., identify) at least a first fractional sample and a second fractional sample for determining the corresponding distortion value, wherein the first fractional sample and the second fractional sample have different fractional pixel resolutions.

[0039] That is, the video encoder may identify both half-pixel samples and quarter-pixel samples, for example, surrounding an integer sample for which distortion values ​​are to be determined in the same stage, compared to other techniques in which the video encoder first determines half-pixel samples surrounding the best integer sample in one stage and then determines quarter-pixel samples surrounding the best half-pixel sample in another stage. For example, in one or more examples, the video encoder may determine (e.g., identify) all samples to be evaluated, including samples at different pixel resolutions, and after determining (e.g., identifying) all samples, the video encoder may evaluate the samples (e.g., such as by generating candidate reference blocks and determining distortion values).

[0040] In one or more examples, the video encoder may determine the distortion values ​​partially or completely in parallel (e.g., simultaneously or overlapping in time). For example, the video encoder may determine a first distortion value based on a first fractional sample having a first fractional pixel resolution, and determine a second distortion value based on a second fractional sample having a second different fractional pixel resolution in parallel with determining the first distortion value.

[0041] In this way, the example techniques can promote codec efficiency because the video encoder does not need to wait for the completion of the evaluation of samples at one resolution before determining the distortion values ​​of samples at other resolutions. For example, in a multi-stage motion search (refinement), there are dependencies and delays in the hardware implementation. As an example, for a multi-stage motion search, the video encoder completes the evaluation of samples at one resolution (e.g., determining and comparing the distortion values ​​of the samples) in a first stage to identify the best sample, and then evaluates the samples at different resolutions around the identified best sample in a second stage. In one or more examples, using a single-stage motion search (refinement), the video encoder can first generate distortion values ​​for all samples and then compare these distortion values, thereby allowing the video encoder to complete the evaluation of all samples without waiting for the completion of the evaluation of samples at one resolution before generating distortion values ​​for samples at other resolutions.

[0042] For example, the video encoder may determine distortion values ​​for all samples for which distortion values ​​are to be determined partially or completely in parallel. In some other techniques, the video encoder cannot generate distortion values ​​for all samples in one stage. Instead, the video encoder needs to wait to determine which samples with lower pixel resolutions to determine distortion values ​​for until the video encoder has completed determining distortion values ​​for samples with higher pixel resolutions. For example, the video encoder may not determine distortion values ​​for samples with quarter-pixel resolution until after distortion values ​​for samples with half-pixel resolution are determined and compared to the distortion values ​​for samples with half-pixel resolution.

[0043] Figure 1 1 is a block diagram illustrating an example video encoding and decoding system 100 that can perform the techniques of the present disclosure. The techniques of the present disclosure are generally directed to encoding and decoding (encoding and / or decoding) video data. Generally, video data includes any data used to process video. Therefore, video data may include original, unencoded video, encoded video, decoded (e.g., reconstructed) video, and video metadata, such as signaling data.

[0044] like Figure 1As shown, in this example, system 100 includes a source device 102 that provides encoded video data to be decoded and displayed by a target device 116. Specifically, source device 102 provides video data to target device 116 via computer-readable medium 110. Source device 102 and target device 116 may include any of a variety of devices, including desktop computers, notebook (i.e., laptop) computers, mobile devices, tablet computers, set-top boxes, telephone handsets (such as smartphones), televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, broadcast receiver devices, etc. In some cases, source device 102 and target device 116 may be equipped for wireless communication and may therefore be referred to as wireless communication devices.

[0045] exist Figure 1 In the example of , source device 102 includes video source 104, memory 106, video encoder 200, and output interface 108. Target device 116 includes input interface 122, video decoder 300, memory 120, and display device 118. According to the present disclosure, video encoder 200 of source device 102 and video decoder 300 of target device 116 can be configured to apply motion search techniques for reference block selection. An example of motion search for motion reference block selection is described for video encoder 200. However, in one or more examples, video decoder 300 can perform similar operations as part of decoder-side motion derivation.

[0046] Therefore, source device 102 represents an example of a video encoding device, and target device 116 represents an example of a video decoding device. In other examples, the source device and the target device may include other components or arrangements. For example, source device 102 may receive video data from an external video source (such as an external camera). Similarly, target device 116 may be docked with an external display device, rather than including an integrated display device.

[0047] like Figure 1The system 100 shown is only an example. In general, any digital video encoding and / or decoding device can perform motion search techniques for reference block selection. The source device 102 and the target device 116 are only examples of such codec devices, wherein the source device 102 generates coded video data to send to the target device 116. The present disclosure refers to a "codec" device as a device that performs data encoding and decoding (encoding and / or decoding). Therefore, the video encoder 200 and the video decoder 300 represent examples of codec devices, especially video encoders and video decoders, respectively. In some examples, the source device 102 and the target device 116 can operate in a substantially symmetrical manner, so that each of the source device 102 and the target device 116 includes a video encoding and decoding component. Therefore, the system 100 can support one-way or two-way video transmission between the source device 102 and the target device 116, for example, for video streaming, video playback, video broadcasting, or video telephony.

[0048] In general, the video source 104 represents the source of video data (i.e., the original, unencoded video data), and provides a continuous sequence of pictures (also referred to as "frames") of video data to the video encoder 200, which encodes the data of the pictures. The video source 104 of the source device 102 may include a video capture device, such as a camera, a video archive containing previously captured original video, and / or a video feed interface that receives video from a video content provider. As another alternative, the video source 104 may generate computer graphics-based data as the source video, or a combination of real-time video, archived video, and computer-generated video. In each case, the video encoder 200 encodes the captured, pre-captured, or computer-generated video data. The video encoder 200 may rearrange the pictures from the received order (sometimes referred to as the "display order") into a codec order for encoding and decoding. The video encoder 200 may generate a bitstream including the encoded video data. Then, the source device 102 may output the encoded video data to the computer-readable medium 110 via the output interface 108, for example, to be received and / or retrieved by the input interface 122 of the target device 116.

[0049] The memory 106 of the source device 102 and the memory 120 of the target device 116 represent general purpose memories. In some examples, the memory 106, memory 120 can store raw video data, for example, raw video from the video source 104 and raw decoded video data from the video decoder 300. Additionally or alternatively, the memory 106, memory 120 can store software instructions that can be executed by, for example, the video encoder 200 and the video decoder 300, respectively. Although in this example, the memory 106 and memory 120 are shown separately from the video encoder 200 and the video decoder 300, it should be understood that the video encoder 200 and the video decoder 300 can also include internal memories for functionally similar or equivalent purposes. In addition, the memory 106, memory 120 can store encoded video data, for example, data output from the video encoder 200 and input to the video decoder 300. In some examples, portions of the memory 106, memory 120 can be allocated as one or more video buffers, for example, for storing raw, decoded and / or encoded video data.

[0050] The computer-readable medium 110 may represent any type of medium or device capable of transmitting the encoded video data from the source device 102 to the target device 116. In one example, the computer-readable medium 110 represents a communication medium that enables the source device 102 to transmit the encoded video data directly to the target device 116 in real time, for example, via a radio frequency network or a computer-based network. According to a communication standard such as a wireless communication protocol, the output interface 108 can modulate a transmission signal including the encoded video data, and the input interface 122 can demodulate the received transmission signal. The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include a router, a switch, a base station, or any other equipment that can be used to facilitate communication from the source device 102 to the target device 116.

[0051] In some examples, source device 102 may output the encoded data from output interface 108 to storage device 112. Similarly, target device 116 may access the encoded data from storage device 112 via input interface 122. Storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, Blu-ray disc, DVD, CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

[0052] In some examples, source device 102 may output the encoded video data to file server 114 or another intermediate storage device that may store the encoded video data generated by source device 102. Target device 116 may access the stored video data from file server 114 via streaming or downloading.

[0053] The file server 114 may be any type of server device capable of storing encoded video data and sending the encoded video data to the target device 116. The file server 114 may represent a web server (e.g., for a website), a server configured to provide a file transfer protocol service (e.g., a file transfer protocol (FTP) or a file delivery over one-way transport (FLUTE) protocol), a content delivery network (CDN) device, a hypertext transfer protocol (HTTP) server, a multimedia broadcast multicast service (MBMS) or an enhanced MBMS (eMBMS) server, and / or a network attached storage (NAS) device. Additionally or alternatively, the file server 114 may implement one or more HTTP streaming protocols, such as dynamic adaptive streaming over HTTP (DASH), HTTP real-time streaming (HLS), real-time streaming protocol (RTSP), HTTP dynamic streaming, etc.

[0054] Target device 116 may access the encoded video data from file server 114 via any standard data connection, including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both, suitable for accessing encoded video data stored on file server 114. Input interface 122 may be configured to operate according to any one or more of the various protocols discussed above for retrieving or receiving media data from file server 114, or other such protocols for retrieving media data.

[0055] Output interface 108 and input interface 122 may represent wireless transmitters / receivers, modems, wired network components (e.g., Ethernet cards), wireless communication components that operate according to any of the various IEEE 802.11 standards, or other physical components. In examples where output interface 108 and input interface 122 include wireless components, output interface 108 and input interface 122 may be configured to transmit data (e.g., encoded video data) according to a cellular communication standard (e.g., 4G, 4G-LTE (Long Term Evolution), LTE Advanced, 5G, etc.). In some examples where output interface 108 includes a wireless transmitter, output interface 108 and input interface 122 may be configured to transmit data (e.g., encoded video data) according to other wireless standards (e.g., IEEE 802.11 specifications, IEEE 802.15 specifications (e.g., ZigBee TM),Bluetooth TM 108). In some examples, source device 102 and / or target device 116 may include corresponding system-on-chip (SoC) devices. For example, source device 102 may include a SoC device for performing functions attributed to video encoder 200 and / or output interface 108, and target device 116 may include a SoC device for performing functions attributed to video decoder 300 and / or input interface 122.

[0056] The technology disclosed herein can be applied to video encoding and decoding supporting any of a variety of multimedia applications such as over-the-air television broadcasting, cable television transmission, satellite television transmission, Internet streaming video transmission (such as Dynamic Adaptive Streaming over HTTP (DASH), digital video encoded to data storage media, decoding of digital video stored on data storage media, or other applications.

[0057] The input interface 122 of the destination device 116 receives the encoded video bitstream from the computer-readable medium 110 (e.g., a communication medium, a storage device 112, a file server 114, etc.). The encoded video bitstream may include signaling information defined by the video encoder 200, which is also used by the video decoder 300, such as syntax elements with values ​​describing characteristics and / or processing of video blocks or other coding and decoding units (e.g., slices, pictures, groups of pictures, sequences, etc.). The display device 118 displays decoded pictures of the decoded video data to a user. The display device 118 may represent any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0058] Despite Figure 1 Not shown, but in some examples, the video encoder 200 and the video decoder 300 may each be integrated with an audio encoder and / or an audio decoder, and may include appropriate MUX-DEMUX units or other hardware and / or software to process a multiplexed stream including both audio and video in a common data stream.

[0059] The video encoder 200 and the video decoder 300 can each be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When these techniques are partially implemented in software, the device can store instructions for the software in an appropriate non-transitory computer-readable medium and use one or more processors to execute these instructions in hardware to perform the techniques of the present disclosure. Each of the video encoder 200 and the video decoder 300 can be included in one or more encoders or decoders, any of which can be integrated into the corresponding device as part of a combined encoder / decoder (CODEC). The device including the video encoder 200 and / or the video decoder 300 may include an integrated circuit, a microprocessor, and / or a wireless communication device, such as a cellular phone.

[0060] The video encoder 200 and the video decoder 300 may operate according to a video codec standard such as ITU-T H.265, also known as High Efficiency Video Codec (HEVC) or an extension thereof such as a multi-view and / or scalable video codec extension. Alternatively, the video encoder 200 and the video decoder 300 may operate according to other proprietary or industry standards such as ITU-T H.266, also known as Universal Video Codec (VVC). In other examples, the video encoder 200 and the video decoder 300 may operate according to a proprietary video codec / format such as AOMedia Video 1 (AV1), an extension of AV1, and / or a successor version of AV1 (e.g., AV2). In other examples, the video encoder 200 and the video decoder 300 may operate according to other proprietary formats or industry standards. However, the technology of the present disclosure is not limited to any particular codec standard or format. Typically, as described in more detail, the video encoder 200 and the video decoder 300 may be configured to perform the technology of the present disclosure in conjunction with any video codec technology that uses motion search (e.g., for inter-frame prediction).

[0061] Typically, the video encoder 200 and the video decoder 300 can perform block-based picture encoding and decoding. The term "block" generally refers to a structure that includes data to be processed (e.g., encoded, decoded, or otherwise used in an encoding process and / or decoding process). For example, a block may include a two-dimensional matrix of samples of luminance data and / or chrominance data. Typically, the video encoder 200 and the video decoder 300 can encode and decode video data represented in a YUV (e.g., Y, Cb, Cr) format. That is, the video encoder 200 and the video decoder 300 can encode and decode luminance components and chrominance components instead of encoding and decoding red, green, and blue (RGB) data for samples of a picture, wherein the chrominance component may include both red-toned and blue-toned chrominance components. In some examples, the video encoder 200 converts the received RGB format data into a YUV representation before encoding, and the video decoder 300 converts the YUV representation into an RGB format. Alternatively, a pre-processing unit and a post-processing unit (not shown) may perform these conversions.

[0062] The present disclosure may generally relate to encoding and decoding (e.g., encoding and decoding) a picture to include the process of encoding or decoding data of the picture. Similarly, the present disclosure may relate to encoding and decoding a block of a picture to include the process of encoding or decoding data of the block, such as prediction and / or residual encoding and decoding. The encoded video bitstream generally includes a series of values ​​of syntax elements that represent codec decisions (e.g., codec modes) and partitioning of the picture into blocks. Therefore, references to encoding and decoding a picture or a block should generally be understood as codec values ​​of syntax elements that constitute the picture or block.

[0063] HEVC defines various blocks, including codec units (CUs), prediction units (PUs), and transform units (TUs). According to HEVC, a video codec (such as video encoder 200) divides a codec tree unit (CTU) into CUs according to a quadtree structure. That is, the video codec divides the CTU and CU into four equal, non-overlapping squares, and each node of the quadtree has zero or four child nodes. A node without child nodes may be referred to as a "leaf node", and the CU of such a leaf node may include one or more PUs and / or one or more TUs. The video codec may further divide PUs and TUs. For example, in HEVC, the residual quadtree (RQT) represents the division of TUs. In HEVC, PU represents inter-frame prediction data, and TU represents residual data. The intra-predicted CU includes intra-frame prediction information, such as an intra-frame mode indication.

[0064] As another example, the video encoder 200 and the video decoder 300 may be configured to operate according to VVC. According to VVC, a video codec (such as the video encoder 200) divides a picture into a plurality of codec tree units (CTUs). The video encoder 200 may divide the CTU according to a tree structure (such as a quadtree-binary tree (QTBT) structure or a multi-type tree (MTT) structure). The QTBT structure eliminates the concept of multiple partition types (such as the distinction between CU, PU, ​​and TU of HEVC). The QTBT structure includes two levels: a first level divided according to quadtree partitioning, and a second level divided according to binary tree partitioning. The root node of the QTBT structure corresponds to the CTU. The leaf nodes of the binary tree correspond to codec units (CUs).

[0065] In the MTT partition structure, blocks can be partitioned using quadtree (QT) partitioning, binary tree (BT) partitioning, and one or more types of ternary tree (TT) (also known as ternary tree (TT)) partitioning. A ternary tree or ternary tree partition is a partition in which a block is split into three sub-blocks. In some examples, a ternary tree or ternary tree partition divides a block into three sub-blocks without dividing the initial block through the center. The partition types in MTT (e.g., QT, BT, and TT) can be symmetric or asymmetric.

[0066] When operating according to the AV1 codec, the video encoder 200 and the video decoder 300 may be configured to encode and decode video data in blocks. In AV1, the largest codec block that can be processed is called a super block. In AV1, a super block may be 128×128 luma samples or 64×64 luma samples. However, in subsequent video codec formats (e.g., AV2), super blocks may be defined by different (e.g., larger) luma sample sizes. In some examples, the super block is the top layer of the block quadtree. The video encoder 200 may further divide the super block into smaller codec blocks. The video encoder 200 may divide super blocks and other codec blocks into smaller blocks using square or non-square partitioning. Non-square blocks may include N / 2×N, N×N / 2, N / 4×N, and N×N / 4 blocks. The video encoder 200 and the video decoder 300 may perform separate prediction and transform processing on each codec block.

[0067] AV1 also defines tiles of video data. A tile is a rectangular array of super blocks that can be encoded and decoded independently of other tiles. That is, the video encoder 200 and the video decoder 300 can encode and decode the codec blocks within a tile, respectively, without using video data from other tiles. However, the video encoder 200 and the video decoder 300 can perform filtering across tile boundaries. The size of the tile can be uniform or non-uniform. Tile-based encoding and decoding can achieve parallel processing and / or multi-threading of encoder and decoder implementations.

[0068] In some examples, the video encoder 200 and the video decoder 300 may use a single QTBT or MTT structure to represent each of the luma component and the chroma components, while in other examples, the video encoder 200 and the video decoder 300 may use two or more QTBT or MTT structures, such as one QTBT / MTT structure for the luma component and another QTBT / MTT structure for the two chroma components (or two QTBT / MTT structures for the corresponding chroma components).

[0069] The video encoder 200 and the video decoder 300 may be configured to use quadtree partitioning, QTBT partitioning, MTT partitioning, super block partitioning, or other partitioning structures.

[0070] In some examples, a CTU includes a codec tree block (CTB) of luma samples, two corresponding CTBs of chroma samples for a picture with three sample arrays, or a CTB of samples for a monochrome picture or a picture encoded and decoded using three separate color planes and a syntax structure for encoding and decoding samples. A CTB can be an N×N block of samples for some value of N, so that the division of components into CTBs is a single division. A component is an array or a single sample from one of the three arrays (luminance and two chroma) that make up a 4:2:0, 4:2:2, or 4:4:4 color format picture, or an array or a single sample in an array that makes up a monochrome format picture. In some examples, a codec block is an M×N block of samples for some values ​​of M and N, so that the division of a CTB into codec blocks is a single division.

[0071] Blocks (e.g., CTUs or CUs) may be grouped in a picture in various ways. For example, a brick may refer to a rectangular area of ​​a CTU row within a particular tile in a picture. A tile may be a rectangular area of ​​a CTU within a particular tile column and a particular tile row in a picture. A tile column refers to a rectangular area of ​​a CTU whose height is equal to the height of the picture and whose width is specified by a syntax element (e.g., in a picture parameter set). A tile row refers to a rectangular area of ​​a CTU whose height is specified by a syntax element (e.g., in a picture parameter set) and whose width is equal to the width of the picture.

[0072] In some examples, a tile may be divided into multiple bricks, each of which may include one or more CTU rows within the tile. A tile that is not divided into multiple bricks may also be referred to as a brick. However, a brick that is a proper subset of a tile may not be referred to as a tile. The bricks in a picture may also be arranged into slices. A slice may be an integer multiple of the bricks of a picture, which may be exclusively contained in a single network abstraction layer (NAL) unit. In some examples, a slice includes multiple complete tiles, or only a continuous sequence of complete bricks of a tile.

[0073] This disclosure may use "N×N" and "N times N" interchangeably to refer to the sample dimensions of a block (such as a CU or other video block) in the vertical and horizontal dimensions, for example, 16×16 samples or 16 times 16 samples. In general, a 16×16 CU will have 16 samples in the vertical direction (y=16) and 16 samples in the horizontal direction (x=16). Similarly, an N×N CU typically has N samples in the vertical direction and N samples in the horizontal direction, where N represents a non-negative integer value. The samples in a CU can be arranged in rows and columns. In addition, a CU does not necessarily have the same number of samples in the horizontal direction as in the vertical direction. For example, a CU may include N×M samples, where M is not necessarily equal to N.

[0074] The video encoder 200 encodes the video data of the CU representing the prediction and / or residual information and other information. The prediction information indicates how the CU will be predicted in order to form a prediction block for the CU. The residual information generally represents the sample-by-sample difference between the samples of the CU before encoding and the prediction block.

[0075] In order to predict a CU, the video encoder 200 can generally form a prediction block of the CU by inter-frame prediction or intra-frame prediction. Inter-frame prediction generally refers to predicting a CU based on data of a previously coded picture, while intra-frame prediction generally refers to predicting a CU based on previously coded data of the same picture. In order to perform inter-frame prediction, the video encoder 200 can use one or more motion vectors to generate a prediction block. The video encoder 200 can generally perform a motion search to identify a reference block that closely matches the CU, for example, based on the difference between the CU and the reference block. The video encoder 200 can calculate a difference metric using the sum of absolute differences (SAD), the sum of squared differences (SSD), the mean absolute difference (MAD), the mean square difference (MSD) and other such difference calculations to determine whether the reference block closely matches the current CU. In some examples, the video encoder 200 can use unidirectional prediction or bidirectional prediction to predict the current CU.

[0076] As described in more detail, the present disclosure describes example techniques for motion search for reference block selection. Video encoder 200 may be configured to perform the example techniques and determine the reference block. In some examples, video decoder 300 may perform the example techniques and determine the reference block as part of a decoder-side derivation technique. For convenience, these examples are described with respect to video encoder 200.

[0077] Furthermore, the terms "reference block" and "prediction block" may be interchangeable. However, in some cases, the reference block and the prediction block may be different. For example, the reference block may include multiple samples, and the multiple samples may be filtered, offset, cropped, etc. to generate a prediction block.

[0078] In some examples, the reference block may include samples directly from the reference picture (e.g., the samples are actual samples within the reference picture). Such samples are referred to as integer samples. In some examples, the reference block may include fractional samples. Fractional samples refer to samples generated based on integer samples within the reference picture. For example, the video encoder 200 may interpolate (e.g., average) two integer samples to generate a sample that is assumed to be between the two integer samples. As another example, the video encoder 200 may perform a weighted average of two integer samples to generate additional samples between the two integer samples (e.g., three samples between the two integer samples). Interpolation is an example way to generate fractional samples, but there may be other ways that can be used with the techniques of the present disclosure.

[0079] Fractional samples can be referred to as samples with different fractional pixel resolutions. For example, a half-pixel refers to having one hypothetical sample between two integer samples in the same row or column, a quarter-pixel refers to having three hypothetical samples between two integer samples in the same row or column, and so on. Because these additional fractional samples increase the resolution of the image, the fractional samples can be at different fractional pixel resolutions.

[0080] According to one or more examples described in this disclosure, the video encoder 200 may be configured to determine which fractional samples of a plurality of fractional samples to determine a corresponding cost metric, wherein the fractional samples include fractional samples with different resolutions. Examples of cost metrics include distortion values. In this disclosure, for convenience, distortion values ​​are used to describe examples, but the technology should not be considered to be limited to values ​​indicating distortion. Instead, the example technology is based on cost metrics. The cost metric may be based on samples between a candidate reference block and a current block, and may indicate distortion but does not necessarily indicate distortion.

[0081] For example, the video encoder 200 can be configured to determine a first fractional sample and a second fractional sample (e.g., which are used to determine corresponding distortion values), wherein the first fractional sample has a first fractional pixel resolution and the second fractional sample has a second fractional pixel resolution different from the first fractional pixel resolution. For example, the video encoder 200 can determine a first cost metric associated with the first fractional sample and a second cost metric associated with the second fractional sample.

[0082] The video encoder 200 may generate candidate reference blocks, from which the video encoder 200 selects a reference block. In the present disclosure, "fractional samples having different resolutions" means that the fractional samples include at least a first fractional sample having a first fractional pixel resolution and a second fractional sample having a different second fractional pixel resolution.

[0083] As an example, the first fractional pixel resolution may be half-pixel, quarter-pixel, eighth-pixel, sixteenth-pixel, thirty-second-pixel, or sixty-fourth-pixel. The second fractional pixel resolution may be, for example, another of half-pixel, quarter-pixel, eighth-pixel, sixteenth-pixel, thirty-second-pixel, or sixty-fourth-pixel. Half-pixel fractional resolution means that there is one interpolated sample between two integer samples in the vertical, horizontal, and diagonal directions. Quarter-pixel fractional resolution means that there are three interpolated samples between two integer samples in the vertical, horizontal, and diagonal directions, and so on. In other words, the half-pixel fractional resolution grid has a step size of half a pixel / integer sample, the quarter-pixel fractional resolution grid has a step size of quarter a pixel / integer sample, and so on.

[0084] As an example, the video encoder 200 may determine integer samples in the reference picture based on the search area. Figure 2 Examples of search areas are described in more detail. For example, the video encoder 200 may utilize the motion vector of a neighboring block of the current block as an initial motion vector. The video encoder 200 may determine an integer sample in a reference picture identified by the initial motion vector. If the initial motion vector does not have an integer resolution, the video encoder 200 may crop the initial motion vector to have an integer resolution so that the cropped initial motion vector identifies an integer sample in the reference picture.

[0085] The video encoder 200 may determine a plurality of integer samples within a search area (of a reference picture) and evaluate each of the plurality of integer samples. For example, the video encoder 200 may generate a candidate reference block having integer samples. For example, for a first integer sample, the video encoder 200 may generate a first candidate reference block having integer samples, and for a second integer sample, the video encoder 200 may generate a second candidate reference block having integer samples. To identify each of the integer samples and the candidate reference blocks, the video encoder 200 may utilize a corresponding motion vector. For example, a first motion vector identifies a first integer sample, and the first integer sample is an upper left sample of the first candidate reference block of integer samples. A second motion vector identifies a second integer sample, and the second integer sample is an upper left sample of the second candidate reference block of integer samples, and so on.

[0086] The video encoder 200 may compare each candidate reference block with the current block. As an example of comparing each candidate reference block with the current block, the video encoder 200 may determine a cost metric between each candidate reference block and the current block. An example of calculating a cost metric may be based on a sum of absolute differences (SAD) or a sum of absolute transform differences (SATD). For SATD, the video encoder 200 may transform each of the candidate reference block and the current block (e.g., using a Hadamard transform, a discrete cosine transform (DCT), etc.), and determine the SAD from the resulting transform. The video encoder 200 may be configured to calculate a cost metric using other techniques. As described, a cost metric may indicate a distortion value, and an example technique may describe that the video encoder 200 determines the distortion value. The distortion value is an example of a cost metric.

[0087] The video encoder 200 may determine a candidate reference block having a minimum distortion value compared to all other candidate reference blocks. The candidate reference block having the minimum distortion value may be referred to as the "best" candidate reference block having an integer number of samples, and the motion vector identifying the best candidate reference block having an integer number of samples may be referred to as the "best" motion vector. In addition, the integer samples identified by the best motion vector (e.g., the top left samples in the best candidate reference block) may be referred to as the "best" integer samples. It should be understood that in the case where multiple candidate reference blocks have the same minimum distortion value, for example, a reference block whose top left sample is the first one in the scanning order (in the reference picture), etc., a rule for selecting the "best" motion vector may be defined.

[0088] According to one or more examples described in the present disclosure, after determining integer samples (e.g., best integer samples) in a reference picture within a search area, the video encoder 200 may determine which fractional samples among a plurality of fractional samples are to determine a corresponding cost metric based on the integer samples. That is, the video encoder 200 may determine a set of fractional samples among a plurality of fractional samples for which a corresponding cost metric is to be determined. The determined fractional samples (e.g., the set of fractional samples) include at least a first fractional sample having a first fractional pixel resolution and a second fractional sample having a second fractional pixel resolution. The first fractional pixel resolution is different from the second fractional pixel resolution. In other words, the video encoder 200 may determine at least a first fractional sample and a second fractional sample (e.g., which are used to determine a corresponding cost metric) based on the integer samples.

[0089] As an example, video encoder 200 may determine, from the integer samples, integer samples adjacent to the determined integer sample associated with the lowest distortion value. In one or more examples, adjacent integer samples may mean that there are no other integer samples between two adjacent integer samples.

[0090] For example, when the video encoder 200 is determining the "best" integer sample (e.g., the candidate reference block of the integer sample with the lowest distortion value, where the best integer sample is the top-left sample), the video encoder 200 may have determined the distortion value of each neighboring integer sample using a technique similar to the technique described above. In other words, in some examples, when the video encoder 200 determines the "best" integer sample, the distortion value associated with the "best" integer sample and the distortion values ​​associated with the neighboring integer samples may have been stored in a memory (e.g., memory 106 or some other memory of the video encoder 200). The video encoder 200 can access the memory to determine the sample with the lowest distortion value (referred to herein as the "best" neighboring integer sample) from the neighboring integer samples. In this way, the video encoder 200 may be able to reuse information that has already been determined, which can facilitate efficient processing.

[0091] The video encoder 200 may determine the fractional samples for which the distortion values ​​are to be determined such that the fractional samples are biased toward the adjacent integer samples having the lowest distortion values. For example, assume that the "best" integer sample is the first integer sample, and the adjacent integer sample having the lowest distortion value is the second integer sample. In this example, the set of fractional samples determined for which the distortion values ​​are to be determined may be "biased toward" the second integer sample, e.g., Figure 3A-C. Therefore, the determined fractional samples for which the distortion value is to be determined may be based on the positions of the first integer sample and the second integer sample in the reference picture. For example, the video encoder 200 may determine the first fractional sample and the second fractional sample for determining the corresponding distortion value based on the positions of the first integer sample and the second integer sample in the reference picture.

[0092] Generally, in the present disclosure, if the geometric centers (i.e., centroids) of all samples of the sample set are located on a connecting line (e.g., a diagonal line) of a determined integer sample and an adjacent integer sample, then the sample set, such as a determined fractional sample set or a determined fractional sample plus a determined integer sample (e.g., Figure 3A The set of integer samples 152A in , may be said to be "biased" toward neighboring integer samples (e.g., Figure 3A Similarly, a sample set may be said to be “biased toward” a first-neighboring integer sample and a second-neighboring integer sample (which itself is a neighboring integer sample of the first-neighboring integer sample) if the geometric centers of all samples of the sample set are within the pie slice / circle segment formed by the connecting lines of the determined integer sample and the corresponding first and second-neighboring integer samples, but not on any of the connecting lines themselves.

[0093] In the above example, the video encoder 200 may determine the second integer sample in the reference picture. The first integer sample and the second integer sample may be adjacent to each other in the reference picture, and there may be no other integer samples between the first integer sample and the second integer sample in the reference picture. In such an example, the video encoder 200 may determine the fractional samples for which the corresponding distortion values ​​are to be determined based on the positions of the first integer sample and the second integer sample in the reference picture. That is, the video encoder 200 may determine the first fractional sample and the second fractional sample based on the positions of the first integer sample and the second integer sample. For example, the set of fractional samples for which the distortion value is to be determined may be biased toward the second integer sample. In other words, the first integer sample and the second integer sample may define an area between the first integer sample and the second integer sample, and most of the fractional samples for which the video encoder 200 determines the distortion value may be within the defined area.

[0094] As described above, to determine a first integer sample (e.g., a "best" integer sample) and a second integer sample (e.g., a "best" neighboring sample), the video encoder 200 may determine respective distortion values ​​associated with respective integer samples of a plurality of integer samples, wherein the plurality of integer samples includes the first integer sample and the second integer sample. The video encoder 200 may store the corresponding distortion values ​​for comparison. For example, the video encoder 200 may determine that the first integer sample is the integer sample associated with the lowest distortion value, and the second integer sample is the integer sample that is adjacent to the first integer sample associated with the lowest distortion value.

[0095] In the above example, the video encoder 200 can determine for which fractional samples to determine corresponding distortion values ​​based on the positions of the first integer sample and the second integer sample. In addition, the example is described for the distortion value, which is an example of a cost metric. However, the example technology is not limited to this.

[0096] As another example, the video encoder 200 may determine a fractional sample centered around the determined integer sample (e.g., a first integer sample or a "best" integer sample). For example, to determine at least the first fractional sample and the second fractional sample, the video encoder 200 may identify a fractional sample set for determining a corresponding distortion value, wherein the fractional sample set includes the first fractional sample and the second fractional sample. In this example, the video encoder 200 may identify the fractional sample set as a fractional sample set centered around the integer sample.

[0097] As another example, the video encoder 200 may determine the fractional samples starting from the fractional samples close to the determined integer samples, and add the fractional samples whose distance from the integer samples increases continuously until the number of fractional samples equals the number of defined fractional samples. For example, in order to determine at least the first fractional sample and the second fractional sample, the video encoder 200 may identify a fractional sample set for determining the corresponding distortion value, wherein the fractional sample set includes the first fractional sample and the second fractional sample. The video encoder 200 may identify the fractional sample set starting from the fractional sample close to the integer sample until the number of fractional samples equals the defined number of fractional samples. In other words, the video encoder 200 may determine a fractional sample set with a defined fractional sample set size, starting from the fractional sample closest to the determined integer sample and continuously filling the set with fractional samples at increasing distances from the determined integer sample. The qualified fractional samples for inclusion in the fractional sample set may have to satisfy one or more (geometric) constraints, such as the above-mentioned bias toward the best adjacent integer sample, the defined order (such as the scanning order), the template shape (e.g., square, diamond, etc.), etc.

[0098] For example, the number of fractional samples (the size of the fractional sample set) determined by the video encoder 200 (e.g., via interpolation or some other technique) can be defined by the parameter numFracPoints. The parameter numFracPoints can be defined as M×N, where, in some examples, M and N are both equal to 5. However, the definition of numFracPoints is provided as an example and should not be considered limiting. In some examples, numFracPoints includes the determined integer samples.

[0099] In general, the number of fractional samples generated by the video encoder 200 for motion search can be based on the hardware capabilities of the video encoder 200 and can be a design choice. For example, if the video encoder 200 includes 25 hardware search engine circuits that can run in parallel to perform the operation of motion search, the number of fractional samples can be 25. That is, as an example, the number of fractional samples generated by the video encoder 200 for motion search can be equal to the number of hardware search engine circuits that can run in parallel to perform the operation of motion search. The number of fractional samples determined by the video encoder 200 based on the number of search engine circuits is provided as an example and should not be considered limiting. The parameter numFracPoints can be determined by the video encoder 200 based on the number of available hardware search engine circuits, or predefined as a fixed value, such as 25.

[0100] By performing the example techniques described in the present disclosure, the parallel operation of the hardware search engines can be maximized to complete the motion search in a relatively shorter time. For example, if there are 25 fractional samples for which distortion values ​​are to be determined, and there are 25 hardware search engines, each hardware search engine can run in parallel (e.g., simultaneously or overlapping in time) to determine the distortion values. In other techniques as described above, where not all fractional samples for which distortion values ​​are to be determined are known at the same time, some hardware search engines can remain idle and not be used until distortion values ​​for some fractional samples are determined.

[0101] Thus, in some examples, the video encoder 200 may determine each of the corresponding cost metrics (e.g., distortion values) in parallel based on the determined fractional samples (e.g., fractional samples for which distortion values ​​are to be determined). In other words, the determined fractional samples include at least a first fractional sample having a first fractional pixel resolution and a second fractional sample having a second fractional pixel resolution, wherein the first fractional pixel resolution is different from the second fractional pixel resolution. In one or more examples, the video encoder 200 may determine a first cost metric (e.g., a first distortion value) based on the first fractional sample having the first fractional pixel resolution, and determine a second cost metric (e.g., a second distortion value) based on the second fractional sample having the second fractional pixel resolution in parallel with determining the first distortion value.

[0102] The example techniques may also have benefits even in cases where the number of hardware search engines is less than the number of fractional samples for which distortion values ​​are to be determined. Even in such examples, the hardware search engines may operate in parallel and not wait to complete determining distortion values ​​for fractional samples having one resolution before determining distortion values ​​for fractional samples having another different resolution.

[0103] An example of determining (e.g., identifying) which fractional samples are used for motion search is described above. As described above, the fractional samples include at least a first fractional sample having a first fractional pixel resolution and a second fractional sample having a second fractional pixel resolution, wherein the first fractional pixel resolution is different from the second fractional pixel resolution. After the video encoder 200 determines all fractional samples (including samples with different fractional pixel resolutions) for which corresponding distortion values ​​are to be determined, the video encoder 200 may determine the distortion values ​​based on the determined fractional samples. For example, after determining (e.g., identifying, calculating, selecting, deriving, etc.) both the first fractional sample and the second fractional sample, the video encoder 200 may determine the corresponding distortion values ​​associated with the first fractional sample and the second fractional sample.

[0104] For example, the video encoder 200 may generate a plurality of candidate reference blocks based on the determined fractional samples, wherein each of the plurality of candidate reference blocks is generated based on a corresponding fractional sample in a plurality of fractional samples. In this case, the candidate reference block may include fractional samples (i.e., a candidate reference block having fractional samples). For example, the first fractional sample may be the upper left sample of the first candidate reference block having fractional samples, the second fractional sample may be the upper left sample of the second candidate reference block having fractional samples, and so on. In some examples, the video encoder 200 may identify each candidate reference block having fractional samples based on a corresponding motion vector from the current block (e.g., the first motion vector of the first candidate reference block, the second motion vector of the second candidate reference block, etc.). The candidate reference block includes fractional samples at a single fractional pixel resolution. The step length between adjacent samples of the candidate reference block in the horizontal and vertical directions is always an integer sample. In other words, a candidate reference block having a quarter pixel fractional sample corresponds to an integer sample candidate reference block shifted a quarter pixel from an integer sample motion vector.

[0105] The video encoder 200 may compare each of the plurality of candidate reference blocks with the current block. For example, as an example of comparison, the video encoder 200 may perform a SAD or SATD operation between each candidate reference block and the current block. The video encoder 200 may determine a corresponding distortion value based on the comparison (e.g., the result of the SAD or SATD may be a corresponding distortion value).

[0106] In one or more examples, the video encoder 200 may determine a reference block for the current block based on at least one of the corresponding distortion values ​​and possibly based on the distortion values ​​of the determined integer samples described above. For example, the video encoder 200 may determine which of the candidate reference blocks (including the candidate reference blocks corresponding to the determined integer samples) has the minimum distortion (e.g., the minimum distortion value) relative to the current block. The video encoder 200 may select the determined candidate reference block as the reference block for the current block. The video encoder 200 may also determine a motion vector associated with the determined candidate reference block. The video encoder 200 may set the motion vector of the current block based on the determined motion vector associated with the determined candidate reference block.

[0107] In other words, the video encoder 200 may determine a distortion value indicating a minimum amount of distortion relative to the current block from the corresponding distortion value and the distortion value of the candidate reference block corresponding to the determined integer sample. The video encoder 200 may determine a candidate reference block associated with the determined distortion value. The video encoder 200 may determine a reference block based on the determined candidate reference block. As an example, the video encoder 200 may set the reference block equal to the determined candidate reference block.

[0108] The video encoder 200 may be configured to encode the current block based on the reference block. As an example, the video encoder 200 may generate a prediction block based on the reference block. The prediction block may be equal to the reference block, or the video encoder 200 may perform some scaling or other modification on the reference block to generate the prediction block.

[0109] The video encoder 200 may determine residual information indicating the difference between the current block and the predicted block. In addition, as described above, the video encoder 200 may determine a motion vector of the current block based on the reference block (e.g., a motion vector for identifying the reference block is set equal to the motion vector of the current block). The video encoder 200 may signal information indicating the residual information and information used by the video decoder 300 to derive the motion vector of the current block. For example, the video encoder 200 may determine the difference between an initial motion vector (e.g., used to generate a search area for motion search) and the determined motion vector, and signal information indicating the difference (also referred to as a motion vector difference (MVD)) as a refinement of the initial motion vector. The video decoder 300 may then reconstruct the current block by generating a predicted block based on the motion vector and adding the residual information to the predicted block.

[0110] The example techniques described in the present disclosure may allow the video encoder 200 to determine the fractional samples for motion search for which distortion values ​​are to be determined in one stage, and may determine the distortion values ​​in one stage. A stage as used in the present disclosure refers to a combination of operations required to determine which fractional samples are to be determined for distortion values, wherein each fractional sample is a sample of a candidate reference block. For example, in a single stage, the video encoder 200 may determine a set of fractional samples for determining a corresponding distortion value. That is, in a single stage, the video encoder 200 may determine a set of fractional samples for which a corresponding distortion value is to be determined.

[0111] The determined (e.g., identified) fractional sample may be the top-left sample of the candidate reference block to which the motion vector of the candidate reference block points. For example, the video encoder 200 may determine all fractional samples for which distortion values ​​are to be determined in one stage, and perform motion search using only those determined fractional samples.

[0112] Each fractional sample determined in a single stage may refer to the top left sample of the corresponding candidate reference block. However, the multiple fractional samples determined by the video encoder 200 in a single stage may be fractional samples of the corresponding candidate reference block pointed to by the motion vector of the corresponding candidate reference block. In addition to the fractional samples determined by the video encoder 200 in a single stage of fractional sample determination, there may be no additional fractional samples that form the samples pointed to by the motion vector.

[0113] As described above, the video encoder 200 may be able to determine distortion values ​​in parallel, which can reduce processing time. For example, the video encoder 200 does not need to wait until the distortion value of one fractional sample is completed before starting to determine the distortion value of another fractional sample. The video encoder 200 does not need to wait until the distortion value of a candidate reference block from one fractional sample with one fractional resolution is completed before starting to determine the distortion value of another candidate reference block from another fractional sample with a different fractional resolution.

[0114] In some other techniques, the video encoder 200 may have utilized multiple stages to determine the fractional samples for which distortion values ​​are to be determined. For example, in these other techniques, the video encoder 200 may determine a "best" integer sample similar to that described above. After determining the "best" integer sample, the video encoder 200 may determine a half-pixel fractional sample centered on the best integer sample (e.g., in the first stage, the video encoder 200 determines the half-pixel fractional sample). As described above, the video encoder 200 may then determine the distortion value to determine the best half-pixel fractional sample. For example, the video encoder 200 may search eight half-pixel fractional samples around the integer sample (e.g., determine their distortion values) to determine the best half-pixel fractional sample. Then, in the second stage, the video encoder 200 may determine a quarter-pixel fractional sample centered on the best half-pixel fractional sample, determine (e.g., search) the distortion values ​​of eight quarter-pixel fractional samples around the best half-pixel fractional sample, and then determine the reference block based on the distortion values ​​determined for the best integer sample, the best half-pixel fractional sample, and the best quarter-pixel fractional sample.

[0115] In the above example conventional technique, the fractional search portion includes two stages. The first stage determines half-pixel fractional samples, and the second stage determines quarter-pixel fractional samples. The quarter-pixel fractional sample search cannot begin until the video encoder 200 completes the half-pixel fractional sample search, where the search refers to the determination and comparison of distortion values. Therefore, there is a dependency and delay in when the video encoder 200 can begin to generate and evaluate distortion values.

[0116] In one or more examples, the video encoder 200 may combine the half-pixel fractional sample search and the quarter-pixel fractional sample search into one stage. That is, the video encoder 200 may determine which fractional samples of a plurality of fractional samples to determine the distortion value for (e.g., determine a set of fractional samples), wherein the determined fractional samples (e.g., the set of fractional samples) include at least a first fractional sample having a first fractional pixel resolution and a second fractional sample having a second fractional pixel resolution, instead of segmentally determining the half-pixel fractional samples and the quarter-pixel fractional samples in different stages (e.g., the video encoder 200 does not determine the quarter-pixel fractional samples until after evaluating the distortion value associated with the half-pixel fractional sample). To determine the fractional samples for which the distortion value is to be determined, the video encoder 200 may identify the fractional samples for which the video encoder 200 is to determine the distortion value, and then may determine the distortion value for the identified fractional samples.

[0117] Thus, the video encoder 200 may identify all fractional samples that the video encoder 200 will use for motion search (refinement) in one stage (e.g., determine all fractional samples that will be the top-left samples of the corresponding reference block in one stage). After determining the fractional samples (e.g., after generating or identifying all fractional samples including fractional samples with different fractional pixel resolutions), the video encoder 200 may determine the corresponding distortion value based on the determined fractional samples. Typically, using a single-stage implementation, the video encoder 200 may determine the fractional samples that form the top-left sample of the corresponding candidate reference block without having to complete the generation of other fractional samples. For example, in one or more examples, the video encoder 200 may not wait until after evaluating the half-pixel fractional samples before evaluating the quarter-pixel fractional samples.

[0118] The above examples are described for half-pixel fractional samples and quarter-pixel fractional samples. However, these techniques are not limited to this. For example, to determine multiple fractional samples, the video encoder 200 may determine, for example, one or more of a half-pixel sample, a quarter-pixel sample, an eighth-pixel sample, a sixteenth-pixel sample, a thirty-second-pixel sample, or a sixty-fourth-pixel sample.

[0119] In addition to performing inter-frame prediction using the example techniques described above, the video encoder 200 can be configured to perform other encoding techniques, such as affine motion compensation and intra-frame prediction described below. In affine motion compensation mode, the video encoder 200 can determine two or more motion vectors representing non-translational motion, such as enlargement or reduction, rotation, perspective motion, or other irregular motion types.

[0120] To perform intra prediction, the video encoder 200 can select an intra prediction mode to generate a prediction block. Some examples of VVC provide sixty-seven intra prediction modes, including various directional modes, as well as a plane mode and a DC mode. Typically, the video encoder 200 selects an intra prediction mode that describes neighboring samples of a current block (e.g., a block of a CU) and predicts the samples of the current block based on the neighboring samples. Assuming that the video encoder 200 encodes and decodes CTUs and CUs in a raster scan order (from left to right, from top to bottom), such samples can typically be above, above and to the left, or to the left of the current block in the same picture as the current block.

[0121] The video encoder 200 encodes data representing the prediction mode of the current block. For example, for inter-frame prediction mode, the video encoder 200 can encode data indicating which of the various available inter-frame prediction modes is used and motion information for the corresponding mode. For unidirectional or bidirectional inter-frame prediction, for example, the video encoder 200 can encode the motion vector using advanced motion vector prediction (AMVP) or merge mode. The video encoder 200 can use a similar mode to encode the motion vector for affine motion compensation mode.

[0122] AV1 includes two general techniques for encoding and decoding codec blocks of video data. The two general techniques are intra prediction (e.g., intra prediction or spatial prediction) and inter prediction (e.g., inter prediction or temporal prediction). In the context of AV1, when predicting a block of a current video data frame using an intra prediction mode, the video encoder 200 and the video decoder 300 do not use video data from other video data frames. For most intra prediction modes, the video encoder 200 encodes a block of the current frame based on the difference between the sample values ​​in the current block and the predicted values ​​generated from the reference samples in the same frame. The video encoder 200 determines the predicted values ​​generated from the reference samples based on the intra prediction mode.

[0123] After prediction (such as intra-frame prediction or inter-frame prediction of a block), the video encoder 200 can calculate residual data for the block. The residual data (such as a residual block) represents the sample-by-sample difference between the block and the predicted block of the block formed using the corresponding prediction mode. The video encoder 200 can apply one or more transforms to the residual block to generate transformed data in the transform domain rather than the sample domain. For example, the video encoder 200 can apply a discrete cosine transform (DCT), an integer transform, a wavelet transform, or a conceptually similar transform to the residual video data. In addition, the video encoder 200 can apply a secondary transform after the first transform, such as a mode-dependent non-separable secondary transform (MDNSST), a signal-dependent transform, a karhun-Loeve transform (KLT), etc. The video encoder 200 generates transform coefficients after applying one or more transforms.

[0124] As described above, after any transform used to produce transform coefficients, the video encoder 200 can perform quantization of the transform coefficients. Quantization generally refers to the process of quantizing transform coefficients to possibly reduce the amount of data used to represent the transform coefficients, thereby providing further compression. By performing the quantization process, the video encoder 200 can reduce the bit depth associated with some or all transform coefficients. For example, the video encoder 200 can round down an n-bit value to an m-bit value during quantization, where n is greater than m. In some examples, to perform quantization, the video encoder 200 can perform a bitwise right shift of the value to be quantized.

[0125] After quantization, the video encoder 200 can scan the transform coefficients to generate a one-dimensional vector from a two-dimensional matrix including the quantized transform coefficients. The scan can be designed to place the transform coefficients with higher energy (and therefore lower frequency) in front of the vector and the transform coefficients with lower energy (and therefore higher frequency) in the back of the vector. In some examples, the video encoder 200 can scan the quantized transform coefficients using a predefined scan order to generate a serialized vector, and then entropy encode the quantized transform coefficients of the vector. In other examples, the video encoder 200 can perform adaptive scanning. After scanning the quantized transform coefficients to form a one-dimensional vector, the video encoder 200 can entropy encode the one-dimensional vector, for example, according to context-adaptive binary arithmetic coding (CABAC). The video encoder 200 can also entropy encode the values ​​of the syntax elements describing the metadata associated with the encoded video data for use by the video decoder 300 when decoding the video data.

[0126] To perform CABAC, the video encoder 200 may assign context within a context model to a symbol to be transmitted. For example, the context may relate to whether the neighboring values ​​of the symbol are zero values. The probability determination may be based on the context assigned to the symbol.

[0127] The video encoder 200 may also generate syntax data, such as block-based syntax data, picture-based syntax data, and sequence-based syntax data, or other syntax data, such as a sequence parameter set (SPS), a picture parameter set (PPS), or a video parameter set (VPS), to the video decoder 300, for example, in a picture header, a block header, a slice header. The video decoder 300 may similarly decode such syntax data to determine how to decode the corresponding video data.

[0128] In this way, the video encoder 200 can generate a bitstream including the encoded video data, such as syntax elements describing the division of a picture into blocks (eg, CUs) and prediction and / or residual information of the blocks. Finally, the video decoder 300 can receive the bitstream and decode the encoded video data.

[0129] In general, the video decoder 300 performs a process that is the opposite of the process performed by the video encoder 200 to decode the encoded video data of the bitstream. For example, the video decoder 300 may decode the values ​​of the syntax elements of the bitstream using CABAC in a manner substantially similar to (although opposite to) the CABAC encoding process of the video encoder 200. The syntax elements may define partitioning information for partitioning a picture into CTUs, and partitioning each CTU according to a corresponding partitioning structure (such as a QTBT structure) to define CUs of the CTU. The syntax elements may also define prediction and residual information for a block (e.g., CU) of video data.

[0130] The residual information may be represented by, for example, quantized transform coefficients. The video decoder 300 may dequantize and inversely transform the quantized transform coefficients of the block to reproduce the residual block of the block. The video decoder 300 uses the signaled prediction mode (intra-frame prediction or inter-frame prediction) and related prediction information (e.g., motion information for inter-frame prediction) to form a prediction block for the block. The video decoder 300 may then combine the prediction block and the residual block (on a sample-by-sample basis) to reproduce the original block. The video decoder 300 may perform additional processing (such as performing a deblocking process) to reduce visual artifacts along the boundaries of the blocks.

[0131] The present disclosure generally refers to "signaling" certain information, such as syntax elements. The term "signaling" may generally refer to the communication of values ​​of syntax elements and / or other data used to decode the encoded video data. That is, the values ​​of syntax elements that the video encoder 200 may signal in a bitstream. Generally, signaling refers to generating values ​​in a bitstream. As described above, the source device 102 may transmit the bitstream to the target device 116 substantially in real time or in non-real time, such as may occur when storing syntax elements to the storage device 112 for later retrieval by the target device 116.

[0132] Figure 2 is a conceptual diagram showing an example of a search area for integer samples. Figure 2 The search area is provided as an example only and should not be considered limiting. In general, the entire area within the reference picture or any area within the reference picture can be the search area. In some examples, the video encoder 200 can determine the integer samples according to different types of integer motion search algorithms (such as down-sampling search, M×N integer search, etc.).

[0133] The video encoder 200 may utilize the search area and Figure 2 The diamond-shaped search pattern shown in the dashed line in the figure determines the "best" integer sample and / or the best integer sample motion vector (MV). As described above, the best integer sample can refer to the integer sample that forms the upper left integer sample of the candidate reference block of integer samples with the lowest distortion value. The best integer sample MV refers to the motion vector used to identify the candidate reference block of integer samples with the lowest distortion value.

[0134] The video encoder 200 may start with an integer sample in the "0" position. As an example, the video encoder 200 may select a motion vector of a neighboring block and use the motion vector as an initial motion vector to identify the integer sample in the "0" position. If the initial motion vector is not an integer motion vector (e.g., does not identify an integer sample), the video encoder 200 may clip the initial motion vector to identify the integer sample. There may be various ways to determine the integer sample in the "0" position, and using the motion vector of the neighboring block is one example. As another example, the integer sample in the "0" position may be co-located with the upper left sample in the current block.

[0135] The video encoder 200 may generate a candidate reference block of integer samples for the integer sample in the "0" position, and determine a distortion value with the current block. The video encoder 200 may perform similar operations on the integer sample marked at the "1" position (closest to the "0" position), the sample marked at the "2" position (closest to the "1" position), the sample marked at the "4" position, and the sample marked at the "8" position. The "1" position, the "2" position, the "4" position, and the "8" position are position numbers indicating the L1 (or taxi) distance from the integer sample at the "0" position. For example, the integer sample in the "1" position is one sample away from the integer sample in the "0" position, the integer sample in the "2" position is two samples away from the integer sample in the "0" position, and so on.

[0136] In one or more examples, based on the corresponding distortion values ​​of the candidate reference blocks of integer samples, the video encoder 200 can determine the candidate reference block of integer samples with the minimum distortion relative to the current block. The video encoder 200 can determine the integer samples used to generate the determined candidate reference block as the "best" integer samples. For example, the best motion vector can identify the best integer samples. After determining the best integer samples, the video encoder 200 can determine the fractional samples that the video encoder 200 can further evaluate to determine the reference block for the current block for inter-frame prediction (e.g., in a single stage).

[0137] Figure 3A-3Cis a conceptual diagram showing an example of a score sample evaluated for selecting a reference block. Figure 3A-3C , black samples represent integer samples of the reference picture. All other samples represent fractional samples. The fractional samples shown with hashed or diagonal lines are examples of fractional samples that the video encoder 200 determines to be fractional samples for which corresponding distortion values ​​are to be determined. In other words, the video encoder 200 may identify fractional samples for which distortion values ​​are to be determined (e.g., in one stage), and Figure 3A-3C The determined fractional samples are shown with hashed or diagonal lines. Figure 3A-3C In the example, there are at least a first fractional sample and a second fractional sample, wherein the first fractional sample and the second fractional sample have different fractional pixel resolutions.

[0138] exist Figure 3A In the example above, video encoder 200 may have determined that integer sample 152A is the “best” integer sample. That is, as an example of performing Figure 2 As part of the operation of , video encoder 200 may have determined that the candidate reference block of integer samples has the lowest distortion relative to the current block, and the top left sample of the candidate reference block is integer sample 152A. Video encoder 200 may similarly determine integer sample 152B and integer sample 152C as Figure 3B and Figure 3C The “best” integer sample in .

[0139] Assuming integer sample 152A is a first integer sample, video encoder 200 may determine a second integer sample (e.g., integer sample 154A) in a reference picture (e.g., the same reference picture as integer sample 152A), for example, as the "best" neighboring integer sample as described above. To determine the fractional samples (e.g., those fractional samples shown as hashed or diagonal lines) from the plurality of fractional samples, video encoder 200 may determine the fractional samples based on the positions of first integer sample 152A and second integer sample 154A in the reference picture. That is, to determine which fractional samples of the plurality of fractional samples to determine corresponding distortion values ​​for, video encoder 200 may determine the fractional samples for which corresponding distortion values ​​are to be determined based on the positions of the first integer sample and the second integer sample in the reference picture. In other words, to determine a set of fractional samples for which corresponding distortion values ​​are to be determined, video encoder 200 may determine a set of fractional samples for which corresponding distortion values ​​are to be determined based on the positions of the first integer sample and the second integer sample. The set of fractional samples for which distortion values ​​are to be determined may be biased toward the second integer sample.

[0140] For example, as Figure 2As part of the operation of , video encoder 200 may determine respective distortion values ​​associated with respective integer samples of a plurality of integer samples. The plurality of integer samples include a first integer sample 152A and a second integer sample 154A. To determine first integer sample 152A, video encoder 200 may determine an integer sample associated with a lowest distortion value. That is, as described above, the distortion value of integer sample 152A may be less than the distortion values ​​of other integer samples (of the integer sample search). To determine second integer sample 154A, video encoder 200 may determine integer samples of the integer samples that are adjacent to first integer sample 152A associated with the lowest distortion value. That is, the distortion value associated with second integer sample 154A may be less than the distortion value associated with any other integer sample shown in black that is adjacent to first integer sample 152A. As shown, first integer sample 152A and second integer sample 154A are adjacent to each other in a reference picture, and there are no other integer samples between first integer sample 152A and second integer sample 154A in the reference picture.

[0141] In some examples, when determining first integer sample 152A, video encoder 200 may have already determined the distortion values ​​associated with second integer sample 154A and other adjacent integer samples. In one or more examples, because the distortion values ​​associated with second integer sample 154A and other adjacent integer samples are known, video encoder 200 may simply access a memory to obtain the corresponding distortion values ​​to determine second integer sample 154A. For example, additional processing may not be required to determine the distortion values ​​associated with adjacent integer samples because the distortion values ​​have been previously determined. Therefore, by accessing the memory, video encoder 200 can determine second integer sample 154A. Although the distortion values ​​associated with second integer sample 154A and other adjacent integer samples are known, video encoder 200 may simply access a memory to obtain the corresponding distortion values ​​to determine second integer sample 154A. Figure 3A described, but this also applies to Figure 3B and Figure 3C The second integer samples 154B and 154C.

[0142] like Figure 3A As shown, the fractional samples of the determined plurality of fractional samples are biased toward second integer sample 154A (e.g., in the form of a hash line or a diagonal line). For example, most of the determined fractional samples are formed within a region defined by first integer sample 152A, with first integer sample 152A serving as one corner of the region and second integer sample 154A serving as the diagonal corner of the region. Typically, first integer sample 152A and second integer sample 154A (diagonally adjacent integer samples) may define a corner of a region between first integer sample 152A and second integer sample 154A, and there may be more determined fractional samples (e.g., those fractional samples for which distortion values ​​are to be determined) within the region than outside the region.

[0143] Generally, in the present disclosure, if the geometric centers (i.e., centroids) of all samples of the sample set are located on a connecting line (e.g., a diagonal line) of the determined integer sample and the adjacent integer samples, then the sample set, e.g., a fractional sample set or a fractional sample plus a determined integer sample (e.g., Figure 3A The set of integer samples 152A in , may be said to be "biased" toward neighboring integer samples (e.g., Figure 3A Similarly, a sample set may be said to be “biased toward” a first-neighboring integer sample and a second-neighboring integer sample (which itself is a neighboring integer sample of the first-neighboring integer sample) if the geometric centers of all samples of the sample set are within the pie slice / circle segment formed by the connecting lines of the determined integer sample and the corresponding first and second-neighboring integer samples, but not on any of the connecting lines themselves.

[0144] According to one or more examples described in the present disclosure, the video encoder 200 may determine which fractional samples of a plurality of fractional samples to determine corresponding distortion values ​​for based on the integer sample 152A. For example, the video encoder 200 may determine a set of fractional samples of a plurality of fractional samples for which corresponding distortion values ​​are to be determined based on the integer sample 152A. The determined fractional samples (e.g., the set of fractional samples) include at least a first fractional sample 156A having a first fractional pixel resolution and a second fractional sample 158A having a second fractional pixel resolution. For example, for ease of visualization, the fractional samples shown with hashed lines are quarter-pixel resolutions, and the fractional samples shown with diagonal lines are half-pixel resolutions.

[0145] In one or more examples, the video encoder 200 may determine (e.g., identify) fractional samples for which the video encoder 200 is to determine distortion values, and as shown, the determined (e.g., identified) fractional samples include fractional samples with different fractional pixel resolutions. The video encoder 200 may also determine (e.g., generate) fractional samples, such as by interpolation. For example, after determining all fractional samples for which corresponding distortion values ​​are to be determined, the video encoder 200 may determine corresponding distortion values ​​based on (e.g., associated with) the determined fractional samples. That is, after determining both the first fractional sample and the second fractional sample, the video encoder 200 may determine corresponding distortion values ​​associated with the first fractional sample and the second fractional sample.

[0146] The video encoder 200 may determine (e.g., identify, select, calculate, derive, etc.) the fractional samples in one stage. In general, the video encoder 200 may determine all of the fractional samples for which distortion values ​​are to be determined, without relying at all on determining some of the fractional samples. For example, in some examples, all of the fractional samples shown as hashed or diagonal lines may form the top-left sample of the corresponding candidate reference block, and the video encoder 200 may identify all of the fractional samples in one stage, rather than identifying some of the fractional samples and then identifying other fractional samples based on the earlier identified fractional samples.

[0147] The above examples are for reference Figure 3A The video encoder 200 may be related to Figure 3B and Figure 3C Perform a similar technique. For example, in Figure 3B , video encoder 200 may determine first integer sample 152B and second integer sample 154B that are diagonally offset from each other along an upper right diagonal line (e.g., adjacent samples with minimal distortion among the adjacent samples). In this example, fractional samples including fractional sample 156B (quarter pixel) and fractional sample 158B (half pixel) are biased toward integer sample 154B (i.e., the centroid of the illustrated set of fractional samples is located on a connecting line between first integer sample 152B and second integer sample 154B).

[0148] exist Figure 3C , video encoder 200 may determine first integer sample 152C and second integer sample 154C that are vertically offset from each other (e.g., the adjacent samples with the least distortion among the adjacent samples). In this example, fractional samples including fractional sample 156C (quarter pixel) and fractional sample 158C (half pixel) are biased toward second integer sample 154C (i.e., the centroid of the illustrated fractional sample set is located on the connecting line between first integer sample 152C and second integer sample 154C). Corresponding biased fractional sample sets may be defined for other horizontal, vertical, or diagonal offsets / biases.

[0149] Figure 4A-4C is an additional conceptual diagram illustrating an example of score samples evaluated for selecting a reference block. Figure 4A-4C FIG. 2 shows an example in which the video encoder 200 determines a plurality of fractional samples centered around an integer sample, that is, the geometric center of the set of determined fractional samples coincides with the integer sample. Figure 4A-4C In the example, the video encoder 200 may have been based on the reference Figure 2 The described technique determines that integer samples 160A, 160B, and 160C are the "best" corresponding integer samples, respectively. Figure 4A-4C, fractional samples that the video encoder 200 may determine in a single stage are shown using hashed or diagonal lines, and integer samples are shown in black.

[0150] In one or more examples, the video encoder 200 may determine fractional samples from a plurality of fractional samples based on the integer samples 160A, 160B, or 160C. The determined fractional samples include at least a first fractional sample having a first fractional pixel resolution and a second fractional sample having a second fractional pixel resolution. For example, in Figure 4A-4C , the determined fractional samples shown with diagonal lines are half-pixel fractional samples, and the determined fractional samples shown with hashed lines are quarter-pixel fractional samples. Figure 4A-4C The determined fractional samples 162A, 162B and 162C in are respectively quarter-pixel fractional samples, and Figure 4A-4C The determined fractional samples 164A, 164B, and 164C in are half-pixel fractional samples.

[0151] exist Figure 4A In the example of , the number of determined fractional samples plus the number of determined integer samples is 25 (eg, numFracPoints=5×5). Figure 4B In the example of , the determined fractional samples are all fractional samples around the integer sample 160B and between any other integer samples (eg, within a pixel region).

[0152] exist Figure 4C In the example of , the determined fractional sample may be based on the precision and the distance to the integer sample 160C. Figure 4C , numFracPoints is equal to 25, and the determined fractional samples include the 20 nearest fractional samples and 4 quarter-pixel fractional samples. Figure 4C and Figure 4A The example may be similar, except that Figure 4C In the example, relative to Figure 4A Four half-pixel fractional samples are removed and four quarter-pixel fractional samples are added next to the integer samples. In some examples, numFracPoints includes integer samples 160A, 160B, or 160C. In the present disclosure, the set consisting of the determined fractional samples can generally be extended to include the determined integer samples. Since the distortion value of the determined integer sample has been determined during the integer search, the integer sample can be added to the set consisting of the determined fractional samples without additional computational cost. Instead, the distortion value of the determined integer sample can be stored in a memory and read from the memory when the distortion value of the determined set of fractional samples is determined.

[0153] In some examples, video encoder 200 may determine multiple fractional samples starting from fractional samples close to an integer sample (e.g., integer samples 152A, 152B, 152C, 160A, 160B, or 160C) until the number of fractional samples equals a defined number of fractional samples (e.g., numFracPoints).

[0154] Figure 5 is a block diagram illustrating an example video encoder 200 that may perform the techniques of this disclosure. Figure 5 It is provided for the purpose of explanation and should not be considered as limiting the techniques widely illustrated and described in this disclosure. For the purpose of explanation, this disclosure describes a video encoder 200 according to the techniques of VVC (ITU-T H.266) and HEVC (ITU-T H.265). However, the techniques of this disclosure may be performed by video encoding devices configured for other video codec standards and video codec formats (such as AV1 and successors of the AV1 video codec format).

[0155] exist Figure 5 In the example of , the video encoder 200 includes a video data memory 230, a mode selection unit 202, a residual generation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a filter unit 216, a decoded picture buffer (DPB) 218, and an entropy coding unit 220. Any or all of the video data memory 230, the mode selection unit 202, the residual generation unit 204, the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the filter unit 216, the DPB 218, and the entropy coding unit 220 can be implemented in one or more processors or processing circuits. For example, the units of the video encoder 200 can be implemented as one or more circuits or logic elements, as part of a hardware circuit, or as part of a processor, ASIC, or FPGA. In addition, the video encoder 200 may include additional or alternative processors or processing circuits to perform these and other functions.

[0156] The video data memory 230 may store video data to be encoded by the components of the video encoder 200. The video encoder 200 may, for example, receive video data from the video source 104 ( Figure 1) receives video data stored in video data memory 230. DPB 218 can act as a reference picture memory, which stores reference video data for use by video encoder 200 when predicting subsequent video data. Video data memory 230 and DPB 218 can be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM) or other types of memory devices. Video data memory 230 and DPB 218 can be provided by the same memory device or a separate memory device. In various examples, as shown in the figure, video data memory 230 can be on the chip with other components of video encoder 200, or outside the chip relative to those components.

[0157] In the present disclosure, references to the video data memory 230 should not be interpreted as limited to memory internal to the video encoder 200 (unless specifically described as such) or memory external to the video encoder 200 (unless specifically described as such). Instead, references to the video data memory 230 should be understood as references to memory storing video data received by the video encoder 200 for encoding (e.g., video data for a current block to be encoded). Figure 1 The memory 106 may also provide temporary storage of outputs from the various units of the video encoder 200 .

[0158] Show Figure 5 The various units of the video encoder 200 are described to help understand the operations performed by the video encoder 200. These units can be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Fixed-function circuits refer to circuits that provide specific functions and are preset on the operations that can be performed. Programmable circuits refer to circuits that are programmable to perform various tasks and provide flexible functions in the operations that can be performed. For example, a programmable circuit can execute software or firmware that causes the programmable circuit to operate in a manner defined by the instructions of the software or firmware. Fixed-function circuits can execute software instructions (e.g., receive parameters or output parameters), but the type of operation performed by the fixed-function circuit is generally unchanged. In some examples, one or more units may be different circuit blocks (fixed function or programmable), and in some examples, one or more units may be integrated circuits.

[0159] The video encoder 200 may include an arithmetic logic unit (ALU), an elementary function unit (EFU), a digital circuit, an analog circuit, and / or a programmable core formed by a programmable circuit. In an example in which the operation of the video encoder 200 is performed using software executed by a programmable circuit, the memory 106 ( Figure 1) may store instructions (eg, object code) for software received and executed by the video encoder 200, or another memory within the video encoder 200 (not shown) may store such instructions.

[0160] The video data memory 230 is configured to store received video data. The video encoder 200 may retrieve a picture of video data from the video data memory 230 and provide the video data to the residual generation unit 204 and the mode selection unit 202. The video data in the video data memory 230 may be original video data to be encoded.

[0161] The mode selection unit 202 includes a motion estimation unit 222, a motion compensation unit 224, and an intra prediction unit 226. The mode selection unit 202 may include additional functional units to perform video prediction according to other prediction modes. As an example, the mode selection unit 202 may include a palette unit, an intra-block copy unit (which may be part of the motion estimation unit 222 and / or the motion compensation unit 224), an affine unit, a linear model (LM) unit, etc.

[0162] The mode selection unit 202 generally coordinates multiple encoding processes to test combinations of encoding parameters and the resulting rate-distortion values ​​for such combinations. The encoding parameters may include the division of a CTU into CUs, a prediction mode of a CU, a transform type of residual data of a CU, a quantization parameter of residual data of a CU, etc. The mode selection unit 202 may ultimately select a combination of encoding parameters that has a better rate-distortion value than other tested combinations.

[0163] The video encoder 200 may divide the picture retrieved from the video data memory 230 into a series of CTUs and encapsulate one or more CTUs in a slice. The mode selection unit 202 may divide the CTUs of the picture according to a tree structure (such as the MTT structure, QTBT structure, super block structure, or quadtree structure described above). As described above, the video encoder 200 may form one or more CUs by dividing the CTUs according to the tree structure. Such CUs may also be generally referred to as "video blocks" or "blocks".

[0164] Typically, the mode selection unit 202 also controls its components (e.g., the motion estimation unit 222, the motion compensation unit 224, and the intra prediction unit 226) to generate a prediction block for the current block (e.g., the current CU, or in HEVC, the overlapping portion of the PU and TU). For inter prediction of the current block, the motion estimation unit 222 may perform a motion search to identify one or more closely matching reference blocks in one or more reference pictures (e.g., one or more previously coded pictures stored in the DPB 218). Specifically, the motion estimation unit 222 may calculate a value representing the degree of similarity of the potential reference block to the current block, for example, based on the sum of absolute differences (SAD), the sum of squared differences (SSD), the mean absolute difference (MAD), the mean squared difference (MSD), etc. The motion estimation unit 222 may typically perform these calculations using the sample-by-sample difference between the current block and the reference block under consideration. The motion estimation unit 222 may identify the reference block having the lowest value resulting from these calculations, which indicates the reference block that most closely matches the current block.

[0165] The motion estimation unit 222 may form one or more motion vectors (MVs) that define the position of a reference block in a reference picture relative to the position of a current block in a current picture. The motion estimation unit 222 may then provide the motion vectors to the motion compensation unit 224. For example, for unidirectional inter prediction, the motion estimation unit 222 may provide a single motion vector, and for bidirectional inter prediction, the motion estimation unit 222 may provide two motion vectors. The motion compensation unit 224 may then use the motion vectors to generate a prediction block. For example, the motion compensation unit 224 may use the motion vectors to retrieve data for the reference block. As another example, if the motion vector has fractional sample precision, the motion compensation unit 224 may interpolate the values ​​of the prediction block according to one or more interpolation filters. In addition, for bidirectional inter prediction, the motion compensation unit 224 may retrieve data for two reference blocks identified by the corresponding motion vectors and combine the retrieved data, for example, by sample-by-sample averaging or weighted averaging.

[0166] When operating according to the AV1 video codec format, the motion estimation unit 222 and the motion compensation unit 224 can be configured to encode codec blocks of the video data (e.g., both luma and chroma codec blocks) using translational motion compensation, affine motion compensation, overlapped block motion compensation (OBMC), and / or composite inter-frame intra-frame prediction.

[0167] As another example, for intra prediction or intra prediction codec, the intra prediction unit 226 can generate a prediction block from samples adjacent to the current block. For example, for directional mode, the intra prediction unit 226 can generally mathematically combine the values ​​of adjacent samples and pad these calculated values ​​in a defined direction across the current block to produce a prediction block. As another example, for DC mode, the intra prediction unit 226 can calculate the average value of adjacent samples of the current block and generate a prediction block to include the average value obtained for each sample of the prediction block.

[0168] When operating according to the AV1 video codec format, the intra prediction unit 226 can be configured to encode codec blocks (e.g., luma and chroma codec blocks) of video data using directional intra prediction, non-directional intra prediction, recursive filter intra prediction, luma chroma (CFL) prediction, intra block copy (IBC), and / or palette mode. The mode selection unit 202 may include additional functional units to perform video prediction according to other prediction modes.

[0169] The mode selection unit 202 provides the prediction block to the residual generation unit 204. The residual generation unit 204 receives the original uncoded version of the current block from the video data memory 230 and receives the prediction block from the mode selection unit 202. The residual generation unit 204 calculates the sample-by-sample difference between the current block and the prediction block. The resulting sample-by-sample difference defines a residual block for the current block. In some examples, the residual generation unit 204 may also determine the difference between the sample values ​​in the residual block to generate the residual block using residual differential pulse code modulation (RDPCM). In some examples, the residual generation unit 204 may be formed using one or more subtractor circuits that perform binary subtraction.

[0170] In the example where the mode selection unit 202 divides the CU into PUs, each PU may be associated with a luma prediction unit and a corresponding chroma prediction unit. The video encoder 200 and the video decoder 300 may support PUs of various sizes. As described above, the size of the CU may refer to the size of the luma codec block of the CU, and the size of the PU may refer to the size of the luma prediction unit of the PU. Assuming that the size of a particular CU is 2N×2N, the video encoder 200 may support PU sizes of 2N×2N or N×N for intra-frame prediction, and symmetric PU sizes of 2N×2N, 2N×N, N×2N, N×N or the like for inter-frame prediction. The video encoder 200 and the video decoder 300 may also support asymmetric partitioning of PU sizes of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter-frame prediction.

[0171] In an example where the mode selection unit 202 does not further divide the CU into PUs, each CU may be associated with a luma codec block and a corresponding chroma codec block. As described above, the size of a CU may refer to the size of the luma codec block of the CU. The video encoder 200 and the video decoder 300 may support a CU size of 2N×2N, 2N×N, or N×2N.

[0172] For other video codec techniques, such as intra-block copy mode codec, affine mode codec, and linear model (LM) mode codec, as some examples, the mode selection unit 202 generates a prediction block for the current block being encoded via various units associated with the codec technique. In some examples, such as palette mode codec, the mode selection unit 202 may not generate a prediction block, but instead generate syntax elements indicating how to reconstruct the block based on the selected palette. In such a mode, the mode selection unit 202 may provide these syntax elements to the entropy coding unit 220 for encoding.

[0173] As described above, the residual generation unit 204 receives video data of a current block and a corresponding prediction block. Then, the residual generation unit 204 generates a residual block for the current block. To generate the residual block, the residual generation unit 204 calculates the sample-by-sample difference between the prediction block and the current block.

[0174] The transform processing unit 206 applies one or more transforms to the residual block to generate a block of transform coefficients (referred to herein as a "transform coefficient block"). The transform processing unit 206 can apply various transforms to the residual block to form a transform coefficient block. For example, the transform processing unit 206 can apply a discrete cosine transform (DCT), a directional transform, a Karhunen-Loeve transform (KLT), or a conceptually similar transform to the residual block. In some examples, the transform processing unit 206 can perform multiple transforms on the residual block, for example, a primary transform and a secondary transform, such as a rotation transform. In some examples, the transform processing unit 206 does not apply a transform to the residual block.

[0175] When operating according to AV1, the transform processing unit 206 may apply one or more transforms to the residual block to generate a block of transform coefficients (referred to herein as a "transform coefficient block"). The transform processing unit 206 may apply various transforms to the residual block to form a transform coefficient block. For example, the transform processing unit 206 may apply a horizontal / vertical transform combination that may include a discrete cosine transform (DCT), an asymmetric discrete sine transform (ADST), a flipped ADST (e.g., inverse ADST), and an identity transform (IDTX). When an identity transform is used, a transform is skipped in one of the vertical or horizontal directions. In some examples, the transform process may be skipped.

[0176] The quantization unit 208 may quantize the transform coefficients in the transform coefficient block to produce a quantized transform coefficient block. The quantization unit 208 may quantize the transform coefficients of the transform coefficient block according to a quantization parameter (QP) value associated with the current block. The video encoder 200 (e.g., via the mode selection unit 202) may adjust the degree of quantization applied to the transform coefficient block associated with the current block by adjusting the QP value associated with the CU. Quantization may introduce information loss, and therefore, the quantized transform coefficients may have lower precision than the original transform coefficients produced by the transform processing unit 206.

[0177] The inverse quantization unit 210 and the inverse transform processing unit 212 may apply inverse quantization and inverse transform to the quantized transform coefficient block, respectively, to reconstruct a residual block from the transform coefficient block. The reconstruction unit 214 may generate a reconstructed block corresponding to the current block (although possibly with some degree of distortion) based on the reconstructed residual block and the prediction block generated by the mode selection unit 202. For example, the reconstruction unit 214 may add samples of the reconstructed residual block to corresponding samples from the prediction block generated by the mode selection unit 202 to generate a reconstructed block.

[0178] Filter unit 216 may perform one or more filter operations on the reconstructed block. For example, filter unit 216 may perform a deblocking operation to reduce blocking artifacts along the edges of the CU. In some examples, the operations of filter unit 216 may be skipped.

[0179] When operating according to AV1, filter unit 216 may perform one or more filter operations on the reconstructed blocks. For example, filter unit 216 may perform a deblocking operation to reduce blocking artifacts along the edges of the CU. In other examples, filter unit 216 may apply a constrained directional enhancement filter (CDEF), which may be applied after deblocking and may include the application of a non-separable, non-linear, low-pass directional filter based on the estimated edge direction. Filter unit 216 may also include a loop recovery filter applied after CDEF, and may include a separable symmetric normalized Wiener filter or a dual self-guided filter.

[0180] The video encoder 200 stores the reconstructed blocks in the DPB 218. For example, in an example where the operation of the filter unit 216 is not performed, the reconstruction unit 214 can store the reconstructed blocks to the DPB 218. In an example where the operation of the filter unit 216 is performed, the filter unit 216 can store the filtered reconstructed blocks to the DPB 218. The motion estimation unit 222 and the motion compensation unit 224 can retrieve a reference picture formed by the reconstructed (and possibly filtered) blocks from the DPB 218 to perform inter-frame prediction on blocks of subsequently encoded pictures. In addition, the intra-frame prediction unit 226 can use the reconstructed blocks in the DPB 218 of the current picture to perform intra-frame prediction on other blocks in the current picture.

[0181] In general, the entropy coding unit 220 may entropy encode syntax elements received from other functional components of the video encoder 200. For example, the entropy coding unit 220 may entropy encode the quantized transform coefficient blocks from the quantization unit 208. As another example, the entropy coding unit 220 may entropy encode the prediction syntax elements (e.g., motion information for inter-frame prediction or intra-frame mode information for intra-frame prediction) from the mode selection unit 202. The entropy coding unit 220 may perform one or more entropy coding operations on syntax elements as another example of video data to generate entropy-encoded data. For example, the entropy coding unit 220 may perform a context-adaptive variable length coding (CAVLC) operation, a CABAC operation, a variable to variable (V2V) length coding operation, a syntax-based context-adaptive binary arithmetic coding (SBAC) operation, a probability interval partitioning entropy (PIPE) coding operation, an exponential-Golomb coding operation, or another type of entropy coding operation on the data. In some examples, the entropy coding unit 220 may operate in a bypass mode in which the syntax elements are not entropy encoded.

[0182] The video encoder 200 may output a bitstream including entropy-coded syntax elements required to reconstruct a block of a slice or picture. Specifically, the entropy encoding unit 220 may output a bitstream.

[0183] According to AV1, the entropy coding unit 220 may be configured as a symbol-to-symbol adaptive multi-symbol arithmetic codec. The syntax elements in AV1 include an alphabet of N elements, and the context (e.g., a probability model) includes a set of N probabilities. The entropy coding unit 220 may store the probabilities as an n-bit (e.g., 15-bit) cumulative distribution function (CDF). The entropy coding unit 22 may perform recursive scaling to update the context using an update factor based on the alphabet size.

[0184] The above operations are described with respect to blocks. Such descriptions should be understood as operations for luma codec blocks and / or chroma codec blocks. As described above, in some examples, the luma codec block and the chroma codec block are the luma and chroma components of the CU. In some examples, the luma codec block and the chroma codec block are the luma and chroma components of the PU.

[0185] In some examples, the operations performed for the luma codec block do not need to be repeated for the chroma codec block. As an example, the operations for identifying the motion vector (MV) and reference picture of the luma codec block do not need to be repeated to identify the MV and reference picture of the chroma block. Instead, the MV of the luma codec block can be scaled to determine the MV of the chroma block, and the reference picture can be the same. As another example, the intra prediction process can be the same for the luma codec block and the chroma codec block.

[0186] The video encoder 200 represents an example of a device configured to encode video data, the device including a memory configured to store the video data, and one or more processing units implemented in circuits and configured to determine integer samples in a reference picture based on a search area. For example, the video encoder 200 may perform Figure 2 An example technique of determining integer samples in a reference picture is Figure 2 An example of a search area is shown.

[0187] The motion estimation unit 222 and the motion compensation unit 224 can determine which fractional samples among multiple fractional samples to determine the corresponding distortion value based on the integer samples. For example, the motion estimation unit 222 and the motion compensation unit 224 can determine a set of fractional samples among multiple fractional samples for which the corresponding distortion value is to be determined based on the integer samples. The determined fractional samples (e.g., a set of fractional samples) include at least a first fractional sample having a first fractional pixel resolution and a second fractional sample having a second fractional pixel resolution. The first fractional pixel resolution is different from the second fractional pixel resolution. Therefore, the motion estimation unit 222 and the motion compensation unit 224 can determine at least a first fractional sample and a second fractional sample for determining the corresponding distortion value based on the integer samples, wherein the first fractional sample has a first fractional pixel resolution, and the second fractional sample has a second fractional pixel resolution different from the first fractional pixel resolution. For example, Figure 3A-3C and Figure 4A-4C Examples of determined (eg, identified) fractional samples among a plurality of fractional samples are shown, the fractional samples including at least a first fractional sample having a first fractional pixel resolution and a second fractional sample having a second fractional pixel resolution.

[0188] As described above, in some examples, the motion estimation unit 222 and the motion compensation unit 224 may determine in one stage which fractional samples of the plurality of fractional samples to determine the corresponding distortion value for (e.g., determine a set of fractional samples of the plurality of fractional samples). For example, the motion estimation unit 222 and the motion compensation unit 224 may determine the corresponding distortion value in parallel based on the determined fractional samples (e.g., the determined set of fractional samples). As described above, the determined fractional samples include fractional samples having different resolutions. For example, the motion estimation unit 222 and the motion compensation unit 224 may determine a first distortion value based on a first fractional sample having a first fractional pixel resolution, and determine a second distortion value based on a second fractional sample having a second fractional pixel resolution in parallel with determining the first distortion value.

[0189] In one or more examples, after determining all fractional samples for which corresponding distortion values ​​are to be determined, the motion estimation unit 222 and the motion compensation unit 224 may determine corresponding distortion values ​​based on the determined fractional samples. For example, after determining both the first fractional sample and the second fractional sample, the motion estimation unit 222 and the motion compensation unit 224 may determine corresponding distortion values ​​associated with the first fractional sample and the second fractional sample. For example, the motion estimation unit 222 and the motion compensation unit 224 may generate a plurality of candidate reference blocks based on the determined fractional samples, wherein each of the plurality of candidate reference blocks is generated based on a corresponding fractional sample in the determined fractional samples. The motion estimation unit 222 and the motion compensation unit 224 may compare each of the plurality of candidate reference blocks with the current block (e.g., perform a SAD or SATD operation). The motion estimation unit 222 and the motion compensation unit 224 may determine corresponding distortion values ​​based on the comparison (e.g., the result of the SAD or SATD may be a distortion value).

[0190] The motion estimation unit 222 and the motion compensation unit 224 may determine a reference block (and a corresponding motion vector) for the current block based on at least one of the corresponding distortion values. For example, the motion estimation unit 222 and the motion compensation unit 224 may determine a distortion value having a lowest value (e.g., indicating minimum distortion). The motion estimation unit 222 and the motion compensation unit 224 may determine a candidate reference block associated with the lowest distortion value. The motion estimation unit 222 and the motion compensation unit 224 may set the candidate reference block associated with the lowest distortion value as the reference block for the current block. For example, the motion estimation unit 222 and the motion compensation unit 224 may determine a distortion value indicating a lowest amount of distortion relative to the current block from the respective distortion values, and determine a candidate reference block associated with the determined distortion value. The motion estimation unit 222 and the motion compensation unit 224 may determine a reference block based on the determined candidate reference blocks.

[0191] The motion estimation unit 222 and the motion compensation unit 224 may encode the current block based on the reference block. For example, the motion estimation unit 222 and the motion compensation unit 224 may generate a prediction block based on the reference block (e.g., set the prediction block equal to the reference block or perform a modification on the reference block to generate the prediction block). The residual generation unit 204 may determine residual information indicating the difference between the current block and the prediction block, and the motion estimation unit 222 and the motion compensation unit 224 may determine a motion vector of the current block based on the reference block (e.g., a motion vector of a candidate reference block having a lowest distortion value). The motion estimation unit 222 and the motion compensation unit 224 may signal information indicating the residual information and information used by the video decoder 300 to derive the motion vector of the current block via the entropy encoding unit 220.

[0192] Figure 6 is a block diagram illustrating an example video decoder 300 that can perform techniques of the present disclosure. In one or more examples, the video decoder 300 can receive signaling of residual information and information used to derive motion vectors. The residual information and information used to derive motion vectors can be generated using example techniques implemented by the video encoder 200 described in the present disclosure.

[0193] Figure 6 It is provided for the purpose of explanation and does not limit the techniques widely illustrated and described in this disclosure. For the purpose of explanation, this disclosure describes a video decoder 300 according to the techniques of VVC (ITU-T H.266, under development) and HEVC (ITU-T H.265). However, the techniques of this disclosure can be performed by video codec devices configured for other video codec standards.

[0194] exist Figure 6 In the example of , the video decoder 300 includes a codec picture buffer (CPB) memory 320, an entropy decoding unit 302, a prediction processing unit 304, an inverse quantization unit 306, an inverse transform processing unit 308, a reconstruction unit 310, a filter unit 312, and a decoded picture buffer (DPB) 314. Any or all of the CPB memory 320, the entropy decoding unit 302, the prediction processing unit 304, the inverse quantization unit 306, the inverse transform processing unit 308, the reconstruction unit 310, the filter unit 312, and the DPB 314 can be implemented in one or more processors or processing circuits. For example, the units of the video decoder 300 can be implemented as one or more circuits or logic elements, as part of a hardware circuit, or as part of a processor, ASIC, or FPGA. In addition, the video decoder 300 may include additional or alternative processors or processing circuits to perform these and other functions.

[0195] The prediction processing unit 304 includes a motion compensation unit 316 and an intra prediction unit 318. The prediction processing unit 304 may include additional units to perform predictions according to other prediction modes. As an example, the prediction processing unit 304 may include a palette unit, an intra-block copy unit (which may form part of the motion compensation unit 316), an affine unit, a linear model (LM) unit, etc. In other examples, the video decoder 300 may include more, fewer, or different functional components.

[0196] As described above, when operating according to AV1, the motion compensation unit 316 can be configured to decode codec blocks of video data (e.g., both luma and chroma codec blocks) using translational motion compensation, affine motion compensation, OBMC, and / or composite inter-frame intra-frame prediction. As described above, the intra-frame prediction unit 318 can be configured to decode codec blocks of video data (e.g., both luma and chroma codec blocks) using directional intra-frame prediction, non-directional intra-frame prediction, recursive filter intra-frame prediction, CFL, intra-frame block copy (IBC), and / or palette mode.

[0197] The CPB memory 320 may store video data, such as an encoded video bitstream, to be decoded by components of the video decoder 300. The video data stored in the CPB memory 320 may be, for example, received from the computer readable medium 110 ( Figure 1 ). The CPB memory 320 may include a CPB that stores encoded video data (e.g., syntax elements) from an encoded video bitstream. In addition, the CPB memory 320 may store video data other than syntax elements of encoded pictures, such as temporary data representing outputs from various units of the video decoder 300. The DPB 314 typically stores decoded pictures, and the video decoder 300 may output and / or use the decoded pictures as reference video data when decoding subsequent data or pictures of the encoded video bitstream. The CPB memory 320 and the DPB 314 may be formed by any of a variety of memory devices, such as DRAM, including SDRAM, MRAM, RRAM or other types of memory devices. The CPB memory 320 and the DPB 314 may be provided by the same memory device or a separate memory device. In various examples, the CPB memory 320 may be on-chip with other components of the video decoder 300, or off-chip relative to those components.

[0198] Additionally or alternatively, in some examples, video decoder 300 may retrieve the video from memory 120 ( Figure 1) retrieves the encoded and decoded video data. That is, memory 120 may store data as discussed above with respect to CPB memory 320. Likewise, when some or all of the functionality of video decoder 300 is implemented in software to be executed by processing circuitry of video decoder 300, memory 120 may store instructions to be executed by video decoder 300.

[0199] Show Figure 6 The various units shown in FIG. 3 are used to help understand the operations performed by the video decoder 300. These units may be implemented as fixed function circuits, programmable circuits, or a combination thereof. Figure 5 Similarly, a fixed-function circuit refers to a circuit that provides a specific function and is preset in the operations that can be performed. A programmable circuit refers to a circuit that can be programmed to perform a variety of tasks and provide flexible functions in the operations that can be performed. For example, a programmable circuit can execute software or firmware that causes the programmable circuit to operate in a manner defined by the instructions of the software or firmware. A fixed-function circuit can execute software instructions (e.g., receive parameters or output parameters), but the type of operation performed by the fixed-function circuit is generally unchanged. In some examples, one or more units can be different circuit blocks (fixed function or programmable), and in some examples, one or more units can be integrated circuits.

[0200] The video decoder 300 may include an ALU, an EFU, a digital circuit, an analog circuit, and / or a programmable core formed by a programmable circuit. In an example where the operation of the video decoder 300 is performed by software executed on the programmable circuit, an on-chip or off-chip memory may store instructions (e.g., object code) of the software received and executed by the video decoder 300.

[0201] The entropy decoding unit 302 may receive the encoded video data from the CPB and entropy decode the video data to reproduce the syntax elements. The prediction processing unit 304, the inverse quantization unit 306, the inverse transform processing unit 308, the reconstruction unit 310, and the filter unit 312 may generate decoded video data based on the syntax elements extracted from the bitstream.

[0202] Typically, the video decoder 300 reconstructs a picture on a block-by-block basis. The video decoder 300 may perform a reconstruction operation on each block individually (where the block currently being reconstructed (ie, decoded) may be referred to as a "current block").

[0203] The entropy decoding unit 302 may entropy decode syntax elements defining the quantized transform coefficients of the quantized transform coefficient block and transform information such as a quantization parameter (QP) and / or transform mode indication(s). The inverse quantization unit 306 may use the QP associated with the quantized transform coefficient block to determine a degree of quantization and, likewise, determine a degree of inverse quantization for the inverse quantization unit 306 to apply. The inverse quantization unit 306 may, for example, perform a bitwise left shift operation to inverse quantize the quantized transform coefficients. The inverse quantization unit 306 may thereby form a transform coefficient block including the transform coefficients.

[0204] After the inverse quantization unit 306 forms the transform coefficient block, the inverse transform processing unit 308 may apply one or more inverse transforms to the transform coefficient block to generate a residual block associated with the current block. For example, the inverse transform processing unit 308 may apply an inverse DCT, an inverse integer transform, an inverse Karhunen-Loeve transform (KLT), an inverse rotational transform, an inverse directional transform, or another inverse transform to the transform coefficient block.

[0205] In addition, prediction processing unit 304 generates a prediction block based on the prediction information syntax element entropy decoded by entropy decoding unit 302. For example, if the prediction information syntax element indicates that the current block is inter-predicted, motion compensation unit 316 can generate the prediction block. In this case, the prediction information syntax element can indicate a reference picture in DPB 314 from which to retrieve the reference block, and a motion vector that identifies the position of the reference block in the reference picture relative to the position of the current block in the current picture. Motion compensation unit 316 can generally generate a prediction block in the same manner as described for motion compensation unit 224 ( Figure 5 )The inter-frame prediction process is performed in a manner substantially similar to that described in detail above.

[0206] As another example, if the prediction information syntax element indicates that the current block is intra-predicted, the intra-prediction unit 318 can generate a prediction block according to the intra-prediction mode indicated by the prediction information syntax element. Similarly, the intra-prediction unit 318 can generally generate a prediction block in the same manner as that for the intra-prediction unit 226 ( Figure 5 The intra prediction process is performed in a manner substantially similar to that described in the foregoing description. The intra prediction unit 318 may retrieve data of neighboring samples of the current block from the DPB 314.

[0207] The reconstruction unit 310 may reconstruct the current block using the prediction block and the residual block. For example, the reconstruction unit 310 may add samples of the residual block to corresponding samples of the prediction block to reconstruct the current block.

[0208] The filter unit 312 may perform one or more filter operations on the reconstructed block. For example, the filter unit 312 may perform a deblocking operation to reduce blocking artifacts along the edges of the reconstructed block. The operations of the filter unit 312 may not necessarily be performed in all examples.

[0209] The video decoder 300 may store the reconstructed block in the DPB 314. For example, in an example where the operation of the filter unit 312 is not performed, the reconstruction unit 310 may store the reconstructed block to the DPB 314. In an example where the operation of the filter unit 312 is performed, the filter unit 312 may store the filtered reconstructed block to the DPB 314. As described above, the DPB 314 may provide reference information to the prediction processing unit 304, such as previously decoded pictures for subsequent motion compensation and samples of the current picture for intra-frame prediction. In addition, the video decoder 300 may output a decoded picture (e.g., a decoded video) from the DPB 314 for subsequent display on a display device (e.g., Figure 1 is presented on a display device 118).

[0210] Although various examples are described with respect to the video encoder 200 that performs the example techniques described in the present disclosure, the example techniques are not limited thereto. In some examples, the video decoder 300 may be configured to perform the example techniques. For example, the video decoder 300 may be configured to perform the example techniques as part of decoder-side motion vector refinement or some other decoder-side techniques. In addition, in order to determine a cost metric, the video encoder 300 may determine a cost metric between a candidate reference block and a block (such as a previously decoded block or a representation of a current block). For example, the video decoder 300 may receive residual data and add the residual data to a previously decoded block to generate a representation of the current block. This representation of the current block may not be the actual current block, but may be considered as a hypothesis of the current block. The representation of the previously decoded block or current block (e.g., the hypothesis of the current block) is provided as an example and should not be considered limiting.

[0211] As an example, the video decoder 300 can be configured to determine integer samples in a reference picture of the video data, and determine at least a first fractional sample and a second fractional sample based on the integer samples, wherein the first fractional sample has a first fractional pixel resolution and the second fractional sample has a second fractional pixel resolution different from the first fractional pixel resolution.

[0212] In some examples, after determining both the first fractional sample and the second fractional sample, the video decoder 300 may determine a first cost metric associated with the first fractional sample and a second cost metric associated with the second fractional sample. In one or more examples, to determine the first cost metric, the video decoder 300 may generate a first candidate reference block based on the first fractional sample and compare the first candidate reference block to a block (e.g., a previously decoded block, a representation of a current block, etc.). The video decoder 300 may determine the first cost metric based on the comparison of the first candidate reference block to the current block. To determine the second cost metric, the video decoder 300 may generate a second candidate reference block based on the second fractional sample and compare the second candidate reference block to the block (e.g., a previously decoded block, a representation of a current block, etc.). The video decoder 300 may determine the second cost metric based on the comparison of the first candidate reference block to the current block.

[0213] The video decoder 300 may determine a reference block of a current block to be decoded based on at least one of the first cost metric or the second cost metric, and decode the current block based on the reference block. For example, the video decoder 300 may generate a prediction block based on the reference block, determine (e.g., receive) residual information indicating a difference between the current block and the prediction block, and add the residual information to the prediction block to reconstruct the current block.

[0214] Various examples described with video encoder 200 may be applicable to video decoder 300, such as a portion of decoder-side motion determination. For example, to determine the cost metric, since the current block may not be available on the decoder side before the current block is reconstructed, video decoder 300 may utilize a previously decoded block, or a representation of the current block (e.g., adding residual information to a candidate reference block), or some other block that may be generated from the decoded data. Video decoder 300 may then perform various example techniques using the cost metric as described with respect to video encoder 200. In this way, although the example techniques are described with respect to video encoder 200, the example techniques may also be applicable to video decoder 300.

[0215] Figure 7 2 is a flowchart illustrating an example method for encoding a current block according to the techniques of the present disclosure. The current block may include a current CU. Although with respect to the video encoder 200 ( Figure 1 and Figure 5 ), but it should be understood that other devices may be configured to perform the same Figure 7 For convenience, use Figure 3A-3C and Figure 4A-4C To describe the example Figure 7 .

[0216] Video encoder 200 may determine integer samples in a reference picture (400).In some examples, video encoder 200 may determine integer samples in a reference picture based on a search area. Figure 2 An example of the video encoder 200 determining integer samples is shown. As an example, the video encoder 200 may begin by identifying an initial motion vector for a first integer sample in a reference picture. For the first integer sample, the video encoder 200 may generate a first candidate reference block of integer samples. The video encoder 200 may determine a first cost metric between the first candidate reference block of integer samples and the current block. The video encoder 200 may perform these same operations on multiple integer samples in the search area to determine corresponding cost metrics between corresponding candidate reference blocks of integer samples and the current block. Based on the cost metrics, the video encoder 200 may determine the integer sample with the lowest cost metric among the candidate reference blocks of integer samples. Examples of integer samples determined by the video encoder 200 include Figure 3A-3C integer samples 152A, 152B, and 152C of Figure 4A-4C Integer samples 160A, 160B and 160C of.

[0217] The video encoder 200 may determine at least a first fractional sample and a second fractional sample (e.g., which are used to determine a corresponding cost metric) based on the integer samples (402). For example, the video encoder 200 may determine which fractional samples of a plurality of fractional samples to determine a corresponding cost metric for based on the integer samples. For example, the video encoder 200 may determine a set of fractional samples of a plurality of fractional samples for which a corresponding cost metric is to be determined. The determined fractional samples (e.g., the determined set of fractional samples) include at least a first fractional sample having a first fractional pixel resolution and a second fractional sample having a second fractional pixel resolution. The first fractional pixel resolution is different from the second fractional pixel resolution. In some examples, the first fractional pixel resolution may be 1 / 2 N pixels, while the second fractional pixel resolution is 1 / 2 N+1 Pixels, where N is an integer greater than or equal to 1.

[0218] For example, Figure 3A-3C and Figure 4A-4C Examples of determined fractional samples are shown as hashed or diagonal lines. Figure 3A-3C The fraction samples 156A, 156B and 156C or Figure 4A-4C The fractional samples 162A, 162B, and 162C in FIG. 1 are examples of first fractional samples having a first fractional pixel resolution (eg, quarter-pixel resolution). Figure 3A-3C The scores in samples 158A, 158B and 158C or Figure 4A-4CThe fractional samples 164A, 164B, and 164C in FIG. 1 are examples of second fractional samples having a second fractional pixel resolution (eg, half-pixel resolution).

[0219] In one or more examples, the video encoder 200 may determine (e.g., identify) the fractional samples in one stage (i.e., a single stage). As described, a single stage refers to the video encoder 200 determining (e.g., identifying) all fractional samples pointed to by corresponding motion vectors used to determine a reference block together. For example, the video encoder 200 may determine the fractional samples for which the corresponding cost metric (e.g., distortion value) is to be determined in one stage by determining all fractional samples in parallel. As another example, the video encoder 200 may determine the fractional samples in one stage by not waiting to complete determining one set of fractional samples before determining another set of fractional samples (i.e., there is no dependency between the determination of the fractional samples).

[0220] For example, the video encoder 200 may determine each corresponding cost metric in parallel based on each determined fractional sample. As another example, the video encoder 200 may determine a first cost metric based on a first fractional sample having a first fractional pixel resolution, and determine a second cost metric based on a second fractional sample having a second fractional pixel resolution in parallel with determining the first distortion value.

[0221] In some examples, to determine the plurality of fractional samples, the video encoder 200 may interpolate the fractional samples from one or more integer samples in the reference picture. Determining the plurality of fractional samples may include determining, for example, one-half pixel samples, one-quarter pixel samples, one-eighth pixel samples, one-sixteenth pixel samples, one-thirty-second pixel samples, or one-sixty-fourth pixel samples.

[0222] The video encoder 200 may determine the fractional samples starting from the fractional samples close to the integer samples, and add fractional samples whose distance from the integer samples increases continuously until the number of fractional samples is equal to the number of defined fractional samples (e.g., numFracPoints). However, the example technology is not limited thereto. For example, Figure 4A-4C As shown, video encoder 200 may determine fractional samples centered around integer samples 160A, 160B, and 160C.

[0223] In some examples, such as Figure 3A-3CIn the example, integer samples 152A, 152B, or 152C may be referred to as first integer samples. The video encoder 200 may determine a second integer sample in a reference picture. In order to determine which fractional samples of the plurality of fractional samples are to be used to determine the corresponding cost metric, the video encoder 200 may determine the fractional samples for which the corresponding cost metric is to be determined based on the positions of the first integer sample and the second integer sample in the reference picture. As described above, the set consisting of the determined fractional samples may be biased toward the second integer sample, or toward a third integer sample that is an adjacent integer sample of the second integer sample and the second integer sample. That is, the video encoder 200 may determine the first fractional sample and the second fractional sample for determining the corresponding cost metric based on the positions of the first integer sample and the second integer sample in the reference picture.

[0224] For example, in Figure 3A-3C In FIG. 1 , integer samples 154A, 154B, and 154C may be examples of second integer samples. Video encoder 200 may identify the first integer sample and the second integer sample based on the positions of the first integer sample and the second integer sample. Figure 3A-3C For example, the first integer sample and the second integer sample may form a corner of a region, and most of the determined (eg, identified) fractional samples may be within the region.

[0225] As mentioned above about Figure 3A-3C As described above, the video encoder 200 may be configured to determine the number of integer samples (eg, Figure 3A-3C ) in the image. The plurality of integer samples include a first integer sample and a second integer sample. In one or more examples, to determine the first integer sample, the video encoder 200 may determine an integer sample associated with a minimum cost metric from the plurality of integer samples. For example, the cost metrics of integer samples 152A, 152B, and 152C are the lowest among the plurality of integer samples in the search area, respectively. To determine the second integer sample, the video encoder 200 may determine integer samples adjacent to the first integer sample associated with the minimum cost metric among the integer samples. For example, the cost metrics of integer samples 154A, 154B, and 154C are the lowest among the integer samples adjacent to integer samples 152A, 152B, and 152C, respectively. The first integer sample and the second integer sample are adjacent to each other in the reference picture, and there are no other integer samples between the first integer sample and the second integer sample in the reference picture. An example of using one or more cost metrics is described above, which includes one or more distortion values.

[0226] After determining both the first fractional sample and the second fractional sample, the video encoder 200 may determine a first cost metric associated with the first fractional sample and a second cost metric associated with the second fractional sample (404). For example, after determining all fractional samples (e.g., for which respective cost metrics are to be determined), the video encoder 200 may determine respective cost metrics associated with the determined fractional samples.

[0227] There may be various ways in which the video encoder 200 may determine a corresponding cost metric. As described above, a distortion value is an example of a cost metric, and the cost metric may indicate distortion. However, the example techniques are not limited thereto. A cost metric is not necessarily considered to indicate distortion.

[0228] As an example, the video encoder 200 may generate a plurality of candidate reference blocks based on the determined fractional samples, wherein each of the plurality of candidate reference blocks is generated based on each of the determined fractional samples. The video encoder 200 may compare each of the plurality of candidate reference blocks with the current block (e.g., by performing a SAD or SATD calculation). The SAD and SATD calculations are example ways to determine a cost metric. The video encoder 200 may determine a corresponding cost metric based on the comparison (e.g., the cost metric is the result of the SAD or SATD).

[0229] The video encoder 200 may determine a reference block for the current block based on at least one of the first cost metric or the second cost metric (406). For example, the video encoder 200 may determine which candidate reference block has the smallest cost metric (e.g., distortion value), and the video encoder 200 may select the candidate reference block as the reference block for the current block. For example, the video encoder 200 may determine a cost metric indicating a lowest amount of distortion relative to the current block from the corresponding cost metrics, and determine a candidate reference block associated with the determined cost metric. The video encoder 200 may determine the reference block based on the determined candidate reference block and the candidate reference block associated with the determined integer samples.

[0230] Video encoder 200 may encode the current block based on the reference block (408). For example, video encoder 200 may generate a prediction block based on the reference block, determine residual information indicating a difference between the current block and the prediction block, and determine a motion vector for the current block based on the reference block. Video encoder 200 may signal information indicating the residual information and information used by a video decoder to derive a motion vector for the current block.

[0231] One or more examples described in this disclosure are described below.

[0232] Clause 1. A method for encoding video data, the method comprising: determining integer samples in a reference picture of the video data; determining at least a first fractional sample and a second fractional sample based on the integer samples, wherein the first fractional sample has a first fractional pixel resolution and the second fractional sample has a second fractional pixel resolution different from the first fractional pixel resolution; after determining both the first fractional sample and the second fractional sample, determining a first cost metric associated with the first fractional sample and a second cost metric associated with the second fractional sample; determining a reference block of a current block based on at least one of the first cost metric or the second cost metric; and encoding the current block based on the reference block.

[0233] Clause 2. The method of clause 1, wherein determining the reference block comprises determining the reference block for the current block based on at least one of the first cost metric or the second cost metric and a cost metric associated with integer samples.

[0234] Clause 3. A method according to any one of clauses 1 and 2, wherein the first fractional pixel resolution is 1 / 2 N pixel resolution, and the second fractional pixel resolution is 1 / 2 N+1 Pixel resolution, where N is an integer greater than or equal to 1.

[0235] Clause 4. A method according to any one of clauses 1-3, wherein the first cost metric and the second cost metric are cost metrics among multiple cost metrics, the method further comprising: determining a cost metric indicating a lowest cost metric relative to a current block from the multiple cost metrics, wherein the determined cost metric includes one of the first cost metric or the second cost metric; and determining a candidate reference block associated with the determined cost metric, wherein determining the reference block includes determining the reference block based on the determined candidate reference block.

[0236] Clause 5. A method according to any one of clauses 1-4, wherein determining at least a first fractional sample and a second fractional sample includes identifying a fractional sample set, wherein the fractional sample set includes the first fractional sample and the second fractional sample, and wherein, after determining both the first fractional sample and the second fractional sample, determining a first cost metric associated with the first fractional sample and a second cost metric associated with the second fractional sample includes, after identifying all fractional samples in the fractional sample set, determining a corresponding cost metric associated with each of the fractional samples in the fractional sample set.

[0237] Clause 6. The method of clause 5, wherein determining the respective cost metric associated with each of the fractional sample sets comprises determining the respective cost metric associated with each of the fractional sample sets at least partially in parallel.

[0238] Clause 7. A method according to any of clauses 1-6, wherein determining a first cost metric associated with a first fractional sample and a second cost metric associated with a second fractional sample comprises: determining the first cost metric associated with the first fractional sample having a first fractional pixel resolution; and determining the second cost metric associated with the second fractional sample having a second fractional pixel resolution at least partially in parallel with determining the first cost metric.

[0239] Clause 8. The method of any of clauses 1-7, wherein determining at least a first fractional sample and a second fractional sample based on the integer samples comprises determining the first fractional sample and the second fractional sample in one stage.

[0240] Clause 9. A method according to any one of clauses 1-8, wherein the integer sample is a first integer sample, and the method further includes: determining a second integer sample in a reference picture, wherein determining the first fractional sample and the second fractional sample includes determining the first fractional sample and the second fractional sample based on positions of the first integer sample and the second integer sample in the reference picture.

[0241] Clause 10. The method according to Clause 9 further includes: determining respective cost metrics associated with respective integer samples of a plurality of integer samples, wherein the plurality of integer samples include a first integer sample and a second integer sample, wherein determining the first integer sample includes determining an integer sample associated with a minimum cost metric from the plurality of integer samples, and wherein determining the second integer sample includes determining integer samples in the integer samples that are adjacent to the first integer sample associated with the minimum cost metric.

[0242] Clause 11. The method of any of clauses 9 and 10, wherein the first integer sample and the second integer sample are adjacent to each other in a reference picture, and there are no other integer samples between the first integer sample and the second integer sample in the reference picture.

[0243] Clause 12. A method according to any one of clauses 9-11, wherein determining at least a first fractional sample and a second fractional sample includes identifying a fractional sample set, wherein the fractional sample set includes a first fractional sample and a second fractional sample, and wherein identifying the fractional sample set includes identifying a fractional sample set whose geometric center is biased toward a second integer sample.

[0244] Clause 13. A method according to any one of clauses 1-8, wherein determining at least a first fractional sample and a second fractional sample includes identifying a fractional sample set, wherein the fractional sample set includes the first fractional sample and the second fractional sample, and wherein identifying the fractional sample set includes identifying the fractional sample set centered around an integer sample.

[0245] Clause 14. A method according to any one of clauses 1-8 and 13, wherein determining at least a first fractional sample and a second fractional sample includes identifying a fractional sample set, wherein the fractional sample set includes a first fractional sample and a second fractional sample, and wherein identifying the fractional sample set includes identifying the fractional sample set starting from fractional samples close to integer samples and adding fractional samples whose distance from the integer samples continuously increases until the number of fractional samples is equal to the defined number of fractional samples.

[0246] Clause 15. A method according to any one of clauses 1-14, wherein determining at least a first fractional sample and a second fractional sample comprises: determining a first fractional pixel resolution as a half-pixel, a quarter-pixel, an eighth-pixel, a sixteenth-pixel, a thirty-second-pixel, or a sixty-fourth-pixel; and determining a second fractional sample with a second fractional pixel resolution as another of half-pixel, a quarter-pixel, an eighth-pixel, a sixteenth-pixel, a thirty-second-pixel, or a sixty-fourth-pixel.

[0247] Clause 16. A method according to any one of clauses 1-15, wherein determining a first cost metric associated with a first fractional sample and a second cost metric associated with a second fractional sample comprises: generating a first candidate reference block based on the first fractional sample; comparing the first candidate reference block with a current block; determining the first cost metric based on the comparison between the first candidate reference block and the current block; generating a second candidate reference block based on the second fractional sample; comparing the second candidate reference block with the current block; and determining the second cost metric based on the comparison between the first candidate reference block and the current block.

[0248] Clause 17. A method according to any one of clauses 1-16, wherein encoding a current block based on a reference block comprises: generating a prediction block based on the reference block; determining residual information indicating the difference between the current block and the prediction block; determining a motion vector of the current block based on the reference block; and signaling information indicating the residual information and information used by a video decoder to derive the motion vector of the current block.

[0249] Item 18. A device for encoding video data, the device comprising: a memory configured to store video data; and one or more processors implemented in a circuit, coupled to the memory and configured to: determine integer samples in a reference picture of the video data; determine at least a first fractional sample and a second fractional sample based on the integer samples, wherein the first fractional sample has a first fractional pixel resolution and the second fractional sample has a second fractional pixel resolution different from the first fractional pixel resolution; after determining both the first fractional sample and the second fractional sample, determine a first cost metric associated with the first fractional sample and a second cost metric associated with the second fractional sample; determine a reference block for a current block based on at least one of the first cost metric or the second cost metric; and encode the current block based on the reference block.

[0250] Clause 19. The apparatus of clause 18, wherein, to determine the reference block, the one or more processors are configured to determine the reference block for the current block based on at least one of the first cost metric or the second cost metric and a cost metric associated with integer samples.

[0251] Clause 20. The apparatus of any one of clauses 18 and 19, wherein the first fractional pixel resolution is 1 / 2 N pixel resolution, and the second fractional pixel resolution is 1 / 2 N+1 Pixel resolution, where N is an integer greater than or equal to 1.

[0252] Clause 21. A device according to any one of clauses 18-20, wherein the first cost metric and the second cost metric are cost metrics among multiple cost metrics, and wherein one or more processors are configured to: determine a cost metric indicating a lowest cost metric relative to a current block from the multiple cost metrics, wherein the determined cost metric includes one of the first cost metric or the second cost metric; and determine a candidate reference block associated with the determined cost metric, wherein in order to determine the reference block, the one or more processors are configured to determine the reference block based on the determined candidate reference block.

[0253] Clause 22. A device according to any of clauses 18-21, wherein, in order to determine at least a first fractional sample and a second fractional sample, one or more processors are configured to identify a fractional sample set, wherein the fractional sample set includes a first fractional sample and a second fractional sample, and wherein, after determining both the first fractional sample and the second fractional sample, in order to determine a first cost metric associated with the first fractional sample and a second cost metric associated with the second fractional sample, the one or more processors are configured to determine a corresponding cost metric associated with each of the fractional samples in the set after identifying all fractional samples in the fractional sample set.

[0254] Clause 23. The apparatus of clause 22, wherein, to determine the respective cost metric associated with each of the fractional sample sets, the one or more processors are configured to determine the respective cost metric associated with each of the fractional sample sets at least partially in parallel.

[0255] Clause 24. A device according to any of clauses 18-23, wherein, in order to determine a first cost metric associated with a first fractional sample and a second cost metric associated with a second fractional sample, one or more processors are configured to: determine the first cost metric associated with the first fractional sample having a first fractional pixel resolution; and determine the second cost metric associated with the second fractional sample having a second fractional pixel resolution at least partially in parallel with determining the first cost metric.

[0256] Clause 25. The apparatus of any of clauses 18-24, wherein, to determine at least the first fractional sample and the second fractional sample based on the integer samples, the one or more processors are configured to determine the first fractional sample and the second fractional sample in one stage.

[0257] Clause 26. A device according to any of clauses 18-25, wherein the integer sample is a first integer sample, and wherein the one or more processors are configured to: determine a second integer sample in a reference picture, wherein, to determine the first fractional sample and the second fractional sample, the one or more processors are configured to determine the first fractional sample and the second fractional sample based on positions of the first integer sample and the second integer sample in the reference picture.

[0258] Clause 27. A device according to clause 26, wherein one or more processors are configured to: determine respective cost metrics associated with respective integer samples of a plurality of integer samples, wherein the plurality of integer samples include a first integer sample and a second integer sample, wherein, to determine the first integer sample, the one or more processors are configured to determine an integer sample associated with a minimum cost metric from the plurality of integer samples, and wherein, to determine the second integer sample, the one or more processors are configured to determine integer samples in the integer samples that are adjacent to the first integer sample associated with the minimum cost metric.

[0259] Clause 28. Apparatus according to any of clauses 26 and 27, wherein the first integer sample and the second integer sample are adjacent to each other in a reference picture and there are no other integer samples between the first integer sample and the second integer sample in the reference picture.

[0260] Clause 29. A device according to any of clauses 26-28, wherein, in order to determine at least a first fractional sample and a second fractional sample, one or more processors are configured to identify a fractional sample set, wherein the fractional sample set includes a first fractional sample and a second fractional sample, and wherein, in order to identify the fractional sample set, one or more processors are configured to identify the fractional sample set whose geometric center is biased toward the second integer sample.

[0261] Clause 30. A device according to any of clauses 18-25, wherein, in order to determine at least a first fractional sample and a second fractional sample, one or more processors are configured to identify a fractional sample set, wherein the fractional sample set includes a first fractional sample and a second fractional sample, and wherein, in order to identify the fractional sample set, one or more samples are configured to identify the fractional sample set centered around an integer sample.

[0262] Clause 31. A device according to any of clauses 18-25 and 30, wherein, in order to determine at least a first fractional sample and a second fractional sample, one or more processors are configured to identify a fractional sample set, wherein the fractional sample set includes a first fractional sample and a second fractional sample, and wherein, in order to identify the fractional sample set, one or more processors are configured to identify the fractional sample set starting from fractional samples close to integer samples and add fractional samples whose distance from the integer samples continuously increases until the number of fractional samples is equal to the defined number of fractional samples.

[0263] Clause 32. A device according to any of clauses 18-31, wherein, in order to determine at least a first fractional sample and a second fractional sample, one or more processors are configured to: determine a first fractional sample whose first fractional pixel resolution is half a pixel, quarter a pixel, eighth a pixel, sixteenth a pixel, thirty-second a pixel, or sixty-fourth a pixel; and determine a second fractional sample whose second fractional pixel resolution is another of half a pixel, quarter a pixel, eighth a pixel, sixteenth a pixel, thirty-second a pixel, or sixty-fourth a pixel.

[0264] Clause 33. A device according to any of clauses 18-32, wherein, in order to determine a first cost metric associated with a first fractional sample and a second cost metric associated with a second fractional sample, one or more processors are configured to: generate a first candidate reference block based on the first fractional sample; compare the first candidate reference block with a current block; determine the first cost metric based on the comparison between the first candidate reference block and the current block; generate a second candidate reference block based on the second fractional sample; compare the second candidate reference block with the current block; and determine the second cost metric based on the comparison between the first candidate reference block and the current block.

[0265] Clause 34. A device according to any one of clauses 18-33, wherein, in order to encode a current block based on a reference block, one or more processors are configured to: generate a prediction block based on the reference block; determine residual information indicating the difference between the current block and the prediction block; determine a motion vector for the current block based on the reference block; and signal information indicating the residual information and information used by a video decoder to derive the motion vector for the current block.

[0266] Item 35. A computer-readable storage medium having instructions stored thereon that, when executed, cause one or more processors to: determine integer samples in a reference picture of video data; determine at least a first fractional sample and a second fractional sample based on the integer samples, wherein the first fractional sample has a first fractional pixel resolution and the second fractional sample has a second fractional pixel resolution different from the first fractional pixel resolution; after determining both the first fractional sample and the second fractional sample, determine a first cost metric associated with the first fractional sample and a second cost metric associated with the second fractional sample; determine a reference block for a current block based on at least one of the first cost metric or the second cost metric; and encode the current block based on the reference block.

[0267] Clause 36. The computer-readable storage medium of clause 31, further comprising instructions for causing one or more processors to perform the operations of any of clauses 2-17.

[0268] Item 37. A device for encoding video data, the device comprising: a component for determining integer samples in a reference picture of the video data; a component for determining at least a first fractional sample and a second fractional sample based on the integer samples, wherein the first fractional sample has a first fractional pixel resolution and the second fractional sample has a second fractional pixel resolution different from the first fractional pixel resolution; a component for determining a first cost metric associated with the first fractional sample and a second cost metric associated with the second fractional sample after determining both the first fractional sample and the second fractional sample; a component for determining a reference block of a current block based on at least one of the first cost metric or the second cost metric; and a component for encoding the current block based on the reference block.

[0269] Clause 38. The apparatus of clause 37, further comprising means for performing the operations of any of clauses 2-17.

[0270] In one or more examples, the described functions can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored or transmitted as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media (which corresponds to tangible media (such as data storage media)), or communication media (which includes any media that facilitates the transfer of computer programs from one place to another (for example, according to a communication protocol)). In this way, computer-readable media can generally correspond to (1) non-transitory tangible computer-readable storage media or (2) communication media (such as signals or carrier waves). Data storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described in the present disclosure. A computer program product may include a computer-readable medium.

[0271] As an example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, flash memory or any other medium that can be used to store the required program code in the form of instructions or data structures and can be accessed by a computer. In addition, any connection can be appropriately referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave) is used to send instructions from a website, server or other remote source, then the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology (such as infrared, radio and microwave) are included in the definition of the medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carriers, signals or other transient media, but are directed to non-transient tangible storage media. The disks and optical disks used herein include compact discs (CDs), laser optical disks, optical disks, digital versatile discs (DVDs), floppy disks and blue-ray discs, wherein the disks usually reproduce data magnetically, while the optical disks reproduce data optically with lasers. The above combination should also be included in the scope of computer-readable media.

[0272] Instructions may be executed by one or more processors, such as one or more DSPs, general-purpose microprocessors, ASICs, FPGAs, or other equivalent integrated or discrete logic circuits. Therefore, the terms "processor" and "processing circuit" used herein may refer to any of the above structures or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the functions described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. In addition, these techniques may be implemented entirely in one or more circuits or logic elements.

[0273] The techniques of the present disclosure may be implemented in a variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs), or IC collections (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily need to be implemented by different hardware units. Instead, as described above, the various units may be combined in a codec hardware unit or provided by a collection of interoperable hardware units, including one or more processors as described above, with appropriate software and / or firmware.

[0274] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

1. A method for encoding video data, the method comprising: determining an integer number of samples in a reference picture of the video data; determining at least a first fractional sample and a second fractional sample based on the integer samples, wherein the first fractional sample has a first fractional pixel resolution and the second fractional sample has a second fractional pixel resolution different from the first fractional pixel resolution; After determining both the first fractional sample and the second fractional sample, determining a first cost metric associated with the first fractional sample and a second cost metric associated with the second fractional sample; determining a reference block for a current block based on at least one of the first cost metric or the second cost metric; and The current block is encoded based on the reference block.

2. The method according to claim 1, wherein: Determining the reference block includes determining the reference block for the current block based on at least one of the first cost metric or the second cost metric and a cost metric associated with the integer samples.

3. The method according to claim 1, wherein: The first fractional pixel resolution is 1 / 2 N pixel resolution, and the second fractional pixel resolution is 1 / 2 N+1 Pixel resolution, where N is an integer greater than or equal to 1.

4. The method according to claim 1, wherein: The first cost metric and the second cost metric are cost metrics among a plurality of cost metrics, and the method further comprises: determining a cost metric indicating a lowest cost metric relative to the current block from among the plurality of cost metrics, wherein the determined cost metric comprises one of the first cost metric or the second cost metric; and determining a candidate reference block associated with the determined cost metric, The step of determining the reference block comprises: determining the reference block based on the determined candidate reference block.

5. The method according to claim 1, in, determining at least the first fractional sample and the second fractional sample comprises identifying a fractional sample set, wherein the fractional sample set comprises the first fractional sample and the second fractional sample, and Wherein, after determining both the first fractional sample and the second fractional sample, determining the first cost metric associated with the first fractional sample and the second cost metric associated with the second fractional sample includes: after identifying all of the fractional samples in the fractional sample set, determining the corresponding cost metric associated with each of the fractional sample sets. 6 . The method of claim 5 , wherein determining a respective cost metric associated with each of the fractional sample sets comprises determining the respective cost metric associated with each of the fractional sample sets at least partially in parallel.

7. The method according to claim 1, wherein: Determining the first cost metric associated with the first fractional sample and the second cost metric associated with the second fractional sample includes: determining the first cost metric associated with the first fractional sample having the first fractional pixel resolution; and At least partially in parallel with determining the first cost metric, determining the second cost metric associated with the second fractional samples having the second fractional pixel resolution.

8. The method according to claim 1, wherein: Determining at least the first fractional sample and the second fractional sample based on the integer samples includes determining the first fractional sample and the second fractional sample in one stage.

9. The method according to claim 1, wherein: The integer sample is a first integer sample, and the method further comprises: determining a second integer sample in the reference picture, The determining of the first fractional sample and the second fractional sample comprises: determining the first fractional sample and the second fractional sample based on positions of the first integer sample and the second integer sample in the reference picture.

10. The method according to claim 9, further comprising: determining respective cost metrics associated with respective integer samples of a plurality of integer samples, wherein the plurality of integer samples includes the first integer sample and the second integer sample, Wherein, determining the first integer sample comprises: determining an integer sample associated with a minimum cost metric from the plurality of integer samples, and The determining of the second integer sample comprises: determining integer samples among the integer samples that are adjacent to the first integer sample associated with the minimum cost metric.

11. The method according to claim 9, wherein: The first integer sample and the second integer sample are adjacent to each other in the reference picture, and there are no other integer samples between the first integer sample and the second integer sample in the reference picture.

12. The method according to claim 9, in, Determining at least the first fractional sample and the second fractional sample comprises: identifying a fractional sample set, wherein the fractional sample set comprises the first fractional sample and the second fractional sample, and The identifying the fractional sample set comprises: identifying the fractional sample set whose geometric center is biased toward the second integer sample.

13. The method according to claim 1, in, determining at least the first fractional sample and the second fractional sample comprises identifying a fractional sample set, wherein the fractional sample set comprises the first fractional sample and the second fractional sample, and The identifying the fractional sample set comprises identifying the fractional sample set centered around the integer sample.

14. The method according to claim 1, in, Determining at least the first fractional sample and the second fractional sample comprises identifying a fractional sample set, wherein the fractional sample set comprises: the first fractional sample and the second fractional sample, and Wherein, identifying the fractional sample set comprises identifying the fractional sample set starting from fractional samples close to the integer samples, and adding fractional samples whose distances from the integer samples continuously increase until the number of the fractional samples is equal to the defined number of fractional samples.

15. The method according to claim 1, wherein: Determining at least the first fractional sample and the second fractional sample comprises: determining the first fractional pixel resolution to be the first fractional samples of half a pixel, a quarter pixel, an eighth pixel, a sixteenth pixel, a thirty-second pixel, or a sixty-fourth pixel; and The second fractional pixel resolution is determined to be the second fractional sample of another one of half a pixel, quarter a pixel, eighth a pixel, sixteenth a pixel, thirty-second a pixel, or sixty-fourth a pixel.

16. The method according to claim 1, wherein: Determining a first cost metric associated with the first fractional sample and a second cost metric associated with the second fractional sample includes: generating a first candidate reference block based on the first fractional samples; comparing the first candidate reference block with the current block; determining the first cost metric based on the comparison of the first candidate reference block and the current block; generating a second candidate reference block based on the second fraction of samples; comparing the second candidate reference block with the current block; and The second cost metric is determined based on the comparison of the first candidate reference block and the current block.

17. The method according to claim 1, wherein: Encoding the current block based on the reference block includes: generating a prediction block based on the reference block; determining residual information indicating a difference between the current block and the predicted block; determining a motion vector of the current block based on the reference block; and Information indicating the residual information and information used by a video decoder to derive the motion vector of the current block are signaled.

18. A device for encoding video data, the device comprising: a memory configured to store video data; as well as One or more processors implemented in circuitry, coupled to the memory and configured to: determining an integer number of samples in a reference picture of the video data; determining at least a first fractional sample and a second fractional sample based on the integer samples, wherein the first fractional sample has a first fractional pixel resolution and the second fractional sample has a second fractional pixel resolution different from the first fractional pixel resolution; After determining both the first fractional sample and the second fractional sample, determining a first cost metric associated with the first fractional sample and a second cost metric associated with the second fractional sample; determining a reference block for a current block based on at least one of the first cost metric or the second cost metric; and The current block is encoded based on the reference block.

19. The apparatus according to claim 18, wherein: To determine the reference block, the one or more processors are configured to determine the reference block for the current block based on at least one of the first cost metric or the second cost metric and a cost metric associated with the integer samples.

20. The apparatus of claim 18, wherein: The first fractional pixel resolution is 1 / 2 N pixel resolution, and the second fractional pixel resolution is 1 / 2 N+1 Pixel resolution, where N is an integer greater than or equal to 1.

21. The apparatus of claim 18, wherein the first cost metric and the second cost metric are cost metrics from a plurality of cost metrics, and wherein: The one or more processors are configured to: determining a cost metric indicating a lowest cost metric relative to the current block from among the plurality of cost metrics, wherein the determined cost metric comprises one of the first cost metric or the second cost metric; and determining a candidate reference block associated with the determined cost metric, In order to determine the reference block, the one or more processors are configured to determine the reference block based on the determined candidate reference blocks.

22. The apparatus according to claim 18, in, To determine at least the first fractional sample and the second fractional sample, the one or more processors are configured to identify a fractional sample set, wherein the fractional sample set includes the first fractional sample and the second fractional sample, and Wherein, after determining both the first fractional sample and the second fractional sample, in order to determine the first cost metric associated with the first fractional sample and the second cost metric associated with the second fractional sample, the one or more processors are configured to determine the corresponding cost metric associated with each of the fractional sample sets after identifying all of the fractional samples in the fractional sample set.

23. The apparatus of claim 22, wherein: To determine the respective cost metric associated with each of the fractional sample sets, the one or more processors are configured to determine the respective cost metric associated with each of the fractional sample sets at least partially in parallel.

24. The apparatus of claim 18, wherein: To determine the first cost metric associated with the first fractional sample and the second cost metric associated with the second fractional sample, the one or more processors are configured to: determining the first cost metric associated with the first fractional sample having the first fractional pixel resolution; as well as At least partially in parallel with determining the first cost metric, determining the second cost metric associated with the second fractional samples having the second fractional pixel resolution.

25. The apparatus of claim 18, wherein: To determine at least the first fractional samples and the second fractional samples based on the integer samples, the one or more processors are configured to determine the first fractional samples and the second fractional samples in one stage.

26. The apparatus of claim 18, wherein: The integer samples are first integer samples, and wherein the one or more processors are configured to: determining a second integer sample in the reference picture, In order to determine the first fractional sample and the second fractional sample, the one or more processors are configured to determine the first fractional sample and the second fractional sample based on positions of the first integer sample and the second integer sample in the reference picture.

27. The apparatus of claim 26, wherein: The one or more processors are configured to: determining respective cost metrics associated with respective integer samples of a plurality of integer samples, wherein the plurality of integer samples includes the first integer sample and the second integer sample, wherein, to determine the first integer sample, the one or more processors are configured to determine an integer sample associated with a lowest cost metric from the plurality of integer samples, and To determine the second integer sample, the one or more processors are configured to determine integer samples that are adjacent to the first integer sample associated with the lowest cost metric.

28. The apparatus of claim 26, wherein: The first integer sample and the second integer sample are adjacent to each other in the reference picture, and there are no other integer samples between the first integer sample and the second integer sample in the reference picture.

29. The apparatus according to claim 26, in, To determine at least the first fractional sample and the second fractional sample, the one or more processors are configured to identify a fractional sample set, wherein the fractional sample set includes the first fractional sample and the second fractional sample, and In order to identify the fractional sample set, the one or more processors are configured to identify the fractional sample set whose geometric center is biased towards the second integer sample.

30. The apparatus according to claim 18, in, To determine at least the first fractional sample and the second fractional sample, the one or more processors are configured to identify a fractional sample set, wherein the fractional sample set includes the first fractional sample and the second fractional sample, and In order to identify the fractional sample set, the one or more samples are configured to identify the fractional sample set centered around the integer sample.

31. The apparatus according to claim 18, in, To determine at least the first fractional sample and the second fractional sample, the one or more processors are configured to identify a fractional sample set, wherein the fractional sample set includes the first fractional sample and the second fractional sample, and In which, in order to identify the fractional sample set, the one or more processors are configured to identify the fractional sample set starting from the fractional samples close to the integer samples, and add fractional samples whose distances from the integer samples continuously increase until the number of fractional samples is equal to the defined number of fractional samples.

32. The apparatus of claim 18, wherein: To determine at least the first fractional sample and the second fractional sample, the one or more processors are configured to: determining the first fractional pixel resolution to be the first fractional samples of half a pixel, a quarter pixel, an eighth pixel, a sixteenth pixel, a thirty-second pixel, or a sixty-fourth pixel; and The second fractional pixel resolution is determined to be the second fractional sample of another one of half a pixel, quarter a pixel, eighth a pixel, sixteenth a pixel, thirty-second a pixel, or sixty-fourth a pixel.

33. The apparatus of claim 18, wherein: To determine a first cost metric associated with the first fractional sample and a second cost metric associated with the second fractional sample, the one or more processors are configured to: generating a first candidate reference block based on the first fractional samples; comparing the first candidate reference block with the current block; determining the first cost metric based on the comparison of the first candidate reference block and the current block; generating a second candidate reference block based on the second fraction of samples; comparing the second candidate reference block with the current block; as well as The second cost metric is determined based on the comparison of the first candidate reference block and the current block.

34. The apparatus of claim 18, wherein: To encode the current block based on the reference block, the one or more processors are configured to: generating a prediction block based on the reference block; determining residual information indicating a difference between the current block and the predicted block; determining a motion vector of the current block based on the reference block; as well as Information indicating the residual information and information used by a video decoder to derive the motion vector of the current block are signaled.

35. A computer-readable storage medium having stored thereon instructions which, when executed, cause one or more processors to: determining integer samples in a reference picture of the video data; At least a first fractional sample and a second fractional sample are determined based on the integer samples, wherein: The first fractional samples have a first fractional pixel resolution, and the second fractional samples have a second fractional pixel resolution different from the first fractional pixel resolution; After determining both the first fractional sample and the second fractional sample, determining a first cost metric associated with the first fractional sample and a second cost metric associated with the second fractional sample; determining a reference block for a current block based on at least one of the first cost metric or the second cost metric; as well as The current block is encoded based on the reference block.

36. An apparatus for encoding video data, the apparatus comprising: means for determining an integer number of samples in a reference picture of said video data; means for determining at least a first fractional sample and a second fractional sample based on the integer samples, wherein the first fractional sample has a first fractional pixel resolution and the second fractional sample has a second fractional pixel resolution different from the first fractional pixel resolution; means for determining a first cost metric associated with the first fractional sample and a second cost metric associated with the second fractional sample after determining both the first fractional sample and the second fractional sample; means for determining a reference block for a current block based on at least one of the first cost metric or the second cost metric; and Means for encoding the current block based on the reference block.

Images