Method for decoding video data, method for encoding video data, electronic device, and method of storing bitstream

Abstract

To provide methods and systems for improvement in encoding chroma samples and luma samples of a video frame in a bitstream of video data.SOLUTION: A method for decoding video data comprises: obtaining a plurality of luma samples for a plurality of pixels in a video frame, where the pixels belong to a coding block and include a boundary pixel located inside the coding block, where the boundary pixel is immediately adjacent to a boundary of the coding block; determining that one or more neighboring pixels of the boundary pixel are outside the coding block and not available; assigning a respective luminance component corresponding to the boundary pixel to a luminance component corresponding to each of the one or more neighboring pixels; determining a boundary luminance component based, at least, on the luminance components of the one or more neighboring pixels and the boundary pixel according to a predefined luma interpolation scheme; and converting a boundary chroma component from the boundary luminance component according to a linear mapping model.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of the "Simplifications of "Cross-Component Linear Model" in U.S. Provisional Application No. No. 62 / 955,348, the entire contents of which are incorporated herein by reference. .

[0002] This application relates generally to video data encoding and compression, and more particularly to bit-by-bit video data encoding and compression. Improvements to the coding of chroma and luma samples of video frames in a stream Related to methods and systems for [Background technology]

[0003] Digital video can be transmitted over a variety of devices, including digital televisions, laptop or desktop computers, and tablets. bullet computers, digital cameras, digital recording devices, digital media players players, video game consoles, smartphones, video conference devices, video streaming It is supported by various electronic devices such as , MPEG-4, ITU-TH.263, ITU-TH.264 / MPEG-4, Part 10, ADVANCED Video Coding (AVC), high efficiency video Coding (HIGH EFFICIENCY Video Coding) (HEVC), and the Versatile Video Coding (VVC) standard It transmits digital video data by implementing the video compression / decompression standards defined in the Video compression is typically performed by transmitting, receiving, encoding, decoding, and / or storing , spatial (intra-frame) prediction and / or temporal (inter-frame) prediction are performed to extract the video data. In the case of block-based video coding, this involves reducing or removing redundancy inherent in A video frame is divided into one or more slices, each of which contains multiple video blocks. These may also be called coding tree units (CTUs). Each CTU is a or until a predefined minimum CU size is reached. It may be recursively divided into smaller CUs. Each CU (also called a leaf CU) has one or more contains multiple transform units (TUs), and each CU contains one or more prediction units (PUs). Each CU can be coded in either intra, inter, or IBC mode. A video block within an intra-coded (I) slice of a video frame may be , using spatial prediction with respect to reference samples in neighboring blocks within the same video frame Within an inter-coded (P or B) slice of a video frame, A video block is calculated with respect to reference samples in adjacent blocks in the same video frame. Use spatial prediction or reference samples in other previous and / or future reference video frames. Time prediction may be used for the pull.

[0004] Previously encoded reference blocks, e.g., spatial or temporal, based on neighboring blocks Inter prediction produces a predicted block for the current video block being coded. The process of finding the blocks may be performed by a block matching algorithm. The residual data representing the pixel difference between the current block to be coded and the predicted block is Inter-coded blocks are called prediction blocks or prediction errors. The encoding is performed according to the motion vectors and residual blocks that point to the reference blocks in the reference frame. The process of determining motion vectors is usually called motion estimation. The coded block is encoded according to the intra prediction mode and the residual block. For further compression, the residual block is converted from the pixel domain to the transform domain, e.g. is transformed into the frequency domain to obtain residual transform coefficients, which may then be quantized. First, the quantized transform coefficients arranged in a two-dimensional array are scanned to obtain the first-order The original vector is generated and then entropy encoded into a video bitstream. and further compression can be achieved.

[0005] The encoded video bitstream is then stored on a computer-readable storage medium (e.g. , flash memory) and other electronic devices with digital video capabilities The electronic device then receives the data and accesses it, or transmits it directly to the electronic device via wired or wireless communication. For example, the device analyzes the encoded video bitstream and extracts the Syntax elements are obtained from the bitstream and some are added to the syntax elements obtained from the bitstream. and extracting digital video data from a video bitstream encoded at least in part based on the Video decompression (as opposed to video compression as described above) is achieved by reconstructing the data into its original format. and then performs a digital image reconstructor (reverse processing) to display the reconstructed digital image data on the electronic device. Render.

[0006] Inter-component prediction modes are used to predict the composition of the image between luma and chroma samples of a video bitstream. It is applied to reduce minute redundancy. Specifically, in the prediction model, the luminance samples are downsampled and used to predict the chroma samples. At some locations in the frame, luma samples are not available and downsampling occurs near these locations. Different downsampling filters are used to generate the downsampled luma samples. The application of different downsampling filters is incompatible with parallel image processing and This can cause a bottleneck in the processing speed of the image bitstream. It would be beneficial to have a more efficient inter-component prediction mechanism. Summary of the Invention [Problem to be solved by the invention]

[0007] This application relates to video data encoding and decoding, and more particularly to video Encoding chroma and luma samples of a video frame in a bitstream of data The present invention describes a method and system for improving the performance of the sample padding. Unavailable luminance values ​​are applied to the luminance samples and used by the luminance downsampling process. Generates samples. Unavailable samples are generated outside the video frame or image slice. These uses may be in the future or may not yet be encoded and may be encoded later. Unusable samples are selectively generated by overlap padding or mirror padding. Therefore, the same downsampling filter is used across the entire video frame. All downsampled luma samples can be generated, which allows parallel image processing. This allows for processing of the video data bitstream, thereby improving the video processing speed. [Means for solving the problem]

[0008] In one aspect of the present application, a method for decoding video data is performed on an electronic device. The method includes extracting from the bitstream a plurality of pixels in a video frame. The method includes obtaining a plurality of luminance samples, the plurality of pixels belonging to a coding block. and the boundary pixels are within the coding block, and the coding block The method further comprises: is unavailable and one or more luminance samples corresponding to the boundary pixels are is assigned to each pixel of several neighboring pixels as a corresponding luminance sample, and At least one or more adjacent pixels and boundary pixels are interpolated according to a defined intensity interpolation scheme. and determining a boundary luminance sample based on the luminance samples of the pixel. In this case, each pixel of the plurality of neighboring pixels is outside the coding block. determining boundary chroma samples from the boundary luma samples according to a mapping model; In some embodiments, one or more neighboring pixels of a boundary pixel may include outside the frame or image slice, or in some embodiments, border pixels. One or more adjacent pixels of the block have not yet been coded, and the block continues to be coded. It is encoded.

[0009] According to another aspect of the present application, an electronic device includes one or more processing units, a memory, and a plurality of programs stored in the memory. The programs include one or more When executed by the processing unit, the electronic device is configured to decode the video data as described above. Have them carry out the method.

[0010] According to another aspect, a non-transitory computer-readable storage medium includes one or more Storing a plurality of programs for execution by an electronic device having a processing unit The program, when executed by one or more processing units, , the method for decoding the video data is executed as described above. [Brief explanation of the drawings]

[0011] The following description is included to provide a further understanding of the implementation of the present invention. The accompanying drawings, which are incorporated herein by reference in their entirety and which form a part of this specification, illustrate implementations described and together with the description: , which are helpful in interpreting the underlying principles. Like reference numerals refer to corresponding parts.

[0012] [Figure 1] FIG. 1 is a block diagram illustrating an example video encoding and decoding system according to some implementations of this disclosure. [Figure 2] FIG. 2 is a block diagram illustrating an example video encoder according to some implementations of this disclosure. [Figure 3] FIG. 3 is a block diagram illustrating an example video decoder according to some implementations of this disclosure. [Figure 4A] FIG. 4A is a block diagram illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes in accordance with some implementations of this disclosure. [Figure 4B] FIG. 4B is a block diagram illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes in accordance with some implementations of this disclosure. [Figure 4C] FIG. 4C is a block diagram illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes in accordance with some implementations of this disclosure. [Figure 4D] FIG. 4D is a block diagram illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes in accordance with some implementations of this disclosure. [Figure 4E] FIG. 4E is a block diagram illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes in accordance with some implementations of this disclosure. [Figure 5] FIG. 5 illustrates a process for deriving chroma samples from luma samples of a coded block of a video frame of a bitstream in accordance with some embodiments. [Figure 6A] FIG. 6A illustrates four example coding blocks, each including multiple luma samples to be converted to multiple chroma samples, in accordance with some embodiments. [Figure 6B] FIG. 6B illustrates four example coding blocks, each including multiple luma samples to be converted to multiple chroma samples, in accordance with some embodiments. [Figure 6C] FIG. 6C illustrates four example coding blocks, each including multiple luma samples to be converted to multiple chroma samples, in accordance with some embodiments. [Figure 6D] FIG. 6D illustrates four example coding blocks, each including multiple luma samples to be converted to multiple chroma samples, in accordance with some embodiments. [Figure 7] FIG. 7 is a flowchart of a video data decoding method performed on an electronic device, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0013] Reference numerals are given to details of exemplary embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous references are made to aid in understanding the subject matter presented herein. Non-limiting specific details are given, however, those skilled in the art will appreciate that such details may be used without departing from the scope of the claims. Various alternatives may be used without these specific details, and the subject matter may be performed without these specific details. For example, if the subject matter presented here is to provide digital video capabilities, It will be apparent to those skilled in the art that this can be implemented in many types of electronic devices.

[0014] FIG. 1 illustrates a block diagram of a video system for encoding and decoding video blocks in parallel according to some implementations of this disclosure. 1 is a block diagram illustrating an exemplary system 10 for encoding. , system 10 generates video data to be subsequently decoded by destination device 14. and a source device 12 that encodes the source device 12 and the destination device The device 14 can be used on desktop or laptop computers, tablet computers, Smartphones, set-top boxes, digital televisions, cameras, display devices , digital media players, video game consoles, video streaming devices In some implementations, the source may include any of a wide variety of electronic devices, including a The source device 12 and the destination device 14 are equipped with wireless communication capabilities.

[0015] In some implementations, the destination device 14 receives the error to be decoded via link 16. Link 16 can receive encoded video data. Any type of communication medium capable of moving image data from source device 12 to destination device 14. In one example, link 16 may include a network or device where source device 12 is Communications that allow coded video data to be sent directly to a destination 14 in real time The encoded video data may be transmitted over a communication standard such as a wireless communication protocol. The signal may be modulated according to the standard and transmitted to the destination device 14. any wireless or radio frequency (RF) spectrum or one or more physical transmission lines The communication medium may include a local area network, a wide area network, packet-based networks such as networks, or global networks such as the Internet The communication medium can be a network of routers, switches, base stations, or may help facilitate communication from source device 12 to destination device 14. The term "device" may also include any other device that is suitable for the device.

[0016] In some other implementations, the encoded video data is transmitted to an output interface 22 The encoded data may then be transmitted to the storage device 32. The received video data is accessed by the destination device 14 via the input interface 28. The storage device 32 may be a hard drive, a Blu-ray disc, or disk, DVD, CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data, It may include any of a variety of distributed or locally accessed data storage media. In another example, storage device 32 may store the encoded data generated by source device 12. This may correspond to a file server or another intermediate storage device that can hold the stored video data. The destination device 14 receives the content from the storage device 32 via streaming or download. The file server can access the stored video data. The data can be stored and the encoded video data can be transmitted to a destination device 14. It can be any type of computer. Exemplary file servers include web servers. (e.g. for websites), FTP servers, Network Attached Storage (NAS) devices The destination device 14 may include a local storage device or a local disk drive. Wi-Fi connection), wired connection (e.g., DSL, cable modem, etc.), or Both of these access the encoded video data stored on the file server. Video data encoded over any standard data connection, including combinations suitable for The encoded video data can be accessed from the storage device 32. may be a streaming transmission, a download transmission, or a combination of both.

[0017] As shown in FIG. 1, source device 12 includes a video source 18, a video encoder 20, and a , and an output interface 22. The video source 18 includes a video capture device (e.g., video cameras), video archives containing previously captured footage, video collections, a video feed interface for receiving video from a content provider; and and / or computer graphics data for generating source footage. computer graphics system, or a combination of such sources. As an example, video source 18 may be a video camera in a security surveillance system. In some cases, source device 12 and destination device 14 may be using camera or video phones. However, the implementations described in this application are generally applicable to video coding. and can be applied to wireless and / or wired applications.

[0018] Captured, pre-captured, or computer-generated footage , may be encoded by video encoder 20. The encoded video data , directly to the destination device 14 via the output interface 22 of the source device 12. The encoded video data may (or alternatively) be stored on storage device 32. Once stored, it can be accessed by the destination device 14 or other devices for decoding. The output interface 22 may be used for modeling and / or playback. The device may further include a system and / or a transmitter.

[0019] The destination device 14 includes an input interface 28, a video decoder 30, and a display The input interface 28 includes a receiver and / or a modem. and can receive encoded video data via link 16. 16 or provided on storage device 32. The video data is used by the video decoder 30 in decoding the video data. It may include various syntax elements generated by the encoder 20. The syntax elements may be transmitted over a communication medium, stored on a storage medium, or stored on a file server. The encoded video data may be included in the encoded video data stored in the memory.

[0020] In some implementations, destination device 14 may be an integrated display device and destination device. a display device configured to communicate with the display device 14; The display device 34 may display the decoded video data to a user. It is displayed on a liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED) Any type of display, such as an OLED display, or another type of display device It may include various display devices.

[0021] The video encoder 20 and the video decoder 30 are compatible with VVC, HEVC, MPEG-4, proprietary or commercial technology, such as Port 10, Advanced Video Coding (AVC), or extensions of such standards. This application is not limited to any particular video encoding / decoding standard. It is understood that this is not a limitation and may be applicable to other video encoding / decoding standards. Video encoder 20 of device 12 may be configured to record video in accordance with any of these current or future standards. It is generally envisioned that the image data may be encoded in the address space. The video decoder 30 of the destination device 14 may be configured to receive video signals according to any of these current or future standards. It is also generally envisioned that the image data may be configured to be decoded.

[0022] The video encoder 20 and the video decoder 30 each include one or more microprocessors. processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), Field Programmable Gate Arrays (FPGAs), discrete logic, and software Any variety of hardware, firmware, or any combination thereof It may be implemented as a suitable encoder circuit, or partially implemented in software. The electronic device stores instructions for the software on a suitable non-transitory computer-readable medium. and storing instructions in hardware using one or more processors to execute the instructions in accordance with the present disclosure. The disclosed video encoding / decoding operations can be performed. Each of the video decoder 30 and the video decoder 30 may include one or more encoders or decoders. The encoder or decoder is an encoder / decoder combined with a corresponding device. It may be integrated as part of a codec.

[0023] FIG. 2 illustrates an example video encoder 20 according to some implementations described in this application. 1 is a block diagram showing a video encoder 20 for encoding an image of a video block within a video frame. Intra- and inter-prediction coding can be performed. Intra-prediction coding is a type of spatial prediction. Relying on measurements to reduce or eliminate spatial redundancy in video data within a given video frame or picture. Inter-predictive coding relies on temporal prediction to predict adjacent pictures in a video sequence. Reduce or remove temporal redundancy in video data within a frame or picture.

[0024] As shown in FIG. 2, the video encoder 20 includes a video data memory 40, a prediction processing unit 42, and a 41, Decoded Picture Buffer (DPB) 54, Summer 50, Transform Processing Unit 5 2, a quantization unit 54, and an entropy encoding unit 56. The motion estimation unit 41 includes a motion estimation unit 42, a motion compensation unit 44, and a division unit 45. 5, intra prediction processing unit 46, and intra block copy (BC) unit 4 8. In some implementations, video encoder 20 may further include a For this purpose, it also includes an inverse quantization unit 58, an inverse transform processing unit 50, and a sub-unit 52. A locking filter (not shown) is located between Summer 52 and DPB 54, and is positioned at the block boundary. can be filtered to remove blocky artifacts from the reconstructed video. In addition to the deblocking filter, an in-loop filter (not shown) is used, The output of summer 52 may also be filtered. may take the form of a programmable hardware unit or may consist of one or more It can be divided into several illustrated fixed or programmable hardware units. This can be done.

[0025] Video data memory 40 stores the data to be encoded by components of video encoder 20. The video data in the video data memory 40 can be stored in the following format: , may be obtained from the video source 18. The DPB 54 is generated by the video encoder 20 ( By encoding video data (e.g., in intra or inter predictive coding modes) The video data memory 40 and the DP are buffers for storing reference video data to be used. B54 may be formed by any of a variety of memory devices. The video data memory 40 is on-chip with the other components of the video encoder 20, and may be off-chip relative to those components.

[0026] As shown in FIG. 2, after receiving the video data, the division unit in the prediction processing unit 41 The block 45 divides the video data into video blocks. The video frame is sliced ​​and tagged according to a predefined partitioning structure, such as a quadtree structure. This may also involve splitting the image into smaller files or other larger coding units (CUs). An image frame is divided into multiple video blocks (or sets of video blocks called tiles). The prediction processing unit 41 may calculate the error results (e.g., coding rate and distortion) select one of a number of possible predictive coding modes for the current video block based on the level of the The selectable predictive coding mode is one of several intra-predictive coding modes, e.g. The prediction processing unit 41 may be one of a plurality of inter-prediction coding modes. Provide the resulting intra- or inter-predictive coded blocks to Summer50 to generate a residual block, which is then provided to Summer52 to later use as part of the reference frame. The block to be encoded can be reconstructed for use as a prediction processing unit. Unit 41 includes motion vectors, intra mode indicators, segmentation information, and other Syntax elements such as syntax information are encoded in entropy encoding units. It will also be available on Net 56.

[0027] In order to select an appropriate intra-prediction coding mode for the current video block, a prediction processing unit The intra prediction processing unit 46 in the unit 41 performs the same intra prediction as the current block to be coded. Intra-block transfer to the current video block relative to one or more adjacent blocks in the same frame Predictive coding can be performed to provide spatial prediction. The estimation unit 42 and the motion compensation unit 44 estimate one or more motions in one or more reference frames. Or, performing inter-prediction coding on the current video block for multiple predicted blocks. Video encoder 20 generates multiple codes for each block of video data to provide temporal prediction. An encoding pass can be performed to select, for example, an appropriate encoding mode.

[0028] In some implementations, motion estimation unit 42 may estimate a given motion within a sequence of video frames. The motion vector is generated according to the pattern to generate the image for the current video frame. The motion vector is determined based on a prediction block in the reference video frame. The displacement of the prediction unit (PU) of the video block within the current video frame is indicated. The motion estimation performed by motion estimation unit 42 is a motion estimation method that estimates the motion of video blocks. A motion vector is generated for a given frame (or other encoding) The reference frame (or units) for the current block being coded The current video frame or pixel is used to predict a block in another coded unit (or block). The predetermined pattern can indicate the displacement of the PU of the video block in the image. Video frames can be designated as P-frames or B-frames. A unit 48 determines motion vectors by the motion estimation unit 42 for inter prediction. For intra BC coding, for example, the vector of the block vector is determined in a similar manner to the vector of the block vector. or the motion estimation unit 42 can be used to determine the block vectors It is possible.

[0029] The predicted block is calculated by the sum of absolute differences (SAD) and sum of squared differences (SSD) of the reference frame. of a video block that is coded with respect to pixel differences that may be determined by a pixel difference metric or other difference metric. This is a block that is identified as an exact match for the PU. The encoder 20 calculates values ​​for sub-integer pixel positions of the reference frame stored in the DPB 54. For example, video encoder 20 may calculate a quarter pixel of the reference frame. You can interpolate values ​​for positions, eighth-pixel positions, or other fractional pixel positions. Therefore, motion estimation unit 42 calculates the motion vectors for whole and fractional pixel locations. It can perform a motion search using the motion vectors and output motion vectors with sub-pixel accuracy.

[0030] Motion estimation unit 42 determines the location of the PU of the video block within the inter-predictive coded frame. to the first reference frame list (list 0) or the second reference frame list (list 1) The motion vector of the PU is calculated by comparing the position of the predicted block in the selected reference frame with the position of the predicted block in the selected reference frame. The vector is calculated, and the first reference frame list or the second reference frame list is , identify one or more reference frames stored in the DPB 54. 42 sends the calculated motion vectors to the motion compensation unit 44, which then and sends it to the coding unit 56.

[0031] The motion compensation performed by the motion compensation unit 44 is and fetching or generating a prediction block based on the determined motion vector. Upon receiving the motion vector of the PU of the current video block, the motion compensation unit 44 locates the predicted block pointed to by the motion vector in one of the reference frame lists. , it can retrieve the predicted block from DPB54 and transfer the predicted block to Summer50. Then, the summer 50 performs motion compensation from the pixel values ​​of the current video block to be coded. The pixel values ​​of the predicted block provided by unit 44 are subtracted to obtain the pixel The pixel difference values ​​forming the residual video block are: The motion compensation unit 44 may include luma and / or chroma difference components. for use by video decoder 30 in decoding video blocks of an image frame It can also generate syntax elements associated with video blocks of a video frame. The syntax elements can, for example, specify the motion vectors used to identify the prediction blocks. Syntax elements that define the prediction mode, any flags that indicate the prediction mode, or The motion estimation unit 42 and the motion estimation unit 43 may also include any other syntax information described in The compensation unit 44 may be highly integrated, but is shown separately for conceptual purposes. Please note.

[0032] In some implementations, the intra BC unit 48 is connected to the motion estimation unit 42 and the motion A vector is generated in a similar manner as described above in connection with the compensation unit 44, and the predicted block is fetched. The predicted block can be coded in the same frame as the actual block being coded. The vectors are called block vectors as opposed to motion vectors. The intra BC unit 48 determines the intra prediction mode to encode the current block. In some examples, the intra BC unit 48 can be used to For example, different intra prediction modes can be used to encode the current block during separate encoding passes. It is possible to encode blocks and test their performance through rate-distortion analysis. Next, the intra BC unit 48 selects from among the various tested intra prediction modes: Select the appropriate intra prediction mode to use and display the intra mode indicator accordingly. For example, the intra BC unit 48 can generate various tested intra Rate-distortion analysis of intra-prediction modes is used to calculate rate-distortion values ​​for the tested modes. The intra prediction mode with the best rate-distortion performance among the modes is selected as the appropriate intra prediction mode to be used. A rate-distortion analysis is generally performed on the encoded The block and the original encoding that was encoded to generate the block being encoded. The amount of distortion (or error) between the encoded block and the unencoded block. Determines the bit rate (i.e., number of bits) used to generate the clock. The BC unit 48 performs distortion and rate optimization for various encoded blocks. Calculate the ratio to indicate which intra prediction mode gives the best rate-distortion value for the block. It can be determined whether

[0033] In another example, the intra BC unit 48 may include the motion estimation unit 42 and the motion compensation unit 43. 44 may be used in whole or in part to perform intra-BC prediction according to the practices described herein. In either case, the intra-blox model can be implemented for the purpose of measurement. For block copy, the predicted block is computed by sum of absolute differences (SAD), sum of squared differences (SSD) or other difference metric. The predicted block is identified as a block that matches the sub-integer pixel This may include calculating the value of the rule position.

[0034] The predicted block is either from the same frame by intra prediction or inter Whether the codes are from different frames due to prediction, video encoder 20 Subtract the pixel values ​​of the predicted block from the pixel values ​​of the current video block to be coded; A residual image block can be formed by forming pixel value differences. The pixel difference values ​​forming a block may include both differences in luminance and chrominance components. stomach.

[0035] The intra-prediction processing unit 46 performs the motion estimation unit 42 and the motion compensation unit 44. or the intra prediction performed by the BC unit 48. As an alternative to intra block copy prediction, the current video block can be intra-block copied as described above. In particular, intra-prediction processing unit 46 can predict the current block. In order to do so, the intra prediction mode to be used for coding can be determined. , intra-prediction processing unit 46 may, for example, perform various The current block can be encoded using intra prediction mode, The measurement processing unit 46 (or, in some examples, the mode selection unit) From the intra prediction modes, an appropriate intra prediction mode to be used can be selected. Intra-prediction processing unit 46 processes the selected intra-prediction mode for the block by The indicative information can be provided to entropy encoding unit 56. The entropy encoding unit 56 encodes the selected intropy in the bitstream. Information indicating the trajectory prediction mode can be encoded.

[0036] A prediction processing unit 41 performs a conventional prediction process via either inter-prediction or intra-prediction. After determining the predicted block of the video block, Summer 50 predicts from the current video block. The residual image block is formed by subtracting the residual image block. The image data can be contained in one or more transformation units (TUs), and the transformation processing unit The transform processing unit 52 is provided with a discrete cosine transform (DCT) or The residual video data is transformed into residual transform coefficients using a transform, such as a substantially similar transform.

[0037] Transform processing unit 52 sends the resulting transform coefficients to quantization unit 54. Quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process reduces the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting the quantization parameter. In some examples, quantization unit 54 then generates a matrix containing the quantized transform coefficients. Alternatively, you can use the entropy encoding unit to Unit 56 is capable of performing the scan.

[0038] Following quantization, the entropy encoding unit 56 generates, e.g., a context Context-adaptive variable length coding (CAVLC), Context-adaptive binary arithmetic coding (CABAC), Syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval segmentation encoding PIPE coding, or another entropy encoding method or technique The quantized transform coefficients are entropy encoded into the video bitstream using a The encoded bitstream is then sent to video decoder 30, and are archived in storage device 32 for later transmission or retrieval to video decoder 30. The entropy encoding unit 56 can encode the actual image data. Entropy encoding motion vectors and other syntax elements of a video frame It is also possible.

[0039] The inverse quantization unit 58 and the inverse transform processing unit 50 perform inverse quantization and inverse transformation, respectively. A transformation is applied to the pixels to generate reference blocks for the prediction of other video blocks. As mentioned above, the motion compensation unit 4 4 is a motion compensated prediction block from one or more reference blocks of the frame stored in the DPB 54. Motion compensation unit 44 may generate one or more interpolated frames. A filter is applied to the predicted block to calculate sub-integer pixel values ​​for use in motion estimation. This can be done.

[0040] The sum 52 is the reconstructed residual block generated by the motion compensation unit 44. Add to the motion compensated prediction block to generate a reference block for storage in the DPB 54 The reference block is then fed to the intra BC unit 48 as a predicted block, the motion estimation unit 49 as a and motion compensation unit 44 to determine the motion of other pixels in subsequent video frames. The video blocks can be inter predicted.

[0041] FIG. 3 is a block diagram illustrating an exemplary video decoder 30 according to some implementations of the present application. The video decoder 30 includes a video data memory 69, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, a summary The prediction processing unit 81 includes a motion compensation unit 82, an image processing unit 90, and a DPB 92. The video decoder 30 further includes an intra prediction processing unit 84 and an intra BC unit 85. follows the encoding process previously described for video encoder 20 in connection with FIG. A roughly reverse decoding process can be performed. For example, the motion compensation unit 8 2 predicts the motion vectors received from the entropy decoding unit 80. The intra prediction unit 84 can generate entropy decoding data. generating prediction data based on the intra-prediction mode indicator received from unit 80; can.

[0042] In some examples, the units of the video decoder 30 may be assigned to perform implementations of the present application. Also, in some examples, implementations of the present disclosure may involve one or more of the following: For example, the intra BC unit 85 may be divided into several units. The implementation of the present invention may be implemented by a motion compensation unit 82, an intra-prediction processing unit 84, or the like, alone or in combination with the motion compensation unit 82, an intra-prediction processing unit 84, or the like. and other units of video decoder 30, such as entropy decoding unit 80. In some examples, video decoder 30 may perform the intra BC unit The function of the intra BC unit 85 may be implemented by the motion compensation unit 8 2 may be performed by other components of the prediction processing unit 81.

[0043] Video data memory 69 is decoded by other components of video decoder 30. It can store video data such as encoded video bitstreams. The video data stored in the data memory 69 is obtained from the storage device 32, for example. It is also possible to transmit video data via wired or wireless network communication to a local camera or other device. The image may be acquired from a source or stored on a physical data storage medium (e.g., a flash drive). The image data may be acquired by accessing the image data memory 6. 9 stores encoded video data from the encoded video bitstream. The video decoder 30 may include a coded picture buffer (CPB) for storing the decoded picture. The data buffer (DPB) 92 is used for decoding the video data by the video decoder 30. A reference for use in coding (e.g., in intra or inter predictive coding modes) The video data memory 69 and the DPB 92 store video data. Dynamic Random Access Memory (DRAM) (including Synchronous DRAM (SDRAM)), Magnetoresistive Random Access Memory (MRRAN) MRAM, resistive RAM (RRAM), or other types of memory devices It may be formed by any of a variety of memory devices. For purposes of illustration, a video data memory 3, the DPB 69 and the DPB 92 are shown as two separate components of the video decoder 30. However, those skilled in the art will recognize that the video data memory 69 and the DPB 92 are the same. It is clear that this can be provided by the memory device or a separate memory device. In some examples, video data memory 69 may be integrated into other components of video decoder 30. The chip may be on-chip or off-chip relative to those components.

[0044] During the decoding process, video decoder 30 decodes the video of the encoded video frames. Encoded video bits representing blocks and associated syntax elements The video decoder 30 receives the video frame level and / or video block level data. The syntax elements can be received at the block level. The bitstream is entropy decoded by a bit decoding unit 80 to obtain a quantity of coefficients, motion vectors or intra prediction mode indicators, and other syntax Entropy decoding unit 80 then generates motion vectors and and other syntax elements to the prediction processing unit 81.

[0045] If the video frame is coded as an intra-predictive (I) frame or other type If the prediction is performed for an intra-coded predicted block in the next frame, The intra prediction processing unit 84 of the unit 81 performs the signaled intra prediction mode. based on the current frame and reference data from previously decoded blocks of the current frame. Predictions for video blocks of video frames can be generated.

[0046] If the video frame is coded as an inter-predictive (i.e., B or P) frame, In this case, the motion compensation unit 82 of the prediction processing unit 81 performs entropy decoding. based on the motion vectors and other syntax elements received from the coding unit 80. generating one or more prediction blocks for a video block of the video frame; The blocks may be generated from a reference frame in one of the reference frame lists. The coder 30 uses a default construction technique based on the reference frame stored in the DPB 92. can be used to construct the reference frame lists, List0 and List1.

[0047] In some examples, video blocks are coded according to the intra BC modes described herein. When the prediction processing unit 81 is entropy decoded, the intra BC unit 85 of the prediction processing unit 81 Based on the block vectors and other syntax elements received from the reading unit 80, The video encoder generates a prediction block for the current video block based on the prediction block. 20 within the same reconstructed area of ​​the picture as the current video block defined by That's fine.

[0048] The motion compensation unit 82 and / or the intra BC unit 85 calculates the motion vectors and and other syntax elements to predict the video blocks of the current video frame. The prediction information is then used to determine a prediction for the current video block being decoded. For example, motion compensation unit 82 may generate a motion compensation block based on the received syntax element Some of these are used to determine the prediction mode used to encode a video block of a video frame. frame type (e.g., intra or inter prediction), inter prediction frame type (e.g., B or P), construction information for one or more reference frame lists for the frame, Motion vectors of inter-prediction encoded video blocks, and inter-prediction for each frame Determine the inter prediction status of the coded video block, and other information, The video block in the video frame is decoded.

[0049] Similarly, the intra BC unit 85 may also interpret some of the received syntax elements, e.g. The flag indicates that the current video block was predicted using intra BC mode. , which video blocks of the frame are in the reconstruction area and are stored in DPB 92. block vectors of each intra BC predicted video block in the frame, Determine the intra BC prediction status of the intra BC predicted video block, and other information. to decode the video blocks in the current video frame.

[0050] Motion compensation unit 82 provides motion compensation to video encoder 20 during the encoding of a video block. The interpolation is performed using the interpolation filter used by the Interpolated values ​​for the received cells can also be calculated by motion compensation unit 82. The interpolation filters used by the video encoder 20 are determined from the syntax elements. , an interpolation filter can be used to generate the prediction block.

[0051] Inverse quantization unit 86 inversely quantizes the image data captured by video encoder 20 to determine the degree of quantization. using the same quantization parameters calculated for each video block in the image frame; provided in the bitstream and decoded by the entropy decoding unit 80 The inverse transform processing unit 88 inverse quantizes the entropy decoded quantized transform coefficients. To reconstruct the residual block in the pixel domain, an inverse transform (e.g., an inverse DCT , an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients.

[0052] The motion compensation unit 82 or the intra BC unit 85 processes the vectors and other syntax After generating a prediction block for the current video block based on the pixel elements, Summer 90 Residual blocks from the inverse transform processing unit 88 and the motion compensation unit 82 and the intra BC and the corresponding predicted block generated by unit 85. Reconstruct the decoded video block of the video block. ) is located between Summer 90 and DPB 92 and further decodes the video blocks. The decoded video blocks in a given frame can then be processed. The reference block DPB92 is stored in the reference block DPB92, which is used for subsequent motion compensation of the next video block. The DPB92 or a memory device separate from the DPB92 is used to store the frames. 1 for presentation on a display device such as display device 34. It is also possible to store coded images.

[0053] In a typical video coding process, a video sequence is typically split into ordered frames. It contains a set of frames or pictures. Each frame is denoted by SL, SCb, and SCr. SL is a two-dimensional array of luminance samples. SCb is a two-dimensional array of Cb chroma samples. SCr is a two-dimensional array of Cr chroma samples. In other examples, the frame may be monochrome. It is small and therefore contains only one two-dimensional array of luma samples.

[0054] As shown in FIG. 4A, video encoder 20 (or more specifically, splitting unit 4 5) first divides a frame into a set of coding tree units (CTUs). The video frame is encoded from left to right and top to bottom. Each can contain an integer number of CTUs arranged consecutively in raster scan order. A CTU is the largest logical coding unit, and its width and height are the same as those of a video sequence. All CTUs within the 128x128, 54x64, 32x32, and 16x16 The sequence parameter sets are the same size as the video encoder 20. However, this application is not necessarily limited to a particular size. As shown in Figure 4B, each CTU is a single luminance sample. Coding tree block (CTB), the coding tree block of the corresponding two chroma samples, and the code It may contain syntax elements used to encode samples of the coding tree block. The syntax elements define the properties of different types of units of a coding block of pixels. and how the video sequence can be reconstructed by the video decoder 30; Inter or intra prediction, intra prediction mode, motion vectors, and other parameters For a monochromatic picture or a picture with three distinct color planes, the CTU contains: A single coding tree block and the coding tree block samples used to encode it. A coding tree block is an NxN block of samples. Good too.

[0055] To achieve better performance, the video encoder 20 may add a For example, binary tree decomposition, ternary tree decomposition, quad tree decomposition, or any combination of the above. The division can be performed recursively to divide the CTU into smaller coding units (CUs). As shown in Figure 4C, the 54x64 CTU 400 is first divided into four smaller CUs. Each smaller CU has a block size of 32x32. Among the CUs, CU410 and CU420 each have four 16x16 block-sized C The two 16x16 CUs 430 and 440 are each divided into four 8x8 blocks. 4C. 0 partitioning process, each leaf node of the quadtree is 32× This corresponds to one CU of a suitable size ranging from 32 to 8x8. Similar to the ,each CU consists of a coded block (CB) of luma samples and two equally sized frames. The corresponding coding block of the chroma samples of the frame and the samples of the coding block are coded. It may also contain syntax elements used to represent monochrome pictures or three different In a picture with color planes, a CU consists of a single coding block and and the syntax structure used to encode the sample. The quadtree decomposition shown in Figure 4D is for illustrative purposes only and does not represent a single CTU divided into multiple CUs. Partitioning to adapt to different local characteristics based on quadtree / ternary / binary tree partitioning Note that in a multi-type tree structure, one CTU can be Each quadtree leaf CU is further divided by binary and ternary tree structures. As shown in FIG. 4E, four-way split, horizontally split in two, vertically split in two, horizontally split in three, There are five vertically divided types.

[0056] In some implementations, video encoder 20 may divide the coding blocks of a CU into one or more The prediction block can be further divided into M × N prediction blocks (PBs) of the same Rectangular (square or non-square) samples to which measurements (inter or intra) are applied The prediction unit (PU) of a CU predicts two blocks of luminance samples. the corresponding predicted block of chroma samples of , and the predicted block used to predict It may contain syntax elements such as: Monochromatic pictures or pictures with three distinct color planes In this case, a PU has a single prediction block and a The video encoder 20 may include a syntax structure for the luminance, Cb and generating predictive luminance, Cb, and Cr blocks for the Cr prediction block. can be done.

[0057] The video encoder 20 uses intra prediction or inter prediction to generate a prediction block for the PU. The video encoder 20 can generate a PU prediction using intra prediction. To generate the block, video encoder 20 decodes the frame associated with the PU. Based on the loaded samples, a prediction block for the PU can be generated. If the coder 20 uses inter prediction to generate the prediction block of the PU, the video encoder The data block 20 is used to decode one or more frames other than those associated with the PU. Based on the obtained samples, a predicted block of the PU can be generated.

[0058] The video encoder 20 calculates the predicted luminance, Cb, and y for one or more PUs of the CU. After generating the Cr and Cr blocks, video encoder 20 may then extract the Cr and Cr blocks from the original luma coding blocks of the CU. The luminance residual block of the CU is generated by subtracting the predicted luminance block from the CU. Thus, each sample of the luma residual block of the CU is The luminance samples in one of the blocks are compared with the corresponding samples in the original luminance coding block of the CU. Similarly, video encoder 20 may indicate the difference between the Cb residual block and the Cb residual block of a CU. The Cb and Cr residual blocks can be generated respectively, which results in the Cb residual block of the CU. Each sample in the block is the combination of a Cb sample in one of the predicted Cb blocks of the CU and the original Cb of the CU. b indicates the difference between the corresponding samples in the coding block, and Cr indicates the residual block of CU. Each sample in is the difference between the Cr sample in one of the predicted Cr blocks of the CU and the original Cr It may indicate the difference between corresponding samples in the coding block.

[0059] Furthermore, as shown in FIG. 4C, video encoder 20 may use quadtree partitioning to generate C Subtract the luma, Cb, and Cr residual blocks of U from one or more luma, Cb, and Cr A transformation block is a rectangular block to which the same transformation is applied. A CU is a block of samples (square or non-square). , a transform block for luma samples, two corresponding transform blocks for chroma samples, and a transform It may also contain syntax elements used to transform block samples. Therefore, each TU of a CU is composed of a luminance transform block, a Cb transform block, and a Cr transform block. In some examples, the luma transform block associated with the TU may be The Cb transform block may be a sub-block of the luminance residual block of U. The Cr transform block may be a sub-block of the Cr residual block of the CU. In a monochromatic picture or a picture with three separate color planes, T U is a single transform block and the signal used to transform the samples of the transform block. It may also include a tax structure.

[0060] Video encoder 20 applies one or more transforms to the luma transform block of the TU to A block of luminance coefficients for a TU can be generated. The block of coefficients is a two-dimensional array of transform coefficients. The transform coefficients may be rays. The transform coefficients may be scalar quantities. The transform can be applied to the Cb transform block of a TU to generate the Cb coefficient block of the TU. Video encoder 20 applies one or more transforms to the Cr transform blocks of the TU. The Cr coefficient block of the TU can be generated by the above.

[0061] A coefficient block (e.g., a luma coefficient block, a Cb coefficient block, or a Cr coefficient block) After generating the coefficient blocks, video encoder 20 may quantize the coefficient blocks. Quantization generally refers to the process of quantizing transform coefficients to reduce the amount of data used to represent them. This refers to a process whereby the video encoder 20 potentially reduces the coefficient After quantizing a block, video encoder 20 may generate a symbol indicating the quantized transform coefficients. The tax elements can be entropy encoded. For example, is a context-adaptive binary quantization for syntax elements that indicate quantized transform coefficients. Finally, video encoder 20 may perform coding. The sequence of bits that form a representation of the encoded frame and associated data is The bitstream can be output to a storage device 32. or transmitted to the destination device 14.

[0062] After receiving the bitstream generated by the video encoder 20, the video decoder 30 parses the bitstream and obtains syntax elements from the bitstream. The video decoder 30 can extract syntax elements from the bitstream. Based at least in part on the image data, a frame of video data can be reconstructed. The process of reconstructing the image data is generally performed by the encoding process performed by video encoder 20. For example, video decoder 30 may The inverse transform is performed on the associated coefficient block to generate the TU associated with the current CU. The video decoder 30 can further reconstruct the residual block of the current CU. Add the samples of the predicted block of the PU to the corresponding samples of the transform block of the TU of the current CU. The coding block of the current CU is reconstructed by calculating the coding block of each CU of the frame. After reconstructing the coded blocks, video decoder 30 can reconstruct the frame. .

[0063] As mentioned above, video coding is primarily performed in two modes: intra-frame prediction (or Video compression is achieved using intra-frame prediction (or inter-prediction) and inter-frame prediction (or inter-prediction) Palette-based coding is another coding style that is used in many video coding standards. This scheme is a palette-based coding scheme that is particularly suited to screen-generated content coding. , a video encoder (e.g., video encoder 20 or video decoder 30) may A palette table of colors representing the image data of a given block is created. contains the most dominant (e.g., frequently used) pixel values ​​in the block of Pixel values ​​that are not frequently represented in the video data of a block are included in the palette table. Not included in the palette table as an escape color.

[0064] Each entry in the palette table is an index of the corresponding pixel value in the palette table. The palette index for the samples in the block is stored in the palette table. Indicates which entries from the table are used to predict or reconstruct which samples. This palette mode can be coded as a picture, slice, tile, or the palette predictor of the first block of any other such grouping of video blocks As explained below, the process begins with generating a palette predictor for the subsequent video block. is usually generated by updating a previously used palette predictor. For this purpose, the palette predictor is assumed to be defined at the picture level. A palette may contain multiple coding blocks, each of which has its own palette. There is one palette predictor for the whole picture, but one palette predictor for the whole picture.

[0065] The number of bits required to signal a palette entry in a video bitstream To reduce the overhead, the video decoder uses a palette predictor to reconstruct video blocks. You can determine a new palette entry in the palette table that will be used to For example, the palette predictor can be a palette entry from a previously used palette table. It may contain all entries in the most recently used pallet table. It may be initialized with the most recently used palette table by The palette predictor uses the most recently used palettes over all entries from the palette table. Contains few entries, then some entries from other previously used pallet tables. A palette predictor can be used to code different blocks. It may have the same size as the palette table used, or it may encode different blocks. It may be larger or smaller than the pallet table used for The palette predictor is a first-in-first-out (FIFO) table containing 54 palette entries. It is executed as a

[0066] to generate a palette table for a block of video data from the palette predictor. The video decoder extracts each element of the palette predictor from the encoded video bitstream. It can receive a 1-bit flag for the palette reservation entry. The first value ( For example, a binary 1), or the associated entry of the palette predictor is a palette predictor. It may have a second value (e.g., a binary 0) that indicates that the If the size of the palette predictor is larger than the palette table used for the block of video data, If the maximum size of the palette table is reached, the video decoder will not You may stop receiving flags.

[0067] In some runs, some entries in the palette table may contain palette predictors. signaled directly in the encoded video bitstream, rather than being determined using For such entries, the video decoder may From the image bitstream, the picture of the luma and two chroma components associated with the entry Three separate m-bit values ​​may be received to indicate the cell value, where m is the video data Represents the bit depth of the data. Multiple directly signaled palette entries are required. Compared to m-bit values, these palette entries obtained from the palette predictor are 1-bit Therefore, you can use the palette predictor to select some or all of the Signaling a palette entry signals a new palette table entry. This significantly reduces the number of bits required for signaling, making palette mode encoding Improves overall coding efficiency.

[0068] In many cases, the palette predictor for a block is one or more pre-signed is determined based on the palette table used to encode the mapped block. However, when encoding the first coding tree unit of a picture, slice, or tile, In this case, the palette table of the previously coded block is not available. A new predictor cannot be generated using a previously used palette table entry. In this case, the sequence of the palette predictor initializer is determined by the sequence parameter set ( may be signaled in the Picture Parameter Set (PPS) and / or Picture Parameter Set (PPS). Sequence Parameter Set (SPS) and / or Picture Parameter Set ( PPS) generates a palette predictor when a previously used palette table is not available. The SPS is typically found in each slice segment header. The syntax element is determined by the content of the syntax element found in the PPS to which it points. A series of consecutive coded images, called a coded video sequence (CVS), as determined by PPS refers to the syntax structure of syntax elements that apply to a video picture. As determined by the syntax elements found in each slice segment header: Syntax for syntax elements that apply to one or more single pictures in a CVS Therefore, SPS generally has a higher level of syntactic structure than PPS. and the syntax elements contained in the SPS are Constraint elements generally change infrequently and apply to the majority of video data. be.

[0069] FIG. 5 illustrates a coding block of a video frame of a bitstream in accordance with some embodiments. 5 shows a process 500 for obtaining chroma samples 502 from luma samples 504 in 506. The coding block 506 of the image contains a plurality of pixels, each of which is made up of a plurality of color components ( For example, blue, green, and red). In image processing, the brightness and color information of multiple pixels is represented by multiple luminance samples. Each pixel of the plurality of pixels is represented by a plurality of chroma samples 504 and a plurality of chroma samples 502. Each chroma sample 502 corresponds uniquely to a single corresponding luma sample 504. Each brightness corresponds to a corresponding set of brightness samples 504 according to the sampling scheme. Each luminance sample has a luma component Y′, and each chroma sample 504 has a blue-difference chroma component Cb and It has a red difference chroma component Cr. The subsampling scheme for the luma and chroma components (Y':C b:Cr) are, for example, 4:1:1, 4:2:0, 4:2:2, 4:4:4 and 4:4: 5, the luminance samples 504 and 505 of the video frame have a ratio of three parts. The saturation and chroma samples 502 are subsampled with a three-part ratio equal to 4:1:1. It complies with the GIS scheme.

[0070] In some embodiments, the coding blocks 506 of a video frame are 2M wide. 2M x 2N pixels corresponding to 2N luminance samples across the luminance samples and height M and N are optionally equal to or different from each other. An exemplary subsampling scheme for the chroma components (Y':Cb:Cr=4:1:1) For example, the luminance samples 504 of a video frame are encoded at a resolution of 2Mx2N, while The chroma samples 502 are encoded at a lower resolution of MxN. The pull 502 is 2M x 2N (for example, 4:4:4 full sampling), 2M x N (for example, 4 :4:0 subsampling), M×2N (e.g. 4:2:2 subsampling), 1 / 2 Encoded at different chroma resolutions such as Mx2N (e.g., 4:1:1 subsampling) This may also be done.

[0071] TIFF2025183342000002.tif97160

[0072] TIFF2025183342000003.tif79160

[0073] In some embodiments, the luma samples 504 and chroma samples 502 of a video frame On average, four luminance samples 504 are assigned to the blue-difference chroma component Cb and the red-difference chroma component Cr. The subsampling scheme corresponds to one chroma sample 504 having In other words, the first luma resolution of luma samples 504 is the second chroma resolution of chroma samples 502. On average, every four luma samples 504 are at the second chroma resolution. One downsample in the set of luma samples 508 having an equal third luma resolution In some embodiments, each luminance sample is combined with the sampled luminance sample 508. For each pixel 508, there are four neighboring luminance samples 504 and more neighboring luminance samples 506. 4 is applied to obtain the corresponding chroma samples 502. A subset of the luma samples 504 For each sample, each luma sample 504 is applied multiple times to produce one or more downsampled A combined luma sample 508 and / or one or more chroma samples 502 are generated. The plurality of luma samples 504 are interpolated based on a predefined interpolation scheme 510. For more information on downsampling to 508 and / or chroma samples 502, see , as described below with reference to Figures 6A-6D.

[0074] In some embodiments, a suitable method for converting luma samples 508 to chroma samples 502 is provided. The linear mapping model 512 used is obtained using the Max-Min method. In practice, the maximum downsampled luma sample is the The smallest downsampled luma sample is identified from the set of luma samples. The maximum chroma sample is identified from the set of unsampled luma samples. The largest downsampled luma sample corresponds to the largest downsampled luma sample, and the smallest chroma sample corresponds to the smallest downsampled luma sample. The maximum and minimum chroma samples correspond to the maximum and the minimum downsampled luminance samples are applied to achieve linear mapping. The linear model 512 is determined, i.e., the two coefficients α and β in equation (1) are determined. After the mapping model 512 is obtained, the video codec uses the linear mapping model 512 to the downsampled luma samples 508 from the luma samples 504, The corresponding chroma samples 502 in the coding block 506 are generated.

[0075] The coded blocks of a video frame are reconstructed according to an ordered sequence. In some circumstances, the coding block 506 may be the first row or the first column of a video frame. When the encoding block 506 is being processed, the group of selected pixels For example, the selected group of pixels is reconstructed in the coding block 506. Contains pixels located directly above and to the left of the pixel. luminance samples 518, and a selected group of downsampled luminance samples 514. The loop is reconstructed for a group of selected pixels, e.g., using the Max-Min method. can be used to obtain a linear mapping model 512 based on the modulus. , luma samples 518, and downsampled luma samples 514 are encoded as It is outside the boundary of block 506 and is directly adjacent to that boundary. In an embodiment, the video codec performs a gradient of the encoded downsampled luminance samples. loop (e.g., the downlink corresponding to the upper neighboring luminance sample and the left neighboring luminance sample) downsampling (a selected group of 514 luminance samples) to find the maximum downsampling downsampled luminance samples 514A-1 and the minimum downsampled luminance The video codec then identifies the maximum and minimum downsampling corresponding to sampled luminance samples 514A-1 and 514B-1 (e.g., A selection of 516 saturation samples, including the adjacent saturation samples above and the adjacent saturation samples to the left. Previously encoded chroma samples 516A-1 and 516B (in a group -1. In this way, the linear mapping model 512 identifies these associated Downsampled luma samples (514A-1 and 514B-1) and chroma can be obtained based on the samples (516A-1 and 516B-1).

[0076] Alternatively, in some embodiments, the video codec may select a group of selected luminance samples. Downsampling is performed on the group to obtain the maximum and minimum luminance samples 518A and 518B. 518B and 518C, without distinguishing between groups of luminance samples (e.g., the top and left neighbors) Search for luminance samples (a selected group of 518 luminance samples, including luminance samples) and Within the group of samples, (i) the luminance sample 518A having the largest luminance value and (ii) the The video codec then identifies the luma samples 518B that have the smallest luma values. and downsampling in the area associated with the lowest luminance samples 518A and 518B. Perform downsampling (e.g., in a region with 6 samples, perform 6 tap downsampling) Any weighted averaging scheme known in the art, including weighted averaging, can be used), downsampling, Generate the pulled luminance sample 514A-2 as the maximum luminance sample (the luminance Sample 514A-2 is indeed the largest downsampled luminance sample 514A (may be -1, but not necessarily), downsampled luma samples 51 4B-2 as the smallest luminance sample (the luminance sample 514B-2 is certainly The minimum downsampled luma sample may be 514B-1, or (It doesn't matter.) The video codec then converts the downsampled luma samples 514A -2 and 514B-2 (e.g., the chroma sample containing the upper and left adjacent chroma samples) saturation samples 516A-2 and 516B-2 (in the group of saturation samples 516) In this way, the linear mapping model 512 identifies the downsampled luminance Samples (514A-2 and 514B-2) and chroma samples (516A-2 and 516B-2).

[0077] Alternatively, in some embodiments, the video codec may use a group of chroma samples (e.g. For example, a group of 516 saturation samples containing the neighboring saturation samples above and to the left , maximum and minimum saturation samples 516A-3 and 516B-3 (e.g., maximum and minimum, respectively) The video codec then identifies the chroma samples with the highest and lowest chroma values. corresponding to the large and smallest saturation samples 514A-3 and 514B-3 (e.g., and the left adjacent luma sample. Downsampled luminance samples 514A-3 and 514B- 3. In this way, the linear mapping model 512 identifies the downsampled Luminance samples (514A-3 and 514B-3) and chroma samples (516A-3 and and 516B-3).

[0078] Alternatively, in some embodiments, the video codec may use downsampled luminance A group of samples (e.g., a selected set of downsampled luminance samples 514) Search for a predefined number (e.g., two) of the groups with the largest brightness values. downsampled luma samples (e.g., 514A-4 and 514A-5), and a predefined number of downsamplings (e.g., two) with the smallest luminance value. Next, identify the luminance samples (e.g., 514B-4 and 514B-5) that are The image codec selects the chroma samples (e.g., 516A-4, 516A-5, 516B-4, and 516B-5) and The samples are the maximum downsampled luma samples 514A-4 and 514A-5, respectively. -5 groups and smallest downsampled luminance samples 514B-4 and 51 4B-5 groups. For each identified saturation and luminance sample group, a value (e.g., saturation or luminance value) Perform a weighted average of the The grand average saturation value, the minimum average saturation (for example, of saturation samples 516B-4 and 516B-5) the intensity value, the maximum average downsampling (e.g., of luminance samples 514A-4 and 514A-4) The filtered luminance values, and (e.g., luminance samples 514B-4 and 514B-5 ) produces the minimum average downsampled luminance value. The model 512 outputs the downsampled luminance samples (514A-4, 514A- 5, 514B-4 and 514B-5) and chroma samples (516A-4, 516A- 5, 516B-4 and 516B-5).

[0079] Additionally, in some embodiments, the linear mapping model 512 may be (e.g., downscaled) Multiple downsampling (within a selected group of sampled luma samples 514) Within a selected group of matched luma samples and (e.g., chroma samples 516 ) is obtained by creating a linear fitting curve for multiple corresponding saturation samples. Such curve fitting has a deviation error that selectively meets the tolerance. Each downsampled luma sample corresponds to a predefined number of chroma samples. A predefined number (e.g., greater than 10) of downsampled luminance samples In some embodiments, the downsampled luminance samples include a linear map. To obtain the ping model 512, the adjacent downsampled luminance samples 514 are randomly selected from the group.

[0080] 6A-6D illustrate four exemplary coding blocks 506A-506D according to some embodiments. 506D, where each coding block is a plurality of luma samples converted into a plurality of chroma samples 502. Each cross ("x") represents the location of a luminance sample 504, and each circle ("○") indicates a chroma sample 502 or a downsampled luma sample 508 Each circled cross ("x in a circle") represents the position of a saturation sample 502, The location where the luma sample 504 and the downsampled luma sample 508 overlap. Each coding block 506 is marked with a block boundary 602. Block 506 determines the corresponding subsampling schemes for the luma and chroma components (Y': Cb:Cr) and adopts a corresponding luminance interpolation scheme 510 to generate a corresponding coded block. 506 to the corresponding downsampled luma samples 508. In some embodiments of the present application, for each coding block 506 In particular, the downsampled luminance samples 508 are directly aligned with the block boundaries 602. If adjacent, all downsampled luma samples 508 are interpolated by luma interpolation scheme. 510 is used to extract adjacent luminance samples located in the same corresponding coding block 506. Each downsampled luma sample 508 (respectively A luma sample 508) always overlaps with the corresponding chroma sample 502, and the chroma sample 502 is therefore the downsampled luminance sample based on the linear mapping model 512. 508.

[0081] In some embodiments, the inter-component filter model 520 may be implemented using the luminance interpolation scheme 510 and linear interpolation scheme 520. The luminance for each coding block 506 is calculated in combination with the shape mapping model 512. The linear mapping model 512 of the interpolation scheme 510 and the two filter coefficients f(i) are Given the linear coefficients α and β, the chroma samples 502 of the coding block 506 are In particular, if the downsampled luminance sample 508 is directly adjacent to the block boundary 602, In this case, the inter-component filter model 520 is used to filter the corresponding components located in the same coding block 506. It is obtained directly from the attached luminance samples 504.

[0082] In some embodiments, the coding block 506 is the first coding block of a video frame. and no other coding blocks 506 have been reconstructed before the coding block 506. Such a coding block 506 is preferably located in the upper left corner of the video frame. The linear mapping model 512 is independent of the coding blocks in the video frame (e.g., For example, based on a previous video frame. The coding block 506 is not the first coding block of the video frame, but One or more other coded blocks 506 have been reconstructed before the block 506. For example, The coding block 506 is positioned at the center of the video frame, and the The coding block above or to the left of the adjacent block has already been processed to determine the linear mapping model 512. or reconstructing luma samples or chroma samples in the coding block 506. In some situations, the encoding block 506 may be used to At least one block is located on the upper boundary of the frame and immediately to the left of the coding block 506. One coding block has already been processed and the linear mapping model 512 has been determined. is used to reconstruct the luma or chroma samples in the coding block 506. In some situations, the coding block 506 may be used to At least one code located on the left boundary side and directly above the coding block 506 The coding block has already been processed to determine the linear mapping model 512 or the coding This is used to reconstruct luma or chroma samples in the reconstruction block 506. Regardless of the position of the coding block 506, the coding block 506 can be positioned to the right or left of the coding block 506. The coding block below has not yet been processed, and the linear matrix for the coding block 506 is determining the mapping model 512 or the luminance samples in the coding block 506 Nor can it be used to reconstruct chroma samples.

[0083] In some embodiments, the same predefined intensity interpolation scheme 510 or inter-component filter may be used. The filter model 520 performs all downsampling across the coding block 506. are applied to generate the calculated luma samples 508 or chroma samples 502, respectively. Alternatively, in some embodiments, the predefined luminance interpolation scheme 510 may be The position of the upsampled luma sample 508 or the downsampled luma sample The pixel position varies depending on the boundary pixel associated with the sample 508. In this embodiment, the inter-component filter model 520 is based on the position of the saturation sample 502 or the saturation sample 503. The pixel position varies depending on the boundary pixel associated with the sample 502. For example, Each intensity interpolation scheme 510 and each inter-component filter model 520 is The filter configurations are either square, square, or T-shaped.

[0084] Referring to FIG. 6A, a coding block 506A is obtained in the bitstream and It contains a number of luma samples 504 reconstructed from the stream, and a number of corresponding chroma samples. The vector 502 is obtained from a number of luma samples 504. In the coding block 506A , the subsampling scheme of the luma component 504 and the chroma component 502 (Y':Cb:Cr ) has a three-part ratio of 4:2:0. Each chroma sample 502 is filtered by the inter-component filter The six adjacent luminance samples 504 are combined according to the model 520. The luma samples 504 are located in the upper left corner, directly above, the upper right corner, and the lower left corner of the corresponding chroma samples 502. In other words, six of each saturation sample 502 are located in the bottom corner, directly below, and in the bottom right corner. Two adjacent luminance samples 504 are interpolated according to a predefined luminance interpolation scheme 510. chroma samples 502 are converted to downsampled luma samples 508 that overlap with the , which are then transformed into the corresponding chroma samples 502 according to a linear mapping model 512. In the example, the first chroma sample 502A and its corresponding six adjacent luma samples 502B are 504A-1 to 504A-6 are completely enclosed within block boundary 602 and are the first color. The intensity sample 502A is used to generate the inter-component filter model 520 or the intensity interpolation scheme 510 and and the linear mapping model 512 based on the combination of the adjacent luminance samples 504A-1 can be obtained from 504A-6.

[0085] Conversely, the second chroma sample 502B is located at the left block boundary 6 of the coding block 506A. Immediately adjacent to 02A, six adjacent luminance samples are filtered according to the inter-component filter model 520. 504B-1 to 504B-6. In some circumstances, the coding block 506 The left block boundary 602A of the image frame overlaps the left boundary of the image frame, and the left block boundary 602A of the image frame overlaps the left boundary of the image frame. In the coding block, two adjacent luminance samples 504B-1 and 504B-6 are Unavailable luma samples 50 that are outside the coding block 506 and are unavailable 4B-1 and 504B-6 are luminance samples 504B-1 and 504B-6, respectively. can be replicated from directly adjacent luminance samples 504B-2 and 504B-5. In one example, luma samples 504B-1 and 504B-6 are added to chroma sample 502B. The luminance samples 504B-4 and 504B-6 are symmetrical with respect to the luminance samples 504B-1 and 504B-6. Alternatively, in some circumstances, the coding block 50 The left block boundary 602A of 6 does not overlap with the left boundary of the video frame. The luminance samples 504B-1 and 504B-6 are utilized within the coding block 506 itself. However, it can be reconstructed by other adjacent coding blocks before coding block 506. The luma samples 504B-1 and 504B-2 outside the coding block 506 may be provided as and 504B-6 are luma samples 504B-1 and 504B-6 from any other coding block. 4B-6, the luma samples 50 in the coding block 506 are 4B-2 to 504B-5 may still be replicated. The second chroma sample 502B and / or the second luma sample 508B may be adjacent luma samples. The adjacent luminance samples 504B-1 to 504B-6 are further obtained. Among the two luminance samples 504B-1 and 504B-6, Two separate luminance samples (e.g., 504B-2 and 504B-3) are included in the coding block 506A. and 504B-5).

[0086] In other words, a video signal having multiple luminance samples for multiple pixels in a video frame is generated. When a bit stream is obtained, multiple pixels belong to the coding block 506A and are coded. Located within the coding block 506 and directly on the boundary 602 of the coding block 506A. The boundary pixels (e.g., the pixel corresponding to luminance sample 504B-2) are included. ) is due to one or more neighboring pixels (e.g., luminance sample 504B- Each pixel of one or more neighboring pixels corresponds to a coding block. In some embodiments, one or more neighbors of the boundary pixel are outside of block 506A. The tangent pixel is outside the video frame or image slice. One or more neighboring pixels of a boundary pixel have not yet been coded, and the coding block In some embodiments, the inter-component filter model 520 is , is limited to the coding block 506 and any neighboring pixels outside the coding block 506 The cell is considered unavailable. and 504B-5 are luminance samples corresponding to each pixel of one or more adjacent pixels. The boundary luminance samples 50 are assigned to blocks 504B-1 and 504B-6, respectively. 8B is a diagram illustrating a method for interpolating at least the boundary pixels and the pixel values ​​according to a predefined intensity interpolation scheme 510. and one or more adjacent pixel luminance samples 504B-1 to 504B-6. and determines the boundary saturation sample 502B according to the linear mapping model 512. Alternatively, the boundary saturation sample 502B is used to filter the inter-component filter model 502B. 20, the intensity samples 504B of the boundary pixel and one or more adjacent pixels Determined from -1 to 504B-6.

[0087] Referring to FIG. 6B, in coding block 506B, the luma component 504 and the chroma component The 502 subsampling scheme (Y':Cb:Cr) is a three part ratio of 4:2: 0. Each chroma sample 502 has an inter-component filter model with a cross filter shape. 520. Five adjacent luminance samples 504 are combined according to One of the brightness samples 504 overlaps with the chroma sample 502, and the remaining four adjacent brightness samples The intensity samples 504 are positioned directly above, below, to the left, and to the right of the corresponding chroma samples 502. In other words, the five adjacent luma samples 504 of each chroma sample 502 are The downsamples are then overlapped with the corresponding chroma samples 502 according to a predefined luma interpolation scheme 510. The sampled luminance samples 508 are then converted to a linear mapping model 512 Therefore, it is converted into the corresponding chroma sample 502. In one example, the third chroma sample 502C and the corresponding five adjacent luminance samples 504C-1 to 504C-5 are completely The third chroma sample 502C is enclosed within the block boundary 602 and is filtered by the inter-component filter model. 520 or a combination of an intensity interpolation scheme 510 and a linear mapping model 512. Based on this, adjacent luminance samples 504C-1 to 504C-5 can be obtained.

[0088] The fourth chroma sample 502D is located at the left block boundary 602A of the coding block 506B. , and five adjacent luminance samples 504 according to the inter-component filter model 520. In some circumstances, the left side of the coding block 506B corresponds to the coding block 506D-1 through 504D-5. The block boundary 602A overlaps the left boundary of the video frame and is not part of this or any other mark. The adjacent luminance sample 504D-5 is unavailable in the encoding block. 504D-5 is derived from the luminance sample 504D-1, which is directly adjacent to the luminance sample 504D-5. or symmetrical with respect to the luminance sample 504D-5 with respect to the central luminance sample 504D-1. In some circumstances, the coding block 5 The left block boundary 602A of 06 does not overlap with the left boundary of the video frame. Sample 504D-5 is not available within coding block 506 itself, but is The reconstructed encoded block may be provided by other adjacent encoded blocks reconstructed before block 506B. The luma samples 504D-5 outside of the coding block 506 are not included in any other coding block. 5D-5 from the coding block 504D-6. 504D-1 or 504D-3 located in the luminance sample 504D-1 or 504D-3 located in the luminance sample 504D-2. By these means, the fourth chroma sample 502D and / or the The fourth luminance sample 508D is derived from adjacent luminance samples 504D-1 to 504D-5. The luminance sample 504D-1 to 504D-5 is included in the adjacent luminance samples 504D-1 to 504D-5. D-5 corresponds to luminance sample 504D-1 or 504D-3 in coding block 506B. are selectively allocated based on

[0089] The fifth chroma sample 502E is located at the left block boundary 602A of the coding block 506B. and immediately adjacent to both the upper block boundary 602B, Thus, it corresponds to five adjacent luminance samples 504E-1 to 504E-5. In an embodiment, coding block 506B is the first coding block in a video frame. , two adjacent luminance samples 504E-2 in this or any other coding block and 504E-5 are not available. Luminance samples 504E-2 and 504E-5 are In some situations, the coding block 506 B is not located in the top left corner of the video frame and is adjacent to its left and above Adjacent luma samples 504E-2 and 504E-5 are coded blocks. Other blocks that are not available within the block 506 itself but are reconstructed before the encoding block 506 Alternatively, the luma sample 504E-2 may be provided by a neighboring coding block. and 504E-5 by loading luma samples from any other coding block. from the luminance sample 504E-1 located in the coding block 506 without relying on By these means, the fifth chroma sample 502E and and / or the fifth luminance sample 508E is selected from the adjacent luminance samples 504E-1 to 504E-2. E-5, and the luminance samples in the adjacent luminance samples 504E-1 to 504E-5 are Samples 504E-2 and 504E-5 are located at block boundary 60 of coding block 506B. 2. I can't stand it.

[0090] Referring to FIG. 6C, in coding block 506C, the luma component 504 and the chroma component The 502 subsampling scheme (Y':Cb:Cr) is a three part ratio of 4:2: Each chroma sample 502 has an inter-component filter model with a 2x3 filter shape. The six adjacent luminance samples 504 are combined according to the formula 520. One of the luma samples 504 overlaps with the chroma sample 502, and the remaining five adjacent The luma samples are located to the left, right, bottom left corner, directly below, and to the right of the corresponding chroma samples 502. These six adjacent luma samples 504 for each chroma sample 502 are According to a predefined luma interpolation scheme 510, the downsamples overlap with the corresponding chroma samples 502. are converted to unsampled luminance samples 508, which are then converted to a linear mapping model 512. In one example, the sixth chroma sample 502 is F and its corresponding six adjacent luminance samples 504F-1 to 504F-6 are The sixth chroma sample 502F is entirely enclosed within the block boundary 602 and is filtered by the inter-component filter model. 520 or a combination of the intensity interpolation scheme 510 and the linear mapping model 512. can be obtained from adjacent luminance samples 504F-1 to 504F-5 based on .

[0091] The seventh chroma sample 502G is located at the left block boundary 602A of the coding block 506C. and immediately adjacent to both the lower block boundary 602C, Thus, it corresponds to six adjacent luminance samples 504G-1 to 504G-6. In an embodiment, coding block 506C is located in the bottom left corner of the video frame, and The left and right block boundaries 602A and 602C correspond to the left and right block boundaries of the video frame. In some embodiments, the coding block 506C overlaps the bottom block boundary. Since the luminance samples 504G-1 and 504G-2 are located on the left boundary side of the frame, G-6 does not exist, and luma samples 504G-4 and 504G-5 have not yet been coded. In the bottom left corner or left border of the video frame, four adjacent luma samples 504G- 1, 504G-4, 504G-5, and 504G-6 are equivalent to said or any other encoding Each of these luma samples is not available within the block. 504G-2 or 504G-3, for example, luminance sample 504G-4 is The luminance samples 504G-1, 504G-5 and 504G-6 are replicated from the luminance sample 504G-3. and 504G-6 are replicated from luminance sample 504G-2. The image block 506C is not located in the bottom left corner of the video frame, but is located to the left and below it. Adjacent coding blocks of luminance samples 504G-1, 504G-4 are available. , 504G-5 and 504G-6 are not available within the coding block 506 itself, , provided by other adjacent coding blocks reconstructed before coding block 506. Alternatively, the luminance samples 504G-1, 504G-4, 504G-5 and 5 04G-6 does not rely on loading luma samples from any other coding block. luminance samples 504G-1 and 504G-2 are located within the coding block 506. By these means, the seventh chroma sample 5 may still be replicated from G-3. 02G and / or the seventh luminance sample 508G are adjacent to the luminance sample 504G-1 504G-1 to 504G-6, and the adjacent luminance samples 504G-1 to 504G-6 The luminance samples 504G-1, 504G-4, 504G-5 and 504G-6 are Two other luminance values ​​are positioned adjacent to the block boundary 602 of the encoding block 506C. The samples 504G-2 and 504G-3 are selectively assigned based on the samples 504G-2 and 504G-3.

[0092] Referring to FIG. 6D, in coding block 506D, the luma component 504 and the chroma component The 502 subsampling scheme (Y':Cb:Cr) is a three part ratio of 4:4: Each chroma sample 502 has an inter-component filter model with a 2x3 filter shape. The six adjacent luminance samples 504 are combined according to the formula 520. One of the luma samples 504 overlaps with the chroma sample 502, and the remaining five adjacent The luma samples are located to the left, right, bottom left corner, directly below, and bottom right corner of the corresponding chroma samples 502. These six adjacent luma samples 504 for each chroma sample 502 are , which overlap with the corresponding chroma samples 502 according to a predefined luma interpolation scheme 510. The unsampled luminance samples 508 are then converted to linear mapping models 51 2, and then converted into the corresponding chroma samples 502 according to

[0093] The eighth chroma sample 502H is located at the bottom block boundary 602C of the coding block 506D. and immediately adjacent to both the right block boundary 602D, Thus, it corresponds to six adjacent luminance samples 504H-1 to 504H-6. In an embodiment, coding block 506D is located in the bottom right corner of the video frame, and The right and bottom block boundaries 602D and 602C correspond to the right and bottom block boundaries of the video frame. overlaps the bottom block boundary. Luminance samples 504H-3 to 504H-6 are not available. In this example, coding block 506D is not located in the bottom left corner of the video frame, but is located to the right of it. Since the adjacent coding blocks below and above have not yet been reconstructed, The data is not yet available. Each of these luminance samples 504H-3 to 504H-6 The sample is replicated from the luminance sample 504H-1 or 504H-2, e.g., Sample 504H-6 is replicated from luminance sample 504H-1 and -3 to 504H-5 are replicated from luminance sample 504H-2. Therefore, the eighth chroma sample 502H and / or the eighth luma sample 508H are adjacent The luminance samples 504H-1 to 504H-6 are obtained from the adjacent luminance samples 50 The luminance samples 504H-3 to 504H-6 are encoded in 504H-1 to 504H-6. Two other luminance samples positioned adjacent to the block boundary 602 of block 506D. The access points 504H-1 and 504H-2 are selectively assigned based on the access points 504H-1 and 504H-2.

[0094] FIG. 7 illustrates a video data decoding process performed on an electronic device, according to some embodiments. 7 is a flowchart of a method 700 for detecting multiple pixels in a video frame. A bitstream is obtained having coded blocks 506 containing luma samples 504. (702). A plurality of pixels are located within the coding block 506 and The electronic device includes boundary pixels that are immediately adjacent to the boundary 602 of the block 506. It is determined (704) that one or more neighboring pixels of the cell are unavailable. Each pixel in the or each of the neighboring pixels is outside the coding block 506. The luminance sample corresponding to a pixel (e.g., 504G-1 in FIG. 6C) is one or more The luminance samples corresponding to each pixel in the neighborhood (e.g., 504G-6 in FIG. 6C) ) (706). The electronic device assigns the , the intensity samples of at least the boundary pixel and one or more adjacent pixels (e.g. , 502G-1 to 502G-6) based on the boundary saturation samples (e.g., 50 in Figure 6C) 2G). Specifically, the electronic device determines a predefined luminance interpolation scheme 51 0, take intensity samples of at least the boundary pixel and one or more adjacent pixels. Based on the rule (e.g., 502G-1 to 502G-6), the boundary luminance samples (e.g., 6C) (710). ) is determined from the boundary luminance samples according to the linear mapping model 512 (712) .

[0095] In some embodiments, the predefined intensity interpolation scheme 510 may include at least one boundary pixel. The pixel location is determined based on the pixel location, and one or more neighboring pixels are selected based on the predefined brightness. neighboring boundary pixels based on a degree interpolation scheme 510. In some situations, a predefined The luma interpolation scheme 510 also interpolates the luma and chroma samples of multiple pixels. is determined based on a subsampling scheme conforming to the three-part Y'CbCr ratio. In some situations, the predefined luminance interpolation scheme 510 may also be used to Syntax elements derived from the stream (e.g., "sps_chroma_colloca ted_vertical_flag"), optionally The defined intensity interpolation scheme 510 may be in a cross, block, or T-shaped filter configuration. corresponds to one.

[0096] In some embodiments, one or more neighboring pixels of a boundary pixel are In some embodiments, one of the boundary pixels or A number of adjacent pixels have not yet been coded and are coded following the coding block.

[0097] In some embodiments, the pixels are first completely surrounded by a coding block. 6A-6C. 8C, and 508F) are used to interpolate the first pixel according to a predefined luminance interpolation scheme 510. The set of luminance samples corresponding to the set of first pixels is determined from the set of luminance samples corresponding to the set of first pixels. The corresponding internal saturation samples (e.g., 502A, 502C, and 50 2F) is determined based on the interior luminance samples according to a linear mapping model 512. do.

[0098] In some embodiments, for example, in FIG. 6A, a plurality of pixels may be coded as a coding block. and the corresponding internal luminance samples (e.g., 504B-3, 504B-4) The boundary luminance samples (e.g., 508B) are based on a predefined luminance. The interior luminance samples of the interior pixels (e.g., 504B-3) are calculated according to the luminance interpolation scheme 510. , 504B-4) and the luminance samples of the boundary pixel and one or more adjacent pixels ( For example, it is determined based on both 504B-1 and 504B-2.

[0099] In some embodiments, for example in FIG. 6A, the boundary pixel is the first boundary pixel. (e.g., corresponding to luminance sample 504B-2) and one or more adjacent pixels The cell is connected to one or more first neighboring pixels (e.g., luminance sample 504B-1). The pixels are located within the coding block 506 and correspond to the coding block. A second boundary pixel (e.g., a luminance pixel) immediately adjacent to the corresponding block boundary 602 of block 506 The electronic device includes a second boundary pixel (corresponding to sample 504B-5). One or more adjacent second neighboring pixels (e.g., corresponding to luminance sample 504B-6) Each pixel of the one or more second neighboring pixels is included in the coding block 50. 6. The luminance sample corresponding to the second boundary pixel (e.g., 504B-5) is the luminance sample corresponding to each pixel of one or more second neighboring pixels (e.g. , 504B-6), and the boundary luminance sample 508B is assigned to a predefined luminance interpolation According to the inter-pixel scheme 510, at least first and second boundary pixels (e.g., luminance subpixels) are (corresponding to samples 504B-2 and 504B-5) and the first and second adjacent pixels Both luminance samples (e.g., corresponding to luminance samples 504B-1 and 504B-6) The decision is based on the pull.

[0100] In some embodiments, for example, in FIG. 6B, one or more neighboring pixels may be: The boundary pixel (e.g., corresponding to luminance sample 504D-1) is compared to the boundary 602A. It is divided from the luminance sample (e.g., 504D-1) that is on the opposite side and corresponds to the boundary pixel. One adjacent pixel ( For example, it contains only luminance sample 504D-5).

[0101] In some embodiments, for example, in FIG. 6C , the boundaries include a first boundary 602A; The boundary pixel (e.g., corresponding to luminance sample 504G-1) is located at the first boundary 602A. and a second boundary 602C that is orthogonal to the first boundary 602A. One or more neighboring pixels are directly adjacent to the corner of the first boundary pixel. The first adjacent pixel on the opposite side of 602A (e.g., corresponding to luminance sample 504G-1) (1) a second adjacent pixel on the other side of the second boundary relative to the boundary pixel; (2) the block of coding block 506C (e.g., corresponding to luma sample 504G-5) A third neighboring pixel across the corner from the boundary pixel (e.g., luminance sample 504 It includes at least one of the following:

[0102] In some embodiments, the luminance values ​​of the pixels are calculated according to a subsampling scheme. The Y, C, and chroma samples are three-part Y', Cb, and Cc samples with one of the following ratio values: Comply with the Cr ratio: (1) 4:1:1, where every four horizontal pixels are four luminance corresponding to the sample, blue-difference chroma sample Cb, and red-difference chroma sample Cr; (2) 4: 2:0, where every 4 pixels in each 2x2 pixel block have 4 luminance samples , corresponding to the blue-difference chroma sample Cb, and the red-difference chroma sample Cr; (3) 4:2:2, Here, each of the four pixels contains four luma samples, two blue-difference chroma samples Cb, and and two red-difference chroma samples Cr; (4) 4:4:4, where four pixels The cell contains four luma samples, four blue-difference chroma samples (Cb), and four red-difference chroma samples (Rb). Corresponds to the Cr.

[0103] In some embodiments, the interpolation is performed according to a predefined luminance interpolation scheme, e.g., as shown in Figs. In C and 6D, six adjacent luminance samples 504 in a 2x3 subarray are alternately The luminance samples are downsampled to 508, and the luminance samples for each left and right column in the 2x3 array are A luminance sample is used at least twice to generate alternative luminance samples in the same row. In some embodiments, the interpolation is performed according to a predefined luminance interpolation scheme, e.g., as shown in FIG. In the example, five adjacent luminance samples 504 are downsampled to alternate luminance samples 508. The pixels are arranged in a cross-shaped sub-array centered on a central pixel.

[0104] In some embodiments, a second plurality of pixels for a second plurality of pixels in a video frame a bitstream having luma samples 518 and a second plurality of chroma samples 516; The alternative plurality of luminance samples 514 are obtained using a predefined luminance interpolation scheme 510. and has the same resolution as the chroma samples 516. The first parameter α and the second parameter β for the delta 512 are used to represent alternative brightness values. The second plurality of saturation samples 514 and the second plurality of saturation samples 516 are used to determine the saturation level. In some embodiments, the boundary saturation sample Y is calculated using a linear matrix as described in equation (1) as follows: It is determined from the boundary luminance samples X according to the mapping model 512. In some embodiments, the two largest luminance samples 514A-4 and 514A-5 are The two smallest luminance samples 514B-4 are determined among the plurality of replacement luminance samples 514. and 514B-5 are determined among the alternative luminance samples 514. The luminance samples 514A-4 and 514A-5 are averaged to form a first luminance value, and the two The smallest luminance samples 514B-4 and 514B-5 are averaged to form a second luminance value. The two first chroma samples 516A-4 and 516A-5 are the two largest luma samples. Among the second plurality of saturation samples 514 associated with 514A-4 and 514A-5 The two second chroma samples 516B-4 and 516B-5 are determined by A second plurality of chroma samples associated with the small luma samples 514B-4 and 514B-5. The two first saturation values ​​516A-4 and 516A-5 are determined in the first and two second saturation values ​​516B-4 and 516B-5 is averaged to a second chroma value corresponding to a second luminance value. The model 512 determines, based on the first and second luminance values ​​and the first and second saturation values, It is taken between the chroma sample 502 and the luma sample 508 .

[0105] In one or more examples, the functions described may be implemented in hardware, software, firmware, or other similar devices. may be implemented in software, or in any combination thereof. In this case, the functions may be stored as one or more instructions or code on a computer-readable medium. or transmitted by a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium may correspond to a tangible medium such as a data storage medium. A computer-readable storage medium or a computer program that can be transmitted from one location, for example, a communication A communication medium, including any medium that can be easily transmitted to another location based on a protocol. In this manner, computer-readable media generally include: (1) non-transitory or (2) a communication medium such as a signal or carrier wave. The data storage medium may correspond to one or more computers, or one or more The processor accesses and retrieves instructions, code, and / or data structures to Any available medium may be capable of carrying out the embodiments described herein. The computer program product may include a computer-readable medium.

[0106] The terms used in the description of the embodiments herein are intended to describe specific embodiments. The above is merely a guide and is not intended to limit the scope of the claims. Unless expressly indicated otherwise, the following terms are used in the description of the embodiments and the appended claims. The singular forms "a kind," "one," and "the" are intended to include the plural forms as well. It is further understood that as used herein, the term "and / or" refers to one or more associated refers to any and all possible combinations of the listed items and includes one or more related It is further understood that the present invention includes any and all possible combinations of the listed items. The terms "comprise" and / or "include" as used herein refer to said features, elements and / or specifies the presence of a component, but does not include one or more other features, elements, components and / or The presence or addition of groups consisting of these is not excluded.

[0107] It will be further understood that the terms "first," "second," etc. may be used herein to describe various elements. These elements may be used to clarify, but are not limited by, these terms. These terms are only used to distinguish one type of element from another. For example, a first electrode may be referred to as a second electrode without departing from the scope of an embodiment, and Similarly, the second electrode may be referred to as the first electrode. Yes, but not the same electrode.

[0108] The description herein is presented for purposes of illustration and description and is not intended to be exhaustive or to limit the invention in the form disclosed. The invention is not intended to be limiting. Many modifications, variations and alternative embodiments will be apparent to those skilled in the art. The above and the description are provided to best understand the principles and practical applications of the present invention and will enable those skilled in the art to understand various aspects of the present invention. The embodiments can be understood and best utilized in various embodiments having basic principles and various modifications. , for example, to adapt to the specific application expected. Thus, the scope of the claims is not limited to the specific embodiments disclosed and includes modifications and other embodiments. Embodiments are intended to fall within the scope of the following claims.

Claims

1. 1. A method for decoding video data, comprising: From the bitstream, multiple luma samples for multiple pixels in a video frame The plurality of pixels belong to a coding block, and the plurality of pixels belong to the coding block. and includes boundary pixels located within the block and immediately adjacent to the boundary of the coding block. And, determining that one or more neighboring pixels of the boundary pixel are unavailable; each pixel of one or more neighboring pixels being outside the coding block; The luminance samples corresponding to the boundary pixels are calculated by dividing the luminance samples by the one or more adjacent pixels. assigning the pixel to a corresponding luminance sample; At least the one or more neighboring pixels are interpolated according to a predefined luminance interpolation scheme. determining a boundary luminance sample based on the boundary pixel and the luminance sample of the boundary pixel; 、 determining boundary chroma samples from the boundary luminance samples according to a linear mapping model; And, a method for decoding said video data, said method comprising:

2. determining the predefined intensity interpolation scheme based on at least the positions of the boundary pixels; determining a luminance interpolation scheme for the one or more neighboring pixels based on the predefined luminance interpolation scheme; the cell is adjacent to said boundary pixel; The method of claim 1 further comprising:

3. The predefined luminance interpolation scheme also includes subsamples corresponding to the plurality of pixels. a coding scheme and at least one syntax element obtained from the bitstream; The method of claim 2 , wherein the determination is based on one of:

4. The one or more neighboring pixels of the boundary pixel are The method according to any one of claims 1 to 3, wherein the device is external to the chair.

5. The one or more neighboring pixels of the boundary pixel have not yet been coded, The method according to any one of claims 1 to 3, wherein the coding block is coded subsequently.

6. The plurality of pixels includes a first set of pixels completely surrounded by the coding block. Including, corresponding to the first set of pixels according to the predefined luminance interpolation scheme determining an interior luminance sample from the set of luminance samples; The first picture is calculated based on the intra-luminance samples according to the linear mapping model. and determining an internal saturation sample corresponding to the set of cells. Any of the methods described above.

7. The plurality of pixels are within the coding block and correspond to intra-luminance samples. further including an interior pixel corresponding to the The boundary luminance samples are interpolated by the intra-pixel interpolation scheme according to the predefined luminance interpolation scheme. the inner luminance sample of a pixel and the one or more adjacent pixels and the boundary pixel and the luminance samples of the 10. A method according to any preceding claim.

8. The boundary pixel is a first boundary pixel, and the one or more adjacent pixels are one or more first neighboring pixels; The plurality of pixels are located within the coding block and are relative to the coding block. a second boundary pixel immediately adjacent to the boundary of the first pixel; 10. A method according to any preceding claim, comprising: forming one or more second adjacent pixels immediately adjacent to the second boundary pixel; Each pixel of the one or more second neighboring pixels is outside the coding block. and, a luminance sample corresponding to the second boundary pixel is added to the one or more second neighboring pixels; For each pixel, a corresponding luminance sample is assigned to the boundary luminance sample. and interpolating at least the first and second boundary pixels according to a predefined intensity interpolation scheme. and both the first and second neighboring pixels; Further includes:

9. The one or more neighboring pixels are on the opposite side of the boundary to the boundary pixel. and the corresponding luminance samples allocated from the luminance samples corresponding to the boundary pixels.

10. The method of claim 9, wherein the pixel includes only one neighboring pixel having the same shape.

10. the boundary includes a first boundary; The boundary pixels are located between the first boundary and a second boundary perpendicular to the first boundary. directly adjacent to a corner of the formed coding block, The one or more neighboring pixels are a reflection of the first boundary relative to the boundary pixel. (1) a first adjacent pixel on the opposite side of the boundary pixel; (2) a second adjacent pixel on the other side of the boundary of the coding block; and (3) across a corner of the coding block. and a third adjacent pixel opposite the boundary pixel.

10. A method according to any preceding claim.

11. Luma samples and chroma samples of the plurality of pixels are subsampled according to a subsampling scheme. The sample conforms to a three-part Y'CbCr ratio with one of the following ratio values: ： (1) 4:1:1, where every four horizontal pixels contain four luminance samples, blue-difference chroma samples, corresponding to the red difference chroma sample Cb, and the red difference chroma sample Cr; (2) 4:2:0, where four pixels in each 2x2 pixel block are corresponding to the luma sample, the blue-difference chroma sample Cb, and the red-difference chroma sample Cr; (3) 4:2:2, where every four pixels contain four luminance samples and two blue-difference samples. corresponding to a dither sample Cb, and two red-difference chroma samples Cr; (4) 4:4:4, where every four pixels contain four luminance samples and four blue-difference samples. corresponding to four red-difference chroma samples Cb, Cc, and four red-difference chroma samples Cr, 10. A method according to any preceding claim.

12. According to the predefined luminance interpolation scheme, six adjacent luminance values ​​in the 2x3 sub-array are interpolated. The luminance samples are downsampled to alternate luminance samples, and each left and right in a 2x3 array The luminance samples in the right and left columns are at least is also used twice, 10. A method according to any preceding claim.

13. According to the predefined luminance interpolation scheme, five adjacent luminance samples are interpolated into alternative luminance samples. are downsampled to 1000 samples and arranged in a cross-shaped subarray centered on the central pixel. To be placed, 10. A method according to any preceding claim.

14. extracting from the bitstream a second plurality of pixels in a video frame; obtaining a first plurality of luma samples and a second plurality of chroma samples; the same resolution as the plurality of chroma samples according to the predefined luma interpolation scheme determining an alternative plurality of luminance samples having Using the alternative plurality of luma samples and the second plurality of chroma samples Thus, the first parameter α and the second parameter β for the linear mapping model are and 10. The method of any preceding claim, further comprising:

15. The boundary chroma sample Y is calculated using the following linear mapping model, which is described using the following equation: The method of claim 14, wherein the boundary luminance samples X are determined from the boundary luminance samples X according to a rule. Y = α * X + β

16. Determine a first parameter α and a second parameter β for the linear mapping model. To determine, identifying two largest luminance samples from the alternative plurality of luminance samples; identifying two smallest luminance samples from the alternative plurality of luminance samples; averaging the two largest luminance samples into a first luminance value; averaging the two smallest luminance samples into a second luminance value; among the second plurality of chroma samples associated with the two largest luma samples. identifying two first chroma samples; among the second plurality of chroma samples associated with the two smallest luma samples. identifying two second saturation samples; Averaging the two first saturation values ​​into a first saturation value corresponding to the first luminance value. And, Averaging the two second saturation values ​​into a second saturation value corresponding to the second luminance value. And, a saturation value and a second saturation value based on the first and second luminance values ​​and the first and second saturation values. obtaining a linear mapping model between the luminance values; The method of claim 14 further comprising:

17. 1. An electronic device comprising: one or more processors; Memory and Including, The memory stores a request to the processor when executed by the one or more processors.

17. An apparatus for storing instructions for carrying out the method according to any one of claims 1 to 16. The electronic device.

18. A non-transitory computer-readable medium, comprising: When executed by one or more processors, the processors are provided with any one of claims 1 to 16. and storing instructions for performing one of the methods described above. The non-transitory computer-readable medium.

Images