Method for decoding video data, electronic device, non-transitory computer-readable storage medium, method for storing video bitstream, and computer program

Abstract

To provide a decoding method that improves the interpolation of luma and chroma samples.SOLUTION: A method obtains a bitstream having a coding block including a plurality of luma samples for a plurality of pixels in a video frame, the plurality of pixels include boundary pixels belonging to the coding block and located inside the coding block and directly adjacent to a boundary of the coding block. The method also determines, for each pixel in the boundary pixels, one or more neighboring pixels outside the coding block are unavailable. Luma samples corresponding to the boundary pixels are assigned to the luma samples corresponding to each pixel in the boundary pixels. The method further determines, on the basis of the luma samples of at least the boundary pixels and the one or more neighboring pixels according to a predefined luma interpolation scheme, a boundary luma sample. The boundary chroma sample is determined from the boundary luma sample according to a linear mapping model.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a joint venture of “Simplifications of "Cross-Component Linear Model" No. 62 / 955,348, the entire contents of which are incorporated herein by reference. .

[0002] This application relates generally to video data encoding and compression, and more particularly to bit-by-bit video data encoding and compression. Improvements to the coding of chroma and luma samples of video frames in a stream The present invention relates to a method and system for [Background technology]

[0003] Digital video can be transmitted over a wide range of devices, including digital televisions, laptop or desktop computers, and tablets. bulletin computers, digital cameras, digital recording devices, digital media players players, video game consoles, smartphones, video teleconferencing devices, video streamers It is supported by various electronic devices such as mobile devices. , MPEG-4, ITU-TH.263, ITU-TH.264 / MPEG-4, Part 10, ADVANCED Video Coding (AVC), high efficiency video Coding (HIGH EFFICIENCY Video Coding) (HEVC), and VERSATILE Video Coding (VVC) standard It transmits digital video data by implementing the video compression / decompression standards defined in the Video compression is usually , perform spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to extract the video data In the case of block-based video coding, the method includes reducing or removing redundancy inherent in A video frame is divided into one or more slices, and each slice contains multiple video blocks. There are blocks, which may be called coding tree units (CTUs). Each CTU is a or until a predefined minimum CU size is reached. Each CU (also called a leaf CU) has one or more Each CU contains multiple transform units (TUs), and each CU contains one or more prediction units (PUs). Each CU is coded in either intra, inter, or IBC mode. A video block in an intra-coded (I) slice of a video frame may be , using spatial prediction with respect to reference samples in adjacent blocks in the same video frame Within an inter-coded (P or B) slice of a video frame, A video block is coded with respect to reference samples in adjacent blocks in the same video frame. Using spatial prediction or other reference samples in previous and / or future reference video frames A time prediction may be used for the pull.

[0004] Previously encoded reference blocks, e.g., spatial or temporal, based on neighboring blocks Inter prediction produces a predicted block for the current video block being coded. The process of finding the block may be performed by a block matching algorithm. The residual data, which represents the pixel difference between the current block to be encoded and the predicted block, is Inter-coded blocks are called prediction blocks or prediction errors. The motion vectors pointing to the reference blocks in the reference frames and the residual blocks are used to form the encoding. The process of determining motion vectors is usually called motion estimation. The coded block is encoded according to the intra prediction mode and the residual block. For further compression, the residual block is transformed from the pixel domain to a transform domain, e.g. The signal is transformed into the frequency domain to obtain residual transform coefficients, which may then be quantized. First, the quantized transform coefficients arranged in a two-dimensional array are scanned to obtain the first-order The source vector is generated and then entropy encoded into a video bitstream. and further compression can be achieved.

[0005] The encoded video bitstream is then stored on a computer-readable storage medium (e.g. , flash memory) and is then transmitted to other electronic devices with digital video capabilities. The information is then accessed or transmitted directly to the electronic device via wire or wirelessly. For example, the device analyzes the encoded video bitstream and extracts Get syntax elements from the bitstream and add a few and recovering digital video data from a video bitstream encoded at least in part based on the Video decompression (different from video compression above) is achieved by reconstructing the video data into its original format. Then, the reconstructed digital image data is sent to the electronic device for display. Render.

[0006] Inter-component prediction modes are predictions made between luma and chroma samples of a video bitstream. In particular, the prediction model is applied to reduce the redundancy of the luminance samples. The chroma samples are downsampled and used to predict the chroma samples. Luma samples are not available at some locations in the frame, and downsampling occurs near these locations. Different downsampling filters are used to generate the downsampled luma samples. The application of different downsampling filters is not compatible with parallel image processing and may result in image degradation. This could cause a bottleneck in the processing speed of the image bitstream. It would be beneficial to have a more efficient inter-component prediction mechanism. Summary of the Invention [Problem to be solved by the invention]

[0007] This application relates to video data encoding and decoding, and more particularly to video Encoding chroma and luma samples of a video frame in a bitstream of data The present invention relates to a method and system for improving the performance of a sample padding method. The unavailable luminance is applied to the luminance samples and used by the luminance downsampling process. Generate samples. Unavailable samples are generated outside the video frame or image slice. These uses may include: Unavailable samples are selectively generated by overlap padding or mirror padding. Therefore, the same downsampling filter is used across the entire video frame. All downsampled luma samples can be generated, which allows parallel image processing. This allows for faster image processing of the corresponding video data bitstreams. [Means for solving the problem]

[0008] In one aspect of the present application, a method for decoding video data is performed on an electronic device. The method includes extracting from the bitstream a plurality of pixels in a video frame. The method includes obtaining a plurality of luminance samples, the plurality of pixels belonging to a coding block. and the boundary pixels are within the coding block, The method further comprises: selecting one or more neighboring pixels of the boundary pixel; is unavailable and one or more luminance samples corresponding to the border pixels are is assigned to each pixel of multiple adjacent pixels as a corresponding luminance sample, and At least one or more adjacent pixels and boundary pixels are interpolated according to a defined intensity interpolation scheme. and determining a boundary luminance sample based on the luminance samples of the pixel. In this case, each pixel of the plurality of adjacent pixels is outside the coding block. determining boundary chroma samples from the boundary luma samples according to a mapping model; In some embodiments, one or more neighboring pixels of the boundary pixel include Alternatively, in some embodiments, the pixel is a border pixel. One or more adjacent pixels of the block have not yet been coded, and the block continues It is encoded.

[0009] According to another aspect of the present application, an electronic device includes one or more processing units, a memory, and a plurality of programs stored in the memory. The programs include one or more When executed by the processing unit, the electronic device is provided with a means for decoding the video data as described above. Have them carry out the method.

[0010] According to another aspect, a non-transitory computer readable storage medium includes one or more Storing a plurality of programs for execution by an electronic device having a processing unit The program, when executed by one or more processing units, causes the electronic device to , and executes the method for decoding the video data as described above. [Brief description of the drawings]

[0011] No. 6,393,962, which is incorporated herein by reference in its entirety to provide a further understanding of the practice of the present invention. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate implementations described and, together with the description, , which are helpful in interpreting the underlying principles. Like reference numbers refer to corresponding parts.

[0012] [Figure 1] FIG. 1 is a block diagram illustrating an example video encoding and decoding system in accordance with some implementations of this disclosure. [Diagram 2] FIG. 2 is a block diagram illustrating an example video encoder according to some implementations of this disclosure. [Diagram 3] FIG. 3 is a block diagram illustrating an example video decoder according to some implementations of this disclosure. [Figure 4A] FIG. 4A is a block diagram illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes in accordance with some implementations of this disclosure. [Figure 4B] FIG. 4B is a block diagram illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes in accordance with some implementations of this disclosure. [Figure 4C] FIG. 4C is a block diagram illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes in accordance with some implementations of this disclosure. [Figure 4D] FIG. 4D is a block diagram illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes in accordance with some implementations of this disclosure. [Figure 4E] FIG. 4E is a block diagram illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes in accordance with some implementations of this disclosure. [Diagram 5] FIG. 5 illustrates a process for deriving chroma samples from luma samples of a coded block of a video frame of a bitstream in accordance with some embodiments. [Figure 6A] FIG. 6A illustrates four example coding blocks, each of which includes a number of luma samples to be converted to a number of chroma samples, in accordance with some embodiments. [Figure 6B] FIG. 6B illustrates four example coding blocks, each of which includes multiple luma samples to be converted to multiple chroma samples, in accordance with some embodiments. [Figure 6C] FIG. 6C illustrates four example coding blocks, each of which includes multiple luma samples to be converted to multiple chroma samples, in accordance with some embodiments. [Figure 6D] FIG. 6D illustrates four example coding blocks, each of which includes multiple luma samples to be converted to multiple chroma samples, in accordance with some embodiments. [Figure 7] FIG. 7 is a flow chart of a video data decoding method performed on an electronic device in accordance with some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Reference will now be made to details of exemplary embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous references are made to aid in understanding the subject matter presented herein. Non-limiting specific details are given, but those skilled in the art will appreciate that such details may be modified without departing from the scope of the claims. Various alternatives may be used without these specific details and the subject matter may be practiced without these specific details. For example, it is clear that the subject matter presented herein may be used to provide digital imaging capabilities. It will be apparent to one skilled in the art that the present invention may be implemented in many types of electronic devices equipped with the

[0014] FIG. 1 illustrates a block diagram of a video block for encoding and decoding in parallel according to some implementations of the present disclosure. FIG. 1 is a block diagram illustrating an exemplary system 10 for coding. , the system 10 generates video data to be subsequently decoded by the destination device 14. and a source device 12 that encodes the source and destination devices. The S14 is a desktop or laptop computer, tablet computer, Smartphones, set-top boxes, digital televisions, cameras, display devices , digital media players, video game consoles, video streaming devices In some implementations, the device may include any of a wide variety of electronic devices, including a source, a processor, a memory, a floppy disk ... The destination device 12 and the destination device 14 are equipped with wireless communication capabilities.

[0015] In some implementations, the destination device 14 receives the encoded data to be decoded over a link 16. Link 16 is capable of receiving encoded video data. Any type of communication medium capable of moving image data from source device 12 to destination device 14. In one example, the link 16 may include a network or device in which the source device 12 Communications that allow for the direct transmission of coded video data to a destination 14 in real time The encoded video data may be transmitted over a communication tag, such as a wireless communication protocol. The signal may be modulated according to the standard and transmitted to the destination device 14. Any wireless or radio frequency (RF) spectrum or one or more physical transmission lines The communication medium may include a local area network, a wide area network, packet-based networks such as the ISP network, or global networks such as the Internet The communication medium may be a router, switch, base station, or may help facilitate communication from the source device 12 to the destination device 14. The present invention may also include any other device that is capable of

[0016] In some other implementations, the encoded video data is output to an output interface 22. The encoded data may then be transmitted to the storage device 32. The received video data is accessed by the destination device 14 via the input interface 28. The storage device 32 may be a hard drive, a Blu-ray disc, Disks, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or Any other suitable digital storage medium for storing encoded video data. It may include any of a variety of distributed or locally accessed data storage media. In another example, storage device 32 may receive the encoded data generated by source device 12. This may correspond to a file server or another intermediate storage device that can hold the stored video data. The destination device 14 receives the content from the storage device 32 via streaming or download. The file server can access the stored video data from the The data can be stored and the encoded video data can be transmitted to a destination device 14. It may be any type of computer. Exemplary file servers include web servers. (e.g. for websites), FTP servers, network attached storage (NAS) devices The destination device 14 may include a wireless device, a local storage device, or a local disk drive. Wi-Fi connection), wired connection (e.g. DSL, cable modem, etc.), or Both of these provide access to the encoded video data stored on the file server. to encoded video data over any standard data connection, including any combination suitable for The encoded video data can be accessed from the storage device 32. may be a streaming transmission, a download transmission, or a combination of both.

[0017] As shown in FIG. 1, the source device 12 includes a video source 18, a video encoder 20, and a , and an output interface 22. The video source 18 includes a video capture device (e.g. video cameras), video archives containing previously captured footage, video a video feed interface for receiving video from a content provider; and and / or computer graphics data as source footage computer graphics system, or a combination of such sources. As an example, video source 18 may be a video camera of a security surveillance system. In some cases, source device 12 and destination device 14 may be using a camera phone or video phone. However, the implementations described in this application are generally applicable to video coding. and can be applied to wireless and / or wired applications.

[0018] Captured, pre-captured or computer-generated footage , may be encoded by video encoder 20. The encoded video data , directly to the destination device 14 via the output interface 22 of the source device 12. The encoded video data may (or alternatively) be stored in storage device 32. After being stored, it can be accessed by the destination device 14 or other devices for decoding. The output interface 22 may be used for modeling and / or playback. The device may further include a radio frequency and / or a transmitter.

[0019] The destination device 14 includes an input interface 28, a video decoder 30, and a display The input interface 28 includes a receiver and / or a modem. and can receive encoded video data via link 16. 16 or provided on a storage device 32. The video data is used by the video decoder 30 in decoding the video data. It may include various syntax elements generated by the encoder 20. The syntax elements may be transmitted over a communication medium, stored on a storage medium, or stored on a file server. The encoded video data may be included in the encoded video data stored in the memory.

[0020] In some implementations, destination device 14 may be an integrated display device and a destination device A display device configured to communicate with the display device 14. The display device 34 may display the decoded video data to a user. The display is displayed on a liquid crystal display (LCD), plasma display, or organic light-emitting diode (OLED). Any type of display, such as an LCD (Light Emitting Diode) display, or another type of display device. The display may include a variety of display devices.

[0021] The video encoder 20 and the video decoder 30 are compatible with VVC, HEVC, MPEG-4, proprietary or commercial technology, such as Port 10, Advanced Video Coding (AVC), or extensions of such standards. This application is not limited to any particular video encoding / decoding standard. It is understood that this specification is not limited to the above and may be applicable to other video encoding / decoding standards. Video encoder 20 of device 12 may be configured to record video in accordance with any of these current or future standards. It is generally envisioned that the image data may be encoded by the addressee. The video decoder 30 of the destination device 14 may receive video signals in accordance with any of these present or future standards. It is also generally envisioned that the image sensor may be configured to decode image data.

[0022] The video encoder 20 and the video decoder 30 each include one or more microprocessors. processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), Field Programmable Gate Arrays (FPGAs), discrete logic, and software , hardware, firmware, or any combination thereof. It may be implemented as a suitable encoder circuit, or may be implemented partially in software. The electronic device stores instructions for the software on a suitable non-transitory computer-readable medium. and executing the instructions in hardware using one or more processors to implement the present disclosure. The disclosed video encoding / decoding operations can be performed. Each of the video decoder 30 and the video decoder 30 includes one or more encoders or decoders. The encoder or decoder is an encoder / decoder combined with a corresponding device. It may be integrated as part of a coder (CODEC).

[0023] FIG. 2 illustrates an example video encoder 20 according to some implementations described in this application. 1 is a block diagram showing a video encoder 20 for encoding an image of a video block in a video frame. Intra-prediction coding can be performed using spatial prediction. Relying on the measurement, the spatial redundancy of the video data within a given video frame or picture is reduced or eliminated. Inter-prediction coding relies on temporal prediction to predict or remove adjacent pictures in a video sequence. Reduce or remove temporal redundancy in video data within a frame or picture.

[0024] As shown in FIG. 2, the video encoder 20 includes a video data memory 40, a prediction processing unit, and 41, decoded picture buffer (DPB) 54, summer 50, transform processing unit 5 2, a quantization unit 54, and an entropy encoding unit 56. The motion estimation unit 41 includes a motion estimation unit 42, a motion compensation unit 44, and a division unit 45. 5, intra prediction processing unit 46, and intra block copy (BC) unit 4 In some implementations, video encoder 20 may further include a filter for video block reconstruction. For this purpose, the inverse quantization unit 58, the inverse transform processing unit 50, and the sub-quantization unit 52 are also included. A locking filter (not shown) is placed between the summer 52 and the DPB 54, and the block boundary can be filtered to remove block artifacts from the reconstructed image. In addition to the deblocking filter, an in-loop filter (not shown) is used, The output of summer 52 may also be filtered. The may take the form of a programmable hardware unit or may be implemented as one or more The number of illustrated fixed or programmable hardware units is divided into This can be done.

[0025] A video data memory 40 stores video data to be encoded by components of the video encoder 20. The video data in the video data memory 40 can be stored in the following format: , may be obtained from the video source 18. The DPB 54 is By encoding video data (e.g., in intra or inter predictive coding modes) The video data memory 40 and the DP B54 may be formed by any of a variety of memory devices. The video data memory 40 is on-chip with the other components of the video encoder 20, and may be off-chip relative to those components.

[0026] As shown in FIG. 2, after receiving the video data, the division unit 41 The block 45 divides the video data into video blocks. The video frame is sliced ​​and tagged according to a predefined partitioning structure, such as a quadtree structure. This may also include splitting the image into multiple image files or other larger coding units (CUs). An image frame is divided into multiple video blocks (or sets of video blocks called tiles). The prediction processing unit 41 may divide the error results (e.g., the coding rate and distortion select one of a plurality of possible predictive coding modes for the current video block based on the level of the The possible predictive coding modes that can be selected are, for example, one of several intra-predictive coding modes. or one of a number of inter-prediction coding modes. The resulting intra- or inter-predictive coded block is then fed to Summer50. This is then fed to Summer52 to later use it as part of the reference frame. The prediction unit can reconstruct the block to be encoded for use as a Unit 41 stores motion vectors, intra mode indicators, segmentation information, and other Syntax elements such as syntax information are encoded in entropy encoding units. It will also be provided to Net56.

[0027] In order to select an appropriate intra-prediction coding mode for the current video block, a prediction processing unit The intra prediction processing unit 46 in the unit 41 performs intra prediction on the same block as the current block to be coded. Intra-blocking of a current video block to one or more adjacent blocks in the same frame Predictive coding can be performed to provide spatial prediction. The estimation unit 42 and the motion compensation unit 44 estimate one or more of the reference frames. or performing inter-prediction coding of a current image block for a plurality of predicted blocks. Video encoder 20 generates multiple codes for each block of video data. An encoding pass can be performed to, for example, select an appropriate encoding mode.

[0028] In some implementations, motion estimation unit 42 may be configured to estimate a given motion in a sequence of video frames. By generating a motion vector according to the pattern, the image for the current video frame is a motion vector for a prediction block in a reference video frame; The displacement of a prediction unit (PU) of a video block in a current video frame is indicated. The motion estimation performed by motion estimation unit 42 is a motion vector that estimates the motion of video blocks. A motion vector is generated based on, for example, the current frame (or other coding The reference frame (or units) for the current block being coded is the current video frame or pixel for a prediction block in another coded unit The predetermined pattern can indicate the displacement of the PU of the image block in the Video frames can be designated as P-frames or B-frames. Intra BC A unit 48 determines a motion vector by the motion estimation unit 42 for inter prediction. For intra BC coding, for example, the vector of the block vector is determined in a similar manner to that of the vector of the block vector. or the motion estimation unit 42 can be used to determine the block vectors It is possible.

[0029] The predicted block is calculated by sum of absolute differences (SAD) and sum of squared differences (SSD) of the reference frame. or other difference metric. A block that is determined to match the PU exactly. In some implementations, The encoder 20 calculates values ​​of sub-integer pixel positions of the reference frame stored in the DPB 54. For example, video encoder 20 may calculate a quarter pixel You can interpolate values ​​for positions, eighth-pixel positions, or other fractional pixel positions. Thus, motion estimation unit 42 performs the following calculations for whole and fractional pixel locations: It is possible to perform a motion search using the motion vectors and output motion vectors with sub-pixel accuracy.

[0030] The motion estimation unit 42 estimates the location of the PU of the video block in the inter-predictive coded frame. to the first reference frame list (list 0) or the second reference frame list (list 1) The motion vector of the PU is calculated by comparing the position of the predicted block of the selected reference frame with the position of the predicted block of the selected reference frame. The vector is calculated by the first reference frame list or the second reference frame list. , identifying one or more reference frames stored in the DPB 54. 42 sends the calculated motion vectors to the motion compensation unit 44, which then and sends it to the coding unit 56.

[0031] The motion compensation performed by the motion compensation unit 44 is and fetching or generating a prediction block based on the determined motion vector. Upon receiving the motion vector of the PU of the current image block, the motion compensation unit 44 locates the prediction block to which the motion vector points in one of the reference frame lists. , we can retrieve the predicted block from DPB54 and transfer the predicted block to Summer50. Then, the summer 50 performs motion compensation from the pixel values ​​of the current video block to be coded. The pixel values ​​of the predicted block provided by unit 44 are subtracted to obtain the pixel forming a residual video block of pixel difference values. The pixel difference values ​​forming the residual video block are The motion compensation unit 44 may include a luma or chroma difference component, or both. for use by the video decoder 30 in decoding a video block of an image frame. Additionally, it can generate syntax elements associated with video blocks of a video frame. The syntax elements can, for example, specify the motion vectors used to identify the prediction blocks. A syntax element that defines a prediction mode, an optional flag that indicates a prediction mode, or The motion estimation unit 42 and the motion estimation unit 43 may include any other syntax information described in . The compensation unit 44 may be highly integrated, but is shown separately for conceptual purposes. Please note.

[0032] In some implementations, the intra BC unit 48 is connected to the motion estimation unit 42 and the motion A vector is generated in a similar manner to that described above in relation to the compensation unit 44, and the prediction block is fetched. However, the predicted block must be in the same frame as the actual block to be coded. The vectors are called block vectors as opposed to motion vectors. The intra BC unit 48 determines the intra prediction mode to encode the current block. In some examples, the intra BC unit 48 can be used to For example, a given block can be encoded using different intra prediction modes during separate encoding passes. It is possible to encode multiple blocks and test their performance through rate-distortion analysis. The intra BC unit 48 then selects from among the various tested intra prediction modes: Select the appropriate intra prediction mode to use and display the intra mode indicator accordingly. For example, the intra BC unit 48 can generate various tested intra Rate-distortion analysis of intra-prediction modes is used to calculate the rate-distortion values ​​for the tested modes. The intra prediction mode that has the best rate-distortion performance among the three modes is used. A rate distortion analysis is generally performed on the encoded The block and the original encoding that was encoded to generate the block being encoded. The amount of distortion (or error) between the encoded block and the unencoded block. Determines the bit rate (i.e. number of bits) used to generate the clock. The BC unit 48 performs distortion and rate optimization for various encoded blocks. A ratio is calculated to indicate which intra prediction mode gives the best rate-distortion value for the block. It can be determined whether:

[0033] In another example, the intra BC unit 48 may include the motion estimation unit 42 and the motion compensation unit 43. Set 44 may be used in whole or in part to perform intra-BC prediction according to the implementations described herein. In either case, the intra-blot model can be used to measure the For block copy, the predicted block is computed by sum of absolute differences (SAD), sum of squared differences (SSD) or other difference metric. The identification of the predicted block is based on the sub-integer pixel This may include calculating the value of the rule position.

[0034] The predicted block is either from the same frame by intra prediction or from inter Whether the encoded data is from a different frame due to prediction, video encoder 20 Subtract pixel values ​​of the prediction block from pixel values ​​of the current image block to be encoded; A residual image block may be formed by forming pixel value differences. The pixel difference values ​​forming a block may include both luminance and chrominance component differences. stomach.

[0035] The intra-prediction processing unit 46 includes a motion estimation unit 42 and a motion compensation unit 44. or the intra prediction performed by the intra BC unit 48. As an alternative to intra block copy prediction, the current video block can be intra In particular, intra-prediction processing unit 46 can predict the current block by encoding the In order to do so, the intra prediction mode to be used for coding can be determined. , intra-prediction processing unit 46 may, for example, perform various The current block can be encoded using intra prediction mode. The measurement processing unit 46 (or, in some examples, the mode selection unit) is From the intra prediction modes, an appropriate intra prediction mode to be used can be selected. The intra-prediction processing unit 46 determines the selected intra-prediction mode for the block by The indicative information may be provided to entropy encoding unit 56. The entropy encoding unit 56 encodes the selected interleave in the bitstream. Information indicating the trajectory prediction mode may be encoded.

[0036] A prediction processing unit 41 performs a current prediction process via either inter-prediction or intra-prediction. After determining the prediction block of the video block, Summer 50 is predicted from the current video block. A residual image block is formed by subtracting the residual image block from the residual image block. The image data can be contained in one or more transformation units (TUs), The transform processing unit 52 is provided with a discrete cosine transform (DCT) or The residual image data is transformed into residual transform coefficients using a transform, such as a substantially similar transform.

[0037] The transform processing unit 52 sends the resulting transform coefficients to a quantization unit 54. Quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process reduces the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting the quantization parameter. In some examples, quantization unit 54 then generates a matrix including the quantized transform coefficients. Alternatively, you can use the entropy encoding unit to perform a scan of the The unit 56 is capable of performing the scan.

[0038] Following quantization, the entropy encoding unit 56 performs entropy encoding on the context Context-adaptive variable length coding (CAVLC), Context-adaptive binary arithmetic coding (CABAC), Syntax-based Context-Adaptive Binary Arithmetic Coding (SBAC), Probability Interval Segmentation Encoding PIPE coding, or another entropy encoding method or technique The quantized transform coefficients are entropy encoded into the video bitstream using a The encoded bitstream is then sent to a video decoder 30, The images are archived in storage device 32 for later transmission or retrieval to video decoder 30. The entropy encoding unit 56 can generate the image data to be encoded. Entropy encoding the motion vectors and other syntax elements of the video frames It is also possible.

[0039] The inverse quantization unit 58 and the inverse transform processing unit 50 perform inverse quantization and inverse transform, respectively. A pixel transformation is applied to generate a reference block for prediction of other video blocks. As described above, the motion compensation unit 4 reconstructs the residual video block in the motion domain. 4, a motion compensated prediction is performed from one or more reference blocks of the frame stored in the DPB 54. The motion compensation unit 44 may generate one or more interpolated frames. A filter is applied to the prediction block to compute sub-integer pixel values ​​for use in motion estimation. This can be done.

[0040] The summation 52 is generated by the motion compensation unit 44 using the reconstructed residual block. Add to the motion compensated prediction block to generate a reference block for storage in the DPB 54 The reference block is then fed to the intra BC unit 48 as a predicted block, the motion estimation unit 49 as a prediction block, and the 42, and motion compensation unit 44 to estimate the difference between the The video blocks can be inter predicted.

[0041] FIG. 3 is a block diagram illustrating an example video decoder 30 according to some implementations of the present application. The video decoder 30 includes a video data memory 69, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, The prediction processing unit 81 includes a motion compensation unit 82, an image processor 90, and a DPB 92. The video decoder 30 further includes an intra prediction processing unit 84 and an intra BC unit 85. The encoding process is similar to that described above for video encoder 20 in conjunction with FIG. A roughly reverse decoding process can be performed. For example, the motion compensation unit 8 2 predicts the motion vector based on the motion vector received from the entropy decoding unit 80. The intra prediction unit 84 can generate the measured data while the intra prediction unit 84 performs entropy decoding. generating prediction data based on the intra-prediction mode indicator received from unit 80; can.

[0042] In some examples, the units of the video decoder 30 may be configured to perform implementations of the present application. Also, in some examples, implementations of the present disclosure may involve one or more of the following: For example, the intra BC unit 85 may be divided into one or more units. The implementation of the present invention may be implemented by a motion compensation unit 82, an intra-prediction processing unit 84, or the like. and other units of video decoder 30, such as entropy decoding unit 80. In some examples, video decoder 30 may perform the intra BC unit The function of the intra BC unit 85 may be the same as that of the motion compensation unit 80. 2 may be performed by other components of prediction processing unit 81.

[0043] The video data memory 69 is decoded by the other components of the video decoder 30. It can store video data, such as encoded video bitstreams. The video data stored in the data memory 69 is obtained from the storage device 32, for example. It is also possible to transmit video data to a local camera or other device via wired or wireless network communication. The image may be acquired from a source or may be stored on a physical data storage medium (e.g., a flash drive). The image data may be acquired by accessing the image data memory 6. 9 stores encoded video data from the encoded video bitstream. The video decoder 30 may include a coded picture buffer (CPB) for storing the decoded picture. The data buffer (DPB) 92 is used for decoding the video data by the video decoder 30. A reference for use in coding (e.g., in intra- or inter-prediction coding modes) The image data memory 69 and the DPB 92 store image data. Dynamic Random Access Memory (DRAM) (including Synchronous DRAM (SDRAM)), Magnetoresistive Random Access Memory (MRRAM) MRAM, resistive RAM (RRAM), or other types of memory devices It may be formed by any of a variety of memory devices. For the purposes of illustration, The buffer 69 and the DPB 92 are shown in FIG. 3 as two separate components of the video decoder 30. However, those skilled in the art will recognize that the video data memory 69 and the DPB 92 are the same. It will be appreciated that this may be provided by the memory device or a separate memory device. In some examples, video data memory 69 may be integrated into other components of video decoder 30. The integrated circuitry may be on-chip or off-chip relative to those components.

[0044] During the decoding process, video decoder 30 decodes the video of the encoded video frames. The encoded video bits representing the block and associated syntax elements The video decoder 30 receives the video frame level and / or video block level The syntax elements can be received at the block level. A p-decoding unit 80 entropy decodes the bitstream to obtain The quantization coefficients, motion vectors or intra prediction mode indicators, and other syntax Entropy decoding unit 80 then generates motion vectors and and other syntax elements to prediction processing unit 81.

[0045] If a video frame is coded as an intra-prediction (I) frame or is coded using other types of When the prediction process is performed for an intra-coded predicted block in a frame of the same group, The intra prediction processing unit 84 of the unit 81 processes the signaled intra prediction mode. Based on the current frame and reference data from previously decoded blocks of the current frame, Predictions can be generated for video blocks of a video frame.

[0046] The video frame is coded as an inter-predictive (i.e., B or P) frame. In this case, the motion compensation unit 82 of the prediction processing unit 81 performs entropy decoding. Based on the motion vectors and other syntax elements received from the motion vector processing unit 80, 2. Generating one or more prediction blocks for a video block of a video frame of the image. The block may be generated from a reference frame in one of the reference frame lists. The coder 30 uses a default construction technique based on the reference frames stored in the DPB 92. The reference frame lists, List0 and List1, can be constructed using

[0047] In some examples, the video block is coded according to the intra BC mode described herein. In this case, the intra BC unit 85 of the prediction processing unit 81 performs entropy decoding. Based on the block vectors and other syntax elements received from the reading unit 80, The video encoder generates a prediction block of the current video block based on the prediction block. 20 within the same reconstructed area of ​​the picture as the current video block defined by This is also fine.

[0048] The motion compensation unit 82 and / or the intra BC unit 85 calculates the motion vectors and and other syntax elements to predict the video block of the current video frame. The prediction information is then used to determine a prediction for the current video block being decoded. For example, motion compensation unit 82 may generate a motion compensation block based on the received syntax element The prediction mode used to encode a video block of a video frame is determined using some of the frame type (e.g., intra or inter prediction), inter prediction frame type (e.g., B or P), construction information for one or more reference frame lists for a frame, Motion vectors of inter-prediction encoded video blocks, and inter-prediction for each frame Determine the inter prediction status of the coded video block, and other information, The video block in the video frame is decoded.

[0049] Similarly, the intra BC unit 85 may also interpret some of the received syntax elements, e.g. The flag indicates that the current video block was predicted using intra BC mode. , which image blocks of a frame are in the reconstruction domain and are stored in DPB 92. The block vector of each intra BC prediction video block of the frame, Determine the intra BC prediction status of the intra BC predicted video block, and other information. , to decode the video block in the current video frame.

[0050] Motion compensation unit 82 provides motion compensation to video encoder 20 during the encoding of a video block. The interpolation is performed using the interpolation filter used by It is also possible for motion compensation unit 82 to calculate an interpolated value for a cell. and determining the interpolation filters to be used by the video encoder 20 from the selected syntax elements. , an interpolation filter can be used to generate the prediction block.

[0051] Inverse quantization unit 86 inversely quantizes the image data captured by video encoder 20 to determine the degree of quantization. using the same quantization parameters calculated for each video block in the image frame; The bitstream is provided and encoded by an entropy decoding unit 80. The inverse transform processing unit 88 inverse quantizes the entropy decoded quantized transform coefficients. To reconstruct the residual block in the pixel domain, an inverse transform (e.g., an inverse DCT , an inverse integer transform, or a conceptually similar inverse transform process) is applied to the transform coefficients.

[0052] The motion compensation unit 82 or the intra BC unit 85 processes the vectors and other syntax After generating a prediction block of the current image block based on the pixel elements, Summer 90 Residual blocks from the inverse transform processing unit 88 and the motion compensation unit 82 and the intra BC and the corresponding prediction block generated by unit 85. Reconstruct the decoded video block of the video block. ) is located between Summer 90 and DPB 92 and further decodes the decoded video block. The decoded video blocks in a given frame are then processed by the DP The reference block B92 is stored in the DPB92, and the DPB92 is used for subsequent motion compensation of the next video block. The DPB92 or a memory device separate from the DPB92 is used to store frames. 1 for presentation on a display device, such as display device 34. It is also possible to store coded images.

[0053] In a typical video coding process, a video sequence is typically divided into ordered frames. A frame contains a set of frames or pictures. Each frame is denoted by SL, SCb, and SCr. SL is a two-dimensional array of luminance samples. SCb is a two-dimensional array of Cb chroma samples. SCr is a two-dimensional array of Cr chroma samples. In another example, the frame can be monochrome. It is small and therefore contains only one two-dimensional array of luma samples.

[0054] As shown in FIG. 4A, a video encoder 20 (or more specifically, a splitting unit 4 5) first divides a frame into a set of coding tree units (CTUs). A video frame is encoded from left to right and top to bottom. Each may contain an integer number of CTUs arranged consecutively in raster scan order. A CTU is the largest logical coding unit, and its width and height are the length of a video sequence. All CTUs in the 128x128, 54x64, 32x32, and 16x16 The sequence parameter set is then set to the same size as the video encoder 20. However, this application is not necessarily limited to a particular size. As shown in FIG. 4B, each CTU is a single luminance sample. Coding tree block (CTB), the coding tree block of the corresponding two chroma samples, and the code It may contain syntax elements used to encode samples of the coding tree block. The syntax elements specify the properties of different types of units of a coding block of pixels. describes the video sequence and how the video decoder 30 can reconstruct the video sequence; Inter or intra prediction, intra prediction mode, motion vectors, and other parameters For a monochromatic picture or a picture with three separate color planes, the CTU includes A single coding tree block and the coding tree block samples are coded using A coding tree block is an N×N block of samples. Good too.

[0055] To achieve better performance, the video encoder 20 may add a For example, a binary tree, a ternary tree, a quad tree, or any combination of the above. By recursively dividing the CTU into smaller coding units (CUs), As shown in FIG. 4C, the 54×64 CTU 400 is first divided into four smaller CUs. Each smaller CU has a block size of 32x32. Among the CUs, CU410 and CU420 each have four 16x16 block size C The two 16x16 CUs 430 and 440 are each divided into four 8x8 blocks. FIG. 4D illustrates a CTU 40 as shown in FIG. 4C. The final result of the partitioning process of 0 is shown in the quadtree data structure. Each leaf node of the quadtree is 32× It corresponds to one CU of a suitable size ranging from 32 to 8×8. Similar to the CU, each CU consists of a coded block (CB) of luma samples and two equally sized frames. The corresponding coding block of the chroma samples of the frame and the samples of the coding block are coded. and syntax elements used to represent monochrome pictures or three distinct In a picture with color planes, a CU consists of a single coding block and and a syntax structure used to encode the sample. The quadtree partitioning shown in Figure 4D is for illustration purposes only and does not represent a CTU divided into multiple CUs. Partitioning to adapt to different local characteristics based on quadtree / ternarytree / binarytree partitioning Note that in a multi-type tree structure, one CTU can be Each quadtree leaf CU is then further divided by a binary tree and a ternary tree structure. As shown in FIG. 4E, four-way split, two-way split, two-way split, three-way split, There are five vertically divided types.

[0056] In some implementations, video encoder 20 may divide the coding blocks of a CU into one or more The prediction block can be further divided into M × N prediction blocks (PBs) of the same prediction Rectangular (square or non-square) samples to which measurements (inter or intra) are applied The prediction unit (PU) of the CU predicts a prediction block of luma samples, two The corresponding prediction block of the chroma samples of, and the prediction block used to predict Syntax elements may be included for monochrome pictures or pictures with three distinct color planes. In this model, a PU is used to predict a single prediction block and The video encoder 20 may include a syntax structure for determining the luminance, Cb and generating predictive luminance, Cb, and Cr blocks for the Cr predictive block. can be done.

[0057] The video encoder 20 uses intra prediction or inter prediction to generate a prediction block of the PU. The video encoder 20 can generate a prediction frame for the PU using intra prediction. To generate the block, video encoder 20 may decode the frame associated with the PU. Based on the downloaded samples, a prediction block of the PU can be generated. If the coder 20 uses inter prediction to generate a prediction block for a PU, The DA 20 is configured to receive the decoded frames of one or more frames other than the frames associated with the PU. Based on the sample thus obtained, a prediction block for the PU can be generated.

[0058] The video encoder 20 calculates predicted luma, Cb, and y for one or more PUs of a CU. After generating the luminance and Cr blocks, video encoder 20 may then subtract the luminance and Cr blocks from the original luminance coding block of the CU. The luminance residual block of the CU is generated by subtracting the luminance prediction block from the CU. Thus, each sample of the luma residual block of the CU can be calculated by the predicted luma block of the CU. The luma sample in one of the blocks and the corresponding sample in the original luma coding block of the CU are Similarly, video encoder 20 may indicate the difference between the Cb residual block and the Cb residual block of the CU. The Cb residual block of the CU can be generated by using the Cr residual block. Each sample in the block is the sum of the Cb samples in one of the predicted Cb blocks of the CU and the original Cb sample of the CU. b indicates the difference between the corresponding samples in the coding block, and Cr the residual block of CU Each sample in is the sum of the Cr sample in one of the predicted Cr blocks of the CU and the original Cr It may indicate the difference between corresponding samples in the coding block.

[0059] Furthermore, as shown in FIG. 4C, video encoder 20 may use quadtree partitioning to Subtract the luma, Cb, and Cr residual blocks of U from one or more luma, Cb, and Cr A transformation block is a rectangular block to which the same transformation is applied. A CU is a block of samples (square or non-square). The transform unit (TU) of a CU is , a transform block for luma samples, two corresponding transform blocks for chroma samples, and a transform It may also contain syntax elements used to transform block samples. Therefore, each TU of a CU is divided into a luminance transformation block, a Cb transformation block, and a Cr transformation block. In some examples, the luma transform block associated with the TU may be The Cb transform block may be a sub-block of the luminance residual block of U. The Cr transform block may be a sub-block of the Cr residual block of the CU. In a monochromatic picture or a picture with three separate color planes, T U is a single transform block and the signal used to transform the samples of the transform block. and a tax structure.

[0060] The video encoder 20 applies one or more transforms to the luma transform block of the TU to obtain A block of luminance coefficients of the TU can be generated. The block of coefficients is a two-dimensional array of transform coefficients. The transform coefficients may be rays. The transform coefficients may be scalar quantities. The transform can be applied to the Cb transform block of a TU to generate the Cb coefficient block of the TU. Video encoder 20 may apply one or more transforms to the Cr transform blocks of the TU. The Cr coefficient block of the TU can be generated by the above.

[0061] A coefficient block (e.g., a luma coefficient block, a Cb coefficient block, or a Cr coefficient block) After generating the coefficient block, video encoder 20 may quantize the coefficient block. Quantization generally refers to the process of quantizing a transform coefficient to reduce the amount of data used to represent the transform coefficient. This refers to the process of potentially reducing the coefficients to provide additional compression. After quantizing a block, video encoder 20 may then generate a signal indicating the quantized transform coefficients. The tax elements can be entropy encoded. is a context-adaptive binary quantization for syntax elements that indicate quantized transform coefficients. Finally, video encoder 20 may perform a coding algorithm such as CABAC. The sequence of bits that form a representation of the encoded frame and associated data is The bitstream can be output to a storage device 32. or transmitted to the destination device 14.

[0062] After receiving the bitstream generated by the video encoder 20, the video decoder 30 parses the bitstream and obtains syntax elements from the bitstream. The video decoder 30 can extract syntax elements from the bitstream. Based at least in part on the image data, a frame of video data can be reconstructed. The process of reconstructing the image data is generally performed by the encoding process performed by video encoder 20. For example, video decoder 30 may 2. Perform an inverse transform on the associated coefficient block to generate the TU associated with the current CU. The video decoder 30 can then reconstruct the residual block of the current CU. Add the samples of the predicted block of the PU to the corresponding samples of the transform block of the TU of the current CU. The coding block of the current CU is reconstructed by calculating the coding block of each CU of the frame. After reconstructing the encoded blocks, video decoder 30 can reconstruct the frame. .

[0063] As mentioned above, video coding is mainly divided into two modes: intra-frame prediction (or Video compression is achieved using intra-frame prediction (or inter-prediction) and inter-frame prediction (or inter-prediction) Palette-based coding is another coding style that is used in many video coding standards. This scheme is a palette-based coding scheme that is particularly suited for screen-generated content coding. , a video encoder (e.g., video encoder 20 or video decoder 30) may The palette table is a table of colors that represent the image data of a given block. contains the most dominant (e.g., frequently used) pixel values ​​in a block of Pixel values ​​that are not frequently represented in the image data of a block are included in the palette table. or included in the palette table as an escape color.

[0064] Each entry in the palette table is an index of the corresponding pixel value in the palette table. The palette index for the samples in the block is stored in the palette table. Indicates which entries from the pool are used to predict or reconstruct which samples. This palette mode can be coded to match a picture, slice, tile, or the palette predictor of the first block of any other such grouping of video blocks The process begins with generating a palette predictor for the subsequent video block, as described below. is usually generated by updating a previously used palette predictor. For this purpose, the palette predictor is assumed to be defined at the picture level. A palette may contain multiple coding blocks, with each coding block having its own palette. In this case, there is one palette predictor for the entire picture, but one for the entire picture.

[0065] The number of bits required to signal a palette entry in a video bitstream To reduce the overhead, the video decoder reconstructs video blocks using a palette predictor. It is possible to determine a new palette entry in the palette table that will be used to For example, the palette predictor can use the palette entries from a previously used palette table. may contain all entries in the palette table that have been recently used, or It may be initialized with the most recently used palette table by The palette predictor uses the most recently used palette table over all entries from the palette table. Contains few entries, then some entries from other previously used palette tables. A palette predictor can be used to code the different blocks. It may have the same size as the palette table used, or it may encode different blocks. The pallet table may be larger or smaller than the pallet table used to The palette predictor is a first-in-first-out (FIFO) table that contains 54 palette entries. It is executed as a bull.

[0066] to generate a palette table for a block of image data from the palette predictor. The video decoder extracts each element of the palette predictor from the encoded video bitstream. It can receive a 1-bit flag for the palette reservation entry. The first value ( For example, a binary 1), or the associated entry of the palette predictor is a palette tile. It may have a second value (e.g., a binary 0) that indicates that the The size of the palette predictor is smaller than the palette table used for a block of video data. If the maximum size of the palette table is reached, the video decoder will not Receiving the flag may be stopped.

[0067] In some runs, some entries in the palette table may not represent the palette predictor. is determined using the For such an entry, the video decoder may From the image bitstream, the picture of the luma and two chroma components associated with the entry Three separate m-bit values ​​may be received to indicate the cell value, where m is the image data. Represents the bit depth of the data. Multiple directly signaled palette entries are required. Compared to m-bit values, these palette entries obtained from the palette predictor are 1-bit Therefore, you can use the palette predictor to select some or all of the Signaling a palette entry in the current palette table signals a new palette table entry. This significantly reduces the number of bits required for signaling, making palette mode encoding possible. Improves overall coding efficiency.

[0068] In many cases, the palette predictor for a block is a pre-signed The color is determined based on the palette table used to encode the colored block. However, when encoding the first coding tree unit of a picture, slice, or tile, In this case, the palette table of the previously coded block is not available. A new predictor cannot be generated using a previously used palette table entry. In this case, the sequence of the palette predictor initializer is determined by the sequence parameter set ( It may be signaled in the Picture Parameter Set (PPS) and / or the Picture Parameter Set (PPS). Sequence Parameter Set (SPS) and / or Picture Parameter Set ( PPS) generates a palette predictor when a previously used palette table is not available. The SPS is typically found in each slice segment header. The syntax elements found in the PPS to which the syntax element is pointed are A series of consecutive coded images, called a Coded Video Sequence (CVS), as determined by the PPS refers to the syntax structure of syntax elements that apply to a video picture. As determined by the syntax elements found in each slice segment header: Syntax for syntax elements that apply to one or more single pictures in a CVS Therefore, an SPS generally has a higher level of syntax structure than a PPS. It has been determined that the syntax elements in the SPS are Consent elements generally change infrequently, meaning they apply to the majority of video data. be.

[0069] FIG. 5 illustrates a coding block of a video frame of a bitstream according to some embodiments. The process 500 of obtaining chroma samples 502 from luma samples 504 of 506 is shown. The coding block 506 of the frame includes a number of pixels, each of which is made up of a number of color components ( For example, blue, green, and red. In this process, the brightness and color information of multiple pixels is represented by multiple luminance samples. Each pixel of the plurality of pixels is represented by a plurality of chroma samples 502 and a plurality of saturation samples 504. Each chroma sample 502 corresponds uniquely to a single corresponding luma sample 504. Each luminance sample corresponds to a corresponding set of luminance samples 504 according to a subsampling scheme. Each luminance sample has a luma component Y′, and each chroma sample 504 has a blue-difference chroma component Cb and It has a red difference chroma component Cr. The subsampling scheme for the luma and chroma components (Y':C b:Cr) are, for example, 4:1:1, 4:2:0, 4:2:2, 4:4:4 and 4:4: 5, the luminance samples 504 and 506 of the video frame have a ratio of three parts: The chroma and saturation samples 502 are subsampled with a three part ratio equal to 4:1:1. It complies with the GIS Scheme.

[0070] In some embodiments, the coding blocks 506 of a video frame may span 2M pixels in width. 2M x 2N pixels corresponding to 2N luminance samples across the luminance samples and height. M and N are optionally equal to or different from each other. An exemplary subsampling scheme for the chroma components (Y':Cb:Cr=4:1:1) For example, the luminance samples 504 of a video frame are encoded with a resolution of 2M×2N, while The chroma samples 502 are encoded at a lower resolution of M×N. The pull 502 is 2M×2N (for example, 4:4:4 full sampling), 2M×N (for example, 4 :4:0 subsampling), M×2N (e.g. 4:2:2 subsampling), 1 / 2 M×2N (e.g. 4:1:1 subsampling) This may also be the case.

[0071] TIFF2025060974000002.tif97160

[0072] TIFF2025060974000003.tif79160

[0073] In some embodiments, luma samples 504 and chroma samples 502 of a video frame may be On average, four luminance samples 504 are divided into blue-difference chroma components Cb and red-difference chroma components Cr. The subsampling scheme corresponds to one chroma sample 504 having In other words, the first luma resolution of the luma samples 504 is the second chroma resolution of the chroma samples 502. On average, every four luma samples 504 are scaled to the second chroma resolution. One downsample in the set of luma samples 508 having an equal third luma resolution In some embodiments, each luma sample is combined with the sampled luma sample 508. For the pixel 508, there are more adjacent luminance samples 504 than there are adjacent luminance samples 506. 4 is applied to obtain the corresponding chroma samples 502. For each sample, each luma sample 504 is applied multiple times to produce one or more downsampled An integrated luma sample 508 and / or one or more chroma samples 502 are generated. A plurality of luminance samples 504 are interpolated based on a predefined interpolation scheme 510. For more information on downsampling to 508 and / or chroma samples 502, see , as described below with reference to Figures 6A-6D.

[0074] In some embodiments, a method is provided for converting luma samples 508 to chroma samples 502. The linear mapping model 512 used is obtained using the Max-Min method. In general, the maximum downsampled luma sample is the The smallest downsampled luma sample is identified from the set of luma samples. The maximum chroma sample is identified from the set of unsampled luma samples. The largest downsampled luma sample corresponds to the largest downsampled chroma sample, and the smallest downsampled chroma sample corresponds to the smallest downsampled chroma sample. The maximum and minimum chroma samples correspond to the maximum and the minimum downsampled luminance samples are applied to produce a linear mapping. The linear model 512 is determined, i.e., the two coefficients α and β in equation (1) are determined. After the mapping model 512 is obtained, the video codec performs a linear mapping model 512 to the downsampled luma samples 508 from the luma samples 504, It generates corresponding chroma samples 502 in a coding block 506 .

[0075] The coding blocks of a video frame are reconstructed according to an ordered sequence. In some circumstances, the encoding block 506 may encode the first row or the first column of a video frame. When the coding block 506 is being processed, the group of selected pixels For example, the selected group of pixels is reconstructed in the coding block 506. Includes pixels located directly above and to the left of the pixel. 518, and a selected group of downsampled luminance samples 514. The loop is reconstructed for a group of selected pixels, e.g. using the Max-Min method. can be used to obtain a linear mapping model 512 based on the method. , luma samples 518, and downsampled luma samples 514 are encoded as It is outside the boundary of block 506 and is directly adjacent to that boundary. In an embodiment, the video codec performs a gradient of the encoded downsampled luminance samples. loop (e.g., the downlink corresponding to the luminance sample of the upper neighbor and the luminance sample of the left neighbor) Searches for the maximum downsampling downsampled luminance sample 514A-1 and the minimum downsampled luminance The video codec then identifies the maximum and minimum downsampling luminance samples 514A-1 and 514B-1 (e.g., A selection of chroma sample 516, including the adjacent chroma sample above and the adjacent chroma sample to the left Previously encoded chroma samples 516A-1 and 516B (in a group -1. In this way, the linear mapping model 512 identifies these associated Downsampled luma samples (514A-1 and 514B-1) and chroma can be obtained based on the samples (516A-1 and 516B-1).

[0076] Alternatively, in some embodiments, the video codec may select a group of luminance samples. Downsampling is performed on the group to obtain the maximum and minimum luminance samples 518A and 518B, a group of luminance samples (e.g., the top and left neighbors) can be identified without identifying 518B. Search for the selected group of 518 luminance samples (including the luminance sample) and Within the group of samples, (i) the luminance sample 518A having the largest luminance value and (ii) the luminance sample 518B having the smallest luminance value. The video codec then identifies the luma sample 518B having the smallest luma value. and downsampling in the area associated with the smallest luminance samples 518A and 518B. Perform downsampling (e.g., in a region with 6 samples, perform 6 tap downsampling) Any weighted averaging scheme known in the art, including weighted averaging schemes such as Downsampling, can be used. Generate the pulled luminance sample 514A-2 as the maximum luminance sample ( Sample 514A-2 is indeed the largest downsampled luminance sample 514A -1, or not), downsampled luma samples 51 4B-2 as the smallest luminance sample (the luminance sample 514B-2 is certainly The minimum downsampled luma sample may be 514B-1, or it may not be Then the video codec converts the downsampled luma samples 514A -2 and 514B-2 (e.g., the chroma samples including the upper and left adjacent chroma samples) Chroma samples 516A-2 and 516B-2 (in the group of chroma samples 516) In this way, the linear mapping model 512 identifies the downsampled luminance Samples (514A-2 and 514B-2) and Chroma Samples (516A-2 and 516B-2).

[0077] Alternatively, in some embodiments, the video codec may select a group of chroma samples (e.g. For example, search for a group of 516 saturation samples that contains the neighboring saturation samples above and to the left. , maximum and minimum saturation samples 516A-3 and 516B-3 (e.g., maximum and minimum saturation samples 516A-3 and 516B-3, respectively) The video codec then identifies the chroma samples with the highest and lowest chroma values. 514A-3 and 514B-3, which correspond to the maximum and minimum saturation samples (e.g., A group of 514 downsampled luma samples, including the left adjacent luma sample, Downsampled luminance samples 514A-3 and 514B- 3. In this way, the linear mapping model 512 identifies the downsampled Luma samples (514A-3 and 514B-3) and chroma samples (516A-3 and and 516B-3).

[0078] Alternatively, in some embodiments, the video codec may use downsampled luminance A group of samples (e.g., a selected set of downsampled luminance samples 514) Search for a predefined number (e.g., two) of the groups with the largest luminance values. downsampled luma samples (e.g., 514A-4 and 514A-5), and downsampling a predefined number (e.g., two) of pixels with a minimum luminance value. Next, identify the luminance samples that are not included in the image (e.g., 514B-4 and 514B-5). The image codec may select the chroma samples within a selected group of chroma samples 516 (e.g., 516A-4, 516A-5, 516B-4, and 516B-5) and The samples are the maximum downsampled luma samples 514A-4 and 514A -5 groups and the smallest downsampled luminance samples 514B-4 and 51 It corresponds to one of the groups of 4B-5. Next, the video codec is For each identified group of saturation and luminance samples, a value (e.g., saturation or luminance value) Perform a weighted average of the most significant saturation samples (e.g., saturation samples 516A-4 and 516A-5) The grand average saturation value, the minimum average saturation (for example, of saturation samples 516B-4 and 516B-5). the maximum average downsample value (for example, of luminance samples 514A-4 and 514A-4); The pulled luminance values, and (e.g., luminance samples 514B-4 and 514B-5 This produces the minimum average downsampled luminance value. The input model 512 outputs the downsampled luminance samples (514A-4, 514A- 5, 514B-4 and 514B-5) and chroma samples (516A-4, 516A- 5, 516B-4 and 516B-5).

[0079] Additionally, in some embodiments, the linear mapping model 512 may be Multiple downsampling (within a selected group of sampled luminance samples 514) Within a selected group of matched luma samples and (e.g., chroma samples 516 ) is obtained by creating a linear fitting curve for multiple corresponding chroma samples. Such curve fitting has a deviation error that selectively meets the tolerance. Each downsampled luma sample corresponds to a predefined number of chroma samples. A predefined number of downsampled luminance samples (e.g., greater than 10) In some embodiments, the downsampled luminance samples include a linear map. To obtain the ping model 512, adjacent downsampled luminance samples 514 are randomly selected from the group.

[0080] 6A-6D are diagrams illustrating four exemplary coding blocks 506A- 506D, where each coding block is a plurality of luma samples 502 converted to a plurality of chroma samples 502. Each cross ("x") represents the location of a luminance sample 504, and each circle ("○") indicates a chroma sample 502 or a downsampled luma sample 508 Each circled cross represents the location of a saturation sample 502, The location where the luma sample 504 and the downsampled luma sample 508 overlap Each coding block 506 is marked with a block boundary 602. Block 506 selects the corresponding subsampling schemes for the luma and chroma components (Y': Cb:Cr) and adopts a corresponding luminance interpolation scheme 510 to generate a corresponding coded block. The luminance samples 504 of block 506 are converted to corresponding downsampled luminance samples 508. In some embodiments of the present application, for each coding block 506, In particular, the downsampled luminance samples 508 are directly aligned to the block boundaries 602. In the case of adjacent pixels, all downsampled luma samples 508 are 510 is used to extract adjacent luminance samples located in the same corresponding coding block 506. Each downsampled luma sample 508 (respective A luma sample 508) always overlaps with its corresponding chroma sample 502, is then the downsampled luminance sample based on the linear mapping model 512. 508.

[0081] In some embodiments, the inter-component filter model 520 may be a linear filter model based on the intensity interpolation scheme 510. The luminance for each coding block 506 is calculated by combining the luminance for each coding block 506 with the shape mapping model 512. The linear mapping model 512 of the interpolation scheme 510 and the two filter coefficients f(i) are Given the linear coefficients α and β, the chroma samples 502 of the coding block 506 are In particular, when the downsampled luminance sample 508 is directly adjacent to the block boundary 602, In this case, the inter-component filter model 520 is used to filter the corresponding The intensity is obtained directly from the attached luminance samples 504.

[0082] In some embodiments, the coding block 506 is the first coding block of a video frame. and no other coding blocks 506 have been reconstructed before the coding block 506. Such a coding block 506 is preferably located in the upper left corner of the video frame. The linear mapping model 512 is independent of the coding blocks in the video frame (e.g., For example, based on a previous video frame. The coding block 506 is not the first coding block of a video frame, but One or more other coding blocks 506 have been reconstructed before the block 506. For example, The coding block 506 is located in the center of the video frame. The adjacent coding block above or to the left has already been processed and a linear mapping model 512 is determined. or reconstructing the luma samples or chroma samples in the coding block 506. In some circumstances, the encoding block 506 may be used to At least one block is located on the upper boundary of the frame and immediately to the left of the coding block 506. One coding block has already been processed and the linear mapping model 512 has been determined. is used to reconstruct the luma or chroma samples in the coding block 506. In some circumstances, the coding block 506 may be used to At least one code located on the left boundary side and directly above the coding block 506 The coding block has already been processed to determine the linear mapping model 512 or the coding The input signal is used to reconstruct the luma or chroma samples in the reconstruction block 506. Regardless of the position of the coding block 506, the coding block 506 can be moved to the right or The coding block below has not yet been processed, and the linear matrix for the coding block 506 is 5. Determining the mapping model 512 or the luminance samples in the coding block 506 Nor can it be used to reconstruct chroma samples.

[0083] In some embodiments, the same predefined intensity interpolation scheme 510 or inter-component filter may be used. The filter model 520 performs all downsampling across the coding block 506. are applied to generate the corresponding luma samples 508 or chroma samples 502, respectively. Alternatively, in some embodiments, the predefined luminance interpolation scheme 510 may be The position of the upsampled luma sample 508 or the downsampled luma sample It varies with the location of the boundary pixel associated with sample 508. In this embodiment, the inter-component filter model 520 is The pixel position varies depending on the location of the boundary pixel associated with the sample 502. For example, Each of the intensity interpolation schemes 510 and each of the inter-component filter models 520 is a cross, block, The filter arrangements are either square, square, or T-shaped.

[0084] Referring to FIG. 6A, a coding block 506A is obtained in the bitstream and The stream includes a number of luma samples 504 reconstructed from the stream. The signal 502 is derived from a number of luminance samples 504. , the subsampling scheme of the luma component 504 and the chroma component 502 (Y':Cb:Cr ) has a three-part ratio of 4:2:0. Each chroma sample 502 is filtered by the inter-component filter The six adjacent luminance samples 504 are combined according to the model 520. The luma samples 504 are located in the upper left corner, directly above, the upper right corner, the lower left corner, and the upper right corner of the corresponding chroma samples 502. In other words, six of each chroma sample 502 are located in the bottom corner, directly below, and in the bottom right corner. The two adjacent luminance samples 504 are interpolated according to a predefined luminance interpolation scheme 510. chroma samples 502 are converted to downsampled luma samples 508 that overlap with the chroma samples 502 of , which are then transformed into the corresponding chroma samples 502 according to a linear mapping model 512. In the example, the first chroma sample 502A and its corresponding six adjacent luma samples 502B are 04A-1 to 504A-6 are completely enclosed within block boundary 602 and are the first color. The intensity sample 502A is a component-by-component filter model 520 or a luminance interpolation scheme 510 and and the linear mapping model 512 based on the combination of adjacent luminance samples 504A-1 can be obtained from 504A-6.

[0085] Conversely, the second chroma sample 502B is located at the left block boundary 6 of the coding block 506A. Immediately adjacent to 02A, six adjacent luminance samples are filtered according to the inter-component filter model 520. 504B-1 through 504B-6. In some circumstances, the coding block 506 The left block boundary 602A of the image frame overlaps with the left boundary of the image frame, and the left block boundary 602A of the image frame overlaps with the left boundary of the image frame. In the coding block, two adjacent luminance samples 504B-1 and 504B-6 are Unavailable. Luma samples 50 that are outside the coding block 506 and are unavailable. 4B-1 and 504B-6 are luminance samples 504B-1 and 504B-6, respectively. can be replicated from directly adjacent luminance samples 504B-2 and 504B-5. In one example, luma samples 504B-1 and 504B-6 are The luminance samples 504B-4 and 504B-5 are symmetrical with respect to the luminance samples 504B-1 and 504B-6. and 504B-3. Alternatively, in some circumstances, the coding block 50 The left block boundary 602A of 6 does not overlap with the left boundary of the video frame. The luminance samples 504B-1 and 504B-6 are utilized within the coding block 506 itself. However, it is possible that the coding block 506 may be reconstructed by other adjacent coding blocks that are reconstructed before the coding block 506. Luminance samples 504B-1 and 504B-2 outside the coding block 506 may be provided. 504B-1 and 504B-6 are the luminance samples 504B-1 and 504B-6 from any other coding block. 4B-6, the luma sample 50 in the coding block 506 is 4B-2 to 504B-5 may still be copied. The second chroma sample 502B and / or the second luma sample 508B are adjacent luma samples. The adjacent luminance samples 504B-1 to 504B-6 are further obtained. Among the two luminance samples 504B-1 and 504B-6, Two separate luminance samples (e.g., 504B-2 and and 504B-5).

[0086] In other words, a video signal having multiple luminance samples for multiple pixels in a video frame is generated. If a bit stream is obtained, then a number of pixels belong to the coding block 506A and are coded. 506A and directly on the boundary 602 of the coding block 506A. The boundary pixels (e.g., the pixel corresponding to luminance sample 504B-2) are included in the pixel. luminance sample 504B- 1). Each pixel of one or more adjacent pixels corresponds to a coding block. In some embodiments, one or more neighbors of the boundary pixel are outside of block 506A. A tangent pixel is outside of a video frame or image slice. One or more adjacent pixels of the boundary pixel have not yet been coded, and the coding block In some embodiments, the inter-component filter model 520 is , is confined to the coding block 506 and any neighboring pixels outside the coding block 506 The cell is considered unavailable. and 504B-5 for detecting a luminance sample corresponding to each pixel of one or more adjacent pixels. The boundary luminance samples 50 are assigned to the filters 504B-1 and 504B-6, respectively. 8B is a diagram showing a method for interpolating at least the boundary pixels and the pixel area according to a predefined intensity interpolation scheme 510. and one or more adjacent pixel luminance samples 504B-1 to 504B-6. and determining the boundary chroma sample 502B according to the linear mapping model 512. Alternatively, the boundary saturation sample 502B is used to filter the inter-component filter model 502B. 20, the brightness samples 504B of the boundary pixel and one or more adjacent pixels Determined from -1 to 504B-6.

[0087] 6B, in the coding block 506B, the luma component 504 and the chroma component The subsampling scheme of 502 (Y':Cb:Cr) is a three part ratio of 4:2: 0. Each chroma sample 502 is filtered using an inter-component filter model with a cross filter shape. 520. Five adjacent luminance samples are combined from 504 according to One of the brightness samples 504 overlaps with the chroma sample 502, and the remaining four adjacent brightness samples The intensity samples 504 are located directly above, below, to the left, and to the right of the corresponding chroma samples 502. In other words, the five adjacent luma samples 504 of each chroma sample 502 are The downsampled sample is then overlapped with the corresponding chroma sample 502 according to a predefined luma interpolation scheme 510. The sampled luminance samples 508 are then converted to a linear mapping model 512. Therefore, it is converted to the corresponding chroma sample 502. In one example, the third chroma sample 502C and the corresponding five adjacent luminance samples 504C-1 to 504C-5 are completely The third chroma sample 502C is enclosed within the block boundary 602 and is filtered by the inter-component filter model. 520 or a combination of a luminance interpolation scheme 510 and a linear mapping model 512. Based on this, adjacent luminance samples 504C-1 to 504C-5 can be obtained.

[0088] The fourth chroma sample 502D is located at the left block boundary 602A of the coding block 506B. 5 adjacent luminance samples 504 according to the inter-component filter model 520. In some circumstances, the left of the coding block 506B corresponds to The block boundary 602A overlaps with the left border of the video frame and is not aligned with this or any other code. The adjacent luminance sample 504D-5 is not available in the coding block. 504D-5 is derived from luminance sample 504D-1, which is directly adjacent to luminance sample 504D-5. or symmetrically with respect to the luminance sample 504D-5 with respect to the central luminance sample 504D-1. In some circumstances, the coding block 5 The left block boundary 602A of 06 does not overlap with the left boundary of the video frame. Sample 504D-5 is not available within coding block 506 itself, but is The reconstructed encoded block may be provided by another adjacent encoded block reconstructed before block 506B. The luma samples 504D-5 that are outside of the coding block 506 can be The coding block 504D-5 is then loaded into the luminance sample 504D-5 without relying on loading the luminance sample 504D-5 from the 504D-1 or 504D-3 located in the luminance sample ...2 or 504D-3 located in the luminance sample 504D-3. By these means, the fourth chroma sample 502D and / or the The fourth luminance sample 508D is derived from adjacent luminance samples 504D-1 to 504D-5. The luminance sample 504D-1 to 504D-5 in the adjacent luminance samples 504D-1 to 504D-5 is D-5 corresponds to luminance sample 504D-1 or 504D-3 in coding block 506B. The allocation is selectively based on the

[0089] The fifth chroma sample 502E is located at the left block boundary 602A of the coding block 506B. and immediately adjacent to the upper block boundary 602B, Thus, it corresponds to five adjacent luminance samples 504E-1 to 504E-5. In an embodiment, coding block 506B is the first coding block in a video frame. , two adjacent luminance samples 504E-2 in this or any other coding block and 504E-5 are unavailable. Luminance samples 504E-2 and 504E-5 are In some circumstances, the coding block 506 B is not located in the upper left corner of the video frame and is adjacent to its left and above Adjacent luminance samples 504E-2 and 504E-5 are encoded blocks. Others that are not available within the block 506 itself but are reconstructed before the encoding block 506 Alternatively, the luminance sample 504E-2 may be provided by a neighboring coding block. and 504E-5 by loading luma samples from any other coding block. Without relying on By these means, the fifth chroma sample 502E and and / or the fifth luminance sample 508E is a luminance sample obtained by subtracting the adjacent luminance samples 504E-1 to 504E-2 from the first luminance sample 504E-3. E-5, and the luminance samples in the adjacent luminance samples 504E-1 to 504E-5 are Samples 504E-2 and 504E-5 are aligned along block boundary 60 of coding block 506B. 2. It can be said.

[0090] 6C, in the coding block 506C, the luma component 504 and the chroma component The subsampling scheme of 502 (Y':Cb:Cr) is a three part ratio of 4:2: Each chroma sample 502 has an inter-component filter model with a 2×3 filter shape. The six adjacent luminance samples 504 are combined according to the filter 520. One of the luma samples 504 overlaps with the chroma sample 502, and the remaining five adjacent The luma samples are located to the left, right, bottom left corner, directly below, and to the right of the corresponding chroma samples 502. These six adjacent luma samples 504 for each chroma sample 502 are According to a predefined luma interpolation scheme 510, the doubling time is calculated by multiplying the doubling time with the corresponding chroma sample 502. The resulting luminance samples are then converted to linear mapping models 512. In one example, the sixth chroma sample 502 is converted to a corresponding chroma sample 502 according to F and its corresponding six adjacent luminance samples 504F-1 to 504F-6 are The sixth chroma sample 502F, which is entirely enclosed within the block boundary 602, is filtered using the inter-component filter model. 520 or a combination of the luminance interpolation scheme 510 and the linear mapping model 512. can be obtained from adjacent luminance samples 504F-1 to 504F-5 based on .

[0091] The seventh chroma sample 502G is located at the left block boundary 602A of the coding block 506C. and immediately adjacent to the lower block boundary 602C, Thus, six adjacent luminance samples 504G-1 to 504G-6 are corresponding to the In an embodiment, coding block 506C is located in the lower left corner of the video frame, and the left and The left and right block boundaries 602A and 602C correspond to the left and right block boundaries of the video frame. In some embodiments, the coding block 506C overlaps the bottom block boundary. Since the luminance samples 504G-1 and 504G-2 are located on the left boundary side of the frame, G-6 does not exist, and luminance samples 504G-4 and 504G-5 have not yet been encoded. In the bottom left corner or left border of the video frame, four adjacent luma samples 504G- 1, 504G-4, 504G-5, and 504G-6 are equivalent to said or any other encoding Each of these luma samples is not available in the block. 504G-2 or 504G-3, for example, luminance sample 504G-4 is The luminance samples 504G-1, 504G-5 and 504G-6 are replicated from the luminance sample 504G-3. The luminance samples 504G-1 and 504G-2 are replicated from the luminance samples 504G-3 and 504G-4. The pixelated block 506C is not located in the bottom left corner of the video frame, but is located to the left and below it. adjacent coding blocks are available. , 504G-5 and 504G-6 are not available within the coding block 506 itself, , provided by other adjacent coding blocks reconstructed before coding block 506. Alternatively, the luminance samples 504G-1, 504G-4, 504G-5 and 5 04G-6 does not rely on loading luma samples from any other coding block. Luminance samples 504G-2 and 504G-3 are located in the coding block 506. G-3 may still be replicated. By these means, the seventh chroma sample 5 02G and / or the seventh luminance sample 508G are adjacent luminance samples 504G-1 504G-1 to 504G-6, and the adjacent luminance samples 504G-1 to 504G-6 are The luminance samples 504G-1, 504G-4, 504G-5 and 504G-6 are Two other luminance values ​​are positioned adjacent to the block boundary 602 of the encoding block 506C. The allocation is selectively based on samples 504G-2 and 504G-3.

[0092] 6D, in the coding block 506D, the luma component 504 and the chroma component The subsampling scheme of 502 (Y':Cb:Cr) is a three part ratio of 4:4: Each chroma sample 502 has an inter-component filter model with a 2×3 filter shape. The six adjacent luminance samples 504 are combined according to the filter 520. One of the luma samples 504 overlaps with the chroma sample 502, and the remaining five adjacent The luma samples are located to the left, right, bottom left corner, directly below, and bottom right corner of the corresponding chroma samples 502. These six adjacent luma samples 504 for each chroma sample 502 are , which overlap with the corresponding chroma samples 502 according to a predefined luma interpolation scheme 510. The unsampled luminance samples 508 are then transformed into a linear mapping model 51 2, the corresponding chroma samples 502 are converted to the corresponding chroma samples 502 according to

[0093] The eighth chroma sample 502H is located at the bottom block boundary 602C of the coding block 506D. and immediately adjacent to the right block boundary 602D, Thus, six adjacent luminance samples 504H-1 to 504H-6 are corresponding to the In an embodiment, the coding block 506D is located in the lower right corner of the video frame, and the right and The right and bottom block boundaries 602D and 602C correspond to the right and bottom block boundaries of the video frame. overlaps the bottom block boundary. Luminance samples 504H-3 to 504H-6 are not available. Or, in some circumstances Now, coding block 506D is not located in the bottom left corner of the video frame, but to the right of it. Since the adjacent coding blocks below and above have not yet been reconstructed, The luminance samples 504H-3 to 504H-6 are not yet available. The sample is replicated from the luminance sample 504H-1 or 504H-2, e.g. Sample 504H-6 is replicated from luminance sample 504H-1, -3 to 504H-5 are replicated from luminance sample 504H-2. Thus, the eighth chroma sample 502H and / or the eighth luma sample 508H may be adjacent The luminance samples 504H-1 to 504H-6 are obtained from the adjacent luminance samples 50 The luminance samples 504H-3 to 504H-6 in the luminance samples 504H-1 to 504H-6 are encoded Two other luminance samples are positioned adjacent to the block boundary 602 of block 506D. The addresses are selectively assigned based on pulls 504H-1 and 504H-2.

[0094] FIG. 7 illustrates a video data decoding process performed on an electronic device, according to some embodiments. 7 is a flow chart of a method 700 for detecting multiple pixels in a video frame. A bitstream is obtained having an encoding block 506 containing luma samples 504. (702). A plurality of pixels are located within the coding block 506 and The electronic device includes boundary pixels that are immediately adjacent to the boundary 602 of the block 506. It is determined (704) that one or more adjacent pixels of the cell are unavailable. Each pixel in the or each of the adjacent pixels is outside the coding block 506. A luminance sample corresponding to a pixel (e.g., 504G-1 in FIG. 6C) is one or more The luminance samples corresponding to each pixel in the neighborhood (e.g., 504G-6 in FIG. 6C) ) based on the inter-component filter model 120 (706). , at least the brightness samples of the boundary pixels and one or more adjacent pixels (e.g. , 502G-1 to 502G-6) based on the boundary saturation samples (e.g., 50 in FIG. 6C Specifically, the electronic device determines a predefined luminance interpolation scheme 51. 0, at least the brightness samples of the boundary pixels and one or more adjacent pixels are taken. Based on the rule (e.g., 502G-1 to 502G-6), the boundary luminance sample (e.g., Fig. 6C 508G) is determined (710). ) is determined (712) from the boundary luminance samples according to the linear mapping model 512. .

[0095] In some embodiments, the predefined intensity interpolation scheme 510 may include at least one boundary pixel. Based on the location of the pixel, one or more neighboring pixels are determined based on a predefined brightness. 5. The pixel values ​​are then interpolated based on a degree interpolation scheme 510. In some situations, a predefined The luma interpolation scheme 510 also interpolates the luma and chroma samples of the multiple pixels. is determined based on a subsampling scheme conforming to the three-part Y'CbCr ratio. In some circumstances, the predefined intensity interpolation scheme 510 may also be used to Syntax elements derived from the stream (e.g., "sps_chroma_colloca ted_vertical_flag"), optionally The defined intensity interpolation scheme 510 may be implemented in a cross, block, or T-shaped filter configuration. Corresponds to one.

[0096] In some embodiments, one or more neighboring pixels of the boundary pixel are In some embodiments, one of the border pixels or A number of adjacent pixels, which have not yet been coded, are coded following the coding block.

[0097] In some embodiments, the pixels are first completely surrounded by a coding block. The set of pixels includes an internal luminance sample (e.g., 508A, 50 8C, and 508F) are used to interpolate the first pixel according to a predefined luminance interpolation scheme 510. The first set of pixels is determined from a set of luminance samples corresponding to the set of pixels. The corresponding internal chroma samples (e.g., 502A, 502C, and 50 2F) is determined based on the interior luminance samples according to a linear mapping model 512. do.

[0098] In some embodiments, for example in FIG. 6A, a number of pixels are coded into a coding block. 504B-3, 504B-4) The border luminance samples (e.g., 508B) are samples of a predefined luminance. The interior luminance samples of the interior pixels (e.g., 504B-3) are calculated according to a luminance interpolation scheme 510. , 504B-4) and the luminance samples of the boundary pixel and one or more adjacent pixels ( For example, it is determined based on both 504B-1 and 504B-2).

[0099] In some embodiments, for example in FIG. 6A, the boundary pixel may be the first boundary pixel. luminance sample 504B-1, corresponding to one or more adjacent pixels The cell may include one or more first neighboring pixels (e.g., luminance sample 504B-1). The pixels are within the coding block 506 and include A second boundary pixel (e.g., a luminance pixel) that is directly adjacent to the corresponding block boundary 602 of block 506 The electronic device may also include a second boundary pixel (corresponding to sample 504B-5). One or more adjacent second neighboring pixels (e.g., corresponding to luminance sample 504B-6) Each pixel of the one or more second neighboring pixels is coded as a coding block 50. 6. The luminance sample corresponding to the second boundary pixel (e.g., 504B-5) is a luminance sample corresponding to each pixel of one or more second neighboring pixels (e.g. , 504B-6), and the boundary luminance sample 508B is assigned to a predefined luminance compensation According to the spacing scheme 510, at least the first and second boundary pixels (e.g., luminance subpixels) are (corresponding to samples 504B-2 and 504B-5) and the first and second adjacent pixels Both luminance samples (e.g., corresponding to luminance samples 504B-1 and 504B-6) The decision is based on pull.

[0100] In some embodiments, for example, in FIG. 6B, one or more neighboring pixels may be: For a boundary pixel (e.g., corresponding to luminance sample 504D-1), It is divided from the luminance sample (e.g. 504D-1) that is on the opposite side and corresponds to the boundary pixel. One adjacent pixel ( For example, it contains only luminance sample 504D-5.

[0101] In some embodiments, for example, in FIG. 6C , the boundaries include a first boundary 602A; The boundary pixel (e.g., corresponding to the luminance sample 504G-1) is located at the first boundary 602A. and a second boundary 602C perpendicular to the first boundary 602A. One or more adjacent pixels are directly adjacent to the corner of the first boundary pixel. The first adjacent pixel on the opposite side of 602A (e.g., corresponding to luminance sample 504G-1) (1) a second adjacent pixel on the opposite side of the second boundary to the boundary pixel; (2) the block of coding block 506C (e.g., corresponding to luma sample 504G-5) A third adjacent pixel across the corner from the border pixel (e.g., luminance sample 504 It includes at least one of the following:

[0102] In some embodiments, the luminance samples of a number of pixels are calculated according to a subsampling scheme. The Y'Cb and chroma samples are three-part Y'Cb samples with one of the following ratio values: Compliant with Cr ratio: (1) 4:1:1, where every four horizontal pixels are four luminance sample, blue-difference chroma sample Cb, and red-difference chroma sample Cr;(2) 4: 2:0, where every 4 pixels in each 2x2 pixel block have 4 luma samples , corresponding to the blue-difference chroma sample Cb, and the red-difference chroma sample Cr; (3) 4:2:2, Here, every 4 pixels have 4 luma samples, 2 blue-difference chroma samples Cb, and and two red-difference chroma samples Cr; (4) 4:4:4, where four pixels The cell contains 4 luma samples, 4 blue-difference chroma samples Cb, and 4 red-difference chroma samples. Corresponds to the Cr.

[0103] In some embodiments, the interpolation is performed according to a predefined luminance interpolation scheme, e.g., FIG. In C and D, six adjacent luminance samples 504 in a 2×3 subarray are alternated. The luminance samples are downsampled to 508, and the luminance samples for each left and right column in the 2x3 array are A luminance sample is used at least twice to generate alternative luminance samples in the same row. In some embodiments, the interpolation is performed according to a predefined intensity interpolation scheme, e.g., as shown in FIG. In the example, five adjacent luminance samples 504 are downsampled to an alternative luminance sample 508. The pixels are ringed and arranged in a cross-shaped subarray centered on a central pixel.

[0104] In some embodiments, a second plurality of pixels for a second plurality of pixels in a video frame is a bitstream having luma samples 518 and a second plurality of chroma samples 516; The alternative plurality of luminance samples 514 are obtained according to a predefined luminance interpolation scheme 510. and has the same resolution as the multiple chroma samples 516. The first parameter α and the second parameter β for the delta 512 are used to represent alternative luminance values. The second plurality of chroma samples 514 and the second plurality of chroma samples 516 are used to determine the chroma intensity. In some embodiments, the border chroma sample Y is calculated using a linear matrix as described in equation (1) as follows: It is determined from the boundary luminance sample X according to the mapping model 512. In some embodiments, the two largest luminance samples 514A-4 and 514A-5 are Among the replacement luminance samples 514, the two smallest luminance samples 514B-4 are determined. and 514B-5 are determined among the alternative luminance samples 514. The first luminance sample 514A-4 and the second luminance sample 514A-5 are averaged to form a first luminance value. The minimum luminance samples 514B-4 and 514B-5 are averaged to produce a second luminance value. The two first chroma samples 516A-4 and 516A-5 are the two largest luma samples. Among the second plurality of chroma samples 514 associated with 514A-4 and 514A-5 The two second chroma samples 516B-4 and 516B-5 are the two highest A second plurality of chroma samples associated with the small luma samples 514B-4 and 514B-5. The two first saturation values ​​516A-4 and 516A-5 are determined in the first sample. and two second chroma values ​​516B-4 and 516B-5 is averaged to a second chroma value corresponding to a second luma value. The model 512 determines, based on the first and second luminance values ​​and the first and second chroma values, It is taken between the chroma sample 502 and the luma sample 508 .

[0105] In one or more examples, the functions described may be implemented in hardware, software, firmware, or other The implementation may be implemented in software, software, or any combination thereof. In this case, the functions may be stored as one or more instructions or code on a computer-readable medium. or transmitted by a computer-readable medium and executed by a hardware-based processing unit. A computer readable medium may correspond to a tangible medium such as a data storage medium. A computer-readable storage medium for storing a computer program, or a computer program for transmitting the computer program from one location, for example, a communication Any communication medium that can be easily transmitted to another location based on a protocol. In this manner, a computer-readable medium generally is: (1) non-transitory; or (2) a communication medium such as a signal or carrier wave. The data storage medium may correspond to one or more computers, or one or more The processor of the device accesses and retrieves instructions, code and / or data structures to Any available medium may be used in carrying out the embodiments described herein. The computer program product may include a computer-readable medium.

[0106] The terms used in the description of the embodiments herein are intended to describe specific embodiments. The above description is merely illustrative and is not intended to limit the scope of the claims. Unless expressly indicated otherwise, the following terms are used in the description of the embodiments and the appended claims: The singular forms "a," "one," and "the" are intended to include the plural forms as well. It will be further understood that, as used herein, the term "and / or" refers to one or more Refers to any and all possible combinations of the listed items and includes one or more of the associated It is to be further understood that the present invention includes any and all possible combinations of the listed items. The terms "comprise" and / or "include" as used herein refer to any of the features, elements and / or specifies the presence of an element, but not one or more other features, elements, components and / or The presence or addition of the group consisting of these is not excluded.

[0107] It will be further understood that the terms "first," "second," etc. may be used herein to describe various elements. These elements may be used to clarify, but are not limited by, these terms. These terms are only used to distinguish one type of element from another. For example, a first electrode may be referred to as a second electrode without departing from the scope of the embodiments, and Similarly, the second electrode may be referred to as the first electrode. But it's not the same electrode.

[0108] The description in this application is presented for purposes of illustration and description and is not intended to be exhaustive or to limit the invention in the form disclosed. The present invention is not intended to be limiting. Many modifications, variations, and alternative embodiments will be apparent to those skilled in the art. The above description is provided to best explain the principles and practical applications of the present invention and will enable those skilled in the art to easily understand the various aspects of the present invention. The present invention can be understood to be most suitable for various embodiments having basic principles and various modifications. For example, the present invention is intended to be adapted to the particular application anticipated. As such, the claims are not limited to the specific embodiments disclosed and are intended to cover modifications and other embodiments. Implementations are intended to fall within the scope of the following claims.

Claims

1. 1. A method for decoding video data, comprising: A plurality of luma samples for a plurality of pixels in a video frame from the bitstream. The plurality of pixels belong to a coding block, and the plurality of pixels are obtained by the boundary pixels located within the region and immediately adjacent to the boundary of the coding block; And, determining that one or more neighboring pixels of the boundary pixel are unavailable; each pixel of one or more neighboring pixels being outside the coding block; A luminance sample corresponding to the boundary pixel is calculated for each of the one or more adjacent pixels. assigning the pixels to corresponding luminance samples; At least the one or more adjacent pixels are interpolated according to a predefined luminance interpolation scheme. determining a boundary luminance sample based on the boundary pixel and the luminance samples of the boundary pixels; 、 determining boundary chroma samples from the boundary luma samples according to a linear mapping model And, 23. A method for decoding said video data comprising:

2. determining the predefined intensity interpolation scheme based on at least the positions of the boundary pixels; and determining a brightness interpolation scheme for the one or more neighboring pixels based on the predefined brightness interpolation scheme. the cell is adjacent to said boundary pixel; The method of claim 1 further comprising:

3. The predefined luminance interpolation scheme also includes subsamples corresponding to the plurality of pixels. A combination of a pulling scheme and syntax elements obtained from the bitstream. The method of claim 2 , wherein the determination is based on one of:

4. The one or more neighboring pixels of the boundary pixel are The method according to any one of claims 1 to 3, wherein the device is external to the chair.

5. The one or more neighboring pixels of the boundary pixel have not yet been coded, The method according to any one of claims 1 to 3, further comprising encoding the encoded block subsequent to said encoded block.

6. The plurality of pixels includes a first set of pixels that are completely surrounded by the coding block. Including, corresponding to the first set of pixels according to the predefined intensity interpolation scheme determining an interior luminance sample from a set of luminance samples; The first pixel is calculated based on the intra-luminance samples according to the linear mapping model. determining an internal saturation sample corresponding to the set of cells. Any method as described above.

7. The pixels are within the coding block and correspond to intra-luminance samples. further comprising an internal pixel for The boundary luminance samples are interpolated according to the predefined luminance interpolation scheme. The inner luminance sample of a pixel and the one or more adjacent pixels and the boundary pixel and the luminance samples of the 13. A method according to any preceding claim.

8. The boundary pixel is a first boundary pixel, and the one or more adjacent pixels are one or more first neighboring pixels; The plurality of pixels are within the coding block and are adjacent to each other. a second boundary pixel immediately adjacent to the boundary of the first pixel; 13. A method according to any preceding claim, comprising the steps of: creating one or more second adjacent pixels immediately adjacent to the second boundary pixel; Each pixel of the one or more second neighboring pixels is outside the coding block. and, a luminance sample corresponding to the second boundary pixel is subtracted from the one or more second neighboring pixels; For each pixel in the image, the boundary luminance samples are assigned to the corresponding luminance samples. At least the first and second boundary pixels are interpolated according to a predefined luminance interpolation scheme. and both the first and second neighboring pixels; Further includes:

9. The one or more neighboring pixels are on the opposite side of the boundary to the boundary pixel. and the corresponding luminance samples assigned from the luminance samples corresponding to the boundary pixels.

2. The method of claim 1, wherein the pixel includes only one adjacent pixel having the same pixel shape.

10. the boundary includes a first boundary; The boundary pixels are between the first boundary and a second boundary perpendicular to the first boundary. Immediately adjacent to a corner of the formed coding block, The one or more neighboring pixels are a reflection of the first boundary relative to the boundary pixel. (1) a first adjacent pixel on the opposite side of the boundary pixel; (2) a second adjacent pixel on the opposite side of the boundary of the coding block; and (3) across a corner of the coding block. and a third adjacent pixel opposite the boundary pixel.

13. A method according to any preceding claim.

11. Luma samples and chroma samples of the plurality of pixels according to a subsampling scheme. The sample conforms to a three-part Y'CbCr ratio having one of the following ratio values: ： (1) 4:1:1, where every 4 horizontal pixels contains 4 luma samples, blue-difference chroma samples, corresponding to the red-difference chroma sample Cb, and the red-difference chroma sample Cr; (2) 4:2:0, where every four pixels in each 2x2 pixel block are divided into four corresponding to the luma sample, the blue-difference chroma sample Cb, and the red-difference chroma sample Cr; (3) 4:2:2, where every 4 pixels contains 4 luma samples and 2 blue-difference samples. corresponding to a dither sample Cb, and two red-difference chroma samples Cr; (4) 4:4:4, where every 4 pixels contains 4 luminance samples, 4 blue-difference samples, corresponding to four dither chroma samples Cb, and four red-difference chroma samples Cr, 13. A method according to any preceding claim.

12. According to the predefined luminance interpolation scheme, six adjacent luminance values ​​in the 2×3 subarray are interpolated. The luminance samples are downsampled to alternate luminance samples, and each left and right The luminance samples in the first and second columns are at least Also used twice, 13. A method according to any preceding claim.

13. According to the predefined luminance interpolation scheme, five adjacent luminance samples are interpolated into alternative luminance samples. The pixels are downsampled to 1000 samples and then split into a cross-shaped subarray centered on the central pixel. To be placed, 13. A method according to any preceding claim.

14. extracting from the bitstream a second composite signal for a second plurality of pixels in a video frame; obtaining a second plurality of luma samples and a second plurality of chroma samples; the same resolution as the plurality of chroma samples according to the predefined luma interpolation scheme determining an alternative plurality of luminance samples having Using the alternative plurality of luma samples and the second plurality of chroma samples Thus, the first parameter α and the second parameter β for the linear mapping model are and 20. The method of any preceding claim, further comprising:

15. The boundary chroma sample Y is calculated using the following linear mapping model, which is described using the following equation: The method of claim 14 , wherein the boundary luminance samples X are determined according to a rule. Y = α * X + β

16. Determine a first parameter α and a second parameter β for the linear mapping model. To determine, identifying two largest luminance samples from the alternative plurality of luminance samples; identifying two smallest luminance samples from the alternative plurality of luminance samples; averaging the two largest luminance samples into a first luminance value; averaging the two smallest luminance samples into a second luminance value; among the second plurality of chroma samples associated with the two largest luma samples identifying two first chroma samples; among the second plurality of chroma samples associated with the two smallest luma samples. identifying two second chroma samples; averaging the two first chroma values ​​into a first chroma value corresponding to the first luminance value; And, averaging the two second chroma values ​​into a second chroma value corresponding to the second luminance value; And, a saturation value based on the first and second luminance values ​​and the first and second saturation values; Obtaining a linear mapping model between the luminance values; The method of claim 14 further comprising:

17. 1. An electronic device comprising: one or more processors; Memory, Including, The memory may include a processor that, when executed by the one or more processors, 17. A method according to claim 1, further comprising: storing instructions for carrying out the method according to claim 1; The electronic device.

18. A non-transitory computer-readable medium, comprising: When executed by one or more processors, the processors are provided with any one of claims 1 to 16. and storing instructions for carrying out one or more of the methods described herein. The non-transitory computer-readable medium.

Images