Apparatus and method for encoding video data

Abstract

A method for decoding a bitstream by an electronic device is provided. A block unit is determined based on an image frame received from the bitstream. First motion information having a first list tag for selecting a first reference frame and second motion information having a second list tag for selecting a second reference frame are received from a candidate list to reconstruct the block unit. When the first list tag is the same as the second list tag, a predefined one of the first motion information and the second motion information is stored for a subblock determined in the block unit without checking whether the first reference frame and the second reference frame are included in a reference list indicated by a tag value different from the two list tags. When a subsequent block is reconstructed based on the subblock, the subsequent block is reconstructed based on the stored motion information.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure claims the benefit of and priority to U.S. Provisional Patent Application Serial No. 62 / 865,443, filed on June 24, 2019, entitled “Motion Storing Process for Triangle Partition Prediction and Reference Sampling Process for Linear Model” (hereinafter “the '443 Provisional”) and U.S. Provisional Patent Application Serial No. 62 / 884,335, filed on August 8, 2019, entitled “High Level Syntax of Affine Related Mode and Sample Selection of Linear Model” (hereinafter “the '335 Provisional”). The disclosures of the '443 Provisional and the '335 Provisional are hereby fully incorporated by reference into this disclosure. Technical Field

[0003] The present disclosure relates generally to video coding, and more particularly, to techniques for storing motion information of a block unit in an image frame to predict a plurality of subsequent blocks. Background Art

[0004] The triangle partition mode and the geometric partition mode are encoding tools in the video encoding method. In the triangle partition mode and the geometric partition mode, the encoder can select two of multiple merge candidates to predict the block unit and provide two merge indexes to the bitstream for the decoder to identify the selected merge candidates.

[0005] The encoder and the decoder may partition a block unit in an image frame to generate a plurality of sub-blocks, and store at least one of two merge candidates for each of the sub-blocks in their buffers to predict a plurality of subsequent blocks. However, in the triangular partition mode and the geometric partition mode, some of the plurality of block components in the block unit may be predicted based on only one of the two merge candidates, and the other block components may be predicted based on the two merge candidates. If two merge candidates are stored for each of the sub-blocks predicted based on the two merge candidates, the stored motion candidates may be too complex for subsequent prediction. Summary of the invention

[0006] The present disclosure relates to an apparatus and method for disabling adjustment of an initial prediction result by comparing at least one component vector to a threshold value.

[0007] In a first aspect of the present disclosure, a method for decoding a bitstream and an electronic device for performing the method are provided. The method includes: receiving an image frame of the bitstream; determining a block unit from the received image frame; receiving first motion information and second motion information from a candidate list to reconstruct the block unit, wherein the first motion information includes a first list tag for selecting a first reference frame and the second motion information includes a second list tag for selecting a second reference frame; determining a subblock in the block unit; comparing the first list tag with the second list tag; when the first list tag is the same as the second list tag, storing a predefined one of the first motion information and the second motion information as the stored motion information for the subblock without checking whether the first reference frame and the second reference frame are included in a specific one of a plurality of reference lists indicated by a specific one of a plurality of tag values ​​different from the first list tag and the second list tag; and when reconstructing a subsequent block based on the subblock, reconstructing the subsequent block based on the stored motion information.

[0008] In a second aspect of the present disclosure, a method for decoding a bitstream and an electronic device for performing the method are provided. The method includes: receiving an image frame of the bitstream; determining a block unit from the received image frame; determining a partition line of the block unit; receiving first motion information and second motion information from a candidate list to reconstruct the block unit, wherein the first motion information includes a first list mark and the second motion information includes a second list mark; determining a subblock in the block unit; determining whether the subblock is included in a specific one of a plurality of block areas separated based on the partition line, wherein the specific one of the plurality of block areas covers the partition line; when the subblock is included in the specific one of the plurality of block areas, determining whether to store the first motion information and the second motion information together as stored motion information for the subblock based only on a comparison between the first list mark and the second list mark; and when a subsequent block is reconstructed based on the subblock, reconstructing the subsequent block based on the stored motion information. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Various aspects of the exemplary disclosure will be best understood from the following detailed description when read in conjunction with the accompanying drawings. Various features are not drawn to scale, and the dimensions of the various features may be arbitrarily increased or reduced for clarity of discussion.

[0010] Figure 1 A block diagram illustrating an exemplary system configured to encode and decode video data according to an exemplary embodiment of the present disclosure is shown.

[0011] Figure 2The exemplary embodiment according to the present application is shown Figure 1 Block diagram of an exemplary decoder module of a second electronic device in.

[0012] Figure 3 A flowchart illustrating an exemplary reconstruction method for reconstructing a block unit according to an exemplary embodiment of the present disclosure is shown.

[0013] FIG. 4A to FIG. 4D is a schematic diagram of a block unit having different prediction areas separated based on a partition line according to an exemplary embodiment of the present application.

[0014] FIG. 5A to FIG. 5C is a schematic diagram of a block unit having a plurality of sub-blocks classified into different block areas according to an exemplary embodiment of the present application.

[0015] Figure 6 A flowchart illustrating another exemplary reconstruction method for reconstructing a block unit according to an exemplary embodiment of the present disclosure is shown.

[0016] Figure 7 is a schematic diagram of a block unit and a plurality of reference samples according to an exemplary embodiment of the present application.

[0017] Figure 8 The exemplary embodiment according to the present application is shown Figure 1 Block diagram of an exemplary encoder module of a first electronic device in. DETAILED DESCRIPTION

[0018] The following description contains specific information related to exemplary embodiments in the present disclosure. The drawings and the detailed description attached thereto in the present disclosure are only for exemplary embodiments. However, the present disclosure is not limited to these exemplary embodiments. Other variations and embodiments of the present disclosure will occur to those skilled in the art.

[0019] Unless otherwise indicated, like or corresponding elements in the figures may be indicated by like or corresponding reference numerals.The drawings and illustrations in the present disclosure are generally not drawn to scale and are not intended to correspond to actual relative sizes.

[0020] For the purpose of consistency and ease of understanding, in the exemplary drawings, similar features may be identified by numbers (but not shown in some examples). However, features in different embodiments may differ in other aspects and should not be narrowly limited to what is shown in the drawings.

[0021] Where the description uses the phrases "in one embodiment" or "in some embodiments", these phrases may refer to one or more of the same or different embodiments. The term "coupled" is defined as connected, whether directly or indirectly through intermediate components, and is not necessarily limited to physical connections. The term "comprising" means "including but not necessarily limited to" and specifically indicates open inclusion or membership in the combinations, groups, series, and equivalents so described.

[0022] For the purpose of explanation rather than limitation, specific details (such as functional entities, technologies, protocols and standards) are set forth to provide an understanding of the disclosed technology. In other examples, detailed descriptions of well-known methods, technologies, systems and architectures are omitted to avoid obscuring the description with unnecessary details.

[0023] Those skilled in the art will immediately recognize that any disclosed coding functions or algorithms described in this disclosure can be implemented by hardware, software, or a combination of software and hardware. The functions described may correspond to modules, which may be software, hardware, firmware, or any combination thereof.

[0024] Software implementations may include computer executable instructions stored on a computer readable medium such as a memory or other type of storage device. For example, one or more microprocessors or general purpose computers with communication processing capabilities may be programmed with executable instructions and perform the disclosed functions or algorithms.

[0025] The microprocessor or general purpose computer may be formed by an application specific integrated circuit (ASIC), a programmable logic array, and / or using one or more digital signal processors (DSP). Although some of the disclosed embodiments are oriented to software installed and executed on computer hardware, alternative embodiments implemented as firmware or hardware or a combination of hardware and software are fully within the scope of the present disclosure. Computer readable media include, but are not limited to, random-access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, compact disc read-only memory (CD-ROM), cassettes, magnetic tapes, disk storage devices, or any other equivalent medium capable of storing computer readable instructions.

[0026] Figure 1 A block diagram of a system 100 configured to encode and decode video data according to an exemplary embodiment of the present disclosure is shown. The system 100 includes a first electronic device 110, a second electronic device 120, and a communication medium 130. The first electronic device 110 may be a source device, including any device configured to encode video data and transmit the encoded video data to the communication medium 130. The second electronic device 120 may be a target device, including any device configured to receive the encoded video data via the communication medium 130 and decode the encoded video data.

[0027] In at least one embodiment, the first electronic device 110 may communicate with the second electronic device 120 via a wire or wirelessly via a communication medium 130. The first electronic device 110 may include a source module 112, an encoder module 114, and a first interface 116. The second electronic device 120 may include a display module 122, a decoder module 124, and a second interface 126. The first electronic device 110 may be a video encoder, and the second electronic device 120 may be a video decoder.

[0028] In at least one embodiment, the first electronic device 110 and / or the second electronic device 120 may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, or other electronic devices. Figure 1 Only one example of the first electronic device 110 and the second electronic device 120 is shown. In other implementations, the first electronic device 110 and the second electronic device 120 may include more or fewer components than shown or have different configurations of the various components.

[0029] In at least one embodiment, the source module 112 may include a video capture device for capturing new video, a video archive for storing previously captured video, and / or a video feed interface for receiving video from a video content provider. The source module 112 may generate computer graphics-based data as the source video, or a combination of live video, archived video, and computer-generated video as the source video. The video capture device may be a charge-coupled device (CCD) image sensor, a complementary metal-oxide-semiconductor (CMOS) image sensor, or a camera.

[0030] In at least one embodiment, the encoder module 114 and the decoder module 124 may each be implemented as any of a variety of suitable encoder / decoder circuits, such as one or more microprocessors, central processing units (CPUs), graphic processing units (GPUs), systems on chips (SoCs), digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When partially implemented in software, the device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the disclosed method. In at least one embodiment, each of the encoder module 114 and the decoder module 124 may be included in one or more encoders or decoders, wherein any encoder or decoder may be integrated in the device as part of a combined encoder / decoder (CODEC).

[0031] In at least one embodiment, the first interface 116 and the second interface 126 may utilize custom protocols or comply with existing standards or de facto standards, including but not limited to Ethernet, IEEE 802.11 or IEEE 802.15 series; Wireless USB or telecommunication standards, including but not limited to GSM, CDMA2000, TD-SCDMA, WiMAX, 3GPP-LTE or TD-LTE. In at least one embodiment, the first interface 116 and the second interface 126 may each include any device configured to transmit and / or store compatible video bitstreams via the communication medium 130 and receive compatible video bitstreams via the communication medium 130.

[0032] In at least one embodiment, the first interface 116 and the second interface 126 may include computer system interfaces that enable compatible video bitstreams to be stored on or received from a storage device. For example, the first interface 116 and the second interface 126 may include chipsets that support Peripheral Component Interconnect (PCI) and Peripheral Component Interconnect Express (PCIe) bus protocols, proprietary bus protocols, Universal Serial Bus (USB) protocols, I2C, or any other logical and physical structures that can be used to interconnect peer devices.

[0033] In at least one embodiment, the display module 122 may include a display using liquid crystal display (LCD) technology, plasma display technology, organic light emitting diode (OLED) display technology, or light emitting polymer display (LPD) technology, as well as other display technologies used in other embodiments. The display module 122 may include a high-definition display or an ultra-high-definition display.

[0034] Figure 2 The present invention shows an exemplary embodiment of the present invention. Figure 12242 is a block diagram of an implementation of a decoder module 124 of a second electronic device 120 in FIG. The decoder module 124 includes an entropy decoder (e.g., an entropy decoding unit 2241), a prediction processor (e.g., a prediction processing unit 2242), an inverse quantization / inverse transform processor (e.g., an inverse quantization / inverse transform unit 2243), an adder (e.g., an adder 2244), a filter (e.g., a filtering unit 2245), and a decoded picture buffer (e.g., a decoded picture buffer 2246). The prediction processing unit 2242 also includes an intra-frame prediction processor (e.g., an intra-frame prediction unit 22421) and an inter-frame prediction processor (e.g., an inter-frame prediction unit 22422). The decoder module 124 receives a bitstream and decodes the bitstream to output a decoded video.

[0035] The entropy decoding unit 2241 can be obtained from Figure 1 The second interface 126 in receives a bitstream including a plurality of syntax elements and performs a parsing operation on the bitstream to extract the syntax elements from the bitstream. As part of the parsing operation, the entropy decoding unit 2241 may entropy decode the bitstream to generate quantized transform coefficients, quantization parameters, transform data, motion vectors, intra-frame modes, partition information, and other syntax information.

[0036] In at least one embodiment, the entropy decoding unit 2241 may perform context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding technique to generate quantized transform coefficients. In at least one embodiment, the entropy decoding unit 2241 may provide the quantized transform coefficients, quantization parameters, and transform data to the inverse quantization / inverse transform unit 2243 and provide the motion vector, intra mode, partition information, and other syntax information to the prediction processing unit 2242.

[0037] In at least one embodiment, the prediction processing unit 2242 may receive syntax elements such as motion vectors, intra modes, partition information, and other syntax information from the entropy decoding unit 2241. The prediction processing unit 2242 may receive syntax elements including partition information and divide the image frame according to the partition information.

[0038] In at least one embodiment, each of the image frames may be divided into at least one image block according to the partition information. The at least one image block may include a luminance block for reconstructing a plurality of luminance samples and at least one chrominance block for reconstructing a plurality of chrominance samples. The luminance block and the at least one chrominance block may be further divided to generate a macroblock, a coding tree unit (CTU), a coding block (CB), a subdivision thereof, and / or another equivalent coding unit.

[0039] In at least one embodiment, during the decoding process, the prediction processing unit 2242 receives prediction data including an intra mode or a motion vector for a current image block of a particular one of the image frames. The current image block may be one of a luminance block or a chrominance block in a particular one of the image frames.

[0040] In at least one embodiment, the intra-frame prediction unit 22421 may perform intra-frame prediction encoding on the current block unit relative to one or more neighboring blocks in the same frame as the current block unit based on syntax elements related to the intra-frame mode to generate a prediction block. The intra-frame mode may specify the position of the reference sample selected from the neighboring blocks in the current frame. In at least one embodiment, when the chroma component is reconstructed by the prediction processing unit 2242, the intra-frame prediction unit 22421 may reconstruct multiple chroma components of the current block unit based on multiple luminance components of the current block unit.

[0041] In at least one embodiment, when the luminance component of the current block is reconstructed by the prediction processing unit 2242, the intra prediction unit 22421 may reconstruct multiple chrominance components of the current block unit based on multiple luminance components of the current block unit.

[0042] In at least one embodiment, the inter prediction unit 22422 may perform inter prediction encoding on the current block unit relative to one or more blocks of the one or more reference image blocks based on syntax elements related to the motion vector to generate a prediction block.

[0043] In at least one embodiment, the motion vector may indicate the displacement of a current block unit within the current image block relative to a reference block unit within the reference image block. The reference block unit is a block determined to closely match the current block unit.

[0044] In at least one embodiment, the inter-frame prediction unit 22422 receives a reference image block stored in the decoded picture buffer 2246 and reconstructs the current block unit based on the received reference image block.

[0045] In at least one embodiment, the inverse quantization / inverse transform unit 2243 may apply inverse quantization and inverse transform to reconstruct the residual block in the pixel domain. The inverse quantization / inverse transform unit 2243 may apply inverse quantization to the residual quantized transform coefficients to generate residual transform coefficients, and then apply inverse transform to the residual transform coefficients to generate a residual block in the pixel domain.

[0046] In at least one embodiment, the inverse transform may be applied in reverse from a transform process such as a discrete cosine transform (DCT), a discrete sine transform (DST), an adaptive multiple transform (AMT), a mode-dependent non-separable secondary transform (MDNSST), a hypercube-givens transform (HyGT), a signal-dependent transform, a Karhunen-Loéve transform (KLT), a wavelet transform, an integer transform, a subband transform, or a conceptually similar transform.

[0047] In at least one embodiment, the inverse transform may convert the residual information from a transform domain such as the frequency domain back to the pixel domain. In at least one embodiment, the degree of inverse quantization may be modified by adjusting a quantization parameter. Summer 2244 adds the residual block to the prediction block from prediction processing unit 2242 to produce a reconstructed block.

[0048] In at least one embodiment, summer 2244 adds the reconstructed residual block to the prediction block provided from prediction processing unit 2242 to generate a reconstructed block.

[0049] In at least one embodiment, the filtering unit 2245 may include a deblocking filter, a sample adaptive offset (SAO) filter, a bilateral filter, and / or an adaptive loop filter (ALF) to remove block artifacts from the reconstructed block. In addition to the deblocking filter, SAO filter, bilateral filter, and ALF, additional filters (in the loop or after the loop) may also be used. Such filters are not explicitly shown for simplicity but may filter the output of the summer 2244.

[0050] After the filtering unit 2245 performs a filtering process on a reconstructed block of a specific image frame, the filtering unit 2245 may output the decoded video to the display module 122 or other video receiving units.

[0051] In at least one embodiment, the decoded picture buffer 2246 may be a reference picture memory for storing reference blocks used by the prediction processing unit 2242 to decode the bitstream (in inter-coding mode). The decoded picture buffer 2246 may be formed by any of a variety of memory devices, such as: dynamic random-access memory (DRAM), including synchronous DRAM (SDRAM), magneto-resistive RAM (MRAM), resistive RAM (RRAM); or other types of memory devices.

[0052] In at least one embodiment, decoded picture buffer 2246 may be on-chip with other components of decoder module 124 or off-chip relative to those components.

[0053] Figure 3 A flow chart of an exemplary reconstruction method 300 for reconstructing a block unit according to an exemplary embodiment of the present disclosure is shown. The method 300 is merely an example, as there are many ways to perform the method.

[0054] Method 300 can be used Figure 1 and Figure 2 The method 300 is performed with reference to the configuration shown, and various elements of these figures are referenced when explaining the method 300 . Figure 3 Each block shown may represent one or more processes, methods, or subroutines performed.

[0055] Furthermore, the order of the blocks is illustrative only and may be varied. Additional blocks may be added or fewer blocks may be utilized without departing from the present disclosure.

[0056] At block 310, the decoder module 124 receives an image frame of video data and determines a block unit from the image frame based on the received video data. The video data may be a bitstream.

[0057] refer to Figure 1 and Figure 2 The second electronic device 120 may receive a bitstream from an encoder or other video provider such as the first electronic device 110 via the second interface 126 . The second interface 126 may provide the bitstream to the decoder module 124 .

[0058] The decoder module 124 may determine an image frame based on the bitstream and divide the image frame according to a plurality of partition indications in the bitstream to determine a block unit. For example, the decoder module 124 may divide the image frame to generate a plurality of coding tree units and may further divide one of the coding tree units according to the partition indication (e.g., based on a video coding standard) to determine a block unit.

[0059] In at least one embodiment, the entropy decoding unit 2241 may decode the bitstream to determine multiple prediction indicators of the block unit and the decoder module 124 may further reconstruct the block unit based on the prediction indicators. The prediction indicators may include multiple flags and multiple indexes.

[0060] At block 320 , the decoder module 124 determines the first motion information and the second motion information from the candidate list and determines a partition line of the block unit to reconstruct the block unit.

[0061] In at least one embodiment, reference Figure 2 , the decoder module 124 may divide the block unit into a plurality of prediction regions, and reconstruct the prediction regions of the block unit based on the first motion information and the second motion information.

[0062] In at least one embodiment, the prediction indication may include a partition index of the block unit. The partition index may indicate a partition line of the block unit. In at least one embodiment, when the partition index is a triangular partition index, the partition line may be one of a diagonal partition line and an anti-diagonal direction. In at least one embodiment, when the partition index is a geometric partition index, the partition index may indicate a partition angle index and a partition distance index. The partition angle index may indicate an inclination angle of the partition line, and the partition distance index may indicate a line offset between the partition line and a center point of the block unit. Therefore, the decoder module 124 may determine the partition line based on the inclination angle and the line offset indicated by the partition index.

[0063] In at least one embodiment, the prediction indication may include a first motion candidate index and a second motion candidate index. In at least one embodiment, the first motion candidate index may indicate the first motion information in the merge candidate list, and the second motion candidate index may indicate the second motion information in the merge candidate list. In at least one embodiment, multiple merge candidate modes in the merge candidate list may be selected from multiple spatial motion prediction modes of multiple adjacent blocks adjacent to the block unit, multiple temporal motion prediction modes of multiple co-located blocks, history-based motion prediction modes stored in a first-in-first-out (FIFO) table, multiple pair-wise average motion prediction modes, and zero motion modes. In at least one embodiment, the first motion information may indicate a first reference frame and a first reference vector, and the second motion information may indicate a second reference frame and a second reference vector. The first reference frame and the second reference frame may be selected from multiple reference lists of the block unit. In one embodiment, the first reference frame and the second reference frame may be selected from the same one in the reference list. In other embodiments, the first reference frame and the second reference frame may be selected from different reference lists. In at least one embodiment, the number of reference lists may be equal to 2.

[0064] FIG. 4A to FIG. 4B is a schematic diagram of a block unit with different segmentation lines according to an exemplary embodiment of the present application. In at least one embodiment, the partition indication of the block unit 400 may indicate the segmentation line 420, the first reference vector V0, the second reference vector V1, the first reference frame, and the second reference frame. FIG. 4C to FIG. 4D 4 is a schematic diagram of a block unit with different prediction areas separated based on a dividing line according to an exemplary embodiment of the present application. In this embodiment, the decoder module 124 may divide the block unit 400 based on the dividing line 420 to determine the prediction areas 441-443. In at least one embodiment, the block unit may include a plurality of block components. In at least one embodiment, the block components in the first prediction area 441 may be reconstructed based on the first reference vector and the first reference frame, and the block components in the second prediction area 442 may be reconstructed based on the second reference vector and the second reference frame. In addition, the block components in the third prediction area 443 may be reconstructed based on the first reference vector, the second reference vector, the first reference frame, and the second reference frame. The block components in the third prediction area 443 may be reconstructed by: deriving a plurality of first reference samples determined based on the first reference vector and the first reference frame and a plurality of second reference samples determined based on the second reference vector and the second reference frame; and merging the first reference samples and the second reference samples based on a plurality of mixing weights. In at least one embodiment, the mixing weights may be derived based on a plurality of component distances between the dividing line 420 and the block components in the third prediction area 443.

[0065] At block 330 , the decoder module 124 determines sub-blocks in the block unit.

[0066] In at least one embodiment, reference Figure 2 , the decoder module 124 may divide the block unit to generate a plurality of sub-blocks. Figure 5A is a schematic diagram of a block unit having a plurality of sub-blocks 561-576 according to an exemplary embodiment of the present application, each sub-block corresponding to a sub-block index (xSbIdx, ySbInx). In at least one embodiment, the decoder module 124 may determine a specific one of the sub-blocks. In one embodiment, the number numSbX of the plurality of first candidate indexes for the first position index xSbIdx may be equal to four, and the number numSbY of the plurality of second candidate indexes for the second position index ySbIdx may be equal to four. Therefore, the first position index xSbIdx may be a number between zero and numSbX, and the second position index ySbIdx may be a number between zero and numSBY.

[0067] At block 340, the decoder module 124 determines whether the sub-block is included in a specific one of the plurality of block regions that covers the partition line. In at least one embodiment, when the decoder module 124 determines that the sub-block is included in the specific block region that covers the partition line, the method 300 may proceed to block 350. When the decoder module 124 determines that the sub-block is not included in the specific block region, the method may end.

[0068] In at least one embodiment, reference Figure 2 , the decoder module 124 may divide the block unit to generate a plurality of block regions. FIG. 5B to FIG. 5C is a schematic diagram of a block unit having different block areas separated based on a dividing line according to an exemplary embodiment of the present application.

[0069] In at least one embodiment, the decoder module 124 may classify the sub-blocks 561-576 into block regions based on the sub-block index. In at least one embodiment, the third block region 583 may cover the partition line 420, and the first block region 581 and the second block region may not cover the partition line 420. Most of the block components of the sub-blocks in the first block region 581 may be reconstructed based on the first motion information, and most of the block components of the sub-blocks in the second block region 582 may be reconstructed based on the second motion information. Therefore, when the sub-block is included in the first block region 581 corresponding to the first prediction area 441, the decoder module 124 may store the first motion information for the sub-block. For example, the decoder module 124 may store the first motion information for the sub-blocks 562-564, 567-568, and 572 because the sub-blocks 562-564, 567-568, and 572 are included in the first block region 581. In addition, when the sub-block is included in the second block area 582 corresponding to the second prediction area 442, the decoder module 124 may store the second motion information for the sub-block. For example, the decoder module 124 may store the second motion information for the sub-blocks 565, 569-570, and 573-575 because the sub-blocks 562-564, 567-568, and 572 are included in the second block area 582. The decoder module 124 may directly store a corresponding one of the first motion information and the second motion information for the sub-blocks included in other block areas different from the specific block area 583 that covers the partition line, and the method 300 may end without performing other procedures.

[0070] In at least one embodiment, most of the block components of the sub-block in the third block area 583 can be reconstructed based on the first motion information and the second motion information. Therefore, when the sub-block is included in the third block area 583 corresponding to the third prediction area 443, the method 300 can proceed to block 350 so that the decoder module 124 determines how to store at least one of the first motion information and the second motion information for the sub-block.

[0071] At block 350, the decoder module 124 determines whether the first list marker of the first motion information is the same as the second list marker of the second motion information. In at least one embodiment, when the first list marker is the same as the second list marker, the method 300 may proceed to block 360. When the first list marker is different from the second list marker, the method 300 may proceed to block 370.

[0072] In at least one embodiment, reference Figure 2 and Figure 5B, when the sub-block is included in the specific block area 583, the decoder module 124 may compare the first list tag with the second list tag. When the sub-block is included in the specific block area 583, most of the block components in the sub-block can be reconstructed based on the first motion information and the second motion information. Therefore, when the first list tag is different from the second list tag, the block components in the specific block area 583 can be reconstructed based on the reference frames in the two reference lists. In addition, when the first list tag is the same as the second list tag, the block components in the specific block area 583 can be reconstructed based on the reference frames in the same reference list.

[0073] At block 360 , when the first list flag is the same as the second list flag, the decoder module 124 stores a predefined one of the first motion information and the second motion information for the sub-block.

[0074] In at least one embodiment, reference Figure 2 and Figure 5B , when the first list mark of the block unit 400 is the same as the second list mark of the block unit 400 and the subblock is included in the specific block area 583, most of the block components in the subblock can be reconstructed by deriving the first reference sample and the second reference sample according to one of the reference lists.

[0075] Since the motion information is stored so as to reconstruct a plurality of subsequent blocks based on the stored motion information, the stored motion information may not affect a plurality of residual components generated based on a mixed result of the first reference sample and the second reference sample. Figure 1 , when the encoder module 114 and the decoder module 124 perform the method 300 to store the predefined one of the first motion information and the second motion information, the encoder module 114 and the decoder module 124 may store the same motion information for the sub-block. In at least one embodiment, when the first reference frame and the second reference frame are included in the same reference list, the decoder module 124 may directly store the predefined one of the first motion information and the second motion information to reduce the use of the buffer. Therefore, the decoder module 124 stores the predefined one of the first motion information and the second motion information for the sub-block.

[0076] In at least one embodiment, the decoder module 124 may directly store a predefined one of the first motion information and the second motion information as the stored motion information for the sub-block without further checking the relationship between the two reference frames and other reference lists. When the first list tag is the same as the second list tag, the decoder module 124 may directly store the predefined motion information without checking whether at least one of the first reference frame and the second reference frame is included in a specific one of the reference lists indicated by a specific one of the multiple tag values ​​that is different from the first list tag and the second list tag. For example, the number of reference lists is equal to two, so the tag value may include zero and one. In one embodiment, when the first list tag and the second list tag are equal to zero, the specific tag value is equal to one. The first list tag and the second list tag may indicate the first reference list L0, and the specific tag value may indicate the second reference list L1. Therefore, when the first list tag and the second list tag are equal to zero, the decoder module 124 may directly store the predefined motion information without checking whether at least one of the first reference frame and the second reference frame is included in the second reference list L1 indicated by the specific tag value equal to one.

[0077] In at least one embodiment, the predefined one of the first motion information and the second motion information may be the second motion information. Therefore, the decoder module 124 directly stores the second motion information for the sub-block. Since the second motion information is derived according to the second candidate index of the block unit in the bitstream, the stored motion information may be derived according to the second candidate index in the bitstream.

[0078] In block 370 , the decoder module 124 stores both the first motion information and the second motion information for the sub-block.

[0079] In at least one embodiment, reference Figure 2 and Figure 5B , when the first list tag of the block unit 400 is different from the second list tag of the block unit 400 and the subblock is included in the specific block area 583, most block components in the subblock can be reconstructed by deriving the first reference sample and the second reference sample according to different reference lists.

[0080] In at least one embodiment, when the first list flag of the block unit 400 is different from the second list flag of the block unit 400 and the sub-block is included in the specific block area 583, most of the block components in the sub-block can be reconstructed by deriving the first reference sample according to the first reference list and deriving the second reference sample according to the second reference list. Therefore, the decoder module 124 can directly store the first motion information and the second motion information for the sub-block without further determination.

[0081] In at least one embodiment, the decoder module 124 may directly store a predefined one of the first motion information and the second motion information for a sub-block having the same reference list and directly store both the first motion information and the second motion information for a sub-block having a different reference list. Therefore, when a sub-block is included in the specific block area 583, the decoder module 124 may determine whether to store the first motion information and the second motion information together as the stored motion information for the sub-block based only on the comparison between the first list flag and the second list flag.

[0082] At block 380 , when the subsequent block is predicted based on the sub-block, the decoder module 124 reconstructs the subsequent block based on the stored motion information.

[0083] In at least one embodiment, reference Figure 2 , the decoder module 124 may maintain the reconstructed image frame and multiple subsequent frames after the block unit is reconstructed. Therefore, the stored information of the block unit can be used to reconstruct the image frame and subsequent blocks in the subsequent frames. In at least one embodiment, the subsequent frame may include multiple unreconstructed frames reconstructed after the reconstruction of the image frame. In addition, the subsequent block may include multiple unreconstructed blocks in the image frame and the subsequent frame. The unreconstructed block may be reconstructed after the reconstruction of the block unit.

[0084] Figure 6 A flow chart of an exemplary method 600 for reconstructing a block unit according to an exemplary embodiment of the present disclosure is shown. The method 600 is merely an example, as there are many ways to perform the method.

[0085] Method 600 can be used Figure 1 and Figure 2 The method 600 is performed with reference to the configuration shown, and various elements of these figures are referenced when explaining the method 600 . Figure 6 Each block shown may represent one or more processes, methods, or subroutines performed.

[0086] Furthermore, the order of the blocks is illustrative only and may be varied. Additional blocks may be added or fewer blocks may be utilized without departing from the present disclosure.

[0087] At block 610, the decoder module 124 receives an image frame of video data and determines a block unit from the image frame based on the received video data. The video data may be a bitstream.

[0088] refer to Figure 1 and Figure 2 The second electronic device 120 may receive a bitstream from an encoder or other video provider such as the first electronic device 110 via the second interface 126 . The second interface 126 may provide the bitstream to the decoder module 124 .

[0089] The decoder module 124 may determine an image frame based on the bitstream and divide the image frame according to a plurality of partition indications in the bitstream to determine a block unit. For example, the decoder module 124 may divide the image frame to generate a plurality of coding tree units and may further divide one of the coding tree units according to the partition indication (e.g., based on a video coding standard) to determine a block unit.

[0090] In at least one embodiment, the entropy decoding unit 2241 may decode the bitstream to determine multiple prediction indicators of the block unit and the decoder module 124 may further reconstruct the block unit based on the prediction indicators. The prediction indicators may include multiple flags and multiple indexes.

[0091] At block 620 , the decoder module 124 determines a plurality of reconstructed components of the block unit by using the first prediction mode determined according to the video data.

[0092] In at least one embodiment, the block unit may include a plurality of first block components and a plurality of second block components. In the embodiment, the first block component may be a plurality of luminance block components, and the second block component may be a plurality of chrominance block components. In at least one embodiment, the prediction indication may include at least one first prediction flag to indicate how to reconstruct the first block component of the block unit. For example, the prediction indication may include a prediction mode flag to select a first prediction mode from an intra-frame prediction mode and an inter-frame prediction.

[0093] In at least one embodiment, reference Figure 2 , the decoder module 124 may determine the first prediction mode based on at least one first prediction flag in the bitstream, and reconstruct the block unit based on the first prediction mode. In at least one embodiment, when the first prediction mode is an intra-frame prediction mode, the decoder module 124 may determine a plurality of neighboring blocks adjacent to the block unit, and reconstruct a first block component of the block unit based on the neighboring blocks to determine the reconstructed component. In at least one embodiment, when the first prediction mode is an inter-frame prediction mode, the decoder module 124 may determine a plurality of reference frames reconstructed before reconstructing the image frame, and reconstruct a first block component of the block unit based on the reference frames to determine the reconstructed component. Therefore, the reconstructed component is derived for the luminance block component of the block unit.

[0094] At block 630 , the decoder module 124 selects a plurality of reference samples for the second prediction mode from a plurality of neighboring blocks adjacent to the block unit based on at least one sampling parameter.

[0095] In at least one embodiment, the prediction indication may include at least one second prediction flag for indicating how to reconstruct the second block component of the block unit. For example, the prediction indication may include a linear model flag for determining whether to predict the second block component based on the linear model mode. In at least one embodiment, the prediction indication may include a linear model index for selecting a prediction model mode from a plurality of candidate model modes when the linear model flag is equal to one.

[0096] In at least one embodiment, the at least one sampling parameter may include at least one of a start parameter, an interval parameter, and a quantity parameter. Figure 7 is a schematic diagram of a block unit and multiple reference samples according to an exemplary embodiment of the present application. In at least one embodiment, a starting parameter of the block unit 700 may be determined according to multiple starting candidates, an interval parameter of the block unit 700 may be determined according to multiple interval candidates, and a quantity parameter of the block unit 700 may be determined according to multiple quantity candidates.

[0097] In at least one embodiment, the starting parameter of the block unit 700 may be equal to a specific starting candidate among the starting candidates determined according to the bitstream. In at least one embodiment, at least one starting point of the block unit 700 may be directly set based on a specific one of the starting candidates determined for the block unit 700. Each of the starting candidates may be an integer candidate Ci. For example, when the specific integer candidate Ci is equal to the starting integer Si, the at least one starting point may include two starting points (Si, -1) and (-1, Si). Figure 7 , the starting integer Si may be equal to two, so the two starting points 712 and 722 in the first reference line 710 and the second reference line 720 are located at (2, -1) and (-1, 2). In at least one embodiment, at least one starting point of the block unit 700 may be calculated based on the block size W×H of the block unit 700 and a specific starting candidate. In the embodiment, each of the starting candidates may be a fractional candidate Cf. For example, when the specific fractional candidate Cf is equal to the starting fraction Sf, the two starting points may be located at (Sf×W, -1) and (-1, Sf×H). Reference Figure 7, the starting fraction Sf may be equal to 1 / 2. In another embodiment, each of the starting candidates may include an integer candidate Si and a fraction candidate Sf. For example, when a specific integer candidate Ci is equal to the starting integer Si and a specific fraction candidate Cf is equal to the starting fraction Sf, the two starting points may be located at (Si+Sf×W,-1) and (-1,Si+Sf×H). In at least one embodiment, at least one starting point of the block unit 700 may be calculated based on the starting base Sb of the block unit 700 and the specific starting candidate. In the embodiment, each of the starting candidates may be an integer candidate Ci. For example, when a specific integer candidate Ci is equal to the starting integer Si, the two starting points may be located at (Sb×Si,-1) and (-1,Sb×Si). In another embodiment, when a specific integer candidate Ci is equal to the starting integer Si, the two starting points may be located at (Sb+Si,-1) and (-1,Sb+Si). In at least one embodiment, at least one starting point of the block unit 700 may be calculated based on the starting base Sb, the interval base Ib, and the specific starting candidate of the block unit 700. In the embodiment, each of the starting candidates may be an integer candidate Ci. For example, when the specific integer candidate Ci is equal to the starting integer Si, the two starting points may be located at (Sb+Ib×Si,-1) and (-1,Sb+Ib×Si). In at least one embodiment, the starting base Sb and the interval base Ib may be derived as follows:

[0098] Sb=numSampN>>(2+numIs4N)

[0099] Ib=Max(1,numSampN>>(1+numIs4N))

[0100] numIs4N=(availT&&availL&&predModeIntra==INTRA_LT_CCLM)? 0:1

[0101] Wherein numSampN represents the number of available adjacent chroma samples on the upper and upper right numSampT or the number of available adjacent chroma samples on the left and lower left numSampL, availT and availL represent the availability of neighboring blocks, predModeIntra represents the intra prediction mode of the block unit, and INTRA_LT_CCLM is one of the cross-component linear model mode candidates.

[0102] In at least one embodiment, each of the interval candidates may be an integer candidate Ii. The distance between two adjacent reference samples may be set equal to the interval parameter selected from the integer candidate Ii. Figure 7, the interval parameter may be equal to four, so the two second reference samples 714 and 724 in the first reference line 710 and the second reference line 720 are located at (6, -1) and (-1, 6). In at least one embodiment, the distance between two adjacent reference samples may be calculated based on the interval bottom Ib of the block unit 700 and the interval parameter. In the embodiment, each of the interval candidates may be equal to a positive integer. For example, when the interval parameter is equal to the interval integer Ii, the distance between two adjacent reference samples may be equal to Ib×Ii. In addition, when the interval parameter is equal to the interval integer Ii, the distance between two adjacent reference samples may be equal to Ib+Ii. In at least one embodiment, the distance between two adjacent reference samples may be calculated based on the interval bottom Ib of the block unit 700, the derived starting bottom Sd, and the interval parameter. In the embodiment, each of the interval candidates may be equal to a positive integer. For example, when the interval parameter is equal to the interval integer Ii, the distance between two adjacent reference samples may be equal to Ib+Sd×Ii. In at least one embodiment, the quantity candidate may include a plurality of upper quantity candidates and a plurality of left quantity candidates. The number of reference samples above the block unit 700 may be equal to the number parameter selected from the upper number candidate, and the number of reference samples on the left side of the block unit 700 may be equal to the number parameter selected from the left number candidate. Figure 7 , the number parameter may be equal to two to indicate that the number of reference samples above the block unit 700 is equal to two and the number of reference samples to the left of the block unit 700 is equal to two. Therefore, there are only four reference samples 712, 714, 722 and 724 for the block unit 700.

[0103] In at least one embodiment, at least one sampling parameter may be a starting parameter. In one embodiment, the starting parameter may be fixed in a plurality of candidate model modes. Therefore, when the decoder module 124 selects a prediction model mode from the candidate model modes, the starting parameter may be determined according to the prediction model mode. In other embodiments, the starting parameter may be determined based on a starting index determined according to the bitstream. Therefore, the decoder module 124 determines the starting parameter based on the value of the starting index.

[0104] In at least one embodiment, at least one sampling parameter may include a starting parameter, an interval parameter, and a quantity parameter. In at least one embodiment, the starting parameter, the interval parameter, and the quantity parameter may be fixed in a plurality of candidate model patterns. For example, the first candidate model pattern may include a starting candidate S1, a first interval candidate I1, and a first quantity candidate Q1, and the Nth candidate model pattern may include an Nth starting candidate Sn, an Nth interval candidate In, and an Nth quantity candidate Qn. In one embodiment, the starting candidates S1-Sn may be equal to or different from each other, the interval candidates I1-In may be equal to or different from each other, and the quantity candidates Q1-Qn may be equal to or different from each other. For example, when the number of candidate model patterns is equal to four, the starting candidates S1-S2 may be equal to each other, the starting candidates S3-S4 may be equal to each other and different from the starting candidates S1-S2, each of the interval candidates I1-I4 may be equal to each other, the quantity candidates Q1 and Q3 may be equal to each other, and the quantity candidates Q2 and Q4 may be equal to each other and different from the quantity candidates Q1 and Q3.

[0105] At block 640 , the decoder module 124 derives a plurality of model parameters for the second prediction mode based on the reference samples.

[0106] In at least one embodiment, reference Figure 2 and Figure 7 , the decoder module 124 may determine a plurality of first reference components and a plurality of second reference components in the reference sample. In at least one embodiment, the first reference component may be a plurality of luma reference components, and the second reference component may be a plurality of chroma reference components. In at least one embodiment, the decoder module 124 may downsample the first reference component to generate a plurality of downsampled reference components. In at least one embodiment, each of the downsampled reference components is generated based on N of the first reference components. In one embodiment, the number N may be equal to 2×K, and the number K may be an integer. For example, the number K may be equal to two and the number N may be equal to four.

[0107] In at least one embodiment, the decoder module 124 may derive model parameters of the model equation based on the reference sample. In one embodiment, the decoder module 124 may derive the model equation based on the first reference component and the second reference component. In other embodiments, the decoder module 124 may derive the model equation based on the downsampled reference component and the second reference component. In at least one embodiment, the model equation of the block unit may be derived as follows:

[0108] Pred c [x,y]=α×Rec′ L [x,y]+β

[0109] where pred C(i, j) represents multiple prediction components in the block unit, and rec L (i, j) denotes multiple downsampled reconstructed components of the same block unit. In at least one embodiment, the model parameters α and β are derived based on reference samples.

[0110] In at least one embodiment, the decoder module 124 may select two smaller values ​​Xa0 and Xa1 and two larger values ​​Xb0 and Xb1 of the downsampled reference components, and determine four second reference components Ya0, Ya1, Yb0, and Yb1 corresponding to the four selected downsampled reference components. Next, the decoder module 124 determines a first average value XA of Xa0 and Xa1, a second average value XB of Xb0 and Xb1, a third average value YA of Ya0 and Ya1, and a fourth average value YB of Yb0 and Yb1. The model parameter α may be equal to (Ya-Yb) / (Xa-Xb), and the model parameter β may be equal to Yb-α×Xb. In at least one embodiment, the model parameters α and β may be derived based on the reference sample by any other derivation method.

[0111] At block 650 , the decoder module 124 generates a plurality of prediction components for the block unit based on the reconstructed components according to the second prediction mode.

[0112] In at least one embodiment, reference Figure 2 , the decoder module 124 may downsample the reconstruction components to generate downsampled reconstruction components. In at least one embodiment, each of the downsampled reconstruction components is generated based on M of the first reference components. In one embodiment, the number M may be equal to 2×L, and the number L may be an integer. For example, the number L may be equal to two and the number M may be equal to four. In at least one embodiment, the number K may be equal to the number L, and the number N may be equal to the number M.

[0113] In at least one embodiment, the decoder module 124 may substitute the downsampled reconstruction component into the model equation to generate the prediction component. Therefore, the prediction component may be generated for the chrominance block component of the block unit. The chrominance block component of the block unit may be reconstructed by adding the prediction component to a plurality of residual components of the block unit determined according to the bitstream. In at least one embodiment, the residual component of the block unit may be a plurality of chrominance residual components. In at least one embodiment, the decoder module 124 may reconstruct all other block units in the image frame for reconstructing the image frame and the video.

[0114] Figure 8 Show Figure 1The encoder module 114 in. The encoder module 114 may include a prediction processor (e.g., a prediction processing unit 8141), at least a first summer (e.g., a first summer 8142) and a second summer (e.g., a second summer 8145), a transform / quantization processor (e.g., a transform / quantization unit 8143), an inverse quantization / inverse transform processor (e.g., an inverse quantization / inverse transform unit 8144), a filter (e.g., a filtering unit 8146), a decoded picture buffer (e.g., a decoded picture buffer 8147), and an entropy encoder (e.g., an entropy encoding unit 8148). The prediction processing unit 8141 of the encoder module 114 may also include a partition processor (e.g., a partition unit 81411), an intra-frame prediction processor (e.g., an intra-frame prediction unit 81412), and an inter-frame prediction processor (e.g., an inter-frame prediction unit 81413). The encoder module 114 may receive a source video and encode the source video to output a bitstream.

[0115] The encoder module 114 may receive a source video including a plurality of image frames and then divide the image frames according to the encoding structure. Each of the image frames may be divided into at least one image block.

[0116] At least one image block may include a luma block having a plurality of luma samples and at least one chroma block having a plurality of chroma samples. The luma block and at least one chroma block may be further divided to generate a macroblock, a coding tree unit (CTU), a coding block (CB), a subdivision thereof, and / or another equivalent coding unit.

[0117] In at least one embodiment, encoder module 114 may perform additional subdivision of the source video.It should be noted that the embodiments are generally applicable to video encoding regardless of how the source video is partitioned prior to and / or during encoding.

[0118] In at least one embodiment, during the encoding process, the prediction processing unit 8141 may receive a current image block of a specific one of the image frames during the encoding process. The current image block may be one of a luminance block or a chrominance block in the specific image frame.

[0119] The partition unit 81411 may divide the current image block into a plurality of block units. The intra prediction unit 81412 may perform intra prediction encoding on the current block unit relative to one or more neighboring blocks in the same frame as the current block unit to provide spatial prediction. The inter prediction unit 81413 may perform inter prediction encoding on the current block unit relative to one or more blocks in one or more reference image blocks to provide temporal prediction.

[0120] In at least one embodiment, the prediction processing unit 8141 may select one of the encoding results generated by the intra prediction unit 81412 and the inter prediction unit 81413 based on a mode selection method such as a cost function. The mode selection method may be a rate-distortion optimization (RDO) process.

[0121] The prediction processing unit 8141 may determine the selected encoding result and provide the prediction block corresponding to the selected encoding result to the first summer 8142 for generating a residual block and to the second summer 8145 for reconstructing the encoded block unit. The prediction processing unit 8141 may further provide syntax elements (such as motion vectors, intra-mode indicators, partition information, and other syntax information) to the entropy encoding unit 8148.

[0122] In at least one embodiment, the intra prediction unit 81412 may perform intra prediction on the current block unit. The intra prediction unit 81412 may determine an intra prediction mode for reconstructed samples adjacent to the current block unit in order to encode the current block unit.

[0123] In at least one embodiment, the intra prediction unit 81412 may use various intra prediction modes to encode the current block unit, and the intra prediction unit 81412 or the prediction processing unit 8141 may select an appropriate intra prediction mode from the selected modes. The intra prediction unit 81412 may use a cross-component prediction mode to encode the current block unit to predict one of the two chroma components of the current block unit based on the luminance component of the current block unit. The intra prediction unit 81412 may predict a first chroma component of the two chroma components of the current block unit based on a second chroma component of the two chroma components of the current block unit.

[0124] In at least one embodiment, the inter-prediction unit 81413 may perform inter-prediction on the current block unit instead of the intra-prediction performed by the intra-prediction unit 81412. The inter-prediction unit 81413 may perform motion estimation to estimate the motion of the current block unit for generating a motion vector.

[0125] The motion vector may indicate the displacement of the current block unit within the current image block relative to the reference block unit within the reference image block. The inter-frame prediction unit 81413 may receive at least one reference image block stored in the decoded picture buffer 8147 and estimate the motion based on the received reference image block to generate a motion vector.

[0126] In at least one embodiment, the first summer 8142 may generate a residual block by subtracting the prediction block determined by the prediction processing unit 8141 from the original current block unit. The first summer 8142 may represent one or more components that perform this subtraction operation.

[0127] In at least one embodiment, the transform / quantization unit 8143 may apply a transform to the residual block to generate residual transform coefficients and then quantize the residual transform coefficients to further reduce the bit rate. The transform may be one of a DCT, DST, AMT, MDNSST, HyGT, a signal-dependent transform, a KLT, a wavelet transform, an integer transform, a subband transform, or a conceptually similar transform.

[0128] In at least one embodiment, the transform may convert the residual information from a pixel value domain to a transform domain such as a frequency domain.The degree of quantization may be modified by adjusting a quantization parameter.

[0129] In at least one embodiment, the transform / quantization unit 8143 may perform a scan of the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 8148 may perform the scan.

[0130] In at least one embodiment, the entropy coding unit 8148 may receive a plurality of syntax elements, including quantization parameters, transform data, motion vectors, intra modes, partition information, and other syntax information, from the prediction processing unit 8141 and the transform / quantization unit 8143. The entropy coding unit 8148 may encode the syntax elements into a bitstream.

[0131] In at least one embodiment, the entropy coding unit 8148 may entropy encode the quantized transform coefficients by CAVLC, CABAC, SBAC, PIPE encoding, or another entropy coding technique to generate an encoded bitstream. The encoded bitstream may be transmitted to another device (ie, Figure 1 to a second electronic device 120) or archived for later transmission or retrieval.

[0132] In at least one embodiment, the inverse quantization / inverse transform unit 8144 may apply inverse quantization and inverse transform to reconstruct the residual block in the pixel domain for later use as a reference block. The second summer 8145 may add the reconstructed residual block to the prediction block provided from the prediction processing unit 8141 to generate a reconstructed block for storage in the decoded picture buffer 8147.

[0133] In at least one embodiment, the filtering unit 8146 may include a deblocking filter, a SAO filter, a bilateral filter, and / or an ALF to remove block artifacts from the reconstructed block. In addition to the deblocking filter, the SAO filter, the bilateral filter, and the ALF, additional filters (in the loop or after the loop) may be used. Such filters are not shown for simplicity and may filter the output of the second summer 8145.

[0134] In at least one embodiment, the decoded picture buffer 8147 may be a reference picture memory that stores reference blocks used by the encoder module 114 to encode video, such as in intra-frame or inter-frame coding modes. The decoded picture buffer 8147 may include a variety of memory devices, such as: DRAM, including SDRAM, MRAM, RRAM; or other types of memory devices. The decoded picture buffer 8147 may be on-chip with other components of the encoder module 114 or off-chip relative to those components.

[0135] In at least one embodiment, the encoder module 114 may perform the following Figure 3 An exemplary reconstruction method 300 for reconstructing a block unit is shown. The method 300 may be used Figure 1 and Figure 8 The method 300 is performed with reference to the configuration shown, and various elements of these figures are referenced when explaining the method 300 . Figure 3 Each frame shown may represent one or more processes, methods or subroutines performed. In addition, the order of frames is illustrative only and may be changed. Additional frames may be added or fewer frames may be utilized without departing from the present disclosure.

[0136] At block 310, the encoder module 114 receives an image frame of video data and determines a block unit from the image frame based on the received video data. The video data may be a video.

[0137] In at least one embodiment, reference Figure 1 and Figure 8 , the first electronic device 110 may receive the video via the source module 112. The encoder module 114 may determine an image frame according to the video and divide the image frame to determine a block unit.

[0138] In at least one embodiment, the prediction processing unit 8141 of the first electronic device 110 may determine the block unit according to the video via the partition unit 81411 , and the encoder module 114 may provide multiple partition indications into the bitstream based on the partition result of the partition unit 81411 .

[0139] At block 320 , the encoder module 114 determines the first motion information and the second motion information from the candidate list and determines a partition line of the block unit to reconstruct the block unit.

[0140] In at least one embodiment, reference Figure 8 , the encoder module 114 may generate a plurality of encoding results generated by the intra prediction unit 81412 and the inter prediction unit 81413. Next, the encoder module 114 may select one of the encoding results based on a mode selection method such as a cost function. The mode selection method may be a rate distortion optimization (RDO) process. In at least one embodiment, when the selected encoding result is generated based on a partition line, first motion information, and second motion information, the encoder module 114 may divide the block unit into a plurality of prediction areas, and predict the prediction area of ​​the block unit based on the first motion information and the second motion information.

[0141] In at least one embodiment, when the partition line is generated based on a triangular partition mode, the partition line may be one of a diagonal partition line and an anti-diagonal direction. In at least one embodiment, when the partition line is generated based on a geometric partition mode, the partition line may be determined by the tilt angle of the partition line and the line offset between the partition line and the center point of the block unit. Therefore, the encoder module 114 may provide a partition index of the partition line into the bitstream, the partition index indicating the tilt angle and the line offset.

[0142] In at least one embodiment, the first motion information and the second motion information may be included in a merge candidate list of a block unit. In at least one embodiment, the first motion information may indicate a first reference frame and a first reference vector, and the second motion information may indicate a second reference frame and a second reference vector. The first reference frame and the second reference frame may be selected from a plurality of reference lists of a block unit.

[0143] In at least one embodiment, reference Figure 8 , the first summer 8142 of the encoder module 114 in the first electronic device 110 may generate a residual block based on the selected encoding result, and provide a bitstream including a plurality of coefficients to the second electronic device 120, wherein the plurality of coefficients correspond to a plurality of residual components in the residual block.

[0144] In at least one embodiment, the encoder module 114 may reconstruct the block unit based on the selected encoding result again for predicting other block units in the image frame. The block unit may be reconstructed by adding the selected encoding result to the residual block of the sub-block.

[0145] At block 330 , the encoder module 114 determines sub-blocks in the block unit.

[0146] In at least one embodiment, reference Figure 5A and Figure 8 , the encoder module 114 may divide the block unit 400 to generate a plurality of sub-blocks 561-576.

[0147] At block 340, the encoder module 114 determines whether the sub-block is included in a specific one of the plurality of block regions that covers the partition line. In at least one embodiment, when the encoder module 114 determines that the sub-block is included in the specific one of the block regions that covers the partition line, the method 300 may proceed to block 350. When the encoder module 114 determines that the sub-block is not included in the specific one of the block regions, the method may end.

[0148] In at least one embodiment, reference FIG. 5B to FIG. 5C and Figure 8 , the encoder module 114 may divide the block unit to generate a plurality of block regions 581-583. In at least one embodiment, the encoder module 114 may classify the sub-blocks 561-576 into the block regions 581-583. In at least one embodiment, the third block region 583 may cover the partition line 420, and the first block region 581 and the second block region may not cover the partition line 420. Most of the block components of the sub-blocks in the first block region 581 may be predicted based only on the first motion information, and most of the block components of the sub-blocks in the second block region 582 may be predicted based only on the second motion information. Therefore, when the sub-block is included in the first block region 581 corresponding to the first prediction area 441, the encoder module 114 may store the first motion information for the sub-block. In addition, when the sub-block is included in the second block region 582 corresponding to the second prediction area 442, the encoder module 114 may store the second motion information for the sub-block.

[0149] In at least one embodiment, most of the block components of the sub-block in the third block area 583 may be predicted based on the first motion information and the second motion information. Therefore, when the sub-block is included in the third block area 583 corresponding to the third prediction area 443, the method 300 may proceed to block 350 so that the encoder module 114 determines how to store at least one of the first motion information and the second motion information for the sub-block.

[0150] At block 350, the encoder module 114 determines whether the first list tag of the first motion information is the same as the second list tag of the second motion information. In at least one embodiment, when the first list tag is the same as the second list tag, the method 300 may proceed to block 360. When the first list tag is different from the second list tag, the method 300 may proceed to block 370.

[0151] In at least one embodiment, reference FIG. 5B to FIG. 5C and Figure 8, when the sub-block is included in the specific one block area 583, the encoder module 114 may compare the first list flag with the second list flag. When the first list flag is different from the second list flag, the block components in the specific one block area 583 may be predicted based on the reference frames in the two reference lists. In addition, when the first list flag is the same as the second list flag, the block components in the specific one block area 583 may be predicted based on the reference frames in the same reference list.

[0152] At block 360 , when the first list flag is the same as the second list flag, the encoder module 114 stores a predefined one of the first motion information and the second motion information for the sub-block.

[0153] In at least one embodiment, reference FIG. 5B to FIG. 5C and Figure 8 , when the first list mark of the block unit 400 is the same as the second list mark of the block unit 400 and the subblock is included in a specific block area 583, most block components in the subblock can be predicted by deriving the first reference sample and the second reference sample according to one of the reference lists.

[0154] Since the motion information is stored so as to predict a plurality of subsequent blocks based on the stored motion information, the stored motion information may not affect a plurality of residual components of a block unit generated based on a mixed result of the first reference sample and the second reference sample. Figure 1 , when the encoder module 114 and the decoder module 124 perform the method 300 to store the predefined one of the first motion information and the second motion information, the encoder module 114 and the decoder module 124 may store the same motion information for the sub-block. In at least one embodiment, when the first reference frame and the second reference frame are included in the same reference list, the encoder module 114 may directly store the predefined one of the first motion information and the second motion information to reduce the use of the buffer. Therefore, the encoder module 114 stores the predefined one of the first motion information and the second motion information for the sub-block.

[0155] In at least one embodiment, the predefined one of the first motion information and the second motion information may be the second motion information. Therefore, the encoder module 114 directly stores the second motion information for the sub-block. In at least one embodiment, the bitstream may include a first candidate index indicating the first motion information and a second candidate index indicating the second motion information. Therefore, the decoder module 124 may determine the predefined one of the first motion information and the second motion information according to the second candidate index of the block unit in the bitstream.

[0156] In block 370 , the encoder module 114 stores both the first motion information and the second motion information for the sub-block.

[0157] In at least one embodiment, reference FIG. 5B to FIG. 5C and Figure 8 , when the first list flag of the block unit 400 is different from the second list flag of the block unit 400 and the sub-block is included in a specific one block area 583, most of the block components in the sub-block can be predicted by deriving the first reference sample and the second reference sample according to different reference lists. Therefore, the encoder module 114 directly stores the first motion information and the second motion information for the sub-block without further determination.

[0158] In at least one embodiment, the encoder module 114 may directly store a predefined one of the first motion information and the second motion information for a sub-block having the same reference list and directly store both the first motion information and the second motion information for a sub-block having a different reference list. Therefore, when a sub-block is included in a specific one block area 583, the encoder module 114 may determine whether to store the first motion information and the second motion information together as the stored motion information for the sub-block based only on the comparison between the first list flag and the second list flag.

[0159] At block 380 , when the subsequent block is predicted based on the sub-block, the encoder module 114 reconstructs the subsequent block based on the stored motion information.

[0160] In at least one embodiment, reference Figure 8 , the encoder module 114 may maintain the predicted image frame and multiple subsequent frames after the block unit is reconstructed. Therefore, the stored information of the block unit can be used to predict subsequent blocks in the image frame and subsequent frames. In at least one embodiment, the subsequent frame may include multiple unpredicted frames predicted after the reconstruction of the image frame. In addition, the subsequent block may include multiple unpredicted blocks in the image frame and subsequent frames. The unpredicted block may be predicted after the reconstruction of the block unit.

[0161] In at least one embodiment, the encoder module 114 may perform the following Figure 6 An exemplary reconstruction method 600 for reconstructing a block unit is shown. The method 600 may be used Figure 1 and Figure 8 The method 600 is performed with reference to the configuration shown, and various elements of these figures are referenced when explaining the method 600 . Figure 6 Each frame shown may represent one or more processes, methods or subroutines performed. In addition, the order of frames is illustrative only and may be changed. Additional frames may be added or fewer frames may be utilized without departing from the present disclosure.

[0162] At block 610, the encoder module 114 receives an image frame of video data and determines a block unit from the image frame based on the received video data. The video data may be a video.

[0163] In at least one embodiment, reference Figure 1 and Figure 8 , the first electronic device 110 may receive the video via the source module 112. The encoder module 114 may determine an image frame according to the video and divide the image frame to determine a block unit.

[0164] In at least one embodiment, the prediction processing unit 8141 of the first electronic device 110 may determine the block unit according to the video via the partition unit 81411 , and the encoder module 114 may provide multiple partition indications into the bitstream based on the partition result of the partition unit 81411 .

[0165] At block 620 , the encoder module 114 determines a plurality of reconstructed components of the block unit by using a first prediction mode determined according to the video data.

[0166] In at least one embodiment, the block unit may include a plurality of first block components and a plurality of second block components. The first block components may be a plurality of luminance block components, and the second block components may be a plurality of chrominance block components. Figure 8 , the encoder module 114 may generate a plurality of encoding results generated by the intra prediction unit 81412 and the inter prediction unit 81413. Next, the encoder module 114 may select one of the encoding results based on a mode selection method such as a cost function. The mode selection method may be a rate-distortion optimization (RDO) process. The luminance prediction mode for the selected encoding result may be set to a first prediction mode. In at least one embodiment, the first summer 8142 may generate a residual block by subtracting the encoding result from the first block component. In at least one embodiment, the residual block may be transformed and quantized by the transform / quantization unit 8143. In at least one embodiment, the inverse quantization / inverse transform unit 8144 may apply inverse quantization and inverse transform to reconstruct the residual block in the pixel domain for later use as a reference block. The second summer 8145 may add the reconstructed residual block to the encoding result provided from the prediction processing unit 8141 so as to generate a reconstructed component for storage in the decoded picture buffer 8147. In at least one embodiment, the reconstructed component is derived for the luminance block component of the block unit.

[0167] At block 630 , the encoder module 114 selects a plurality of reference samples from a plurality of neighboring blocks adjacent to the block unit for the second prediction mode based on the at least one sampling parameter.

[0168] In at least one embodiment, reference Figure 8, the encoder module 114 may select one of the encoding results based on the mode selection method for the second block component. The chrominance prediction mode for the selected encoding result may be set to the second prediction mode. In at least one embodiment, the second prediction mode may be selected from a plurality of candidate model modes.

[0169] In at least one embodiment, at least one sampling parameter may include at least one of a start parameter, an interval parameter, and a quantity parameter. In at least one embodiment, the start parameter of the block unit 700 may be determined according to a plurality of start candidates, the interval parameter of the block unit 700 may be determined according to a plurality of interval candidates, and the quantity parameter of the block unit 700 may be determined according to a plurality of quantity candidates.

[0170] In at least one embodiment, the starting parameter of the block unit 700 may be equal to a specific one of the starting candidates determined according to the bitstream. In one embodiment, at least one starting point of the block unit 700 may be directly set based on the specific starting candidate determined for the block unit 700. In the embodiment, each of the starting candidates may be an integer candidate Ci. In one embodiment, at least one starting point of the block unit 700 may be calculated based on the block size W×H of the block unit 700 and the specific starting candidate. In the embodiment, each of the starting candidates may be a fractional candidate Cf, or include an integer candidate Ci and a fractional candidate Cf. In one embodiment, at least one starting point of the block unit 700 may be calculated based on the starting base Sb of the block unit 700 and the specific starting candidate by adding the starting base Sb to the specific starting candidate or by multiplying the starting base Sb by the specific starting candidate. In the embodiment, each of the starting candidates may be an integer candidate Ci. In one embodiment, at least one starting point of the block unit 700 may be calculated based on the starting base Sb of the block unit 700, the interval base Ib and the specific starting candidate. In the embodiment, each of the starting candidates may be an integer candidate Ci. In at least one embodiment, each of the interval candidates may be an integer candidate Ii. The distance between two adjacent reference samples may be set equal to the interval parameter selected from the integer candidate Ii. In one embodiment, the distance between two adjacent reference samples may be calculated based on the interval bottom Ib and the interval parameter of the block unit 700 by adding the interval bottom Ib to the interval parameter or by multiplying the interval bottom Ib by the interval parameter. In one embodiment, the distance between two adjacent reference samples may be calculated based on the interval bottom Ib of the block unit 700, the derived starting bottom Sd and the interval parameter. In at least one embodiment, the quantity candidate may include a plurality of upper quantity candidates and a plurality of left quantity candidates. The number of reference samples above the block unit 700 may be equal to the quantity parameter selected from the upper quantity candidate, and the number of reference samples on the left side of the block unit 700 may be equal to the quantity parameter selected from the left quantity candidate.

[0171] In at least one embodiment, at least one sampling parameter may be a starting parameter. In one embodiment, the starting parameter may be fixed in a plurality of candidate model modes.

[0172] In at least one embodiment, the at least one sampling parameter may include a start parameter, an interval parameter, and a quantity parameter. In at least one embodiment, the start parameter, the interval parameter, and the quantity parameter may be fixed in each of the candidate model modes.

[0173] At block 640 , the encoder module 114 derives a plurality of model parameters for the second prediction mode based on the reference samples.

[0174] In at least one embodiment, reference Figure 8 , the encoder module 114 may determine a plurality of first reference components and a plurality of second reference components in the reference sample. In at least one embodiment, the first reference component may be a plurality of luma reference components, and the second reference component may be a plurality of chroma reference components. In at least one embodiment, the encoder module 114 may downsample the first reference component to generate a plurality of downsampled reference components.

[0175] In at least one embodiment, the encoder module 114 may derive model parameters of the model equation based on the reference sample. In at least one embodiment, the model equation of the block unit may be derived as follows:

[0176] Pred c [x,y]=α×Rec′ L [x,y]+β

[0177] where pred C (i, j) represents multiple prediction components in the block unit, and rec L (i, j) denotes multiple downsampled reconstructed components of the same block unit. In at least one embodiment, the model parameters α and β are derived based on reference samples.

[0178] At block 650 , the encoder module 114 generates a plurality of prediction components for the block unit based on the reconstructed components according to the second prediction mode.

[0179] In at least one embodiment, reference Figure 8, the encoder module 114 may downsample the reconstruction component to generate a downsampled reconstruction component. In at least one embodiment, the encoder module 114 may substitute the downsampled reconstruction component into the model equation to generate a prediction component. Therefore, a prediction component may be generated for the chroma block component of the block unit. In at least one embodiment, a chroma residual block may be generated in a first summer 8142 by subtracting the prediction component from the original second block component in the video and the chroma residual block may be added back to the prediction component in a second summer 8145 to generate a plurality of chroma reconstruction components. In at least one embodiment, the encoder module 114 may predict all other block units in the image frame and reconstruct the encoded block unit for predicting the video.

[0180] The disclosed embodiments are to be considered in all respects as illustrative and not restrictive. It should also be understood that the present disclosure is not limited to the particular embodiments described above, but that many rearrangements, modifications and substitutions are possible without departing from the scope of the present disclosure.

Claims

1. A method for decoding a bit stream by an electronic device, the method comprising: Receiving an image frame of the bit stream; determining a block unit from a received image frame; receiving first motion information and second motion information from a candidate list to reconstruct the block unit, wherein the first motion information includes a first list flag for selecting a first reference frame, and the second motion information includes a second list flag for selecting a second reference frame; Determining a sub-block in the block unit; comparing the first list marker to the second list marker; When the first list flag is the same as the second list flag, storing a predefined one of the first motion information and the second motion information as the stored motion information for the subblock without checking whether the first reference frame and the second reference frame are included in a specific one of a plurality of reference lists indicated by a specific one of a plurality of flag values ​​different from the first list flag and the second list flag; as well as When a subsequent block is reconstructed based on the sub-block, the subsequent block is reconstructed based on the stored motion information.

2. The method according to claim 1, further comprising: Determining a dividing line of the block unit; as well as determining whether the sub-block is included in a specific one of the plurality of block areas separated based on the partition line, wherein the specific one of the plurality of block areas covers the dividing line, and wherein when the sub-block is included in the specific one of the plurality of block areas, the first list mark is compared with the second list mark, When the first list flag is identical to the second list flag and the subblock is included in the specific one of the plurality of block regions, storing the predefined one of the first motion information and the second motion information as the stored motion information for the subblock without checking whether the first reference frame and the second reference frame are included in the specific one of the plurality of reference lists indicated by the specific one of the plurality of flag values ​​different from the first list flag and the second list flag; as well as When the first list flag is different from the second list flag and the subblock is included in the specific one of the plurality of block regions, the first motion information and the second motion information are stored together as stored motion information for the subblock.

3. The method according to claim 2, characterized in that Also includes: When the subblock is included in a first block region corresponding to the first motion information among the plurality of block regions, storing the first motion information as stored motion information for the subblock; as well as when the subblock is included in a second block region corresponding to the second motion information among the plurality of block regions, storing the second motion information as stored motion information for the subblock, Each of the first block area among the plurality of block areas and the second block area among the plurality of block areas is different from the specific one of the plurality of block areas.

4. The method according to claim 3, characterized in that The predefined one of the first motion information and the second motion information is the second motion information.

5. The method according to claim 1, characterized in that When the number of the multiple reference lists is equal to two and the first list tag is the same as the second list tag, the specific one of the multiple reference lists indicated by the specific one of the multiple tag values ​​is different from another reference list indicated by the first list tag in the multiple reference lists.

6. An electronic device for decoding a bit stream, the electronic device comprising: at least one processor; as well as a storage device coupled to the at least one processor and storing a plurality of instructions that, when executed by the at least one processor, cause the at least one processor to: Receiving an image frame of the bit stream; determining a block unit from a received image frame; receiving first motion information and second motion information from a candidate list to reconstruct the block unit, wherein the first motion information includes a first list flag for selecting a first reference frame, and the second motion information includes a second list flag for selecting a second reference frame; Determining a sub-block in the block unit; comparing the first list marker to the second list marker; when the first list flag is the same as the second list flag, storing a predefined one of the first motion information and the second motion information as the stored motion information for the subblock without checking whether the first reference frame and the second reference frame are included in a specific one of a plurality of reference lists indicated by a specific one of a plurality of flag values ​​different from the first list flag and the second list flag; and When a subsequent block is reconstructed based on the sub-block, the subsequent block is reconstructed based on the stored motion information.

7. The electronic device according to claim 6, characterized in that: The plurality of instructions, when executed by the at least one processor, further cause the at least one processor to: Determining a dividing line of the block unit; and determining whether the sub-block is included in a specific one of the plurality of block areas separated based on the partition line, wherein the specific one of the plurality of block areas covers the dividing line, and Wherein when the sub-block is included in the specific one of the plurality of block areas, the first list mark is compared with the second list mark.

8. The electronic device according to claim 7, characterized in that: The plurality of instructions, when executed by the at least one processor, further cause the at least one processor to: When the subblock is included in a first block region corresponding to the first motion information among the plurality of block regions, storing the first motion information as stored motion information for the subblock; and when the subblock is included in a second block region corresponding to the second motion information among the plurality of block regions, storing the second motion information as stored motion information for the subblock, wherein each of the first block area among the plurality of block areas and the second block area among the plurality of block areas is different from the specific one of the plurality of block areas, When the first list flag is identical to the second list flag and the subblock is included in the specific one of the plurality of block regions, storing the predefined one of the first motion information and the second motion information as the stored motion information for the subblock without checking whether the first reference frame and the second reference frame are included in the specific one of the plurality of reference lists indicated by the specific one of the plurality of flag values ​​different from the first list flag and the second list flag; and When the first list flag is different from the second list flag and the subblock is included in the specific one of the plurality of block regions, the first motion information and the second motion information are stored together as stored motion information for the subblock.

9. The electronic device according to claim 8, characterized in that: The predefined one of the first motion information and the second motion information is the second motion information.

Images