Intra prediction method and apparatus, and electronic device
Patent Information
- Application Number
- PCT/CN2026/080920
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-03-10
- Filing Date
- 2026-03-03
- Publication Date
- 2026-09-17
Smart Images
Figure CN2026080920_17092026_PF_FP_ABST
Abstract
Description
Intra-frame prediction methods, apparatus and electronic equipment
[0001] Cross-references to related applications
[0002] This application claims priority to Chinese Patent Application No. 202510278556.5, filed in China on March 10, 2025, the entire contents of which are incorporated herein by reference. Technical Field
[0003] This application belongs to the field of video encoding and decoding technology, specifically relating to an intra-frame prediction method, apparatus, and electronic device. Background Technology
[0004] The video encoder employs a block-based hybrid coding framework. The coding process includes: block partitioning, intra-frame prediction, inter-frame prediction, transform, quantization, loop filtering, and entropy coding. Intra-frame prediction includes extrapolation filter-based intra-prediction mode (EIP).
[0005] EIP requires the use of extrapolation filters for intra-frame prediction, and the shape of the extrapolation filter in related techniques is shown in Figure 1a. When using the extrapolation filter shown in Figure 1a, the reconstructed or predicted sample points at the upper right and lower left of the current test sample point are used. If the test sample point is located at the right or lower boundary of the current coding unit, as shown in the three positions in Figure 1b, the sample at the diagonal fill position is unavailable. In this case, the value of the sample point at the diagonal fill position can only be replaced by the value of a sample point at a non-true position, such as the value of the sample point at the gray position obtained by using the left or upper edge of that position. This will lead to inaccurate prediction values for the test sample point and reduce coding accuracy. Summary of the Invention
[0006] This application provides an intra-frame prediction method, apparatus, and electronic device that can solve the problem of inaccurate prediction values of sample points under test in EIP mode.
[0007] Firstly, an intra-frame prediction method is provided, executed by either the encoder or decoder, the method comprising:
[0008] Obtain the position of the test sample point in the current coding unit;
[0009] The target filter is determined based on the location of the sample point to be tested;
[0010] Based on the target filter, intra-frame prediction is performed on the test sample points to obtain the predicted values of the test sample points;
[0011] Wherein, when the position of the sample point to be tested is located at the first position of the current coding unit, the target filter is the first filter;
[0012] The first position is located in the vicinity of at least one of the lower boundary and the right boundary of the current coding unit. The first filter includes N first inputs and one first output, where N is a positive integer. The N first inputs correspond one-to-one with the N first sample points. The first output corresponds to the sample point to be tested. The positions of the N first sample points include at least one of the upper and left sides of the sample point to be tested, but do not include at least one of the right and lower sides of the sample point to be tested.
[0013] Secondly, an intra-frame prediction apparatus is provided, comprising:
[0014] The acquisition module is used to acquire the position of the test sample point in the current coding unit;
[0015] The determination module is used to determine the target filter based on the position of the sample point to be tested;
[0016] The prediction module is used to perform intra-frame prediction on the test sample points based on the target filter to obtain the predicted values of the test sample points.
[0017] Wherein, when the position of the sample point to be tested is located at the first position of the current coding unit, the target filter is the first filter;
[0018] The first position is located in the vicinity of at least one of the lower boundary and the right boundary of the current coding unit. The first filter includes N first inputs and one first output, where N is a positive integer. The N first inputs correspond one-to-one with the N first sample points. The first output corresponds to the sample point to be tested. The positions of the N first sample points include at least one of the upper and left sides of the sample point to be tested, but do not include at least one of the right and lower sides of the sample point to be tested.
[0019] Thirdly, an intra-frame prediction apparatus is provided, the apparatus being configured to perform the steps of the method described in the first aspect.
[0020] Fourthly, an electronic device is provided, the terminal including a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the method as described in the first aspect.
[0021] Fifthly, an electronic device is provided, including a processor and a communication interface, wherein the processor is used to obtain the position of the sample point to be tested in the current encoding unit;
[0022] The target filter is determined based on the location of the sample point to be tested;
[0023] Based on the target filter, intra-frame prediction is performed on the test sample points to obtain the predicted values of the test sample points;
[0024] Wherein, when the position of the sample point to be tested is located at the first position of the current coding unit, the target filter is the first filter;
[0025] The first position is located in the vicinity of at least one of the lower boundary and the right boundary of the current coding unit. The first filter includes N first inputs and one first output, where N is a positive integer. The N first inputs correspond one-to-one with the N first sample points. The first output corresponds to the sample point to be tested. The positions of the N first sample points include at least one of the upper and left sides of the sample point to be tested, but do not include at least one of the right and lower sides of the sample point to be tested.
[0026] A sixth aspect provides an electronic device comprising: a memory configured to store video data, and processing circuitry configured to implement the steps of the method described in the first aspect.
[0027] In a seventh aspect, a readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.
[0028] Eighthly, a coding / decoding system is provided, comprising: an encoding end device and a decoding end device, wherein the encoding end device is configured to perform the steps of the method described in the first aspect, and the decoding end device is configured to perform the steps of the method described in the first aspect.
[0029] In a ninth aspect, a chip is provided, the chip including a processor and a communication interface coupled to the processor, the processor being configured to run a program or instructions to implement the steps of the method described in the first aspect.
[0030] In a tenth aspect, a computer program / program product is provided, the computer program / program product being stored in a storage medium, the computer program / program product being executed by at least one processor to perform the steps of the method as described in the first aspect.
[0031] Eleventhly, an electronic device is provided for performing the steps of the method as described in the first aspect.
[0032] In this embodiment, when the position of the sample point to be tested is located at the first position of the current coding unit, the filter corresponding to the sample point to be tested is determined as the first filter. Intra-frame prediction is then performed on the sample point to be tested based on the first filter to obtain the predicted value of the sample point. Specifically, the N first inputs of the first filter correspond one-to-one with the N first sample points. The positions of the N first sample points are located above and / or to the left of the sample point to be tested. Therefore, the N first sample points corresponding to the first filter will not exceed the lower and right boundaries of the current coding unit. That is, the N first sample points are either sample points within the current coding unit that have already obtained predicted values or sample points outside the current coding unit that have already been decoded. Thus, when the sample point to be tested is located in the region adjacent to the lower boundary and / or the region adjacent to the right boundary of the current coding unit, the encoding or decoding end performs intra-frame prediction on the sample point to be tested based on the first filter, which effectively improves the accuracy of the predicted value of the sample point to be tested, thereby helping to improve the coding efficiency of the EIP mode. Attached Figure Description
[0033] Figure 1a is one of the schematic diagrams of extrapolation filters in related technologies;
[0034] Figure 1b is one of the schematic diagrams of intra-frame prediction of the current coding unit based on extrapolation filters;
[0035] Figure 2 is a schematic diagram of an encoding / decoding system provided in an embodiment of this application;
[0036] Figure 3 is a schematic diagram of the structure of an encoder provided in an embodiment of this application;
[0037] Figure 4 is a schematic diagram of the structure of a decoder provided in an embodiment of this application;
[0038] Figure 5a is a schematic diagram of the current coding unit and the reference line;
[0039] Figure 5b is the second schematic diagram of intra-frame prediction of the current coding unit based on the extrapolation filter;
[0040] Figure 5c is a schematic diagram of one of the extrapolation filters applicable to embodiments of this application;
[0041] Figure 5d is a second schematic diagram of an extrapolation filter applicable to the embodiments of this application;
[0042] Figure 5e is the third schematic diagram of intra-frame prediction of the current coding unit based on extrapolation filters;
[0043] Figure 5f is the fourth schematic diagram of intra-frame prediction of the current coding unit based on the extrapolation filter;
[0044] Figure 6 is a flowchart illustrating an intra-frame prediction method provided in an embodiment of this application;
[0045] Figure 7a is a third schematic diagram of an extrapolation filter applicable to embodiments of this application;
[0046] Figure 7b is a fourth schematic diagram of an extrapolation filter applicable to embodiments of this application;
[0047] Figure 7c is a fifth schematic diagram of an extrapolation filter applicable to embodiments of this application;
[0048] Figure 8 is a schematic diagram of an intra-frame prediction device provided in an embodiment of this application;
[0049] Figure 9 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;
[0050] Figure 10 is a schematic diagram of the structure of a terminal provided in an embodiment of this application. Detailed Implementation
[0051] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.
[0052] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited in number; for example, the first object can be one or more. Furthermore, "or" in this application indicates at least one of the connected objects. For example, the scope of protection for "A or B" covers at least three scenarios: Scenario 1: including A but not B; Scenario 2: including B but not A; Scenario 3: including both A and B. In addition, the terms "A and / or B," "at least one of A and B," and "at least one of A or B" also cover at least the above three scenarios. The character " / " generally indicates that the preceding and following objects are in an "or" relationship.
[0053] Figure 2 is a schematic diagram of the codec system 10 provided in an embodiment of this application. The technical solution of this application embodiment relates to codec-decoder (CODEC) of video data (including encoding or decoding). The video data includes original unencoded video, encoded video, decoded (e.g., reconstructed) video, or syntax elements, etc.
[0054] As shown in Figure 2, the encoding / decoding system 10 includes a source device 100, which provides encoded video data to be decoded and displayed by the destination device 110. Specifically, the source device 100 provides video data to the destination device 110 via a communication medium 120. The source device 100 and the destination device 110 may include any one or more of the following: desktop computer, laptop computer, tablet computer, set-top box, mobile phone, wearable device (e.g., smartwatch or wearable camera), television, camera, display device, in-vehicle device, virtual reality (VR) device, augmented reality (AR) device, mixed reality (MR) device, digital media player, video game console, video conferencing equipment, video streaming equipment, broadcast receiver equipment, broadcast transmitter equipment, spacecraft, aircraft, robot, satellite, etc.
[0055] In the example of Figure 2, source device 100 includes a data source 101, memory 102, encoder 200, and output interface 104. Destination device 110 includes an input interface 111, decoder 300, memory 113, and display device 114. Source device 100 represents an example of a video encoding device, while destination device 110 represents an example of a video decoding device. In other examples, source device 100 and destination device 110 may not include some of the components shown in Figure 2, or they may include components other than those shown in Figure 2. For example, source device 100 may receive video data from an external data source (such as an external camera). Similarly, destination device 110 may interface with an external display device instead of including an integrated display device. As another example, memory 102 and memory 113 may be external memories.
[0056] Although Figure 2 illustrates the source device 100 and the destination device 110 as separate devices, in some examples, they may be integrated into a single device. In such embodiments, the same hardware or software, separate hardware or software, or any combination thereof may be used to implement the functionality corresponding to the source device 100 and the functionality corresponding to the destination device 110.
[0057] In some examples, source device 100 and destination device 110 can perform unidirectional or bidirectional video transmission. If it is bidirectional video transmission, source device 100 and destination device 110 can operate in a substantially symmetrical manner, that is, each of source device 100 and destination device 110 includes an encoder and a decoder.
[0058] Data source 101 represents the source of video data (i.e., raw, unencoded video data) and provides encoder 200 with a series of images containing video data, which encoder 200 encodes. Data source 101 of source device 100 may include a video acquisition device (such as a video camera), a video archive containing previously acquired raw video, or a video feed interface for receiving video from a video content provider. Alternatively, data source 101 may generate computer graphics-based data as source video, or combine live video, archived video, and computer-generated video. In these cases, encoder 200 encodes the acquired, pre-acquired, or computer-generated video data. Encoder 200 may rearrange the images from the received order (sometimes referred to as the "display order") according to the encoded order. Encoder 200 may generate a bitstream including the encoded video data. Source device 100 may then output the encoded video data to communication medium 120 via output interface 104 for reception or retrieval, for example, by input interface 111 of destination device 110.
[0059] The memory 102 of the source device 100 and the memory 113 of the destination device 110 represent general-purpose memory. In some examples, memory 102 may store raw video data from data source 101, and memory 113 may store decoded video data from decoder 300. Additionally or alternatively, memories 102 and 113 may respectively store software instructions executable by, for example, encoder 200 and decoder 300. Although memories 102 and 113 are shown separately from encoder 200 and decoder 300 in this example, it should be understood that encoder 200 and decoder 300 may also include internal memory for functionally similar or equivalent purposes. If encoder 200 and decoder 300 are deployed on the same hardware device, memories 102 and 113 may be the same memory. Furthermore, memories 102 and 113 may store, for example, encoded video data output from encoder 200 and input to decoder 300. In some examples, portions of memories 102 and 113 may be allocated as one or more video buffers, for example, to store raw, decoded, or encoded video data.
[0060] In some examples, source device 100 can output encoded data from output interface 104 to memory 113. Similarly, destination device 110 can access encoded data from memory 113 via input interface 111. Memory 113 or memory 102 can include any of a variety of distributed or locally accessed data storage media, such as hard drives, Blu-ray discs, digital versatile discs (DVDs), compact disc read-only memory (CD-ROMs), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.
[0061] Output interface 104 may include any type of medium or device capable of transmitting encoded video data from source device 100 to destination device 110. For example, output interface 104 may include a transmitter or transceiver, such as an antenna, configured to transmit encoded video data directly from source device 100 to destination device 110 in real time. The encoded video data may be modulated according to the communication standards of a wireless communication protocol and transmitted to destination device 110.
[0062] Communication medium 120 may include transient media, such as wireless broadcasting or wired network transmission. For example, communication medium 120 may include radio frequency (RF) spectrum or one or more physical transmission lines (e.g., cables). Communication medium 120 may form part of a packet-based network (such as a local area network, a wide area network, or a global network such as the Internet). Communication medium 120 may also take the form of a storage medium (e.g., a non-transitory storage medium), such as a hard disk, flash drive, compact disc, digital video disc, Blu-ray disc, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.
[0063] In some implementations, the communication medium 120 may include a router, switch, base station, or any other device that can be used to facilitate communication from source device 100 to destination device 110. For example, a server (not shown) may receive encoded video from source device 100 and provide the encoded video data to destination device 110, for example, via network transmission. The server may include (e.g., a web server for a website), a server configured to provide file transfer protocol services (such as File Transfer Protocol (FTP) or File Delivery Over Unidirectional Transport (FLUTE) protocol), a Content Delivery Network (CDN) device, a Hypertext Transfer Protocol (HTTP) server, a Multimedia Broadcast Multicast Services (MBMS) or Evolved Multimedia Broadcast Multicast Service (eMBMS) server, or a Network-attached Storage (NAS) device, etc. The server can implement one or more HTTP streaming protocols, such as MPEG Media Transport (MMT), Dynamic Adaptive Streaming over HTTP (DASH), HTTP Live Streaming (HLS), or Real Time Streaming Protocol (RTSP).
[0064] Destination device 110 can access encoded video data from a server, for example via a wireless channel (e.g., Wireless Fidelity (WIFI) connection) or a wired connection (e.g., Digital Subscriber Line (DSL), Cable Modem, etc.) for accessing encoded video data stored on the server.
[0065] Output interface 104 and input interface 111 can represent a wireless transmitter / receiver, a modem, a wired networking component (e.g., an Ethernet card), a wireless communication component operating according to the Institute of Electrical and Electronics Engineers (IEEE) 802.11 or IEEE 802.15 standard (e.g., ZigBee™ transmission mode), Bluetooth standard, or other physical components. In an example where output interface 104 and input interface 111 include a wireless component, output interface 104 and input interface 111 can be configured to operate according to Wi-Fi, Ethernet, cellular networks (such as 4G (4... th Generation 4G mobile communication networks, Long Term Evolution (LTE), Advanced LTE, 5G (5G) th Generation 5G mobile communication network, sixth generation (6G) th Generation 6G mobile communication networks, etc., are used to transmit data, such as encoded video data.
[0066] The technology provided in this application can be applied to support video encoding and decoding in one or more multimedia applications such as video conferencing, over-the-air television broadcasting, cable television transmission, satellite television transmission, internet streaming video transmission, digital video encoded onto a data storage medium, decoding of digital video stored on a data storage medium, or other applications.
[0067] Destination device 110 receives an encoded video bitstream from communication medium 120 via its input interface 111. The encoded video bitstream may include syntax elements and encoded data units (e.g., sequences, image groups, images, slices, blocks, etc.), where the syntax elements are used to decode the encoded data units to obtain decoded video data. Display device 114 displays the decoded video data to the user. Display device 114 may include a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED) display, or other types of display devices.
[0068] The encoder 200 and decoder 300 can be implemented as one or more of various processing circuits, which may include microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. When the technology is implemented wholly or partially in software, the device may store instructions for the software in a suitable non-transitory computer-readable storage medium and use one or more processors to execute the instructions in hardware to perform the technology provided in the embodiments of this application.
[0069] The encoder 200 and decoder 300 can process based on the following video codec standards: H.263, H.264, H.265 (also known as High Efficiency Video Coding, HEVC), H.266 (also known as Versatile Video Coding, VVC), Moving Picture Experts Group 2 (MPEG-2), MPEG-4, VP8, VP9, Alliance for Open Media Video 1 (AV1), Audio Video Coding Standard 1 (AVS1), AVS2, AVS3, or next-generation video standard protocols. This application embodiment does not specifically limit the specific implementation.
[0070] Typically, encoder 200 and decoder 300 can perform block-based encoding and decoding of images. The term "block" generally refers to a structure that includes data to be processed (e.g., encoded, decoded, or otherwise used during encoding or decoding). For example, a block can include a two-dimensional matrix of samples of luminance or chrominance data. For example, encoder 200 and decoder 300 can encode and decode video data represented in YUV format, where "Y" represents luminance (or Luma), and "U" and "V" are the two components of chrominance (or Chroma), with "U" representing the blue chrominance component (Cb) and "V" representing the red chrominance component (Cr).
[0071] Referring to Figure 3, which is a schematic diagram of the encoder 200 provided in an embodiment of this application, the encoder 200 can be the encoder 200 in Figure 2. In the example of Figure 3, the encoder 200 includes a memory 201, an encoding parameter determination unit 210, a residual generation unit 202, a transform processing unit 203, a quantization unit 204, an inverse quantization unit 205, an inverse transform processing unit 206, a reconstruction unit 207, a filter unit 208, a decoded picture buffer (DPB) 209, and an entropy encoding unit 220.
[0072] The memory 201 can store video data to be encoded. For example, the encoder 200 can receive and store video data from the data source 101 shown in Figure 2. In some examples, the memory 201 can be on the same chip as other components of the encoder 200 (as shown in Figure 3), or it can be on a separate chip from those components.
[0073] The coding parameter determination unit 210 includes a mode selection unit 211, an inter-frame prediction unit 212, and an intra-frame prediction unit 213. The inter-frame prediction unit 212 is used to obtain a first prediction block for the current block using an inter-frame prediction mode. The intra-frame prediction unit 213 is used to obtain a second prediction block for the current block using an intra-frame prediction mode. The mode selection unit 211 is used to obtain a target prediction block based on the first and second prediction blocks and determine the final prediction mode. Furthermore, the coding parameter determination unit 210 may also include other functional units, such as functional units for determining the partitioning method of coding units (CUs), functional units for determining the transformation type of the residual data of the CUs, or functional units for determining the quantization parameters of the residual data of the CUs.
[0074] For ease of description and understanding, in the embodiments of this application, the CU to be processed in the current image is referred to as the current CU, and the image block to be processed in the current CU is referred to as the current block or the image block to be processed. For example, in encoding, it refers to the block currently being encoded; in decoding, it refers to the block currently being decoded.
[0075] Inter-frame prediction unit 212 may include a motion estimation unit and a motion compensation unit. For inter-frame prediction of the current block, the motion estimation unit may perform a motion search to identify one or more matching reference blocks in one or more reference pictures (e.g., one or more previously encoded / decoded pictures stored in DPB 209).
[0076] The motion estimation unit can generate one or more motion vectors (MVs) representing the position of a reference block in a reference image relative to the position of the current block in the current image. The motion compensation unit can then use interpolation to obtain a predicted value with the precision indicated by the motion vectors.
[0077] The encoding parameter determination unit 210 can provide the target prediction block to the residual generation unit 202. The residual generation unit 202 receives the raw uncoded video data of the current block from the memory 201 and calculates the residual between the current block and the target prediction block to obtain the residual block. In some examples, the function of the residual generation unit 202 can be implemented using one or more subtractor circuits that perform binary subtraction.
[0078] As an example, the encoding parameter determination unit 210 can provide the entropy encoding unit 220 with syntax elements representing encoding parameters for encoding. The encoding parameters include one or more of the following: the partitioning method of the CU, the final prediction mode, the transformation type of the residual data of the CU, or the quantization parameters of the residual data of the CU.
[0079] The transformation processing unit 203 transforms the residual block output by the residual generation unit 202 to obtain a transform coefficient block. This transformation may include Discrete Cosine Transform (DCT), integer transformation, direction transformation, or Karhunen-Loeve Transform (KL Transform), etc. In some examples, the encoder 200 may not include the transformation processing unit 203.
[0080] Quantization unit 204 can quantize the transform coefficients in the transform coefficient block according to the quantization parameter (QP) value associated with the current block to generate a quantized transform coefficient block.
[0081] The inverse quantization unit 205 and the inverse transform processing unit 206 can perform inverse quantization and inverse transform on the transform coefficient block, respectively, to obtain the reconstructed residual block. The reconstruction unit 207 can generate a reconstructed block corresponding to the current block based on the reconstructed residual block and the target prediction block generated by the coding parameter determination unit 210.
[0082] Filter unit 208 can perform one or more filter operations on the reconstructed block. For example, filter unit 208 can be a deblocking filter (DBF), an adaptive loop filter (ALF), a sample adaptive offset (SAO) filter, etc. In some examples, encoder 200 may not include filter unit 208.
[0083] Encoder 200 stores the reconstructed image obtained from the reconstructed blocks in DPB 209. For example, in an example where the operation of filter unit 208 is not required, reconstruction unit 207 can store the reconstructed blocks in DPB 209. In an example where the operation of filter unit 208 is required, filter unit 208 can store the filtered reconstructed blocks in DPB 209. Inter-frame prediction unit 212 retrieves the reconstructed image from DPB 209 to perform inter-frame prediction on blocks of subsequent images to be encoded. In some examples, DPB 209 can be replaced with other types of memory.
[0084] Entropy coding unit 220 can entropy code the syntax elements of other components in encoder 200 to output encoded video data. For example, entropy coding unit 220 can entropy code the quantized transform coefficient block from quantization unit 204. As another example, entropy coding unit 220 can entropy code the syntax elements (e.g., motion information for inter-frame prediction or intra-frame mode information for intra-frame prediction) from coding parameter determination unit 210.
[0085] It is understood that the composition of the encoder 200 shown in Figure 3 is only illustrative and does not constitute a limitation on the embodiments of this application.
[0086] Figure 4 is a schematic diagram of the decoder 300 provided in an embodiment of this application. The decoder 300 can be the decoder 300 described in Figure 2. In the example of Figure 3, the decoder 300 includes a Coded Picture Buffer (CPB) 301, an entropy decoding unit 302, a prediction processing unit 310, an inverse quantization unit 303, an inverse transform processing unit 304, a reconstruction unit 305, a filter unit 306, and a DPB 307.
[0087] The entropy decoding unit 302 can receive encoded video data from the CPB 301 and perform entropy decoding on the video data to obtain syntax elements. The syntax elements indicate encoding parameters, including one or more of the following: CU partitioning method, final prediction mode, transformation type of CU residual data, or quantization parameters of CU residual data.
[0088] When the syntax element includes the final prediction mode, the prediction processing unit 310 obtains the final prediction mode. If the final prediction mode is an inter-frame prediction mode, the prediction block of the current CU can be obtained through the inter-frame prediction unit 311 of the prediction processing unit 310; if the final prediction mode is an intra-frame prediction mode, the prediction block of the current CU can be obtained through the intra-frame prediction unit 312 of the prediction processing unit 310. In some examples, the prediction processing unit 310 may also include a unit for performing prediction functions according to other prediction modes.
[0089] CPB 301 can acquire and store encoded video data from the communication medium 120 shown in Figure 2. DPB 307 is used to store decoded images. Optionally, CPB 301 and DPB 307 can be replaced with other types of memory, which are not specifically limited in this application. In some examples, CPB 301 can be on the same chip as other components of decoder 300 (as shown in the figure), or it can be on a separate chip from those components.
[0090] Decoder 300 can perform reconstruction operations on each block individually. Entropy decoding unit 302 can entropy decode the syntax elements and transform information (e.g., QP or transform mode indication) of the quantized transform coefficients to obtain the quantized transform coefficients. Dequantization unit 303 dequantizes the quantized transform coefficients to obtain a transform coefficient block including the transform coefficients. Inverse transform processing unit 304 performs an inverse transform on the transform coefficient block to generate a residual block corresponding to the current block; this inverse transform is the reverse operation of the above transform.
[0091] Reconstruction unit 305 can reconstruct the current block based on the prediction block and the residual block. For example, reconstruction unit 305 can add samples from the residual block to the corresponding samples from the prediction block to reconstruct the current block.
[0092] Filter unit 306 can perform one or more filter operations on the reconstructed block. For example, the type of filter unit 306 can be referenced to the type of filter unit 208, and will not be described again here. In some examples, the operations of filter unit 306 can be skipped.
[0093] Decoder 300 can store the reconstructed image obtained from the reconstructed blocks in DPB 307. For example, in an example where filter unit 306 is not operated, reconstruction unit 305 can store the reconstructed blocks in DPB 307. In an example where filter unit 306 is operated, filter unit 306 can store the filtered reconstructed blocks in DPB 307. Decoder 300 can output the decoded image (e.g., decoded video) from DPB 307 for subsequent rendering on a display device (such as display device 114 of FIG. 2).
[0094] In related technologies, video encoders employ a block-based hybrid coding framework. The coding process includes: block partitioning, intra-frame prediction, inter-frame prediction, transform, quantization, loop filtering, and entropy coding. The encoder first divides the image into non-overlapping Coding Tree Units (CTUs). Then, it further divides the CTUs into Coding Units (CUs) of different sizes according to quadtree, binary tree, and ternary tree partitioning methods. The final partitioning method and CU size are determined by rate-distortion cost. The encoder encodes each CU in a top-to-bottom, left-to-right order, and the decoder decodes the current frame's CUs in the same order. The encoder uses intra-frame prediction technology based on image texture correlation. Using the reconstructed samples above and to the left of the current CU as references, it traverses Direct Current (DC) mode, Planar mode, and angular prediction mode, utilizing rate-distortion cost to select the optimal intra-frame prediction mode to remove spatial redundancy in the image. The latest video compression standards, such as Versatile Video Coding (VVC), include 65 prediction modes for intra-frame prediction. VVC supports multiple reference rows. In addition to the most adjacent top row and the left column of reconstructed samples, there are a total of 6 reference rows with index values of {1,3,5,7,12}, as shown in Figure 5a. The index value indicates the distance between the reference row and the current coding unit.
[0095] Intra-frame prediction includes extrapolation filter-based intra-prediction mode (EIP). Taking the coding end as an example, the implementation method of EIP includes the following steps:
[0096] Step 1. Obtain the reconstructed pixel region above and / or to the left of the current coding unit as shown in Figure 5b. Calculate the 15-tap filter coefficients for the three shapes shown in Figure 5c. Alternatively, some techniques use the 15-tap filter coefficients for the three shapes shown in Figure 5d. The calculation method follows the same approach as the filter system calculation method in the Convolutional Cross-Component Intra-Model (CCCM) of the Enhanced Compression Model (ECM) in related technologies. Specifically:
[0097] Taking Figure 5e as an example, starting from the top left corner of the reconstructed pixel region, proceeding downwards from the first row, each row is traversed in a left-to-right order, or in a top-left-to-bottom-right order as shown in Figure 5f. The weighted average of the reconstructed values of the corresponding gray position sample points in each of the 14 filter coefficients is calculated. This is achieved by minimizing the mean-square error (MSE) between the weighted average calculated using the gray samples and the white samples in all 4×4 regions of the reconstructed pixel region. MSE minimization is performed by calculating the autocorrelation matrix with the reconstructed values of the 14 gray sample points as input, and the cross-correlation vector between the 14 gray input sample points and one white output sample point. The autocorrelation matrix is then decomposed using LDL, and the final filter coefficients are calculated using inverse permutation.
[0098] Step 2. Using the three extrapolation filters shown in Figure 5c or Figure 5d respectively, calculate the predicted value of each sample point starting from the first sample point in the upper left corner of the current coding unit, in the order from the upper left to the lower right as shown in Figure 5f; where the value of the gray sample point is the input, and the value of the white sample point in the lower right corner is the predicted output value.
[0099] The number of sample points to be predicted in the current coding unit is W×H, where W and H are the width and height of the current coding unit, respectively. Taking a 15-tap (i.e., a filter with 15 coefficients) as an example, to obtain the predicted value of the sample point in the lower right corner, the values of 14 sample points in the upper left part are needed. These 14 sample points may be decoded and reconstructed pixels outside the current coding unit, or they may be sample points inside the current coding unit that have already obtained predicted values. There is a one-to-one correspondence between the 14 filter coefficients and the 14 sample point values.
[0100] Step 3. Calculate the prediction value of the current coding unit obtained using the three shapes of extrapolation filters in EIP mode, and then calculate the rate-distortion cost, comparing it with the rate-distortion cost of other prediction modes. If the rate-distortion cost of EIP mode is the lowest, set the EIP mode identifier information to 1, and write it into the bitstream along with the selected filter index information.
[0101] After the decoder obtains that the current coding unit to be decoded is in EIP mode, it parses the index value of the selected filter. Following the same method as the encoder, it obtains the filter coefficients using the decoded and reconstructed pixel values above or to the left of the current coding unit. Then, it calculates the predicted value of each sample point in the current coding unit in order from top left to bottom right. This predicted value is added to the residual value calculated from the residual information of the current coding unit obtained from the bitstream to obtain the reconstructed value.
[0102] In related technologies, when using the 15-tap filter coefficients of the first and second shapes from the left in Figure 1a, the EIP technique utilizes the reconstructed or predicted sample points at the upper right and lower left of the current prediction position. If the sample point is located at the right or lower boundary of the current coding unit, as shown in the three positions (x, y) in Figure 1b, the sample at the diagonal position is unavailable. In this case, the value of the sample point at the diagonal position can only be replaced by the value of a sample point at a non-true position, such as the value of the sample point at the gray position obtained by using the left or upper edge of that position. This leads to inaccurate prediction values for these boundary position samples, resulting in low intra-frame prediction accuracy and reduced coding efficiency of the EIP mode. To address the above problems, this application proposes an intra-frame prediction method.
[0103] The intra-frame prediction method provided in the embodiments of this application is described below with reference to the accompanying drawings. The intra-frame prediction method provided in the embodiments of this application can be executed by an encoding end, such as the encoder 200 shown in Figure 2 or Figure 3, or by a decoding end, such as the decoder 300 shown in Figure 2 or Figure 4. The encoding end and decoding end can be implemented by software, hardware, or a combination thereof. When implemented by hardware, the encoding end can be referred to as an encoding device or a video encoding device, and the decoding end can be referred to as a decoding device or a video decoding device.
[0104] Please refer to Figure 6, which is a flowchart of an intra-frame prediction method provided in an embodiment of this application. For better understanding, the following embodiments use the decoding end as an example to illustrate the specific implementation of the intra-frame prediction method provided in this application. Of course, the specific implementation methods in the following embodiments are also applicable to the encoding end.
[0105] As shown in Figure 6, the method includes the following steps:
[0106] Step 601: Obtain the position of the sample point to be tested in the current coding unit.
[0107] Understandably, at the decoding end, the current coding unit (CU) is also the coding unit to be decoded; at the encoding end, the current coding unit is also the coding unit to be encoded. In some embodiments, the current coding unit may also be referred to as the unit to be encoded.
[0108] In this embodiment of the application, the sample point to be tested is also the sample point in the current coding unit that has not completed intra-frame prediction.
[0109] Step 602: Determine the target filter based on the position of the sample point to be tested.
[0110] Wherein, when the position of the sample point to be tested is located at the first position of the current coding unit, the target filter is a first filter; the first position is located in the vicinity of at least one of the lower boundary and the right boundary of the current coding unit, the first filter includes N first inputs and one first output, N is a positive integer, the N first inputs correspond one-to-one with the N first sample points, the first output corresponds to the sample point to be tested, and the position of the N first sample points includes at least one of the upper side and the left side of the sample point to be tested, but does not include at least one of the right side and the lower side of the sample point to be tested.
[0111] Optionally, the location of the sample point to be tested is located at the first position of the current coding unit, including at least the following cases:
[0112] (1) The sample point to be tested is adjacent to the lower boundary of the current coding unit, which can be understood as the sample point to be tested being right next to the lower boundary of the current coding unit;
[0113] (2) The test sample point is adjacent to the right boundary of the current coding unit, which can be understood as the test sample point being right next to the right boundary of the current coding unit;
[0114] (3) The sample point to be tested is adjacent to the lower boundary and the right boundary of the current coding unit, which can be understood as the sample point to be tested being right next to the lower boundary and the right boundary of the current coding unit.
[0115] (4) The sample point to be tested is separated from the lower boundary of the current coding unit by a first preset number of sample points, for example, the sample point to be tested is separated from the lower boundary of the current coding unit by 1 or 2 sample points.
[0116] (5) The sample point to be tested is separated from the right boundary of the current coding unit by a second preset number of sample points, for example, the sample point to be tested is separated from the right boundary of the current coding unit by 1 or 2 sample points.
[0117] (6) The sample point to be tested is separated from the lower boundary of the current coding unit by a first preset number of sample points and from the right boundary of the current coding unit by a second preset number of sample points. For example, the sample point to be tested is separated from the lower boundary of the current coding unit by 1 sample point and from the right boundary of the current coding unit by 2 sample points.
[0118] Understandably, the current coding unit performs intra-frame prediction on the sample points in the current coding unit in order from top left to bottom right, so that the sample points to be predicted will not exceed the lower and right boundaries of the current coding unit.
[0119] In this embodiment of the application, when the sample point to be tested is located at the first position of the current coding unit, the decoding end determines the target filter as the first filter. The first output of the first filter corresponds to the sample point to be tested. The N first sample points corresponding to the N first outputs of the first filter are located above and / or to the left of the sample point to be tested. The sample point to be tested is located in the vicinity of at least one of the lower boundary and right boundary of the current coding unit. Therefore, the N first sample points will not exceed the lower boundary and / or right boundary of the current coding unit. That is, the N first sample points are either sample points in the current coding unit that have already obtained predicted values or sample points outside the current coding unit that have already been decoded.
[0120] For example, the shape of the first filter may be as shown in Figure 7a(b) and / or (c), or it may be as shown in Figure 7b(b) and / or (c), or it may be as shown in Figure 7c(b) and / or (c). The input position of each first sample point in the first filter will be described in detail below.
[0121] When the shape of the first filter is as shown in Figure 7a(b), the input positions of each first sample point in the first filter include: {Position(-1, 0), Position(-2, 0), Position(0, -1), Position(0, -2), Position(-1, -1), Position(-2, -1), Position(-1, -2), Position(-2, -2)}.
[0122] When the shape of the first filter is as shown in Figure 7a(c), the input positions of each first sample point in the first filter include: {Position(-1, 0),Position(-2, 0),Position(-3, 0),Position(0, -1),Position(-1, -1),Position(-2, -1),Position(-3, -1),Position(0, -2),Position(-1, -2),Position(-2, -2),Position(-3, -2),Position(0, -3),Position(-1, -3),Position(-2, -3)}.
[0123] When the shape of the first filter is as shown in Figure 7b(b), the input positions of each first sample point in the first filter include: {Position(-1, 0), Position(-2, 0), Position(-3, 0), Position(-4, 0), Position(0, -1), Position(-1, -1), Position(-2, -1), Position(-3, -1)}.
[0124] When the shape of the first filter is as shown in Figure 7b(c), the input positions of each first sample point in the first filter include: {Position(-1, 0), Position(-2, 0), Position(-3, 0), Position(-4, 0), Position(-5, 0), Position(-6, 0), Position(-7, 0), Position(0, -1), Position(-1, -1), Position(-2, -1), Position(-3, -1), Position(-4, -1), Position(-5, -1), Position(-6, -1)}.
[0125] When the shape of the first filter is as shown in Figure 7c(b), the input positions of each first sample point in the first filter include: {Position(-1, 0), Position(0, -1), Position(-1, -1), Position(0, -2), Position(-1, -2), Position(0, -3), Position(-1, -3), Position(0, -4)}.
[0126] When the shape of the first filter is as shown in Figure 7c(c), the input positions of each first sample point in the first filter include: {Position(-1, 0), Position(0, -1), Position(-1, -1), Position(0, -2), Position(-1, -2), Position(0, -3), Position(-1, -3), Position(0, -4), Position(-1, -4), Position(0, -5), Position(-1, -5), Position(0, -6)}.
[0127] Step 603: Perform intra-frame prediction on the test sample points based on the target filter to obtain the predicted values of the test sample points.
[0128] Understandably, after determining the target filter based on the position of the sample point to be tested, the decoding end performs intra-frame prediction on the sample point to be tested based on the target filter to obtain the predicted value of the sample point to be tested. For example, the predicted value of the sample point to be tested is calculated based on the filter coefficients of the target filter and the reconstructed or predicted values of each sample point in the target filter. The method of obtaining the filter coefficients of the target filter and the reconstructed or predicted values of each sample point in the target filter will be described in detail in subsequent embodiments, and will not be repeated here.
[0129] In this embodiment, the decoding end obtains the position of the test sample point in the current coding unit. When the position of the test sample point is located at the first position of the current coding unit, the filter corresponding to the test sample point is determined to be the first filter. Intra-frame prediction is then performed on the test sample point based on the first filter to obtain the predicted value of the test sample point. Specifically, the N first inputs of the first filter correspond one-to-one with the N first sample points. The positions of the N first sample points are located above and / or to the left of the test sample point. Therefore, the N first sample points corresponding to the first filter will not exceed the lower boundary and / or right boundary of the current coding unit. That is, the N first sample points are either sample points in the current coding unit that have already obtained predicted values or sample points outside the current coding unit that have already been decoded. Thus, when the test sample point is located in the region adjacent to the lower boundary and / or the region adjacent to the right boundary of the current coding unit, the decoding end performs intra-frame prediction on the test sample point based on the first filter, which effectively improves the accuracy of the predicted value of the test sample point, thereby helping to improve the coding efficiency of the EIP mode.
[0130] Optionally, when the position of the sample point to be tested is not located at the first position of the current coding unit, the target filter is a second filter; wherein, the second filter includes M second inputs and one second output, M is a positive integer, the M second inputs correspond one-to-one with the M second sample points, the second output corresponds to the sample point to be tested, and the positions of the M second sample points include at least one of the following:
[0131] Located above the sample point to be tested;
[0132] Located to the left of the sample point to be tested;
[0133] Located on the upper right side of the sample point to be tested;
[0134] It is located on the lower left side of the sample point to be tested.
[0135] It should be noted that the position of the sample point to be tested is not located in the first position of the current coding unit, that is, the position of the sample point to be tested is not located in the adjacent area of the lower boundary and / or the adjacent area of the right boundary of the current coding unit.
[0136] Optionally, the position of the sample point to be tested is not located at the first position of the current coding unit, including at least the following cases:
[0137] (1) The sample point to be tested is not adjacent to the lower boundary of the current coding unit;
[0138] (2) The sample point to be tested is not adjacent to the right boundary of the current coding unit;
[0139] (3) The sample point to be tested is neither adjacent to the lower boundary of the current coding unit nor to the right boundary of the current coding unit;
[0140] (4) The number of sample points between the sample point to be tested and the lower boundary of the current coding unit exceeds the first preset number. For example, if the first preset number is 1, then the sample point to be tested and the lower boundary of the current coding unit are separated by at least two sample points.
[0141] (5) The number of sample points between the sample point to be tested and the right boundary of the current coding unit exceeds the second preset number. For example, if the second preset number is 2, then the sample point to be tested and the right boundary of the current coding unit are separated by at least 3 sample points.
[0142] (6) The number of sample points between the test sample point and the lower boundary of the current coding unit exceeds a first preset number, and the number of sample points between the test sample point and the right boundary of the current coding unit exceeds a second preset number.
[0143] In this embodiment, when the sample point to be tested is not located at the first position of the current encoding unit, the decoding end determines the target filter as a second filter. The second input of the second filter corresponds to the sample point to be tested, and each of the M second inputs of the second filter corresponds one-to-one with one of the M second sample points. The positions of the M second sample points include at least one of the following: located above the sample point to be tested, located to the left of the sample point to be tested, located to the upper right of the sample point to be tested, or located to the lower left of the sample point to be tested.
[0144] Optionally, the M second inputs of the second filter include at least one target second input, the target second input satisfying at least one of the following:
[0145] The offset of the second target input relative to the second output in the x-direction is greater than 0;
[0146] The offset of the second target input relative to the second output in the y-direction is greater than 0.
[0147] For example, the shape of the second filter can be as shown in Figure 7a(a), or as shown in Figure 7b(a), or as shown in Figure 7c(a). The input position of each second sample point in the second filter will be described in detail below.
[0148] When the shape of the second filter is as shown in Figure 7a(a), the input positions of each first sample point in the second filter include: {Position(-1, 0), Position(-2, 0), Position(-2, 1), Position(0, -1), Position(-1, -1), Position(-2, -1), Position(1, -2), Position(0, -2), Position(-3, 1), Position(-3, 0), Position(-1, -2), Position(-2, -2), Position(1, -3), Position(0, -3)}.
[0149] When the shape of the second filter is as shown in Figure 7b(a), the input positions of each first sample point in the second filter include: {Position(0,-1),Position(-1,0),Position(-1,-1),Position(-2,0),Position(-2,-1),Position(-3,0),Position(-3,-1),Position(-4,0),Position(-4,-1),Position(-5,0),Position(-6,0),Position(-7,0),Position(-2,1),Position(-3,1)}.
[0150] When the shape of the second filter is as shown in Figure 7c(a), the input positions of each first sample point in the second filter include: {Position(-1,0),Position(0,-1),Position(-1,-1),Position(0,-2),Position(-1,-2),Position(0,-3),Position(-1,-3),Position(0,-4),Position(-1,-4),Position(0,-5),Position(0,-6),Position(0,-7),Position(1,-3),Position(1,-2)}.
[0151] Taking the second filter shown in Figure 7a(a) as an example, the second sample point (gray square) corresponding to the second filter includes sample points located on the lower left and upper right sides of the sample point to be tested (x,y). The position of the sample point to be tested is not located at the first position of the current coding unit, that is, the sample point to be tested is not adjacent to the lower boundary and right boundary of the current coding unit. Therefore, even if the second filter is used, the second sample point corresponding to the second filter will not exceed the lower boundary and right boundary of the current coding unit. Thus, when performing intra-frame prediction on each sample point in the current coding unit in the order from the upper left to the lower right, the reconstructed value or predicted value of each second input of the second filter can be obtained, thereby effectively ensuring the accuracy of the predicted value of the sample point to be tested.
[0152] In this embodiment, when the position of the sample point to be tested is located at the first position of the current coding unit, the first filter is used to perform intra-frame prediction on the sample point to be tested; when the position of the sample point to be tested is not located at the first position of the current coding unit, the second filter is used to perform intra-frame prediction on the sample point to be tested. This allows the decoder to select an appropriate filter for intra-frame prediction based on the position of the sample point to be tested within the current coding unit, making the intra-frame prediction method more flexible and ensuring the accuracy of the predicted value of the sample point to be tested, effectively improving the decoding or coding efficiency of the EIP mode.
[0153] Optionally, the values of N and M may be the same or different. That is, the number of first sample points corresponding to the first filter and the number of second sample points corresponding to the second filter may be the same or different. For example, as shown in FIG7c, when the second filter is the shape shown in FIG7b(a) and the first filter is the shape shown in FIG7b(c), the number of first sample points (gray squares) corresponding to the first filter and the number of second sample points (gray squares) corresponding to the second filter are the same. When the second filter is the shape shown in FIG7c(a) and the first filter is the shape shown in FIG7c(b), the number of first sample points (gray squares) corresponding to the first filter and the number of second sample points (gray squares) corresponding to the second filter are different.
[0154] In this embodiment of the application, the number of first sample points corresponding to the first filter and the number of second sample points corresponding to the second filter can be the same or different, thereby making the selection of the first filter and the second filter more flexible at the decoding end or the encoding end, and also making the intra-frame prediction method more flexible.
[0155] Optionally, the shape of the first filter may be the same as or different from the shape of the second filter. For example, the first filter shown in FIG7a(b) has a different shape than the second filter described in FIG7a(a). Alternatively, in some embodiments, the shape of the first filter is the same as the shape of the second filter. This allows for greater flexibility in the selection of the first and second filters at the decoding or encoding end, and also makes the intra-frame prediction method more flexible.
[0156] Optionally, the step of performing intra-frame prediction on the test sample point based on the target filter to obtain the predicted value of the test sample point includes:
[0157] Obtain the filter coefficients of the target filter, and obtain the reconstructed or predicted values of each sample point in the target filter;
[0158] Based on the filter coefficients and the reconstructed or predicted values of each sample point in the target filter, the predicted value of the sample point to be tested is determined.
[0159] The filter coefficients of the target filter can be obtained using the method for calculating filter coefficients in CCCM within ECM. For example, taking the first filter as an example, assuming the first filter is the filter shown in Figure 7a(c), according to the shape of the first filter, starting from the upper left corner of the reconstructed pixel region of the current coding unit, from the top row downwards, each row is traversed in a left-to-right order or a left-to-right order. For each 4×4 region, based on the initial 15 filter coefficients and the reconstructed values of 14 input samples, a predicted value for one output sample is determined. Then, by minimizing the mean square error (MSE) between the reconstructed values of all output samples corresponding to all 4×4 regions in the reconstructed region and the predicted values of all output samples, the 15 filter coefficients are updated, i.e., a set of filter coefficients is found that minimizes the sum of squared errors between the reconstructed values and the predicted values.
[0160] Further, the reconstructed or predicted values of each sample point in the target filter are obtained. Taking the first filter as an example, if the first sample point of the first filter is located outside the current coding unit, the reconstructed value corresponding to the first sample point is obtained; if the first sample point of the first filter is located inside the current coding unit, the predicted value corresponding to the first sample point is obtained. The calculation method for the reconstructed or predicted value corresponding to the first sample point can refer to related technologies, which will not be elaborated upon in this embodiment.
[0161] Understandably, the filter coefficients of the target filter correspond one-to-one with the reconstructed or predicted values of the sample points of the target filter. Taking the first filter as an example, the N+1 filter coefficients of the first filter correspond one-to-one with the reconstructed or predicted values of the N+1 sample points of the first filter, that is, one filter coefficient corresponds to one reconstructed or predicted value of a sample point.
[0162] In this embodiment, the predicted value of the sample point to be tested is determined based on the filter coefficients and the reconstructed or predicted values of each sample point in the target filter. Optionally, the predicted value of the sample point to be tested can be determined according to the following formula (1):
[0163] Among them, pred (x,y) c is the predicted value of the sample point to be tested. i Let N be the filter coefficients of the target filter, i = 0, ..., R-1, where R is the value of N or M mentioned above. offsetX represents the reconstructed or predicted value of each sample point in the target filter. iOffsetY represents the relative positional offset between the first sample point and the test sample point in the first filter along the x-axis, or the relative positional offset between the second sample point and the test sample point in the second filter along the y-axis. i It is the relative position offset of the first sample point and the test sample point in the first filter in the y-axis direction, or the relative position offset of the second sample point and the test sample point in the second filter in the x-axis direction, where bitdepth is the bit depth.
[0164] In this embodiment of the application, whether the first filter or the second filter is used, the predicted value of the sample point to be tested can be calculated based on the above formula (1), thereby realizing the intra-frame prediction of the sample point to be tested and effectively ensuring the accuracy of the intra-frame prediction.
[0165] The intra-frame prediction method provided in this application is applicable to both the decoding and encoding ends. Specifically, when the method is applied to the encoding end, the encoding end writes the index information of the selected target filter into the bitstream, enabling the decoding end to determine which filter is being used by the encoding end based on the index information in the bitstream. Correspondingly, when the method is applied to the decoding end, upon receiving the bitstream sent by the encoding end, the decoding end parses the bitstream and extracts the index information of the target filter from it. Based on this index information, the decoding end determines whether the encoding end is using a first filter or a second filter, thereby effectively ensuring the decoding accuracy of the decoding end.
[0166] The intra-prediction method provided in this application can be executed by an intra-prediction device. As an example, the device can be an electronic device or a component within an electronic device, such as a chip or circuit. This application uses an intra-prediction device executing the intra-prediction method as an example to illustrate the intra-prediction device provided in this application.
[0167] Please refer to Figure 8, which is a structural diagram of an intra-frame prediction device provided in an embodiment of this application. The device can be applied to an encoding end or a decoding end, or it can be an encoding end device or a decoding end device. As shown in Figure 8, the intra-frame prediction device 800 includes:
[0168] The acquisition module 801 is used to acquire the position of the sample point to be tested in the current coding unit;
[0169] The determination module 802 is used to determine the target filter based on the position of the sample point to be tested;
[0170] Prediction module 803 is used to perform intra-frame prediction on the test sample point based on the target filter to obtain the predicted value of the test sample point;
[0171] Wherein, when the position of the sample point to be tested is located at the first position of the current coding unit, the target filter is the first filter;
[0172] The first position is located in the vicinity of at least one of the lower boundary and the right boundary of the current coding unit. The first filter includes N first inputs and one first output, where N is a positive integer. The N first inputs correspond one-to-one with the N first sample points. The first output corresponds to the sample point to be tested. The positions of the N first sample points include at least one of the upper and left sides of the sample point to be tested, but do not include at least one of the right and lower sides of the sample point to be tested.
[0173] Optionally, when the position of the sample point to be tested is not located at the first position of the current coding unit, the target filter is a second filter;
[0174] The second filter includes M second inputs and one second output, where M is a positive integer. Each of the M second inputs corresponds one-to-one with one of the M second sample points, and the second output corresponds to the sample point to be tested. The positions of the M second sample points include at least one of the following:
[0175] Located above the sample point to be tested;
[0176] Located to the left of the sample point to be tested;
[0177] Located on the upper right side of the sample point to be tested;
[0178] It is located on the lower left side of the sample point to be tested.
[0179] Optionally, the M second inputs of the second filter include at least one target second input, the target second input satisfying at least one of the following:
[0180] The offset of the second target input relative to the second output in the x-direction is greater than 0;
[0181] The offset of the second target input relative to the second output in the y-direction is greater than 0.
[0182] Optionally, the values of N and M may be the same or different.
[0183] Optionally, the shape of the first filter may be the same as or different from the shape of the second filter.
[0184] Optionally, the prediction module 803 is further configured to:
[0185] Obtain the filter coefficients of the target filter, and obtain the reconstructed or predicted values of each sample point in the target filter;
[0186] Based on the filter coefficients and the reconstructed or predicted values of each sample point in the target filter, the predicted value of the sample point to be tested is determined.
[0187] Optionally, the position of the sample point to be tested is located at the first position of the current coding unit, including at least one of the following:
[0188] The sample point to be tested is adjacent to the lower boundary of the current coding unit;
[0189] The sample point to be tested is adjacent to the right boundary of the current coding unit;
[0190] The sample point to be tested is separated from the lower boundary of the current coding unit by a first preset number of sample points;
[0191] The sample point to be tested is separated from the right boundary of the current encoding unit by a second preset number of sample points.
[0192] The intra-frame prediction apparatus provided in this application determines the filter corresponding to the test sample point as a first filter when the position of the test sample point is located at the first position of the current coding unit. Based on the first filter, intra-frame prediction is performed on the test sample point to obtain its predicted value. Specifically, the N first inputs of the first filter correspond one-to-one with N first sample points, and the positions of the N first sample points are located above and / or to the left of the test sample point. Therefore, the N first sample points corresponding to the first filter will not exceed the lower and right boundaries of the current coding unit. That is, the N first sample points are either sample points within the current coding unit that have already obtained predicted values or sample points outside the current coding unit that have already been decoded. Thus, when the test sample point is located in the region adjacent to the lower boundary and / or the region adjacent to the right boundary of the current coding unit, the apparatus performs intra-frame prediction on the test sample point based on the first filter, which effectively improves the accuracy of the predicted value of the test sample point, thereby helping to improve the coding efficiency of the EIP mode.
[0193] The intra-frame prediction apparatus provided in this application embodiment can implement all the processes implemented in the intra-frame prediction method embodiment shown in FIG6 and achieve the same technical effect. To avoid repetition, it will not be described again here.
[0194] As shown in Figure 9, this application embodiment also provides an electronic device 900, including a processor 901 and a memory 902. The memory 902 stores programs or instructions that can run on the processor 901. For example, when the electronic device 900 is an encoding device, the program or instructions executed by the processor 901 implement the various steps of the above-described intra-frame prediction method embodiment and achieve the same technical effect. When the electronic device 900 is a decoding device, the program or instructions executed by the processor 901 implement the various steps of the above-described intra-frame prediction method embodiment and achieve the same technical effect. To avoid repetition, this will not be described again here. Optionally, the memory 902 can be the memory 102 or memory 113 in the embodiment shown in Figure 2, and the processor 901 can implement the functions of the encoder 200 or decoder 300 in the embodiments shown in Figures 3-4.
[0195] This application also provides an electronic device, including: a memory configured to store video data; and a processing circuit configured to implement the various steps of the intra-frame prediction method embodiments described above. Optionally, the memory may be memory 102 or memory 113 in the embodiment shown in FIG2, and the processing circuit may implement the functions of encoder 200 or decoder 300 in the embodiments shown in FIG3-4.
[0196] This application also provides an electronic device, including a processor and a communication interface. The communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the steps in the method embodiment shown in FIG6. This device embodiment corresponds to the above method embodiment, and all implementation processes and methods of the above method embodiments can be applied to this terminal embodiment and can achieve the same technical effect.
[0197] The processor or processing circuit in this application embodiment may include general-purpose processors, special-purpose processors, etc., such as central processing units (CPUs), microprocessors, digital signal processors (DSPs), artificial intelligence (AI) processors, graphics processing units (GPUs), application-specific integrated circuits (ASICs), network processors (NPs), field-programmable gate arrays (FPGAs), or other programmable logic devices, gate circuits, transistors, discrete hardware components, etc. The communication interface in this application embodiment may include transceivers, pins, circuits, buses, etc.
[0198] The aforementioned electronic devices can be terminals or other devices besides terminals, such as servers, network attached storage (NAS), etc.
[0199] Among them, the terminal can also be called user equipment (UE), which can be a mobile phone, tablet computer, laptop computer, notebook computer, personal digital assistant (PDA), handheld computer, netbook, ultra-mobile personal computer (UMPC), mobile internet device (MID), augmented reality (AR), virtual reality (VR) device, mixed reality (MR) device, robot, wearable device, flight vehicle, vehicle user equipment (VUE), shipborne equipment, pedestrian user equipment (PUE), smart home (home devices with wireless communication functions, such as refrigerators, televisions, washing machines or furniture, etc.), game console, personal computer (PC), ATM or self-service machine, etc. Wearable devices include: smartwatches, smart bracelets, smart earphones, smart glasses, smart jewelry (smart bracelets, smart chains, smart rings, smart necklaces, smart anklets, smart anklets, etc.), smart wristbands, smart clothing, etc. Among these, in-vehicle devices can also be referred to as in-vehicle terminals, in-vehicle controllers, in-vehicle modules, in-vehicle components, in-vehicle chips, or in-vehicle units, etc. It should be noted that the embodiments in this application do not limit the specific type of terminal.
[0200] A server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server. A cloud server can provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), or cloud computing services based on big data and artificial intelligence platforms.
[0201] For example, the aforementioned electronic device may include, but is not limited to, the type of source device 100 or destination device 110 shown in FIG2.
[0202] Taking an electronic device as an example, Figure 10 is a schematic diagram of the hardware structure of a terminal implementing an embodiment of this application.
[0203] The terminal 1000 includes, but is not limited to, at least some of the following components: radio frequency unit 1001, network module 1002, audio output unit 1003, input unit 1004, sensor 1005, display unit 1006, user input unit 1007, interface unit 1008, memory 1009, and processor 1010.
[0204] Those skilled in the art will understand that the terminal 1000 may also include a power supply (such as a battery) for powering various components. The power supply can be logically connected to the processor 1010 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The terminal structure shown in Figure 10 does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.
[0205] It should be understood that, in this embodiment, the input unit 1004 may include a graphics processor 10041 and a microphone 10042. The graphics processor 10041 processes image data of still images or videos obtained by an image acquisition device (such as a camera) in video acquisition mode or image acquisition mode, or it may process the obtained point cloud data. The display unit 1006 may include a display panel 10061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 1007 includes at least one of a touch panel 10071 and other input devices 10072. The touch panel 10071 is also called a touch screen. The touch panel 10071 may include a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.
[0206] In this embodiment, after receiving downlink data from the network-side device, the radio frequency unit 1001 can transmit it to the processor 1010 for processing; in addition, the radio frequency unit 1001 can send uplink data to the network-side device. Typically, the radio frequency unit 1001 includes, but is not limited to, antennas, amplifiers, transceivers, couplers, low-noise amplifiers, duplexers, etc.
[0207] The memory 1009 can be used to store software programs or instructions, as well as various data. The memory 1009 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1009 may include volatile memory or non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1009 in this embodiment includes, but is not limited to, these and any other suitable types of memory.
[0208] The processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into the processor 1010.
[0209] The processor 1010 is used for:
[0210] Obtain the position of the test sample point in the current coding unit;
[0211] The target filter is determined based on the location of the sample point to be tested;
[0212] Based on the target filter, intra-frame prediction is performed on the test sample points to obtain the predicted values of the test sample points;
[0213] Wherein, when the position of the sample point to be tested is located at the first position of the current coding unit, the target filter is the first filter;
[0214] The first position is located in the vicinity of at least one of the lower boundary and the right boundary of the current coding unit. The first filter includes N first inputs and one first output, where N is a positive integer. The N first inputs correspond one-to-one with the N first sample points. The first output corresponds to the sample point to be tested. The positions of the N first sample points include at least one of the upper and left sides of the sample point to be tested, but do not include at least one of the right and lower sides of the sample point to be tested.
[0215] Optionally, when the position of the sample point to be tested is not located at the first position of the current coding unit, the target filter is a second filter;
[0216] The second filter includes M second inputs and one second output. The M second inputs correspond one-to-one with the M second sample points, and the second output corresponds to the sample point to be tested. The positions of the M second sample points include at least one of the following:
[0217] Located above the sample point to be tested;
[0218] Located to the left of the sample point to be tested;
[0219] Located on the upper right side of the sample point to be tested;
[0220] It is located on the lower left side of the sample point to be tested.
[0221] Optionally, the M second inputs of the second filter include at least one target second input, the target second input satisfying at least one of the following:
[0222] The offset of the second target input relative to the second output in the x-direction is greater than 0;
[0223] The offset of the second target input relative to the second output in the y-direction is greater than 0.
[0224] Optionally, the values of N and M may be the same or different.
[0225] Optionally, the shape of the first filter may be the same as or different from the shape of the second filter.
[0226] Optionally, the processor 1010 is also used for:
[0227] Obtain the filter coefficients of the target filter, and obtain the reconstructed or predicted values of each sample point in the target filter;
[0228] Based on the filter coefficients and the reconstructed or predicted values of each sample point in the target filter, the predicted value of the sample point to be tested is determined.
[0229] Optionally, the position of the sample point to be tested is located at the first position of the current coding unit, including at least one of the following:
[0230] The sample point to be tested is adjacent to the lower boundary of the current coding unit;
[0231] The sample point to be tested is adjacent to the right boundary of the current coding unit;
[0232] The sample point to be tested is separated from the lower boundary of the current coding unit by a first preset number of sample points;
[0233] The sample point to be tested is separated from the right boundary of the current encoding unit by a second preset number of sample points.
[0234] In this embodiment, when the position of the sample point to be tested is located at the first position of the current coding unit, the filter corresponding to the sample point to be tested is determined as the first filter. Intra-frame prediction is then performed on the sample point to be tested based on the first filter to obtain the predicted value of the sample point. Specifically, the N first inputs of the first filter correspond one-to-one with the N first sample points. The positions of the N first sample points are located above and / or to the left of the sample point to be tested. Therefore, the N first sample points corresponding to the first filter will not exceed the lower and right boundaries of the current coding unit. That is, the N first sample points are either sample points within the current coding unit that have already obtained predicted values or sample points outside the current coding unit that have already been decoded. Thus, when the sample point to be tested is located in the region adjacent to the lower boundary and / or the region adjacent to the right boundary of the current coding unit, the terminal performs intra-frame prediction on the sample point to be tested based on the first filter, which effectively improves the accuracy of the predicted value of the sample point to be tested, thereby helping to improve the coding efficiency of the EIP mode.
[0235] It is understood that the implementation process of each implementation method mentioned in this embodiment can refer to the relevant description of the method embodiment described in Figure 6, and achieve the same or corresponding technical effects. To avoid repetition, it will not be described again here.
[0236] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described intra-frame prediction method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.
[0237] The processor mentioned above is the processor in the terminal described in the above embodiments. The readable storage medium includes computer-readable storage media, such as ROM, RAM, magnetic disk, or optical disk. In some examples, the readable storage medium may be a non-transient readable storage medium.
[0238] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described intra-frame prediction method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.
[0239] It should be understood that the chips mentioned in the embodiments of this application may include system-on-a-chip (also known as system chip, chip system, or system-on-a-chip) or discrete display chips, etc.
[0240] This application also provides a computer program / program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described intra-frame prediction method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.
[0241] This application also provides an encoding / decoding system, including: an encoding end device and a decoding end device. The encoding end device can be used to perform the steps of the intra-frame prediction method as described above, and the decoding end device can be used to perform the steps of the intra-frame prediction method as described above.
[0242] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
[0243] From the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of computer software products plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, magnetic disk, optical disk, etc.), and the computer software product includes several instructions to cause the terminal or network-side device to execute the methods described in the various embodiments of this application.
[0244] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other implementations under the guidance of this application without departing from the spirit and scope of the claims. All of these implementations are within the protection scope of this application.
Claims
1. An intra-frame prediction method, comprising: Obtain the position of the test sample point in the current coding unit; The target filter is determined based on the location of the sample point to be tested; Based on the target filter, intra-frame prediction is performed on the test sample points to obtain the predicted values of the test sample points; Wherein, when the position of the sample point to be tested is located at the first position of the current coding unit, the target filter is the first filter; The first position is located in the vicinity of at least one of the lower boundary and the right boundary of the current coding unit. The first filter includes N first inputs and one first output, where N is a positive integer. The N first inputs correspond one-to-one with the N first sample points. The first output corresponds to the sample point to be tested. The positions of the N first sample points include at least one of the upper and left sides of the sample point to be tested, but do not include at least one of the right and lower sides of the sample point to be tested.
2. The method according to claim 1, wherein, When the position of the sample point to be tested is not located at the first position of the current coding unit, the target filter is the second filter; The second filter includes M second inputs and one second output, where M is a positive integer. Each of the M second inputs corresponds one-to-one with one of the M second sample points, and the second output corresponds to the sample point to be tested. The positions of the M second sample points include at least one of the following: Located above the sample point to be tested; Located to the left of the sample point to be tested; Located on the upper right side of the sample point to be tested; It is located on the lower left side of the sample point to be tested.
3. The method according to claim 2, wherein, The second filter includes at least one target second input among its M second inputs, the target second input satisfying at least one of the following: The offset of the second target input relative to the second output in the x-direction is greater than 0; The offset of the second target input relative to the second output in the y-direction is greater than 0.
4. The method according to claim 2, wherein, The values of N and M may be the same or different.
5. The method according to any one of claims 2-4, wherein, The shape of the first filter may be the same as or different from the shape of the second filter.
6. The method according to any one of claims 1-5, wherein, The step of performing intra-frame prediction on the test sample points based on the target filter to obtain the predicted values of the test sample points includes: Obtain the filter coefficients of the target filter, and obtain the reconstructed or predicted values of each sample point in the target filter; Based on the filter coefficients and the reconstructed or predicted values of each sample point in the target filter, the predicted value of the sample point to be tested is determined.
7. The method according to any one of claims 1-6, wherein, The location of the sample point to be tested is located at the first position of the current coding unit, including at least one of the following: The sample point to be tested is adjacent to the lower boundary of the current coding unit; The sample point to be tested is adjacent to the right boundary of the current coding unit; The sample point to be tested is separated from the lower boundary of the current coding unit by a first preset number of sample points; The sample point to be tested is separated from the right boundary of the current encoding unit by a second preset number of sample points.
8. An intra-frame prediction apparatus, comprising: The acquisition module is used to acquire the position of the test sample point in the current coding unit; The determination module is used to determine the target filter based on the position of the sample point to be tested; The prediction module is used to perform intra-frame prediction on the test sample points based on the target filter to obtain the predicted values of the test sample points. Wherein, when the position of the sample point to be tested is located at the first position of the current coding unit, the target filter is the first filter; The first position is located in the vicinity of at least one of the lower boundary and the right boundary of the current coding unit. The first filter includes N first inputs and one first output, where N is a positive integer. The N first inputs correspond one-to-one with the N first sample points. The first output corresponds to the sample point to be tested. The positions of the N first sample points include at least one of the upper and left sides of the sample point to be tested, but do not include at least one of the right and lower sides of the sample point to be tested.
9. The apparatus according to claim 8, wherein, When the position of the sample point to be tested is not located at the first position of the current coding unit, the target filter is the second filter; The second filter includes M second inputs and one second output, where M is a positive integer. Each of the M second inputs corresponds one-to-one with one of the M second sample points, and the second output corresponds to the sample point to be tested. The positions of the M second sample points include at least one of the following: Located above the sample point to be tested; Located to the left of the sample point to be tested; Located on the upper right side of the sample point to be tested; It is located on the lower left side of the sample point to be tested.
10. The apparatus according to claim 9, wherein, The second filter includes at least one target second input among its M second inputs, the target second input satisfying at least one of the following: The offset of the second target input relative to the second output in the x-direction is greater than 0; The offset of the second target input relative to the second output in the y-direction is greater than 0.
11. The apparatus according to claim 9, wherein, The values of N and M may be the same or different.
12. The apparatus according to any one of claims 9-11, wherein, The shape of the first filter may be the same as or different from the shape of the second filter.
13. The apparatus according to any one of claims 8-12, wherein, The prediction module is also used for: Obtain the filter coefficients of the target filter, and obtain the reconstructed or predicted values of each sample point in the target filter; Based on the filter coefficients and the reconstructed or predicted values of each sample point in the target filter, the predicted value of the sample point to be tested is determined.
14. The apparatus according to any one of claims 8-13, wherein, The location of the sample point to be tested is located at the first position of the current coding unit, including at least one of the following: The sample point to be tested is adjacent to the lower boundary of the current coding unit; The sample point to be tested is adjacent to the right boundary of the current coding unit; The sample point to be tested is separated from the lower boundary of the current coding unit by a first preset number of sample points; The sample point to be tested is separated from the right boundary of the current encoding unit by a second preset number of sample points.
15. An electronic device comprising a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the intra-frame prediction method as claimed in any one of claims 1-7.
16. A readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the intra-frame prediction method as described in any one of claims 1-7.
17. A chip comprising a processor and a communication interface coupled to the processor, the processor being configured to run a program or instructions to implement the steps of the intra-frame prediction method as described in any one of claims 1-7.
18. A computer program product stored in a storage medium, the computer program product being executed by at least one processor to implement the steps of the intra-frame prediction method as claimed in any one of claims 1-7.
19. An electronic device for performing the steps of the intra-frame prediction method as claimed in any one of claims 1-7.