Image block processing method and apparatus, and related devices

Abstract

The present application belongs to the technical field of computers. Disclosed are an image block processing method and apparatus, and related devices. The image block processing method in the embodiments of the present application comprises: determining a filtering region, wherein a first prediction block of an image block comprises the filtering region; and performing filtering processing on the first prediction block on the basis of the filtering region, so as to obtain a second prediction block.

Description

Image patch processing methods, apparatus and related equipment

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese Patent Application No. 202510051301.5, filed in China on January 13, 2025, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application belongs to the field of computer technology, and specifically relates to an image block processing method, apparatus and related equipment. Background Technology

[0004] In image or video processing, prediction blocks for image patches are frequently generated. In some related technologies, these prediction blocks are often generated directly based on prediction modes, such as intra-frame or inter-frame prediction modes. This generated prediction block is the final prediction block for the image patch. However, because the prediction block is generated directly based on the prediction mode, its prediction accuracy is relatively poor. Summary of the Invention

[0005] This application provides an image patch processing method, apparatus, and related equipment, which can solve the problem of poor prediction accuracy of image patch prediction blocks.

[0006] Firstly, an image patch processing method is provided, including:

[0007] Determine the filtering region, wherein the first prediction block of the image block includes the filtering region;

[0008] The first prediction block is filtered based on the filtering region to obtain the second prediction block.

[0009] Secondly, an image block processing apparatus is provided, comprising:

[0010] A first determining module is used to determine a filtering region, wherein a first prediction block of an image block includes the filtering region;

[0011] The processing module is used to filter the first prediction block based on the filtering region to obtain the second prediction block.

[0012] Thirdly, an image patch processing apparatus is provided, the apparatus being configured to perform the steps of the method described in the first aspect.

[0013] Fourthly, an electronic device is provided, the terminal including a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the method as described in the first aspect.

[0014] Fifthly, an electronic device is provided, including a processor and a communication interface, wherein the processor is configured to determine a filtering region, wherein a first prediction block of an image block includes the filtering region; and a second prediction block is obtained by filtering the first prediction block based on the filtering region.

[0015] A sixth aspect provides an electronic device comprising: a memory configured to store video data, and processing circuitry configured to implement the steps of the method described in the first aspect.

[0016] In a seventh aspect, a readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0017] Eighthly, a chip is provided, the chip including a processor and a communication interface coupled to the processor, the processor being used to run programs or instructions to implement the steps of the method as described in the first aspect.

[0018] In a ninth aspect, a computer program / program product is provided, the computer program / program product being stored in a storage medium, the computer program / program product being executed by at least one processor to perform the steps of the method as described in the first aspect.

[0019] In this embodiment, a filtering region is determined, wherein a first prediction block of an image block includes the filtering region; the first prediction block is filtered based on the filtering region to obtain a second prediction block. Since the second prediction block is obtained by filtering the first prediction block based on the filtering region, the prediction accuracy of the second prediction block of the image block is higher, thus improving the prediction accuracy of the image block's prediction blocks. Attached Figure Description

[0020] Figure 1 is a schematic diagram of the encoding and decoding system provided in an embodiment of this application;

[0021] Figure 2 is a schematic diagram of the encoder provided in an embodiment of this application;

[0022] Figure 3 is a schematic diagram of the decoder provided in an embodiment of this application;

[0023] Figure 4 is a flowchart of an image block processing method provided in an embodiment of this application;

[0024] Figure 5 is a schematic diagram of the filtering region provided in an embodiment of this application;

[0025] Figure 6 is a structural diagram of an image block processing device provided in an embodiment of this application;

[0026] Figure 7 is a structural diagram of an electronic device provided in an embodiment of this application;

[0027] Figure 8 is a structural diagram of a terminal provided in an embodiment of this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0029] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited in number; for example, the first object can be one or more. Furthermore, "or" in this application indicates at least one of the connected objects. For example, the scope of protection for "A or B" covers at least three scenarios: Scenario 1: including A but not B; Scenario 2: including B but not A; Scenario 3: including both A and B. In addition, the terms "A and / or B," "at least one of A and B," and "at least one of A or B" also cover at least the above three scenarios. The character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0030] Figure 1 is a schematic diagram of the encoding / decoding system provided in an embodiment of this application. The technical solution of this application embodiment relates to encoding and decoding (CODEC) video data (including encoding or decoding). The video data includes original unencoded video, encoded video, decoded (e.g., reconstructed) video, or syntax elements, etc.

[0031] As shown in Figure 1, the encoding / decoding system includes a source device 100, which provides encoded video data to the destination device 110 for decoding and display. Specifically, the source device 100 provides video data to the destination device 110 via a communication medium 120. The source device 100 and the destination device 110 may include any one or more of the following: desktop computer, laptop computer, tablet computer, set-top box, mobile phone, wearable device (e.g., smartwatch or wearable camera), television, camera, display device, in-vehicle device, virtual reality (VR) device, augmented reality (AR) device, mixed reality (MR) device, digital media player, video game console, video conferencing equipment, video streaming equipment, broadcast receiver device, broadcast transmitter device, spacecraft, aircraft, robot, satellite, etc.

[0032] In the example of Figure 1, source device 100 includes a data source 101, memory 102, encoder 200, and output interface 104. Destination device 110 includes an input interface 111, decoder 300, memory 113, and display device 114. Source device 100 represents an example of a video encoding device, while destination device 110 represents an example of a video decoding device. In other examples, source device 100 and destination device 110 may not include some of the components shown in Figure 1, or they may include components other than those shown in Figure 1. For example, source device 100 may receive video data from an external data source (such as an external camera). Similarly, destination device 110 may interface with an external display device instead of including an integrated display device. As another example, memory 102 and memory 113 may be external memories.

[0033] Although Figure 1 illustrates the source device 100 and the destination device 110 as separate devices, in some examples, they may be integrated into a single device. In such embodiments, the same hardware or software, separate hardware or software, or any combination thereof may be used to implement the functionality corresponding to the source device 100 and the functionality corresponding to the destination device 110.

[0034] In some examples, source device 100 and destination device 110 can perform unidirectional or bidirectional video transmission. If it is bidirectional video transmission, source device 100 and destination device 110 can operate in a substantially symmetrical manner, that is, each of source device 100 and destination device 110 includes an encoder and a decoder.

[0035] Data source 101 represents the source of video data (i.e., raw, unencoded video data) and provides encoder 200 with a series of images containing video data, which encoder 200 encodes. Data source 101 of source device 100 may include a video acquisition device (such as a video camera), a video archive containing previously acquired raw video, or a video feed interface for receiving video from a video content provider. Alternatively, data source 101 may generate computer graphics-based data as source video, or combine live video, archived video, and computer-generated video. In these cases, encoder 200 encodes the acquired, pre-acquired, or computer-generated video data. Encoder 200 may rearrange the images from the received order (sometimes referred to as the "display order") according to the encoded order. Encoder 200 may generate a bitstream including the encoded video data. Source device 100 may then output the encoded video data to communication medium 120 via output interface 104 for reception or retrieval, for example, by input interface 111 of destination device 110.

[0036] The memory 102 of the source device 100 and the memory 113 of the destination device 110 represent general-purpose memory. In some examples, memory 102 may store raw video data from data source 101, and memory 113 may store decoded video data from decoder 300. Additionally or alternatively, memories 102 and 113 may respectively store software instructions executable by, for example, encoder 200 and decoder 300. Although memories 102 and 113 are shown separately from encoder 200 and decoder 300 in this example, it should be understood that encoder 200 and decoder 300 may also include internal memory for functionally similar or equivalent purposes. If encoder 200 and decoder 300 are deployed on the same hardware device, memories 102 and 113 may be the same memory. Furthermore, memories 102 and 113 may store, for example, encoded video data output from encoder 200 and input to decoder 300. In some examples, portions of memories 102 and 113 may be allocated as one or more video buffers, for example, to store raw, decoded, or encoded video data.

[0037] In some examples, source device 100 can output encoded data from output interface 104 to memory 113. Similarly, destination device 110 can access encoded data from memory 113 via input interface 111. Memory 113 or memory 102 can include any of a variety of distributed or locally accessed data storage media, such as hard drives, Blu-ray discs, digital versatile discs (DVDs), compact disc read-only memory (CD-ROMs), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

[0038] Output interface 104 may include any type of medium or device capable of transmitting encoded video data from source device 100 to destination device 110. For example, output interface 104 may include a transmitter or transceiver, such as an antenna, configured to transmit encoded video data directly from source device 100 to destination device 110 in real time. The encoded video data may be modulated according to the communication standards of a wireless communication protocol and transmitted to destination device 110.

[0039] Communication medium 120 may include transient media, such as wireless broadcasting or wired network transmission. For example, communication medium 120 may include radio frequency (RF) spectrum or one or more physical transmission lines (e.g., cables). Communication medium 120 may form part of a packet-based network (such as a local area network, a wide area network, or a global network such as the Internet). Communication medium 120 may also take the form of a storage medium (e.g., a non-transitory storage medium), such as a hard disk, flash drive, compact disc, digital video disc, Blu-ray disc, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

[0040] In some implementations, the communication medium 120 may include a router, switch, base station, or any other device that can be used to facilitate communication from source device 100 to destination device 110. For example, a server (not shown) may receive encoded video from source device 100 and provide the encoded video data to destination device 110, for example, via network transmission. The server may include (e.g., a web server for a website), a server configured to provide file transfer protocol services (such as File Transfer Protocol (FTP) or File Delivery Over Unidirectional Transport (FLUTE) protocol), a Content Delivery Network (CDN) device, a Hypertext Transfer Protocol (HTTP) server, a Multimedia Broadcast Multicast Services (MBMS) or Evolved Multimedia Broadcast Multicast Service (eMBMS) server, or a Network-attached Storage (NAS) device, etc. The server can implement one or more HTTP streaming protocols, such as MPEG Media Transport (MMT), Dynamic Adaptive Streaming over HTTP (DASH), HTTP Live Streaming (HLS), or Real Time Streaming Protocol (RTSP).

[0041] Destination device 110 can access encoded video data from a server, for example via a wireless channel (e.g., Wireless Fidelity (WIFI) connection) or a wired connection (e.g., Digital Subscriber Line (DSL), Cable Modem, etc.) for accessing encoded video data stored on the server.

[0042] Output interface 104 and input interface 111 can represent a wireless transmitter / receiver, a modem, a wired networking component (e.g., an Ethernet card), a wireless communication component operating according to the Institute of Electrical and Electronics Engineers (IEEE) 802.11 or IEEE 802.15 standard (e.g., ZigBee™ transmission mode), Bluetooth standard, or other physical components. In an example where output interface 104 and input interface 111 include a wireless component, output interface 104 and input interface 111 can be configured to operate according to Wi-Fi, Ethernet, cellular networks (such as 4G (4... th Generation 4G mobile communication networks, Long Term Evolution (LTE), Advanced LTE, 5G (5G) th Generation 5G mobile communication network, sixth generation (6G) th Generation 6G mobile communication networks, etc., are used to transmit data, such as encoded video data.

[0043] The technology provided in this application can be applied to support video encoding and decoding in one or more multimedia applications such as video conferencing, over-the-air television broadcasting, cable television transmission, satellite television transmission, internet streaming video transmission, digital video encoded onto a data storage medium, decoding of digital video stored on a data storage medium, or other applications.

[0044] Destination device 110 receives an encoded video bitstream from communication medium 120 via its input interface 111. The encoded video bitstream may include syntax elements and encoded data units (e.g., sequences, image groups, images, slices, blocks, etc.), where the syntax elements are used to decode the encoded data units to obtain decoded video data. Display device 114 displays the decoded video data to the user. Display device 114 may include a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED) display, or other types of display devices.

[0045] The encoder 200 and decoder 300 can be implemented as one or more of various processing circuits, which may include microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. When the technology is implemented wholly or partially in software, the device may store instructions for the software in a suitable non-transitory computer-readable storage medium and use one or more processors to execute the instructions in hardware to perform the technology provided in the embodiments of this application.

[0046] The encoder 200 and decoder 300 can process based on the following video codec standards: H.263, H.264, H.265 (also known as High Efficiency Video Coding, HEVC), H.266 (also known as Versatile Video Coding, VVC), Moving Picture Experts Group 2 (MPEG-2), MPEG-4, VP8, VP9, ​​Alliance for Open Media Video 1 (AV1), Audio Video Coding Standard 1 (AVS1), AVS2, AVS3, or next-generation video standard protocols. This application embodiment does not specifically limit the specific implementation.

[0047] Typically, encoder 200 and decoder 300 can perform block-based encoding and decoding of images. The term "block" generally refers to a structure that includes data to be processed (e.g., encoded, decoded, or otherwise used during encoding or decoding). For example, a block can include a two-dimensional matrix of samples of luminance or chrominance data. For example, encoder 200 and decoder 300 can encode and decode video data represented in YUV format, where "Y" represents luminance (or Luma), and "U" and "V" are the two components of chrominance (or Chroma), with "U" representing the blue chrominance component (Cb) and "V" representing the red chrominance component (Cr).

[0048] Referring to Figure 2, which is a schematic diagram of the encoder 200 provided in an embodiment of this application, the encoder 200 can be the encoder 200 in Figure 1. In the example of Figure 2, the encoder 200 includes a memory 201, an encoding parameter determination unit 210, a residual generation unit 202, a transform processing unit 203, a quantization unit 204, an inverse quantization unit 205, an inverse transform processing unit 206, a reconstruction unit 207, a filter unit 208, a decoded picture buffer (DPB) 209, and an entropy encoding unit 220.

[0049] The memory 201 can store video data to be encoded. For example, the encoder 200 can receive and store video data from the data source 101 shown in Figure 1. In some examples, the memory 201 can be on the same chip as other components of the encoder 200 (as shown in Figure 2), or it can be on a separate chip from those components.

[0050] The coding parameter determination unit 210 includes a mode selection unit 211, an inter-frame prediction unit 212, and an intra-frame prediction unit 213. The inter-frame prediction unit 212 is used to obtain a first prediction block for the current block using an inter-frame prediction mode. The intra-frame prediction unit 213 is used to obtain a second prediction block for the current block using an intra-frame prediction mode. The mode selection unit 211 is used to obtain a target prediction block based on the first and second prediction blocks and determine the final prediction mode. Furthermore, the coding parameter determination unit 210 may also include other functional units, such as functional units for determining the partitioning method of coding units (CUs), functional units for determining the transformation type of the residual data of the CUs, or functional units for determining the quantization parameters of the residual data of the CUs.

[0051] For ease of description and understanding, in the embodiments of this application, the CU to be processed in the current image is referred to as the current CU, and the image block to be processed in the current CU is referred to as the current block or the image block to be processed. For example, in encoding, it refers to the block currently being encoded; in decoding, it refers to the block currently being decoded.

[0052] Inter-frame prediction unit 212 may include a motion estimation unit and a motion compensation unit. For inter-frame prediction of the current block, the motion estimation unit may perform a motion search to identify one or more matching reference blocks in one or more reference pictures (e.g., one or more previously encoded / decoded pictures stored in DPB 209).

[0053] The motion estimation unit can generate one or more motion vectors (MVs) representing the position of a reference block in a reference image relative to the position of the current block in the current image. The motion compensation unit can then use interpolation to obtain a predicted value with the precision indicated by the motion vectors.

[0054] The encoding parameter determination unit 210 can provide the target prediction block to the residual generation unit 202. The residual generation unit 202 receives the raw uncoded video data of the current block from the memory 201 and calculates the residual between the current block and the target prediction block to obtain the residual block. In some examples, the function of the residual generation unit 202 can be implemented using one or more subtractor circuits that perform binary subtraction.

[0055] As an example, the encoding parameter determination unit 210 can provide the entropy encoding unit 220 with syntax elements representing encoding parameters for encoding. The encoding parameters include one or more of the following: the partitioning method of the CU, the final prediction mode, the transformation type of the residual data of the CU, or the quantization parameters of the residual data of the CU.

[0056] The transformation processing unit 203 transforms the residual block output by the residual generation unit 202 to obtain a transform coefficient block. This transformation may include Discrete Cosine Transform (DCT), integer transformation, direction transformation, or Karhunen-Loeve Transform (KL Transform), etc. In some examples, the encoder 200 may not include the transformation processing unit 203.

[0057] Quantization unit 204 can quantize the transform coefficients in the transform coefficient block according to the quantization parameter (QP) value associated with the current block to generate a quantized transform coefficient block.

[0058] The inverse quantization unit 205 and the inverse transform processing unit 206 can perform inverse quantization and inverse transform on the transform coefficient block, respectively, to obtain the reconstructed residual block. The reconstruction unit 207 can generate a reconstructed block corresponding to the current block based on the reconstructed residual block and the target prediction block generated by the coding parameter determination unit 210.

[0059] Filter unit 208 can perform one or more filter operations on the reconstructed block. For example, filter unit 208 can be a deblocking filter (DBF), an adaptive loop filter (ALF), a sample adaptive offset (SAO) filter, etc. In some examples, encoder 200 may not include filter unit 208.

[0060] Encoder 200 stores the reconstructed image obtained from the reconstructed blocks in DPB 209. For example, in an example where the operation of filter unit 208 is not required, reconstruction unit 207 can store the reconstructed blocks in DPB 209. In an example where the operation of filter unit 208 is required, filter unit 208 can store the filtered reconstructed blocks in DPB 209. Inter-frame prediction unit 212 retrieves the reconstructed image from DPB 209 to perform inter-frame prediction on blocks of subsequent images to be encoded. In some examples, DPB 209 can be replaced with other types of memory.

[0061] Entropy coding unit 220 can entropy code the syntax elements of other components in encoder 200 to output encoded video data. For example, entropy coding unit 220 can entropy code the quantized transform coefficient block from quantization unit 204. As another example, entropy coding unit 220 can entropy code the syntax elements (e.g., motion information for inter-frame prediction or intra-frame mode information for intra-frame prediction) from coding parameter determination unit 210.

[0062] It is understood that the composition of the encoder 200 shown in Figure 2 is only illustrative and does not constitute a limitation on the embodiments of this application.

[0063] Figure 3 is a schematic diagram of the decoder 300 provided in an embodiment of this application. The decoder 300 may be the decoder 300 described in Figure 1. In the example of Figure 3, the decoder 300 includes a Coded Picture Buffer (CPB) 301, an entropy decoding unit 302, a prediction processing unit 310, an inverse quantization unit 303, an inverse transform processing unit 304, a reconstruction unit 305, a filter unit 306, and a DPB 307.

[0064] The entropy decoding unit 302 can receive encoded video data from the CPB 301 and perform entropy decoding on the video data to obtain syntax elements. The syntax elements indicate encoding parameters, including one or more of the following: CU partitioning method, final prediction mode, transformation type of CU residual data, or quantization parameters of CU residual data.

[0065] When the syntax element includes the final prediction mode, the prediction processing unit 310 obtains the final prediction mode. If the final prediction mode is an inter-frame prediction mode, the prediction block of the current CU can be obtained through the inter-frame prediction unit 311 of the prediction processing unit 310; if the final prediction mode is an intra-frame prediction mode, the prediction block of the current CU can be obtained through the intra-frame prediction unit 312 of the prediction processing unit 310. In some examples, the prediction processing unit 310 may also include a unit for performing prediction functions according to other prediction modes.

[0066] CPB 301 can acquire and store encoded video data from the communication medium 120 shown in Figure 1. DPB 307 is used to store decoded images. Optionally, CPB 301 and DPB 307 can be replaced with other types of memory, which are not specifically limited in this application. In some examples, CPB 301 can be on the same chip as other components of decoder 300 (as shown in the figure), or it can be on a separate chip from those components.

[0067] Decoder 300 can perform reconstruction operations on each block individually. Entropy decoding unit 302 can entropy decode the syntax elements and transform information (e.g., QP or transform mode indication) of the quantized transform coefficients to obtain the quantized transform coefficients. Dequantization unit 303 dequantizes the quantized transform coefficients to obtain a transform coefficient block including the transform coefficients. Inverse transform processing unit 304 performs an inverse transform on the transform coefficient block to generate a residual block corresponding to the current block; this inverse transform is the reverse operation of the above transform.

[0068] Reconstruction unit 305 can reconstruct the current block based on the prediction block and the residual block. For example, reconstruction unit 305 can add samples from the residual block to the corresponding samples from the prediction block to reconstruct the current block.

[0069] Filter unit 306 can perform one or more filter operations on the reconstructed block. For example, the type of filter unit 306 can be referenced to the type of filter unit 208, and will not be described again here. In some examples, the operations of filter unit 306 can be skipped.

[0070] Decoder 300 can store the reconstructed image obtained from the reconstructed blocks in DPB 307. For example, in an example where filter unit 306 is not operated, reconstruction unit 305 can store the reconstructed blocks in DPB 307. In an example where filter unit 306 is operated, filter unit 306 can store the filtered reconstructed blocks in DPB 307. Decoder 300 can output the decoded image (e.g., decoded video) from DPB 307 for subsequent rendering on a display device (such as display device 114 of FIG. 1).

[0071] The image block processing method provided in this application embodiment is described below with reference to the accompanying drawings. The image block processing method provided in this application embodiment can be executed by an encoding end, such as the encoder 200 shown in Figure 1 or Figure 2. The image block processing method provided in this application embodiment can be executed by a decoding end, such as the decoder 300 shown in Figure 1 or Figure 3. The encoding end and decoding end can be implemented by software, hardware, or a combination thereof. When implemented by hardware, the encoding end can be referred to as an encoding end device or a video encoding device, and the decoding end can be referred to as a decoding end device or a video decoding device.

[0072] Please refer to Figure 4, which is a flowchart of an image block processing method provided in an embodiment of this application. As shown in Figure 4, it includes the following steps:

[0073] Step 401: Determine the filtering region, wherein the first prediction block of the image block includes the filtering region.

[0074] The aforementioned image block can be an image block in a video frame or an image block in an image. The aforementioned image block can be any image block in a video frame or an image, or it can be an image block to be predicted or processed, such as the current image block, or the aforementioned image block can be a target image block, or the aforementioned image block can be the currently predicted image block, etc.

[0075] The first prediction block mentioned above can be a prediction block of the image block generated by intra-frame prediction mode or inter-frame prediction mode.

[0076] The first prediction block mentioned above includes the filtering region, which may refer to a portion of the first prediction block. In some embodiments, the filtering region may also be the entire region of the first prediction block.

[0077] It should be noted that since the first prediction block is the prediction block of the image block, the filtering region can also be understood as the filtering region of the image block.

[0078] It should be noted that the above-mentioned determination of the filtering region can be made before generating the first prediction block, or it can be made after generating the first prediction block.

[0079] Step 402: Filter the first prediction block based on the filtering region to obtain the second prediction block.

[0080] Wherein, the above-mentioned filtering of the first prediction block based on the filtering region to obtain the second prediction block may be the filtering of the filtering region of the first prediction block to obtain the second prediction block; or, the above-mentioned filtering of the first prediction block based on the filtering region may be the filtering of the first prediction block with the filtering region as the core, that is, the actual filtering process involves not only the filtering region, but also the boundary interval or adjacent region of the filtering region.

[0081] In this embodiment, the above filtering process can be a bilateral filter (BIF). This embodiment of the application does not limit the filtering process. For example, in some implementations, it can also be a value filter, mean filter, Gaussian filter, low-pass filter or high-pass filter, etc.

[0082] In this embodiment, since the second prediction block is obtained by filtering the first prediction block based on the filtering region, the prediction accuracy of the second prediction block of the image block is higher, that is, the prediction accuracy of the prediction block of the image block is improved.

[0083] As an optional implementation, determining the filtering region includes:

[0084] The filtering region is determined based on the information of the image block;

[0085] The information of the image block includes at least one of the following:

[0086] The size information of the image patch and the prediction mode of the image patch.

[0087] The size information of the aforementioned image block can be the dimension information of the image block.

[0088] The prediction mode of the above image patch refers to the prediction mode of the prediction block that generates the above image patch, such as intra-frame prediction mode or inter-frame prediction mode.

[0089] The aforementioned determination of the filtering region based on the image patch information can be based on a pre-acquired mapping relationship between information and the filtering region. For example, the mapping relationship between size information and the filtering region can be pre-configured, or the mapping relationship between the mapping mode and the filtering region can be predicted and configured. Alternatively, the aforementioned determination of the filtering region based on the image patch information can be based on a pre-acquired filtering region division rule to determine the filtering region corresponding to the aforementioned information; no specific limitation is imposed on this.

[0090] It should be noted that since the first prediction block is the prediction block of the image block, the information of the image block can also be called the information of the first prediction block, that is, the size information of the first prediction block, or the prediction mode of the first prediction block.

[0091] In the above embodiments, since the filtering region is determined based on at least one of the image block size information and the prediction mode of the image block, the filtering region can be better matched with the image block, that is, better matched with the first prediction block, thereby improving the filtering effect.

[0092] It should be noted that the embodiments of this application are not limited to determining the filtering region based on the information of the image block. For example, in some implementations, a default filtering region may be used, such as the left half of the first prediction block or the lower half of the first prediction block.

[0093] In some implementations, the size information of the image patch includes at least one of the following:

[0094] The width of the image block;

[0095] The height of the image block;

[0096] The area of ​​the image block.

[0097] The width of the image block can also refer to the width of the first prediction block, the height of the image block can also refer to the height of the first prediction block, and the area of ​​the image block can also refer to the area of ​​the first prediction block.

[0098] In this embodiment, the filtering region can be determined based on at least one of the width, height, and area of ​​the image block, so that the filtering region matches the image block better, that is, matches the first prediction block better, thereby improving the filtering effect.

[0099] In some implementations, the above-mentioned filtering region satisfies one of the following:

[0100] When the width of the image patch is greater than or equal to the height of the image patch, the width of the filtering region is w1 times the width of the image patch, where w1 is a real number less than 1 and greater than 0.

[0101] When the width of the image patch is less than the height of the image patch, the width of the filtering region is w2 times the width of the image patch, where w2 is a real number less than 1 and greater than 0.

[0102] When the height of the image block is less than or equal to the width of the image block, the height of the filtering region is h1 times the height of the image block, where h1 is a real number less than 1 and greater than 0.

[0103] When the height of the image block is greater than the width of the image block, the height of the filtering region is h² times the height of the image block, where h² is a real number less than 1 and greater than 0.

[0104] The values ​​of w1 and w2 can be predefined, such as 1 / 2, 1 / 3, 1 / 4, etc. If the value of w1 is 1 / 2, then the width of the filtering region is half the width of the image block, and the filtering region is the left region of the image block. In some embodiments, it can also be the right region.

[0105] The values ​​of h1 and h2 can be predefined, such as 1 / 2, 1 / 3, 1 / 4, etc. If the value of h1 is 1 / 2, then the height of the filtering region is half the height of the image block, and the filtering region is the upper region of the image block. In some embodiments, it can also be the lower region.

[0106] In this embodiment, at least one of the width and height of the filtering region can be determined based on the ratio between the width and height of the image block, which can make the filtering region more suitable and thus improve the filtering effect.

[0107] In some implementations, if the width of the filtering region is greater than a preset width, the width of the filtering region can be determined to be 1 / 2, 1 / 3, or 1 / 4 of the width of the image block.

[0108] In some implementations, if the height of the filtering region is greater than a preset height, the width of the filtering region can be determined to be 1 / 2, 1 / 3, or 1 / 4 of the width of the image block.

[0109] In some implementations, at least one of the width and height of the filtering region described above may be predefined or use a default value.

[0110] In some implementations, the filtering region is configured such that, when the area is greater than a preset area, the width of the filtering region is determined to be 1 / 2, 1 / 3, or 1 / 4 of the width of the image block.

[0111] In some implementations, the prediction mode includes an intra-frame prediction mode, and the filtering region satisfies one of the following:

[0112] When the intra-frame prediction mode is the vertical angle prediction mode, the width of the filtering region is w3 times the width of the image patch, where w3 is a real number less than 1 and greater than 0.

[0113] When the intra-frame prediction mode is the horizontal angle prediction mode, the height of the filtering region is h3 times the height of the image block, where h3 is a real number less than 1 and greater than 0.

[0114] The value of w2 can be predefined, such as 1 / 2, 1 / 3, 1 / 4, etc. If the value of w3 is 1 / 2, then the width of the filtering region is half the width of the image block.

[0115] The value of h3 can be predefined, such as 1 / 2, 1 / 3, 1 / 4, etc. If the value of h3 is 1 / 2, then the height of the filter region is half the height of the image block.

[0116] In this embodiment, since the width of the filtering region is determined based on the vertical angle prediction pattern, the filtering region can better match the prediction pattern, thereby improving the filtering effect. Furthermore, since the height of the filtering region is determined based on the horizontal angle prediction pattern, the filtering region can better match the prediction pattern, further improving the filtering effect.

[0117] It should be noted that in some implementations, when the prediction mode is an inter-frame prediction mode, the filtering region can be a preset or default region.

[0118] In some implementations, the filtering region satisfies one of the following:

[0119] When the intra-frame prediction mode is a vertical angle prediction mode, and the intra-frame prediction mode is greater than the vertical mode, the width of the filtering region is w3 times the width of the image patch.

[0120] When the intra-frame prediction mode is a horizontal angle prediction mode, and the intra-frame prediction mode is smaller than the horizontal mode, the height of the filtering region is h3 times the height of the image block.

[0121] The aforementioned vertical mode refers to the vertical mode in the intra-frame prediction mode, and the aforementioned horizontal mode refers to the horizontal mode in the intra-frame prediction mode.

[0122] Wherein, the above-mentioned intra-prediction mode being greater than the vertical mode can be that the direction of the intra-prediction mode is to the right of the direction of the vertical mode, the above-mentioned intra-prediction mode being greater than the vertical mode can be that the index or number of the intra-prediction mode is greater than the index or number of the vertical mode, or the above-mentioned intra-prediction mode being greater than the vertical mode can be that the angle corresponding to the intra-prediction mode is greater than the angle corresponding to the vertical mode.

[0123] Wherein, the above-mentioned intra-prediction mode being less than the horizontal mode can be that the direction of the intra-prediction mode is below the direction of the horizontal mode, the above-mentioned intra-prediction mode being less than the horizontal mode can be that the index or number of the intra-prediction mode is less than the index or number of the horizontal mode, or the above-mentioned intra-prediction mode being less than the horizontal mode can be that the angle corresponding to the intra-prediction mode is less than the angle corresponding to the horizontal mode.

[0124] In this implementation, since the intra-frame prediction mode is greater than the vertical mode, the width of the filtering region is w³ times the width of the image patch. This allows the filtering region to better match the prediction mode, thereby improving the filtering effect. Furthermore, since the intra-frame prediction mode is less than the horizontal mode, the height of the filtering region is h³ times the height of the image patch. This also allows the filtering region to better match the prediction mode, further improving the filtering effect.

[0125] It should be noted that, in the embodiments of this application, the intra-frame prediction mode is not limited to the angle prediction mode; for example, it can also be a non-angle intra-frame prediction mode.

[0126] As an optional implementation, the method further includes:

[0127] Determine at least one of the first scaling factor and the second scaling factor for the filtering process;

[0128] Wherein, the first scaling factor is determined based on the predicted value of the filtered region, or the first scaling factor is determined based on the predicted value of the image patch;

[0129] The second scaling factor is determined based on the size information of the image block.

[0130] The predicted value of the aforementioned filtering region may refer to the predicted value of the filtering region in the first prediction block, and the predicted value of the aforementioned image block may refer to the value of the aforementioned first prediction block.

[0131] The first and second scaling factors mentioned above can be scaling factors in BIF filtering, for example:

[0132] The first scaling factor mentioned above is in BIF filtering. The second scaling factor mentioned above is in BIF filtering.

[0133] In some implementations, the first scaling factor is determined based on the predicted value of the filtered region. This can be achieved by looking up the predicted value of the filtered region in a lookup table. For example, the first scaling factor can be determined by looking up the predicted value of the filtered region in the corresponding lookup table (LUT).

[0134] In some implementations, the first scaling factor is determined based on the predicted value of the image patch. This can be achieved by looking up the predicted value of the image patch in a table. For example, the second scaling factor can be determined by using the predicted value of the image patch in the corresponding LUT.

[0135] In some implementations, the second scaling factor is determined based on the size information of the image block. This can be achieved by determining the first scaling factor through a lookup table based on the size information of the image block. For example, the second scaling factor can be determined by using the width and height of the image block in the corresponding LUT.

[0136] In some implementations, at least one of the first scaling factor and the second scaling factor can be determined by the following formula:

[0137] Among them, the above The second proportional factor mentioned above, the above The first scaling factor mentioned above, the width mentioned above TU The width of the aforementioned image patch, and the aforementioned height TU The height of the above image block, the above LUT w,hFor the table corresponding to the second scaling factor mentioned above, the above LUT MAD For the table corresponding to the i-th scaling factor mentioned above, MAD TU The mean absolute deviation (MAD) is the difference between the predicted values ​​of the image patch or the predicted values ​​of the filtered region.

[0138] In some implementations, the above-mentioned LUT w,h and LUT MAD Predefined tables, such as protocol agreement tables.

[0139] In some implementations, the aforementioned MAD TU It can be determined using the following formula:

[0140] Where h and w represent the height and width of the aforementioned image patch, respectively, and s i,j This represents the predicted value of the pixel at coordinates (i, j) in the image block or filtered region, where i and j represent the i-th row and j-th column of the image block or filtered region, respectively.

[0141] It should be noted that in some implementations, the aforementioned MAD TU The MAD is not limited to being calculated using the above formula; in some implementations, the above MAD... TU Alternatively, it can be determined by looking up a table.

[0142] It should be noted that in some implementation methods, the above... and The determination is not limited to the formula described above. For example, in some implementations, the above formula can also be used to determine the determination. and

[0143] Among them, a and b are preset constants, which can be determined based on empirical values.

[0144] Since the first scaling factor is determined based on the predicted value of the filtering region or the predicted value of the image block, and the second scaling factor is determined based on the size information of the image block, the scaling factor used in the filtering process can be better matched with the image block, thereby improving the filtering effect.

[0145] It should be noted that the embodiments of this application are not limited to the filtering method based on the first scaling factor and the second scaling factor described above. For example, in some embodiments, the BIF filtering process agreed upon in the protocol can be used to perform filtering based on the first scaling factor and the second scaling factor. That is, the application of the first scaling factor and the second scaling factor in the BIF filtering process is agreed upon in the protocol. For example, a scale factor can be calculated based on the first scaling factor and the second scaling factor, and then BIF filtering can be performed based on the scale factor. For example, the scale factor can be calculated in the following way:

[0146] Among them, C TU The above is the scaling factor.

[0147] Alternatively, in some implementations, BIF filtering can be performed using a newly defined protocol based on the first and second scaling factors described above.

[0148] It should be noted that the above implementation method is mainly described using BIF filtering. In some implementation methods, other filtering methods can also be performed by determining the corresponding parameters based on the predicted values ​​of the above image blocks or prediction blocks, and then performing filtering based on those parameters.

[0149] As an optional implementation, before filtering the first prediction block based on the filtering region to obtain the second prediction block, the method further includes:

[0150] The boundary region of the filtering region of the first prediction block is filled.

[0151] In this context, the boundary region of the aforementioned filtering region can be understood as the adjacent region of the filtering region. Taking Figure 5 as an example, the first prediction block is the region shown in 501, the filtering region is the black region shown in 502, and the boundary region is the area surrounding the filtering region.

[0152] The above-mentioned filling of the boundary region of the filtering region of the first prediction block can be done by filling all or part of the boundary region of the filtering region of the first prediction block.

[0153] The above-mentioned filling of the boundary area can be done using the default value.

[0154] In this embodiment, since the boundary region of the filtering region is filled before the filtering process, the area involved in the filtering process can be larger. That is, filtering process can be carried out based on the above-mentioned filtering region and the above-mentioned boundary region, so as to support more types of filters and improve the filtering effect.

[0155] In some implementations, the predicted values ​​of the sub-regions belonging to the first prediction block in the boundary region of the filtered region are used to fill the sub-regions.

[0156] In this embodiment, the predicted values ​​of the sub-regions are used for filling, which makes the boundary prediction values ​​of the filtering process more accurate and helps to improve the filtering effect.

[0157] The aforementioned sub-region can be region 503 as shown in Figure 5.

[0158] The above-mentioned filling of the sub-region with the predicted value of the sub-region can be understood as the filling process being the prediction process of the sub-region, because the predicted value of the sub-region was already obtained when the first prediction block was generated, that is, the filling of the sub-region was achieved.

[0159] In this implementation, since the predicted values ​​of the aforementioned sub-regions are filled in, no additional values ​​need to be filled in for the sub-regions, thus saving computational resources.

[0160] It should be noted that the various implementation methods described above can be combined with each other or implemented individually, and there is no limitation on this.

[0161] In this embodiment, a filtering region is determined, wherein a first prediction block of an image block includes the filtering region; the first prediction block is filtered based on the filtering region to obtain a second prediction block. Since the second prediction block is obtained by filtering the first prediction block based on the filtering region, the prediction accuracy of the second prediction block of the image block is higher, thus improving the prediction accuracy of the image block's prediction blocks.

[0162] The following example illustrates the method provided in this application, using intra-frame prediction mode at the encoding end and the aforementioned image block as the target image block:

[0163] Example 1:

[0164] Based on the intra-frame prediction mode, width, height, area, and other information of the target image block (i.e., the image block in the above embodiment), the filtering region of the prediction block of the target image block is determined.

[0165] In some implementations, a filtering region of the target image block is determined based on information about the target image block, and a first prediction block of the target image block is filtered based on the filtering region of the target image block to obtain a filtered second prediction block.

[0166] The aforementioned first information includes at least one of the following:

[0167] Width, height, and area of ​​the target image patch;

[0168] Intra-frame prediction mode of the target image patch;

[0169] In some implementations, if the width of the target image block is greater than or equal to its height, then the width is equal to half (or 1 / 3, etc.) of the original width; or, if the width of the target image block is less than or equal to its height, then the width is equal to half (or 1 / 3, etc.) of the original width.

[0170] If the height of the target image patch is less than or equal to its width, then the height is equal to half (or 1 / 3, etc.) of the original height; or, if the height of the target image patch is greater than or equal to its width, then the height is equal to half (or 1 / 3, etc.) of the original height.

[0171] In some implementations, if the intra-frame prediction mode is a vertical angular prediction mode, then the width is half of the original width; or, if the intra-frame prediction mode is a vertical angular prediction mode and is larger than the vertical mode, then the width is half of the original width.

[0172] If the intra-frame prediction mode is a horizontal angular prediction mode, then the height is equal to half of the original height; or, if the intra-frame prediction mode is a horizontal angular prediction mode and is smaller than the horizontal mode, then the height is equal to half of the original height.

[0173] In some implementations, for BIF filtering, filtering is performed on the first predicted block of the target image patch within the filtering region based on the target image patch, further including:

[0174] Sure and

[0175] in, Calculate the predicted value using only the filtered region. Determined using the width and height of the target image patch; or,

[0176] Calculate the predicted value using the target image patch. Determined using the width and height of the target image patch;

[0177] in, and The corresponding descriptions of the above embodiments are not repeated here.

[0178] In some implementations, filtering is performed on the first prediction block of the target image block based on the filtering region of the target image block, and this process further includes:

[0179] Fill the boundaries of the filtered region;

[0180] The boundary of the aforementioned filtering region can also be called the filter region boundary.

[0181] The right boundary is filled with the predicted pixels of its location, as shown in Figure 5.

[0182] The image block processing method provided in this application can be executed by an image block processing device. As an example, the device can be an electronic device or a component within an electronic device, such as a chip or circuit. This application uses an image block processing device executing the image block processing method as an example to illustrate the image block processing device provided in this application.

[0183] Please refer to Figure 6, which is a structural diagram of an image block processing apparatus 600 provided in an embodiment of this application. As shown in Figure 6, it includes:

[0184] The first determining module 601 is used to determine the filtering region, wherein the first prediction block of the image block includes the filtering region;

[0185] The processing module 602 is used to perform filtering processing on the first prediction block based on the filtering region to obtain a second prediction block.

[0186] Optionally, the first determining module 601 is used to determine the filtering region based on the information of the image block;

[0187] The information of the image block includes at least one of the following:

[0188] The size information of the image patch and the prediction mode of the image patch.

[0189] Optionally, the size information of the image patch includes at least one of the following:

[0190] The width of the image block;

[0191] The height of the image block;

[0192] The area of ​​the image block.

[0193] Optionally, the filtering region satisfies one of the following:

[0194] When the width of the image patch is greater than or equal to the height of the image patch, the width of the filtering region is w1 times the width of the image patch, where w1 is a real number less than 1 and greater than 0.

[0195] When the width of the image patch is less than the height of the image patch, the width of the filtering region is w2 times the width of the image patch, where w2 is a real number less than 1 and greater than 0.

[0196] When the height of the image block is less than or equal to the width of the image block, the height of the filtering region is h1 times the height of the image block, where h1 is a real number less than 1 and greater than 0.

[0197] When the height of the image block is greater than the width of the image block, the height of the filtering region is h² times the height of the image block, where h² is a real number less than 1 and greater than 0.

[0198] Optionally, the prediction mode includes an intra-frame prediction mode, and the filtering region satisfies one of the following:

[0199] When the intra-frame prediction mode is a vertical angle prediction mode, the width of the filtering region is w3 times the width of the image patch, where w3 is a real number less than 1 and greater than 0.

[0200] When the intra-frame prediction mode is the horizontal angle prediction mode, the height of the filtering region is h3 times the height of the image block, where h3 is a real number less than 1 and greater than 0.

[0201] Optionally, the filtering region satisfies one of the following:

[0202] When the intra-frame prediction mode is a vertical angle prediction mode, and the intra-frame prediction mode is greater than the vertical mode, the width of the filtering region is w3 times the width of the image patch.

[0203] When the intra-frame prediction mode is a horizontal angle prediction mode, and the intra-frame prediction mode is smaller than the horizontal mode, the height of the filtering region is h3 times the height of the image block.

[0204] Optionally, the device further includes:

[0205] The second determining module is used to determine at least one of the first scaling factor and the second scaling factor of the filtering process;

[0206] Wherein, the first scaling factor is determined based on the predicted value of the filtered region, or the first scaling factor is determined based on the predicted value of the image patch;

[0207] The second scaling factor is determined based on the size information of the image block.

[0208] Optionally, the device further includes:

[0209] A filling module is used to fill the boundary region of the filtering region of the first prediction block.

[0210] Optionally, for the sub-regions belonging to the first prediction block in the boundary region of the filtered region, the predicted values ​​of the sub-regions are used for filling.

[0211] The aforementioned image block processing device can improve the prediction accuracy of prediction blocks.

[0212] The image block processing apparatus provided in this application embodiment can implement the various processes implemented in the method embodiment shown in FIG1 and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0213] As shown in Figure 7, this application embodiment also provides an electronic device 700, including a processor 701 and a memory 702. The memory 702 stores a program or instructions that can run on the processor 701. For example, when the electronic device 700 is an encoding device, the program or instructions executed by the processor 701 implement the various steps of the above-described image block processing method embodiment and achieve the same technical effect. When the electronic device 700 is a decoding device, the program or instructions executed by the processor 701 implement the various steps of the above-described image block processing method embodiment and achieve the same technical effect. To avoid repetition, further details are omitted here.

[0214] This application also provides an electronic device, including: a memory configured to store video data; and a processing circuit configured to implement the various steps of the image block processing method embodiments described above.

[0215] This application also provides an electronic device, including a processor and a communication interface. The communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the steps in the method embodiment shown in FIG4. This device embodiment corresponds to the above method embodiment, and all implementation processes and methods of the above method embodiments can be applied to this terminal embodiment and achieve the same technical effect.

[0216] The processor or processing circuit in this application embodiment may include general-purpose processors, special-purpose processors, etc., such as central processing units (CPUs), microprocessors, digital signal processors (DSPs), artificial intelligence (AI) processors, graphics processing units (GPUs), application-specific integrated circuits (ASICs), network processors (NPs), field-programmable gate arrays (FPGAs), or other programmable logic devices, gate circuits, transistors, discrete hardware components, etc. The communication interface in this application embodiment may include transceivers, pins, circuits, buses, etc.

[0217] The aforementioned electronic devices can be terminals or other devices besides terminals, such as servers, network attached storage (NAS), etc.

[0218] Among them, the terminal can also be called user equipment (UE), which can be a mobile phone, tablet computer, laptop computer, notebook computer, personal digital assistant (PDA), handheld computer, netbook, ultra-mobile personal computer (UMPC), mobile internet device (MID), augmented reality (AR), virtual reality (VR) device, mixed reality (MR) device, robot, wearable device, flight vehicle, vehicle user equipment (VUE), shipborne equipment, pedestrian user equipment (PUE), smart home (home devices with wireless communication functions, such as refrigerators, televisions, washing machines or furniture, etc.), game console, personal computer (PC), ATM or self-service machine, etc. Wearable devices include: smartwatches, smart bracelets, smart earphones, smart glasses, smart jewelry (smart bracelets, smart chains, smart rings, smart necklaces, smart anklets, smart anklets, etc.), smart wristbands, smart clothing, etc. Among these, in-vehicle devices can also be referred to as in-vehicle terminals, in-vehicle controllers, in-vehicle modules, in-vehicle components, in-vehicle chips, or in-vehicle units, etc. It should be noted that the embodiments in this application do not limit the specific type of terminal.

[0219] A server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server. A cloud server can provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), or cloud computing services based on big data and artificial intelligence platforms.

[0220] Taking an electronic device as an example, Figure 8 is a schematic diagram of the hardware structure of a terminal implementing an embodiment of this application.

[0221] The terminal 800 includes, but is not limited to, at least some of the following components: radio frequency unit 801, network module 802, audio output unit 803, input unit 804, sensor 805, display unit 806, user input unit 807, interface unit 808, memory 809, and processor 810.

[0222] Those skilled in the art will understand that the terminal 800 may also include a power supply (such as a battery) for powering various components. The power supply can be logically connected to the processor 810 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The terminal structure shown in Figure 8 does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0223] It should be understood that, in this embodiment, the input unit 804 may include a graphics processor 8041 and a microphone 8042. The graphics processor 8041 processes image data of still images or videos obtained by an image acquisition device (such as a camera) in video acquisition mode or image acquisition mode, or it may process the obtained point cloud data. The display unit 806 may include a display panel 8061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 807 includes at least one of a touch panel 8071 and other input devices 8072. The touch panel 8071 is also called a touch screen. The touch panel 8071 may include two parts: a touch detection device and a touch controller. Other input devices 8072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0224] In this embodiment, after receiving downlink data from the network-side device, the radio frequency unit 801 can transmit it to the processor 810 for processing; in addition, the radio frequency unit 801 can send uplink data to the network-side device. Typically, the radio frequency unit 801 includes, but is not limited to, antennas, amplifiers, transceivers, couplers, low-noise amplifiers, duplexers, etc.

[0225] The memory 809 can be used to store software programs or instructions, as well as various data. The memory 809 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 809 may include volatile memory or non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 809 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0226] Processor 810 may include one or more processing units; optionally, processor 810 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 810.

[0227] The processor 810 is configured to determine a filtering region, wherein a first prediction block of an image block includes the filtering region; and to perform filtering processing on the first prediction block based on the filtering region to obtain a second prediction block.

[0228] Optionally, determining the filtering region includes:

[0229] The filtering region is determined based on the information of the image block;

[0230] The information of the image block includes at least one of the following:

[0231] The size information of the image patch and the prediction mode of the image patch.

[0232] Optionally, the size information of the image patch includes at least one of the following:

[0233] The width of the image block;

[0234] The height of the image block;

[0235] The area of ​​the image block.

[0236] Optionally, the filtering region satisfies one of the following:

[0237] When the width of the image patch is greater than or equal to the height of the image patch, the width of the filtering region is w1 times the width of the image patch, where w1 is a real number less than 1 and greater than 0.

[0238] When the width of the image patch is less than the height of the image patch, the width of the filtering region is w2 times the width of the image patch, where w2 is a real number less than 1 and greater than 0.

[0239] When the height of the image block is less than or equal to the width of the image block, the height of the filtering region is h1 times the height of the image block, where h1 is a real number less than 1 and greater than 0.

[0240] When the height of the image block is greater than the width of the image block, the height of the filtering region is h² times the height of the image block, where h² is a real number less than 1 and greater than 0.

[0241] Optionally, the prediction mode includes an intra-frame prediction mode, and the filtering region satisfies one of the following:

[0242] When the intra-frame prediction mode is a vertical angle prediction mode, the width of the filtering region is w3 times the width of the image patch, where w3 is a real number less than 1 and greater than 0.

[0243] When the intra-frame prediction mode is the horizontal angle prediction mode, the height of the filtering region is h3 times the height of the image block, where h3 is a real number less than 1 and greater than 0.

[0244] Optionally, the filtering region satisfies one of the following:

[0245] When the intra-frame prediction mode is a vertical angle prediction mode, and the intra-frame prediction mode is greater than the vertical mode, the width of the filtering region is w3 times the width of the image patch.

[0246] When the intra-frame prediction mode is a horizontal angle prediction mode, and the intra-frame prediction mode is smaller than the horizontal mode, the height of the filtering region is h3 times the height of the image block.

[0247] Optionally, the processor 810 is further configured to determine at least one of a first scaling factor and a second scaling factor for the filtering process;

[0248] Wherein, the first scaling factor is determined based on the predicted value of the filtered region, or the first scaling factor is determined based on the predicted value of the image patch;

[0249] The second scaling factor is determined based on the size information of the image block.

[0250] Optionally, the processor 810 is also configured to fill the boundary regions of the filtered region of the first prediction block.

[0251] Optionally, for the sub-regions belonging to the first prediction block in the boundary region of the filtered region, the predicted values ​​of the sub-regions are used for filling.

[0252] The aforementioned electronic devices can improve the prediction accuracy of the prediction blocks.

[0253] It is understood that the implementation process of each implementation method mentioned in this embodiment can refer to the relevant description of the method embodiment and achieve the same or corresponding technical effect. To avoid repetition, it will not be described again here.

[0254] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image block processing method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0255] The processor mentioned above is the processor in the terminal described in the above embodiments. The readable storage medium includes computer-readable storage media, such as ROM, RAM, magnetic disk, or optical disk. In some examples, the readable storage medium may be a non-transient readable storage medium.

[0256] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described image block processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0257] It should be understood that the chips mentioned in the embodiments of this application may include system-on-a-chip (also known as system chip, chip system, or system-on-a-chip) or discrete display chips, etc.

[0258] This application also provides a computer program / program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described image block processing method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0259] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0260] From the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of computer software products plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, magnetic disk, optical disk, etc.), and the computer software product includes several instructions to cause the terminal or network-side device to execute the methods described in the various embodiments of this application.

[0261] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other implementations under the guidance of this application without departing from the spirit and scope of the claims. All of these implementations are within the protection scope of this application.

Claims

1. An image patch processing method, comprising: Determine the filtering region, wherein the first prediction block of the image block includes the filtering region; The first prediction block is filtered based on the filtering region to obtain the second prediction block.

2. The method according to claim 1, characterized in that, The defined filtering region includes: The filtering region is determined based on the information of the image block; The information of the image block includes at least one of the following: The size information of the image patch and the prediction mode of the image patch.

3. The method according to claim 2, wherein, The size information of the image block includes at least one of the following: The width of the image block; The height of the image block; The area of ​​the image block.

4. The method according to claim 3, wherein, The filtering region satisfies one of the following conditions: When the width of the image patch is greater than or equal to the height of the image patch, the width of the filtering region is w1 times the width of the image patch, where w1 is a real number less than 1 and greater than 0. When the width of the image patch is less than the height of the image patch, the width of the filtering region is w2 times the width of the image patch, where w2 is a real number less than 1 and greater than 0. When the height of the image block is less than or equal to the width of the image block, the height of the filtering region is h1 times the height of the image block, where h1 is a real number less than 1 and greater than 0. When the height of the image block is greater than the width of the image block, the height of the filtering region is h² times the height of the image block, where h² is a real number less than 1 and greater than 0.

5. The method according to any one of claims 2 to 4, wherein, The prediction mode includes an intra-frame prediction mode, and the filtering region satisfies one of the following: When the intra-frame prediction mode is the vertical angle prediction mode, the width of the filtering region is w3 times the width of the image patch, where w3 is a real number less than 1 and greater than 0. When the intra-frame prediction mode is the horizontal angle prediction mode, the height of the filtering region is h3 times the height of the image block, where h3 is a real number less than 1 and greater than 0.

6. The method according to claim 5, wherein, The filtering region satisfies one of the following conditions: When the intra-frame prediction mode is a vertical angle prediction mode, and the intra-frame prediction mode is greater than the vertical mode, the width of the filtering region is w3 times the width of the image patch. When the intra-frame prediction mode is a horizontal angle prediction mode, and the intra-frame prediction mode is smaller than the horizontal mode, the height of the filtering region is h3 times the height of the image block.

7. The method according to any one of claims 1 to 6, further comprising: Determine at least one of the first scaling factor and the second scaling factor for the filtering process; Wherein, the first scaling factor is determined based on the predicted value of the filtered region, or the first scaling factor is determined based on the predicted value of the image patch; The second scaling factor is determined based on the size information of the image block.

8. The method according to any one of claims 1 to 7, wherein, Before filtering the first prediction block based on the filtering region to obtain the second prediction block, the method further includes: The boundary region of the filtering region of the first prediction block is filled.

9. The method according to claim 8, wherein, For the sub-regions belonging to the first prediction block within the boundary region of the filtered region, the predicted values ​​of the sub-regions are used for filling.

10. An image patch processing apparatus, comprising: A first determining module is used to determine a filtering region, wherein a first prediction block of an image block includes the filtering region; The processing module is used to filter the first prediction block based on the filtering region to obtain the second prediction block.

11. The apparatus according to claim 10, wherein, The first determining module is used to determine the filtering region based on the information of the image block; The information of the image block includes at least one of the following: The size information of the image patch and the prediction mode of the image patch.

12. The apparatus according to claim 10 or 11, further comprising: The second determining module is used to determine at least one of the first scaling factor and the second scaling factor of the filtering process; Wherein, the first scaling factor is determined based on the predicted value of the filtered region, or the first scaling factor is determined based on the predicted value of the image patch; The second scaling factor is determined based on the size information of the image block.

13. The apparatus according to any one of claims 10 to 12, further comprising: A filling module is used to fill the boundary region of the filtering region of the first prediction block.

14. The apparatus according to claim 11, wherein, The size information of the image block includes at least one of the following: The width of the image block; The height of the image block; The area of ​​the image block.

15. The apparatus according to claim 14, wherein, The filtering region satisfies one of the following conditions: When the width of the image patch is greater than or equal to the height of the image patch, the width of the filtering region is w1 times the width of the image patch, where w1 is a real number less than 1 and greater than 0. When the width of the image patch is less than the height of the image patch, the width of the filtering region is w2 times the width of the image patch, where w2 is a real number less than 1 and greater than 0. When the height of the image block is less than or equal to the width of the image block, the height of the filtering region is h1 times the height of the image block, where h1 is a real number less than 1 and greater than 0. When the height of the image block is greater than the width of the image block, the height of the filtering region is h² times the height of the image block, where h² is a real number less than 1 and greater than 0.

16. The apparatus according to any one of claims 11, 14, and 15, wherein, The prediction mode includes an intra-frame prediction mode, and the filtering region satisfies one of the following: When the intra-frame prediction mode is the vertical angle prediction mode, the width of the filtering region is w3 times the width of the image patch, where w3 is a real number less than 1 and greater than 0. When the intra-frame prediction mode is the horizontal angle prediction mode, the height of the filtering region is h3 times the height of the image block, where h3 is a real number less than 1 and greater than 0.

17. The apparatus according to claim 16, wherein, The filtering region satisfies one of the following conditions: When the intra-frame prediction mode is a vertical angle prediction mode, and the intra-frame prediction mode is greater than the vertical mode, the width of the filtering region is w3 times the width of the image patch. When the intra-frame prediction mode is a horizontal angle prediction mode, and the intra-frame prediction mode is smaller than the horizontal mode, the height of the filtering region is h3 times the height of the image block.

18. The apparatus according to claim 13, wherein, For the sub-regions belonging to the first prediction block within the boundary region of the filtered region, the predicted values ​​of the sub-regions are used for filling.

19. An electronic device comprising a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the image block processing method as claimed in any one of claims 1 to 9.

20. A readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the image block processing method as described in any one of claims 1 to 9.

21. A chip comprising a processor and a communication interface coupled to the processor, the processor being configured to run a program or instructions to implement the steps of the image block processing method as described in any one of claims 1 to 9.

22. A computer program product stored in a storage medium, the computer program product being executed by at least one processor to implement the steps of the image block processing method as claimed in any one of claims 1 to 9.

Images