Processing method, processing device, and storage medium

Abstract

The application provides a processing method, a processing device and a storage medium. The processing method can be applied to the processing device and includes: performing prediction processing on at least one sub-block of a current block. The technical scheme can improve the prediction effect on an image block with a large size, and thus can support improving the prediction effect of video encoding and / or decoding.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to a processing method, processing device, and storage medium. Background Technology

[0002] The existing video coding standard (H.266 / VVC) proposes a video frame coding technique. For example, when encoding and decoding video frames, the protocol divides each frame into different blocks and performs prediction processing and encoding / decoding processing.

[0003] In conceiving and implementing this application, the inventors discovered at least the following problem: the prediction effect for large image blocks is not ideal during intra-frame prediction and / or inter-frame prediction, which in turn leads to poor encoding and / or decoding quality in the video encoding and / or decoding process.

[0004] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides a processing method, processing device, and storage medium that can improve the prediction performance for large image blocks, thereby supporting improved prediction performance for video encoding and / or decoding.

[0006] This application provides a processing method applicable to a processing device, comprising the following steps:

[0007] S10, perform prediction processing on at least one sub-block of the current block.

[0008] Optionally, step S10 includes at least one of the following:

[0009] Based on at least one prediction pattern, perform prediction processing on at least one sub-block of the current block;

[0010] The prediction result of the current block is determined or obtained based on at least one reference region, at least one reference block and / or at least one reference pixel of at least one sub-block of the current block.

[0011] Optionally, prediction processing is performed on at least one sub-block of the current block according to at least one prediction mode, including at least one of the following:

[0012] Based on the same prediction pattern, at least one sub-block of the current block is predicted.

[0013] Based on the prediction pattern corresponding to the first sub-block of the current block, perform prediction processing on the second sub-block of the current block.

[0014] Optionally, the method of determining or obtaining the prediction model includes at least one of the following:

[0015] The prediction mode is determined or obtained based on at least one reference region, at least one reference block and / or at least one reference pixel of at least one sub-block of the current block;

[0016] Based on the prediction patterns corresponding to the adjacent blocks, non-adjacent blocks, and / or the first sub-block of the current block, determine or obtain the prediction pattern corresponding to the second sub-block of the current block;

[0017] The predicted pattern is determined or obtained based on the list of most likely patterns and / or the list of non-most likely patterns corresponding to the current block;

[0018] Based on the list of most likely patterns corresponding to the first sub-block of the current block, determine or obtain the prediction pattern corresponding to the second sub-block of the current block.

[0019] Optionally, the prediction model includes at least one of the following:

[0020] Decoding-side intra-frame mode derivation mode;

[0021] Derivation of the mode based on the intra-frame coding mode of the event;

[0022] Template-based intra-frame coding mode derivation;

[0023] Local illumination compensation mode;

[0024] Intra-frame prediction mode based on neural networks.

[0025] Optionally, the first model is applied to a neural network-based intra-frame prediction mode.

[0026] Optionally, the first model is determined or obtained based on at least one of the following:

[0027] The current block's width, height, block size, and block area must be at least one of these.

[0028] The width, height, size, and area of ​​at least one reference region of the current block;

[0029] The width, height, size, and area of ​​at least one of the reference blocks of the current block.

[0030] Optionally, the reference region, reference block, and / or reference pixel are determined or obtained based on at least one of the following:

[0031] The current block includes at least one of the following: region, block, and pixel;

[0032] The predicted block and / or reconstructed block of the first sub-block of the current block;

[0033] At least one of the following: the pixel above the current block, the non-adjacent pixel above the current block, the pixel to the left of the current block, the non-adjacent pixel to the left of the current block, the pixel above the left of the current block, and the non-adjacent pixel above the left of the current block;

[0034] The current block is at least one of the following: neighboring block, non-neighboring block, sibling block, temporal block, and default block;

[0035] The current block's width, height, block size, and block area must be at least one of these.

[0036] The candidate motion vector or candidate block vector of the current block is determined or the candidate block is obtained.

[0037] Optionally, the processing method further includes at least one of the following:

[0038] Determine or obtain at least one sub-block of the current block;

[0039] At least one sub-block of the current block is determined or obtained if the width, height, block size and / or block area of ​​the current block satisfy the first condition.

[0040] This application also provides a processing device, including: a memory and a processor, wherein the memory stores a processing program, and when the processing program is executed by the processor, it implements the steps of any of the processing methods described above.

[0041] This application also provides a storage medium storing a computer program that, when executed by a processor, implements the steps of any of the processing methods described above.

[0042] As described above, the processing method of this application can be applied to a processing device, including: performing prediction processing on at least one sub-block of the current block. Through the technical solution of this application, prediction processing on at least one sub-block of the current block can be achieved, which can improve the prediction accuracy of the current block, especially larger image blocks, thereby supporting improved encoding and / or decoding quality in the video encoding and / or decoding process. Attached Figure Description

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings without creative effort, and / or based on these drawings.

[0044] Figure 1 A schematic diagram of the hardware structure of a mobile terminal to implement the various embodiments of this application;

[0045] Figure 2 A communication network system architecture diagram provided in this application embodiment;

[0046] Figure 3 A schematic diagram of the hardware structure of a controller 140 provided in this application;

[0047] Figure 4 A schematic diagram of the hardware structure of a network node 150 provided in this application;

[0048] Figure 5 This is a flowchart illustrating the processing method according to the first embodiment;

[0049] Figure 6 This is a schematic diagram of the encoder's encoding process in the processing method shown in the first embodiment;

[0050] Figure 7 This is a schematic diagram of the decoding process of the decoder in the processing method shown in the first embodiment;

[0051] Figure 8 This is a schematic diagram of a DIMD mode template in the processing method shown in the first embodiment;

[0052] Figure 9 This is a schematic diagram of the gradient magnitude in the processing method shown in the first embodiment;

[0053] Figure 10 This is a schematic diagram of the encoded region corresponding to the block to be predicted in the processing method shown in the first embodiment;

[0054] Figure 11 This is a schematic diagram of the decoded region corresponding to the block to be predicted in the processing method shown in the first embodiment;

[0055] Figure 12 This is a schematic diagram of a TIMD pattern in the processing method shown in the first embodiment;

[0056] Figure 13 This is a schematic diagram of the LIC mode in the processing method shown in the third embodiment;

[0057] Figure 14 This is a schematic diagram of the NNIP mode in the processing method shown in the third embodiment;

[0058] Figure 15 This is a schematic diagram of the structure of a neural network model based on a fully connected layer, as shown in the third embodiment;

[0059] Figure 16 This is a schematic diagram of the structure of a neural network model based on convolutional layers, as shown in the third embodiment;

[0060] Figure 17 This is a schematic diagram of the structure of a neural network model based on hybrid convolutional and fully connected layers, as shown in the third embodiment;

[0061] Figure 18 This is a pixel sampling diagram of the TIMD mode in the processing method shown in the fourth embodiment. Figure 1 ;

[0062] Figure 19 This is a pixel sampling diagram of the TIMD mode in the processing method shown in the fourth embodiment. Figure 2 ;

[0063] Figure 20 This is a pixel sampling diagram of the TIMD mode in the processing method shown in the fourth embodiment. Figure 3 ;

[0064] Figure 21 This is a pixel sampling diagram of the TIMD mode in the processing method shown in the fourth embodiment. Figure 4 ;

[0065] Figure 22 This is a pixel sampling diagram of the OBIC mode in the processing method shown in the fourth embodiment. Figure 1 ;

[0066] Figure 23 This is a pixel sampling diagram of the OBIC mode in the processing method shown in the fourth embodiment. Figure 2 ;

[0067] Figure 24 This is a pixel sampling diagram of the OBIC mode in the processing method shown in the fourth embodiment. Figure 3 ;

[0068] Figure 25 This is a pixel sampling diagram of the OBIC mode in the processing method shown in the fourth embodiment. Figure 4 ;

[0069] Figure 26 This is a pixel sampling diagram of the OBIC mode in the processing method shown in the fourth embodiment. Figure 5 ;

[0070] Figure 27 This is a pixel sampling diagram of the OBIC mode in the processing method shown in the fourth embodiment. Figure 6 ;

[0071] Figure 28 This is a pixel sampling diagram of the OBIC mode in the processing method shown in the fourth embodiment. Figure 7 ;

[0072] Figure 29 This is a pixel sampling diagram of the OBIC mode in the processing method shown in the fourth embodiment. Figure 8 ;

[0073] Figure 30 This is a pixel sampling diagram of the OBIC mode in the processing method shown in the fourth embodiment. Figure 9 ;

[0074] Figure 31 This is a reference template illustration of the DIMD mode or NNIP mode in the processing method shown in the fifth embodiment. Figure 1 ;

[0075] Figure 32 This is a schematic reference template of the TIMD mode in the processing method shown in the fifth embodiment. Figure 1 ;

[0076] Figure 33 This is a reference template illustration of the LIC mode in the processing method shown in the fifth embodiment. Figure 1 ;

[0077] Figure 34 This is a reference template illustration of the LIC mode in the processing method shown in the fifth embodiment. Figure 2 ;

[0078] Figure 35 This is a reference template illustration of the DIMD mode or NNIP mode in the processing method shown in the fifth embodiment. Figure 2 ;

[0079] Figure 36 This is a schematic reference template of the TIMD mode in the processing method shown in the fifth embodiment. Figure 2 ;

[0080] Figure 37 This is a schematic diagram of the processing module of the processing device.

[0081] The realization of the objectives, functional features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0082] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0083] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0084] It should be understood that although the terms first, second, third, etc., may be used herein to describe various information, this information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another; for example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if," as used herein, can be interpreted as "when," "when," or "in response to determination." Furthermore, as used herein, the singular forms "a," "an," and "the" are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms "comprising," "including," indicate the presence of the stated feature, step, operation, element, component, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms “or,” “and / or,” and “including at least one of the following” as used in this application may be interpreted as inclusive, or mean any one or any combination thereof. For example, “including at least one of the following: A, B, C” means “any one of the following: A; B; C; A and B; A and C; B and C; A and B and C”, or “A, B or C” or “A, B and / or C” means “any one of the following: A; B; C; A and B; A and C; B and C; A and B and C”. Exceptions to this definition will only occur if the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0085] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0086] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0087] It should be noted that step designations such as S10 are used in this paper to more clearly and concisely describe the corresponding content, and do not constitute a substantial restriction on the order.

[0088] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0089] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0090] The processing device can be implemented in various forms. For example, the processing device described in this application may include processing devices such as mobile phones, servers, tablet computers, laptops, handheld computers, personal digital assistants (PDAs), portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, and fixed terminals such as digital TVs and desktop computers.

[0091] The following description will use a mobile terminal as an example. Those skilled in the art will understand that, apart from elements specifically designed for mobile purposes, the construction according to the embodiments of this application can also be applied to fixed-type terminals.

[0092] Please see Figure 1 This is a schematic diagram of the hardware structure of a mobile terminal implementing various embodiments of this application. The mobile terminal 100 may include: an RF (Radio Frequency) unit 101, a WiFi module 102, an audio output unit 103, an A / V (Audio / Video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111, etc. Those skilled in the art will understand that... Figure 1 The mobile terminal structure shown does not constitute a limitation on the mobile terminal. The mobile terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0093] The following is combined with Figure 1 A detailed introduction to each component of the mobile terminal:

[0094] The radio frequency (RF) unit 101 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 110; additionally, it transmits uplink data to the base station. Typically, the RF unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, and a duplexer. Furthermore, the RF unit 101 can communicate wirelessly with networks and other devices. The aforementioned wireless communications may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution), 5G, and 6G.

[0095] WiFi is a short-range wireless transmission technology. Mobile terminals, through the WiFi module 102, can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 1 WiFi module 102 is shown, but it is understood that it is not a necessary component of a mobile terminal and can be omitted as needed without changing the nature of the invention.

[0096] The audio output unit 103 can convert audio data received by the radio frequency unit 101 or the WiFi module 102 or stored in the memory 109 into audio signals and output them as sound when the mobile terminal 100 is in call signal receiving mode, call mode, recording mode, voice recognition mode, broadcast receiving mode, etc. Furthermore, the audio output unit 103 can provide audio output related to specific functions performed by the mobile terminal 100 (e.g., call signal receiving sound, message receiving sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.

[0097] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos acquired by an image capture device (such as a camera) in video capture mode or image capture mode. The processed image frames can be displayed on the display unit 106. The image frames processed by the GPU 1041 can be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) in operating modes such as telephone call mode, recording mode, and voice recognition mode, and can process such sound into audio data. The processed audio (voice) data can be converted into a format that can be transmitted to a mobile communication base station via the radio frequency unit 101 in telephone call mode. The microphone 1042 can implement various types of noise cancellation (or suppression) algorithms to eliminate (or suppress) noise or interference generated during the reception and transmission of audio signals.

[0098] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Optionally, the light sensor includes an ambient light sensor and a proximity sensor. Optionally, the ambient light sensor can adjust the brightness of the display panel 1061 according to the ambient light level, and the proximity sensor can turn off the display panel 1061 and / or backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc. Other sensors that may be configured in the phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0099] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0100] User input unit 107 can be used to receive input numerical or character information, and generate key signal inputs related to user settings and function control of the mobile terminal. Optionally, user input unit 107 may include touch panel 1071 and other input devices 1072. Touch panel 1071, also known as touch screen, can collect touch operations on or near the user (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near touch panel 1071), and drive corresponding connection devices according to a pre-set program. Touch panel 1071 may include two parts: touch detection device and touch controller. Optionally, touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to touch controller; touch controller receives touch information from touch detection device, converts it into touch point coordinates, and sends it to processor 110, and can also receive and execute commands sent by processor 110. In addition, touch panel 1071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may include other input devices 1072. Optionally, other input devices 1072 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc. The specifics are not limited here.

[0101] Optionally, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. Subsequently, the processor 110 provides corresponding visual output on the display panel 1061 based on the type of touch event. Although in Figure 1 In this embodiment, the touch panel 1071 and the display panel 1061 are two independent components to realize the input and output functions of the mobile terminal. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to realize the input and output functions of the mobile terminal. The specific implementation is not limited here.

[0102] Interface unit 108 serves as an interface through which at least one external device can connect to mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, and so on. Interface unit 108 may be used to receive input (e.g., data, power, etc.) from the external device and transmit the received input to one or more components within mobile terminal 100, or it may be used to transmit data between mobile terminal 100 and the external device.

[0103] The memory 109 can be used to store software programs and various data. The memory 109 may primarily include a program storage area and a data storage area. Optionally, the program storage area may store the operating system, applications required for at least one function (such as sound playback, image playback, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory 109 may include high-speed random access memory and may include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0104] The processor 110 is the control center of the mobile terminal. It connects various parts of the mobile terminal via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 109, and by calling data stored in the memory 109, it performs various functions and processes data of the mobile terminal, thereby providing overall monitoring of the mobile terminal. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor. Optionally, the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 110.

[0105] The mobile terminal 100 may include a power supply 111 (such as a battery) to power various components. Preferably, the power supply 111 can be logically connected to the processor 110 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0106] although Figure 1 As not shown, the mobile terminal 100 may include a Bluetooth module, etc., which will not be described in detail here.

[0107] To facilitate understanding of the embodiments of this application, the communication network system on which the mobile terminal of this application is based is described below.

[0108] Please see Figure 2 , Figure 2 This application provides a communication network system architecture diagram. The communication network system is an LTE system based on the universal mobile communication technology. The LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203, and the operator's IP services 204, which are connected in sequence.

[0109] Optionally, UE201 can be the aforementioned terminal 100, which will not be described in detail here.

[0110] E-UTRAN202 includes eNodeB2021 and other eNodeB2022, etc. Optionally, eNodeB2021 can connect to other eNodeB2022 via backhaul (e.g., X2 interface). eNodeB2021 connects to EPC203 and can provide UE201 to EPC203 access.

[0111] EPC203 may include MME (Mobility Management Entity) 2031, HSS (Home Subscriber Server) 2032, other MMEs 2033, SGW (Serving Gateway) 2034, PGW (Packet Data Network Gateway) 2035, and PCRF (Policy and Charging Rules Function) 2036, etc. Optionally, MME2031 is the control node that handles signaling between UE201 and EPC203, providing bearer and connection management. HSS2032 is used to provide registers to manage functions such as the Home Location Register (not shown in the figure) and stores user-specific information such as service characteristics and data rates. All user data can be transmitted through SGW2034. PGW2035 can provide IP address allocation and other functions for UE 201. PCRF2036 is the policy and charging control decision point for service data streams and IP bearer resources. It selects and provides available policy and charging control decisions for the Policy and Charging Enforcement Function Unit (not shown in the figure). IP services 204 may include the Internet, intranet, IMS (IP Multimedia Subsystem), or other IP services.

[0112] Although the above description uses the LTE system as an example, those skilled in the art should know that this application is not only applicable to the LTE system, but also to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, 5G and future new network systems (such as 6G), etc., without limitation.

[0113] Figure 3 This is a schematic diagram of the hardware structure of a controller 140 provided in this application. The controller 140 includes a memory 1401 and a processor 1402. The memory 1401 is used to store program instructions, and the processor 1402 is used to call the program instructions in the memory 1401 to execute the steps performed by the controller in the first embodiment of the above method. The implementation principle and beneficial effects are similar, and will not be described again here.

[0114] Optionally, the controller further includes a communication interface 1403, which can be connected to the processor 1402 via a bus 1404. The processor 1402 can control the communication interface 1403 to implement the receiving and sending functions of the controller 140.

[0115] Figure 4This application provides a schematic diagram of the hardware structure of a network node 150. The network node 150 includes a memory 1501 and a processor 1502. The memory 1501 is used to store program instructions, and the processor 1502 is used to call the program instructions in the memory 1501 to execute the steps performed by the first node in the first embodiment of the above method. The implementation principle and beneficial effects are similar, and will not be described again here.

[0116] Optionally, the controller further includes a communication interface 1503, which can be connected to the processor 1502 via a bus 1504. The processor 1502 can control the communication interface 1503 to implement the receiving and sending functions of the network node 150.

[0117] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0118] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk, SSD), etc.

[0119] First Embodiment

[0120] Reference Figure 5 , Figure 5 This is a flowchart illustrating the processing method according to the first embodiment. The processing method of this application embodiment can be applied to a processing device, including:

[0121] Step S10: Perform prediction processing on at least one sub-block of the current block.

[0122] In this embodiment, the processing device can be a smart terminal, such as a mobile phone or computer, or a server, such as a local server or a cloud server. This embodiment and this application primarily use a smart terminal as an example for illustration.

[0123] Optionally, the technical solution of this embodiment can be applied to fields such as image encoding and decoding, video encoding and decoding, hardware video encoding and decoding, dedicated circuit video encoding and decoding, and real-time video encoding and decoding.

[0124] Optionally, the processing device can acquire video image data from a video source, segment each frame of the video image data to obtain at least one image block, and determine the image block to be predicted in the at least one image block as the current block.

[0125] Optionally, the current block can be a block of image to be predicted in the encoder and / or decoder. For example, the current block is a coding unit, and the sub-block is a sub-block within the coding unit. For the sub-block, the corresponding syntax elements and encoding information are derived from the syntax elements and encoding information of the current block. For example, the aspect ratio of the sub-block is the same as that of the current block.

[0126] Optionally, the encoding mode of the sub-block is the same as that of the current block.

[0127] Optionally, sub-blocks do not need to send signaling to indicate the syntax elements or encoding information of the sub-block.

[0128] Optionally, the current block includes four sub-blocks of the same area. When the current block is divided into four sub-blocks of the same size, it means that each sub-block can apply a unified algorithm and parameter settings when performing encoding and decoding operations, without the need to adjust them separately for different sizes. This simplifies the calculation and determination process of the encoding information, prediction information and syntax elements of the sub-blocks.

[0129] Optionally, in order to improve the prediction effect for larger image blocks (i.e., the current block), the current block can be further divided into at least one sub-block, and the sub-blocks can be predicted. By predicting the sub-blocks of the current block, it is helpful to capture and analyze the local features of the current block in more detail, thereby improving the prediction effect and / or reducing the amount of data processed in a single operation, making the computation more efficient.

[0130] Optionally, the number of sub-blocks contained in the current block can be set according to the actual situation. For example, the current block may contain 4, 8, or 16 sub-blocks. The size of each sub-block may be at least partially the same or different.

[0131] Optionally, the current block can be divided into four sub-blocks of the same size, and each sub-block can be subjected to prediction processing according to a preset processing order.

[0132] Optionally, after performing prediction processing on at least one sub-block of the current block, the prediction result of the sub-block can be obtained; the prediction result can be a predicted pixel, a predicted sub-block, etc., and the prediction result of the current block can be determined or obtained based on the prediction result of the sub-block, so as to improve the prediction accuracy of the current block.

[0133] Optionally, prediction processing can be performed on at least one sub-block of the current block to determine or obtain the prediction result of each sub-block. The prediction results of each sub-block can be directly spliced, weighted fusion, or smoothed for overlapping areas to determine or obtain the prediction result of the current block.

[0134] Optionally, direct splicing refers to directly splicing the prediction results of each sub-block according to its position in the current block; it is suitable for situations where there is no overlap between sub-blocks or the overlapping parts can be ignored.

[0135] Optionally, if there is overlap between sub-blocks, smoothing techniques can be used in the overlapping areas to reduce boundary effects. For example, averaging can be used to average the pixel values ​​in the overlapping areas as the final predicted pixel, and / or a weighted average can be used with Gaussian weights to achieve a smoother transition effect.

[0136] Optionally, weighted fusion refers to assigning different weight information to each sub-block based on the confidence level or other indicators of the prediction results of each sub-block, and then fusing based on these weight information. For example, if the prediction results of a certain sub-block are considered to be more accurate, it can be given higher weight information; otherwise, it can be given lower weight information.

[0137] Optionally, the weights may include at least one of the following: weight coefficients, weight vectors, and weight matrices.

[0138] Optionally, prediction processing for at least one sub-block of the current block may include intra-frame prediction processing and / or inter-frame prediction processing.

[0139] Reference Figure 6When the processing device is an encoder on the encoding side, the encoder can receive video data from the video source, such as receiving video images from the video source, determining the image to be predicted in the video images, dividing the image to be predicted into at least one image block, and using the temporal and / or spatial correlation between video images, performing prediction processing on each of the at least one image block, including intra-frame prediction processing and / or inter-frame prediction processing. The intra-frame prediction processing and / or inter-frame prediction processing each include multiple prediction modes. For these prediction modes, the encoder uses, for example, rate-distortion cost to determine the prediction mode finally adopted for each of the at least one image block. For example, it calculates the rate-distortion cost corresponding to each prediction mode or the rate-distortion cost of combining several prediction modes to determine the minimum rate-distortion cost from multiple rate-distortion costs. The prediction mode or combination of prediction modes corresponding to the minimum rate-distortion cost is the prediction mode finally adopted for the image block.

[0140] Optionally, these prediction modes include intra-frame prediction modes and inter-frame prediction modes; and the intra-frame prediction mode can be determined using the Decoder-side Intra-mode Derivation (DIMD) mode, the Occurrence-based Intra Coding (OBIC) mode, or the Template-based Intra-mode Derivation (TIMD) mode proposed in this application.

[0141] Optionally, the processing method includes: determining the prediction mode of the current block based on the first information of the current block.

[0142] Optionally, the first information may include: reference area information of the current block, reference block information, reference template information, reference pixel information, etc.

[0143] Optionally, the processing method further includes: after determining the prediction mode of the image block to be predicted (i.e. the current block) by the above method, using the prediction mode to perform prediction processing on at least one sub-block of the image block to be predicted, thereby determining or obtaining the prediction block of the image block to be predicted.

[0144] Optionally, a residual block between the predicted block and the current block can be further calculated; the residual block can be transformed and quantized, and then encoded by an entropy encoder to form an encoded bit stream.

[0145] Optionally, the encoded bitstream may include prediction parameters corresponding to a defined prediction mode and related side information.

[0146] Optionally, the prediction parameters are entropy-encoded and packed into the encoded bitstream; optionally, the prediction parameters include indication information of the prediction mode.

[0147] Optionally, the transformed and quantized residual block can be added to the corresponding prediction data (e.g., the prediction block) obtained using the prediction mode after inverse quantization and inverse transformation to obtain a reconstructed block. After obtaining the reconstructed block, the loop filtering module performs loop filtering on the reconstructed block according to the filter control parameters to reduce distortion. After the loop filtering process, the reconstructed block after loop filtering is stored according to the encoded image buffer.

[0148] Reference Figure 7 When the processing device is a decoder on the decoding side, after receiving the encoded bit stream, the decoder's entropy decoding unit will parse and decode the encoded bit stream to obtain the transform coefficients. The decoder's inverse transform unit and inverse quantization unit will perform inverse transform and inverse quantization processing on the transform coefficients to obtain the residual block.

[0149] Optionally, the entropy decoding unit of the decoder parses and decodes the encoded bit stream to obtain prediction data, such as prediction parameters and related auxiliary information; the prediction processing unit of the decoder uses the prediction parameters to perform prediction processing, thereby determining the prediction block corresponding to the residual block.

[0150] Optionally, the prediction processing includes intra-frame prediction processing and / or inter-frame prediction processing, and the intra-frame prediction processing and / or inter-frame prediction processing each include one or more prediction modes combined with each other.

[0151] Optionally, when the auxiliary information indicates that the DIMD mode, OBIC mode or TIMD mode proposed in this application is adopted, the prediction mode corresponding to the residual block is determined according to the DIMD mode, OBIC mode or TIMD mode proposed in this application.

[0152] Optionally, the processing method includes: determining the prediction mode of the current block based on the first information of the current block.

[0153] Optionally, the processing method further includes: after determining the prediction mode of the image block to be predicted (i.e. the current block) by the above method, using the prediction mode to perform prediction processing on at least one sub-block of the image block to be predicted, thereby determining or obtaining the prediction block of the image block to be predicted.

[0154] Optionally, the processing method further includes: adding the obtained residual block and the corresponding prediction block (including the predicted luminance block and the predicted chrominance block) to obtain the reconstructed block; the loop filtering unit of the decoder performs loop filtering on the reconstructed block to reduce distortion and improve video quality.

[0155] Optionally, the reconstructed blocks after loop filtering are further combined into a decoded image and stored in a decoded image buffer or output as a decoded video signal.

[0156] Optionally, when the processing device is an encoder, the initially obtained prediction value can be the prediction value obtained in the corresponding prediction mode, which can be directly used in the rate-distortion cost process.

[0157] Optionally, when the processing device is a decoder, the initially obtained prediction value can be the prediction value obtained through the prediction mode corresponding to the block to be predicted (i.e., the image block located at the decoding end) indicated by the syntax elements parsed in the bitstream.

[0158] Optionally, the predicted block can be used as the target image block, the residual block between the target image block and the current block can be calculated, and then encoded by an entropy encoder through transformation and quantization to form an encoded bit stream; and / or some corresponding processing can be performed on the predicted block, such as processing it through other models, using the processed image block as the target image block, and performing the steps of calculating the residual block between the target image block and the current block and subsequent steps.

[0159] Optionally, the basic principle of the DIMD mode proposed in this application can be to deduce the intra-prediction mode of the current block by analyzing the gradient magnitude and direction of adjacent pixels, or the usage of intra-prediction modes of adjacent / non-adjacent image blocks.

[0160] Optionally, the implementation process of the DIMD mode proposed in this application may include: determining the gradient magnitude and gradient direction of the pixels in the DIMD template, determining the gradient histogram or statistical results of the DIMD template with respect to the intra-prediction direction, and determining the intra-prediction direction of the block to be predicted based on the gradient histogram.

[0161] Alternatively, the DIMD template refers to a set of neighboring pixels surrounding the current block. The reason for these neighboring pixels is that they can provide important clues about the edges and texture direction of the block to be predicted. Therefore, the optimal intra-frame prediction mode for the current block can be analyzed and derived based on these pixels.

[0162] Reference Figure 8The DIMD templates adjacent to the block to be predicted (the current block) are determined above and to the left of the block to be predicted, namely the three lines on the left and above the block to be predicted. Next, a pixel in the middle line (e.g., pixel A) is taken as the pixel for calculating the gradient. By calculating the gradient direction of at least one pixel in the middle line, as well as the magnitude of the horizontal and vertical gradients, the gradient direction and the corresponding gradient magnitude value of at least one pixel can be obtained. The gradient magnitude value is the sum of the absolute values ​​of the horizontal and vertical gradients. If the gradient magnitude values ​​with the same gradient direction in at least one pixel are added together, the sum of the gradient magnitude values ​​corresponding to that gradient direction can be obtained. A histogram of gradient magnitude values ​​with different gradient directions for at least one pixel can be established. The prediction direction perpendicular to the gradient direction with the maximum gradient magnitude value can be used as the prediction direction of the intra-frame prediction mode of the current block.

[0163] Optionally, the horizontal gradient Gx and vertical gradient Gy can be calculated using the 3x3 horizontal Sober operator and the vertical Sober operator, respectively. For example, the horizontal gradient Gx and vertical gradient Gy of a pixel x 4 in a pixel line can be calculated according to the following formulas (I) and (II):

[0164]

[0165] Optionally, A can be a matrix consisting of 9 pixels centered at pixel x4, including the pixel x1 above it, the pixel x3 to its left, the pixel x7 below it, the pixel x5 to its right, the pixel x0 at its top left, the pixel x6 at its bottom left, the pixel x2 at its top right, and the pixel x8 at its bottom right, as shown in Formula (III) below:

[0166]

[0167] Optionally, the magnitude of gradient G is the sum of the absolute values ​​of the horizontal and vertical gradients, and its calculation formula is shown in formula (iv):

[0168] Formula (IV) is G = |Gx| + |Gy|.

[0169] Alternatively, the gradient direction corresponding to a pixel can be calculated using arctan(Gx / Gy).

[0170] Optionally, since each gradient direction corresponds to a specific gradient direction range, and each gradient direction range corresponds to the prediction direction of an intra-prediction mode, for at least one pixel in the pixel line, the gradient magnitude values ​​with the same gradient direction range in at least one pixel can be added together to obtain the sum of the gradient magnitude values ​​corresponding to the gradient direction range.

[0171] Optionally, the sum of the gradient magnitude values ​​of the prediction direction of the corresponding intra-frame prediction mode can be obtained.

[0172] Reference Figure 9 This includes the gradient magnitude values ​​corresponding to the prediction directions of each intra-frame prediction mode, based on... Figure 9 The final intra-frame prediction mode selected was mode 3.

[0173] Optionally, the basic principle of the OBIC mode proposed in this application can be to determine the intra prediction mode of the current block by analyzing the use of intra prediction modes by adjacent coding blocks and / or non-adjacent coding blocks.

[0174] Optionally, the implementation process of OBIC mode may include: determining the intra-prediction mode usage of at least one image block, calculating the area amplitude value and intra-prediction direction of at least one image block, determining the area amplitude histogram or statistical results of at least one image block with respect to the intra-prediction direction, and determining the intra-prediction direction of the block to be predicted based on the area amplitude histogram or statistical results.

[0175] Optionally, determining the intra-prediction mode usage of multiple coded blocks includes: first determining a coded region or a decoded region, then determining the intra-prediction mode usage of coded blocks in the coded region, and / or determining the intra-prediction mode usage of decoded blocks in the decoded region.

[0176] Optionally, the encoding unit includes an encoding block of three color components, which include a luminance component and two chrominance components.

[0177] Optionally, determining the intra-prediction mode usage of at least one image block may include: first determining at least one coded region or at least one decoded region, then determining the intra-prediction mode usage of coded blocks within the coded region, and / or determining the intra-prediction mode usage of decoded blocks within the decoded region, for example, such as... Figure 10 As shown, in the encoded region, there are coded blocks 4 and 6 adjacent to the block to be predicted, and coded blocks 1, 2, 3, 5, 7, and 8 that are not adjacent to the block to be predicted. Figure 11 As shown, in the decoded region there are decoded blocks 4 and 6 that are adjacent to the block to be predicted, and decoded blocks 1, 2, 3, 5, 7 and 8 that are not adjacent to the block to be predicted.

[0178] Optionally, the basic principle of the TIMD mode proposed in this application is to determine or generate a predicted value for the template by using each mode in the MPM (Most Probable Mode), and then calculate the SATD (Sum of Absolute Transformed Differences) of the template predicted value and the reconstructed value, select the mode with the smallest SATD as the TIMD mode, and use it for the prediction of the current block.

[0179] Reference Figure 12 The implementation process of TIMD mode may include: determining the template of the block to be predicted, which can be constructed using the reconstructed pixels on the left or top of the block to be predicted, determining or generating predicted values ​​for the template by using each mode in MPM in combination with the reference pixels of the template, and selecting the mode with the smallest SATD value from the MPM modes for prediction based on the predicted value and the SATD of the reconstructed value.

[0180] Optionally, the template can serve as a reference benchmark for pattern selection, simulating local features of the current block (such as edge direction and texture).

[0181] Optionally, the reference pixels of the template are used to determine or generate the predicted values ​​of the template, which are then combined with candidate patterns in the MPM to derive the predicted results of the template.

[0182] Optionally, in embodiments of this application, the current block can be divided into at least one sub-block for prediction processing when performing prediction derivation mode processing.

[0183] Optionally, the prediction derivation mode includes at least one of the DIMD mode, OBIC mode, and TIMD mode.

[0184] In this embodiment, prediction processing can be performed on at least one sub-block of the current block. By predicting the sub-blocks of the current block, it is helpful to capture and analyze the local features of the current block in more detail, thereby improving the prediction accuracy of the current block, especially the current block with a larger size. This can support the improvement of encoding and / or decoding quality in the video encoding and / or decoding process, and / or reduce the amount of data processed in a single operation, making the computation more efficient.

[0185] Second Embodiment

[0186] Based on the first embodiment, a second embodiment is proposed.

[0187] In this embodiment, the processing method further includes at least one of the following methods:

[0188] Method 1: Determine or obtain at least one sub-block of the current block;

[0189] Optionally, the current block can be divided into at least one sub-block to obtain at least one sub-block of the current block.

[0190] Optionally, the block sizes of at least one determined or obtained sub-block can be at least partially the same or completely different.

[0191] Optionally, the block size may include the block's width, height, aspect ratio, depth, area, resolution, and number of pixels.

[0192] Alternatively, the current block can be divided into multiple sub-blocks of the same size by uniform division.

[0193] Alternatively, the current block can be divided into multiple sub-blocks of different sizes through non-uniform partitioning.

[0194] Alternatively, the current block can be divided into four sub-blocks of the same size using a quadtree partitioning method.

[0195] Optionally, at the encoding end, the video image is divided into at least one image block, and rate-distortion optimization is performed on these image blocks to determine the optimal image block partitioning mode and prediction mode. In the process of image block partitioning, a quadtree partitioning method is usually adopted to divide an image block into four lower-level image blocks. Then, all prediction modes are traversed for each lower-level image block. During this process, the lower-level image block calculates the prediction results for each mode, which includes the prediction results of DIMD mode, OBIC mode, TIMD mode, LIC (Local Illumination Compensation) mode, NNIP (Neural Network Intra Prediction) mode, etc. After an image block is divided into at least one image block, if this partitioning method is adopted, this image block will be replaced by at least one image block. The sub-block-based processing method proposed in this embodiment can at least partially reuse the prediction results of these lower-level image blocks without adding additional computation.

[0196] Method 2: At least one sub-block of the current block is determined or obtained if the width, height, block size and / or block area of ​​the current block satisfy the first condition;

[0197] Optionally, if the width, height, block size and / or block area of ​​the current block meet the first condition, the current block can be divided to determine or obtain at least one sub-block of the current block, and prediction processing can be performed on at least one sub-block of the current block. If the width, height, block size and / or block area of ​​the current block do not meet the first condition, the current block can be divided without dividing it, and prediction processing can be performed directly on the current block.

[0198] Optionally, the first condition can be a pre-set condition. Optionally, the first condition is not fixed and can be adaptively adjusted according to different scenarios.

[0199] Optionally, the width, height, block size, and / or block area of ​​the current block satisfy the first condition, including at least one of the following:

[0200] The current block's width, height, block size, and / or block area are set to the first value;

[0201] The current block's width, height, block size, and / or block area are within the first numerical range;

[0202] The width, height, block size, and / or block area of ​​the current block are greater than or equal to the first threshold.

[0203] Optionally, the first value can be a value of at least one of the pre-set width, height, block size, and block area of ​​the image block.

[0204] Optionally, the first numerical range may be a range of at least one of the pre-defined values ​​of the width, height, size, and area of ​​the image block.

[0205] Optionally, the first threshold may be a threshold for at least one of the pre-set width, height, block size, and block area of ​​the image block.

[0206] Optionally, if the block size of the current block is greater than or equal to a first threshold (e.g., 128), then at least one sub-block of the current block is determined or obtained, and prediction processing is performed on at least one sub-block of the current block.

[0207] In this embodiment, if the width, height, block size, and / or block area of ​​the current block meet the first condition, it indicates that the current block is large and may contain complex textures and details. Directly predicting large blocks may lead to lower prediction accuracy. Therefore, this application divides large blocks into smaller sub-blocks, which can better capture local features, thereby improving prediction accuracy and coding efficiency. If the width, height, block size, and / or block area of ​​the current block do not meet the first condition, it indicates that the current block is small. In this case, sub-block division can be skipped, and prediction processing can be performed directly on the current block. This can reduce the amount of computation, optimize the allocation of computing resources, and avoid introducing additional signaling overhead.

[0208] Optionally, step S10 includes at least one of the following methods three and four:

[0209] Method 3: Based on at least one prediction pattern, perform prediction processing on at least one sub-block of the current block;

[0210] Optionally, prediction processing is performed on at least one sub-block of the current block according to at least one of the DIMD mode, OBIC mode, TIMD mode, LIC mode and NNIP mode.

[0211] Optionally, the prediction mode includes at least one of the following: prediction derivation mode, LIC mode, and NNIP mode.

[0212] Optionally, the prediction derivation mode includes at least one of the following: DIMD mode, TIMD mode, and OBIC mode.

[0213] Optionally, at least one sub-block of the current block is predicted according to at least one prediction derivation mode. In the process of predicting at least one sub-block of the current block through the prediction derivation mode, the most suitable angle mode or non-angle prediction mode for each sub-block can be derived first through the prediction derivation mode, and then the prediction process can be further performed through the derived mode to determine or obtain the prediction result of at least one sub-block of the current block.

[0214] Optionally, the angle prediction mode is a technique for predicting the current pixel block. It determines or generates the predicted block by propagating the values ​​of neighboring pixels along a specific direction. The angle prediction mode is mainly used to process directional textures in images, which can effectively reduce spatial redundancy and improve compression efficiency.

[0215] In the HEVC (High Efficiency Video Coding) standard, intra-frame prediction modes include 33 angle prediction modes. These modes cover different angles from horizontal to vertical, ensuring accurate prediction of various texture directions. Optionally, the angle prediction mode predicts the pixel value of the current block by selecting a specific angle and extracting values ​​from neighboring pixels along that direction. For example, if a 45-degree angle is selected, the predicted value will be extracted from the neighboring pixels in the top left or bottom right corner.

[0216] Optionally, the non-angle prediction mode can be a prediction mode other than the angle prediction mode.

[0217] Optionally, the non-angle prediction mode includes at least one of the following: NNIP mode, intra-frame prediction mode based on extrapolation filter, block vector prediction mode, planar mode, and DC mode.

[0218] Optionally, in the prediction derivation mode, at least one non-angle predictor is determined or obtained through a neural network-based prediction mode. Optionally, based on at least one non-angle predictor and / or at least one angle predictor, the prediction result of at least one sub-block of the current block is determined or generated.

[0219] Optionally, in the prediction derivation mode, the prediction result of the current block and / or at least one sub-block of the current block is determined or generated based on the first prediction result and the second prediction result. Optionally, the prediction derivation mode includes at least one of DIMD mode, OBIC mode and TIMD mode. Optionally, the first prediction result is determined or obtained by a prediction mode based on a neural network.

[0220] Optionally, the first prediction result is determined or obtained by at least one first non-angle predictor and / or at least one first angle predictor.

[0221] Optionally, the second prediction result is determined or obtained by at least one second non-angle predictor and / or at least one second angle predictor.

[0222] Optionally, the first prediction result and the second prediction result may differ.

[0223] Optionally, at least one first non-angle predictor and at least one second non-angle predictor are different, and at least one first angle predictor and at least one second angle predictor are different.

[0224] Optionally, the first prediction result and the second prediction result are determined or obtained by a prediction mode based on a neural network and a prediction mode not based on a neural network, respectively. Optionally, the prediction mode not based on a neural network includes: angle prediction mode, DC mode and / or Planar mode.

[0225] Optionally, the prediction result of at least one sub-block of the current block can be determined or generated by fusing the first prediction result and the second prediction result.

[0226] Optionally, in the intra-frame prediction mode, the prediction result of at least one sub-block of the current block is determined or generated by the first prediction result and the second prediction result. Optionally, the first prediction result is determined or obtained by a neural network-based prediction mode, and the second prediction result is determined or obtained by a prediction derivation mode.

[0227] Optionally, the above-mentioned prediction derivation modes include at least one of the following: DIMD mode, OBIC mode, and TIMD mode.

[0228] Optionally, the prediction result of the current block is determined or obtained based on the prediction result of at least one sub-block of the current block. Optionally, the size of at least one sub-block of the current block may be the same as or different from the size of the current block.

[0229] Optionally, if the size of at least one sub-block of the current block is the same as the size of the current block, the prediction result of the current block is determined or obtained by fusing the first prediction result and the second prediction result.

[0230] Optionally, if the size of at least one sub-block of the current block is not the same as the size of the current block, the prediction result of the current block is determined or obtained by splicing and / or fusing the first prediction result and the second prediction result.

[0231] Optionally, different prediction modes can be used to perform prediction processing on each sub-block of the current block.

[0232] Alternatively, the same prediction pattern can be used to predict each sub-block of the current block.

[0233] Optionally, the same prediction derivation pattern can be used to perform prediction processing on each sub-block of the current block.

[0234] Optionally, at least one sub-block of the current block is determined or obtained; and prediction processing is performed on at least one sub-block of the current block according to at least one prediction mode.

[0235] Optionally, if at least one sub-block of the current block satisfies a first condition in terms of the width, height, block size, and / or block area of ​​the current block, at least one sub-block of the current block is determined or obtained, and prediction processing is performed on at least one sub-block of the current block according to at least one prediction mode.

[0236] Optionally, in this embodiment, the prediction mode includes at least one of modes C to G in the third embodiment.

[0237] Optionally, in this embodiment, the prediction mode is determined or obtained according to at least one of methods five to eight in the fourth embodiment.

[0238] In this embodiment, by performing prediction processing on at least one sub-block of the current block using an appropriate prediction mode, local features in the image block can be captured more accurately, thereby improving the prediction effect of the image block and thus supporting the improvement of the prediction effect of video encoding and / or decoding.

[0239] Optionally, prediction processing is performed on at least one sub-block of the current block according to at least one prediction mode, including at least one of mode A and mode B:

[0240] Method A: Based on the same prediction pattern, perform prediction processing on at least one sub-block of the current block;

[0241] Optionally, the same prediction mode may be used to perform prediction processing on at least one sub-block of the current block.

[0242] Optionally, the prediction processing is performed on each sub-block of the current block using the same prediction derivation mode. Optionally, the prediction derivation mode includes at least one of the following: DIMD mode, TIMD mode, and OBIC mode.

[0243] Optionally, prediction processing is performed on at least one sub-block of the current block according to the DIMD pattern.

[0244] Optionally, prediction processing is performed on at least one sub-block of the current block according to the OBIC pattern.

[0245] Optionally, prediction processing is performed on at least one sub-block of the current block according to the TIMD pattern.

[0246] Optionally, prediction processing is performed on at least one sub-block of the current block according to the LIC mode.

[0247] Optionally, prediction processing is performed on at least one sub-block of the current block according to the NNIP pattern.

[0248] Optionally, signaling may be used to instruct at least one sub-block of the current block to perform prediction processing using the same prediction mode.

[0249] Optionally, at least one sub-block of the current block is determined or obtained, and prediction processing is performed on at least one sub-block of the current block according to the same prediction mode.

[0250] Optionally, if at least one sub-block of the current block satisfies the first condition in terms of the width, height, block size, and / or block area of ​​the current block, at least one sub-block of the current block is determined or obtained, and prediction processing is performed on at least one sub-block of the current block according to the same prediction mode.

[0251] Optionally, in this embodiment, the prediction mode includes at least one of modes C to G in the third embodiment.

[0252] Optionally, in this embodiment, the prediction mode is determined or obtained according to at least one of methods five to eight in the fourth embodiment.

[0253] Optionally, if the current block is divided into 4 sub-blocks, and rate-distortion optimization determines that DIMD mode is used for all 4 sub-blocks, in the conventional approach, a separate signaling message needs to be sent for each sub-block to indicate the use of this mode. However, in this embodiment, by using the same prediction mode for each sub-block, only one signaling message is needed to indicate the prediction mode of the current block, effectively saving signaling overhead in the encoding and decoding process. At the encoding end, an image block is divided into at least one next-level image block, and then all prediction modes are traversed for each next-level image block. During this process, the next-level image block calculates each mode. The prediction results under the formula include prediction results of DIMD mode, OBIC mode, TIMD mode, LIC mode, NNIP mode, etc. The sub-block-based processing method proposed in this application can at least partially reuse the prediction results of these next-level image blocks without increasing the amount of additional computation. Optionally, the sub-block-based processing method proposed in this application can effectively save signaling overhead without increasing the amount of additional computation; and / or consistent prediction mode selection can help reduce block artifacts, such as artifacts and discontinuities, making the reconstructed image look more natural and smooth, and improving coding quality.

[0254] Method B involves performing prediction processing on the second sub-block of the current block based on the prediction pattern corresponding to the first sub-block of the current block.

[0255] Optionally, the first sub-block and the second sub-block are adjacent sub-blocks in the current block.

[0256] Optionally, the prediction process for the second sub-block of the current block can be performed using the prediction mode corresponding to the first sub-block of the current block.

[0257] Optionally, since in many cases, especially for image regions with consistent texture or continuous features, adjacent sub-blocks often have high similarity, predicting the second sub-block adjacent to the first sub-block based on the prediction pattern corresponding to the first sub-block can effectively improve the prediction accuracy of the second sub-block based on the correlation between the first and second sub-blocks.

[0258] Optionally, the first sub-block and the second sub-block are non-adjacent sub-blocks in the current block. Although the first sub-block and the second sub-block are non-adjacent sub-blocks, since they are both part of the current block, there is still a certain correlation between them. This correlation stems from the local features and environment they share, such as common lighting conditions, color distribution, or texture patterns. Based on this inherent correlation, the prediction pattern of the first sub-block can be used to predict the second sub-block, thereby improving the prediction accuracy of the second sub-block.

[0259] Optionally, based on the prediction mode corresponding to the first sub-block of the current block, the prediction mode corresponding to the second sub-block of the current block is determined or obtained, and the second sub-block is predicted based on the prediction mode corresponding to the second sub-block.

[0260] Optionally, in this embodiment, the prediction mode includes at least one of modes C to G in the third embodiment.

[0261] Optionally, in this embodiment, the prediction mode is determined or obtained according to at least one of methods five to eight in the fourth embodiment.

[0262] Optionally, at least one sub-block of the current block is determined or obtained, including a first sub-block and a second sub-block; and the second sub-block of the current block is subjected to prediction processing according to the prediction mode corresponding to the first sub-block of the current block.

[0263] Optionally, if at least one sub-block of the current block satisfies the first condition in terms of the width, height, block size, and / or block area of ​​the current block, at least one sub-block of the current block is determined or obtained, including a first sub-block and a second sub-block; and the second sub-block of the current block is subjected to prediction processing according to the prediction mode corresponding to the first sub-block of the current block.

[0264] In this embodiment, there is a certain correlation between the first sub-block and the second sub-block from the same current block. Therefore, by performing prediction processing on the second sub-block of the current block according to the prediction mode corresponding to the first sub-block of the current block, the accuracy of sub-block prediction can be improved based on this inherent correlation. It also helps to maintain the consistency and coherence within the entire current block and improve the encoding and decoding quality.

[0265] Method 4: Determine or obtain the prediction result of the current block based on at least one reference region, at least one reference block and / or at least one reference pixel of at least one sub-block of the current block;

[0266] Optionally, the prediction result of the current block is determined or obtained based on at least one prediction mode and at least one reference region, at least one reference block and / or at least one reference pixel of at least one sub-block of the current block.

[0267] Optionally, the reference region can be an image region used to determine or obtain the prediction result of the current block, and can be an image region adjacent to the current block and / or an image region not adjacent to the current block.

[0268] Optionally, the reference region may be a region consisting of encoded and reconstructed pixels or blocks from the same frame and / or different frames.

[0269] Optionally, the reference block can be a pixel block corresponding to the current block extracted from a reference frame (also known as a reference image) or the current frame (also known as the current image). The reference block can have the same size as the current block or sub-block. By using the reference block, the encoder can take advantage of temporal or spatial correlation to reduce redundant information in the current frame, thereby achieving efficient compression.

[0270] Optionally, a reference frame refers to a frame (also called an image) that has been encoded and reconstructed. These frames can be forward reference frames (i.e., frames / images that are preceding the current frame in the playback order) or backward reference frames (i.e., frames / images that are following the current frame in the playback order).

[0271] Optionally, the reference pixel refers to the pixel used to predict the pixel value within the current block.

[0272] Optionally, the prediction result of each sub-block is determined or obtained based on the prediction mode and at least one reference region, at least one reference block and / or at least one reference pixel of each sub-block of the current block. Based on the prediction result of each sub-block, the prediction result of the current block is determined or obtained. By using the reference region, reference block and / or reference information of each sub-block, it is helpful to capture and analyze the local features of each sub-block in more detail, thereby improving the prediction accuracy of the current block, especially the current block with a larger size, and thus improving the encoding and decoding quality in the video encoding and / or decoding process.

[0273] Optionally, in this embodiment, the prediction mode includes at least one of modes C to G in the third embodiment.

[0274] Optionally, in this embodiment, the prediction mode is determined or obtained according to at least one of methods five to eight in the fourth embodiment.

[0275] Optionally, in this embodiment, the reference region, reference block, and / or reference pixel are determined or obtained according to at least one of methods nine to fourteen in the fifth embodiment.

[0276] Optionally, at least one sub-block of the current block is determined or obtained, and the prediction result of the current block is determined or obtained based on at least one reference region, at least one reference block and / or at least one reference pixel of at least one sub-block of the current block.

[0277] Optionally, if at least one sub-block of the current block satisfies a first condition in terms of the width, height, block size, and / or block area of ​​the current block, at least one sub-block of the current block is determined or obtained, and the prediction result of the current block is determined or obtained based on at least one reference region, at least one reference block, and / or at least one reference pixel of at least one sub-block of the current block.

[0278] Optionally, if at least one sub-block of the current block satisfies a first condition in terms of width, height, block size, and / or block area, at least one sub-block of the current block is determined or obtained, and the prediction result of the current block is determined or obtained based on at least one prediction mode and at least one reference region, at least one reference block, and / or at least one reference pixel of at least one sub-block of the current block.

[0279] Optionally, if at least one sub-block of the current block satisfies the first condition in terms of width, height, block size and / or block area of ​​the current block, at least one sub-block of the current block is determined or obtained, and prediction processing is performed on at least one sub-block of the current block according to the same prediction mode and at least one reference region, at least one reference block and / or at least one reference pixel of at least one sub-block of the current block.

[0280] Optionally, if at least one sub-block of the current block satisfies a first condition in terms of the width, height, block size, and / or block area of ​​the current block, at least one sub-block of the current block is determined or obtained, including a first sub-block and a second sub-block. Based on the prediction mode corresponding to the first sub-block and at least one reference region, at least one reference block, and / or at least one reference pixel of the first sub-block, the prediction result of the first sub-block is determined or obtained. Based on the prediction mode corresponding to the first sub-block and at least one reference region, at least one reference block, and / or at least one reference pixel of the second sub-block, the prediction result of the second sub-block is determined or obtained. Based on the prediction results of the first sub-block and the second sub-block, the prediction result of the current block is determined or obtained.

[0281] In this embodiment, when predicting the current block, by considering the reference region, reference block, or reference pixel of each sub-block, the unique local features of each sub-block can be captured more accurately, thereby improving the prediction accuracy and thus supporting the improvement of prediction performance in video encoding and / or decoding.

[0282] Third Embodiment

[0283] Based on any of the above embodiments, a third embodiment is proposed.

[0284] In this embodiment, the prediction mode includes at least one of mode C to mode G:

[0285] Method C, decoding-side intra-frame mode derivation mode;

[0286] Optionally, the basic principle of DIMD mode is to deduce the intra-prediction mode of the current block by analyzing the gradient magnitude and direction of adjacent pixels, or the usage of intra-prediction modes of adjacent / non-adjacent image blocks.

[0287] Optionally, the implementation process of DIMD mode may include: determining the gradient magnitude and gradient direction of pixels in the DIMD template, determining the gradient histogram or statistical results of the DIMD template with respect to the intra-prediction direction, and determining the intra-prediction direction of the block to be predicted based on the gradient histogram.

[0288] Optionally, the reference template is a key region used to analyze and derive intra-frame prediction modes. It typically consists of decoded pixels surrounding the block to be predicted, which provide important contextual information about the content of the block to be predicted. By analyzing the features of these pixels (such as gradients), the most suitable prediction mode can be inferred.

[0289] Optionally, the reference template may include a reference region, a reference block, and / or a reference pixel.

[0290] Optionally, reference pixels are the specific pixels that constitute the reference template. They are usually decoded pixels located at the top, left, or upper left corner of the block to be predicted. Reference pixels provide important clues about the local image structure (such as edges and texture direction). Therefore, the quality of reference pixels directly affects the accuracy of the prediction mode selection. High-quality reference pixels help to derive the optimal prediction mode more accurately, thereby improving the quality of the reconstructed image.

[0291] Reference Figure 8 The DIMD templates adjacent to the block to be predicted (the current block) are determined above and to the left of the block to be predicted, namely the three lines on the left and above the block to be predicted. Next, a pixel in the middle line (e.g., pixel A) is taken as the pixel for calculating the gradient. By calculating the gradient direction of at least one pixel in the middle line, as well as the magnitude of the horizontal and vertical gradients, the gradient direction and the corresponding gradient magnitude value of at least one pixel can be obtained. The gradient magnitude value is the sum of the absolute values ​​of the horizontal and vertical gradients. If the gradient magnitude values ​​with the same gradient direction in at least one pixel are added together, the sum of the gradient magnitude values ​​corresponding to that gradient direction can be obtained. A histogram of gradient magnitude values ​​with different gradient directions for at least one pixel can be established. The prediction direction perpendicular to the gradient direction with the maximum gradient magnitude value can be used as the prediction direction of the intra-frame prediction mode of the current block.

[0292] Optionally, in this embodiment, the reference region, reference block, and / or reference pixel are determined or obtained according to at least one of methods nine to fourteen in the fifth embodiment.

[0293] In this embodiment, by performing prediction processing on at least one sub-block of the current block using the DIMD mode, the optimal intra-frame prediction mode can be automatically derived at the decoder, thereby reducing the amount of mode information that the encoder needs to transmit to the decoder, thus reducing the bit rate; at the same time, by using information from neighboring decoded blocks to derive the optimal prediction mode for the current block, high-quality image reconstruction can be achieved while maintaining good visual effects.

[0294] Method D, a mode derived from the intra-frame coding mode of the event that occurred;

[0295] Optionally, the basic principle of OBIC mode can be to determine the intra prediction mode of the current block by analyzing the use of intra prediction modes by adjacent and / or non-adjacent coding blocks.

[0296] Optionally, the implementation process of OBIC mode may include: determining the intra-prediction mode usage of at least one image block, calculating the area amplitude value and intra-prediction direction of at least one image block, determining the area amplitude histogram or statistical results of at least one image block with respect to the intra-prediction direction, and determining the intra-prediction direction of the block to be predicted based on the area amplitude histogram or statistical results.

[0297] Optionally, determining the intra-prediction mode usage of multiple coded blocks includes: first determining a coded region or a decoded region, and then determining the intra-prediction mode usage of coded blocks in the coded region, or determining the intra-prediction mode usage of decoded blocks in the decoded region.

[0298] Optionally, the intra-prediction mode usage of at least one image block may include: first determining at least one encoded region or at least one decoded region, then determining the intra-prediction mode usage of encoded blocks in the encoded region, and / or determining the intra-prediction mode usage of decoded blocks in the decoded region.

[0299] Reference Figure 10 In the encoded region, there are coded blocks 4 and 6 that are adjacent to the block to be predicted, and coded blocks 1, 2, 3, 5, 7 and 8 that are not adjacent to the block to be predicted.

[0300] Reference Figure 11 In the decoded region, there are decoded blocks 4 and 6 that are adjacent to the block to be predicted, and decoded blocks 1, 2, 3, 5, 7 and 8 that are not adjacent to the block to be predicted.

[0301] Optionally, in this embodiment, the reference region, reference block, and / or reference pixel are determined or obtained according to at least one of methods nine to fourteen in the fifth embodiment.

[0302] In this embodiment, predicting at least one sub-block of the current block using the OBIC mode can better capture and preserve important details and features in the image, which helps to improve the quality of the reconstructed image and thus enhance the encoding and decoding effect.

[0303] Method E, template-based intra-frame coding mode derivation mode;

[0304] Optionally, the basic principle of the TIMD pattern is to determine or generate a predicted value for the template by using each pattern in the MPM list, then calculate the SATD of the template predicted value and the reconstructed value, select the pattern with the smallest SATD as the TIMD pattern, and use it for the prediction of the current block.

[0305] Reference Figure 12 The specific implementation process of the TIMD mode may include: determining a template for the block to be predicted, which can be constructed using reconstructed pixels on the left or top of the block; determining or generating predicted values ​​for the template by combining each mode in the MPM with reference pixels of the template; selecting the mode with the smallest SATD value from the MPM modes for prediction based on the predicted value and the SATD of the reconstructed value; and so on. Figure 12 In the diagram, the width of the block to be predicted is M, the height is H, and the widths of the templates are L1 and L2, respectively.

[0306] Optionally, the construction of the MPM list depends on the patterns of the neighboring blocks of the current block. The predicted patterns of the neighboring blocks are usually spatially related to the current block (e.g., the continuity of the edges of the same object). Therefore, the MPM list can use this characteristic to quickly narrow down the candidate range and effectively reduce the number of bits for pattern encoding.

[0307] Optionally, in this embodiment, the reference region, reference block, and / or reference pixel are determined or obtained according to at least one of methods nine to fourteen in the fifth embodiment.

[0308] In this embodiment, at least one sub-block of the current block is predicted using the TIMD mode. The optimal intra-frame prediction mode is derived using the template region, thereby reducing the coding bits of the mode information and thus reducing transmission overhead. The TIMD mode selects the optimal mode from the MPM list by calculating the SATD of the predicted pixels and reconstructed pixels in the template region. This mechanism can more accurately select the prediction mode suitable for the current block, thereby improving the encoding and decoding quality.

[0309] Mode F, local illumination compensation mode;

[0310] Optionally, the LIC mode is a mode designed to improve prediction accuracy by eliminating prediction errors caused by local illumination variations (such as changes in brightness or contrast). The LIC mode scales and offsets the predicted values ​​of the reference block by establishing a linear model between adjacent reconstructed pixels (templates) of the current block and the reference block, making it more adaptable to the illumination conditions of the current block. The mathematical form of the linear model is:

[0311] Predicted value = a × P + b;

[0312] a is the scaling factor, b is the offset, and P is the original predicted value of the reference block.

[0313] Reference Figure 13 The current block is the region to be encoded. Its adjacent reconstructed samples (e.g., the encoded pixels above or to the left) are used to derive the linear model parameters. The reference block can be an adjacent encoded region of the current block or a reference frame block in the temporal domain. Its pixel values ​​have been obtained through reconstructed samples (i.e., pixels reconstructed at the decoder). Using the pixel values ​​of the current sample and the reference sample, the optimal scaling factor a and offset b are calculated using the least squares method or a similar method. Then, the original prediction value P of the reference block is substituted into the linear model to obtain the compensated prediction value. This step compensates for the illumination difference (such as brightness shift or contrast change) between the reference block and the current block, making the prediction more consistent with the actual content of the current block. The compensated prediction value (predictor) is used for inter-frame or intra-frame prediction of the current block, reducing the amount of residual data and thus improving coding efficiency.

[0314] Optionally, in this embodiment, the reference region, reference block, and / or reference pixel are determined or obtained according to at least one of methods nine to fourteen in the fifth embodiment.

[0315] In this embodiment, predicting at least one sub-block of the current block using LIC mode can compensate for brightness differences caused by illumination changes, thereby improving the prediction accuracy of the current block and thus improving the encoding and decoding quality.

[0316] Method G is based on neural network intra-frame prediction patterns;

[0317] Alternatively, the NNIP model is a prediction method using a neural network model. A neural network model is a computational model that mimics how neurons in the human brain process information. It performs various tasks by learning complex patterns and features in data. A neural network model consists of multiple layers of neurons, including an input layer, at least one hidden layer, and an output layer. Each neuron receives an input signal, which is then weighted and summed before being passed through a non-linear activation function to produce an output. The weights of these connected neurons determine the degree of influence of the input signal on the output, while the bias is used to adjust the activation threshold of the neurons.

[0318] Alternatively, activation functions, such as ReLU, Sigmoid, or Tanh, can endow neural networks with the ability to model nonlinearity, enabling them to solve problems that linear models cannot.

[0319] Reference Figure 14 The specific implementation process of the NNIP pattern may include: determining or obtaining the neural network model corresponding to the current block, and using the reference template of the neural network of the current block (e.g., Figure 14 The reconstructed reference pixels of the reference template 1 on the left, the reference template 2 above, and the reference template 3 on the upper left are used as input to the neural network model. The neural network model obtains the output of the neural network model by utilizing the relationship between the current block and its reference templates learned by the neural network model, such as the predicted value of the current block, i.e., the predictor.

[0320] Optionally, the reference template includes at least one of a reference region, a reference block, and / or a reference pixel.

[0321] Optionally, a neural network model may include an input layer, a hidden layer, a dropout layer, and an output layer.

[0322] Optionally, the input layer is the first layer that receives input data, in which each node (or neuron) typically represents a feature of the input data.

[0323] Optionally, hidden layers are layers located between the input layer and the output layer. There can be at least one hidden layer. These layers process the input data in a weighted manner. Hidden layers include at least one of the following: fully connected layer, convolutional layer, pooling layer, recursive layer, and dropout layer.

[0324] Optionally, each neuron in a dense or fully connected layer is connected to each neuron in the layer above it.

[0325] Alternatively, convolutional layers are used to extract local features from the input data and are commonly used in image processing.

[0326] Optionally, a pooling layer is used for downsampling to reduce the amount of data and computation.

[0327] Alternatively, a recurrent layer, such as LSTM or GRU, can be used to process sequential data.

[0328] Alternatively, a dropout layer is used to randomly "drop out" (i.e. temporarily set the output to zero) a portion of neurons during model training. This can reduce the complex co-adaptation relationships between neurons, improve the model's generalization ability, and reduce the risk of overfitting.

[0329] Optionally, dropout layers can be added between multiple fully connected layers to prevent neurons in these layers from becoming overly dependent on the specific output of the previous layer, thereby enhancing the model's generalization ability; and / or dropout layers can be added between convolutional layers and fully connected layers to help reduce overfitting introduced by fully connected layers.

[0330] Optionally, the output layer is the layer that determines or generates the final prediction result.

[0331] Optionally, in this embodiment, the reference region, reference block, and / or reference pixel are determined or obtained according to at least one of methods nine to fourteen in the fifth embodiment.

[0332] Optionally, a first neural network model for applying the NNIP mode can be determined or obtained from a set of neural network models; and prediction processing is performed on at least one sub-block of the current block according to the first neural network model.

[0333] Optionally, the first neural network model can be one or more models.

[0334] Optionally, a first neural network model is determined or obtained from a set of neural network models based on at least one reference region, at least one reference block and / or at least one reference pixel of at least one sub-block of the current block, and a preset model mapping / correspondence rule.

[0335] Optionally, the pre-training may include a set of neural network models, which may include at least one of the following: a neural network model based on fully connected layers, a neural network based on convolutional layers, and a neural network based on a mixture of convolutional and fully connected layers.

[0336] Optionally, a neural network model based on fully connected layers, also known as a multilayer perceptron (MLP), includes an input layer, one or more hidden layers (i.e., fully connected layers), and an output layer. Each layer contains a certain number of neurons, and all neurons between adjacent layers are interconnected, i.e., "fully connected" between layers.

[0337] Optionally, dropout layers may be set between at least one set of fully connected layers in a neural network model based on fully connected layers to help reduce overfitting introduced by the fully connected layers.

[0338] Reference Figure 15A fully connected neural network model includes an input layer, multiple hidden layers (i.e., fully connected layers), and an output layer. The fully connected layers include linear layers and activation function layers (e.g., LeakyReLU layers). The model inputs pixels from a reference template of at least one sub-block of the current block. Since the pixels of the reference template may exist in the form of a two-dimensional or three-dimensional matrix, to enable processing of this data in the fully connected layer, the input layer can be flattened, flattening the pixels of at least one reference template into a one-dimensional vector f. Then, a nonlinear transformation using the ReLU activation function is performed to determine or obtain f. i This operation introduces nonlinearity, allowing the model to learn more complex patterns, f i The input will be fed into a network composed of fully connected layers. Each layer will perform linear transformations and activation function processing, and output a new feature vector f. r This is to extract higher-level abstract features from the input, and then to transform the high-level features f r With low-level features f i Concatenate along the channel dimension to determine or generate the feature vector f. c ; the feature vector f c The prediction results are fed into the linear layer and output through the output layer to produce the prediction results of at least one sub-block of the current block.

[0339] Optionally, a convolutional neural network (CNN) includes an input layer, at least one hidden layer, and an output layer. The hidden layer includes a convolutional layer for extracting features. Each convolutional layer may be followed by a pooling layer to reduce spatial size and computational complexity. Each convolutional layer may also be followed by a non-linear activation function (e.g., ReLU) to introduce non-linearity, enabling the network to learn more complex patterns. As the depth of the convolutional neural network increases, it can automatically learn feature representations from low to high levels, improving prediction accuracy.

[0340] Reference Figure 16 A convolutional neural network (CNN) includes an input layer, multiple hidden layers, and an output layer. The hidden layers include convolutional layers, activation function layers (e.g., Leaky ReLU layers), and linear layers. Pixels of a reference template for the current block are input into the CNN. At least one convolutional layer performs convolution operations on the pixels of at least one reference template, and the ReLU activation function after each convolutional layer performs a non-linear transformation to obtain the feature f. 1 f 2 and f 3 ; will feature f 1 f 2 and f3 The features are fused and concatenated along the channel dimension to determine or generate a new merged feature f containing three features. This preserves information from different convolutional paths, allowing the model to combine low-level and high-level features to enhance representational power. The fused feature f is then processed again through the ReLU activation function to obtain fi. i This additional activation step helps to further enhance the model's expressiveness and flexibility, making f i The data is fed into a linear layer to obtain the prediction results for at least one sub-block of the current block.

[0341] Optionally, a neural network model based on hybrid convolutional and fully connected layers is an architecture that combines the advantages of CNNs and MLPs. It includes convolutional layers and fully connected layers. Convolutional layers can automatically learn the spatial hierarchy of input data, such as features like edges, textures, and shapes in images. Convolutional operations can effectively capture local patterns and reduce model complexity through parameter sharing mechanisms. Fully connected layers can flatten the features extracted by the convolutional layers into one-dimensional vectors, which are then fed into a series of fully connected layers. These layers are responsible for combining low-level features to determine or generate higher-level abstract representations. A neural network model based on hybrid convolutional and fully connected layers can combine the advantages of two different types of layers, enabling efficient feature extraction from input data and accurate predictions based on those features.

[0342] Optionally, the neural network model based on hybrid convolutional and fully connected layers includes: an input layer, at least one convolutional layer, at least one fully connected layer, and an output layer.

[0343] Reference Figure 17 The neural network model based on hybrid convolutional and fully connected layers includes an input layer, multiple hidden layers, and an output layer. The hidden layers include convolutional layers, fully connected layers (i.e., linear layers), and activation function layers (e.g., Leaky ReLU layers). The model inputs pixels from the reference template of the current block. At least one convolutional layer performs a convolution operation on the pixels of at least one reference template. A non-linear transformation is then performed using the ReLU activation function after each convolutional layer to obtain the feature f. 1 f 2 and f 3 ; will feature f 1 f 2 and f 3 The fusion and splicing are performed along the channel dimension to determine or generate a new merged feature f containing three features. 4 This preserves information from different convolutional paths and allows the model to combine low-level and high-level features, enhancing representational power. It obtains features f through multiple stacked linear layers.5 , will feature f 4 and f 5 The data is spliced ​​along the channel dimension to form a fused feature f. i and through at least one convolutional layer to fuse features f i Perform convolution processing to obtain the prediction results of at least one sub-block of the current block.

[0344] Optionally, the construction of a neural network model mainly includes: dataset construction and neural network construction and training.

[0345] Alternatively, during the dataset construction process, the data used for intra-frame encoding in the video encoding and decoding process can be extracted from the ECM codebase and used to construct the dataset.

[0346] Optionally, the dataset includes the following fields: frame index (poc), image patch (e.g., coding unit) location information (x, y), image patch size (width, height), channel type (compidx), split angle (splitdir), pixel value offset (shiftWeighted) and compensation amount (offsetWeighted), original image pixel value (org) of the image patch, and pixel value of the reference patch of the image patch. After constructing the dataset, it needs to be further processed and divided to adapt to the input requirements of the neural network.

[0347] Optionally, the predicted pixel of the neural network is defined as p. ij Let the target label of the predicted pixel be the original pixel p. ij =org ij Define the fusion matrix k i,j This indicates that the weights of each pixel in src0 (reference block 0) and src1 (reference block 1) are fused using the GPM tool, and the fused predicted pixel satisfies: p i,j =k i,j ·src0 i,j +(1-k i,j )·src1 i,j Let the target label of the predicted pixel be the original pixel value p. i,j =org i,j The target label of the fusion matrix is ​​obtained by solving for:

[0348]

[0349] Optionally, based on the position and size of the current block, a reference region adjacent to the current block is extracted from the decoded reconstructed frame. This region, also known as a reference template, includes three areas: top, left, and topleft. Figure 14 As shown.

[0350] Optionally, during the construction and training of the neural network model, the neural network function can be designed first, for example, predicting the current block's predicted value based on intra-frame reference pixels.

[0351] Optionally, the reconstructed reference pixels within the reference templates of the left, top, and topleft sides of the current block can be used as input, and the predicted value p of the current block can be used as output. The target label of the neural network model is that the predicted value p of the current block is equal to the original pixel value corresponding to the current block.

[0352] In this embodiment, the powerful nonlinear modeling capability of the neural network model can automatically learn the complex mapping relationship between input and output from a large amount of data. Therefore, the neural network can identify and utilize the subtle relationship between the current block and its reference template. For example, it can effectively capture the temporal correlation between frames and the spatial correlation within frames, understand the continuity and change patterns between different frames or adjacent regions within the same frame; and / or perceive the consistency of local features such as edges, textures, etc., even if these features are shifted or deformed between different frames. Through feature extraction at least one level, it ensures that the prediction results are not only accurate in local details, but also conform to the actual content in the overall layout, so as to improve the accuracy of the current block prediction from multiple dimensions and improve the encoding and decoding quality in the video encoding and / or decoding process.

[0353] Optionally, the first model is applied to a neural network-based intra-frame prediction mode.

[0354] Optionally, the first model is a neural network model.

[0355] Optionally, the first model is determined or obtained according to at least one of the following methods H to J:

[0356] Method H, at least one of the following: width, height, block size, and block area of ​​the current block;

[0357] Optionally, a first model corresponding to the target model type is determined from at least one model type based on at least one of the width, height, size, and area of ​​the current block.

[0358] Optionally, based on at least one of the geometric attributes of the current block (i.e., width, height, block size, and block area), the model type most suitable for processing these features is selected, so that the first model can more accurately reflect the characteristics of the input data and improve the prediction effect.

[0359] Optionally, the structure of the first model is determined based on at least one of the width, height, size, and area of ​​the current block.

[0360] Optionally, since at least one of the width, height, size, and area of ​​the current block is associated with at least one reference region and / or at least one reference block of the current block, a first model suitable for processing the reference region and / or reference block data of the current block can be determined or obtained by at least one of the width, height, size, and area of ​​the current block.

[0361] Optionally, the specific structure of the selected model can be adjusted based on at least one of the geometric attributes of the current block (i.e., width, height, block size, and block area). For example, when the first model is a neural network model, the number of hidden layers (e.g., convolutional layers, fully connected layers), filter size, pooling strategy, etc. can be adjusted based on at least one of the width, height, size, and area of ​​the current block.

[0362] Optionally, the structure of the first model can be determined based on the width and height of the current block and the first mapping table. Optionally, the first mapping table can be as shown in Table 1 below:

[0363] Table 1

[0364]

[0365]

[0366] Optionally, the structure of the first model can be determined based on the block size of the current block and the second mapping table. Optionally, the second mapping table can be as shown in Table 2 below:

[0367] Table 2

[0368] <![CDATA[The block size of the current block is smaller than X4]]> First Structure <![CDATA[The block size of the current block is greater than X4 and less than X5]]> Second Structure <![CDATA[The block size of the current block is greater than X5 and less than X6]]> Third Structure <![CDATA[The block size of the current block is greater than X6 and less than X7]]> Fourth Structure

[0369] Optionally, the structure of the first model can be determined based on the block size of the current block and the third mapping table. Optionally, the third mapping table can be as shown in Table 3 below:

[0370] Table 3

[0371] <![CDATA[The block area of the current block is less than X8]]> First Structure <![CDATA[The block area of the current block is greater than X8 and less than X9]]> Second structure <![CDATA[The block area of the current block is greater than X9 and less than X 10 > Third Structure <![CDATA[The block area of the current block is greater than X 10 and less than X 11 > Fourth Structure

[0372] Optionally, if the current block, at least one reference region of the current block, and / or at least one reference block do not match any of the neural network models, the reconstructed reference pixels of the current block, at least one reference region of the current block, and / or at least one reference block can be transposed and / or downsampled to match the geometric properties of the current block, at least one reference region of the current block, and / or at least one reference block with the neural network model (e.g., the same size). Then, the reconstructed reference pixels obtained by transposition and / or downsampling are input into the neural network model to obtain the prediction result of the current block. By transposing or downsampling, more current blocks, reference regions, and / or reference blocks can be matched with the target model or target model type, thereby reducing the overall number of target models or target model types.

[0373] Optionally, the aforementioned at least one reference region and / or at least one reference block of the current block can be at least one reference region and / or at least one reference block of at least one sub-block of the current block.

[0374] Optionally, the number of neural network models is N, for example, including neural network model NN0, neural network model NN1, neural network model NN2... neural network model NN N-1 Neural network model NN0, neural network model NN1, neural network model NN2... neural network model NN N-1 These correspond to block sizes W0xH0, W1xH1, W2xH2, ..., W respectively. N- 1xH N-1 .

[0375] In this embodiment, based on at least one of the width, height, block size, and block area of ​​the current block, the model most suitable for processing these features is determined or obtained, so that the first model can more accurately reflect the characteristics of the input data, improve the prediction accuracy of the current block, and thus improve the prediction effect in the video encoding and / or decoding process.

[0376] Method I: at least one of the following: width, height, size, and area of ​​at least one reference region of the current block;

[0377] Optionally, a first model corresponding to the target model type is determined from at least one model type based on at least one of the width, height, size, and area of ​​at least one reference region of the current block.

[0378] Optionally, at least one reference region of the current block can be at least one reference region of at least one sub-block of the current block.

[0379] Optionally, based on at least one of the geometric attributes (i.e., width, height, block size, and block area) of at least one reference region of the current block, the model type most suitable for processing these features is selected, so that the first model can more accurately reflect the characteristics of the input data and improve the prediction effect.

[0380] Optionally, the structure of the first model is determined based on at least one of the width, height, size, and area of ​​at least one reference region of the current block.

[0381] Optionally, the specific structure of the selected model can be adjusted based on at least one of the geometric properties (i.e., width, height, block size, and block area) of at least one reference region of the current block. For example, when the first model is a neural network model, the number of hidden layers (e.g., convolutional layers, fully connected layers), filter size, pooling strategy, etc. can be adjusted based on at least one of the width, height, size, and area of ​​at least one reference region.

[0382] Optionally, the structure of the first model can be determined based on the width and height of at least one reference region of the current block and a fourth mapping table. Optionally, the fourth mapping table can be as shown in Table 4 below:

[0383] Table 4

[0384] <![CDATA[Either the width or the height of the reference area is less than X1]]> First Structure <![CDATA[The width and height of the reference area are both greater than or equal to X1, and either the width or the height is less than X2]]> Second Structure <![CDATA[The width and height of the reference area are both greater than or equal to X2]]> Third Structure <![CDATA[The width and height of the reference area are both greater than or equal to X2, and either the width or the height is less than X3]]> Fourth Structure

[0385] Optionally, the structure of the first model can be determined based on the block size of at least one reference region of the current block and the fifth mapping table. Optionally, the fifth mapping table can be as shown in Table 5 below:

[0386] Table 5

[0387] <![CDATA[The block size of the reference area is smaller than X4]]> First Structure <![CDATA[The block size of the reference area is greater than X4 and less than X5]]> Second Structure <![CDATA[The block size of the reference area is greater than X5 and less than X6]]> Third Structure <![CDATA[The block size of the reference area is greater than X6 and less than X7]]> Fourth Structure

[0388] Optionally, the block size includes at least one of the block's width, height, scale, depth, area, resolution, and number of pixels. Optionally, X4 to X7 can be a preset threshold corresponding to at least one of the block size's width, height, scale, depth, area, resolution, and number of pixels.

[0389] Optionally, the structure of the first model can be determined based on the block size of at least one reference region of the current block and the sixth mapping table. Optionally, the sixth mapping table can be as shown in Table 6 below:

[0390] Table 6

[0391] <![CDATA[The block area of the reference region is less than X8]]> First Structure <![CDATA[The block area of the reference region is greater than X8 and less than X9]]> Second Structure <![CDATA[The block area of the reference region is greater than X9 and less than X 10 > Third Structure <![CDATA[The block area of the reference region is greater than X 10 and less than X 11 > Fourth Structure

[0392] In this embodiment, based on at least one of the width, height, size, and area of ​​at least one reference region of the current block, the model most suitable for processing these features is determined or obtained, so that the first model can more accurately reflect the characteristics of the input data, improve the prediction accuracy of the current block, and thus improve the prediction effect in the video encoding and / or decoding process.

[0393] Method J: at least one of the width, height, size, and area of ​​at least one reference block of the current block;

[0394] Optionally, a first model corresponding to the target model type is determined from at least one model type based on at least one of the width, height, size, and area of ​​at least one reference block of the current block.

[0395] Optionally, at least one reference block of the current block can be at least one reference block of at least one sub-block of the current block.

[0396] Optionally, based on at least one of the geometric attributes (i.e., width, height, block size, and block area) of at least one reference block of the current block, the model type most suitable for processing these features is selected, so that the first model can more accurately reflect the characteristics of the input data and improve the prediction effect.

[0397] Optionally, the structure of the first model is determined based on at least one of the width, height, size, and area of ​​at least one reference block of the current block.

[0398] Optionally, the specific structure of the selected model can be adjusted based on at least one of the geometric properties (i.e., width, height, block size, and block area) of at least one reference block of the current block. For example, when the first model is a neural network model, the number of hidden layers (e.g., convolutional layers, fully connected layers), filter size, pooling strategy, etc. can be adjusted based on at least one of the width, height, size, and area of ​​at least one reference block.

[0399] Optionally, the structure of the first model can be determined based on the width and height of at least one reference block of the current block and the seventh mapping table. Optionally, the seventh mapping table can be as shown in Table 7 below:

[0400] Table 7

[0401] <![CDATA[Either the width or the height of the reference block is less than X1]]> First Structure <![CDATA[The width and height of the reference block are both greater than or equal to X1, and either the width or the height is less than X2]]> Second structure <![CDATA[The width and height of the reference block are both greater than or equal to X2]]> Third Structure <![CDATA[The width and height of the reference block are both greater than or equal to X2, and either the width or the height is less than X3]]> Fourth Structure

[0402] Optionally, the structure of the first model can be determined based on the block size of at least one reference block of the current block and the eighth mapping table. Optionally, the eighth mapping table can be as shown in Table 8 below:

[0403] Table 8

[0404] <![CDATA[The block size of the reference block is smaller than X4]]> First Structure <![CDATA[The block size of the reference block is greater than X4 and less than X5]]> Second structure <![CDATA[The block size of the reference block is greater than X5 and less than X6]]> Third Structure <![CDATA[The block size of the reference block is greater than X6 and less than X7]]> Fourth Structure

[0405] Optionally, the block size includes at least one of the block's width, height, scale, depth, area, resolution, and number of pixels. Optionally, X4 to X7 can be a preset threshold corresponding to at least one of the block size's width, height, scale, depth, area, resolution, and number of pixels.

[0406] Optionally, the structure of the first model can be determined based on the block size of at least one reference block of the current block and the ninth mapping table. Optionally, the ninth mapping table can be as shown in Table 9 below:

[0407] Table 9

[0408] <![CDATA[The block area of the reference block is less than X8]]> First Structure <![CDATA[The block area of the reference block is greater than X8 and less than X9]]> Second structure <![CDATA[The block area of the reference block is greater than X9 and less than X 10 > Third Structure <![CDATA[The block area of the reference block is greater than X 10 and less than X 11 > Fourth Structure

[0409] In this embodiment, based on at least one of the width, height, size, and area of ​​at least one reference block of the current block, the model most suitable for processing these features is determined or obtained, so that the first model can more accurately reflect the characteristics of the input data, improve the prediction accuracy of the current block, and thus improve the prediction effect in the video encoding and / or decoding process.

[0410] Fourth embodiment

[0411] Based on any of the above embodiments, a fourth embodiment is proposed.

[0412] In this embodiment, the method for determining or obtaining the prediction mode includes at least one of methods five to eight:

[0413] Method 5: Determine or obtain the prediction mode based on at least one reference region, at least one reference block and / or at least one reference pixel of at least one sub-block of the current block;

[0414] Optionally, the prediction mode determined or obtained based on at least one reference region, at least one reference block and / or at least one reference pixel of at least one sub-block of the current block is an angular prediction mode and / or a non-angular prediction mode.

[0415] Optionally, based on the prediction derivation mode and at least one reference region, at least one reference block and / or at least one reference pixel of at least one sub-block of the current block, the prediction mode is determined or obtained as an angle prediction mode and / or a non-angle prediction mode.

[0416] Optionally, the angular prediction modes and / or non-angular prediction modes of the determined or obtained sub-blocks may be the same or different.

[0417] Optionally, processing module A10 determines or obtains the angular prediction mode and / or non-angular prediction mode of the sub-blocks based on the same prediction derivation mode and at least one reference region, at least one reference block, and / or at least one reference pixel of at least one sub-block of the current block. The determined or obtained angular prediction modes and / or non-angular prediction modes of the various sub-blocks may be the same or different, and the prediction derivation modes include DIMD mode, OBIC mode, and / or TIMD mode.

[0418] Optionally, by utilizing features (such as texture, gradient, brightness distribution, etc.) of the reconstructed reference region, reference block, and / or reference pixel surrounding at least one sub-block of the current block, a prediction mode best suited to at least one sub-block of the current block can be derived or selected.

[0419] Optionally, the reference template of at least one sub-block of the current block includes at least one reference region of at least one sub-block of the current block, at least one reference block, and / or at least one reference pixel.

[0420] Optionally, a prediction mode for predicting at least one sub-block of the current block is determined from candidate modes based on at least one reference region, at least one reference block, and / or at least one reference pixel of at least one sub-block of the current block.

[0421] Optionally, a prediction mode for predicting the first sub-block of the current block is determined from candidate modes based on at least one reference region, at least one reference block, and / or at least one reference pixel of the first sub-block of the current block. Optionally, the first sub-block can be any sub-block in the current block, such as a sub-block that has been predicted or a sub-block that has not yet been predicted.

[0422] Optionally, candidate modes include: prediction derivation modes, such as DIMID mode, OBIC mode, TIMD mode; and / or candidate modes may include: angle prediction modes and / or non-angle prediction modes, where non-angle prediction modes include: NNIP mode, EIP mode, DC mode and / or planar mode; and / or candidate modes may include: compensation modes, such as LIC mode.

[0423] Optionally, for ease of description, the prediction mode used to perform prediction processing on at least one sub-block of the current block, which is determined or obtained through the prediction derivation mode, is referred to as the first mode. Optionally, the first mode is an angle prediction mode or a non-angle prediction mode.

[0424] Optionally, each sub-block adopts the same prediction derivation mode to determine the first mode of each sub-block. The first modes of each sub-block can be the same or different. For example, the first mode of the first sub-block is the first angle prediction mode, the first mode of the second sub-block is the second angle prediction mode, the first mode of the third sub-block is the third angle prediction mode, and the first mode of the fourth sub-block is the fourth angle prediction mode; or, for example, the first mode of the first sub-block is the first angle prediction mode, the first mode of the second sub-block is the first non-angle prediction mode, the first mode of the third sub-block is the second non-angle prediction mode, and the first mode of the fourth sub-block is the second angle prediction mode; or, for example, the first modes of the first to fourth sub-blocks are all non-angle prediction modes; or, for example, the first modes of the first to fourth sub-blocks are all angle prediction modes.

[0425] Optionally, when performing prediction processing on at least one sub-block of the current block using the DIMD mode, a reference template of at least one sub-block of the current block is determined or obtained based on at least one reference region, at least one reference block and / or at least one reference pixel of at least one sub-block of the current block. The gradient magnitude value and gradient direction of the pixels in the reference template are determined, the statistical results of the reference template with respect to the intra-prediction direction are determined, the intra-prediction direction of at least one sub-block of the current block is determined based on the statistical results, and the mode corresponding to the intra-prediction direction is determined as the first mode (e.g., angle mode 34) and used to perform prediction processing on at least one sub-block of the current block.

[0426] Optionally, when performing prediction processing on at least one sub-block of the current block using the DIMD mode, a reference template of the first sub-block is determined or obtained based on at least one reference region, at least one reference block, and / or at least one reference pixel of the first sub-block of the current block. The gradient magnitude and gradient direction of the pixels in the reference template are determined, the statistical results of the reference template with respect to the intra-prediction direction are determined, the intra-prediction direction of the first sub-block is determined based on the statistical results, and the mode corresponding to the intra-prediction direction is determined as the first mode and used for prediction processing on the first sub-block.

[0427] Optionally, when performing prediction processing on at least one sub-block of the current block using OBIC mode, the intra-prediction mode usage of multiple reference blocks (i.e., adjacent coding blocks and / or non-adjacent coding blocks) of the current block is determined, the area amplitude value and intra-prediction direction of the multiple reference blocks are calculated, the area amplitude histogram or statistical results of the multiple reference blocks with respect to the intra-prediction direction are determined, the intra-prediction direction of the block to be predicted is determined based on the area amplitude histogram or statistical results, and the mode corresponding to the intra-prediction direction is determined as the first mode of each sub-block (e.g., angle mode 34), and each sub-block of the current block is performed prediction processing based on the first mode determined for each sub-block.

[0428] Optionally, when performing prediction processing on at least one sub-block of the current block using OBIC mode, the intra-prediction mode usage of multiple reference blocks (i.e., adjacent coding blocks and / or non-adjacent coding blocks) of the first sub-block of the current block is determined, the area amplitude value and intra-prediction direction of the multiple reference blocks are calculated, the area amplitude histogram or statistical results of the multiple reference blocks with respect to the intra-prediction direction are determined, the intra-prediction direction of the block to be predicted is determined based on the area amplitude histogram or statistical results, and the mode corresponding to the intra-prediction direction is determined as the first mode and used to perform prediction processing on the first sub-block of the current block.

[0429] Optionally, when performing prediction processing on at least one sub-block of the current block using TIMD modes, a reference template for at least one sub-block of the current block is determined. The reference template includes a reference region, a reference block, and / or a reference pixel. Predicted values ​​are determined or generated for the reference template using modes in the MPM list. Based on the SATD of the predicted values ​​and the reconstructed values, a first mode is determined from the MPM list and used to perform prediction processing on at least one sub-block of the current block. Optionally, the MPM list is the MPM list of the current block, or it can be the MPM list of the sub-blocks of the current block.

[0430] Optionally, when performing prediction processing on at least one sub-block of the current block using the TIMD mode, a reference template for the first sub-block of the current block is determined. The reference template includes a reference region, a reference block, and / or a reference pixel. Predicted values ​​are determined or generated for the reference template using each mode in the MPM list. Based on the SATD of the predicted values ​​and the reconstructed values, a prediction mode is determined from the MPM list and used to perform prediction processing on the first sub-block of the current block.

[0431] Optionally, in this embodiment, the reference region, reference block, and / or reference pixel are determined or obtained according to at least one of methods nine to fourteen in the fifth embodiment.

[0432] In this embodiment, since the reference region, reference block and / or reference pixel of at least one sub-block of the current block are usually highly correlated with each sub-block in space or time, the prediction mode obtained by using this information can capture local texture, edge or brightness changes more accurately, thereby improving the prediction accuracy for the current block and improving the prediction effect of video encoding and / or decoding.

[0433] Method 6: Determine or obtain the prediction pattern corresponding to the second sub-block of the current block based on the prediction patterns corresponding to the adjacent blocks, non-adjacent blocks, and / or the first sub-block of the current block;

[0434] Optionally, the prediction mode corresponding to the second sub-block of the current block is determined or obtained based on the prediction modes corresponding to the adjacent blocks and / or non-adjacent blocks of the current block.

[0435] Optionally, the prediction modes corresponding to adjacent blocks and / or non-adjacent blocks include angle prediction modes and non-angle prediction modes. The angle prediction mode determines or generates the prediction value by specifying an angle value (such as 45°, 30°, etc.) and copying the reference pixels of the graphic block into the current block along that direction. The non-angle prediction modes include: DIMD mode, OBIC mode, TIMD mode, NNIP mode and / or EIP mode; the prediction mode corresponding to the second sub-block includes the angle prediction mode.

[0436] Optionally, if the neighboring and / or non-neighboring blocks of the current block use non-angle prediction modes, the angular prediction mode they point to will be determined based on these non-angle prediction modes, and the angular mode will be used in the encoding change process. In the case where the neighboring and / or non-neighboring blocks of the current block use non-angle prediction modes, the angular mode pointed to by the non-angle mode is added to the candidate mode list (e.g., MPM list) as a candidate for the angular prediction mode of the second sub-block. Since the prediction modes of neighboring or non-neighboring blocks (even non-angle modes) are usually related to the texture features of the current block, by adding the angular modes pointed to by these non-angle modes to the candidate list, the directional texture of the current block can be captured better and the prediction accuracy can be improved.

[0437] Optionally, the prediction patterns corresponding to the adjacent blocks, non-adjacent blocks, and / or the first sub-block of the current block are used to determine the most likely pattern list and the least likely pattern list, and the prediction pattern corresponding to the second sub-block of the current block is determined or obtained based on the most likely pattern list and the least likely pattern list.

[0438] Optionally, a candidate mode list for the second sub-block is determined or obtained based on the prediction modes of the adjacent and / or non-adjacent blocks of the current block, and a prediction mode is determined or obtained from the candidate mode list based on at least one reference region, at least one reference block, and / or at least one reference pixel of the second sub-block, for use in prediction processing of the second sub-block.

[0439] Optionally, the prediction mode corresponding to the second sub-block of the current block is determined or obtained based on the prediction modes corresponding to the adjacent blocks, non-adjacent blocks and / or the first sub-block of the current block, and at least one reference region, at least one reference block and / or at least one reference pixel of at least one sub-block of the current block.

[0440] Optionally, the prediction mode of the current block is determined or obtained based on the prediction modes corresponding to the adjacent and / or non-adjacent blocks of the current block, and at least one reference region, at least one reference block and / or at least one reference pixel of at least one sub-block of the current block.

[0441] Optionally, the prediction mode corresponding to the second sub-block of the current block is determined or obtained based on the prediction mode corresponding to the first sub-block of the current block. Optionally, the prediction mode corresponding to the first sub-block and the prediction mode corresponding to the second sub-block can be the same.

[0442] Optionally, the angle prediction mode corresponding to the second sub-block of the current block can be determined or obtained based on the angle prediction mode corresponding to the first sub-block of the current block.

[0443] Optionally, the angle prediction mode corresponding to the second sub-block of the current block is determined or obtained based on the non-angle prediction mode corresponding to the first sub-block of the current block.

[0444] In this embodiment, since the adjacent blocks, non-adjacent blocks and / or the first sub-block of the current block are usually highly correlated with the second sub-block of the current block, the prediction pattern obtained by using the graph block can more accurately capture the features of the second sub-block, thereby improving the prediction accuracy for the second sub-block and the current block, and improving the prediction effect of video encoding and / or decoding.

[0445] Method 7: Determine or obtain the predicted pattern based on the list of most likely patterns and / or the list of non-most likely patterns corresponding to the current block;

[0446] Optionally, when performing prediction processing on at least one sub-block of the current block using the TIMD mode, it is necessary to determine the candidate prediction mode of the current block through the MPM list. Since the sub-block is part of the current block, there is a strong correlation between the two. Therefore, when determining the prediction mode corresponding to the sub-block of the current block, the prediction mode corresponding to the sub-block can also be determined based on this inherent correlation and the MPM list of the current block. Optionally, by at least partially reusing the MPM list of the current block, this application can improve the prediction accuracy without increasing the amount of computation.

[0447] Reference Figure 18 The solid line block represents the current block and is divided into sub-blocks A, B, C, and D. Some adjacent blocks of the current block will be sampled as pixels. The prediction modes corresponding to the pixel samples will be analyzed to determine or generate the most likely candidate mode set (i.e., MPM list) for the current block. Then, the prediction mode corresponding to at least one sub-block of the current block can be determined or obtained based on the most likely mode list corresponding to the current block.

[0448] Reference Figure 18 The pixel samples used to determine or generate the MPM list of the current block include at least one of the following positions: left (L), top (A), bottom left (BL), top right (AR), and top left (AL).

[0449] Optionally, the prediction mode corresponding to at least one sub-block of the current block is determined or obtained based on the list of most likely modes corresponding to the current block, and at least one reference region, at least one reference block and / or at least one reference pixel of at least one sub-block of the current block.

[0450] Optionally, a candidate mode list is determined or obtained based on the most likely mode list corresponding to the current block and the prediction modes corresponding to the adjacent blocks, non-adjacent blocks and / or the first sub-block of the current block. The prediction mode corresponding to at least one sub-block of the current block is determined or obtained from the candidate mode list based on at least one reference region, at least one reference block and / or at least one reference pixel of at least one sub-block of the current block.

[0451] Optionally, the prediction mode can be determined or obtained based on the list of non-most likely modes corresponding to the current block. Optionally, the prediction modes included in the list of non-most likely modes may be different from the prediction modes included in the list of most likely modes.

[0452] Optionally, at least one prediction pattern can be determined or obtained based on the list of non-most likely patterns corresponding to the current block, and at least one prediction pattern can also be determined or obtained based on the list of most likely patterns corresponding to the current block. Then, at least one sub-block of the current block can be predicted based on the prediction pattern determined or obtained above.

[0453] In this embodiment, by at least partially reusing the MPM list of the current block, the prediction mode of at least one sub-block of the current block is determined or obtained. Since the sub-block is part of the current block, there is a strong correlation between the two. Therefore, based on the inherent correlation between the current block and its sub-block, a more suitable prediction mode can be selected for sub-block prediction without increasing the amount of computation, thereby improving the prediction accuracy.

[0454] Method 8: Based on the list of most likely patterns corresponding to the first sub-block of the current block, determine or obtain the predicted pattern corresponding to the second sub-block of the current block;

[0455] Optionally, based on the MPM list corresponding to the first sub-block of the current block, the MPM list corresponding to the second sub-block of the current block is determined or obtained, and based on the MPM list corresponding to the second sub-block, the prediction mode corresponding to the second sub-block is determined or obtained.

[0456] Optionally, based on the MPM list corresponding to the first sub-block of the current block, the MPM list corresponding to the second sub-block of the current block is determined or obtained, and based on the MPM list corresponding to the second sub-block and at least one reference template, the prediction mode corresponding to the second sub-block is determined or obtained.

[0457] Optionally, the reference template includes at least one of a reference block, a reference region, and / or a reference pixel.

[0458] For example, the selection of pixel samples for each sub-region of the current block can be referred to Figure 19 The solid line block is the current block and is divided into sub-blocks A, B, C and D. For sub-block A, the MPM list corresponding to sub-block A is determined or obtained based on the adjacent pixels (a1 to a5) of sub-block A.

[0459] like Figure 18 As shown, the MPM list for the current block uses the prediction modes corresponding to the sampling of adjacent pixels at the five positions of the current block (left (L), top (A), bottom left (BL), top right (AR), and top left (AL)) as prediction candidates in the MPM list for the current block. Since... Figure 18 The pixel sample at the top left (AL) position of the current block is the same as the pixel sample at position a1 of sub-block A. Therefore, the prediction mode corresponding to the pixel sample at position a1 of sub-block A can be directly determined when determining the MPM list of the current block. However, the prediction modes corresponding to the remaining pixel samples at positions a2 to a5 of sub-block A need to be determined or searched separately and added to the MPM list corresponding to sub-block A. Optionally, the prediction modes of the pixel samples at positions a2 to a5 of sub-block A are the prediction modes of the coding unit or prediction unit in which they are located. Based on the MPM list corresponding to sub-block A, the prediction mode corresponding to sub-block A is determined or obtained. Based on the prediction mode corresponding to sub-block A and the reference template corresponding to sub-block A, the prediction processing of sub-block A is performed.

[0460] Reference Figure 19 This application can encode or decode sub-blocks according to a preset order. When encoding a subsequent sub-block, the pixel samples of the preceding sub-block with the preset encoding order can be referenced. Therefore, for the subsequent sub-block B, the prediction mode of the adjacent pixel sample b2 can be determined or obtained using the prediction result of sub-block A. Since the prediction modes corresponding to the pixel samples at positions b3 and b4 of sub-block B can be directly determined when determining the MPM list of the current block, the computational load can be effectively reduced. For the pixel sample at position b1 of sub-block B, the prediction modes of these pixel samples need to be determined or searched separately and added to the MPM list corresponding to sub-block B. Optionally, the prediction mode of the pixel sample at position b1 of sub-block B is the prediction mode of the encoding unit or prediction unit in which it is located. Optionally, the prediction processing of sub-block B is performed according to the prediction mode corresponding to sub-block B and the reference template corresponding to sub-block B.

[0461] Reference Figure 19For sub-block C, the prediction modes corresponding to the pixel samples at positions c4 and c5 can be determined or obtained by using the prediction results of sub-blocks A and B, respectively. The prediction modes corresponding to the pixel samples at positions c2 and c3 of sub-block C can be determined directly when determining the MPM list of the current block. For the pixel samples at position c1 of sub-block C, it is necessary to determine or find the prediction modes of these pixel samples separately and add them to the MPM list corresponding to sub-block C. Optionally, the prediction mode of the pixel samples at position c1 of sub-block C is the prediction mode of the coding unit or prediction unit in which it is located.

[0462] Reference Figure 19 For sub-block D, the prediction modes corresponding to the pixel samples at positions d1, d9, d10, d4, and d6 of sub-block D can be directly determined when determining the MPM list of the current block. For the pixel samples at positions d2, d3, d7, and d8 of sub-block D, the prediction modes of these pixel samples need to be determined or searched separately and added to the MPM list corresponding to sub-block D. Optionally, the prediction modes of the pixel samples at positions d2, d3, d7, and d8 of sub-block B are the prediction modes of the coding unit or prediction unit where each pixel sample is located. Optionally, the prediction processing of sub-block D is performed according to the prediction mode corresponding to sub-block D and the reference template corresponding to sub-block D.

[0463] Optionally, in this embodiment, the reference region, reference block, and / or reference pixel are determined or obtained according to at least one of methods nine to fourteen in the fifth embodiment.

[0464] For example, the selection of pixel samples for determining the MPM list of each sub-block in the current block can be referred to Figure 20 The solid line block represents the current block and is divided into sub-blocks A, B, C, and D; for sub-blocks A to C, the selection of adjacent pixel samples is... Figure 19 Similarly, for sub-block D, the prediction modes of sub-blocks A to C can be used to determine or obtain the prediction modes of adjacent pixel samples of sub-block D, thereby obtaining the MPM list corresponding to sub-block D. Based on the MPM list corresponding to sub-block D, the prediction mode corresponding to sub-block D can be determined or obtained. Based on the prediction mode corresponding to sub-block D and the reference template corresponding to sub-block D, the prediction processing of sub-block D is performed.

[0465] For example, the selection of adjacent pixel samples for each sub-region of the current block can be referred to Figure 21 The solid-line block represents the current block and is divided into sub-blocks A, B, C, and D. For sub-block A, ... Figure 19 The difference in how the MPM list of neutron block A is constructed is that sub-block A does not utilize... Figure 19The prediction mode of the pixel sample at position a3 is determined because both positions a2 and a3 of sub-block A are adjacent to the left edge of the current block and are located at the midpoint. The probability that the image blocks at the two positions are the same image block is relatively high. To reduce redundant operations, the prediction mode of only one of the two positions is used to determine the MPM list of sub-block A.

[0466] For sub-block D, and Figure 19 The difference in how the MPM list of neutron block D is constructed is that sub-block D does not utilize... Figure 19 The reason for using the prediction modes of pixel samples at positions d3 and d8 in the image is that positions d2 and d3 of sub-block D are both adjacent to the left edge of the current block and are both located at the midpoint of the edge. The probability that the image blocks at these two positions are the same image block is relatively high. On the other hand, positions d7 and d8 of sub-block D are both adjacent to the top edge of the current block and are both located at the midpoint of the edge. To reduce redundant operations, the prediction mode of only one of the two positions is used to determine the MPM list of sub-block D.

[0467] Optionally, when performing prediction processing on at least one sub-block of the current block using OBIC, the surrounding pixel samples of at least one sub-block of the current block include adjacent pixel samples and non-adjacent pixel samples. Taking the first sub-block and the second sub-block as examples, the adjacent pixel samples of the first sub-block and the second sub-block can both be adjacent pixel samples of the current block, and the adjacent pixel samples of the second sub-block can also be pixel samples located inside the current block. The non-adjacent pixel samples of the first sub-block and the second sub-block can be non-adjacent pixel samples selected according to a preset rule with the current block as the center; or the non-adjacent pixel samples of the first sub-block and the second sub-block and / or non-adjacent pixel samples selected according to a preset rule with the first sub-block and the second sub-block as the center, respectively.

[0468] Optionally, based on the prediction modes corresponding to the surrounding pixel samples of at least one sub-block of the current block, the area amplitude histogram of each of the at least one sub-blocks can be determined. Optionally, the area amplitude histogram of each of the at least one sub-blocks can also be determined based on the prediction modes corresponding to the coding blocks (i.e., the adjacent coding blocks and / or non-adjacent coding blocks of the sub-blocks) where the surrounding pixel samples of at least one sub-block of the current block are located. Optionally, the area amplitude histogram of each of the at least one sub-blocks can also be determined based on the prediction modes corresponding to the sub-blocks of the coding blocks where the surrounding pixel samples of at least one sub-block of the current block are located. The prediction mode with the largest amplitude of each area amplitude histogram is used as the prediction mode of each sub-block of the current block for the prediction processing of the sub-block.

[0469] For example, the selection of surrounding pixel samples of the current block can be referred to Figure 22The solid line block represents the current block. The surrounding pixel samples of the current block include: adjacent pixel samples and non-adjacent pixel samples. The adjacent pixel samples of the current block include pixel samples located above, to the upper left, and to the left of the current block, while the non-adjacent pixel samples of the current block include at least one pixel sample selected above, to the upper left, to the upper right, to the left, and to the lower left, centered on the current block, according to a preset rule.

[0470] For example, when the current block includes at least one sub-block, the selection of surrounding pixel samples for each sub-block can be referenced. Figure 23 The solid-line block represents the current block and is divided into 4 sub-blocks, including sub-block A, such as... Figure 22 and 23 As shown, for sub-block A, since the pixel samples adjacent to the current block are also adjacent to sub-block A, at least some of the adjacent pixel samples of the current block can be used as adjacent pixel samples of sub-block A. Optionally, non-adjacent pixel samples of sub-block A can be selected with the current block as the center, and then at least some of the non-adjacent pixel samples of the current block can be used as non-adjacent pixel samples of sub-block A. Then, based on the prediction mode corresponding to the surrounding pixel samples of the current block, the area amplitude histogram of sub-block A is determined; or, based on the prediction mode corresponding to the surrounding pixel samples of sub-block A, the area amplitude histogram of sub-block A is determined, and then the prediction mode with the largest amplitude of each area amplitude histogram is used as the prediction mode of sub-block A for prediction processing of sub-block A.

[0471] For example, the method for selecting the sampling of surrounding pixels of at least one sub-block of the current block can refer to Figure 24 The solid line block represents the current block and is divided into 4 sub-blocks. For sub-block A, ... Figure 23 The difference in the selection method of pixels around sub-block A is that the non-adjacent pixels of sub-block A are no longer selected with the current block as the center. Instead, with sub-block A as the center, at least one pixel is selected from above, upper left, upper right, left and lower left according to a preset rule. Then, based on the prediction mode corresponding to the pixels around sub-block A, the area amplitude histogram of sub-block A is determined, and the prediction mode with the largest amplitude of each area amplitude histogram is used as the prediction mode of sub-block A for prediction processing of sub-block A.

[0472] For example, the method for selecting the sampling of surrounding pixels of at least one sub-block of the current block can refer to Figure 25 The solid-line block represents the current block and is divided into four sub-blocks, including sub-block B, such as... Figure 22 and 25As shown, for sub-block B, non-adjacent pixel samples of sub-block B can be selected with the current block as the center. At least a portion of the non-adjacent pixel samples of the current block can then be used as non-adjacent pixel samples of sub-block B. Pixel samples located above and to the upper left of sub-block B are selected as adjacent pixel samples of sub-block B. Since this application can encode or decode sub-blocks according to a preset order, when encoding a subsequent sub-block, the pixel samples of the preceding sub-block with the preset encoding order can be referenced. Therefore, for sub-block B in the next order, the prediction result of the preceding sub-block can be used to select pixel samples located inside the current block as adjacent pixel samples to the left of sub-block B. Furthermore, based on the prediction modes corresponding to the surrounding pixel samples of sub-block B, the area amplitude histogram of sub-block B is determined, and the prediction mode with the largest amplitude in each area amplitude histogram is used as the prediction mode for sub-block B, for prediction processing.

[0473] For example, the method for selecting the sampling of surrounding pixels of at least one sub-block of the current block can refer to Figure 26 The solid-line block represents the current block and is divided into 4 sub-blocks; for sub-block B, it is related to... Figure 25 The difference in the selection method of pixels around sub-block B is that the selection of non-adjacent pixels is no longer centered on the current block, but rather on sub-block B. At least one pixel sample is selected from above, upper left, upper right, left, and lower left according to a preset rule. Optionally, since the sub-block to the left of sub-block B has already been encoded or decoded, the non-adjacent pixel samples of sub-block B can include pixel samples located inside the current block. Then, based on the prediction mode corresponding to the pixel samples around sub-block B, the area amplitude histogram of sub-block B is determined, and the prediction mode with the largest amplitude of each area amplitude histogram is used as the prediction mode of sub-block B for prediction processing.

[0474] For example, the method for selecting the sampling of surrounding pixels of at least one sub-block of the current block can refer to Figure 27 The solid-line block represents the current block and is divided into four sub-blocks, including sub-block C, as shown below. Figure 22 and 27As shown, for sub-block C, non-adjacent pixel samples of sub-block C can be selected with the current block as the center. Then, at least a portion of the non-adjacent pixel samples of the current block can be used as non-adjacent pixel samples of sub-block C. Pixel samples located to the left and upper left of sub-block C are selected as adjacent pixel samples of sub-block C. Optionally, since this application can encode or decode sub-blocks according to a preset order, when encoding a subsequent sub-block, the pixel samples of the previous sub-block with the preset encoding order can be referenced. Therefore, for a subsequent sub-block C, the prediction result of the previous sub-block can be used to select pixel samples located inside the current block as adjacent pixel samples above sub-block C. Then, based on the prediction mode corresponding to the surrounding pixel samples of sub-block C, the area amplitude histogram of sub-block C is determined, and the prediction mode with the largest amplitude of each area amplitude histogram is used as the prediction mode of sub-block C for prediction processing of sub-block C.

[0475] For example, the method for selecting the sampling of surrounding pixels of at least one sub-block of the current block can refer to Figure 28 The solid line block represents the current block and is divided into 4 sub-blocks. For sub-block C, ... Figure 27 The difference in the selection method of pixels around sub-block C is that the selection of non-adjacent pixels is no longer centered on the current block, but rather on sub-block C. At least one pixel sample is selected from above, upper left, upper right, left, and lower left according to a preset rule. Optionally, since the sub-blocks above and to the upper right of sub-block C have already been encoded or decoded, the non-adjacent pixels of sub-block C can include pixels located inside the current block. Then, based on the prediction mode corresponding to the pixels around sub-block C, the area amplitude histogram of sub-block C is determined, and the prediction mode with the largest amplitude of each area amplitude histogram is used as the prediction mode of sub-block C for prediction processing.

[0476] For example, the method for selecting the sampling of surrounding pixels of at least one sub-block of the current block can refer to Figure 29 The solid-line block represents the current block and is divided into four sub-blocks, including sub-block D, as shown below. Figure 22 and 29As shown, for sub-block D, non-adjacent pixel samples of sub-block D can be selected with the current block as the center. Then, at least some non-adjacent pixel samples of the current block can be used as non-adjacent pixel samples of sub-block D. Since this application can encode or decode sub-blocks according to a preset order, when encoding the next sub-block, the pixel samples of the previous sub-block with the preset encoding order can be referenced. Therefore, for the next sub-block D, the prediction result of the previous sub-block can be used to select pixel samples located inside the current block as adjacent pixel samples above, to the upper left and to the left of sub-block D. Then, according to the prediction mode corresponding to the surrounding pixel samples of sub-block D, the area amplitude histogram of sub-block D is determined, and the prediction mode with the largest amplitude of each area amplitude histogram is used as the prediction mode of sub-block D for prediction processing of sub-block D.

[0477] For example, the method for selecting the sampling of surrounding pixels of at least one sub-block of the current block can refer to Figure 30 The solid-line block represents the current block and is divided into 4 sub-blocks. For sub-block D, the relationship with... Figure 29 The difference in the selection method of pixels around sub-block D is that the selection of non-adjacent pixels is no longer centered on the current block, but rather on sub-block D. At least one pixel sample is selected from above, upper left, upper right, left, and lower left according to a preset rule. Optionally, since the sub-blocks above, upper left, and left of sub-block D have already been encoded or decoded, the non-adjacent pixels of sub-block D can include pixels located inside the current block. Then, based on the prediction mode corresponding to the pixels around sub-block D, the area amplitude histogram of sub-block D is determined, and the prediction mode with the largest amplitude of each area amplitude histogram is used as the prediction mode of sub-block D for prediction processing of sub-block D.

[0478] In this embodiment, since both the first sub-block and the second sub-block are part of the current block and there is a strong correlation between them, a more suitable prediction mode can be selected to predict the sub-block based on the inherent correlation between the first sub-block and the second sub-block, thereby improving the prediction accuracy.

[0479] Fifth embodiment

[0480] Based on any of the above embodiments, a fifth embodiment is proposed.

[0481] In this embodiment, the method for determining or obtaining the reference region, reference block, and / or reference pixel includes at least one of methods nine to fourteen:

[0482] Method nine: The current block includes at least one of the following: region, block, and pixel;

[0483] Optionally, the reference region of at least one sub-block of the current block includes at least one of regions, blocks, and pixels located inside the current block, and / or at least one of regions, blocks, and pixels located outside the current block.

[0484] Optionally, the reference block of at least one sub-block of the current block includes at least one of the regions, blocks and pixels located inside the current block, and / or at least one of the regions, blocks and pixels located outside the current block.

[0485] Optionally, the reference pixel of at least one sub-block of the current block includes at least one of the regions, blocks and pixels located inside the current block, and / or at least one of the regions, blocks and pixels located outside the current block.

[0486] Optionally, the reference template includes at least one of a reference region, a reference block, and / or a reference pixel.

[0487] Optionally, at least a portion of the reference template for the second sub-block is determined or obtained based on the prediction results of the first sub-block.

[0488] Optionally, at least a portion of the reference template for the second sub-block is determined or obtained based on the reconstructed block of the first sub-block.

[0489] For example, when performing prediction processing on at least one sub-block of the current block using DIMD mode or NNIP mode, the selection of the reference template for at least one sub-block of the current block can refer to... Figure 31 The solid line represents the current block, which is divided into sub-blocks A, B, C, and D. The reference template for sub-block A is the external region adjacent to sub-block A. Optionally, since this application can encode or decode sub-blocks in a preset order, when encoding a subsequent sub-block, the sampling of the preceding sub-block in the preset encoding order can be referenced. Therefore, for sub-block B in the next order, the left reference template corresponding to sub-block B can be determined or obtained using the prediction result and / or reconstruction block of sub-block A, and then the upper reference template can be determined or obtained based on the external region adjacent to sub-block B. For sub-block C, the upper reference template corresponding to sub-block C can also be determined or obtained using the prediction result and / or reconstruction block of sub-block A, and the left reference template can be determined or obtained based on the external region adjacent to sub-block C. For sub-block D, the reference template corresponding to sub-block D can be determined or obtained using the prediction results and / or reconstruction blocks of sub-blocks A, B, and C, respectively.

[0490] Optionally, the boundary of the reference template can be the same as or smaller than the boundary of the sub-block.

[0491] Optionally, the size of the reference template of at least one sub-block of the current block can be smaller than the size of the reference template of the current block, thereby reducing the complexity of the algorithm during the prediction process.

[0492] For example, the width of the reference template of the current block is 4, while the width of the reference target of each sub-block of the current block is 2, and the sum of the sizes of all reference templates of at least one sub-block of the current block is less than the sum of the sizes of all reference templates of the current block. Thus, in the prediction process, the complexity of the algorithm can be reduced by decreasing the size of the reference template.

[0493] For example, when performing prediction processing on at least one sub-block of the current block using the TIMD pattern, the selection of the reference template for at least one sub-block of the current block can refer to... Figure 32 The solid line represents the current block, which is divided into sub-blocks A, B, C, and D. The reference template for sub-block A is the external region adjacent to sub-block A. Optionally, since this application can encode or decode sub-blocks according to a preset order, when encoding a subsequent sub-block, the sampling of the previous sub-block with the preset encoding order can be referenced. Therefore, for sub-block B in the next order, the prediction result and / or reconstruction block of sub-block A can be used to determine or obtain a partial reference template corresponding to sub-block B; then, based on the external region adjacent to sub-block B, the remaining reference template can be determined or obtained. For sub-block C, the prediction result and / or reconstruction block of sub-block A can also be used to determine or obtain a partial reference template corresponding to sub-block C, and based on the external region adjacent to sub-block C, the remaining reference template can be determined or obtained. For sub-block D, the prediction result and / or reconstruction block of sub-blocks A, B, and C can be used respectively to determine or obtain the reference template corresponding to sub-block D.

[0494] For example, when performing prediction processing on at least one sub-block of the current block using LIC mode, the selection of the reference template for at least one sub-block of the current block can refer to... Figure 33 The solid line represents the current block, which is divided into sub-blocks A, B, C, and D. The reference template for sub-block A is the external region adjacent to sub-block A. Optionally, since this application can encode or decode sub-blocks in a preset order, when encoding a subsequent sub-block, the sampling of the previous sub-block with the preset encoding order can be referenced. Therefore, for sub-block B in the next order, the prediction result and / or reconstruction block of sub-block A can be used to determine or obtain a portion of the reference template corresponding to sub-block B, and the remaining reference template can be determined or obtained based on the external region adjacent to sub-block B. For sub-block C, the prediction result and / or reconstruction block of sub-block A can also be used to determine or obtain a portion of the reference template corresponding to sub-block C, and the remaining reference template can be determined or obtained based on the external region adjacent to sub-block C. For sub-block D, the prediction result and / or reconstruction block of sub-blocks A, B, and C can be used respectively to determine or obtain the reference template corresponding to sub-block D.

[0495] Reference Figure 34 The reference template corresponding to sub-block D can be determined or obtained based on the region adjacent to the current block.

[0496] In this embodiment, since the distance between the previous sub-block and the current sub-block may be closer than that between the external adjacent regions, there may be a higher correlation between the previous sub-block and the current sub-block. Therefore, by determining or obtaining the reference template of the current sub-block based on the prediction result and / or reconstructed region of the previous sub-block, the prediction accuracy can be effectively improved based on the correlation between sub-blocks, thereby improving the encoding and decoding quality.

[0497] Method 10: The predicted block and / or reconstructed block of the first sub-block of the current block;

[0498] Optionally, based on the predicted block and / or reconstructed block of the first sub-block of the current block, the reference region, reference block and / or reference pixel corresponding to the second sub-block of the current block are determined or obtained according to at least one of the following.

[0499] Optionally, the first sub-block is processed before the second sub-block.

[0500] In this embodiment, since the distance between the previous sub-block and the current sub-block may be closer than that between the external adjacent regions, there may be a higher correlation between the previous sub-block and the current sub-block. Therefore, by determining or obtaining the reference template of the current sub-block based on the prediction result and / or reconstructed region of the previous sub-block, the prediction accuracy can be effectively improved based on the correlation between sub-blocks, thereby improving the encoding and decoding quality.

[0501] Method 11: At least one of the following: the upper adjacent pixel, the upper non-adjacent pixel, the left adjacent pixel, the left non-adjacent pixel, the upper left adjacent pixel, and the upper left non-adjacent pixel;

[0502] Optionally, a reference region of at least one sub-block of the current block is determined or obtained based on at least one of the above adjacent pixel, above non-adjacent pixel, left adjacent pixel, left non-adjacent pixel, upper left adjacent pixel, and upper left non-adjacent pixel. Optionally, at least one sub-block may include a first sub-block.

[0503] Optionally, a reference block of at least one sub-block of the current block is determined or obtained based on at least one of the above adjacent pixel, above non-adjacent pixel, left adjacent pixel, left non-adjacent pixel, upper left adjacent pixel, and upper left non-adjacent pixel. Optionally, at least one sub-block may include a first sub-block.

[0504] Optionally, a reference pixel of at least one sub-block of the current block is determined or obtained based on at least one of the above adjacent pixel, above non-adjacent pixel, left adjacent pixel, left non-adjacent pixel, upper left adjacent pixel, and upper left non-adjacent pixel. Optionally, at least one sub-block may include a first sub-block.

[0505] Optionally, the upper adjacent pixel can be a pixel located above and adjacent to the current block in the same frame of the image.

[0506] Alternatively, the non-adjacent pixels above can be pixels in the same frame that are above the current block but not adjacent to it.

[0507] Optionally, the left-adjacent pixel can be a pixel located to the left and adjacent to the current block in the same frame of the image.

[0508] Optionally, the non-adjacent pixels on the left can be pixels located to the left of the current block in the same frame of the image, but not adjacent to the current block.

[0509] Optionally, the upper left adjacent pixel can be a pixel located in the same frame image that is adjacent to the upper left of the current block.

[0510] Optionally, the non-adjacent pixel in the upper left corner can be a pixel in the same frame that is located in the upper left corner of the current block, but is not adjacent to the current block.

[0511] Optionally, at least one of the above adjacent pixel, above non-adjacent pixel, left adjacent pixel, left non-adjacent pixel, upper left adjacent pixel, and upper left non-adjacent pixel can be a reconstructed pixel or a predicted pixel.

[0512] Optionally, at least one of the following can be used as a reference region, reference block, and / or reference pixel: the upper adjacent pixel, the upper non-adjacent pixel, the left adjacent pixel, the left non-adjacent pixel, the upper left adjacent pixel, and the upper left non-adjacent pixel. Alternatively, the reference region, reference block, and / or reference pixel can be derived or calculated from at least one obtained pixel.

[0513] Optionally, reference regions, reference blocks, and / or reference pixels of at least one sub-block of the current block can be obtained according to preset mapping / correspondence rules.

[0514] Optionally, a reference region, reference block, and / or reference pixel can be selected from the above adjacent pixels, above non-adjacent pixels, left adjacent pixels, left non-adjacent pixels, upper left adjacent pixels, and upper left non-adjacent pixels of the current block, based on the prediction model.

[0515] Alternatively, if some locations lack valid pixel data, they can be filled using neighboring valid pixels.

[0516] For example, when performing prediction processing on at least one sub-block of the current block using DIMD mode or NNIP mode, the selection of the reference template for at least one sub-block of the current block can refer to... Figure 35 The solid line represents the current block, which is divided into sub-blocks A, B, C, and D. The reference templates for sub-blocks A through D are all outside the current block.

[0517] For example, when performing prediction processing on at least one sub-block of the current block using the TIMD pattern, the selection of the reference template for at least one sub-block of the current block can refer to... Figure 36 The solid line represents the current block, which is divided into sub-blocks A, B, C, and D. The reference templates for sub-blocks A through D are all outside the current block.

[0518] In this embodiment, since the current block usually has a high similarity to at least one of the above adjacent pixel, above non-adjacent pixel, left adjacent pixel, left non-adjacent pixel, upper left adjacent pixel, and upper left non-adjacent pixel, the prediction accuracy for at least one sub-block of the current block can be improved based on the reference region, reference block, and / or reference pixel determined by these adjacent pixels and / or non-adjacent pixels.

[0519] Method 12 requires at least one of the following: the neighboring block, the non-neighboring block, the sibling block, the temporal block, and the default block corresponding to the current block.

[0520] Optionally, a reference region of at least one sub-block of the current block is determined or obtained based on image block information of at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks and default blocks corresponding to the current block. Optionally, at least one sub-block may include a first sub-block.

[0521] Optionally, a reference block for at least one sub-block of the current block is determined or obtained based on image block information from at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, and default blocks corresponding to the current block. Optionally, at least one sub-block may include a first sub-block.

[0522] Optionally, the reference pixels of at least one sub-block of the current block are determined or obtained based on image block information of at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks and default blocks corresponding to the current block. Optionally, at least one sub-block may include a first sub-block.

[0523] Optionally, at least one of the following can be determined as the reference region of at least one sub-block of the current block: the neighboring block, the non-neighboring block, the co-located block, the temporal block, and the default block.

[0524] Optionally, at least one of the following: the neighboring block, non-neighboring block, co-located block, temporal block, and default block corresponding to the current block, is determined as the reference block for at least one sub-block of the current block.

[0525] Optionally, at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, and default blocks corresponding to the current block is determined as the reference pixel of at least one sub-block of the current block.

[0526] Optionally, the image block information may include at least one of the following: block size, block area, image block attributes, and image block type.

[0527] Optionally, the block size includes the block's width, height, aspect ratio, depth, area, resolution, and number of pixels.

[0528] Optionally, image block attributes may include the block's location and / or image texture.

[0529] Optionally, the image block type may include natural images or screen content images, etc.

[0530] Optionally, the reference region of at least one sub-block of the current block can be determined or obtained based on at least one of the following: the upper adjacent pixel, the upper non-adjacent pixel, the left adjacent pixel, the left non-adjacent pixel, the upper left adjacent pixel, and the upper left non-adjacent pixel, among the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, and default blocks corresponding to the current block.

[0531] Optionally, a reference block for at least one sub-block of the current block can be determined or obtained based on at least one of the following: the upper adjacent pixel, the upper non-adjacent pixel, the left adjacent pixel, the left non-adjacent pixel, the upper left adjacent pixel, and the upper left non-adjacent pixel, among the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, and default blocks corresponding to the current block.

[0532] Optionally, a reference pixel of at least one sub-block of the current block can be determined or obtained based on at least one of the following: the upper adjacent pixel, the upper non-adjacent pixel, the left adjacent pixel, the left non-adjacent pixel, the upper left adjacent pixel, and the upper left non-adjacent pixel, among the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, and default blocks corresponding to the current block.

[0533] Optionally, the reference region of at least one sub-block of the current block can be determined or obtained based on at least one of the width, height, block size, and block area of ​​at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, and default blocks corresponding to the current block.

[0534] Optionally, a reference block for at least one sub-block of the current block can be determined or obtained based on at least one of the width, height, block size, and block area of ​​at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, and default blocks corresponding to the current block.

[0535] Optionally, the reference pixel of at least one sub-block of the current block can be determined or obtained based on at least one of the width, height, block size, and block area of ​​at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, and default blocks corresponding to the current block.

[0536] Optionally, a neighboring block can be a block adjacent to the current block, and can be a block that has already been predicted or reconstructed.

[0537] Optionally, a non-neighbor block can be a block that is not adjacent to the current block, and can be a block that has already been predicted or reconstructed.

[0538] Optionally, a co-location block can be an image block in a co-location image that has the same position and size as the current block, and the co-location image can be the image in the reference image that is closest to the current image in time.

[0539] Optionally, a temporal block can be a block that is distinguished in the time domain, such as an image block in other frames before or after the current frame. For example, if there is video data containing a first frame image, a second frame image, and a third frame image played in the first second, second second, and third second, respectively, and the current block is a block divided from the second frame image, then the temporal block corresponding to the current block can be determined to be the corresponding image block in other frames other than the second frame.

[0540] Optionally, the default block can be a pre-set block, such as a block with typical pixel characteristics pre-set by the encoder and / or decoder.

[0541] In this embodiment, at least one reference region is determined or obtained based on at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, and default blocks corresponding to the current block. This ensures that the determined or obtained reference region, reference block, and / or reference pixel are closely related to at least one sub-block of the current block, thereby making the prediction results obtained subsequently based on the reference region, reference block, and / or reference pixel more accurate.

[0542] Method 13: At least one of the following: width, height, block size, and block area of ​​the current block;

[0543] Optionally, at least one reference region and / or at least one reference block can be determined or obtained based on the width, height, and tenth mapping table of the current block. Optionally, the tenth mapping table can be as shown in Table 10 below:

[0544] Table 10

[0545]

[0546]

[0547] Optionally, the height of at least one reference region and / or reference block is equal to the height of the current block by a first preset multiple, and the width of at least one reference region and / or reference block is equal to the width of the current block by a second preset multiple.

[0548] Optionally, the reference area or the image area within the reference area can be an encoded area or a decoded area.

[0549] Optionally, the encoded or decoded region can be determined by the position of the top-left pixel, the height and width of the encoded or decoded region. For example, the position of the top-left pixel can be the position of the image block at a height N (N is greater than 1) times above the top-left position of the current block and at a position N times the width of the image block to the left. The width of the encoded or decoded region is an integer multiple of the width of the current block, and the height of the encoded or decoded region is an integer multiple of the height of the current block.

[0550] Optionally, the reference region and / or reference block can be determined based on the block size of the current block and the eleventh mapping table. Optionally, the eleventh mapping table can be as shown in Table 11 below:

[0551] Table 11

[0552] <![CDATA[The block size of the current block is smaller than X4]]> <![CDATA[Dimension B1]]> <![CDATA[The block size of the current block is greater than X4 and less than X5]]> <![CDATA[Dimension B2]]> <![CDATA[The block size of the current block is greater than X5 and less than X6]]> <![CDATA[Dimension B3]]> <![CDATA[The block size of the current block is greater than X6 and less than X7]]> <![CDATA[Size B4]]>

[0553] Optionally, the block size includes at least one of the block's width, height, scale, depth, area, resolution, and number of pixels. Optionally, X4 to X7 can be a preset threshold corresponding to at least one of the block size's width, height, scale, depth, area, resolution, and number of pixels.

[0554] Optionally, a reference region and / or reference block can be determined based on the block area of ​​the current block and the twelfth mapping table. Optionally, the twelfth mapping table can be as shown in Table 12 below:

[0555] Table 12

[0556] <![CDATA[The block area of the current block is less than X8]]> <![CDATA[Area C1]]> <![CDATA[The block area of the current block is greater than X8 and less than X9]]> <![CDATA[Area C2]]> <![CDATA[The block area of the current block is greater than X9 and less than X 10 > <![CDATA[Area C3]]> <![CDATA[The block area of the current block is greater than X 10 and less than X 11 > <![CDATA[Area C4]]>

[0557] Optionally, the positions of the reference region, reference block, and / or reference pixel can be determined or obtained based on the upper adjacent pixel, left adjacent pixel, and upper left adjacent pixel of the current block, and then the size of the reference region, reference block, and / or reference pixel can be determined based on the width and height of the current block and the first mapping table.

[0558] For example, the reference area of ​​the current block includes: a first area adjacent to the top of the current block, a second area adjacent to the left of the current block, and a third area adjacent to the upper left of the current block; the width of the first area is equal to twice the width of the current block, and the height of the second area is equal to twice the height of the current block.

[0559] Optionally, if the width and height of the current block are both greater than or equal to 8, then the height of the first region is 8, the width of the second region is 8, and the width and height of the third region are both 8.

[0560] Optionally, if either the width or height of the current block is less than 8, then the height of the first region is 4, the width of the second region is 4, and the width and height of the third region are both 4.

[0561] In this embodiment, by determining or obtaining a reference region, reference block, and / or reference pixel based on at least one of the width, height, block size, and block area of ​​the current block, it is ensured that the reference region, reference block, and / or reference pixel are closely related to the current block, thereby making the subsequent prediction results obtained based on the reference region, reference block, and / or reference pixel more accurate.

[0562] Method 14: The candidate motion vector or candidate block vector of the current block is determined or the candidate block is obtained;

[0563] Optionally, a candidate block can be determined or obtained based on at least one candidate motion vector or at least one candidate block vector in the candidate list of the current block, and used as at least one reference region, at least one reference block and / or at least one reference pixel.

[0564] Optionally, a candidate block can be determined or obtained based on the candidate motion vector or candidate block vector of the current block; at least one reference region, at least one reference block and / or at least one reference pixel can be determined or obtained based on at least one of the above adjacent pixels, above non-adjacent pixels, left adjacent pixels, left non-adjacent pixels, upper left adjacent pixels and upper left non-adjacent pixels of the candidate block. For the specific implementation process, you can refer to the scheme in Method 11 above, that is, replace the current block in Method 11 with the candidate block, which will not be repeated here.

[0565] Optionally, a candidate block can be determined or obtained based on the candidate motion vector or candidate block vector of the current block; at least one of the above adjacent pixels, above non-adjacent pixels, left adjacent pixels, left non-adjacent pixels, upper left adjacent pixels, and upper left non-adjacent pixels of the candidate block can be used as at least one reference region, at least one reference block, and / or at least one reference pixel.

[0566] Optionally, a candidate block can be determined or obtained based on the candidate motion vector or candidate block vector of the current block; at least one reference region, at least one reference block and / or at least one reference pixel can be determined or obtained based on at least one of the width, height, block size and block area of ​​the candidate block. The specific implementation process can refer to the scheme in Method 13 above, that is, the current block in Method 13 can be replaced with the candidate block, which will not be repeated here.

[0567] Optionally, a candidate block can be determined or obtained based on the candidate motion vector or candidate block vector of the current block; at least one reference region, at least one reference block and / or at least one reference pixel can be determined or obtained based on at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks and default blocks corresponding to the candidate block. For the specific implementation process, refer to the scheme in Method Twelve above, that is, replace the current block in Method Twelve with the candidate block, which will not be repeated here.

[0568] Optionally, a candidate block can be determined or obtained based on the candidate motion vector or candidate block vector of the current block; and at least one of the following: the neighboring block, non-neighboring block, co-located block, temporal block, and default block corresponding to the candidate block can be used as at least one reference region, at least one reference block, and / or at least one reference pixel.

[0569] In this embodiment, the candidate block determined or obtained based on the candidate motion vector or candidate block vector of the current block ensures that the determined or obtained reference region is closely related to the current block, thereby making the subsequent prediction results obtained based on the reference region, reference block and / or reference pixel more accurate.

[0570] Sixth Embodiment

[0571] This application also provides a processing device, please refer to... Figure 37 , Figure 37 This is a functional block diagram of the processing device of this application, which can be installed in or is the processing equipment. The processing device includes:

[0572] Processing module A10 is used to perform prediction processing on at least one sub-block of the current block.

[0573] Optionally, the processing module A10 is also used for:

[0574] Based on at least one prediction pattern, perform prediction processing on at least one sub-block of the current block;

[0575] The prediction result of the current block is determined or obtained based on at least one reference region, at least one reference block and / or at least one reference pixel of at least one sub-block of the current block.

[0576] Optionally, the processing module A10 is also used for:

[0577] Based on the same prediction pattern, at least one sub-block of the current block is predicted.

[0578] Based on the prediction pattern corresponding to the first sub-block of the current block, perform prediction processing on the second sub-block of the current block.

[0579] Optionally, the method of determining or obtaining the prediction model includes at least one of the following:

[0580] The prediction mode is determined or obtained based on at least one reference region, at least one reference block and / or at least one reference pixel of at least one sub-block of the current block;

[0581] Based on the prediction patterns corresponding to the adjacent blocks, non-adjacent blocks, and / or the first sub-block of the current block, determine or obtain the prediction pattern corresponding to the second sub-block of the current block;

[0582] The predicted pattern is determined or obtained based on the list of most likely patterns and / or the list of non-most likely patterns corresponding to the current block;

[0583] Based on the list of most likely patterns corresponding to the first sub-block of the current block, determine or obtain the prediction pattern corresponding to the second sub-block of the current block.

[0584] Optionally, the prediction model includes at least one of the following:

[0585] Decoding-side intra-frame mode derivation mode;

[0586] Derivation of the mode based on the intra-frame coding mode of the event;

[0587] Template-based intra-frame coding mode derivation;

[0588] Local illumination compensation mode;

[0589] Intra-frame prediction mode based on neural networks.

[0590] Optionally, the first model is applied to a neural network-based intra-frame prediction mode, and / or the first model is determined or obtained according to at least one of the following:

[0591] The current block's width, height, block size, and block area must be at least one of these.

[0592] The width, height, size, and area of ​​at least one reference region of the current block;

[0593] The width, height, size, and area of ​​at least one of the reference blocks of the current block.

[0594] Optionally, the reference region, reference block, and / or reference pixel are determined or obtained based on at least one of the following:

[0595] The current block includes at least one of the following: region, block, and pixel;

[0596] The predicted block and / or reconstructed block of the first sub-block of the current block;

[0597] At least one of the following: the pixel above the current block, the non-adjacent pixel above the current block, the pixel to the left of the current block, the non-adjacent pixel to the left of the current block, the pixel above the left of the current block, and the non-adjacent pixel above the left of the current block;

[0598] The current block is at least one of the following: neighboring block, non-neighboring block, sibling block, temporal block, and default block;

[0599] The current block's width, height, block size, and block area must be at least one of these.

[0600] The candidate motion vector or candidate block vector of the current block is determined or the candidate block is obtained.

[0601] Optionally, the processing module A10 is also configured to: determine or obtain at least one sub-block of the current block;

[0602] At least one sub-block of the current block is determined or obtained if the width, height, block size and / or block area of ​​the current block satisfy the first condition.

[0603] The processing device provided in this application embodiment is similar in implementation principle and beneficial effect to the technical solution shown in the corresponding method embodiment above, and will not be described again here.

[0604] This application also provides a processing device, including a memory and a processor. The memory stores a processing program, and when the processing program is executed by the processor, it implements the steps of the processing method in any of the above embodiments.

[0605] This application also provides a storage medium storing a processing program, which, when executed by a processor, implements the steps of the processing method in any of the above embodiments.

[0606] In the embodiments of the processing device and storage medium provided in this application, all the technical features of any of the above-described processing method embodiments may be included. The extended and explanatory content of the specification is basically the same as that of the embodiments of the above methods, and will not be repeated here.

[0607] This application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to perform the methods described in the various possible implementations above.

[0608] This application also provides a chip, including a memory and a processor. The memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that a device with the chip installed performs the methods described in the various possible implementations above.

[0609] It is understood that the above scenarios are merely examples and do not constitute a limitation on the application scenarios of the technical solutions provided in the embodiments of this application. The technical solutions of this application can also be applied to other scenarios. For example, as those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0610] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0611] The steps in the method of this application embodiment can be adjusted, combined, or deleted according to actual needs.

[0612] The units in the device of this application embodiment can be merged, divided, and deleted according to actual needs.

[0613] In this application, the same or similar terms, concepts, technical solutions and / or application scenario descriptions are generally described in detail only when they appear for the first time. When they appear again, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of this application, the same or similar terms, concepts, technical solutions and / or application scenario descriptions that are not described in detail later can be referred to their previous relevant detailed descriptions.

[0614] In this application, the descriptions of the various embodiments have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0615] The technical features of the present application can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present application.

[0616] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, controlled terminal, or network device, etc.) to execute the methods of each embodiment of this application.

[0617] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, storage disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0618] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A processing method, characterized in that, Including the following steps: S10, perform prediction processing on at least one sub-block of the current block according to at least one prediction mode; The methods for determining or obtaining the prediction model include: Determine the area magnitude histogram of at least one sub-block based on the prediction mode sampled from the surrounding pixels of at least one sub-block of the current block. The prediction pattern with the largest amplitude in the area amplitude histogram of at least one sub-block is taken as the prediction pattern corresponding to at least one sub-block. The sampling of surrounding pixels of at least one sub-block includes: adjacent pixel sampling and non-adjacent pixel sampling; Non-adjacent pixel sampling includes at least one pixel sample selected above, above left, above right, to the left, and below left, centered on the sub-block, according to a preset rule. Adjacent pixel sampling includes pixel samples located inside the current block, selected based on the prediction result of the previous sub-block in the processing order.

2. The processing method as described in claim 1, characterized in that, Step S10 includes: The prediction result of the current block is determined or obtained based on at least one prediction mode and at least one reference region, at least one reference block and / or at least one reference pixel of at least one sub-block of the current block.

3. The processing method as described in claim 2, characterized in that, Based on at least one prediction pattern, perform prediction processing on at least one sub-block of the current block, including at least one of the following: Based on the same prediction pattern, at least one sub-block of the current block is predicted. Based on the prediction pattern corresponding to the first sub-block of the current block, perform prediction processing on the second sub-block of the current block.

4. The processing method as described in claim 2, characterized in that, The method of determining or obtaining the prediction model also includes at least one of the following: The prediction mode is determined or obtained based on at least one reference region, at least one reference block and / or at least one reference pixel of at least one sub-block of the current block; Based on the prediction patterns corresponding to the adjacent blocks, non-adjacent blocks, and / or the first sub-block of the current block, determine or obtain the prediction pattern corresponding to the second sub-block of the current block; The predicted pattern is determined or obtained based on the list of most likely patterns and / or the list of non-most likely patterns corresponding to the current block; Based on the list of most likely patterns corresponding to the first sub-block of the current block, determine or obtain the prediction pattern corresponding to the second sub-block of the current block.

5. The processing method according to any one of claims 2 to 4, characterized in that, The reference region, reference block, and / or reference pixel are determined or obtained based on at least one of the following: The current block includes at least one of the following: region, block, and pixel; The predicted block and / or reconstructed block of the first sub-block of the current block; At least one of the following: the pixel above the current block, the non-adjacent pixel above the current block, the pixel to the left of the current block, the non-adjacent pixel to the left of the current block, the pixel above the left of the current block, and the non-adjacent pixel above the left of the current block; The current block is at least one of the following: neighboring block, non-neighboring block, sibling block, temporal block, and default block; The current block's width, height, block size, and block area must be at least one of these. The candidate motion vector or candidate block vector of the current block is determined or the candidate block is obtained.

6. The processing method as described in claim 1 or 2, characterized in that, It also includes at least one of the following: Determine or obtain at least one sub-block of the current block; At least one sub-block of the current block is determined or obtained if the width, height, block size and / or block area of ​​the current block satisfy the first condition.

7. A processing device, characterized in that, include: A memory and a processor, wherein the memory stores a processing program, and the processing program, when executed by the processor, implements the steps of the processing method as described in any one of claims 1 to 6.

8. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the processing method as described in any one of claims 1 to 6.

Images