Image processing method, processing device, and storage medium

Abstract

Provided in the present application are an image processing method, a processing device, and a storage medium. The image processing method comprises: determining or obtaining a first prediction result on the basis of at least one image region partitioned from a current block. The technical solution of the present application can improve the prediction accuracy of intra prediction, thereby improving the prediction effect of video encoding and / or decoding.

Description

Image processing methods, processing devices and storage media Technical Field

[0001] This application relates to the field of image processing technology, specifically to an image processing method, processing device, and storage medium. Background Technology

[0002] The video coding techniques proposed by existing video coding standards aim to improve coding performance without significantly increasing computational complexity. Specifically, these techniques include dividing each frame into different blocks during video encoding and decoding, followed by prediction, transformation, and quantization processing, as well as entropy coding or entropy decoding.

[0003] In the process of conceiving and implementing this application, the inventors discovered at least the following problems:

[0004] When performing image patch prediction in the image patch prediction stage, the prediction mode can be determined by derivation at the decoding end. However, the accuracy of estimating the probability of each prediction mode by deriving the prediction mode at the decoding end is not high, resulting in poor prediction performance of the prediction mode determined in this way.

[0005] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the Invention

[0006] To address the aforementioned technical problems, this application provides an image processing method, processing device, and storage medium, aiming to solve the technical problem of how to improve the prediction accuracy of intra-frame prediction, thereby supporting the improvement of prediction performance in video encoding and / or decoding.

[0007] This application provides an image processing method, applicable to a processing device, comprising the following steps:

[0008] S10, determine or obtain a first prediction result based on at least one image region divided in the current block.

[0009] Optionally, the current block may be divided based on at least one of the following:

[0010] The first reference template for the current block;

[0011] At least one dividing line;

[0012] The partition parameters for the current block;

[0013] First neural network and / or first lookup table;

[0014] If the syntax elements obtained from the bitstream satisfy the first condition, then the division method is determined or obtained according to the first division strategy.

[0015] Optionally, at least one dividing line is determined or obtained based on at least one of the following:

[0016] The gradient magnitude and / or gradient direction of at least one element within the first reference template;

[0017] The position of at least one element with the maximum gradient magnitude within the first reference template and the direction perpendicular to the direction of the maximum gradient;

[0018] The first gradient histogram and / or statistical feature information corresponding to the first reference template;

[0019] A first reference template, and at least one fitting coefficient and / or partition weight coefficient;

[0020] At least one first weight coefficient matrix and / or a first fitting coefficient matrix corresponding to the first reference template;

[0021] The current block is at least one of the following: neighboring block, non-neighboring block, co-occurring block, temporal block, default block, cross-component block, and candidate block.

[0022] Optionally, the partitioning parameters are determined or obtained based on at least one of the following:

[0023] Template feature information of the first reference template;

[0024] Gradient information of the first reference template;

[0025] Residual fluctuations in the first reference template;

[0026] The structure tensor of the first reference template;

[0027] Phase consistency of the first reference template;

[0028] Second derivative information of the first reference template;

[0029] Structural characteristic parameters of the first reference template;

[0030] Edge structure description information of the first reference template;

[0031] The variance and / or mean of at least one element in the first reference template;

[0032] Texture parameters of the first reference template;

[0033] Partition information of at least one of the following: neighboring blocks, non-neighboring blocks, co-occurring blocks, temporal blocks, default blocks, cross-component blocks, and candidate blocks of the current block;

[0034] Second weighting coefficient matrix and / or second fitting coefficient matrix;

[0035] Fit coefficients;

[0036] Partition weighting coefficients;

[0037] Statistical information for the first reference template;

[0038] A second neural network and / or a second lookup table.

[0039] Optionally, the first prediction result is determined or obtained based on at least one of the following for at least one image region:

[0040] At least one first prediction pattern;

[0041] At least one candidate prediction pattern;

[0042] Second prediction result.

[0043] Optionally, a first prediction model and / or candidate prediction models are determined or obtained based on at least one of the following:

[0044] The second gradient histogram corresponding to the second reference template of at least one image region;

[0045] At least one second reference template corresponding to statistical feature information;

[0046] At least one image region and / or at least one of the following: neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, default blocks, cross-component blocks, and candidate blocks of the current block;

[0047] Texture parameters of at least one second reference template;

[0048] At least one image region and / or a second reference template, including size and / or position parameters;

[0049] At least one of the following: template feature information, gradient information, residual fluctuation, structural feature parameters, edge structure description information, structural tensor, phase consistency, and statistical information of at least one second reference template;

[0050] The variance and / or mean of at least one element in at least one second reference template;

[0051] At least one of the following: the fitting coefficients, partition weight coefficients, third weight coefficient matrix, and third fitting coefficient matrix corresponding to at least one second reference template;

[0052] At least one third neural network and / or a third lookup table;

[0053] If the syntax elements obtained in the bitstream satisfy the second condition, then the prediction pattern is determined or obtained according to the first pattern determination strategy.

[0054] If the syntax elements obtained from the bitstream do not meet the second condition, then the prediction pattern is determined or obtained according to the second pattern determination strategy.

[0055] Optionally, the first reference template and / or the second reference template are determined or obtained according to at least one of the following:

[0056] The reference template above the current block;

[0057] The left-hand reference template for the current block;

[0058] The upper left reference template of the current block;

[0059] The image block is determined or obtained by the motion vector of the current block;

[0060] Image blocks determined or obtained through the block vector of the current block;

[0061] Current block size parameters;

[0062] Template instruction information;

[0063] Matching information;

[0064] Rate distortion costs.

[0065] Optionally, the image processing method further includes at least one of the following:

[0066] The second prediction result is determined or obtained based on a second prediction mode that is a weighted sum of at least one first prediction mode and / or candidate prediction modes, and at least one image region.

[0067] The first prediction result is determined or obtained based on at least one of the following: at least one second prediction result, at least one first weighted and non-angle prediction result;

[0068] The first prediction result is determined or obtained based on at least one of the fitting coefficient, partition weight coefficient, function formula, fourth neural network and fourth lookup table, and at least one second prediction result.

[0069] Optionally, the image processing method further includes at least one of the following:

[0070] The first weight is determined or obtained based on the gradient information of at least one image region;

[0071] The first weight is determined or obtained based on the fusion boundary and / or bending boundary in different directions;

[0072] In at least one image region, the smaller the distance between the elements in a certain region and the fusion boundary, the larger the first weight corresponding to the elements in that region; the smaller the distance between the elements in another region and the fusion boundary, the smaller the first weight corresponding to the elements in that other region.

[0073] In at least one image region, the smaller the distance between the elements in a certain region and the curved boundary, the larger the first weight corresponding to the elements in that region; the smaller the distance between the elements in another region and the curved boundary, the smaller the first weight corresponding to the elements in that other region.

[0074] At least two image regions have different first weights corresponding to the same element position;

[0075] At least two image regions, wherein the first weight corresponding to the elements of a portion of at least one image region is greater than the first weight corresponding to the elements of a portion of at least another image region;

[0076] In at least one image region, the first weight corresponding to elements in a portion of the region is greater than the first weight corresponding to elements in another portion of the region.

[0077] In at least one image region, the smaller the distance between the elements of a certain region and the dividing boundary, the larger the first weight corresponding to the elements of that certain region; and the smaller the distance between the elements of another region and the dividing boundary, the smaller the first weight corresponding to the elements of that other region.

[0078] This application also provides an image processing apparatus, comprising:

[0079] The processing module is used to determine or obtain a first prediction result based on at least one image region divided in the current block.

[0080] This application also provides a processing device, including: a memory and a processor, wherein the memory stores an image processing program, and when the image processing program is executed by the processor, it implements the steps of any of the image processing methods described above.

[0081] This application also provides a storage medium storing a computer program that, when executed by a processor, implements the steps of any of the image processing methods described above.

[0082] As described above, the image processing method of this application can be applied to a processing device, including: determining or obtaining a first prediction result based on at least one image region divided in the current block. Through the technical solution of this application, during the prediction stage of video encoding and / or decoding, such as intra-frame prediction, at least one image region divided in the current block can be considered to determine or obtain a first prediction result. This takes into account different image regions in the current block, making the determined or obtained prediction mode more accurate, improving the prediction accuracy for the current block, and thus supporting improved prediction efficiency in the video encoding and / or decoding process. Attached Figure Description

[0083] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0084] Figure 1 is a schematic diagram of the hardware structure of a smart terminal that implements various embodiments of this application;

[0085] Figure 2 is a communication network system architecture diagram provided in an embodiment of this application;

[0086] Figure 3 is a schematic diagram of the hardware structure of a controller 140 provided in this application;

[0087] Figure 4 is a schematic diagram of the hardware structure of a network node 150 provided in this application;

[0088] Figure 5 is a schematic diagram of an image patch processed using the DI MD mode for prediction.

[0089] Figure 6 is a schematic diagram of the encoding process of the encoder based on the image processing method of this application;

[0090] Figure 7 is a schematic diagram of the decoding process of the decoder based on the image processing method of this application;

[0091] Figure 8 is a flowchart illustrating the image processing method according to the first embodiment;

[0092] Figure 9 is a schematic diagram of the intra-frame prediction process of the image processing method according to the first embodiment;

[0093] Figure 10 is a scene diagram showing the gradient direction in an image block according to the second embodiment;

[0094] Figure 11 is a schematic diagram of a scene where the dividing lines in the image block are used to divide the image according to the second embodiment;

[0095] Figure 12 is a schematic diagram of the partitioned DIMD process of the image processing method according to the third embodiment;

[0096] Figure 13 is a schematic diagram of the prediction mode determination process of the image processing method according to the third embodiment;

[0097] Figure 14 is a schematic diagram of a process for determining the prediction result of the image processing method according to the third embodiment;

[0098] Figure 15 is a schematic diagram of another process for determining the prediction result of the image processing method according to the third embodiment;

[0099] Figure 16 is a schematic diagram of the prediction result fusion of the image processing method according to the third embodiment;

[0100] Figure 17 is a schematic diagram of the current block and reference template of the image processing method according to the fifth embodiment;

[0101] Figure 18 is a schematic diagram of the boundary in an image according to the image processing method shown in the sixth embodiment;

[0102] Figure 19 is a schematic diagram of image region fusion in an image processing method according to the sixth embodiment;

[0103] Figure 20 is a schematic diagram of the prediction module in the image processing device.

[0104] The realization of the objectives, functional features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0105] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0106] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0107] It should be understood that although the terms first, second, third, etc., may be used herein to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word “if” as used herein may be interpreted as “when…” or “in response to determination”. Furthermore, as used herein, the singular forms “a,” “an,” and “the” are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms “comprising,” “including,” indicate the presence of the stated feature, step, operation, element, component, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms “or,” “and / or,” “including at least one of the following,” etc., as used in this application may be interpreted as inclusive, or mean any one or any combination thereof. For example, "including at least one of the following: A, B, C" means "any one of the following: A; B; C; A and B; A and C; B and C; A and B and C." Similarly, "A, B, or C" or "A, B, and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A and B and C." Exceptions to this definition only occur when the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0108] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0109] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0110] It should be noted that in this article, codes such as step S10, method one, and method two are used to describe the corresponding content more clearly and concisely, and do not constitute a substantial restriction on the order.

[0111] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0112] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0113] In this application, the processing device can be a smart terminal, such as a mobile phone or computer, or a server, such as a local server or cloud server. This application primarily uses the processing device as an example of a smart terminal. Optionally, the smart terminal can be implemented in various forms. For example, the smart terminals described in this application can include smart terminals such as mobile phones, tablets, laptops, PDAs, personal digital assistants (PDAs), portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, etc., as well as fixed terminals such as digital TVs and desktop computers.

[0114] The following description will use a mobile terminal as an example. Those skilled in the art will understand that, apart from elements specifically designed for mobile purposes, the construction according to the embodiments of this application can also be applied to fixed-type terminals.

[0115] Please refer to Figure 1, which is a schematic diagram of the hardware structure of a mobile terminal implementing various embodiments of this application. The mobile terminal 100 may include: an RF (Radio Frequency) unit 101, a WiFi module 102, an audio output unit 103, an A / V (Audio / Video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111, etc. Those skilled in the art will understand that the mobile terminal structure shown in Figure 1 does not constitute a limitation on the mobile terminal. The mobile terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0116] The following section, with reference to Figure 1, provides a detailed description of each component of the mobile terminal:

[0117] The radio frequency unit 101 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 110; additionally, it transmits uplink data to the base station. Typically, the radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, and a duplexer. Furthermore, the radio frequency unit 101 can also communicate wirelessly with networks and other devices. The aforementioned wireless communications may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution), 5G, and 6G.

[0118] WiFi is a short-range wireless transmission technology. Mobile terminals using WiFi module 102 can help users send and receive emails, browse web pages, and access streaming media, providing wireless broadband internet access. Although Figure 1 shows WiFi module 102, it is understood that it is not an essential component of the mobile terminal and can be omitted as needed without altering the essence of the invention.

[0119] The audio output unit 103 can convert audio data received by the radio frequency unit 101 or the WiFi module 102 or stored in the memory 109 into audio signals and output them as sound when the mobile terminal 100 is in call signal receiving mode, call mode, recording mode, voice recognition mode, broadcast receiving mode, etc. Furthermore, the audio output unit 103 can also provide audio output related to specific functions performed by the mobile terminal 100 (e.g., call signal receiving sound, message receiving sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.

[0120] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos acquired by an image capture device (such as a camera) in video capture mode or image capture mode. The processed image frames can be displayed on the display unit 106. The image frames processed by the GPU 1041 can be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) in operating modes such as telephone call mode, recording mode, and voice recognition mode, and can process such sound into audio data. The processed audio (voice) data can be converted into a format that can be transmitted to a mobile communication base station via the radio frequency unit 101 in telephone call mode. The microphone 1042 can implement various types of noise cancellation (or suppression) algorithms to eliminate (or suppress) noise or interference generated during the reception and transmission of audio signals.

[0121] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Optionally, the light sensor includes an ambient light sensor and a proximity sensor. Optionally, the ambient light sensor can adjust the brightness of the display panel 1061 according to the ambient light level, and the proximity sensor can turn off the display panel 1061 and / or backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc. Other sensors that may be configured in the phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0122] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0123] User input unit 107 can be used to receive input numerical or character information, and generate key signal inputs related to user settings and function control of the mobile terminal. Optionally, user input unit 107 may include touch panel 1071 and other input devices 1072. Touch panel 1071, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near touch panel 1071), and drive corresponding connection devices according to a pre-set program. Touch panel 1071 may include a touch detection device and a touch controller. Optionally, the touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to processor 110, and can receive and execute commands sent by processor 110. In addition, touch panel 1071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may also include other input devices 1072. Optionally, other input devices 1072 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc., without being specifically limited here.

[0124] Optionally, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. Subsequently, the processor 110 provides corresponding visual output on the display panel 1061 according to the type of touch event. Although in FIG. 1, the touch panel 1071 and the display panel 1061 are implemented as two independent components to realize the input and output functions of the mobile terminal, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to realize the input and output functions of the mobile terminal. The specific implementation is not limited here.

[0125] Interface unit 108 serves as an interface through which at least one external device can connect to mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, and so on. Interface unit 108 may be used to receive input (e.g., data, power, etc.) from the external device and transmit the received input to one or more elements within mobile terminal 100, or it may be used to transmit data between mobile terminal 100 and the external device.

[0126] The memory 109 can be used to store software programs and various data. The memory 109 may primarily include a program storage area and a data storage area. Optionally, the program storage area may store the operating system, applications required for at least one function (such as sound playback, image playback, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory 109 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0127] The processor 110 is the control center of the mobile terminal. It connects various parts of the mobile terminal via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 109, and by calling data stored in the memory 109, it performs various functions and processes data of the mobile terminal, thereby providing overall monitoring of the mobile terminal. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor. Optionally, the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 110.

[0128] The mobile terminal 100 may also include a power supply 111 (such as a battery) that supplies power to various components. Preferably, the power supply 111 can be logically connected to the processor 110 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0129] Although not shown in Figure 1, the mobile terminal 100 may also include a Bluetooth module, etc., which will not be described in detail here.

[0130] To facilitate understanding of the embodiments of this application, the communication network system on which the mobile terminal of this application is based is described below.

[0131] Please refer to Figure 2, which is a communication network system architecture diagram provided in an embodiment of this application. The communication network system is an LTE system based on the universal mobile communication technology. The LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203, and the operator's IP services 204, which are connected in sequence.

[0132] Optionally, UE201 can be the aforementioned mobile terminal 100, which will not be described in detail here.

[0133] E-UTRAN202 includes eNodeB2021 and other eNodeB2022, etc. Optionally, eNodeB2021 can connect to other eNodeB2022 via backhaul (e.g., X2 interface), and eNodeB2021 connects to EPC203, providing access from UE201 to EPC203.

[0134] EPC203 may include MME (Mobility Management Entity) 2031, HSS (Home Subscriber Server) 2032, other MMEs 2033, SGW (Serving Gateway) 2034, PGW (Packet Data Network Gateway) 2035, and PCRF (Policy and Charging Rules Function) 2036, etc. Optionally, MME2031 is the control node that handles signaling between UE201 and EPC203, providing bearer and connection management. HSS2032 is used to provide registers to manage functions such as the Home Location Register (not shown in the figure) and stores user-specific information such as service characteristics and data rates. All user data can be sent through SGW2034. PGW2035 can provide UE 201 IP address allocation and other functions. PCRF2036 is the policy and charging control decision point for service data flow and IP bearer resources. It selects and provides available policy and charging control decisions for the policy and charging enforcement function unit (not shown in the figure).

[0135] IP services 204 may include the Internet, intranet, IMS (IP Multimedia Subsystem), or other IP services.

[0136] Although the above description uses the LTE system as an example, those skilled in the art should know that this application is not only applicable to the LTE system, but also to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, 5G and future new network systems (such as 6G), etc., without limitation.

[0137] Figure 3 is a schematic diagram of the hardware structure of a controller 140 provided in this application. The controller 140 includes a memory 1401 and a processor 1402. The memory 1401 is used to store program instructions, and the processor 1402 is used to call the program instructions in the memory 1401 to execute the steps performed by the controller in the first embodiment of the above method. The implementation principle and beneficial effects are similar, and will not be described again here.

[0138] Optionally, the controller further includes a communication interface 1403, which can be connected to the processor 1402 via a bus 1404. The processor 1402 can control the communication interface 1403 to implement the receiving and sending functions of the controller 140.

[0139] Figure 4 is a schematic diagram of the hardware structure of a network node 150 provided in this application. The network node 150 includes a memory 1501 and a processor 1502. The memory 1501 is used to store program instructions, and the processor 1502 is used to call the program instructions in the memory 1501 to execute the steps performed by the first node in the above method embodiment. The implementation principle and beneficial effects are similar, and will not be described again here.

[0140] Optionally, the controller further includes a communication interface 1503, which can be connected to the processor 1502 via a bus 1504. The processor 1502 can control the communication interface 1503 to implement the receiving and sending functions of the network node 150.

[0141] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0142] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk, SSD), etc.

[0143] Based on the above-described mobile terminal hardware structure and communication network system, various embodiments of this application are proposed.

[0144] Optionally, in DIMD (Decoder-side intra-mode derivation) technology, the best prediction mode is selected by calculating and statistically analyzing the gradient histogram of surrounding pixels. However, when processing image blocks with obvious boundaries, the best prediction mode obtained from the gradient histogram is affected by the boundary gradient and predicts according to the direction of the boundary. But it is possible that using different prediction modes for two image regions on the left and right of the boundary will result in better prediction. That is, the original technology cannot select the best and correct prediction mode under certain circumstances.

[0145] Alternatively, this deficiency can be addressed by mixing multiple prediction modes or assigning different weights to the predicted pixels. However, this lacks analysis of image texture features. Therefore, for image blocks with prominent texture boundaries, partitioning can be used and weighted encoding and decoding can be performed using two or more prediction modes.

[0146] Optionally, if DIMD technology is used for image blocks that are not partitioned, different prediction regions within the image block may not be predicted well at the same time, and the boundaries within the image block may also affect the selection of prediction modes, resulting in poor prediction performance for the image block and hindering the improvement of encoding and decoding performance.

[0147] Optionally, the DIMD algorithm can be optimized to address this deficiency, resulting in a DIMD-based partition prediction algorithm. This algorithm can achieve better encoding and decoding performance when compressing image blocks with obvious texture features, thereby improving the prediction effect of image blocks and enhancing the performance of encoding and decoding (i.e., encoding and / or decoding).

[0148] Optionally, in this embodiment, the first prediction result can be determined or obtained based on at least one image region divided into the current block. In this way, the current block can be partitioned during the prediction process for the video image, thus taking into account different image regions within the current block. This makes the determined or obtained prediction result more accurate, improves the prediction accuracy for the current block, and ultimately supports improved prediction efficiency in the video encoding and / or decoding process.

[0149] For ease of understanding, the DIMD and encoding and / or decoding processes that may be involved in the embodiments of this application will be explained below.

[0150] (I) Basic Working Principle of DIMD

[0151] If the processing device is the encoding end, when performing intra-frame prediction, when DIMD mode is enabled, the processing device will use the Sobel operator (an edge detection operator) to calculate the gradient of the reconstructed pixels around the coding unit. The area to be calculated is the template, such as the first reference template. A gradient can be calculated for every 3x3 image area. The moving gradient operator can calculate a series of gradients, and the calculated gradients are statistically summarized into a gradient histogram.

[0152] Optionally, the horizontal axis of the gradient histogram represents different prediction modes, and the vertical axis represents the sum of the gradient magnitudes corresponding to that gradient direction.

[0153] For example, as shown in Figure 5, DIMD templates adjacent to the block to be predicted (i.e., the current block) are determined above and to the left of the block to be predicted. These might be three lines composed of various image blocks on the left and above the block to be predicted. Pixels in the middle of these three lines (e.g., pixel x 4) are used to calculate the gradient. By calculating the gradient direction of at least one pixel in the pixel line, as well as the magnitudes of the horizontal and vertical gradients, the gradient direction and corresponding gradient magnitude of at least one pixel can be obtained. The gradient magnitude is the sum of the absolute values ​​of the horizontal and vertical gradient magnitudes. Adding the gradient magnitude values ​​of the same prediction mode corresponding to the gradient direction in at least one pixel yields the sum of the gradient magnitude values ​​corresponding to that prediction mode. A histogram of gradient magnitude values ​​for different prediction modes of at least one pixel can be constructed, and the prediction mode with the largest sum of gradient magnitude values, or at least one candidate prediction mode, can be selected as the intra-frame prediction mode for the current block.

[0154] Optionally, the horizontal gradient Gx and vertical gradient Gy can be calculated using the 3x3 horizontal Sober operator and the vertical Sober operator. For example, the horizontal gradient Gx and vertical gradient Gy for a pixel x 4 in a pixel line can be calculated according to the following formulas (I) and (II):

[0155] Optionally, A can be a matrix consisting of 9 pixels centered at pixel x4, including the pixel x1 above it, the pixel x3 to its left, the pixel x7 below it, the pixel x5 to its right, the pixel x0 at its top left, the pixel x6 at its bottom left, the pixel x2 at its top right, and the pixel x8 at its bottom right, as shown in Formula (III) below:

[0156] Optionally, the magnitude of gradient G is the sum of the absolute values ​​of the horizontal and vertical gradients, and its calculation formula is shown in formula (iv):

[0157] Formula (IV) is G = |Gx| + |Gy|.

[0158] Alternatively, the gradient direction corresponding to a pixel can be calculated using arctan(Gx / Gy).

[0159] Optionally, since each gradient direction corresponds to a specific gradient direction range, and each gradient direction range corresponds to a prediction direction of an intra-frame prediction mode, for at least one pixel in the pixel line, the gradient magnitude values ​​with the same gradient direction range in at least one pixel can be added together to obtain the sum of the gradient magnitude values ​​corresponding to that gradient direction range.

[0160] Optionally, the prediction mode with the highest energy can be selected from the gradient histogram and added to the intra-frame prediction mode candidate set, such as the candidate prediction mode list.

[0161] Optionally, the processing device can select the best prediction mode from the intra-prediction mode candidate set based on the rate-distortion cost, for example, selecting the candidate prediction mode with the lowest rate-distortion cost as the best prediction mode, and then performing prediction processing.

[0162] Optionally, after selecting the DIMD mode, the processing device needs to mark the use of the DIMD mode in the bit stream, but the prediction mode specifically derived from the DIMD mode does not need to be transmitted in the bit stream.

[0163] Optionally, if the processing device is a decoding end, and if the processing device determines that DIMD mode is enabled during decoding in the bitstream, the processing device can use the same reconstructed pixels to calculate the gradient again to ensure consistency with the calculation method of the encoding end. The processing device can select the prediction mode with the highest energy in the gradient histogram and add it to the intra-prediction mode candidate set, such as the candidate prediction mode list. The processing device can then select the best prediction mode in the intra-prediction mode candidate set, perform prediction processing, determine or generate prediction blocks, add the prediction blocks to the residuals to obtain the reconstructed blocks, and thus complete the video decoding.

[0164] (II) Encoding process of the image processing method in this embodiment

[0165] Optionally, when the processing device is an encoder (i.e., the encoding end), referring to FIG6, the encoder receives video data input from a video source, such as receiving video images from a video source, determining the image to be predicted in the video images, dividing the image to be predicted into at least one image block (optionally including luma blocks and chroma blocks), and performing prediction processing on each of the at least one image block using the temporal and / or spatial correlation between video images. The prediction processing includes intra-frame prediction processing and / or inter-frame prediction processing, and the intra-frame prediction processing and / or inter-frame prediction processing includes at least one prediction mode. For these prediction modes, the encoder uses, for example, rate-distortion optimization to determine the prediction mode finally adopted for each of the at least one image block. For example, it calculates the rate-distortion cost corresponding to each prediction mode or the rate-distortion cost of combining several prediction modes to determine the minimum rate-distortion cost from at least one rate-distortion cost. The prediction mode or combination of prediction modes corresponding to the minimum rate-distortion cost is the prediction mode finally adopted for the image block.

[0166] Optionally, these prediction modes include intra-frame prediction modes and inter-frame prediction modes: the intra-frame prediction modes can be derived or determined using the methods of this embodiment, such as by using the improved DIMD mode; the inter-frame prediction modes can also be determined using the methods of this embodiment.

[0167] Alternatively, the basic principle of the improved DIMD mode can be to determine the partition (i.e. the divided image region) by gradient, apply the corresponding best prediction mode to different partitions, and obtain the pixel values ​​of all pixels to be predicted in the image block by fusing the best prediction results of at least two image regions.

[0168] Optionally, the pixel value of the pixel sample in the element image block corresponding to the image block to be predicted is subtracted from the predicted value of the corresponding pixel sample in the prediction block to obtain the residual value of the pixel sample and the residual block corresponding to the original image block. The residual block is transformed and quantized, and then encoded by the entropy encoder to form an encoded bit stream.

[0169] Alternatively, the improved DIMD mode can be embodied by performing any of the following embodiments.

[0170] Optionally, the encoded bitstream may also include prediction parameters corresponding to a given prediction mode and related side information.

[0171] Optionally, the prediction parameters are entropy-encoded and then packed into the encoded bitstream.

[0172] Optionally, the relevant auxiliary information may include the indication information of the intra-frame mode derivation at the decoding end in this embodiment.

[0173] Optionally, the transformed and quantized residual block is inversely quantized and inversely transformed, then added to the corresponding prediction data (e.g., the prediction block) obtained using the prediction mode to obtain the reconstructed block. After obtaining the reconstructed block, the loop filtering module performs loop filtering on the reconstructed block according to the filter control data to reduce distortion. After the loop filtering process, the reconstructed block after loop filtering is stored according to the encoded image buffer.

[0174] (III) Decoding process using the image processing method in this embodiment

[0175] Optionally, when the processing device is a decoder on the decoding side, referring to Figure 7, after receiving the encoded bitstream, the decoder's entropy decoding unit parses and decodes the encoded bitstream to obtain transform coefficients. The decoder's inverse transform unit and inverse quantization unit perform inverse transform and inverse quantization processing on the transform coefficients to obtain residual blocks.

[0176] Optionally, the decoder's entropy decoding unit parses and decodes the encoded bitstream to obtain prediction data, such as prediction parameters and related auxiliary information. The decoder's prediction processing unit uses the prediction parameters to perform prediction processing, thereby determining the prediction block corresponding to the residual block.

[0177] Optionally, the prediction processing includes intra-frame prediction processing and / or inter-frame prediction processing, and the intra-frame prediction processing and / or inter-frame prediction processing includes at least one prediction mode or a combination of at least one prediction mode. When the auxiliary information indicates the improved DIMD mode in this embodiment (such as DIMD partition prediction), the prediction mode is used to obtain the pixel values ​​of all pixels to be predicted in the image block to be predicted. After determining that all pixels in the image block to be predicted have been sampled, the prediction result of the image block to be predicted is obtained.

[0178] Optionally, the obtained residual block and the corresponding prediction block (including the predicted luminance block and the predicted chrominance block) are added together to obtain the reconstructed block. The loop filtering unit in the decoder performs loop filtering on the reconstructed block according to the filter control data to reduce distortion and improve video quality. The reconstructed block after loop filtering is further combined into a decoded image and stored in the decoded image buffer or output as decoded video data.

[0179] Optionally, the image processing method provided in this application embodiment can be applied to scenarios where chromaticity prediction and / or luminance prediction of image blocks are performed during the above-mentioned video image encoding process (e.g., intra-frame prediction during video image encoding).

[0180] Optionally, the image processing method provided in this application embodiment can also be used in scenarios where chromaticity prediction and / or luminance prediction are performed on image blocks to be decoded during video decoding, such as intra-frame prediction during video image decoding.

[0181] First Embodiment

[0182] Referring to Figure 8, which is a flowchart illustrating an image processing method according to a first embodiment, the image processing method of this application embodiment can be applied to a processing device, including step S10:

[0183] Step S10: Determine or obtain a first prediction result based on at least one image region divided for the current block.

[0184] In this embodiment, the processing device can be a smart terminal, such as a mobile phone or computer, or a server, such as a local server or a cloud server. This embodiment and this application primarily use a smart terminal as an example for illustration.

[0185] Optionally, the technical solution of this embodiment can be applied to fields such as image encoding and decoding, video encoding and decoding, hardware video encoding and decoding, dedicated circuit video encoding and decoding, and real-time video encoding and decoding.

[0186] Optionally, the processing device can acquire video image data from a video source, segment each frame of the video image data to obtain at least one image block, and determine the image block to be predicted in the at least one image block as the current block.

[0187] Optionally, the current block can be a block of image to be predicted in the encoder and / or decoder.

[0188] Optionally, the image region can be an image region after dividing the current block, a sub-block after dividing the current block, or a set of pixels after dividing the current block into pixels, etc. The following will use an image region after dividing the current block as an example for illustration.

[0189] Optionally, the first prediction result can be the prediction result of intra-frame prediction for the current block, such as the prediction block, the reconstructed block, etc.

[0190] Optionally, the current block can be partitioned to obtain at least one image region for the current block.

[0191] Optionally, the following example only illustrates the partitioning of the current block using the DIMD-based partitioning algorithm, but it is not limited to DIMD and can also be other scenarios applicable to partitioning the current block in this embodiment.

[0192] Optionally, when using a DIMD-based partitioning algorithm, the encoder and decoder do not need to transmit the partitioning mode and the prediction mode corresponding to each partition. They only need to transmit partitioning signaling to determine whether to enable the DIMD-based partitioning algorithm.

[0193] For example, as shown in Figure 9, when performing intra-frame prediction, the DIMD signaling can be used to determine whether the DIMD partition prediction mode is enabled.

[0194] Optionally, if the DIMD signaling is 1 and the partition signaling is 0, then the normal DIMD mode (such as the existing DIMD mode) is used for intra-frame prediction processing, that is, the encoding end uses the normal DIMD mode for encoding and the decoding end uses the normal DIMD mode for decoding.

[0195] Optionally, if the DIMD signaling is 1 and the partition signaling is 1, then the partition DIMD mode (i.e., the improved DIMD mode) is used for intra-frame prediction processing. The encoding end uses the partition DIMD mode for encoding, and the decoding end uses the partition DIMD mode for decoding.

[0196] Optionally, when it is determined that the DIMD-based partition prediction algorithm is used to predict the current block, the current block can be divided into at least one image region using a partitioning mode, and the at least one image region can be predicted to obtain the corresponding prediction result. Then, the prediction results of the at least one image region can be fused to obtain the first prediction result of the current block.

[0197] Optionally, the partitioning mode can be set by the user in advance, or it can be determined according to certain rules (such as using the same partitioning mode as neighboring blocks, or using the element with the largest gradient in the current block (such as the pixel to be predicted) as the starting point (i.e., the dividing line passes through a point in the current block), and the partitioning direction is perpendicular to the gradient direction of that element (i.e., the dividing line passes through the partitioning direction of the current block). There are no restrictions here.

[0198] Optionally, the processing device is an encoding end, which can determine or obtain a first prediction result based on at least one image region divided for the current block.

[0199] Optionally, the processing device is a decoding end, which can determine or obtain a first prediction result based on at least one image region divided for the current block.

[0200] In this embodiment, by determining or obtaining a first prediction result based on at least one image region divided in the current block, it is possible to consider at least one image region divided in the current block during the prediction stage of video encoding and / or decoding, such as intra-frame prediction, in order to determine or obtain a first prediction result. This takes into account different image regions in the current block, making the determined or obtained prediction mode more accurate, improving the prediction accuracy for the current block, and thus supporting the improvement of prediction efficiency in the video encoding and / or decoding process.

[0201] Second Embodiment

[0202] Based on the first embodiment, a second embodiment is proposed.

[0203] In this embodiment, the current block is divided according to at least one of the following methods one to five:

[0204] Method 1: The first reference template of the current block;

[0205] Optionally, since the original pixels of the current block do not exist at the decoding end, in order to measure the cost of a certain decoding mode, such as prediction mode P, a neighboring region of the current block can be used as an approximation of the current block, that is, the first reference template of the current block. For example, the n (n is an integer greater than 1, such as 2) rows of L-shaped regions on the left and above the current block can be used as the first reference template of the current block.

[0206] Optionally, the decoding mode can be applied to the first reference template of the current block, and the prediction result can be subtracted from the reconstruction of the L-shaped region to obtain the prediction residual, the corresponding cost of which can be used as the cost of the prediction mode in the current block.

[0207] Optionally, the first reference template of the current block can be L-shaped, T-shaped, or a row of pixels; there are no restrictions here.

[0208] Optionally, the partitioning mode may include dividing the current block according to a first reference template of the current block.

[0209] Optionally, at least one dividing line (i.e., segmentation line, partition line) in the current block can be determined based on the first reference template of the current block, and the current block can be divided based on the at least one dividing line to obtain at least one image region.

[0210] Optionally, the position of the element with the maximum gradient magnitude (e.g., pixel position) can be determined by performing gradient histogram statistics on the first reference template, and a dividing line passing through the current block can be determined or generated starting from the element position to divide the current block and obtain at least one image region.

[0211] Optionally, regions with large variations in texture complexity of the first reference template can be identified (for example, if the texture complexity of at least one location region in the first reference template is greater than the texture complexity of at least another location region adjacent to it, then the region with large variations in texture complexity between the at least one location region and the at least another location region can be considered as the region with large variations in texture complexity), and at least one dividing line can be identified or generated in the region with large variations in texture complexity, wherein the texture complexity of the regions on both sides of the at least one dividing line in the first reference template is different.

[0212] Optionally, texture complexity may include shape complexity and color complexity. It may be based on the distance and direction between pixels in the first reference template and / or image region, and establish a gray-level co-occurrence matrix through the joint probability distribution between pixel gray levels. Then, starting from this matrix, some statistical quantities are extracted as texture features, and the texture complexity is approximated by the texture features. Alternatively, texture complexity may be approximated by at least one of the variance, mean, standard deviation, etc. of at least one element in the first reference template and / or image region.

[0213] Optionally, it may involve determining at least one dividing direction corresponding to the dividing line of the first reference template, and selecting a pixel position in the current block as the dividing starting point to divide the current block according to at least one dividing direction to obtain at least one image region.

[0214] Optionally, the dividing lines of the first reference template can be determined, and the dividing lines of the first reference template can be translated or copied to the current block to divide the current block and obtain at least one image region.

[0215] Optionally, the current block can be divided according to the first reference template of the current block to obtain at least one image region, and the first prediction result can be determined or obtained based on the at least one image region.

[0216] This method allows the current block to be divided into at least one image region based on the first reference template of the current block, and the first prediction result can be determined or obtained based on the at least one image region. This ensures the effective division of the current block during the prediction stage, so that the first prediction result obtained in the final prediction can take into account different image regions, improve the prediction accuracy of the current block prediction processing, and / or ensure that the decoding end can correctly obtain the best division and the corresponding prediction mode.

[0217] Method 2 requires at least one dividing line;

[0218] Optionally, at least one dividing line can be determined for the current block, and the current block can be divided according to the at least one dividing line to obtain at least one image region.

[0219] Optionally, the dividing line of the current block can be set in advance according to user needs to divide the current block. Alternatively, the dividing line of the current block can be determined according to the gradient information of at least one element in the first reference template to divide the current block. Or, during the prediction stage, the dividing instruction input by the user can be received, at least one dividing line can be determined or generated according to the dividing instruction, and the current block can be divided according to the at least one dividing line.

[0220] Optionally, at least one dividing line is determined or obtained according to at least one of the following methods 1 to 6:

[0221] Method 1: The gradient magnitude and / or gradient direction of at least one element within the first reference template;

[0222] Optionally, a gradient histogram of the first reference template can be calculated, the gradient histogram including the gradient magnitude and gradient direction of at least one element in the first reference template.

[0223] Alternatively, at least one dividing line can be obtained from a lookup table based on the gradient magnitude and / or gradient direction of at least one element.

[0224] Optionally, it can be determined whether the current block needs to be divided based on the gradient magnitude of at least one element in the first reference template. If the current block needs to be divided, the element with the largest gradient magnitude can be determined, and at least one dividing line can be determined or generated based on the position of the element corresponding to the element with the largest gradient magnitude. The current block can be divided based on the at least one dividing line to obtain at least one image region.

[0225] Optionally, the gradient direction corresponding to the element with the largest gradient magnitude in the gradient histogram of the first reference template can be determined, and at least one dividing line can be determined or generated with the dividing direction or angle perpendicular to the gradient direction as the dividing line. Then, the current block is divided according to the at least one dividing line to obtain at least one image region.

[0226] In this method, by determining or obtaining at least one dividing line based on the gradient magnitude and / or gradient direction of at least one element in the first reference template, and dividing the current block into at least one image region based on the at least one dividing line, and determining or obtaining the first prediction result based on the at least one image region, it is possible to ensure the effective division of the current block during the prediction stage, so that the first prediction result obtained in the final prediction can take into account different image regions, improve the prediction accuracy of the current block prediction processing, and / or ensure that the decoding end can correctly obtain the best division and the corresponding prediction mode.

[0227] Method 2: The position of at least one element with the maximum gradient magnitude within the first reference template and the direction perpendicular to the direction of the maximum gradient;

[0228] Optionally, the gradient magnitude and gradient direction of each element in the first reference template can be determined from the gradient corresponding to the first reference template, thereby determining the position of the element with at least one maximum gradient magnitude and the direction perpendicular to the maximum gradient direction in the first reference template.

[0229] Optionally, the position of at least one element with the maximum gradient magnitude and the direction of the maximum gradient can be found in a pre-set lookup table, and then the direction perpendicular to the direction of the maximum gradient can be determined based on the direction of the maximum gradient. The pre-set lookup table stores the gradient magnitude and gradient direction corresponding to each element of the first reference template.

[0230] Optionally, at least one dividing line can be determined or generated, starting from at least one element with the maximum gradient magnitude within the first reference template, with the dividing angle or dividing direction perpendicular to the direction of the maximum gradient as the dividing angle, and the current block can be divided according to the dividing line to obtain at least one image region.

[0231] Optionally, since the first reference template can be used to approximate the current block at the decoding end, the dividing line of the current block can be determined based on the dividing line of the first reference template. For example, if the pose parameters of the two correspond to each other, the current block can be divided according to the dividing line of the current block to obtain at least one image region.

[0232] In this method, by determining or obtaining at least one dividing line based on the position of at least one element with the maximum gradient magnitude within the first reference template and the direction perpendicular to the maximum gradient direction, and dividing the current block according to the at least one dividing line to obtain at least one image region, and determining or obtaining the first prediction result based on the at least one image region, it is possible to ensure the effective division of the current block during the prediction stage, so that the first prediction result obtained in the final prediction can take into account different image regions, improve the prediction accuracy of the current block prediction processing, and / or ensure that the decoding end can correctly obtain the optimal division and the corresponding prediction mode.

[0233] Method 3: The first gradient histogram and / or statistical feature information corresponding to the first reference template;

[0234] Optionally, the statistical feature information may include the gradient information of each element in the first reference template, such as the gradient magnitude, gradient direction, gradient magnitude in the horizontal direction, gradient magnitude in the vertical direction, etc.

[0235] Optionally, gradient histogram statistics can be performed on the first reference template to obtain the first gradient histogram of the first reference template, and the first gradient histogram can be approximated as the first gradient histogram of the current block.

[0236] Optionally, the position of the element with the largest gradient magnitude and the direction of the maximum gradient corresponding to that element position can be determined based on a gradient histogram and / or statistical feature information corresponding to the first reference template.

[0237] Optionally, within the current block, starting from the position corresponding to the element with the largest gradient magnitude, and using the direction perpendicular to the direction of the largest gradient as the dividing angle or dividing direction, at least one dividing line can be determined or generated, and the current block can be divided according to the at least one dividing line to obtain at least one image region.

[0238] For example, as shown in Figure 10, the prediction mode diagram of the current block can be determined based on the first gradient histogram. The directions indicated by the arrows in the figure can correspond to different gradient directions. Therefore, the dividing line can be determined based on the gradient direction at different positions in Figure 10 to divide the current block and obtain at least one image region. For example, as shown in Figure 11, a dividing line can be determined or generated in the direction perpendicular to the gradient direction corresponding to the element with the maximum gradient magnitude to divide the current block.

[0239] In this method, at least one dividing line is determined or obtained based on the first gradient histogram and / or statistical feature information corresponding to the first reference template. The current block is then divided into at least one image region based on the at least one dividing line. The first prediction result is then determined or obtained based on the at least one image region. This ensures that the current block is effectively divided during the prediction stage, so that the first prediction result obtained at the end can take into account different image regions. This improves the prediction accuracy of the current block and / or ensures that the decoding end can correctly obtain the optimal division and the corresponding prediction mode.

[0240] Method 4, a first reference template, and at least one fitting coefficient and / or partition weight coefficient;

[0241] Optionally, the fitting coefficient can be the slope in the linear regression equation, used to quantify the strength and direction of the linear relationship between the independent and dependent variables. Its core function is to assess the degree of association between variables. It can be calculated through linear regression and used to partition the blocks.

[0242] Optionally, the partition weight coefficients can be pre-set or obtained from a lookup table, and can be calculated through linear regression to partition the blocks.

[0243] Optionally, the fitting coefficients and / or partition weight coefficients can be represented in matrix form, or in the form of data, numerical values, etc.

[0244] Optionally, linear regression calculations can be performed based on the image patch to determine at least one fitting coefficient and / or partition weight coefficient, and at least one dividing line can be determined based on the at least one fitting coefficient and / or partition weight coefficient, and the first reference template, to divide the current patch.

[0245] Optionally, a linear regression calculation can be performed on the first reference template and at least one fitting coefficient and / or partition weight coefficient. Based on the linear regression calculation result, at least one dividing line can be determined or obtained. For example, the index corresponding to the linear regression calculation result can be determined and input into a lookup table containing information on dividing lines passing through at least one block (such as the current block) (such as the coordinate position and division direction of the dividing line in at least one image block). The output is at least one dividing line passing through the current block.

[0246] Optionally, at least one element, at least one fitting coefficient, and at least one partition weight coefficient from the first reference template can be input into the regression matrix function calculation formula to obtain at least one regression matrix. Based on the at least one regression matrix, at least one dividing line can be determined or generated to divide the current block. For example, if each element in the regression matrix is ​​examined, and there is an element at a certain position in the regression matrix that is significantly different from the surrounding elements (for example, one element is 5, and another element around that element is 20), then the element position point in the current block corresponding to that position in the regression matrix can be determined. At least one dividing line can be determined or generated starting from the element position point in the current block, and the current block can be divided based on the dividing line to obtain at least one image region.

[0247] Optionally, the fitting coefficient corresponding to at least one element in the first reference template can be determined, and the fitting coefficient can be compared with a preset fitting coefficient threshold (e.g., 5). Elements corresponding to fitting coefficients greater than the preset fitting coefficient threshold are classified into one category, and / or elements corresponding to fitting coefficients less than or equal to the preset fitting coefficient threshold are classified into one category. In this way, at least one dividing line is determined or generated, and the current block is divided according to the at least one dividing line to obtain at least one image region.

[0248] Optionally, the partition weight coefficient corresponding to at least one element in the first reference template can be determined, and elements with partition weight coefficients greater than a preset partition weight coefficient threshold (e.g., 8) can be grouped into one category, and / or elements with partition weight coefficients less than the preset partition weight coefficient threshold can be grouped into one category, thereby determining or generating at least one dividing line, and dividing the current block according to the at least one dividing line to obtain at least one image region.

[0249] In this method, by determining or obtaining at least one dividing line based on a first reference template and at least one fitting coefficient and / or partition weight coefficient, and dividing the current block into at least one image region based on the at least one dividing line, and determining or obtaining a first prediction result based on the at least one image region, it is possible to ensure the effective division of the current block during the prediction stage, so that the first prediction result obtained in the final prediction can take into account different image regions, improve the prediction accuracy of the current block prediction processing, and / or ensure that the decoding end can correctly obtain the optimal division and the corresponding prediction mode.

[0250] Method 5: At least one first weight coefficient matrix and / or a first fitting coefficient matrix corresponding to the first reference template;

[0251] Optionally, the first fitting coefficient matrix may include at least one fitting coefficient, and can be obtained by performing linear regression calculation on the first reference template. For example, the elements of the first reference template are input into a pre-set linear regression function for calculating the fitting coefficient matrix to obtain a first weight coefficient matrix containing at least one fitting coefficient.

[0252] Optionally, the first weight coefficient matrix may include at least one partition weight coefficient, which can be obtained by performing linear regression calculation on the first reference template. For example, the elements of the first reference template are input into a pre-set linear regression function for calculating the weight coefficient matrix to obtain the first weight coefficient matrix containing at least one partition weight coefficient.

[0253] Optionally, the first fitting coefficient matrix and / or the first weight coefficient matrix can be a regression matrix. The following example uses the first regression matrix instead of the first fitting coefficient matrix and / or the first weight coefficient matrix.

[0254] Optionally, at least one index corresponding to the first regression matrix can be determined and entered into a lookup table for searching. Based on the search result (such as the position of the element passed by the dividing line in the current block), at least one dividing line passing through the current block can be determined or obtained, and the current block can be divided according to the at least one dividing line to obtain at least one image region.

[0255] Optionally, at least one first regression matrix can be input into a neural network for model training. Based on the output of the neural network, at least one dividing line passing through the current block can be determined or obtained, and the current block can be divided according to the at least one dividing line to obtain at least one image region.

[0256] Alternatively, the first regression matrix can be used to perform prediction processing on the current block to obtain the first prediction result.

[0257] In this method, by determining or obtaining at least one dividing line based on at least one first weight coefficient matrix or first fitting coefficient matrix corresponding to the first reference template, and dividing the current block into at least one image region based on the at least one dividing line, and determining or obtaining the first prediction result based on the at least one image region, it is possible to ensure the effective division of the current block during the prediction stage, so that the first prediction result obtained in the final prediction can take into account different image regions, improve the prediction accuracy of the current block prediction processing, and / or ensure that the decoding end can correctly obtain the best division and the corresponding prediction mode.

[0258] Method 6: At least one of the following: neighboring blocks, non-neighboring blocks, co-occurring blocks, temporal blocks, default blocks, cross-component blocks, and candidate blocks of the current block.

[0259] Optionally, the default block can be a pre-set block, such as a block with typical pixel characteristics pre-set by the encoder and / or decoder.

[0260] Optionally, the neighboring block can be a block adjacent to the current block, and / or a block that has already been predicted or reconstructed.

[0261] Optionally, a non-neighbor block can be a block that is not adjacent to the current block, and / or a block that has already been predicted or reconstructed.

[0262] Optionally, the co-position block can be an image block in the co-position image that has the same position and size as the current block. Optionally, the co-position image can be the image in the reference image that is closest to the current image in time.

[0263] Optionally, the temporal block can be a block that is distinguished in the time domain, such as an image block in the previous frame. For example, if there is video data containing three frames of images, the first frame is played in the first second, the second frame is played in the second second, and the third frame is played in the third second, if the image block predicted at the current moment (such as the current block) is an image block after the second frame is divided, then the temporal block can be determined to be the image block corresponding to it in the first frame.

[0264] Optionally, the cross-component block can be an image block that is in a different component from at least one current block. For example, if the image block to be predicted is an image block of the Y component, then the cross-component block can be an image block of the U component and / or the V component.

[0265] Optionally, if at least one current block is an image block of the U component, then the cross-component block can be an image block of the Y component and / or the V component.

[0266] Optionally, if at least one current block is an image block of the V component, then the cross-component block can be an image block of the U component and / or the Y component.

[0267] Optionally, block vector calculation is performed on the current block, and candidate blocks corresponding to the current block are determined based on the block vector calculation results. For example, the pixels corresponding to the block vector calculation results are used as pixels in the candidate blocks.

[0268] Optionally, motion vector calculation is performed on the current block, and candidate blocks corresponding to the current block are determined based on the motion vector calculation results. For example, the pixels corresponding to the motion vector calculation results are used as pixels in the candidate blocks.

[0269] Optionally, the dividing line corresponding to at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, default blocks, cross-component blocks, and candidate blocks of the current block can be used as the dividing line of the current block to divide the current block and obtain at least one image region.

[0270] For example, if neighboring blocks are predicted using a partitioning mode, the dividing lines for these blocks can be determined. The starting point of the dividing lines, their direction or angle, can also be determined. Furthermore, the position of the element within the current block corresponding to the starting point of the neighboring block's dividing lines can be identified. For instance, if the starting point of the neighboring block's dividing lines is the position of the first element at the top left corner of the neighboring block, then the first element at the top left corner of the current block can be determined as the starting point of the current block's dividing lines. A dividing line with the same direction or angle as the neighboring block's dividing lines can then be determined or generated based on the starting point of the current block to divide the current block, resulting in at least one image region.

[0271] Optionally, the dividing lines of neighboring or non-neighboring blocks can be extended. If the line passes through the current block, the part passing through the current block is taken as the dividing line of the current block, and the current block is divided according to the dividing line of the current block to obtain at least one image region.

[0272] In this method, at least one dividing line is determined or obtained based on at least one of the following: neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, default blocks, cross-component blocks, and candidate blocks of the current block. The current block is then divided into at least one image region based on the at least one dividing line, and a first prediction result is determined or obtained based on the at least one image region. This ensures that the prediction processing of the current block can take into account other blocks, making the final prediction result more accurate.

[0273] Optionally, in method two, at least one dividing line can be determined or obtained according to at least one of methods 1 to 6, the current block can be divided according to the at least one dividing line to obtain at least one image region, and the first prediction result of the current block can be determined or obtained according to the at least one image region.

[0274] In this method, by dividing the current block according to at least one dividing line to obtain at least one image region, and determining or obtaining a first prediction result based on at least one image region, the current block can be effectively divided, so that the first prediction result obtained in the final prediction can take into account different image regions, thereby improving the prediction accuracy of the current block prediction processing.

[0275] Method 3: Partition parameters of the current block;

[0276] Optionally, the partitioning mode may include dividing the current block according to the partitioning parameters of the current block.

[0277] Optionally, at least one dividing line can be determined based on the partitioning parameters of the current block, and the current block can be divided into at least one image region based on the determined at least one dividing line.

[0278] Optionally, the partitioning parameters may include partitioning information for the current block, such as the partition line of the current block. They may also include parameters used to deduce the partitioning information for the current block.

[0279] For example, partitioning parameters include the texture distribution characteristics of the current block (such as the distribution pattern of texture complexity; if the current block is a 2x2 image block, it can be approximated by a first reference template of size 2x2; the texture complexity of the four pixel regions in the first reference template can be used as the texture distribution characteristics of the current block; for example, the texture complexity of the upper two pixel regions (which can be approximated by the average of the upper two pixels) is higher than the texture complexity of the lower two pixel regions (which can be approximated by the average of the lower two pixels)). Based on these texture distribution characteristics, a dividing line can be determined or generated to divide the current block, resulting in image regions with high texture complexity and image regions with low texture complexity.

[0280] Optionally, the current block can be divided according to the partitioning parameters of the current block to obtain at least one image region, and a first prediction result can be determined or obtained based on the at least one image region.

[0281] Optionally, the partitioning parameters are determined or obtained according to at least one of the following methods 7 to 22:

[0282] Method 7, template feature information of the first reference template;

[0283] Optionally, the template feature information can be the relevant feature information of the first reference template, or it can be a type of feature information calculated based on the template feature operator of the first reference template. For example, it can be the gradient information of elements at different positions in the first reference template, or the texture complexity information of different regions in the first reference template.

[0284] Optionally, an element in the first reference template can be a value at a certain position in the first reference template.

[0285] Optionally, the element can be a pixel, or the value of a pixel after transformation, quantization, or other processing.

[0286] Optionally, the template feature information of the first reference template can be used as the partitioning parameter of the current block, or the template feature information of the first reference template can be transformed, such as enlarged or reduced, or its format changed, to obtain the partitioning parameter of the current block.

[0287] Optionally, the current block can be divided based on the template feature information of the first reference template to obtain at least one image region.

[0288] Optionally, the dividing line of the current block can be determined or obtained based on the template feature information of the first reference template, and the current block can be divided according to the dividing line to obtain at least one image region.

[0289] Optionally, regions with large variations in texture complexity in the first reference template can be determined using template feature information of the first reference template (for example, the boundary region between regions with high texture complexity and regions with low texture complexity in the first reference template can be considered as regions with large variations in texture complexity). At least one dividing line can be determined or generated in the region with large variations in texture complexity to distinguish the texture complexity of different regions in the first reference template. The at least one dividing line can be translated or copied to the current block to divide the current block and obtain at least one image region.

[0290] Optionally, if the template feature information of the first reference template includes gradient information, such as the gradient magnitude of each element in the first reference template, the position of the element corresponding to the largest gradient magnitude can be selected as the starting point to determine or generate a dividing line passing through the current block to divide the current block and obtain at least one image region.

[0291] Optionally, if the template feature information of the first reference template satisfies a condition, such as the template feature information belonging to a specific one or more template feature information (for example, when the template feature information contains gradient information, gradient information containing a gradient magnitude greater than a preset gradient threshold (e.g., 100) can be used as specific media feature information; when the template feature information contains texture complexity information, texture complexity information containing at least one element with high texture complexity and at least one element with low texture complexity can be used as specific template feature information), then the current block can be divided according to the first reference template of the current block, the partitioning parameters of the current block, at least one dividing line, the first neural network, and at least one item in the first lookup table to obtain at least one image region.

[0292] Optionally, the variance of at least one element can be used to approximate the texture complexity. If the variance of at least one element is greater than a preset variance threshold (e.g., 10), the texture complexity of the at least one element can be determined to be high texture complexity. If the variance of at least one element is less than the preset variance threshold, the texture complexity of the at least one element can be determined to be low texture complexity.

[0293] Optionally, at least one partitioning parameter can be determined or obtained based on the template feature information of the first reference template, the current block can be divided based on the at least one partitioning parameter to obtain at least one image region, and the first prediction result can be determined or obtained based on the at least one image region.

[0294] This method allows for the determination or acquisition of partitioning parameters based on the template feature information of the first reference template. The current block is then divided into at least one image region based on the partitioning parameters, and the first prediction result is determined or obtained based on the at least one image region. This ensures the effective division of the current block during the prediction stage, enabling the final first prediction result to take into account different image regions. This improves the prediction accuracy of the current block and / or ensures that the decoding end can correctly obtain the optimal partitioning and corresponding prediction mode.

[0295] Method 8, gradient information of the first reference template;

[0296] Optionally, the gradient information of the first reference template can be obtained through the gradient histogram of the first reference template. The gradient information may include the gradient magnitude, gradient direction, angle, etc. of each element in the first reference template.

[0297] Optionally, the gradient information of the first reference template can be used as the partitioning parameter of the current block, or the gradient information of the first reference template can be transformed, such as increased or decreased, or its format changed, to obtain the partitioning parameter of the current block.

[0298] Optionally, the maximum gradient magnitude can be determined by the gradient information of the first reference template, and the texture boundary of the current block can be determined based on the maximum gradient magnitude, so as to determine or generate a dividing line, divide the texture boundary of the current block, and obtain at least one image region.

[0299] Optionally, the position of the element with the maximum gradient can be determined by the gradient information of the first reference template, and this can be used as the starting point of the dividing line. The direction perpendicular to the gradient direction corresponding to the position of the element with the maximum gradient can be determined, and this perpendicular direction can be used as the direction of the dividing line. A dividing line can be determined or generated based on the determined starting point and direction of the dividing line to divide the current block and obtain at least one image region.

[0300] Optionally, if the gradient information of the first reference template satisfies a condition, such as the gradient information belonging to a specific one or more gradient information (for example, the gradient information with a gradient magnitude greater than a preset gradient magnitude threshold (e.g., 655) is taken as the specific gradient information), then the current block can be divided according to the first reference template of the current block, the partitioning parameters of the current block, at least one dividing line, the first neural network and at least one item in the first lookup table to obtain at least one image region.

[0301] Optionally, at least one partitioning parameter can be determined or obtained based on the gradient information of the first reference template, the current block can be divided based on the at least one partitioning parameter to obtain at least one image region, and the first prediction result can be determined or obtained based on the at least one image region.

[0302] This method allows for the determination or acquisition of partitioning parameters based on the gradient information of the first reference template. The current block is then divided into at least one image region based on the partitioning parameters, and the first prediction result is determined or obtained based on the at least one image region. This ensures the effective division of the current block during the prediction stage, enabling the final first prediction result to take into account different image regions. This improves the prediction accuracy of the current block and / or ensures that the decoding end can correctly obtain the optimal partitioning and corresponding prediction mode.

[0303] Method 9, residual fluctuation of the first reference template;

[0304] Optionally, a possible prediction mode can be selected from the existing candidate mode list for encoding prediction. The residual is determined based on the prediction result. If the residual shows an image region with high energy (i.e., residual fluctuation), it indicates that the encoding effect of the image region is not good, and the encoding region needs to be partitioned for encoding. That is, the current block needs to be divided into image regions and each image region needs to be encoded.

[0305] Optionally, on the decoding side, a first reference template can be used to approximate the current block, so as to divide the current block into at least one image region based on the residual fluctuation of the first reference template.

[0306] Optionally, the residual block corresponding to the first reference template can be determined, and the value of at least one residual element in the residual block can be approximated as the residual fluctuation. For example, if the value of the residual element is closer to 0, it means that the residual fluctuation is smaller, and vice versa.

[0307] Optionally, a residual element can be an element in the residual block or a value at a certain position in the residual block.

[0308] Optionally, at least one dividing line can be determined or obtained based on the residual fluctuation of the first reference template, and the current block can be divided according to the at least one dividing line to obtain at least one image region.

[0309] Optionally, at least two image regions with different residual fluctuations in the current block can be determined based on the residual fluctuations of the first reference template, and the current block can be divided in this way. For example, the residual fluctuation of one image region is 0, and the residual fluctuation of the other image region is any data greater than 0.

[0310] Optionally, if the residual fluctuation of the first reference template satisfies a condition, such as the residual fluctuation matching a preset residual fluctuation threshold range (e.g., the range of 1-10), then the current block can be divided according to the first reference template of the current block, the partitioning parameters of the current block, at least one dividing line, the first neural network, and at least one item in the first lookup table to obtain at least one image region.

[0311] Optionally, the residual fluctuation of the first reference template can be used as the partitioning parameter of the current block, or the residual fluctuation of the first reference template can be transformed, such as increased or decreased, or its format changed, to obtain the partitioning parameter of the current block.

[0312] Optionally, at least one partitioning parameter can be determined or obtained based on the residual fluctuation of the first reference template, the current block can be divided according to the at least one partitioning parameter to obtain at least one image region, and the first prediction result can be determined or obtained based on the at least one image region.

[0313] This method allows for the determination or acquisition of partitioning parameters based on the residual fluctuations of the first reference template. The current block is then divided into at least one image region based on these partitioning parameters. The first prediction result is then determined or obtained based on this at least one image region. This ensures the effective division of the current block during the prediction stage, enabling the final prediction result to take into account different image regions. This improves the prediction accuracy of the current block and / or guarantees that the decoding end can correctly obtain the optimal partitioning and corresponding prediction mode.

[0314] Method 10, the structure tensor of the first reference template;

[0315] Optionally, the structure tensor of the first reference template can be a symmetric matrix, a second-order matrix, or an inertial tensor, which can be calculated based on the gradient information of the first reference template.

[0316] Optionally, the structure tensor of the first reference template may include the horizontal gradient and / or vertical gradient of the first reference template, which can approximate the texture features of the first reference template.

[0317] Optionally, the matrix formula corresponding to the structure tensor J can be shown in Formula (V) below.

[0318] Optionally, J is the structure tensor, g x g is the gradient in the horizontal direction of at least one element within the first reference template. y The gradient is the vertical direction of at least one element within the first reference template.

[0319] Alternatively, the eigenvalues ​​can be calculated by solving the characteristic equation (such as formula (VI)).

[0320] Optionally, λ is an eigenvalue.

[0321] Optionally, at least two calculations can be performed based on formula (VI) to obtain two feature values ​​of different sizes (e.g., feature value 1 and feature value 2). If both feature value 1 and feature value 2 are equal to or greater than a certain value (e.g., 0.01 or 0), then the image region corresponding to these two feature values ​​in the current block corresponding to the first reference template is a flat region and does not need to be divided. If one feature value is greater than a certain value (e.g., 0.01 or 0) and the other feature value is equal to a certain value (e.g., 0.01 or 0), then the image region corresponding to these two feature values ​​in the current block corresponding to the first reference template is an edge region. If both feature value 1 and feature value 2 are much greater than a certain value (e.g., 0.01 or 0), then the image region corresponding to these two feature values ​​in the current block corresponding to the first reference template is a region with high texture complexity and requires image region division processing.

[0322] Optionally, at least one dividing line can be determined or obtained through the structural tensor of the first reference template, and the current block can be divided according to the dividing line to obtain at least one image region.

[0323] Optionally, regions with high texture complexity in the block to be predicted can be determined based on the structural tensor of the first reference template, and at least one dividing line can be determined or generated at the boundary of the region with high texture complexity. The current block can be divided according to the at least one dividing line to obtain at least one image region.

[0324] Optionally, the structure tensor of the first reference template can be used as the partitioning parameter of the current block, or the structure tensor of the first reference template can be deformed, such as increased or decreased, or its format changed, to obtain the partitioning parameter of the current block.

[0325] Optionally, if the structure tensor of the first reference template satisfies a condition, such as the existence of at least two feature values ​​greater than 0 in the structure tensor, the current block can be divided according to the first reference template of the current block, the partitioning parameters of the current block, at least one dividing line, the first neural network, and at least one item in the first lookup table to obtain at least one image region.

[0326] Optionally, at least one partitioning parameter can be determined or obtained based on the structural tensor of the first reference template, the current block can be divided based on the at least one partitioning parameter to obtain at least one image region, and the first prediction result can be determined or obtained based on the at least one image region.

[0327] This method allows for the determination or acquisition of partitioning parameters based on the structure tensor of the first reference template. The current block is then divided into at least one image region based on these partitioning parameters. A first prediction result is then determined or obtained based on this at least one image region. This ensures the effective division of the current block during the prediction phase, enabling the final first prediction result to take into account different image regions. This improves the prediction accuracy of the current block and / or guarantees that the decoding end can correctly obtain the optimal partitioning and corresponding prediction mode.

[0328] Method 11, Phase consistency of the first reference template;

[0329] Optionally, for any element in the first reference template, a set of multi-scale, different-direction filters (typically Log-Gabor) can be used to calculate the phase of each pixel under these filters, and the phase consistency of the first reference template can be evaluated based on the phase of each pixel under these filters.

[0330] Optionally, if the phase of each direction / scale is consistent at a certain element position of the first reference template (e.g., the filter response is around 0° or 180°), it can be determined that there is a real boundary at that element position, and image region segmentation processing is required. A segmentation line passing through the element position can be determined or generated to segment the current block into an image region.

[0331] Optionally, if the phase consistency of the first reference template satisfies a condition, such as the phase of each direction / scale is consistent at any element position of the first reference template, then the current block can be divided according to the first reference template of the current block, the partitioning parameters of the current block, at least one partitioning line, the first neural network and at least one item in the first lookup table to obtain at least one image region.

[0332] Optionally, the phase consistency of the first reference template can be used as the partitioning parameter of the current block, or the phase consistency of the first reference template can be transformed, such as increased or decreased, or its format changed, to obtain the partitioning parameter of the current block.

[0333] Optionally, at least one partitioning parameter can be determined or obtained based on the phase consistency of the first reference template, the current block can be divided based on the at least one partitioning parameter to obtain at least one image region, and the first prediction result can be determined or obtained based on the at least one image region.

[0334] This method allows for the determination or acquisition of partitioning parameters based on the phase consistency of the first reference template. The current block is then divided into at least one image region based on the partitioning parameters, and the first prediction result is determined or obtained based on the at least one image region. This ensures the effective division of the current block during the prediction stage, enabling the final first prediction result to take into account different image regions. This improves the prediction accuracy of the current block and / or ensures that the decoding end can correctly obtain the optimal partitioning and corresponding prediction mode.

[0335] Method 12, second derivative information of the first reference template;

[0336] Alternatively, the second derivative information can be a type of higher-order statistical information.

[0337] Optionally, the gradient information of the first reference template is the first derivative information. When the gradient is at its maximum, its corresponding second derivative information (Lap la ac ian or LoG) will cross zero at the boundary center. Therefore, the boundary line can be marked by the position of Laplace = 0.

[0338] Optionally, the second derivative information of the first reference template can be determined based on the gradient information of the current block, and it can be checked whether the second derivative information corresponding to the position of the element with the largest gradient magnitude is zero at the boundary center. If it is zero, it can be determined that there is a real boundary at the position of the element, and image region segmentation processing is required. A segmentation line passing through the position of the element can be determined or generated to segment the current block into an image region.

[0339] Optionally, if the second derivative information of the first reference template satisfies a condition, such as whether the second derivative information crosses zero at the boundary center, then the current block can be divided according to the first reference template of the current block, the partitioning parameters of the current block, at least one dividing line, the first neural network, and at least one item in the first lookup table to obtain at least one image region.

[0340] Optionally, the second derivative information of the first reference template can be used as the partitioning parameter of the current block, or the second derivative information of the first reference template can be transformed, such as increased or decreased, or its format changed, to obtain the partitioning parameter of the current block.

[0341] Optionally, at least one partitioning parameter can be determined or obtained based on the second derivative information of the first reference template, the current block can be divided based on the at least one partitioning parameter to obtain at least one image region, and the first prediction result can be determined or obtained based on the at least one image region.

[0342] This method allows for the determination or acquisition of partitioning parameters based on the second derivative information of the first reference template. The current block is then divided into at least one image region based on the partitioning parameters, and the first prediction result is determined or obtained based on the at least one image region. This ensures the effective division of the current block during the prediction stage, enabling the final first prediction result to take into account different image regions. This improves the prediction accuracy of the current block and / or guarantees that the decoding end can correctly obtain the optimal partitioning and corresponding prediction mode.

[0343] Method 13, structural feature parameters of the first reference template;

[0344] Optionally, the structural feature parameters of the first reference template may include the structural tensor, phase consistency, and second derivative information of the first reference template.

[0345] Optionally, at least one dividing line can be determined or generated based on the structural feature parameters of the first reference template, and the current block can be divided based on the at least one dividing line to obtain at least one image region.

[0346] Optionally, the texture complexity in the current block can be determined based on the structural feature parameters of the first reference template, and regions with different texture complexities in the current block (such as high texture complexity regions and low texture complexity regions) can be determined. In this way, a dividing line (i.e., a dividing line) can be determined or generated to divide the current block into two image regions, where the texture complexity of one image region is higher than that of the other image region.

[0347] Optionally, the texture complexity corresponding to at least one element in a high-complexity texture region is high-texture-complexity, and the texture complexity corresponding to at least one element in a low-texture-complexity texture region is low-texture-complexity.

[0348] Optionally, if the structural feature parameters of the first reference template satisfy a condition, such as the second derivative information crossing zero at the boundary center, and the phase of each direction / scale is consistent at any element position of the first reference template, then the current block can be divided according to the first reference template of the current block, the partitioning parameters of the current block, at least one partitioning line, the first neural network, and at least one item in the first lookup table to obtain at least one image region.

[0349] Optionally, the structural feature parameters of the first reference template can be used as the partitioning parameters of the current block, or the structural feature parameters of the first reference template can be transformed, such as increased or decreased, or the format changed, to obtain the partitioning parameters of the current block.

[0350] Optionally, at least one partitioning parameter can be determined or obtained based on the structural feature parameters of the first reference template, the current block can be divided based on the at least one partitioning parameter to obtain at least one image region, and the first prediction result can be determined or obtained based on the at least one image region.

[0351] This method allows for the determination or acquisition of partitioning parameters based on the structural feature parameters of the first reference template. The current block is then divided into at least one image region based on the partitioning parameters, and a first prediction result is determined or obtained based on the at least one image region. This ensures the effective division of the current block during the prediction stage, enabling the final first prediction result to take into account different image regions. This improves the prediction accuracy of the current block and / or ensures that the decoding end can correctly obtain the optimal partitioning and corresponding prediction mode.

[0352] Method 14, edge structure description information of the first reference template;

[0353] Optionally, the edge structure description information of the first reference template may include residual fluctuations, phase consistency, etc. of the first reference template.

[0354] Optionally, at least one dividing line can be determined or generated based on the edge structure description information of the first reference template, and the current block can be divided based on the at least one dividing line to obtain at least one image region.

[0355] Optionally, the texture complexity in the current block can be determined based on the edge structure description information of the first reference template, and the region where the texture features change in the current block can be determined, thereby determining or generating a dividing line to divide the current block into two image regions, where the texture complexity of one image region is higher than that of the other image region.

[0356] Optionally, if the edge structure description information of the first reference template satisfies a condition, such as the phase of each direction / scale is consistent at any element position of the first reference template, and there is a residual fluctuation greater than a preset residual fluctuation threshold (e.g., 5), then the current block can be divided according to the first reference template of the current block, the partitioning parameters of the current block, at least one dividing line, the first neural network, and at least one item in the first lookup table to obtain at least one image region.

[0357] Optionally, the edge structure description information of the first reference template can be used as the partitioning parameter of the current block, or the edge structure description information of the first reference template can be transformed, such as enlarged or shrunk, or its format changed, to obtain the partitioning parameter of the current block.

[0358] Optionally, at least one partitioning parameter can be determined or obtained based on the edge structure description information of the first reference template, the current block can be divided based on the at least one partitioning parameter to obtain at least one image region, and the first prediction result can be determined or obtained based on the at least one image region.

[0359] This method allows for the determination or acquisition of partitioning parameters based on the edge structure description information of the first reference template. The current block is then divided into at least one image region based on the partitioning parameters, and a first prediction result is determined or obtained based on the at least one image region. This ensures the effective division of the current block during the prediction stage, enabling the final first prediction result to take into account different image regions. This improves the prediction accuracy of the current block and / or ensures that the decoding end can correctly obtain the optimal partitioning and corresponding prediction mode.

[0360] Method 15: The variance and / or mean of at least one element in the first reference template;

[0361] Optionally, the element in the first reference template can be a value at a certain position in the first reference template, which can be a predicted pixel or a reconstructed pixel.

[0362] Optionally, the pixel location point in the current block can be determined based on the variance and / or mean of at least one element in the first reference template, and a dividing line passing through the pixel location point can be determined or generated to divide the current block, thereby obtaining at least one image region.

[0363] Optionally, the location points of elements with large variance and / or mean deviation can be determined, and a dividing line passing through the current block can be determined or generated based on these location points to divide the current block and obtain at least one image region.

[0364] Optionally, if the variance and / or mean of at least one element in the first reference template satisfies a condition, such as the variance being greater than a preset variance threshold (e.g., 10) and / or the mean being greater than a preset mean threshold (e.g., 5), then the current block can be divided according to the first reference template of the current block, the partitioning parameters of the current block, at least one dividing line, the first neural network, and at least one item in the first lookup table to obtain at least one image region.

[0365] Optionally, the variance and / or mean of at least one element in the first reference template can be used as the partitioning parameter of the current block. Alternatively, the variance and / or mean of at least one element in the first reference template can be transformed, such as increased or decreased, or the format changed, to obtain the partitioning parameter of the current block.

[0366] Optionally, at least one partitioning parameter can be determined or obtained based on the variance and / or mean of at least one element in the first reference template, the current block can be divided based on the at least one partitioning parameter to obtain at least one image region, and the first prediction result can be determined or obtained based on the at least one image region.

[0367] This method allows for the determination or acquisition of partitioning parameters based on the variance and / or mean of at least one element in the first reference template. The current block is then divided into at least one image region based on these partitioning parameters, and a first prediction result is determined or obtained based on this at least one image region. This ensures the effective partitioning of the current block during the prediction phase, enabling the final first prediction result to take into account different image regions. This improves the prediction accuracy of the current block and / or guarantees that the decoding end can correctly obtain the optimal partitioning and corresponding prediction mode.

[0368] Method 16, texture parameters of the first reference template;

[0369] Optionally, the texture parameters of the first reference template may include texture complexity, texture features, and texture changes at each element position.

[0370] Optionally, a dividing line can be determined or generated based on the texture parameters of the first reference template to divide the current block into regions with different texture complexities, thereby obtaining at least one image region.

[0371] Optionally, the texture parameters of the first reference template can be approximated as the texture parameters of the current block, and then the region with obvious texture change characteristics of the current block can be determined based on the texture parameters of the first reference template. This region can then be divided to obtain at least one image region.

[0372] Optionally, if the texture parameters of the first reference template satisfy a condition, such as the texture parameters being greater than a preset texture parameter threshold (e.g., if the texture parameters include texture complexity, and the texture complexity of the first reference template is measured by the variance of at least one element in the first reference template, then the preset texture parameter threshold can be a preset variance threshold, such as 10), then the current block can be divided according to the first reference template of the current block, the partition parameters of the current block, at least one dividing line, the first neural network, and at least one item in the first lookup table to obtain at least one image region.

[0373] Optionally, the texture parameters of the first reference template can be used as the partitioning parameters of the current block, or the texture parameters of the first reference template can be deformed, such as enlarged or shrunk, or the format changed, to obtain the partitioning parameters of the current block.

[0374] Optionally, at least one partitioning parameter can be determined or obtained based on the texture parameters of the first reference template, the current block can be divided based on the at least one partitioning parameter to obtain at least one image region, and the first prediction result can be determined or obtained based on the at least one image region.

[0375] This method allows for the determination or acquisition of partitioning parameters based on the texture parameters of the first reference template. The current block is then divided into at least one image region based on the partitioning parameters, and the first prediction result is determined or obtained based on the at least one image region. This ensures the effective division of the current block during the prediction stage, enabling the final first prediction result to take into account different image regions. This improves the prediction accuracy of the current block and / or ensures that the decoding end can correctly obtain the optimal partitioning and corresponding prediction mode.

[0376] Method 17: Partition information of at least one of the following: neighboring blocks, non-neighboring blocks, co-occurring blocks, temporal blocks, default blocks, cross-component blocks, and candidate blocks of the current block;

[0377] Optionally, at least one of the following can be used as the reference block for the current block: neighboring block, non-neighboring block, co-occurring block, temporal block, default block, cross-component block, and candidate block.

[0378] Optionally, the partitioning information of the reference block may be: information related to the division of the image region when the reference block adopts a partitioning mode, information related to the prediction mode used by the image region divided by the reference block, the partitioning boundary line of the reference block (i.e. the partitioning line of the reference block), etc.

[0379] Optionally, the partition information of the reference block can be the partition information of the neighboring blocks of the current block, the partition information of the non-neighboring blocks of the current block, the partition information of the co-occurring blocks of the current block, the partition information of the temporal blocks of the current block, the partition information of the default block of the current block, the partition information of the cross-component blocks of the current block, or the partition information of the candidate blocks of the current block.

[0380] Optionally, the partition line of the current block can be determined or obtained based on the partition information of at least one reference block, and the current block can be divided according to the partition line to obtain at least one image region.

[0381] Optionally, a dividing line in at least one reference block can be determined, and the dividing line can be copied and moved to the current block to divide the current block and obtain at least one image region.

[0382] Alternatively, the dividing line of a neighboring or non-neighboring block can be extended through the current block to divide the current block and obtain at least one image region.

[0383] Optionally, if the partition information of at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, default blocks, cross-component blocks, and candidate blocks of the current block satisfies a condition, such as the partition information of at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, default blocks, cross-component blocks, and candidate blocks of the current block matching preset partition information, then the current block can be divided according to at least one of the first reference template of the current block, the partition parameters of the current block, at least one dividing line, the first neural network, and the first lookup table to obtain at least one image region.

[0384] Optionally, the partition information of at least one of the neighboring blocks, non-neighboring blocks, co-occurring blocks, temporal blocks, default blocks, cross-component blocks, and candidate blocks of the current block can be used as the partition parameters of the current block. Alternatively, the partition information of at least one of the neighboring blocks, non-neighboring blocks, co-occurring blocks, temporal blocks, default blocks, cross-component blocks, and candidate blocks of the current block can be transformed, such as by increasing or decreasing the size or changing the format, to obtain the partition parameters of the current block.

[0385] Optionally, at least one partitioning parameter can be determined or obtained based on the partitioning information of at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, default blocks, cross-component blocks, and candidate blocks of the current block. The current block is then divided based on the at least one partitioning parameter to obtain at least one image region. The first prediction result is then determined or obtained based on the at least one image region.

[0386] This method allows for the determination or acquisition of partitioning parameters based on at least one of the partitioning information of the current block's neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, default blocks, cross-component blocks, and candidate blocks. The current block is then divided into at least one image region based on the partitioning parameters, and a first prediction result is determined or obtained based on this at least one image region. This ensures that the prediction processing of the current block can comprehensively consider other blocks, making the final prediction result more accurate.

[0387] Method 18, the second weighting coefficient matrix and / or the second fitting coefficient matrix;

[0388] Optionally, the first weight coefficient matrix and the second weight coefficient matrix may be the same or different, and the first fitting coefficient matrix and the second fitting coefficient matrix may be the same or different.

[0389] For example, a first weight coefficient matrix and / or a first fitting coefficient matrix are determined or obtained based on a first first reference template, and a second weight coefficient matrix and / or a second fitting coefficient matrix are determined or obtained based on a second first reference template.

[0390] Optionally, the second weight coefficient matrix can be a regression matrix, and the specific determination or acquisition method is similar to that of the first weight coefficient matrix. The second weight coefficient matrix may include at least one partition weight coefficient.

[0391] Optionally, the second fitting coefficient matrix can be a regression matrix, and the specific method of determining or obtaining it is similar to that of determining or obtaining the first fitting coefficient matrix. The second fitting coefficient matrix may include at least one fitting coefficient.

[0392] Optionally, the second regression matrix will be used in place of the second weighting coefficient matrix or the second fitting coefficient matrix for the following example.

[0393] Optionally, at least one index corresponding to a second regression matrix can be determined and entered into a lookup table for searching. Based on the search result (such as the position of the element passed by the dividing line in the current block), at least one partitioning parameter can be determined or obtained. The current block can be divided according to the at least one partitioning parameter to obtain at least one image region.

[0394] Optionally, at least one first regression matrix can be input into a neural network for model training. Based on the output of the neural network, at least one partitioning parameter (such as a dividing line) passing through the current block can be determined or obtained. The current block can then be divided according to the at least one partitioning parameter to obtain at least one image region.

[0395] Optionally, if the second weight coefficient matrix or the second fitting coefficient matrix satisfies a condition, such as when there is an element position in the second weight coefficient matrix that is significantly different from the surrounding elements, the current block can be divided according to at least one of the first reference template of the current block, the partitioning parameters of the current block, at least one dividing line, the first neural network, and the first lookup table to obtain at least one image region.

[0396] Optionally, the second weighting coefficient matrix or the second fitting coefficient matrix can be used as the partitioning parameters of the current block. Alternatively, the second weighting coefficient matrix or the second fitting coefficient matrix can be transformed, such as by increasing or decreasing its size or changing its format, to obtain the partitioning parameters of the current block.

[0397] Optionally, at least one partitioning parameter can be determined or obtained based on the second weighting coefficient matrix or the second fitting coefficient matrix, the current block can be divided based on the at least one partitioning parameter to obtain at least one image region, and the first prediction result can be determined or obtained based on the at least one image region.

[0398] This method allows for the determination or acquisition of partitioning parameters based on the second weighting coefficient matrix or the second fitting coefficient matrix. The current block is then divided into at least one image region based on the partitioning parameters, and a first prediction result is determined or obtained based on the at least one image region. Furthermore, by utilizing the characteristics of the regression matrix, accurate partitioning of the current block can be achieved during the prediction stage, ensuring that the final first prediction result can take into account different image regions and improving the prediction accuracy of the current block.

[0399] Method 19, Fit coefficient;

[0400] Optionally, the fitting coefficient can be calculated by combining the fitting coefficient calculation formula with at least one element in the first reference template. The fitting coefficient calculation formula can be set in advance, such as the least squares method.

[0401] Optionally, it can be determined whether to divide the current block into regions by calculating the fitting coefficient of the first reference template. If the fitting coefficient of the first reference template is greater than a preset fitting coefficient threshold (e.g., 1), it can be determined that the current block needs to be divided. The current block can be divided according to the first reference template of the current block, the partitioning parameters of the current block, at least one dividing line, the first neural network, and at least one item in the first lookup table to obtain at least one image region.

[0402] Optionally, the fitting coefficients can be used as the partitioning parameters of the current block, or the fitting coefficients can be transformed, such as by increasing or decreasing their size, or by changing their format, to obtain the partitioning parameters of the current block.

[0403] Optionally, at least one partitioning parameter can be determined or obtained based on the fitting coefficient, the current block can be divided based on the at least one partitioning parameter to obtain at least one image region, and a first prediction result can be determined or obtained based on the at least one image region.

[0404] This method allows for the determination or acquisition of partitioning parameters based on fitting coefficients, the division of the current block into at least one image region based on these parameters, and the determination or acquisition of a first prediction result based on this at least one image region. Furthermore, by utilizing the characteristics of fitting coefficients, precise partitioning of the current block can be achieved during the prediction stage, ensuring that the final first prediction result can take into account different image regions and improving the prediction accuracy of the current block.

[0405] Method 20, partition weight coefficient;

[0406] Optionally, the first reference template of the current block can be pre-partitioned to obtain at least one first image region, and a corresponding partition weight coefficient, such as 2 or 4, can be set for each first image region.

[0407] Optionally, different partition weight coefficients can be set according to different texture parameters in each first image region. For example, a larger partition weight coefficient, such as 8, can be set for the first image region with high texture complexity, while a smaller partition weight coefficient, such as 2, can be set for the first image region with low texture complexity.

[0408] Alternatively, a smaller partition weight coefficient, such as 8, can be set for the first image region with high texture complexity, while a larger partition weight coefficient, such as 2, can be set for the first image region with low texture complexity.

[0409] Optionally, the partition weight coefficients corresponding to each first image region can be compared and detected. If the partition weight coefficient of at least one first image region is different from the partition weight coefficients of other first image regions, the current block can be divided according to the first image region to obtain at least one image region. For example, the region in the current block corresponding to the first image region can be determined and divided to obtain at least one image region.

[0410] Optionally, the partition weight coefficient can be used as the partition parameter of the current block, or the partition weight coefficient can be transformed, such as by increasing or decreasing it, or by changing its format, to obtain the partition parameter of the current block.

[0411] Optionally, at least one partitioning parameter can be determined or obtained based on the partitioning weight coefficient, the current block can be divided based on the at least one partitioning parameter to obtain at least one image region, and the first prediction result can be determined or obtained based on the at least one image region.

[0412] This method allows for the determination or acquisition of partitioning parameters based on partitioning weight coefficients, the division of the current block into at least one image region based on the partitioning parameters, and the determination or acquisition of a first prediction result based on the at least one image region. This enables precise division of the current block during the prediction stage, ensuring that the final first prediction result takes into account different image regions and improves the prediction accuracy of the current block.

[0413] Method 21, statistical information of the first reference template;

[0414] Optionally, the statistical information of the first reference template may include at least one of methods 1 to 14, or may include other parameter information of the first reference template, such as KL divergence.

[0415] Optionally, the statistical information of the first reference template can be used to determine whether there are regions with differences in the current block, such as regions with varying texture complexity (e.g., regions with elements of high texture complexity and elements with low texture complexity). If such regions exist, a dividing line can be determined or generated in the regions with varying texture complexity to divide the current block and obtain at least one image region.

[0416] Optionally, the statistical information of the first reference template can be used as the partitioning parameters of the current block, or the statistical information of the first reference template can be transformed, such as increased or decreased, or its format changed, to obtain the partitioning parameters of the current block.

[0417] Optionally, at least one partitioning parameter can be determined or obtained based on the statistical information of the first reference template, the current block can be divided based on the at least one partitioning parameter to obtain at least one image region, and the first prediction result can be determined or obtained based on the at least one image region.

[0418] This method allows for the determination or acquisition of partitioning parameters based on statistical information from a first reference template. The current block is then divided into at least one image region based on these partitioning parameters. A first prediction result is then determined or obtained based on this at least one image region. This enables precise partitioning of the current block during the prediction stage, ensuring that the final first prediction result takes into account different image regions and improves the prediction accuracy for the current block.

[0419] Method 22, a second neural network and / or a second lookup table.

[0420] Optionally, the second neural network can be a neural network.

[0421] Alternatively, the neural network can be a neural network based on fully connected layers; a neural network based on convolutional layers; a neural network based on Transformer; or a neural network based on a hybrid of convolutional and fully connected layers and Transformer, etc.

[0422] Optionally, the second lookup table can be a lookup table, such as a table, list, set, etc.

[0423] Optionally, the first reference template or the current block can be input into the second neural network to output at least one image region.

[0424] Optionally, the index corresponding to the first reference template or the current block can be determined, and the index can be entered into the second lookup table for searching. At least one image region can be determined or obtained based on the search result. For example, at least one dividing line can be determined based on the search result, and the current block can be divided according to the at least one dividing line to obtain at least one image region.

[0425] Optionally, the current block or the first reference template can be input into the second neural network, and the index corresponding to the output of the second neural network can be input into the second lookup table. At least one image region of the current block can be determined or obtained based on the lookup result of the second lookup table.

[0426] Optionally, the index corresponding to the current block or the first reference template can be input into the second lookup table for searching, and the search result of the second lookup table can be input into the second neural network. Based on the output result of the second neural network, at least one image region of the current block can be determined or obtained, such as the output result being the image region of the current block.

[0427] Alternatively, a second neural network and / or a second lookup table can be used as partitioning parameters for the current block.

[0428] Optionally, at least one partitioning parameter can be determined or obtained based on the second neural network and / or the second lookup table, the current block can be divided based on the at least one partitioning parameter to obtain at least one image region, and the first prediction result can be determined or obtained based on the at least one image region.

[0429] This method allows for the determination or acquisition of partitioning parameters based on a second neural network and / or a second lookup table. The current block is then divided into at least one image region based on these partitioning parameters. A first prediction result is then determined or obtained based on this at least one image region. This approach enables the comprehensive consideration of the advantages of the neural network and / or lookup table during the prediction stage, thereby achieving accurate partitioning of the current block. The resulting first prediction result can take into account different image regions, improving the prediction accuracy of the current block.

[0430] Optionally, in method three, the partitioning parameters of the current block can be determined or obtained according to at least one of methods 7 to 22, and the current block can be divided according to the partitioning parameters of the current block to obtain at least one image region. For example, at least one dividing line of the current block can be determined according to the partitioning parameters of the current block, and the current block can be divided according to the at least one dividing line to obtain at least one image region. The first prediction result of the current block can be determined or obtained according to the at least one image region.

[0431] In this method, the current block can be divided into at least one image region based on the partitioning parameters of the current block, and a first prediction result can be determined or obtained based on the at least one image region. This enables the effective division of the current block, so that the first prediction result obtained in the final prediction can take into account different image regions, thereby improving the prediction accuracy of the current block.

[0432] Method four: a first neural network and / or a first lookup table;

[0433] Optionally, the first neural network can be a neural network, and it can be the same as or different from the second neural network.

[0434] Optionally, the first lookup table can be a lookup table, such as a table, list, set, etc.

[0435] Optionally, the first lookup table may be the same as or different from the second lookup table.

[0436] Optionally, the first reference template or the current block can be input into the first neural network, and at least one dividing line can be output. The current block can then be divided according to the at least one dividing line to obtain at least one image region.

[0437] Optionally, the index corresponding to the first reference template or the current block can be determined, the index can be entered into the first lookup table for searching, at least one dividing line can be determined or obtained based on the search result, and the current block can be divided according to the at least one dividing line to obtain at least one image region.

[0438] Optionally, the current block or the first reference template can be input into the first neural network, and the index corresponding to the output of the first neural network can be input into the first lookup table. At least one dividing line of the current block can be determined or obtained based on the lookup result of the first lookup table, and the current block can be divided according to the at least one dividing line to obtain at least one image region.

[0439] Optionally, the index corresponding to the current block or the first reference template can be input into the first lookup table for searching, the search result of the first lookup table can be input into the first neural network, at least one dividing line of the current block can be determined or obtained based on the output result of the first neural network, and the current block can be divided according to the at least one dividing line to obtain at least one image region.

[0440] This method allows for the division of the current block into at least one image region based on a first neural network and / or a first lookup table, and the determination or acquisition of a first prediction result based on the at least one image region. This enables the comprehensive consideration of the advantages of the neural network and / or lookup table during the prediction stage, thereby achieving accurate division of the current block. The resulting first prediction result can take into account different image regions, improving the prediction accuracy of the current block.

[0441] Method 5: If the syntax elements obtained in the code stream meet the first condition, then the division method is determined or obtained according to the first division strategy.

[0442] Optionally, the first partitioning strategy may include at least one of methods one through five.

[0443] Optionally, the first condition can be any pre-set condition, such as the syntax element obtained in the bitstream being a partition signaling, or the syntax element obtained in the bitstream indicating that the current block is to be predicted using a partition prediction mode (such as the improved DIMD partition prediction mode).

[0444] Optionally, syntax elements may include tagging information such as tags and indexes. These can be tags encoded into the bitstream during encoding at the encoding end.

[0445] Alternatively, a syntax element can be a text number, a binary character, or a markup in other forms.

[0446] Optionally, when the processing device is a decoder, it can detect whether the syntax elements obtained in the bitstream meet the first condition. If the syntax elements meet the first condition, the current block can be divided into image regions using the division method determined or obtained according to the first division strategy, and the first prediction result of the current block can be determined or obtained based on at least one divided image region.

[0447] Optionally, if the syntax elements obtained in the bitstream do not meet the first condition, other methods can be used to predict the current block, such as using the ordinary DIMD mode to predict the current block.

[0448] This method allows for the division of the current block into at least one image region based on the first partitioning strategy when the syntax elements obtained from the bitstream meet the first condition. The first prediction result is then determined or obtained based on the at least one image region. This ensures that the encoding and decoding ends use the same mode for prediction processing. Furthermore, only the syntax elements obtained from the bitstream need to meet the first condition, without requiring the encoding end to upload the entire prediction process to the bitstream. This reduces signaling consumption and improves the prediction accuracy for the current block.

[0449] In this embodiment, the current block is divided using at least one of methods one to five to obtain at least one more accurate image region. This allows the current block to be divided into at least one image region during the prediction stage of video encoding and / or decoding, such as intra-frame prediction, in order to determine or obtain a first prediction result. This takes into account different image regions in the current block, making the determined or obtained prediction mode more accurate and improving the prediction accuracy for the current block. This, in turn, supports improving the prediction efficiency in the video encoding and / or decoding process.

[0450] Third Embodiment

[0451] Based on the first or second embodiment, a third embodiment is proposed.

[0452] In this embodiment, the first prediction result is determined or obtained based on at least one prediction mode of at least one image region, and / or at least one candidate prediction mode, and / or the second prediction result.

[0453] Optionally, the process of determining or obtaining the first prediction result may be illustrated by taking at least one of methods six to eight as an example.

[0454] Method 6: Determine or obtain a first prediction result based on at least one first prediction mode for at least one image region.

[0455] Optionally, the first prediction mode can be an angle prediction mode or other types of prediction modes.

[0456] Optionally, the first prediction mode can be a prediction mode used by the image region.

[0457] Optionally, the current block can be divided according to at least one of the methods one to five in the second embodiment to obtain at least one image region, and for each image region, at least one first prediction mode of the image region can be determined, and the first prediction result of the current block can be determined or obtained according to the at least one first prediction mode of the at least one image region.

[0458] Optionally, at least one first prediction mode of at least one image region can be determined, and the prediction mode of the current block can be determined or obtained based on the at least one first prediction mode of the at least one image region. The current block can be predicted based on the prediction mode of the current block to obtain a first prediction result.

[0459] Optionally, the prediction mode of the current block may include at least one first prediction mode for at least one image region.

[0460] Optionally, for each image region, at least one first prediction mode corresponding to the image region can be determined, and the image region can be predicted according to the at least one first prediction mode to obtain a prediction result. The prediction results of at least one image region are weighted and summed to obtain the first prediction result of the current block.

[0461] For example, as shown in Figure 12, the gradient histogram of the current block (such as the first gradient histogram corresponding to the first reference template) can be statistically analyzed to calculate the gradient magnitude of the prediction mode corresponding to each gradient direction in the first reference template. The mode to be used for prediction processing of the current block can be determined according to the DIMD partition signaling. For example, when the DIMD partition signaling is 0, the ordinary DIMD mode can be used, and the candidate prediction modes derived by the gradient are recorded. It is determined whether to fuse the prediction modes (i.e., whether to fuse the derived candidate prediction modes). If so, the derived candidate prediction modes are selected and fused to obtain the corresponding prediction mode, and the current block is predicted according to the prediction mode.

[0462] If the DIMD partitioning signaling is 1, the partitioned DIMD mode can be adopted, and the gradient is recorded and the optimal partition is derived to partition the current block to obtain at least one image region. Then, the gradient histogram (such as the second gradient histogram corresponding to the second reference template) is calculated for each partition (i.e., the image region). Based on the second gradient histogram, the corresponding prediction mode, such as the first prediction mode, is derived for each partition. It is then determined whether the prediction modes are fused. If so, the first prediction modes corresponding to each image region can be fused, such as by weighted summation, to obtain the prediction mode corresponding to the current block, so as to perform prediction processing on the current block and obtain the first prediction result.

[0463] For example, as shown in Figure 13, if the original image in Figure 13 is the image block to be predicted, i.e. the current block, the original reference region of the current block can be determined and used as the first reference template. Then, the operation step ① can be performed to perform the statistical calculation of the gradient histogram based on the first reference template to obtain the first gradient histogram. For example, the gradient histogram includes 2-34 and the maximum gradient magnitude is 20.

[0464] Optionally, if it is determined that the original image needs to be partitioned based on the first gradient histogram, then step ② can be executed. Based on the element (point) and direction (angle) of the maximum gradient magnitude in the first gradient histogram, a straight line is determined to divide the coding block (i.e., element image) to obtain at least one image region. After partitioning, the reference region is divided accordingly to obtain the P0 reference region and the P1 reference region. The P0 reference region is the second reference template of one image region, and the P1 reference region is the reference template of another image region. Gradient histograms can be calculated based on these two second reference templates. The gradient histogram 1 corresponding to the P0 reference region has a maximum gradient magnitude of 11, and the gradient histogram 2 corresponding to the P1 reference region has a maximum gradient magnitude of 35. Then, based on these two gradient histograms, step ③ is executed, that is, based on the new gradient histogram, the prediction modes of the two prediction regions are selected, that is, the first prediction modes corresponding to the two image regions are selected, and the prediction processing is performed based on the first prediction modes to obtain the second prediction results of the two image regions. Then, the two second prediction results are weighted and fused to obtain the first prediction result.

[0465] Optionally, referring to Figure 14, the original image is divided into two image regions, and reference regions for these two image regions, namely P0 reference region and P1 reference region, are determined. These two reference regions may be different from the original reference region of the original image. Then, a first prediction mode for these two image regions is determined based on the P0 reference region and the P1 reference region to perform prediction processing on these two image regions and obtain a second prediction result for these two image regions, such as P1 and P0. The two image regions are then weighted and fused to obtain the first prediction result.

[0466] Optionally, referring to Figure 15, a regression matrix combined with the partitioned DIMD mode can also be used to predict the current block. Optionally, the pixels of three regions (such as the original reference region, P0 prediction, and P1 prediction in the figure) can be input into the regression formula to solve for the three different partition weight coefficients a, b, and c.

[0467] Alternatively, the regression formula can be formula (VII): Rec=a(p1-p0)x+b(p1-p0)y+c(p1-p0)+p0 Formula (VII);

[0468] Optionally, Rec is the regression matrix, p1 is the element position in the P1 prediction, p0 is the element position in the P0 prediction, P1 prediction is the P1 reference region after prediction processing, P0 prediction is the P0 reference region after prediction processing, and x and y are the horizontal and vertical coordinates of the element (or predicted pixel) used to determine the position of the predicted pixel or reconstructed pixel in the current block.

[0469] Optionally, a regression matrix can be obtained by combining the solved a, b, and c with the regression formula, corresponding to the weights (i.e., the first weights) of pixel points P1 and P2. For example, the weight (i.e., the first weight) corresponding to the position of the second element from the top left corner is 0.1, and the weights of the neighboring elements of this element can include 0.5, 0.3, etc.

[0470] Optionally, the second prediction results of at least two image regions (such as P0 and P1 in the figure) can be fused with the first weight (such as weighted summation) to obtain the first prediction result.

[0471] For example, the final predicted value of the pixels in the box in the figure is P1*0.1+P0*(1-0.1).

[0472] In this method, by determining or obtaining a first prediction result based on at least one first prediction mode of at least one image region, different prediction modes can be used for different image regions, avoiding the phenomenon that all image regions use the same prediction mode, and improving the prediction accuracy of the current block.

[0473] Method 7: Determine or obtain the first prediction result based on at least one candidate prediction pattern of at least one image region.

[0474] Optionally, the candidate prediction mode can be an angle prediction mode or other types of prediction modes (such as neural network-based prediction modes).

[0475] Optionally, the candidate prediction mode can be a prediction mode to be used for the image region, such as selecting at least one prediction mode from at least one candidate prediction mode to perform prediction processing on the image region.

[0476] Optionally, at least one candidate prediction pattern for at least one image region can be obtained from at least one lookup table, or at least one candidate prediction pattern for at least one image region can be obtained by other means (such as obtaining candidate prediction patterns based on neural networks).

[0477] Optionally, for any image region, at least one candidate prediction mode for the image region can be determined, and at least one prediction mode can be selected from the at least one candidate prediction mode to perform prediction processing on the image region, such as selecting at least one candidate prediction mode with the lowest rate-distortion cost to perform prediction processing on the image region.

[0478] Optionally, the first prediction result of the current block can be determined or obtained based on the prediction result of at least one image region, for example, by weighted summation of the prediction results of at least one image region to obtain the first prediction result of the current block.

[0479] In this method, by determining or obtaining a first prediction result based on at least one candidate prediction mode of at least one image region, different candidate prediction modes can be used for different image regions, avoiding the phenomenon that all image regions use the same prediction mode, and improving the prediction accuracy of the current block.

[0480] Method 8: Determine or obtain the first prediction result based on the second prediction result of at least one image region.

[0481] Optionally, the result of predicting the image region can be used as the second prediction result.

[0482] Optionally, the second prediction results of at least one image region can be fused to obtain the first prediction result of the current block.

[0483] Optionally, the second prediction results of at least one image region can be weighted and summed to obtain the first prediction result of the current block.

[0484] For example, as shown in Figure 16, if the current block is divided into two image regions, the second prediction result of one image region is P1 and the second prediction result of the other image region is P0. P0 and P1 can be fused to obtain the first prediction result of the current block, such as a reconstructed block or a predicted block.

[0485] In this method, by determining or obtaining a first prediction result based on a second prediction result of at least one image region, it is possible to perform prediction processing on the current block from a more accurate image region level. Compared to predicting only from the block level, the first prediction result determined or obtained based on the second prediction result is more accurate.

[0486] In this embodiment, by determining or obtaining the first prediction result of the current block according to at least one of methods six to eight, it is possible to perform prediction at the image region level during the prediction stage of video encoding and / or decoding, such as intra-frame prediction, so that the first prediction result obtained is more accurate, thereby improving the prediction accuracy of the prediction processing for the current block and thus supporting the improvement of prediction efficiency in the video encoding and / or decoding process.

[0487] Fourth embodiment

[0488] A fourth embodiment is proposed based on any one of the first to third embodiments.

[0489] In this embodiment, a first prediction mode is determined or obtained according to at least one of the following methods nine to nineteen, and / or a candidate prediction mode is determined or obtained according to at least one of the following methods nine to nineteen.

[0490] Method 9: The second gradient histogram corresponding to the second reference template of at least one image region;

[0491] Alternatively, the second reference template can be approximated as the image region of the current block.

[0492] Optionally, the second reference template may be the same as or different from the first reference template.

[0493] Optionally, after dividing the current block into at least one image region, a second reference template for at least one image region can be determined, and gradient calculation can be performed on each element in the second reference template to construct a gradient histogram, which is then used as the second gradient histogram.

[0494] Optionally, the angle prediction mode can be determined based on the gradient direction corresponding to each position in the second gradient histogram, with each gradient direction corresponding to an angle prediction mode.

[0495] Optionally, a defined angular prediction pattern can be used as the defined first prediction pattern and / or candidate prediction pattern.

[0496] Optionally, for each element in the image region, a prediction process can be performed based on the first prediction mode corresponding to each element to obtain a second prediction result for the image region, and the second prediction results can be weighted and summed to obtain the first prediction result for the current block.

[0497] Optionally, the candidate prediction modes corresponding to the image region can be stored in a list to obtain a candidate prediction mode list. The candidate prediction mode list includes at least one candidate prediction mode, such as at least one directional (e.g., diagonal) angle prediction mode.

[0498] Optionally, at least one candidate prediction mode with the lowest rate-distortion cost can be selected from the candidate prediction mode list to perform prediction processing on the image region, thereby obtaining a second prediction result for the image region. The second prediction results are then weighted and summed to obtain a first prediction result for the current block.

[0499] In this method, a first prediction mode and / or a candidate prediction mode are determined or obtained based on the second gradient histogram of a second reference template of at least one image region. This allows for the accurate acquisition of the first prediction mode or the candidate prediction mode that the image region may adopt by utilizing the characteristics of the second gradient histogram.

[0500] Method 10: At least one second reference template corresponding to statistical feature information;

[0501] Optionally, the statistical feature information corresponding to the second reference template may include the gradient information of each element in the second reference template, such as the gradient magnitude, gradient direction, gradient magnitude in the horizontal direction, and gradient magnitude in the vertical direction of each element.

[0502] Optionally, at least one angle prediction pattern can be determined or obtained from the statistical feature information corresponding to at least one second reference template, and can be used as a candidate prediction pattern and / or a first prediction pattern.

[0503] Optionally, the gradient direction corresponding to each element in the statistical feature information corresponding to the second reference template can be determined, and the angle prediction mode corresponding to each element in the second reference template can be determined according to the gradient direction. For example, one gradient direction corresponds to one angle prediction mode. Different gradient directions correspond to different angle prediction modes, and the determined angle prediction mode can be used as a candidate prediction mode and / or the first prediction mode.

[0504] Optionally, at least one candidate prediction mode can be selected from at least one candidate prediction mode determined or obtained based on at least one second reference template to perform prediction processing on at least one image region, thereby obtaining a second prediction result for the image region. Alternatively, at least one image region can be predicted based on a first prediction mode determined or obtained based on at least one second reference template, thereby obtaining a second prediction result. Furthermore, the second prediction results of at least one image region can be fused to obtain a first prediction result for the current block.

[0505] In this approach, a first prediction mode and / or a candidate prediction mode are determined or obtained by statistical feature information corresponding to at least one second reference template, thereby enabling the accurate acquisition of the first prediction mode or the candidate prediction mode that the image region may adopt by utilizing statistical feature information.

[0506] Method 11: At least one image region and / or at least one of the following: neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, default blocks, cross-component blocks, and candidate blocks of the current block;

[0507] Optionally, the neighboring blocks of the current block may include at least one neighboring block of an image region, and the neighboring block of the image region may be a block that is adjacent to the location of the image region.

[0508] Optionally, the prediction mode adopted by at least one image region and / or the neighboring blocks of the current block can be used as the first prediction mode and / or candidate prediction mode of at least one image region.

[0509] Optionally, the non-neighbor blocks of the current block may include non-neighbor blocks of the image region, which may be blocks that are not adjacent to the location of the image region.

[0510] Optionally, the prediction mode adopted by at least one image region and / or the non-neighboring blocks of the current block can be used as the first prediction mode and / or candidate prediction mode of at least one image region.

[0511] Optionally, the co-location block of the current block may include a co-location block of an image region. The co-location block of the image region may be an image block in the co-location image that has the same position and size as the image region of the current block. Optionally, the co-location image may be the image in the reference image that is closest to the current image in time.

[0512] Optionally, the prediction mode adopted by at least one image region and / or the co-occurring block of the current block can be used as the first prediction mode and / or candidate prediction mode for at least one image region.

[0513] Optionally, the temporal block of the current block may include a temporal block of the image region, which may be a block distinguished in the time domain.

[0514] Optionally, the prediction mode adopted by at least one image region and / or the temporal block of the current block can be used as the first prediction mode and / or candidate prediction mode for at least one image region.

[0515] Optionally, the default block of the image region can be the same as or different from the default block of the current block. For example, the size parameter of the default block of the current block may be larger than the size parameter of the default block of the image region.

[0516] Optionally, the prediction mode adopted by at least one image region and / or the default block of the current block can be used as the first prediction mode and / or candidate prediction mode for at least one image region.

[0517] Optionally, the cross-component block of the image region can be the same as or different from the cross-component block of the current block; no restriction is imposed here.

[0518] Optionally, the prediction mode adopted by at least one image region and / or the cross-component block of the current block can be used as the first prediction mode and / or candidate prediction mode for at least one image region.

[0519] Optionally, the candidate blocks of the image region can be the same as or different from the candidate blocks of the current block; no restriction is imposed here.

[0520] Optionally, the prediction mode adopted by at least one image region and / or the candidate block of the current block can be used as the first prediction mode and / or candidate prediction mode of at least one image region.

[0521] Optionally, at least one first prediction mode can be determined or obtained based on at least one image region and / or at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, default blocks, cross-component blocks, and candidate blocks of the current block, and the image region can be predicted based on the at least one first prediction mode to obtain a second prediction result. Alternatively, at least one candidate prediction mode can be determined or obtained based on at least one image region and / or at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, default blocks, cross-component blocks, and candidate blocks of the current block, and at least one of the at least one candidate prediction modes can be selected as a prediction mode to predict the image region to obtain a second prediction result. The second prediction result of the at least one image region can be fused to obtain a first prediction result of the current block.

[0522] In this method, a first prediction mode and / or a candidate prediction mode are determined or obtained based on at least one of the following: neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, default blocks, cross-component blocks, and candidate blocks of at least one image region and / or the current block. This enables the reference to other blocks when performing prediction processing on the image region, making the final determined or obtained first prediction mode and / or candidate prediction mode more accurate.

[0523] Method 12, at least one second reference template texture parameters;

[0524] Optionally, the texture parameters of the second reference template may include texture complexity, texture features, texture changes at each element position, etc.

[0525] Optionally, at least one prediction mode, such as an angle prediction mode, can be determined or obtained based on the texture parameters of at least one second reference template, and used as the first prediction mode or a candidate prediction mode.

[0526] Optionally, the texture parameters of at least one image region can be determined based on the texture parameters of at least one second reference template, and at least one prediction mode of at least one image region can be determined based on the texture parameters of at least one image region, and used as a first prediction mode or a candidate prediction mode.

[0527] Optionally, the texture parameters corresponding to each image region can be different, and different texture parameters can be used in different quantities and different types of prediction modes.

[0528] For example, if the current block is divided into two image regions with different texture complexities, the image region with higher texture complexity can use at least two different prediction modes (such as a planar mode and a vertical angle prediction mode), while the image region with lower texture complexity can use at least one angle prediction mode (such as an approximately 45-degree angle prediction mode (the prediction direction can be from the lower left to the upper right)). Furthermore, the angle prediction mode used by the image region with higher texture complexity can be different from the angle prediction mode used by the image region with lower texture complexity.

[0529] Optionally, at least one first prediction mode can be determined or obtained based on the texture parameters of at least one second reference template, and prediction processing can be performed on at least one image region based on the at least one first prediction mode to obtain a second prediction result. Alternatively, at least one candidate prediction mode can be determined or obtained based on the texture parameters of at least one second reference template, and at least one candidate prediction mode can be selected from the at least one candidate prediction mode to perform prediction processing on at least one image region to obtain a second prediction result. Furthermore, the second prediction result of at least one image region can be fused to obtain a first prediction result for the current block.

[0530] In this approach, by determining or obtaining a first prediction mode and / or a candidate prediction mode based on the texture parameters of at least one second reference template, it is possible to comprehensively consider the texture parameters when performing prediction processing on an image region, thereby avoiding the phenomenon that the prediction effect deteriorates due to the interference of the texture parameters.

[0531] Method 13: Size and / or position parameters of at least one image region and / or a second reference template;

[0532] Optionally, the size parameters of the image region may include the width, height, perimeter, area, etc. of the image region.

[0533] Optionally, the position parameter of the image region may include the pixel position of the image region in the current block, such as the upper left corner region.

[0534] Optionally, the dimensional parameters of the second reference template may include the width, height, perimeter, area, etc. of the second reference template.

[0535] Optionally, the position parameters of the second reference template may include the pixel position of the second reference template relative to the current block, such as above the current block, to the left of the current block, etc.

[0536] Optionally, at least one prediction mode, such as an angle prediction mode, can be determined or obtained based on the size parameters of at least one image region and / or a second reference template, and used as the first prediction mode or a candidate prediction mode.

[0537] For example, if the aspect ratio of at least one image region and / or the second reference template is greater than a preset aspect ratio threshold (e.g., 1 / 2), then at least one angle prediction mode (e.g., the selected approximately 45-degree angle prediction mode) can be selected from at least one preset angle prediction mode (e.g., 65 angle prediction modes, such as the approximately 45-degree angle prediction mode, the approximately 135-degree angle prediction mode, etc.) as the first prediction mode or candidate prediction mode.

[0538] Optionally, at least one prediction mode, such as an angle prediction mode, can be determined or obtained based on the position parameters of at least one image region and / or a second reference template, and used as the first prediction mode or a candidate prediction mode.

[0539] For example, if at least one image region is located in the upper left corner of the current block, at least one angle prediction mode can be selected as the first prediction mode or a candidate prediction mode from at least one preset angle prediction mode (such as 65 angle prediction modes, such as 45-degree angle prediction mode, 30-degree angle prediction mode, etc.).

[0540] For example, if the second reference template is located to the left of the current block, at least one prediction mode can be selected as the first prediction mode or a candidate prediction mode from at least one preset prediction mode (such as DC mode, plane mode and 65 angle prediction modes (such as the approximately 45-degree angle prediction mode)).

[0541] Optionally, at least one first prediction mode can be determined or obtained based on the size parameters and / or position parameters of at least one image region and / or the second reference template, and the at least one image region can be predicted based on the at least one first prediction mode to obtain a second prediction result. Alternatively, at least one candidate prediction mode can be determined or obtained based on the size parameters and / or position parameters of at least one image region and / or the second reference template, and at least one candidate prediction mode can be selected from the at least one candidate prediction mode to predict the at least one image region to obtain a second prediction result. The second prediction result of the at least one image region can be fused to obtain a first prediction result for the current block.

[0542] In this method, by determining or obtaining a first prediction mode and / or candidate prediction mode based on the size parameters and / or position parameters of at least one image region and / or a second reference template, it is possible to comprehensively consider the corresponding size parameters and / or position parameters when performing prediction processing on the image region, so that the first prediction result obtained in the final prediction is more accurate.

[0543] Method 14: At least one of the following: template feature information, gradient information, residual fluctuation, structural feature parameters, edge structure description information, structural tensor, phase consistency, and statistical information of at least one second reference template;

[0544] Optionally, the template feature information of the second reference template can be related feature information of the second reference template, or it can be a type of feature information calculated based on the template feature operator of the second reference template. For example, it can be the gradient information of elements at different positions in the second reference template, or the texture complexity information of different regions in the second reference template.

[0545] Optionally, at least one prediction mode, such as an angle prediction mode, can be determined or obtained based on the template feature information of at least one second reference template, and used as the first prediction mode or a candidate prediction mode.

[0546] Optionally, when the template feature information of at least one second reference template meets certain conditions, such as the presence of a texture change model in the second reference template, or elements with gradient magnitude changes, at least one angle prediction mode (such as the approximately 45-degree angle prediction mode) can be selected as the first prediction mode or candidate prediction mode from at least one preset prediction mode (such as DC mode, planar mode and 65 angle prediction modes (such as the approximately 45-degree angle prediction mode)).

[0547] Optionally, the gradient information of the second reference template can be obtained through the second gradient histogram of the second reference template, which may include the gradient magnitude, gradient direction, angle, etc. of each element in the second reference template.

[0548] Optionally, at least one prediction mode, such as an angle prediction mode, can be determined or obtained based on the gradient information of at least one second reference template, and used as the first prediction mode or a candidate prediction mode.

[0549] Optionally, when the gradient information of at least one second reference template meets certain conditions, such as the existence of elements with gradient magnitude changes in the second reference template, at least one prediction mode can be selected as the first prediction mode or candidate prediction mode from at least one preset prediction mode (such as DC mode, plane mode and 65 angle prediction modes (such as the approximately 45-degree angle prediction mode)).

[0550] Optionally, the residual fluctuation of the second reference template can be similar to that of the first reference template.

[0551] Optionally, the closer the value of at least one residual element in the residual block corresponding to the second reference template is to 0, the smaller the corresponding residual fluctuation, and vice versa.

[0552] Optionally, at least one prediction model, such as an angle prediction model, can be determined or obtained based on the residual fluctuation of at least one second reference template, and used as the first prediction model or a candidate prediction model.

[0553] Optionally, when the residual fluctuation of at least one second reference template meets certain conditions, such as the residual fluctuation of the second reference template being large, at least one prediction mode (e.g., DC mode) can be selected as the first prediction mode or candidate prediction mode from at least one preset prediction mode (e.g., DC mode, plane mode and 65 angle prediction modes (e.g., angle prediction mode of about 45 degrees)).

[0554] Optionally, the structural tensor of the second reference template can be a symmetric matrix, a second-order matrix, or an inertial tensor, which can be calculated based on the gradient information of the second reference template. The specific calculation and determination process of the structural tensor of the second reference template can refer to the specific calculation and determination process of the structural tensor of the first reference template in method 10 of the second embodiment.

[0555] Optionally, at least one prediction mode, such as an angle prediction mode, can be determined or obtained based on the structural tensor of at least one second reference template, and used as the first prediction mode or a candidate prediction mode.

[0556] Optionally, when the structure tensor of at least one second reference template satisfies certain conditions, such as the existence of at least two eigenvalues ​​greater than 0 in the structure tensor of the second reference template, at least one prediction mode (e.g., the plane mode) can be selected as the first prediction mode or candidate prediction mode from at least one preset prediction mode (e.g., DC mode, plane mode and 65 angle prediction modes (e.g., the angle prediction mode of about 45 degrees)).

[0557] Optionally, for any element in the second reference template, a set of multi-scale, different-direction filters (typically Log-Gabor) can be used to calculate the phase of each pixel under these filters, and the phase consistency of the second reference template can be evaluated based on the phase of each pixel under these filters.

[0558] Optionally, at least one prediction mode, such as an angle prediction mode, can be determined or obtained based on the phase consistency of at least one second reference template, and used as the first prediction mode or a candidate prediction mode.

[0559] Optionally, when the phase consistency of at least one second reference template meets certain conditions, such as the phase being consistent in all directions / scales at any element position of the second reference template, at least one angle prediction mode (such as the angle prediction mode of approximately 45 degrees) can be selected as the first prediction mode or candidate prediction mode from at least one preset prediction mode (such as DC mode, planar mode and 65 angle prediction modes (such as the angle prediction mode of approximately 45 degrees)).

[0560] Optionally, the structural feature parameters of the second reference template may include the structural tensor, phase consistency, and second derivative information of the second reference template.

[0561] Optionally, at least one prediction mode, such as an angle prediction mode, can be determined or obtained based on the structural feature parameters of at least one second reference template, and used as the first prediction mode or a candidate prediction mode.

[0562] Optionally, when the structural feature parameters of at least one second reference template meet certain conditions, such as the phase of each direction / scale being consistent at any element position of the second reference template, at least one prediction mode (e.g., DC mode) can be selected as the first prediction mode or candidate prediction mode from at least one preset prediction mode (e.g., DC mode, planar mode and 65 angle prediction modes (e.g., an angle prediction mode of about 45 degrees)).

[0563] Optionally, the edge structure description information of the second reference template may include residual fluctuations, phase consistency, etc. of the second reference template.

[0564] Optionally, at least one prediction mode, such as an angle prediction mode, can be determined or obtained based on the edge structure description information of at least one second reference template, and used as the first prediction mode or a candidate prediction mode.

[0565] Optionally, when the structural characteristic parameters of at least one second reference template meet certain conditions, such as the residual fluctuation of the second reference template being large, and the phase of each direction / scale being consistent at any element position of the second reference template, at least one prediction mode (e.g., DC mode) can be selected as the first prediction mode or candidate prediction mode from at least one preset prediction mode (e.g., DC mode, planar mode, and 65 angle prediction modes (e.g., an angle prediction mode of about 45 degrees)).

[0566] Optionally, the statistical information of the second reference template may include at least one of methods nine to thirteen, and may also include at least one of the template feature information, gradient information, residual fluctuation, structural feature parameters, edge structure description information, structural tensor, and phase consistency of at least one second reference template, and may also include the variance and / or mean of at least one element in at least one second reference template, and may also include at least one of the fitting coefficients, partition weight coefficients, and third weight coefficient matrix or third fitting coefficient matrix corresponding to at least one second reference template, etc.

[0567] Optionally, at least one prediction model, such as an angle prediction model, can be determined or obtained based on statistical information from at least one second reference template, and used as the first prediction model or a candidate prediction model.

[0568] Optionally, when the statistical information of at least one second reference template meets certain conditions, such as the residual fluctuation of the second reference template being greater than a preset residual fluctuation threshold (e.g., 5) and the corresponding width-to-height ratio being greater than (1 / 2), at least one prediction mode (e.g., the plane mode) can be selected as the first prediction mode or a candidate prediction mode from at least one preset prediction mode (e.g., DC mode, plane mode, and 65 angle prediction modes (e.g., an angle prediction mode of about 45 degrees)).

[0569] Optionally, at least one first prediction mode can be determined or obtained based on at least one of the template feature information, gradient information, residual fluctuation, structural feature parameters, edge structure description information, structural tensor, phase consistency, and statistical information of at least one second reference template. The prediction processing is then performed on at least one image region based on the at least one first prediction mode to obtain a second prediction result. Alternatively, at least one candidate prediction mode can be determined or obtained based on at least one of the template feature information, gradient information, residual fluctuation, structural feature parameters, edge structure description information, structural tensor, phase consistency, and statistical information of at least one second reference template. The prediction processing is then performed on at least one candidate prediction mode to obtain a second prediction result. Finally, the second prediction results of at least one image region can be fused to obtain a first prediction result for the current block.

[0570] In this approach, a first prediction mode and / or a candidate prediction mode are determined or obtained based on at least one of the template feature information, gradient information, residual fluctuation, structural feature parameters, edge structure description information, structural tensor, phase consistency, and statistical information of at least one second reference template. This enables the acquisition of the corresponding prediction mode based on the relevant information of the second reference template when performing prediction processing on an image region, thus avoiding the phenomenon that the prediction mode cannot be accurately obtained due to the inability to obtain the relevant information of the current block.

[0571] Method 15: The variance and / or mean of at least one element in at least one second reference template;

[0572] Optionally, at least one prediction pattern, such as an angle prediction pattern, can be determined or obtained based on the variance of at least one element in at least one second reference template, and used as the first prediction pattern or a candidate prediction pattern.

[0573] Optionally, when the variance of at least one element in at least one second reference template meets certain conditions, such as the variance of at least one element in the second reference template being greater than a preset variance threshold (e.g., 5), at least one prediction mode (e.g., the approximately 45-degree angle prediction mode) can be selected as the first prediction mode or a candidate prediction mode from at least one preset prediction mode (e.g., DC mode, plane mode, and 65 angle prediction modes (e.g., the approximately 45-degree angle prediction mode)).

[0574] Optionally, at least one prediction pattern, such as an angle prediction pattern, can be determined or obtained based on the mean of at least one element in at least one second reference template, and used as the first prediction pattern or a candidate prediction pattern.

[0575] Optionally, when the mean of at least one element in at least one second reference template meets certain conditions, such as the mean of at least one element in the second reference template being greater than a preset mean threshold (e.g., 5), at least one prediction mode (e.g., DC mode) can be selected as the first prediction mode or candidate prediction mode from at least one preset prediction mode (e.g., DC mode, planar mode, and 65 angle prediction modes (e.g., an angle prediction mode of about 45 degrees)).

[0576] Optionally, at least one first prediction mode can be determined or obtained based on the variance and / or mean of at least one element in at least one second reference template, and at least one image region can be predicted based on the at least one first prediction mode to obtain a second prediction result. Alternatively, at least one candidate prediction mode can be determined or obtained based on the variance and / or mean of at least one element in at least one second reference template, and at least one candidate prediction mode can be selected from the at least one candidate prediction mode to predict at least one image region to obtain a second prediction result. Furthermore, the second prediction results of at least one image region can be fused to obtain a first prediction result for the current block.

[0577] In this method, a first prediction mode and / or candidate prediction mode are determined or obtained based on the variance and / or mean of at least one element in at least one second reference template. This enables the acquisition of the corresponding prediction mode based on the relevant information of the second reference template when performing prediction processing on an image region, thus avoiding the phenomenon that the prediction mode cannot be accurately obtained due to the inability to obtain the relevant information of the current block.

[0578] Method 16: At least one of the following: fitting coefficients, partition weight coefficients, third weight coefficient matrix, and third fitting coefficient matrix corresponding to a second reference template;

[0579] Optionally, the fitting coefficient of the second reference template can be the fitting coefficient of at least one image region, and the fitting coefficient of the second reference template can be a positive integer.

[0580] Optionally, the fitting coefficient can be calculated by combining the fitting coefficient calculation formula with at least one element in the second reference template. The fitting coefficient calculation formula can be set in advance, such as the least squares method.

[0581] Optionally, at least one prediction mode, such as an angle prediction mode, can be determined or obtained based on the fitting coefficients corresponding to at least one second reference template, and used as the first prediction mode or a candidate prediction mode.

[0582] Optionally, when the fitting coefficient corresponding to at least one second reference template meets certain conditions, such as the fitting coefficient corresponding to at least one second reference template being greater than a preset threshold (e.g., 5), at least one prediction mode (e.g., the plane mode) can be selected as the first prediction mode or candidate prediction mode from at least one preset prediction mode (e.g., DC mode, plane mode, and 65 angle prediction modes (e.g., the angle prediction mode of about 45 degrees)).

[0583] Optionally, the partition weight coefficient corresponding to the second reference template can be the partition weight coefficient of at least one image region. The partition weight coefficient can be set in advance or obtained from a lookup table.

[0584] Optionally, at least one prediction mode, such as an angle prediction mode, can be determined or obtained based on the partition weight coefficients corresponding to at least one second reference template, and used as the first prediction mode or a candidate prediction mode.

[0585] Optionally, when the partition weight coefficient corresponding to at least one second reference template meets certain conditions, such as the partition weight coefficient corresponding to at least one second reference template being greater than a preset threshold (e.g., 8), at least one prediction mode (e.g., DC mode) can be selected as the first prediction mode or candidate prediction mode from at least one preset prediction mode, such as DC mode, plane mode and 65 angle prediction modes (e.g., angle prediction mode of about 45 degrees).

[0586] Optionally, the third weight coefficient matrix and / or the third fitting coefficient matrix corresponding to the second reference template can be a regression matrix, which can be obtained by performing regression calculation on the second reference template of the current block, or by inputting each element of the second reference template into a pre-set linear regression function for calculation.

[0587] Optionally, the third weight coefficient matrix may include fitting coefficients corresponding to at least one second reference template, and the third fitting coefficient matrix may include partition weight coefficients corresponding to at least one second reference template.

[0588] Optionally, at least one prediction mode, such as an angle prediction mode, can be determined or obtained based on the third weight coefficient matrix and / or the third fitting coefficient matrix corresponding to at least one second reference template, and used as the first prediction mode or a candidate prediction mode.

[0589] Optionally, when the third weight coefficient matrix and / or the third fitting coefficient matrix corresponding to at least one second reference template meets certain conditions, such as when there are elements in the third weight coefficient matrix and / or the third fitting coefficient matrix that are significantly different from the surrounding elements (e.g., different element values), at least one prediction mode (e.g., the approximately 45-degree angle prediction mode) can be selected as the first prediction mode or a candidate prediction mode from at least one preset prediction mode (e.g., DC mode, planar mode, and 65 angle prediction modes (e.g., the approximately 45-degree angle prediction mode)).

[0590] Optionally, at least one first prediction mode can be determined or obtained based on at least one of the fitting coefficients, partition weight coefficients, third weight coefficient matrix, and third fitting coefficient matrix corresponding to at least one second reference template, and at least one image region can be predicted based on the at least one first prediction mode to obtain a second prediction result. Alternatively, at least one candidate prediction mode can be determined or obtained based on at least one of the fitting coefficients, partition weight coefficients, third weight coefficient matrix, and third fitting coefficient matrix corresponding to at least one second reference template, and at least one candidate prediction mode can be selected from the at least one candidate prediction mode to predict at least one image region to obtain a second prediction result. Furthermore, the second prediction results of at least one image region can be fused to obtain a first prediction result for the current block.

[0591] In this method, a first prediction mode and / or a candidate prediction mode are determined or obtained based on at least one of the fitting coefficients, partition weight coefficients, third weight coefficient matrix and third fitting coefficient matrix corresponding to at least one second reference template. This enables the acquisition of the corresponding prediction mode based on the relevant information of the second reference template when performing prediction processing on an image region, thus avoiding the phenomenon that the prediction mode cannot be accurately obtained due to the inability to obtain the relevant information of the current block.

[0592] Method 17: At least one third neural network and / or a third lookup table;

[0593] Optionally, the third neural network may be the same as or different from the first or second neural network, and may be a type of neural network or a partial structure of a neural network, such as a 3x3 convolutional layer.

[0594] Optionally, the third lookup table can be a lookup table, such as a table, list, set, etc.

[0595] Optionally, a second reference template or at least one image region, or at least one of methods nine to sixteen, can be input into a third neural network to output at least one image region.

[0596] Optionally, an index corresponding to a second reference template or at least an image region can be determined, and the index can be entered into a third lookup table for searching. Based on the search result, at least one prediction mode, such as an angle prediction mode, can be determined or obtained, and it can be used as the first prediction mode or a candidate prediction mode.

[0597] Optionally, a second reference template or at least one image region can be input into a third neural network, and the index corresponding to the output of the third neural network can be input into a third lookup table. At least one prediction mode can be determined or obtained based on the lookup result of the third lookup table, and it can be used as the first prediction mode or a candidate prediction mode.

[0598] Optionally, the index corresponding to the second reference template or at least one image region can be input into the third lookup table for searching, and the search result of the third lookup table can be input into the third neural network. At least one prediction mode can be determined or obtained based on the output result of the third neural network, and it can be used as the first prediction mode or candidate prediction mode.

[0599] Optionally, a first prediction mode can be determined or obtained based on at least one third neural network and / or a third lookup table, and a prediction processing can be performed on at least one image region based on the at least one first prediction mode to obtain a second prediction result. Alternatively, at least one candidate prediction mode can be determined or obtained based on at least one third neural network and / or a third lookup table, and at least one candidate prediction mode can be selected from the at least one candidate prediction mode to perform prediction processing on at least one image region to obtain a second prediction result. Furthermore, the second prediction results of at least one image region can be fused to obtain a first prediction result for the current block.

[0600] In this approach, a first prediction pattern and / or candidate prediction pattern are determined or obtained based on at least one third neural network and / or a third lookup table. This allows the advantages of neural networks (such as greater accuracy) and / or lookup tables (such as lower complexity) to be utilized to obtain accurate and effective first prediction patterns and / or candidate prediction patterns.

[0601] Method 18: If the syntax elements obtained in the code stream meet the second condition, then the prediction pattern is determined or obtained according to the first pattern determination strategy.

[0602] Optionally, the second condition can be any pre-set condition, which can be the same as or different from the first condition. For example, the syntax element may contain partition signaling and special marking information, such as the character 'a'.

[0603] Optionally, the first mode determination strategy may be at least one of methods nine to seventeen, and the prediction mode determined or obtained according to at least one of methods nine to seventeen, such as the angle prediction mode, may be used as the prediction mode determined or obtained according to the first mode determination strategy, and the prediction mode determined or obtained according to the first mode determination strategy may be used as the first prediction mode or the candidate prediction mode.

[0604] Optionally, if the syntax elements obtained in the bitstream satisfy the second condition, then at least one image region is predicted according to the prediction mode determined or obtained by the first mode determination strategy to obtain a second prediction result, and the second prediction result of at least one image region can be fused to obtain the first prediction result of the current block.

[0605] In this approach, when the syntax elements obtained from the bitstream satisfy the second condition, the prediction mode determined or obtained according to the first mode determination strategy can be used to obtain the corresponding first prediction mode and / or candidate prediction mode. This ensures that the encoding and decoding ends can use the same prediction mode for prediction processing, thereby improving the prediction effect.

[0606] Method 19: If the syntax elements obtained in the code stream do not meet the second condition, then the prediction mode is determined or obtained according to the second mode determination strategy.

[0607] Optionally, the second mode determination strategy is different from the first mode determination strategy, and the second mode determination strategy can be at least one of the methods nine to seventeen. The prediction mode determined or obtained according to at least one of the methods nine to seventeen, such as the angle prediction mode, can be used as the prediction mode determined or obtained according to the first mode determination strategy. The prediction mode determined or obtained according to the first mode determination strategy can be used as the first prediction mode or the candidate prediction mode.

[0608] Optionally, if the syntax elements obtained in the bitstream do not meet the second condition, then the prediction mode determined or obtained according to the first mode determination strategy is used to perform prediction processing on at least one image region to obtain a second prediction result, and the second prediction result of at least one image region can be fused to obtain the first prediction result of the current block.

[0609] In this approach, when the syntax elements obtained from the bitstream do not meet the second condition, the prediction mode determined or obtained according to the second mode determination strategy can be used to obtain the corresponding first prediction mode and / or candidate prediction mode. This ensures that the encoding and decoding ends can use the same prediction mode for prediction processing, thereby improving the prediction effect.

[0610] In this embodiment, a first prediction mode and / or a candidate prediction mode are determined or obtained through at least one of methods nine to nineteen, and at least one image region can be predicted based on at least one first prediction mode and / or candidate prediction mode to determine or obtain the first prediction result of the current block based on the prediction result of at least one image region. Thus, during the prediction stage of video encoding and / or decoding, such as intra-frame prediction, the corresponding prediction mode can be determined from the image region level to make the first prediction result more accurate, thereby improving the prediction accuracy of the prediction processing for the current block and thus supporting the improvement of prediction efficiency in the video encoding and / or decoding process.

[0611] Fifth Embodiment

[0612] A fifth embodiment is proposed based on any one of the first to fourth embodiments.

[0613] In this embodiment, the first reference template is determined or obtained according to at least one of the following methods 20 to 28, and / or the second reference template is determined or obtained according to at least one of the following methods 20 to 28.

[0614] Method 20: Refer to the template above the current block;

[0615] Optionally, the neighboring block above the current block can be used as the upper reference template of the current block, or a non-neighboring block that is not above the current block can be used as the upper reference template of the current block, or both the neighboring block and the non-neighboring block of the current block can be used as the upper reference template of the current block.

[0616] For example, the reference template of the area above the current block, as shown in Figure 17, can be used as the upper reference template of the current block.

[0617] Optionally, the reference template above the current block can be used as the first reference template, and the steps involving the first reference template in methods one to five of the second embodiment can be executed.

[0618] Optionally, the reference template above the current block can be used as the second reference template, and at least one first prediction mode and / or candidate prediction mode of at least one image region can be determined or obtained based on the second reference template.

[0619] In this approach, by determining or obtaining the first reference template and / or the second reference template of the current block based on the upper reference template of the current block, it is possible to ensure the effective subsequent segmentation of the current block based on the first reference template of the current block, and the effective determination of the first prediction mode and / or candidate prediction mode based on the second reference template, thereby ensuring the effective intra-frame prediction of the current block.

[0620] Method 21: Refer to the template to the left of the current block;

[0621] Optionally, the neighboring block adjacent to the left side of the current block can be used as the left reference template of the current block, or the non-neighboring block not adjacent to the left side of the current block can be used as the left reference template of the current block, or the neighboring block and non-neighboring block of the current block can be used as the left reference template of the current block.

[0622] For example, the reference template of the left region of the current block, as shown in Figure 17, can be used as the left reference template of the current block.

[0623] Optionally, the left-side reference template of the current block can be used as the first reference template, and the steps involving the first reference template in methods one to five of the second embodiment can be executed.

[0624] Optionally, the left reference template of the current block can be used as the second reference template, and at least one first prediction mode and / or candidate prediction mode of at least one image region can be determined or obtained based on the second reference template.

[0625] In this approach, by determining or obtaining the first reference template and / or the second reference template of the current block based on the left reference template of the current block, it is possible to ensure the effective subsequent segmentation of the current block based on the first reference template of the current block, and the effective determination of the first prediction mode and / or candidate prediction mode based on the second reference template, thereby ensuring the effective intra-frame prediction of the current block.

[0626] Method 22: Refer to the template at the top left of the current block;

[0627] Optionally, the neighboring block adjacent to the upper left of the current block can be used as the upper left reference template of the current block, or the non-neighboring block that is not adjacent to the upper left of the current block can be used as the upper left reference template of the current block, or the neighboring block and non-neighboring block of the current block can be used as the upper left reference template of the current block.

[0628] For example, the reference template of the upper left region of the current block, as shown in Figure 17, can be used as the upper left reference template of the current block.

[0629] Optionally, the upper left reference template of the current block can be used as the first reference template, and the steps involving the first reference template in methods one to five of the second embodiment can be executed.

[0630] Optionally, the upper left reference template of the current block can be used as the second reference template, and at least one first prediction mode and / or candidate prediction mode of at least one image region can be determined or obtained based on the second reference template.

[0631] In this approach, by determining or obtaining the first reference template and / or the second reference template of the current block based on the upper left reference template of the current block, it is possible to ensure the effective subsequent segmentation of the current block based on the first reference template of the current block, and the effective determination of the first prediction mode and / or candidate prediction mode based on the second reference template, thus ensuring the effective intra-frame prediction of the current block.

[0632] Method 23: Determine or obtain the image block through the motion vector of the current block;

[0633] Optionally, motion vector calculations can be performed on at least one element in the current block, its neighboring blocks, or non-neighboring blocks, and an image block can be determined or obtained based on the motion vector calculation results, such as the image block pointed to by the motion vector calculation results.

[0634] Optionally, an image block determined or obtained by the motion vector of the current block can be used as a first reference template, and the method steps involving the first reference template in methods one to five of the second embodiment can be performed.

[0635] Optionally, an image block determined or obtained by the motion vector of the current block can be used as a second reference template, and at least one first prediction mode and / or candidate prediction mode of at least one image region can be determined or obtained based on the second reference template.

[0636] In this method, by determining or obtaining the first reference template and / or the second reference template of the current block based on the image block determined or obtained by the motion vector of the current block, it is possible to ensure the effective subsequent segmentation of the current block based on the first reference template and the determination of the first prediction mode and / or candidate prediction mode based on the second reference template, thus ensuring the effective intra-frame prediction of the current block.

[0637] Method 24: Determine or obtain the image block through the block vector of the current block;

[0638] Optionally, block vector calculation can be performed on at least one element in the current block or its neighboring or non-neighboring blocks, and the image block can be determined or obtained based on the block vector calculation result, such as the image block pointed to by the block vector calculation result.

[0639] Optionally, an image block determined or obtained by the block vector of the current block can be used as a first reference template, and the method steps involving the first reference template in methods one to five of the second embodiment can be performed.

[0640] Optionally, an image block determined or obtained by the block vector of the current block can be used as a second reference template, and at least one first prediction mode and / or candidate prediction mode of at least one image region can be determined or obtained based on the second reference template.

[0641] In this method, by determining or obtaining the first reference template and / or the second reference template of the current block based on the image block determined or obtained by the block vector of the current block, it is possible to ensure the effective subsequent division of the current block based on the first reference template and the determination of the first prediction mode and / or candidate prediction mode based on the second reference template, thus ensuring the effective intra-frame prediction of the current block.

[0642] Method 25: Current block size parameters;

[0643] Optionally, a first reference template and / or a second reference template for the current block can be determined or obtained based on the size parameters of the current block.

[0644] For example, if the size parameter of the current block is greater than the size parameter threshold (e.g., the width-to-height ratio threshold is 1 / 2), then the reference template above the current block is selected as the first reference template, and the reference template to the left of the current block is selected as the second reference template.

[0645] Optionally, a first reference template can be determined or obtained based on the size parameters of the current block, and the method steps involving the first reference template in methods one to five of the second embodiment can be executed.

[0646] Optionally, a second reference template can be determined or obtained based on the size parameters of the current block, and at least one first prediction mode and / or candidate prediction mode of at least one image region can be determined or obtained based on the second reference template.

[0647] In this method, by determining or obtaining the first reference template and / or the second reference template of the current block based on the size parameters of the current block, it is possible to ensure the effective subsequent division of the current block based on the first reference template and the determination of the first prediction mode and / or candidate prediction mode based on the second reference template, thus ensuring the effective intra-frame prediction of the current block.

[0648] Method 26, Template Instruction Information;

[0649] Optionally, template indication information obtained in the bitstream can be determined, and the first reference template and / or the second reference template of the current block can be determined based on the template indication information.

[0650] Optionally, if the template indication information indicates a reference template to the left of the current block, it can be used as the first reference template and / or the second reference template of the current block; if the template indication information indicates a reference template above the current block, it can be used as the first reference template and / or the second reference template of the current block; if the template indication information indicates a reference template to the upper left of the current block, it can be used as the first reference template and / or the second reference template of the current block.

[0651] For example, if the template indication information indicates that the reference template to the left of the current block is the first reference template and the reference template above the current block is the second reference template, then the reference template to the left of the current block can be used as the first reference template, and the method steps involving the first reference template in methods one to five of the second embodiment can be executed; the reference template above the current block can be used as the second reference template, and at least one first prediction mode and / or candidate prediction mode of at least one image region can be determined or obtained based on the second reference template.

[0652] In this method, by determining or obtaining the first reference template and / or the second reference template of the current block based on the template indication information, it is possible to ensure the effective subsequent partitioning of the current block based on the first reference template and the determination of the first prediction mode and / or candidate prediction mode based on the second reference template, thereby ensuring the effective intra-frame prediction of the current block.

[0653] Method 27: Matching Information;

[0654] Optionally, at least one reference template can be set in advance, and the matching information obtained by matching the current block with at least one reference template can be used as the first reference template. The method steps involving the first reference template in the first to fifth methods of the second embodiment can be executed.

[0655] Optionally, at least one reference template set in advance may include the reference template to the left of the current block, the reference template above the current block, the reference template to the upper left of the current block, etc.

[0656] Optionally, at least one reference template can be set in advance, and matching information obtained by matching at least one image region with at least one reference template can be obtained. The reference template that matches at least one image region can be used as a second reference template, and at least one first prediction mode and / or candidate prediction mode of at least one image region can be determined or obtained based on the second reference template.

[0657] In this approach, by determining or obtaining the first reference template and / or the second reference template of the current block based on the matching information, it is possible to ensure the effective subsequent segmentation of the current block based on the first reference template and the determination of the first prediction mode and / or candidate prediction mode based on the second reference template, thereby ensuring the effective intra-frame prediction of the current block.

[0658] Method 28: Rate distortion cost.

[0659] Optionally, at least one reference template can be set in advance. For example, at least one reference template can be determined or obtained according to any one of the methods 20 to 27. Rate distortion cost is calculated for each reference template, and the reference template with the lowest cost is selected as the first reference template and / or the second reference template.

[0660] In this approach, by determining or obtaining the first reference template and / or the second reference template of the current block based on the rate-distortion cost, it is possible to ensure the effective subsequent segmentation of the current block based on the first reference template and the determination of the first prediction mode and / or candidate prediction mode based on the second reference template, thereby ensuring the effective intra-frame prediction of the current block.

[0661] In this embodiment, by determining or obtaining the first reference template and / or the second reference template through at least one of the methods 20 to 28, it is possible to ensure the effective subsequent partitioning of the current block based on the first reference template and the determination of the first prediction mode and / or candidate prediction mode based on the second reference template, thereby ensuring the effective intra-frame prediction of the current block.

[0662] Sixth Embodiment

[0663] A sixth embodiment is proposed based on any one of the first to fifth embodiments.

[0664] In this embodiment, the image processing method further includes at least one of the following methods 29 to 31.

[0665] Method 29: The second prediction result is determined or obtained based on a second prediction mode that is a weighted sum of at least one first prediction mode and / or candidate prediction modes, and at least one image region.

[0666] Optionally, for any image region, at least one first prediction mode can be determined or obtained according to any one of the methods nine to nineteen in the fourth embodiment, and the at least one first prediction mode can be weighted and summed to obtain a second prediction mode. Then, the image region can be predicted according to the second prediction mode to obtain a second prediction result.

[0667] For example, the second prediction mode to be used for a certain image region includes the weighted sum of two angle prediction modes, and the two angle prediction modes can use the same or different weights, such as the weight of the first angle prediction mode being 2 and the weight of the other angle prediction mode being 8.

[0668] Optionally, for any image region, at least one candidate prediction mode can be determined or obtained according to any one of methods nine to nineteen in the fourth embodiment. At least two candidate prediction modes are then selected from the at least one candidate prediction mode and weighted together to obtain a second prediction mode. The weights corresponding to the selected at least two candidate prediction modes can be the same or different. The image region is then predicted according to the second prediction mode to obtain a second prediction result.

[0669] In this approach, the validity of the determined or obtained second prediction result can be guaranteed by determining or obtaining the second prediction result based on a second prediction mode that weights and sums at least one first prediction mode and / or candidate prediction modes, and at least one image region.

[0670] Method 30: Determine or obtain the first prediction result based on at least one of at least one second prediction result, at least one first weight, and at least one non-angle prediction result; Optionally, when the at least one second prediction result is represented in matrix form, the first weight can also be represented in matrix form, such as a weight matrix, and each element in the weight matrix corresponds one-to-one with each element in the matrix representing the second prediction result in matrix form.

[0671] Optionally, when at least one second prediction result is expressed in data or numerical form, the first weight can also be expressed in data or numerical form.

[0672] Optionally, the first weight can be represented as an integer during the setting phase and as a decimal in the range of 0-1 during the calculation phase. For example, when setting the first weight, 2 to the power of n can be set as the first weight, where n is zero or a positive integer. During the calculation phase, it can be reduced to the range of 0-1 for calculation.

[0673] For example, if the weight matrix of the first weight is set to 2 to the power of 3, then the elements in the weight matrix are integers from 0 to 8. When using the first weight for calculation, all elements in the weight matrix are uniformly divided by 8 to obtain a weight matrix in the range of 0 to 1, and then the corresponding calculation is performed, such as performing a weighted fusion operation based on at least one second prediction result and at least one first weight.

[0674] Optionally, at least one second prediction result for at least one image region can be determined, and at least one second prediction result for at least one image region can be fused to obtain a first prediction result for the current block. Alternatively, at least one second prediction result can be input into a lookup table for searching to obtain the first prediction result.

[0675] Optionally, at least one first weight corresponding to at least one image region can be determined, such as 8, and the first prediction result of the current block can be determined or obtained based on the at least one first weight, such as by inputting the index corresponding to the at least one first weight into a lookup table for searching, and obtaining the first prediction result.

[0676] Optionally, the prediction result of the non-angle prediction mode corresponding to the current block can be determined and used as the non-angle prediction result. The non-angle prediction mode can be Planar mode, DC mode, etc.

[0677] Optionally, the first prediction result of the current block can be determined or obtained based on the non-angle prediction result corresponding to the current block.

[0678] Optionally, a first weight (e.g., 4) corresponding to at least one second prediction result and a first weight (e.g., 1) corresponding to a non-angle prediction result can be determined. Based on the at least one second prediction result, the first weight corresponding to the at least one second prediction result, the non-angle prediction result, and the first weight corresponding to the non-angle prediction result, a weighted sum is performed to obtain the first prediction result.

[0679] In this method, the validity of the determined or obtained first prediction result can be guaranteed by determining or obtaining the first prediction result based on at least one of at least one second prediction result, at least one first weighted and non-angle prediction result.

[0680] Optionally, the image processing method further includes at least one of the following methods 23 to 30.

[0681] Method 23: Determine or obtain the first weight based on the gradient information of at least one image region;

[0682] Optionally, the gradient information of at least one image region can be obtained from the second gradient histogram corresponding to at least one image region, such as the gradient area and gradient magnitude.

[0683] Optionally, at least one first prediction mode or candidate prediction mode, such as an angle prediction mode, can be determined based on the gradient information of at least one image region. The gradient magnitude value corresponding to each angle prediction mode can be determined, and a different first weight can be set for each angle prediction mode according to the magnitude of the gradient magnitude value. For example, the larger the gradient magnitude value, the larger the first weight.

[0684] Optionally, the ratio of the amplitudes of at least two candidate prediction modes can be used as the first weight.

[0685] Optionally, the ratio between any two gradient areas in at least one image region can be used as the first weight, or the ratio between any two gradient magnitude values ​​in at least one image region can be used as the first weight.

[0686] Optionally, the gradient area corresponding to each element in at least one image region can be determined, and different first weights can be set for different gradient areas in at least one image region. For example, the larger the gradient area, the larger the first weight corresponding to the prediction mode adopted by the element.

[0687] Optionally, the gradient magnitude corresponding to each element in at least one image region can be determined, and different first weights can be set for different gradient magnitudes in at least one image region. For example, the larger the gradient magnitude value, the larger the first weight corresponding to the prediction mode adopted by the element.

[0688] Optionally, a fixed first weight can be determined based on the gradient information. For example, if the gradient magnitude values ​​in the gradient information are all no greater than a preset gradient magnitude threshold (e.g., 685), then the first weight can be determined to be a fixed weight value, such as 8.

[0689] Optionally, at least one first weight can be determined or obtained according to method 23, and the first prediction result can be determined or obtained according to at least one of the at least one first weight, at least one second prediction result, and non-angle prediction result.

[0690] In this method, a first weight is determined or obtained based on the gradient information of at least one image region. This allows the determined or obtained first weight to take into account the gradient information and to be indirectly associated with the first prediction mode or candidate prediction mode based on the gradient information. In other words, the first weight determined or obtained through the gradient information can be associated with the second prediction result, thereby improving the accuracy of the determined or obtained first weight.

[0691] Method 24: Determine or obtain the first weight based on the fusion boundary and / or bending boundary in different directions;

[0692] Optionally, the division boundaries of the image regions obtained by dividing the current block can be determined, such as the location of the dividing line, and the division boundaries of at least two adjacent image regions can be determined. When fusing the second prediction results of at least two image regions, the second prediction results of the division boundaries of at least two adjacent image regions need to be fused. In order to obtain the fusion results at the division boundaries of the image regions more accurately, the second prediction results fused at the division boundaries can be optimized according to the set first weight to obtain a more accurate first weight.

[0693] Optionally, the boundary of the division after fusing (e.g., weighted summation) the second prediction results of at least two image regions can be used as the fusion boundary.

[0694] Optionally, the prediction results of the boundary division using different prediction directions can be fused to obtain the prediction results of the fused boundary.

[0695] Optionally, a new first weight can be set separately for the fusion boundary. The first weight of the fusion boundary is relatively large, such as 8.

[0696] Optionally, the first weight corresponding to the fusion boundary can be determined or obtained based on the gradient information of the image region, or it can be determined or obtained in other ways, such as a lookup table, etc., without any restrictions.

[0697] Optionally, since the texture complexity of the current block is unevenly distributed, a curved dividing line can be used to divide the current block to obtain at least one image region, so that regions with high texture complexity are in the same image region as much as possible, and regions with low texture complexity are in the same image region as much as possible, and the curved dividing line can be used as a curved boundary.

[0698] Optionally, the curved boundary of the current block can be determined or obtained based on the texture features of the current block, and the start and end points of the curved boundary can be on the dividing line of the current block.

[0699] For example, as shown in Figure 18, if the dividing line is the diagonal of the current block, the current block can be divided according to the diagonal to obtain two image regions. The diagonal in Figure 18 can be used as the dividing boundary of the image region, and the semi-circular dashed line in Figure 18 can be used as the curvature boundary of the current block.

[0700] Optionally, a separate first weight can be set for the region where the curved boundary is located.

[0701] Optionally, as shown in Figure 18, if the current block is divided into two image regions (such as the upper image region and the lower image region) by using the diagonal of the current block as the dividing line, then different first weights can be set for these two image regions. The way or rule for setting the first weights for these two image regions can be the same or different.

[0702] Optionally, the method or rule for determining the first weight in region B of Figure 18 can be the same as that in region A, or different from that in region C.

[0703] For example, if the lower half of the image region in Figure 18 includes region A, and the upper half of the image region includes regions B and C, then regions A and B can both use the same first weight, such as 8, while region C can use a different first weight, such as 2.

[0704] Alternatively, regions A, B, and C may each use a different first weight.

[0705] At least one first weight can be determined or obtained according to method 24, and the first prediction result can be determined or obtained according to at least one of the at least one first weight, at least one second prediction result and non-angle prediction result.

[0706] In this approach, by determining or obtaining the first weight based on the fusion boundary and / or curvature boundary in different directions, it is possible to comprehensively consider the fusion boundary and / or curvature boundary of the image region when determining the first weight, thereby avoiding the phenomenon that the prediction effect of the current block deteriorates due to the influence of the boundary.

[0707] Method 25: In at least one image region, the smaller the distance between the elements of a certain region and the fusion boundary, the larger the first weight corresponding to the elements of that certain region; and the smaller the distance between the elements of another region and the fusion boundary, the smaller the first weight corresponding to the elements of that other region.

[0708] Alternatively, the elements in the image region can be pixels.

[0709] Optionally, at least one first weight may be determined or obtained according to at least one of methods 23 to 24, and the determined or obtained first weight needs to comply with the rule requirements of method 25.

[0710] Optionally, the centroid formula can be used to calculate the distance from the position of at least one element in at least one image region to the fusion boundary, and different first weights can be set according to the distance, such as the smaller the distance, the larger the first weight, or the larger the distance, the smaller the first weight.

[0711] Alternatively, at least one first weight satisfying mode 25 can be determined or obtained based on a lookup table or neural network.

[0712] For example, as shown in Figure 19, in image region 1, the diagonal is the fusion boundary. The smaller the distance between any element (pixel to be predicted or reconstructed) and the fusion boundary in the lower half of the fusion boundary, the larger the first weight corresponding to that element. Conversely, the smaller the distance between any element and the fusion boundary in the upper half of the fusion boundary, the smaller the first weight corresponding to that element. Similarly, in image region 2, the diagonal is the fusion boundary. The smaller the distance between any element (pixel to be predicted or reconstructed) and the fusion boundary in the lower half of the fusion boundary, the smaller the first weight corresponding to that element. Conversely, the smaller the distance between any element and the fusion boundary in the upper half of the fusion boundary, the larger the first weight corresponding to that element. The second prediction results of at least one element in image region 1 and at least one element in image region 2 can be weighted and fused to obtain the first prediction result for the current block, such as a reconstructed block or a predicted block.

[0713] Optionally, at least one first weight can be determined or obtained according to method 25, and the first prediction result can be determined or obtained according to at least one of the at least one first weight, at least one second prediction result, and non-angle prediction result.

[0714] In this method, when the first weight satisfies the requirements of method 25, different elements can be predicted using different weights, which improves the accuracy of the first prediction result determined or obtained based on at least one of the following: at least one second prediction result, at least one first weight, and at least one non-angle prediction result.

[0715] Method 26: In at least one image region, the smaller the distance between the elements of a partial region and the curved boundary, the larger the first weight corresponding to the elements of the partial region; and the smaller the distance between the elements of another partial region and the curved boundary, the smaller the first weight corresponding to the elements of the other partial region.

[0716] Optionally, at least one first weight may be determined or obtained according to at least one of methods 23 to 24, and the determined or obtained first weight needs to comply with the rule requirements of method 26.

[0717] Optionally, the centroid formula can be used to calculate the distance from the position of at least one element in at least one image region to the curved boundary, and different first weights can be set according to the distance, such as the smaller the distance, the larger the first weight, or the larger the distance, the smaller the first weight.

[0718] Alternatively, at least one first weight satisfying mode 26 can be determined or obtained based on a lookup table or neural network.

[0719] Optionally, at least one first weight can be determined or obtained according to method 26, and the first prediction result can be determined or obtained according to at least one of the at least one first weight, at least one second prediction result, and non-angle prediction result.

[0720] In this method, when the first weight satisfies the requirements of method 26, different elements can be predicted using different weights, which improves the accuracy of the first prediction result determined or obtained based on at least one of the following: at least one second prediction result, at least one first weight, and at least one non-angle prediction result.

[0721] Method 27: The first weights corresponding to the same element position are different in at least two image regions;

[0722] Optionally, the first weight corresponding to the same element position in at least two image regions can be set to the same value, such as 2, or they can be set to different values, such as the first weight corresponding to the element at the top left corner of one image region being 4, and the first weight corresponding to the element at the top left corner of another image region being 8.

[0723] Optionally, at least one first weight may be determined or obtained according to at least one of methods 23 to 24, and the determined or obtained first weight needs to comply with the rule requirements of method 27.

[0724] Alternatively, at least one first weight satisfying mode 27 can be determined or obtained based on a lookup table or neural network.

[0725] Optionally, at least one first weight can be determined or obtained according to method 27, and the first prediction result can be determined or obtained according to at least one of the at least one first weight, at least one second prediction result, and non-angle prediction result.

[0726] In this method, when the first weight satisfies the requirements of method 27, different elements can be predicted using different weights, which improves the accuracy of the first prediction result determined or obtained based on at least one of the following: at least one second prediction result, at least one first weight, and at least one non-angle prediction result.

[0727] Method 28: At least two image regions, where the first weight of elements in a subset of at least one image region is greater than the first weight of elements in a subset of at least another image region;

[0728] Optionally, in at least two image regions, the first weight corresponding to an element at any position in at least one image region is greater than the first weight corresponding to an element at the same position in at least another image region. For example, the first weight corresponding to the element at the top left corner of one image region is 8, and the first weight corresponding to the element at the top left corner of another image region is 4.

[0729] For example, if the current block is divided into two image regions, the filtering results of the two image regions are fused. The weights within the fusion width region (including the fusion boundary region of the two image regions) can follow the rule of method 40, that is, the first weight corresponding to at least one element within the fusion width region is greater than the first weight corresponding to the other element, and the first weight outside the fusion width region is 0 or 1.

[0730] Optionally, at least one first weight may be determined or obtained according to at least one of methods 23 to 24, and the determined or obtained first weight needs to comply with the rule requirements of method 28.

[0731] Alternatively, at least one first weight satisfying mode 28 can be determined or obtained based on a lookup table or neural network.

[0732] Optionally, at least one first weight can be determined or obtained according to method 28, and the first prediction result can be determined or obtained according to at least one of the at least one first weight, at least one second prediction result, and non-angle prediction result.

[0733] In this method, when the first weight satisfies the requirements of method 28, different elements can be predicted using different weights, which improves the accuracy of the first prediction result determined or obtained based on at least one of the following: at least one second prediction result, at least one first weight, and at least one non-angle prediction result.

[0734] Method 29: In at least one image region, the first weight corresponding to the elements of a portion of the region is greater than the first weight corresponding to the elements of another portion of the region.

[0735] Optionally, the at least two first weights corresponding to different element positions in at least one image region can be different or the same.

[0736] Optionally, at least one first weight may be determined or obtained according to at least one of methods 23 to 24, and the determined or obtained first weight needs to comply with the rule requirements of method 29.

[0737] Alternatively, at least one first weight satisfying mode 29 can be determined or obtained based on a lookup table or neural network.

[0738] Optionally, at least one first weight can be determined or obtained according to method 29, and the first prediction result can be determined or obtained according to at least one of the at least one first weight, at least one second prediction result, and non-angle prediction result.

[0739] In this method, when the first weight satisfies the requirements of method 29, different elements can be predicted using different weights, which improves the accuracy of the first prediction result determined or obtained based on at least one of the following: at least one second prediction result, at least one first weight, and at least one non-angle prediction result.

[0740] Method 30: In at least one image region, the smaller the distance between the elements of a partial region and the dividing boundary, the larger the first weight corresponding to the elements of that partial region; and the smaller the distance between the elements of another partial region and the dividing boundary, the smaller the first weight corresponding to the elements of that other partial region.

[0741] Optionally, the centroid formula can be used to calculate the distance from the position of at least one element in at least one image region to the dividing boundary, and different first weights can be set according to the distance, such as the smaller the distance, the larger the first weight, or the larger the distance, the smaller the first weight.

[0742] For example, as shown in Figure 19, in image region 1, the diagonal line serves as the dividing boundary. The smaller the distance between any element (pixel to be predicted or reconstructed) and the dividing boundary in the lower half of the dividing boundary, the larger its corresponding first weight. Conversely, the smaller the distance between any element and the dividing boundary in the upper half of the dividing boundary, the smaller its corresponding first weight. Similarly, in image region 2, the diagonal line serves as the dividing boundary. The smaller the distance between any element (pixel to be predicted or reconstructed) and the dividing boundary in the lower half of the dividing boundary, the smaller its corresponding first weight. The smaller the distance between any element and the dividing boundary in the upper half of the dividing boundary, the larger its corresponding first weight. The second prediction results of at least one element in image region 1 and at least one element in image region 2 can be weighted and fused to obtain the first prediction result for the current block, such as a reconstructed block or a predicted block.

[0743] Optionally, at least one first weight may be determined or obtained according to at least one of methods 23 to 24, and the determined or obtained first weight needs to comply with the rule requirements of method 30.

[0744] Alternatively, at least one first weight satisfying mode 30 can be determined or obtained based on a lookup table or neural network.

[0745] Optionally, at least one first weight can be determined or obtained according to method 30, and the first prediction result can be determined or obtained according to at least one of the at least one first weight, at least one second prediction result, and non-angle prediction result.

[0746] In this method, when the first weight satisfies the requirements of method 30, different elements can be predicted using different weights, which improves the accuracy of the first prediction result determined or obtained based on at least one of the following: at least one second prediction result, at least one first weight, and at least one non-angle prediction result.

[0747] Method 31: Determine or obtain the first prediction result based on at least one of the following: fitting coefficient, partition weight coefficient, function formula, fourth neural network and fourth lookup table, and at least one second prediction result.

[0748] Optionally, the fourth neural network can be a neural network, and the fourth lookup table can be a lookup table, such as a table, list, set, etc.

[0749] Optionally, the function formula of the regression matrix can be determined or obtained based on the fitting coefficient and / or partition weight coefficient, and at least one second prediction result can be input into the function formula for calculation to obtain the first prediction result.

[0750] Optionally, at least one second prediction result can be input into a fourth neural network for model training, and the first prediction result can be output.

[0751] Optionally, at least one index corresponding to the second prediction result can be input into the fourth lookup table for searching, and the first prediction result can be output.

[0752] In this method, the first prediction result is determined or obtained based on at least one of the fitting coefficient, partition weight coefficient, function formula, fourth neural network and fourth lookup table, as well as at least one second prediction result. In this way, the advantages of the parameters such as neural network, lookup table and function formula can be used to make the first prediction result obtained in the final prediction more accurate.

[0753] Optionally, the current block can be divided according to at least one of the methods one to five in the second embodiment to obtain at least one image region. A first prediction mode or candidate prediction mode of at least one image region can be determined or obtained according to at least one of the methods nine to nineteen in the fourth embodiment. A second prediction result can be obtained by predicting at least one image region according to at least one first prediction mode or at least one selected candidate prediction mode.

[0754] Optionally, the first prediction result can be determined or obtained based on the second prediction result and at least one of the first weight and non-angle prediction results determined or obtained according to at least one of the methods 23 to 30 in the sixth embodiment.

[0755] Optionally, the first prediction result can be determined or obtained by combining the second prediction result of method thirty-one and at least one image region.

[0756] In this embodiment, by performing corresponding image processing according to at least one of methods 29 to 31, it is possible to consider the second prediction result of at least one image region of the current block during the prediction stage of video encoding and / or decoding, such as intra-frame prediction, in order to determine or obtain the first prediction result. This takes into account different image regions in the current block, making the determined or obtained prediction mode more accurate, improving the prediction accuracy of prediction processing for the current block, and thus supporting the improvement of prediction efficiency in the video encoding and / or decoding process.

[0757] Referring to Figure 20, this application embodiment also provides an image processing apparatus, which includes:

[0758] Processing module A10 is used to determine or obtain a first prediction result based on at least one image region divided in the current block.

[0759] Optionally, the current block may be divided based on at least one of the following:

[0760] The first reference template of the current block; at least one dividing line; the partitioning parameters of the current block; a first neural network and / or a first lookup table; if the syntax elements obtained in the bitstream satisfy the first condition, then the partitioning method is determined or obtained according to the first partitioning strategy.

[0761] Optionally, at least one dividing line is determined or obtained based on at least one of the following:

[0762] The gradient magnitude and / or gradient direction of at least one element in the first reference template; the position of the element with the largest gradient magnitude and the direction perpendicular to the direction of the largest gradient in the first reference template; the first gradient histogram and / or statistical feature information corresponding to the first reference template; the first reference template, and at least one fitting coefficient and / or partition weight coefficient; at least one first weight coefficient matrix or first fitting coefficient matrix corresponding to the first reference template; at least one of the following: neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, default blocks, cross-component blocks, and candidate blocks of the current block.

[0763] Optionally, the partitioning parameters are determined or obtained based on at least one of the following:

[0764] Template feature information of the first reference template; gradient information of the first reference template; residual fluctuation of the first reference template; structure tensor of the first reference template; phase consistency of the first reference template; second derivative information of the first reference template; structural feature parameters of the first reference template; edge structure description information of the first reference template; variance and / or mean of at least one element in the first reference template; texture parameters of the first reference template; partitioning information of at least one of the following: neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, default blocks, cross-component blocks, and candidate blocks of the current block; second weighting coefficient matrix or second fitting coefficient matrix; fitting coefficients; partitioning weighting coefficients; statistical information of the first reference template; second neural network and / or second lookup table.

[0765] Optionally, the first prediction result is determined or obtained based on at least one of the following for at least one image region:

[0766] At least one first prediction pattern; at least one candidate prediction pattern; and a second prediction result.

[0767] Optionally, a first prediction model and / or candidate prediction models are determined or obtained based on at least one of the following:

[0768] The following are considered as a whole: a second gradient histogram corresponding to a second reference template of at least one image region; statistical feature information corresponding to at least one second reference template; at least one of the following: neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, default blocks, cross-component blocks, and candidate blocks of at least one image region and / or the current block; texture parameters of at least one second reference template; size parameters and / or position parameters of at least one image region and / or the second reference template; at least one of the following: template feature information, gradient information, residual fluctuation, structural feature parameters, edge structure description information, structural tensor, phase consistency, and statistical information of at least one second reference template; variance and / or mean of at least one element in at least one second reference template; at least one of the following: fitting coefficient, partition weight coefficient, third weight coefficient matrix, and third fitting coefficient matrix corresponding to at least one second reference template; at least one third neural network and / or third lookup table; if the syntax elements obtained in the bitstream satisfy the second condition, then the prediction mode is determined or obtained according to the first mode determination strategy; if the syntax elements obtained in the bitstream do not satisfy the second condition, then the prediction mode is determined or obtained according to the second mode determination strategy.

[0769] Optionally, the first reference template and / or the second reference template are determined or obtained according to at least one of the following:

[0770] The reference template above the current block; the reference template to the left of the current block; the reference template to the upper left of the current block; the image block determined or obtained by the motion vector of the current block; the image block determined or obtained by the block vector of the current block; the size parameters of the current block; template indication information; matching information; rate distortion cost.

[0771] Optionally, processing module A10 is configured to perform at least one of the following:

[0772] The second prediction result is determined or obtained based on a second prediction mode that is a weighted sum of at least one first prediction mode and / or candidate prediction modes, and at least one image region.

[0773] The first prediction result is determined or obtained based on at least one of the following: at least one second prediction result, at least one first weighted and non-angle prediction result;

[0774] The first prediction result is determined or obtained based on at least one of the fitting coefficient, partition weight coefficient, function formula, fourth neural network and fourth lookup table, and at least one second prediction result.

[0775] Optionally, processing module A10 is configured to perform at least one of the following:

[0776] The first weight is determined or obtained based on the gradient information of at least one image region;

[0777] The first weight is determined or obtained based on the fusion boundary and / or bending boundary in different directions;

[0778] In at least one image region, the smaller the distance between the elements in a certain region and the fusion boundary, the larger the first weight corresponding to the elements in that region; the smaller the distance between the elements in another region and the fusion boundary, the smaller the first weight corresponding to the elements in that other region.

[0779] In at least one image region, the smaller the distance between the elements in a certain region and the curved boundary, the larger the first weight corresponding to the elements in that region; the smaller the distance between the elements in another region and the curved boundary, the smaller the first weight corresponding to the elements in that other region.

[0780] At least two image regions have different first weights corresponding to the same element position;

[0781] At least two image regions, wherein the first weight corresponding to the elements of a portion of at least one image region is greater than the first weight corresponding to the elements of a portion of at least another image region;

[0782] In at least one image region, the first weight corresponding to elements in a portion of the region is greater than the first weight corresponding to elements in another portion of the region.

[0783] In at least one image region, the smaller the distance between the elements of a certain region and the dividing boundary, the larger the first weight corresponding to the elements of that certain region; and the smaller the distance between the elements of another region and the dividing boundary, the smaller the first weight corresponding to the elements of that other region.

[0784] The image processing apparatus provided in this application embodiment is similar in implementation principle and beneficial effect to the corresponding method embodiment described above, and will not be repeated here.

[0785] This application also provides a processing device, including a memory and a processor. The memory stores an image processing program, and when the image processing program is executed by the processor, it implements the steps of the image processing method in any of the above embodiments.

[0786] This application also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the image processing method in any of the above embodiments.

[0787] In the embodiments of the processing device and storage medium provided in this application, all the technical features of any of the above-described image processing method embodiments may be included. The extended and explanatory content of the specification is basically the same as that of the embodiments of the above methods, and will not be repeated here.

[0788] This application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to perform the image processing methods described in the various possible implementations above.

[0789] This application also provides a chip, including a memory and a processor. The memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that a device with the chip installed performs the image processing methods as described in the various possible implementations above.

[0790] It is understood that the above scenarios are merely examples and do not constitute a limitation on the application scenarios of the technical solutions provided in the embodiments of this application. The technical solutions of this application can also be applied to other scenarios. For example, those skilled in the art will know that with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems. The sequence numbers of the embodiments of this application above are merely for description and do not represent the superiority or inferiority of the embodiments. The steps in the method of the embodiments of this application can be adjusted, merged, and deleted according to actual needs. The units in the device of the embodiments of this application can be merged, divided, and deleted according to actual needs.

[0791] In this application, the same or similar terms, concepts, technical solutions and / or application scenario descriptions are generally described in detail only when they appear for the first time. When they appear again, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of this application, the same or similar terms, concepts, technical solutions and / or application scenario descriptions that are not described in detail later can be referred to their previous relevant detailed descriptions.

[0792] In this application, the descriptions of the various embodiments have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0793] The technical features of the present application can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present application.

[0794] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, controlled terminal, or network device, etc.) to execute the methods of each embodiment of this application.

[0795] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. A computer program product includes one or at least one computer instruction. When the computer program instruction is loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instruction can be stored in a storage medium or transmitted from one storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, storage disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0796] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An image processing method, wherein, Including the following steps: S10, determine or obtain a first prediction result based on at least one image region divided in the current block.

2. The method as described in claim 1, wherein, The current block is divided according to at least one of the following: The first reference template for the current block; At least one dividing line; The partition parameters for the current block; First neural network and / or first lookup table; If the syntax elements obtained from the bitstream satisfy the first condition, then the division method is determined or obtained according to the first division strategy.

3. The method as described in claim 2, wherein, At least one dividing line is determined or obtained based on at least one of the following: The gradient magnitude and / or gradient direction of at least one element within the first reference template; The position of at least one element with the maximum gradient magnitude within the first reference template and the direction perpendicular to the direction of the maximum gradient; The first gradient histogram and / or statistical feature information corresponding to the first reference template; A first reference template, and at least one fitting coefficient and / or partition weight coefficient; At least one first weight coefficient matrix and / or a first fitting coefficient matrix corresponding to the first reference template; The current block is at least one of the following: neighboring block, non-neighboring block, co-occurring block, temporal block, default block, cross-component block, and candidate block.

4. The method of claim 2, wherein, The partitioning parameters are determined or obtained based on at least one of the following: Template feature information of the first reference template; Gradient information of the first reference template; Residual fluctuations in the first reference template; The structure tensor of the first reference template; Phase consistency of the first reference template; Second derivative information of the first reference template; Structural characteristic parameters of the first reference template; Edge structure description information of the first reference template; The variance and / or mean of at least one element in the first reference template; Texture parameters of the first reference template; Partition information of at least one of the following: neighboring blocks, non-neighboring blocks, co-occurring blocks, temporal blocks, default blocks, cross-component blocks, and candidate blocks of the current block; Second weighting coefficient matrix and / or second fitting coefficient matrix; Fit coefficients; Partition weighting coefficients; Statistical information for the first reference template; A second neural network and / or a second lookup table.

5. The method of claim 1, wherein, The first prediction result is determined or obtained based on at least one of the following from at least one image region: At least one first prediction pattern; At least one candidate prediction pattern; Second prediction result.

6. The method of claim 5, wherein, The first prediction model and / or candidate prediction models are determined or obtained based on at least one of the following: The second gradient histogram corresponding to the second reference template of at least one image region; At least one second reference template corresponding to statistical feature information; At least one image region and / or at least one of the following: neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, default blocks, cross-component blocks, and candidate blocks of the current block; Texture parameters of at least one second reference template; At least one image region and / or a second reference template, including size and / or position parameters; At least one of the following: template feature information, gradient information, residual fluctuation, structural feature parameters, edge structure description information, structural tensor, phase consistency, and statistical information of at least one second reference template; The variance and / or mean of at least one element in at least one second reference template; At least one of the following: the fitting coefficients, partition weight coefficients, third weight coefficient matrix, and third fitting coefficient matrix corresponding to at least one second reference template; At least one third neural network and / or a third lookup table; If the syntax elements obtained in the bitstream satisfy the second condition, then the prediction pattern is determined or obtained according to the first pattern determination strategy. If the syntax elements obtained from the bitstream do not meet the second condition, then the prediction pattern is determined or obtained according to the second pattern determination strategy.

7. The method of claim 6, wherein, The first reference template and / or the second reference template are determined or obtained based on at least one of the following: The reference template above the current block; The left-hand reference template for the current block; The upper left reference template of the current block; The image block is determined or obtained by the motion vector of the current block; Image blocks determined or obtained through the block vector of the current block; Current block size parameters; Template instruction information; Matching information; Rate distortion costs.

8. The method of claim 5, wherein, It also includes at least one of the following: The second prediction result is determined or obtained based on a second prediction mode that is a weighted sum of at least one first prediction mode and / or candidate prediction modes, and at least one image region. The first prediction result is determined or obtained based on at least one of the following: at least one second prediction result, at least one first weighted and non-angle prediction result; The first prediction result is determined or obtained based on at least one of the fitting coefficient, partition weight coefficient, function formula, fourth neural network and fourth lookup table, and at least one second prediction result.

9. The method of claim 8, wherein, It also includes at least one of the following: The first weight is determined or obtained based on the gradient information of at least one image region; The first weight is determined or obtained based on the fusion boundary and / or bending boundary in different directions; In at least one image region, the smaller the distance between the elements in a certain region and the fusion boundary, the larger the first weight corresponding to the elements in that region; the smaller the distance between the elements in another region and the fusion boundary, the smaller the first weight corresponding to the elements in that other region. In at least one image region, the smaller the distance between the elements in a certain region and the curved boundary, the larger the first weight corresponding to the elements in that region; the smaller the distance between the elements in another region and the curved boundary, the smaller the first weight corresponding to the elements in that other region. At least two image regions have different first weights corresponding to the same element position; At least two image regions, wherein the first weight corresponding to the elements of a portion of at least one image region is greater than the first weight corresponding to the elements of a portion of at least another image region; In at least one image region, the first weight corresponding to elements in a portion of the region is greater than the first weight corresponding to elements in another portion of the region. In at least one image region, the smaller the distance between the elements of a certain region and the dividing boundary, the larger the first weight corresponding to the elements of that certain region; and the smaller the distance between the elements of another region and the dividing boundary, the smaller the first weight corresponding to the elements of that other region.

10. A processing apparatus, wherein, include: The system includes a memory and a processor, wherein the memory stores an image processing program, and when the image processing program is executed by the processor, it implements the steps of the image processing method as described in claim 1.

11. A storage medium, wherein, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the image processing method as described in claim 1.

Images