Image processing method, processing device, and storage medium

Abstract

Provided in the present application are an image processing method, a processing device, and a storage medium. The image processing method comprises: on the basis of a first cost of a cost evaluation block of a current block, performing selection and / or sorting on encoding and / or decoding modes of the current block. The technical solution of the present application can improve the accuracy of sorting or selection performed on a plurality of encoding and / or decoding modes, thereby improving the efficiency of video encoding and / or decoding.

Description

Image processing methods, processing devices and storage media Technical Field

[0001] This application relates to the field of image processing technology, specifically to an image processing method, processing device, and storage medium. Background Technology

[0002] The existing high-efficiency video coding standard protocol (H.266 / VVC) proposes a video frame coding technique to improve coding performance without significantly increasing computational complexity. Specifically, when encoding and decoding video frames, the protocol divides each frame into different blocks and performs prediction, transformation and quantization processing before encoding and decoding.

[0003] In the process of conceiving and implementing this application, the inventors discovered at least the following problems:

[0004] At various stages of encoding and decoding, when sorting or filtering encoding and / or decoding modes (such as sorting or filtering partitioning modes, or sorting or filtering prediction modes), cost evaluation is used. For example, in the prediction stage, the cost of prediction modes needs to be estimated to sort or filter multiple prediction modes. Currently, cost evaluation functions such as SAD (Sum of absolute differences) or SATD (Sum of Absolute Transformed Difference) are used for evaluation. However, SAD does not consider the correlation between residuals, and SATD ignores the weights of DC and high-frequency variable amplitudes, resulting in insufficient accuracy in estimating the cost of prediction modes. This leads to insufficient accuracy in sorting or filtering multiple encoding and / or decoding modes, thus limiting the efficiency of video encoding and / or decoding.

[0005] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the Invention

[0006] To address the aforementioned technical problems, this application provides an image processing method, processing device, and storage medium, aiming to solve the technical problem of how to improve the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby supporting the improvement of video encoding and / or decoding efficiency.

[0007] This application provides an image processing method, applicable to a processing device, comprising the following steps:

[0008] S10, evaluate the first cost of the block based on the cost of the current block, and select and / or sort the encoding and / or decoding modes of the current block.

[0009] Optionally, the cost evaluation block of the current block is determined or obtained based on at least one of the following:

[0010] The pixel block of the current block; at least one first difference block; at least one second difference block; at least one third difference block; at least one fourth difference block; residual block; absolute value block of the difference; transform block of the difference; absolute value block of the transform block of the difference; quantization block of the difference; absolute value block of the quantization block of the difference.

[0011] Optionally, the image processing method further includes at least one of the following:

[0012] The first difference block is determined or obtained based on the difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode;

[0013] The second difference block is determined or obtained based on the difference between image blocks corresponding to at least two candidate encoding and / or decoding modes;

[0014] The third difference block is determined or obtained based on the difference between the two cost evaluation blocks of the current block;

[0015] The fourth difference block is determined or obtained based on the difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode.

[0016] Optionally, the image processing method further includes at least one of the following:

[0017] The cost calculation methods are different for cost evaluation elements at at least two different positions in the same cost evaluation block;

[0018] The cost calculation methods are different for at least two cost evaluation blocks with different components;

[0019] The cost calculation methods differ for at least two cost evaluation blocks with different size parameters;

[0020] The cost calculation methods differ for at least two cost evaluation blocks with different texture features;

[0021] The cost evaluation blocks are categorized, and at least two different categories of cost evaluation blocks have different cost calculation methods.

[0022] Optionally, the first cost of the cost evaluation block is determined or obtained according to at least one of the following:

[0023] The second cost of at least one cost evaluation element within the cost evaluation block;

[0024] The third cost of at least one non-uniform block;

[0025] The fourth cost of at least one element of the transformed block after transforming the cost evaluation block;

[0026] The fifth cost of at least one element in the quantized block after quantizing the cost evaluation block;

[0027] At least one neural network and / or cost lookup table are used to determine or obtain the first cost of the cost evaluation block.

[0028] Optionally, the non-uniform block is obtained by non-uniformizing the cost evaluation block;

[0029] Optionally, the image processing method further includes at least one of the following:

[0030] If the cost evaluation element is within the first range, then the second cost is determined or obtained based on the cost evaluation element and the first weight;

[0031] If the cost evaluation element is located in a second range that is different from the first range, then the second cost is determined or obtained based on the second weight that is different from the first weight and the cost evaluation element.

[0032] If the cost evaluation element is located in a third range that is different from both the first and second ranges, then the second cost is determined or obtained based on at least one neural network and / or a cost lookup table, and the cost evaluation element.

[0033] If the cost evaluation element is in the fourth range, then the second cost is determined or obtained based on the absolute value of at least one cost evaluation element;

[0034] If the cost evaluation block meets the first condition, then the second cost is determined or obtained according to the first parameter table;

[0035] If the cost evaluation block does not meet the first condition, then the second cost is determined or obtained according to the second parameter table, which is different from the first parameter table;

[0036] The second cost is determined or obtained based on the third parameter table.

[0037] Optionally, the image processing method further includes at least one of the following:

[0038] Based on the third weight, at least one cost evaluation element located in at least one column on the far right and / or at least one row on the far bottom of the cost evaluation block is weighted to obtain the first non-uniform element, and the non-uniform block is determined or obtained based on the at least one first non-uniform element.

[0039] Based on the distance between at least one cost evaluation element and the last column or last row in the cost evaluation block, a fourth weight is determined or obtained. Based on the fourth weight, at least one cost evaluation element is weighted to obtain a second non-uniform element. Based on at least one second non-uniform element, a non-uniform block is determined or obtained.

[0040] Based on a non-uniform weight matrix with at least two different weights, at least one cost evaluation element in the cost evaluation block is weighted to obtain a third non-uniform element, and a non-uniform block is determined or obtained based on at least one third non-uniform element.

[0041] At least one cost evaluation element in the cost evaluation block is weighted according to the neural network and / or the first lookup table to obtain the fourth non-uniform element, and the non-uniform block is determined or obtained based on the at least one fourth non-uniform element.

[0042] Optionally, the cost evaluation blocks may be categorized based on at least one of the following:

[0043] The sum of the absolute values ​​of the cost evaluation elements in the cost evaluation block; the feature complexity of the cost evaluation block; the first eigenvalue of the cost evaluation block; the size parameter of the cost evaluation block; the sum of the absolute values ​​of the transformation elements in the transformation block; the sum of the absolute values ​​of the quantization elements in the quantization block.

[0044] Optionally, step S10 includes the following steps:

[0045] S11, determine or obtain the first candidate list based on the first cost of the cost evaluation block;

[0046] S12, Select and / or sort the encoding and / or decoding modes of the current block according to the first candidate list.

[0047] Optionally, the first candidate list includes at least one of the second candidate list, the third candidate list, the fourth candidate list, the fifth candidate list, the sixth candidate list, the seventh candidate list, the eighth candidate list, the ninth candidate list, the first candidate sublist, the second candidate sublist, the third candidate sublist, and the fourth candidate sublist.

[0048] Optionally, the image processing method further includes at least one of the following:

[0049] Based on the first sorting result of sorting at least two candidate encoding and / or decoding modes of the current block according to the first cost, a second candidate list is determined or obtained;

[0050] Based on a second cost calculation method that is different from the cost calculation method used to determine the first cost, a sixth cost of at least one cost evaluation block is determined or obtained. Based on the second sorting result of sorting at least two candidate encoding and / or decoding modes according to the at least one sixth cost, the second candidate list is updated to obtain a third candidate list.

[0051] When at least one cost evaluation block satisfies the second condition, a fourth candidate list is determined or obtained based on the second candidate list.

[0052] When at least one cost evaluation block does not meet the second condition, a seventh cost is determined or obtained based on a third cost calculation method that is different from the cost calculation method for determining the first cost. The second candidate list is updated based on a third sorting result of sorting at least two candidate encoding and / or decoding modes according to the at least one seventh cost to obtain a fifth candidate list.

[0053] Based on a fourth cost calculation method that differs from the cost calculation method used to determine the first cost, the cost of at least one list element in the head of the second candidate list is calculated to obtain the eighth cost. The second candidate list is then updated based on the eighth cost to obtain the sixth candidate list.

[0054] A seventh candidate list is determined or obtained based on the decision factors of at least one candidate encoding and / or decoding mode and the decision factors of at least one list element in the second candidate list.

[0055] The cost of at least one element in the seventh candidate list is calculated based on the fifth cost calculation method, which is different from the cost calculation method used to determine the first cost, to obtain the ninth cost. The seventh candidate list is then updated based on the at least one ninth cost to obtain the eighth candidate list.

[0056] Split the second candidate list to obtain at least one first candidate sublist;

[0057] Based on a sixth cost calculation method that is different from the cost calculation method for determining the first cost, the cost of at least one list element in at least one first candidate sublist is calculated to obtain a tenth cost. Based on the at least tenth cost, the at least one first candidate sublist is updated to obtain at least one second candidate sublist.

[0058] Based on at least two distinct seventh and eighth cost calculation methods, cost calculations are performed on at least one list element in at least two first candidate sublists to obtain eleventh and twelfth costs. At least one first candidate sublist is updated based on the eleventh cost to obtain a third candidate sublist. At least one first candidate sublist is updated based on the twelfth cost to obtain a fourth candidate sublist. The ninth candidate sublist is determined or obtained based on the third and fourth candidate sublists.

[0059] Optionally, the image processing method further includes at least one of the following:

[0060] The list elements in the first candidate list include at least one of the following: prediction mode, block partitioning mode, motion vector, block vector, filter, and cost evaluation block;

[0061] The shape of the cost evaluation block includes at least one of the following: rectangular, T-shaped, L-shaped, and irregular shape;

[0062] The cost evaluation block contains at least one row of cost evaluation elements and / or at least one column of cost evaluation elements;

[0063] The encoding and / or decoding modes include at least one of prediction mode, block partitioning mode, motion vector, block vector, and filter;

[0064] Step S12 includes: determining or obtaining the cost of at least one encoding and / or decoding mode based on the first candidate list, and selecting and / or sorting the encoding and / or decoding modes of the current block according to the cost of the at least one encoding and / or decoding mode.

[0065] This application also provides a processing apparatus, which includes a processing module:

[0066] The processing module is used to evaluate the first cost of the current block based on the cost of the current block, and to select and / or sort the encoding and / or decoding modes of the current block.

[0067] This application also provides a processing device, including: a memory and a processor, wherein the memory stores an image processing program, and when the image processing program is executed by the processor, it implements the steps of any of the image processing methods described above.

[0068] This application also provides a storage medium storing a computer program that, when executed by a processor, implements the steps of any of the image processing methods described above.

[0069] As described above, the image processing method of this application includes: evaluating a first cost of a block based on the cost of the current block, and selecting and / or sorting the encoding and / or decoding modes of the current block. Through the technical solution of this application, when calculating the cost of entropy encoding at different stages of video encoding and / or decoding, the first cost of the current block can be comprehensively considered, which can improve the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby supporting improved efficiency in video encoding and / or decoding. Attached Figure Description

[0070] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0071] Figure 1 is a schematic diagram of the hardware structure of a smart terminal that implements various embodiments of this application;

[0072] Figure 2 is a communication network system architecture diagram provided in an embodiment of this application;

[0073] Figure 3 is a schematic diagram of the hardware structure of a controller 140 provided in this application;

[0074] Figure 4 is a schematic diagram of the hardware structure of a network node 150 provided in this application;

[0075] Figure 5 is a flowchart illustrating the image processing method according to the first embodiment;

[0076] Figure 6 is a schematic diagram of the coding framework of the image processing method according to the first embodiment;

[0077] Figure 7 is a schematic diagram of pattern filtering for the predicted pattern in the image processing method according to the first embodiment;

[0078] Figure 8 is a schematic diagram of vector filtering for vectors in the image processing method according to the first embodiment;

[0079] Figure 9 is a schematic diagram of filter selection for a filter in an image processing method according to the first embodiment;

[0080] Figure 10 is a flowchart illustrating the block cost calculation according to the third embodiment;

[0081] Figure 11 is a flowchart illustrating the cost calculation based on block feature classification according to the third embodiment;

[0082] Figure 12 is a flowchart illustrating the element cost calculation according to the third embodiment;

[0083] Figure 13 is a flowchart illustrating the block cost calculation based on non-uniform weights according to the third embodiment;

[0084] Figure 14 is a flowchart illustrating the block cost calculation based on a multi-level lookup table according to the third embodiment;

[0085] Figure 15 is a flowchart illustrating the generation of the most likely list based on secondary sorting according to the fifth embodiment;

[0086] Figure 16 is a schematic diagram of a processing device provided in an embodiment of this application.

[0087] The realization of the objectives, functional features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0088] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0089] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0090] It should be understood that although the terms first, second, third, etc., may be used herein to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word “if” as used herein may be interpreted as “when…” or “in response to determination”. Furthermore, as used herein, the singular forms “a,” “an,” and “the” are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms “comprising,” “including,” indicate the presence of the stated feature, step, operation, element, component, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms “or,” “and / or,” “including at least one of the following,” etc., as used in this application may be interpreted as inclusive, or mean any one or any combination thereof. For example, "including at least one of the following: A, B, C" means "any one of the following: A; B; C; A and B; A and C; B and C; A and B and C." Similarly, "A, B, or C" or "A, B, and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A and B and C." Exceptions to this definition only occur when the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0091] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0092] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0093] It should be noted that step designations such as S11 and S12 are used in this document for the purpose of more clearly and concisely describing the corresponding content, and do not constitute a substantial limitation on the order. In specific implementation, those skilled in the art may execute S12 first and then S11, etc., but these should all be within the protection scope of this application.

[0094] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0095] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0096] The processing device in this application can be a server, a smart terminal, or other devices. Optionally, the smart terminal can be implemented in various forms. For example, the smart terminal described in this application can include smart terminals such as mobile phones, tablets, laptops, handheld computers, personal digital assistants (PDAs), portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, etc., as well as fixed terminals such as digital TVs and desktop computers.

[0097] The following description will use a mobile terminal as an example. Those skilled in the art will understand that, apart from elements specifically designed for mobile purposes, the construction according to the embodiments of this application can also be applied to fixed-type terminals.

[0098] Please refer to Figure 1, which is a schematic diagram of the hardware structure of a mobile terminal implementing various embodiments of this application. The mobile terminal 100 may include: an RF (Radio Frequency) unit 101, a WiFi module 102, an audio output unit 103, an A / V (Audio / Video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111, etc. Those skilled in the art will understand that the mobile terminal structure shown in Figure 1 does not constitute a limitation on the mobile terminal. The mobile terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0099] The following section, with reference to Figure 1, provides a detailed description of each component of the mobile terminal:

[0100] The radio frequency unit 101 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 110; additionally, it transmits uplink data to the base station. Typically, the radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, and a duplexer. Furthermore, the radio frequency unit 101 can also communicate wirelessly with networks and other devices. The aforementioned wireless communications may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution), 5G, and 6G.

[0101] WiFi is a short-range wireless transmission technology. Mobile terminals using WiFi module 102 can help users send and receive emails, browse web pages, and access streaming media, providing wireless broadband internet access. Although Figure 1 shows WiFi module 102, it is understood that it is not an essential component of the mobile terminal and can be omitted as needed without altering the essence of the invention.

[0102] The audio output unit 103 can convert audio data received by the radio frequency unit 101 or the WiFi module 102 or stored in the memory 109 into audio signals and output them as sound when the mobile terminal 100 is in call signal receiving mode, call mode, recording mode, voice recognition mode, broadcast receiving mode, etc. Furthermore, the audio output unit 103 can also provide audio output related to specific functions performed by the mobile terminal 100 (e.g., call signal receiving sound, message receiving sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.

[0103] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos acquired by an image capture device (such as a camera) in video capture mode or image capture mode. The processed image frames can be displayed on the display unit 106. The image frames processed by the GPU 1041 can be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) in operating modes such as telephone call mode, recording mode, and voice recognition mode, and can process such sound into audio data. The processed audio (voice) data can be converted into a format that can be transmitted to a mobile communication base station via the radio frequency unit 101 in telephone call mode. The microphone 1042 can implement various types of noise cancellation (or suppression) algorithms to eliminate (or suppress) noise or interference generated during the reception and transmission of audio signals.

[0104] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Optionally, the light sensor includes an ambient light sensor and a proximity sensor. Optionally, the ambient light sensor can adjust the brightness of the display panel 1061 according to the ambient light level, and the proximity sensor can turn off the display panel 1061 and / or backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc. Other sensors that may be configured in the phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0105] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0106] User input unit 107 can be used to receive input numerical or character information, and generate key signal inputs related to user settings and function control of the mobile terminal. Optionally, user input unit 107 may include touch panel 1071 and other input devices 1072. Touch panel 1071, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near touch panel 1071), and drive corresponding connection devices according to a pre-set program. Touch panel 1071 may include a touch detection device and a touch controller. Optionally, the touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to processor 110, and can receive and execute commands sent by processor 110. In addition, touch panel 1071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may also include other input devices 1072. Optionally, other input devices 1072 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc., without being specifically limited here.

[0107] Optionally, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. Subsequently, the processor 110 provides corresponding visual output on the display panel 1061 according to the type of touch event. Although in FIG. 1, the touch panel 1071 and the display panel 1061 are implemented as two independent components to realize the input and output functions of the mobile terminal, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to realize the input and output functions of the mobile terminal. The specific implementation is not limited here.

[0108] Interface unit 108 serves as an interface through which at least one external device can connect to mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, and so on. Interface unit 108 may be used to receive input (e.g., data, power, etc.) from the external device and transmit the received input to one or more elements within mobile terminal 100, or it may be used to transmit data between mobile terminal 100 and the external device.

[0109] The memory 109 can be used to store software programs and various data. The memory 109 may primarily include a program storage area and a data storage area. Optionally, the program storage area may store the operating system, applications required for at least one function (such as sound playback, image playback, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory 109 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0110] The processor 110 is the control center of the mobile terminal. It connects various parts of the mobile terminal via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 109, and by calling data stored in the memory 109, it performs various functions and processes data of the mobile terminal, thereby providing overall monitoring of the mobile terminal. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor. Optionally, the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 110.

[0111] The mobile terminal 100 may also include a power supply 111 (such as a battery) that supplies power to various components. Preferably, the power supply 111 can be logically connected to the processor 110 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0112] Although not shown in Figure 1, the mobile terminal 100 may also include a Bluetooth module, etc., which will not be described in detail here.

[0113] To facilitate understanding of the embodiments of this application, the communication network system on which the mobile terminal of this application is based is described below.

[0114] Please refer to Figure 2, which is a communication network system architecture diagram provided in an embodiment of this application. The communication network system is an LTE system based on the universal mobile communication technology. The LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203, and the operator's IP services 204, which are connected in sequence.

[0115] Optionally, UE201 can be the aforementioned terminal 100, which will not be described in detail here.

[0116] E-UTRAN202 includes eNodeB2021 and other eNodeB2022, etc. Optionally, eNodeB2021 can connect to other eNodeB2022 via backhaul (e.g., X2 interface), and eNodeB2021 connects to EPC203, providing access from UE201 to EPC203.

[0117] EPC203 may include MME (Mobility Management Entity) 2031, HSS (Home Subscriber Server) 2032, other MMEs 2033, SGW (Serving Gateway) 2034, PGW (Packet Data Network Gateway) 2035, and PCRF (Policy and Charging Rules Function) 2036, etc. Optionally, MME2031 is the control node that handles signaling between UE201 and EPC203, providing bearer and connection management. HSS2032 is used to provide registers to manage functions such as the Home Location Register (not shown in the figure) and stores user-specific information such as service characteristics and data rates. All user data can be sent through SGW2034. PGW2035 can provide UE 201 IP address allocation and other functions. PCRF2036 is the policy and charging control decision point for service data flow and IP bearer resources. It selects and provides available policy and charging control decisions for the policy and charging enforcement function unit (not shown in the figure).

[0118] IP services 204 may include the Internet, intranet, IMS (IP Multimedia Subsystem), or other IP services.

[0119] Although the above description uses the LTE system as an example, those skilled in the art should know that this application is not only applicable to the LTE system, but also to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, 5G and future new network systems (such as 6G), etc., without limitation.

[0120] Figure 3 is a schematic diagram of the hardware structure of a controller 140 provided in this application. The controller 140 includes a memory 1401 and a processor 1402. The memory 1401 is used to store program instructions, and the processor 1402 is used to call the program instructions in the memory 1401 to execute the steps performed by the controller in the first embodiment of the above method. The implementation principle and beneficial effects are similar, and will not be described again here.

[0121] Optionally, the controller further includes a communication interface 1403, which can be connected to the processor 1402 via a bus 1404. The processor 1402 can control the communication interface 1403 to implement the receiving and sending functions of the controller 140.

[0122] Figure 4 is a schematic diagram of the hardware structure of a network node 150 provided in this application. The network node 150 includes a memory 1501 and a processor 1502. The memory 1501 is used to store program instructions, and the processor 1502 is used to call the program instructions in the memory 1501 to execute the steps performed by the first node in the above method embodiment. The implementation principle and beneficial effects are similar, and will not be described again here.

[0123] Optionally, the controller further includes a communication interface 1503, which can be connected to the processor 1502 via a bus 1504. The processor 1502 can control the communication interface 1503 to implement the receiving and sending functions of the network node 150.

[0124] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0125] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk, SSD), etc.

[0126] Based on the above-described mobile terminal hardware structure and communication network system, various embodiments of this application are proposed.

[0127] First Embodiment

[0128] Referring to Figure 5, which is a flowchart illustrating the image processing method according to the first embodiment, the image processing method of this application embodiment can be applied to a processing device, including step S10:

[0129] Step S10: Evaluate the first cost of the current block based on the cost of the current block, and select and / or sort the encoding and / or decoding modes of the current block.

[0130] In this embodiment, the processing device can be a smart terminal, such as a mobile phone or computer, or a server, such as a local server or a cloud server. This embodiment and this application primarily use a smart terminal as an example for illustration.

[0131] Optionally, the technical solution of this embodiment can be applied to the fields of image encoding and decoding, video encoding and decoding, hardware video encoding and decoding, dedicated circuit video encoding and decoding, real-time video encoding and decoding, etc.

[0132] Optionally, for ease of understanding, a brief introduction to the encoding and decoding architecture is provided first: Referring to Figure 6, in the encoding and decoding framework, the input video frame can be divided into blocks to obtain multiple blocks. Each block undergoes frame transformation processing through a frame transformation module. The transformed video frame is then input into modules such as transformation, quantization, intra-frame prediction, inter-frame coding, inverse quantization, inverse transform, and loop filtering. The video frame processed by the quantization module is entropy encoded, outputting bits 01011100, etc. Motion estimation is performed based on the video frame obtained after loop filtering and the video frame processed by frame transformation to obtain motion information. Inter-frame prediction is then performed based on the motion information.

[0133] Optionally, at the encoding end, video frames are input to a frame processing module. This module executes a certain selection rule and then performs one or more of the following operations on the frame: geometric transformation, downsampling, linear transformation, nonlinear transformation, and pre-filtering (in the case of filtering, mode information may not be available). The transformed frame is then input to subsequent transformation, quantization, intra-frame prediction, inter-frame coding, inverse quantization, and inverse transform modules. The results of these processes are input to the inverse frame transform module, where inverse geometric transformation, upsampling, and filtering are performed before outputting the decoded image.

[0134] Alternatively, loop filtering can be considered as part of the frame inverse transform module.

[0135] Optionally, at the decoding end, the bitstream is read and the transform mode information is parsed (in the case of filtering, there may be no mode information); the video stream is subjected to inverse quantization, transform, inter-frame prediction, intra-frame prediction, etc., to decode the video frames (after transformation at the encoding end); the video frames are subjected to inverse frame transform according to the mode information and the decoded image is output. The inverse frame transform includes inverse geometric transformation, upsampling, filtering, etc.

[0136] Alternatively, loop filtering can be considered as part of the frame inverse transform module.

[0137] Optionally, cost calculation is required at each stage of encoding and / or decoding to select and / or sort based on the calculated cost. For example, in the block partitioning stage where the input video frame is divided into blocks, the cost of at least one block partitioning mode can be calculated according to this embodiment for selection and / or sorting. Similarly, in the prediction stage, when determining the prediction mode used for intra-frame prediction and / or inter-frame prediction, the cost of candidate prediction modes is calculated for selection and / or sorting. Furthermore, the methods described in this embodiment can also be used for various stages such as block vectors, motion vectors, and filters.

[0138] Optionally, this embodiment can address the problems existing in the cost evaluation function, such as the absolute value of each element in SAD or SATD failing to express the true entropy coding cost of the coefficient block. The technical solution of this embodiment can circumvent this defect and improve the accuracy of its estimation of the true entropy coding cost by controlling the complexity of the cost evaluation function.

[0139] Optionally, the current block can be an image block to be encoded and / or decoded.

[0140] Before executing step S10, the cost evaluation block of the current block can be determined first. For example, the block after the current block has been predicted and the residual has been processed can be used as the cost evaluation block, or the block after the current block has been predicted can be used as the cost evaluation block, etc.

[0141] Optionally, the image processing method further includes: determining or obtaining the cost evaluation block of the current block according to at least one of the following methods one through eleven:

[0142] Method 1: The pixel block of the current block;

[0143] Optionally, the pixel block of the current block can be an image block containing at least one pixel.

[0144] Optionally, the pixel blocks of the current block can be directly used as the cost evaluation block of the current block; the pixel blocks of the current block can be transformed (e.g., format conversion, pixel enlargement or reduction, etc.) to obtain the cost evaluation block of the current block; the pixel blocks of the current block can also be processed according to other rules to obtain the cost evaluation block of the current block, etc.

[0145] For example, if the size parameter of the pixel block of the current block is greater than a preset size parameter threshold, then the preset first image block is determined as the cost evaluation block of the current block; and / or, if the size parameter of the pixel block of the current block is less than or equal to the preset size parameter threshold, then the preset second image block is determined as the cost evaluation block of the current block.

[0146] Optionally, the pixel block size parameters of the current block may include the width, height, area, and perimeter of the current block.

[0147] Optionally, the preset size parameter thresholds may include width threshold, height threshold, area threshold, and perimeter threshold, etc. For example, the preset size parameter thresholds can be set to width x height as 16x16, 8x8, etc.

[0148] For example, if the texture features of the pixel blocks of the current block meet the preset texture feature requirements (such as high texture complexity), then the preset first image block is determined as the cost evaluation block of the current block; and / or, if the texture features of the pixel blocks of the current block do not meet the preset texture feature requirements, then the preset second image block is determined as the cost evaluation block of the current block.

[0149] Optionally, the first image block and the second image block are different from each other, and the first image block and / or the second image block may include at least one of the following:

[0150] The residual block of the current block;

[0151] A first difference block is determined or obtained based on the difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, and a residual block is determined or obtained based on the first difference block;

[0152] A second difference block is determined or obtained based on the difference between image blocks corresponding to at least two candidate encoding and / or decoding modes, and a residual block is determined or obtained based on the second difference block;

[0153] The third difference block is determined or obtained based on the difference between the two cost evaluation blocks of the current block, and the residual block is determined or obtained based on the third difference block;

[0154] A fourth difference block is determined or obtained based on the difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, and a residual block is determined or obtained based on the fourth difference block;

[0155] The difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, and at least one of the following: the absolute value block of the difference, the transform block of the difference, the absolute value block of the transform block of the difference, the quantization block of the difference, and the absolute value block of the quantization block of the difference;

[0156] The difference between image blocks corresponding to at least two candidate encoding and / or decoding modes, and at least one of the following: absolute value block of the difference, transform block of the difference, absolute value block of the transform block of the difference, quantization block of the difference, and absolute value block of the quantization block of the difference;

[0157] The current block contains at least one of the following: the absolute value block of the difference between the two cost evaluation blocks; the transformation block of the difference; the absolute value block of the transformation block of the difference; the quantization block of the difference; and the absolute value block of the quantization block of the difference.

[0158] The difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, and at least one of the following: the absolute value block of the difference, the transform block of the difference, the absolute value block of the transform block of the difference, the quantization block of the difference, and the absolute value block of the quantization block of the difference.

[0159] In this method, the cost evaluation block of the current block is determined or obtained based on the pixel blocks of the current block, so that the cost evaluation block is closely related to the pixel blocks, and thus the first cost is closely related to the pixel blocks. This makes the selection and / or sorting of the encoding and / or decoding modes of the current block based on the first cost more accurate, thereby supporting the improvement of the efficiency of video encoding and / or decoding.

[0160] Method 2, at least one first difference block;

[0161] Optionally, at least one first difference block corresponding to the current block can be determined or obtained.

[0162] Optionally, the image processing method further includes: the first difference block is determined or obtained based on the difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode.

[0163] Optionally, the template for the current block can be determined or obtained based on the reconstructed pixels adjacent to and / or non-adjacent to the current block, such as a T-shaped or L-shaped pixel region. For example, at the decoding end, the reconstructed L-shaped neighbors on the left and top sides of the current block are used as templates, denoted as Lrec. The L-shaped region is predicted according to the prediction mode (decoding mode) to obtain the prediction result of the L-shaped region, which is also the image block corresponding to the decoding mode, denoted as Lpred. Then, the difference between the prediction result Lpred and the reconstructed L-shaped template Lrec, Lpred-Lrec, is the first difference.

[0164] Optionally, the candidate encoding and / or decoding modes can be candidate encoding and / or decoding modes.

[0165] Optionally, the image block corresponding to the encoding and / or decoding mode can be an image block obtained by processing the current block using the encoding and / or decoding mode.

[0166] Optionally, the image block corresponding to the encoding and / or decoding mode can be an image block obtained by processing the template of the current block using the encoding and / or decoding mode.

[0167] Optionally, the difference block corresponding to the difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode can be used as the first difference block. For example, the first difference block is obtained by subtracting pixels between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode.

[0168] Optionally, the cost evaluation block of the current block can be determined or obtained based on at least one first difference block.

[0169] Optionally, the first difference block can be directly used as the cost evaluation block of the current block; the first difference block can be transformed (e.g., format conversion, pixel enlargement or reduction, etc.) to obtain the cost evaluation block of the current block; the first difference block can also be processed according to other rules to obtain the cost evaluation block of the current block, etc.

[0170] For example, if the pixel corresponding to the first difference block is located in the first pixel range, the preset third image block can be determined as the cost evaluation block of the current block; and / or, if the pixel corresponding to the first difference block is located in the second pixel range, which is different from the first pixel range, the preset fourth image block can be determined as the cost evaluation block of the current block.

[0171] Optionally, the third image block and the fourth image block are different from each other, and the third image block and / or the fourth image block may include at least one of the following:

[0172] The residual block of the current block;

[0173] A second difference block is determined or obtained based on the difference between image blocks corresponding to at least two candidate encoding and / or decoding modes, and a residual block is determined or obtained based on the second difference block;

[0174] The third difference block is determined or obtained based on the difference between the two cost evaluation blocks of the current block, and the residual block is determined or obtained based on the third difference block;

[0175] A fourth difference block is determined or obtained based on the difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, and a residual block is determined or obtained based on the fourth difference block;

[0176] The difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, and at least one of the following: the absolute value block of the difference, the transform block of the difference, the absolute value block of the transform block of the difference, the quantization block of the difference, and the absolute value block of the quantization block of the difference;

[0177] The difference between image blocks corresponding to at least two candidate encoding and / or decoding modes, and at least one of the following: absolute value block of the difference, transform block of the difference, absolute value block of the transform block of the difference, quantization block of the difference, and absolute value block of the quantization block of the difference;

[0178] The current block contains at least one of the following: the absolute value block of the difference between the two cost evaluation blocks; the transformation block of the difference; the absolute value block of the transformation block of the difference; the quantization block of the difference; and the absolute value block of the quantization block of the difference.

[0179] The difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, and at least one of the following: the absolute value block of the difference, the transform block of the difference, the absolute value block of the transform block of the difference, the quantization block of the difference, and the absolute value block of the quantization block of the difference.

[0180] In this method, a cost evaluation block is determined or obtained based on at least one first difference block. Since the first difference block can be determined or obtained based on the difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, the cost evaluation block can be closely associated with the template of the current block and at least one candidate encoding and / or decoding mode. This makes the selection and / or sorting of the encoding and / or decoding mode of the current block based on the first cost more accurate, thereby supporting the improvement of video encoding and / or decoding efficiency.

[0181] Method 3, at least one second interpolation block;

[0182] Optionally, at least one second difference block corresponding to the current block can be determined or obtained.

[0183] Optionally, the image processing method further includes: the second difference block is determined or obtained based on the difference between image blocks corresponding to at least two candidate encoding and / or decoding modes.

[0184] Optionally, during the same stage of encoding and decoding, at least two different candidate encoding and / or decoding modes can be used to process the current block to obtain image blocks between at least two candidate encoding and / or decoding modes. Then, the image block difference between the image blocks of the at least two candidate encoding and / or decoding modes can be calculated (e.g., subtracting pixels at the same image position) to obtain the corresponding difference. The image block corresponding to the difference is then used as the second difference block. For example, the at least two different image blocks processed by the candidate encoding and / or decoding modes can be subtracted from each other to obtain the second difference block.

[0185] Alternatively, the difference can be the difference between pixels.

[0186] Optionally, the cost evaluation block of the current block can be determined or obtained based on at least one second difference block.

[0187] Optionally, the second difference block can be directly used as the cost evaluation block of the current block; the second difference block can be transformed (e.g., format conversion, pixel enlargement or reduction, etc.) to obtain the cost evaluation block of the current block; the second difference block can also be processed according to other rules to obtain the cost evaluation block of the current block, etc.

[0188] For example, if the pixel corresponding to the second difference block is located in the third pixel range, the preset fifth image block can be determined as the cost evaluation block of the current block; and / or, if the pixel corresponding to the second difference block is located in the fourth pixel range, which is different from the third pixel range, the preset sixth image block can be determined as the cost evaluation block of the current block.

[0189] Optionally, the fifth image block and the sixth image block are different from each other, and the fifth image block and / or the sixth image block may include at least one of the following:

[0190] The residual block of the current block;

[0191] A first difference block is determined or obtained based on the difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, and a residual block is determined or obtained based on the first difference block;

[0192] The third difference block is determined or obtained based on the difference between the two cost evaluation blocks of the current block, and the residual block is determined or obtained based on the third difference block;

[0193] A fourth difference block is determined or obtained based on the difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, and a residual block is determined or obtained based on the fourth difference block;

[0194] The difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, and at least one of the following: the absolute value block of the difference, the transform block of the difference, the absolute value block of the transform block of the difference, the quantization block of the difference, and the absolute value block of the quantization block of the difference;

[0195] The difference between image blocks corresponding to at least two candidate encoding and / or decoding modes, and at least one of the following: absolute value block of the difference, transform block of the difference, absolute value block of the transform block of the difference, quantization block of the difference, and absolute value block of the quantization block of the difference;

[0196] The current block contains at least one of the following: the absolute value block of the difference between the two cost evaluation blocks; the transformation block of the difference; the absolute value block of the transformation block of the difference; the quantization block of the difference; and the absolute value block of the quantization block of the difference.

[0197] The difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, and at least one of the following: the absolute value block of the difference, the transform block of the difference, the absolute value block of the transform block of the difference, the quantization block of the difference, and the absolute value block of the quantization block of the difference.

[0198] In this method, a cost evaluation block is determined or obtained based on at least one second difference block. Since the second difference block can be determined or obtained based on the difference between image blocks corresponding to at least two candidate encoding and / or decoding modes, the cost evaluation block can be closely associated with at least two candidate encoding and / or decoding modes. This makes the selection and / or sorting of the encoding and / or decoding modes of the current block based on the first cost more accurate, thereby supporting the improvement of video encoding and / or decoding efficiency.

[0199] Method 4, at least one third interpolation block;

[0200] Optionally, at least one third difference block corresponding to the current block can be determined or obtained.

[0201] Optionally, the image processing method further includes: the third difference block is determined or obtained based on the difference between the two cost evaluation blocks of the current block.

[0202] Optionally, two different methods can be used to determine or obtain the two cost evaluation blocks of the current block, perform difference calculation on the two cost evaluation blocks of the current block to obtain a third difference block, determine or obtain a new cost evaluation block based on at least one third difference block, and select and / or sort the encoding and / or decoding modes of the current block based on the new cost evaluation block.

[0203] Optionally, the cost evaluation block of the current block can be determined or obtained based on at least one third difference block.

[0204] Optionally, the third difference block can be directly used as the cost evaluation block of the current block; the third difference block can be transformed (e.g., format conversion, pixel enlargement or reduction, etc.) to obtain the cost evaluation block of the current block; the third difference block can also be processed according to other rules to obtain the cost evaluation block of the current block, etc.

[0205] For example, if the pixel corresponding to the third difference block is located in the fifth pixel range, the preset seventh image block can be determined as the cost evaluation block of the current block; and / or, if the pixel corresponding to the third difference block is located in the sixth pixel range, which is different from the fifth pixel range, the preset eighth image block can be determined as the cost evaluation block of the current block.

[0206] Optionally, the seventh image block and the eighth image block are different from each other, and the seventh image block and / or the eighth image block may include at least one of the following:

[0207] The residual block of the current block;

[0208] A first difference block is determined or obtained based on the difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, and a residual block is determined or obtained based on the first difference block;

[0209] A second difference block is determined or obtained based on the difference between image blocks corresponding to at least two candidate encoding and / or decoding modes, and a residual block is determined or obtained based on the second difference block;

[0210] A fourth difference block is determined or obtained based on the difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, and a residual block is determined or obtained based on the fourth difference block;

[0211] The difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, and at least one of the following: the absolute value block of the difference, the transform block of the difference, the absolute value block of the transform block of the difference, the quantization block of the difference, and the absolute value block of the quantization block of the difference;

[0212] The difference between image blocks corresponding to at least two candidate encoding and / or decoding modes, and at least one of the following: absolute value block of the difference, transform block of the difference, absolute value block of the transform block of the difference, quantization block of the difference, and absolute value block of the quantization block of the difference;

[0213] The current block contains at least one of the following: the absolute value block of the difference between the two cost evaluation blocks; the transformation block of the difference; the absolute value block of the transformation block of the difference; the quantization block of the difference; and the absolute value block of the quantization block of the difference.

[0214] The difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, and at least one of the following: the absolute value block of the difference, the transform block of the difference, the absolute value block of the transform block of the difference, the quantization block of the difference, and the absolute value block of the quantization block of the difference.

[0215] In this method, a cost evaluation block is determined or obtained based on at least one third difference block. Since the third difference block can be determined or obtained based on the difference between the two cost evaluation blocks of the current block, the accuracy of the subsequently determined first cost is improved. This makes the selection and / or sorting of the encoding and / or decoding modes of the current block based on the first cost more accurate, thereby supporting the improvement of the efficiency of video encoding and / or decoding.

[0216] Method 5, at least one fourth interpolation block;

[0217] Optionally, at least one fourth difference block corresponding to the current block can be determined or obtained.

[0218] Optionally, the image processing method further includes: the fourth difference block is determined or obtained based on the difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode.

[0219] Optionally, the current block can be processed according to at least one candidate encoding and / or decoding mode to obtain an image block corresponding to at least one candidate encoding and / or decoding mode, and the difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, such as pixel difference, can be calculated to obtain at least one fourth difference block.

[0220] Optionally, the cost evaluation block of the current block can be determined or obtained based on at least one fourth difference block, and the encoding and / or decoding modes of the current block can be selected and / or sorted based on the first cost of the cost evaluation block.

[0221] Optionally, the fourth difference block can be directly used as the cost evaluation block of the current block; the fourth difference block can be transformed (e.g., format conversion, pixel enlargement or reduction, etc.) to obtain the cost evaluation block of the current block; the fourth difference block can also be processed according to other rules to obtain the cost evaluation block of the current block, etc.

[0222] For example, if the pixel corresponding to the fourth difference block is located in the seventh pixel range, the preset ninth image block can be determined as the cost evaluation block of the current block; and / or, if the pixel corresponding to the first difference block is located in the eighth pixel range, which is different from the seventh pixel range, the preset tenth image block can be determined as the cost evaluation block of the current block.

[0223] Optionally, the ninth image block and the tenth image block are different from each other, and the ninth image block and / or the tenth image block may include at least one of the following:

[0224] The residual block of the current block;

[0225] A first difference block is determined or obtained based on the difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, and a residual block is determined or obtained based on the first difference block;

[0226] A second difference block is determined or obtained based on the difference between image blocks corresponding to at least two candidate encoding and / or decoding modes, and a residual block is determined or obtained based on the second difference block;

[0227] The third difference block is determined or obtained based on the difference between the two cost evaluation blocks of the current block, and the residual block is determined or obtained based on the third difference block;

[0228] The difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, and at least one of the following: the absolute value block of the difference, the transform block of the difference, the absolute value block of the transform block of the difference, the quantization block of the difference, and the absolute value block of the quantization block of the difference;

[0229] The difference between image blocks corresponding to at least two candidate encoding and / or decoding modes, and at least one of the following: absolute value block of the difference, transform block of the difference, absolute value block of the transform block of the difference, quantization block of the difference, and absolute value block of the quantization block of the difference;

[0230] The current block contains at least one of the following: the absolute value block of the difference between the two cost evaluation blocks; the transformation block of the difference; the absolute value block of the transformation block of the difference; the quantization block of the difference; and the absolute value block of the quantization block of the difference.

[0231] The difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, and at least one of the following: the absolute value block of the difference, the transform block of the difference, the absolute value block of the transform block of the difference, the quantization block of the difference, and the absolute value block of the quantization block of the difference.

[0232] In this method, a cost evaluation block is determined or obtained based on at least one fourth difference block. Since the fourth difference block can be determined or obtained based on the difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, the cost evaluation block can be closely associated with the current block and at least one candidate encoding and / or decoding mode. This makes the selection and / or sorting of the encoding and / or decoding mode of the current block based on the first cost more accurate, thereby supporting the improvement of video encoding and / or decoding efficiency.

[0233] Method 6: Residual Block;

[0234] Optionally, the residual block can be a residual block of the current block. For example, in the encoding and / or decoding stage, residual processing is performed based on the current block to obtain the residual block of the current block.

[0235] Optionally, a first difference block can be determined or obtained based on the difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, and a residual block can be determined or obtained based on the first difference block; a second difference block can be determined or obtained based on the difference between the image blocks corresponding to at least two candidate encoding and / or decoding modes, and a residual block can be determined or obtained based on the second difference block; a third difference block can be determined or obtained based on the difference between the two cost evaluation blocks of the current block, and a residual block can be determined or obtained based on the third difference block; a fourth difference block can be determined or obtained based on the difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, and a residual block can be determined or obtained based on the fourth difference block.

[0236] Optionally, the cost evaluation block of the current block can be determined or obtained based on the residual block of the current block, and the encoding and / or decoding modes of the current block can be selected and / or sorted based on the first cost of the cost evaluation block.

[0237] In this approach, by determining or obtaining a cost evaluation block based on the residual block, and then selecting and / or sorting the encoding and / or decoding modes of the current block based on the first cost of the cost evaluation block, it is possible to comprehensively consider the residual block when calculating the cost of entropy encoding at different stages of video encoding and / or decoding. This can improve the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby supporting the improvement of video encoding and / or decoding efficiency.

[0238] Method 7: Absolute value block of the difference;

[0239] Optionally, residual processing can be performed based on the current block, and absolute value processing can be performed to obtain the absolute value of the difference.

[0240] Optionally, the absolute value block of the difference can be determined or obtained based on the difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode; the absolute value block of the difference can be determined or obtained based on the difference between the image blocks corresponding to at least two candidate encoding and / or decoding modes; the absolute value block of the difference can be determined or obtained based on the difference between the two cost evaluation blocks of the current block; the absolute value block of the difference can be determined or obtained based on the difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode.

[0241] Optionally, the pixels in the absolute value block of the difference can be the absolute value of the difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode; can be the absolute value of the difference between the image blocks corresponding to at least two candidate encoding and / or decoding modes; can be the absolute value of the difference between the two cost evaluation blocks of the current block; or can be the absolute value of the difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode.

[0242] Optionally, the cost evaluation block of the current block can be determined or obtained based on the absolute value block of at least one difference, and the encoding and / or decoding modes of the current block can be selected and / or sorted based on the first cost of the cost evaluation block.

[0243] In this approach, a cost evaluation block is determined or obtained based on the absolute value of the difference. Then, the encoding and / or decoding modes of the current block are selected and / or sorted based on the first cost of the cost evaluation block. This enables the calculation of the cost of entropy encoding at different stages of video encoding and / or decoding to comprehensively consider the current block and at least one candidate encoding and / or decoding mode, thereby improving the accuracy of sorting or filtering multiple encoding and / or decoding modes and thus supporting the improvement of video encoding and / or decoding efficiency.

[0244] Method 8: Difference Transformation Block;

[0245] Optionally, the transformed block of the difference can be determined or obtained based on the difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode; the transformed block of the difference can be determined or obtained based on the difference between the image blocks corresponding to at least two candidate encoding and / or decoding modes; the transformed block of the difference can be determined or obtained based on the difference between the two cost evaluation blocks of the current block; the transformed block of the difference can be determined or obtained based on the difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode.

[0246] Optionally, the image block containing the difference can be transformed to obtain the transformed block of the difference.

[0247] Optionally, the image block containing the difference can be determined or obtained based on the difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode, and / or based on the difference between the image blocks corresponding to at least two candidate encoding and / or decoding modes, and / or based on the difference between the two cost evaluation blocks of the current block, and / or based on the difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode.

[0248] Optionally, the cost evaluation block of the current block can be determined or obtained based on the transform block with at least one difference, and the encoding and / or decoding modes of the current block can be selected and / or sorted based on the first cost of the cost evaluation block.

[0249] In this approach, a cost evaluation block is determined or obtained based on the transform block of the difference. Then, the encoding and / or decoding modes of the current block are selected and / or sorted based on the first cost of the cost evaluation block. This enables the calculation of the cost of entropy encoding at different stages of video encoding and / or decoding to comprehensively consider the current block and at least one candidate encoding and / or decoding mode, thereby improving the accuracy of sorting or filtering multiple encoding and / or decoding modes and thus supporting the improvement of video encoding and / or decoding efficiency.

[0250] Method 9: Absolute value block of the difference transformation block;

[0251] Optionally, the absolute value block of the transform block of the difference can be determined or obtained based on the difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode; the absolute value block of the transform block of the difference can be determined or obtained based on the difference between the image blocks corresponding to at least two candidate encoding and / or decoding modes; the absolute value block of the transform block of the difference can be determined or obtained based on the difference between the two cost evaluation blocks of the current block; the absolute value block of the transform block of the difference can be determined or obtained based on the difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode.

[0252] Optionally, at least one pixel in the difference transformation block can be absolute valued to obtain the absolute value block of the difference transformation block.

[0253] Optionally, the absolute value blocks of the difference transformation blocks can all be positive numbers.

[0254] Optionally, the cost evaluation block of the current block can be determined or obtained based on the absolute value block of the transform block with at least one difference, and the encoding and / or decoding modes of the current block can be selected and / or sorted based on the first cost of the cost evaluation block.

[0255] In this approach, a cost evaluation block is determined or obtained based on the absolute value block of the transform block of the difference. Then, the encoding and / or decoding modes of the current block are selected and / or sorted based on the first cost of the cost evaluation block. This enables the calculation of the cost of entropy encoding at different stages of video encoding and / or decoding to comprehensively consider the current block and at least one candidate encoding and / or decoding mode, thereby improving the accuracy of sorting or filtering multiple encoding and / or decoding modes and thus supporting the improvement of video encoding and / or decoding efficiency.

[0256] Method 10, Quantization block for difference;

[0257] Optionally, a quantized block of the difference can be determined or obtained based on the difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode; a quantized block of the difference can be determined or obtained based on the difference between the image blocks corresponding to at least two candidate encoding and / or decoding modes; a quantized block of the difference can be determined or obtained based on the difference between the two cost evaluation blocks of the current block; or a quantized block of the difference can be determined or obtained based on the difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode.

[0258] Optionally, the image block containing the difference can be quantized to obtain a quantized block of the difference.

[0259] Optionally, the cost evaluation block of the current block can be determined or obtained based on the quantization block with at least one difference, and the encoding and / or decoding modes of the current block can be selected and / or sorted based on the first cost of the cost evaluation block.

[0260] In this approach, a cost evaluation block is determined or obtained based on the quantization block of the difference. Then, the encoding and / or decoding modes of the current block are selected and / or sorted based on the first cost of the cost evaluation block. This enables the calculation of the cost of entropy encoding at different stages of video encoding and / or decoding to comprehensively consider the current block and at least one candidate encoding and / or decoding mode, thereby improving the accuracy of sorting or filtering multiple encoding and / or decoding modes and thus supporting the improvement of video encoding and / or decoding efficiency.

[0261] Method 11: Absolute value block of the quantization block of the difference.

[0262] Optionally, the absolute value block of the quantized block of the difference can be determined or obtained based on the difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode; the absolute value block of the quantized block of the difference can be determined or obtained based on the difference between the image blocks corresponding to at least two candidate encoding and / or decoding modes; the absolute value block of the quantized block of the difference can be determined or obtained based on the difference between the two cost evaluation blocks of the current block; the absolute value block of the quantized block of the difference can be determined or obtained based on the difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode.

[0263] Optionally, at least one pixel in the quantization block of the difference can be absolute valued to obtain the absolute value block of the quantization block of the difference.

[0264] Optionally, the absolute value blocks of the quantization blocks of the difference can all be positive numbers.

[0265] Optionally, the cost evaluation block of the current block can be determined or obtained based on the absolute value block of the quantization block with at least one difference, and the encoding and / or decoding modes of the current block can be selected and / or sorted based on the first cost of the cost evaluation block.

[0266] In this approach, a cost evaluation block is determined or obtained based on the absolute value block of the quantization block of the difference. Then, the encoding and / or decoding modes of the current block are selected and / or sorted based on the first cost of the cost evaluation block. This enables the calculation of the cost of entropy encoding at different stages of video encoding and / or decoding to comprehensively consider the current block and at least one candidate encoding and / or decoding mode, thereby improving the accuracy of sorting or filtering multiple encoding and / or decoding modes and thus supporting the improvement of video encoding and / or decoding efficiency.

[0267] Optionally, for step S10, during the encoding and / or decoding process, the first cost of the cost evaluation block of the current block can be determined first. For example, it can be calculated using a pre-set function calculation formula, or it can be queried in a pre-set lookup table containing the mapping key between the cost evaluation block and the cost, or it can be determined in other ways, without any restrictions.

[0268] Optionally, the function calculation formula may include at least one of the following: Sum of absolute differences (SAD), Sum of absolute transformed differences (SATD), Sum of squared differences (SSD), Mean absolute difference (MAD), and Mean squared difference (MSD).

[0269] Optionally, after determining the cost of the current block and evaluating the cost of the current block, the processing device can select and / or sort the encoding and / or decoding modes of the current block based on the cost.

[0270] Optionally, the encoding and / or decoding modes include at least one of prediction modes, block partitioning modes, motion vectors, block vectors, and filters.

[0271] Optionally, the prediction mode may include prediction modes for angle prediction, prediction modes for cross-component prediction, prediction modes based on block vectors, and prediction modes for inter-frame prediction such as motion prediction / motion estimation.

[0272] Optionally, if the encoding and / or decoding mode is a prediction mode of encoding and / or decoding, and the cost evaluation block is a residual block, then step S10 can be performed for each prediction mode.

[0273] For example, as shown in Figure 7, the information of the block to be predicted is obtained, and K prediction modes are used for prediction to generate K prediction blocks respectively. That is, prediction (mode 1) is used to predict to obtain prediction block 1, and prediction (mode K) is used to predict to obtain prediction block K. K is a positive integer.

[0274] Optionally, the subtractor in Figure 7 operates on K prediction blocks to obtain K residual blocks. That is, residual block 1 = original block - prediction block 1, ..., residual block K = original block - prediction block K. The cost calculation module in Figure 7 calculates the cost for each of the K residual blocks to obtain K cost values, such as cost 1 corresponding to residual block 1 and cost K corresponding to residual block K. The cost sorting module in Figure 7 sorts the K costs, for example, according to the size of cost 1 to cost K, to obtain an ordered list [mode, cost]. The mode filtering module in Figure 7 filters the ordered list to obtain a prediction mode [list]. For example, the prediction mode with the lowest cost is selected to obtain a prediction mode or a list of prediction modes composed of multiple prediction modes.

[0275] Optionally, at the encoding end, a rapid, low-complexity coarse screening of the prediction modes can be performed according to this embodiment. Then, the remaining modes after coarse screening are precisely transformed, quantized, and entropy encoded to obtain the optimal prediction mode. For example, at the encoding end, there are a total of 67 angle prediction modes, mode 1,..., mode 67. The framework shown in Figure 7 can be used to first calculate the cost of the 34 prediction modes with odd numbers, i.e., prediction blocks 1, 3,..., 67. Then, these 34 costs are sorted and filtered in ascending order to obtain the four prediction modes with the lowest costs, let them be prediction modes 61, 21, 35, and 3. Then, the costs of the neighboring prediction modes 62, 60, 22, 20, 36, 34, 4, and 2 of these five prediction modes are calculated, and then the prediction modes 61, 62, 60, 21, 22, 20, 35, 36, 34, 3, 4, and 2 are sorted by cost. Select 5 cost patterns from these, for example, prediction patterns 60, 21, 36, 34, 3. This will form the final list of prediction patterns.

[0276] Optionally, referring to Figure 7, the method applied to the decoding end can be modified. Taking the most likely candidate prediction mode list for angle prediction modes as an example, there are a total of 67 angle prediction modes, and the framework in Figure 7 can be used. First, obtain the prediction modes of the reconstructed neighboring blocks of the current block to obtain the prediction mode sequence, such as mode 61,21,35,3,18,36. Calculate the residual blocks for the prediction blocks of these 6 modes according to the process in Figure 7 to obtain the residual block sequence 61,21,35,3,18,36. Then, calculate the cost to obtain the cost sequence [61,1111],[21,2123],[35,926],[3,875],[18,7854],[36,3542]. After sorting by cost, the prediction mode sequence 3,35,61,21,36,18 is obtained. Select the top 5 lowest costs as the most likely mode list, i.e., 3,35,61,21,36.

[0277] Optionally, if the encoding and / or decoding mode is an encoded and / or decoded block vector or motion vector (hereinafter referred to as vector), and the cost evaluation block is a residual block, then for each vector, the first cost of the residual block can be determined or obtained based on the cost of the elements in the current block, and at least one vector can be selected and / or sorted based on the first cost corresponding to each vector.

[0278] Optionally, the element within the current block can be a value at a specific position within the current block, such as a pixel value.

[0279] For example, let's take the decoding end as an example. As shown in Figure 8, the template region information of the current block is obtained, and K types of vectors are used to predict the template region, generating K types of predicted regions respectively. That is, predicting vector 1 yields predicted region 1, ..., and predicting vector K yields predicted region K. Optionally, the vector can be a block vector or a motion vector. Optionally, the decoding end cannot obtain the original pixels of the current block. Therefore, the decoding end usually finds a similar block / template, an approximate block / template, or a neighboring block / template for the current block to calculate the cost, which is denoted as the template region in this example. This template region has already been reconstructed.

[0280] Optionally, the K predicted regions are processed by the subtractor in Figure 8 to obtain K residual regions. That is, residual region 1 = reconstructed template region - predicted region 1, ..., residual region K = reconstructed template region - predicted region K. The cost calculation module in Figure 8 can be used to calculate the cost of each of the K residual regions, obtaining K cost values, such as cost 1 corresponding to residual region 1 and cost K corresponding to residual region K. The cost sorting module in Figure 8 sorts the K costs, for example, according to the magnitude of cost 1 to cost K, to obtain an ordered list [vector, cost]. The vector filtering module in Figure 8 filters the ordered list to obtain a list of most likely vectors. For example, selecting the top 5 vectors with the smallest cost yields the list of most likely vectors.

[0281] Optionally, if the encoding and / or decoding mode is an encoding and / or decoding filter, and the cost evaluation block is a residual block, then for each filter, the first cost of the residual block can be determined or obtained based on the cost of the elements in the current block, and at least one filter can be selected and / or sorted based on the first cost corresponding to each filter.

[0282] For example, let's take the decoding end as an example. As shown in Figure 9, the template region information of the current block is obtained, and K filters are applied to the template region to generate K filtered regions. That is, filtering with filter 1 yields filtered region 1, ..., filtering with filter K yields filtered region K. Optionally, the decoding end cannot obtain the original pixels of the current block. Therefore, the decoding end usually finds a similar block / template, an approximate block / template, or a neighboring block / template for the current block to calculate the cost, which is denoted as the cost of the current block in this example as the template region, which has already been reconstructed.

[0283] Optionally, the K filtering regions are processed by the subtractor in Figure 9 to obtain K residual regions. That is, residual region 1 = reconstruction template region - filtering region 1, ..., residual region K = reconstruction template region - filtering region K. The cost calculation module in Figure 9 can be used to calculate the cost of each of the K residual regions to obtain K cost values, such as cost 1 corresponding to residual region 1 and cost K corresponding to residual region K. The cost sorting module in Figure 9 sorts the K costs, for example, according to the magnitude of cost 1 to cost K, to obtain an ordered list [filter, cost]. The filter filtering module in Figure 9 filters the ordered list to obtain a list of most likely filters. For example, selecting the top 5 filters with the lowest cost yields the list of most likely filters.

[0284] Optionally, on the encoding side, a first cost of the block can be evaluated based on the cost of the current block, and the encoding mode of the current block can be selected and / or sorted.

[0285] Optionally, on the decoding side, the first cost of the block can be evaluated based on the cost of the current block, and the decoding mode of the current block can be selected and / or sorted.

[0286] In this embodiment, by evaluating the first cost of the current block based on the cost of the current block, the encoding and / or decoding modes of the current block can be selected and / or sorted. This allows for comprehensive consideration of the first cost of the current block when calculating the cost of entropy encoding at different stages of video encoding and / or decoding. This improves the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby supporting improved efficiency in video encoding and / or decoding.

[0287] Second Embodiment

[0288] Based on the first embodiment, a second embodiment of this application is proposed. In the second embodiment, the image processing method further includes at least one of the following methods twelve to sixteen:

[0289] Method 12: Cost calculation methods differ for cost evaluation elements at at least two different locations within the same cost evaluation block;

[0290] Optionally, different cost calculation methods can be pre-set. Different cost calculation methods can use different function calculation formulas. The function calculation formulas can include at least one of the following: Sum of absolute differences (SAD), Sum of absolute transformed differences (SATD), Sum of squared differences (SSD), Mean absolute difference (MAD), and Mean squared difference (MSD).

[0291] Optionally, the cost evaluation element can be a value at a location within the cost evaluation block.

[0292] Optionally, when the cost evaluation block is a pixel block of the current block, the cost evaluation element can be a pixel at any pixel position in the pixel block. Method 12 can be that the cost calculation methods of at least two pixels at different pixel positions in the same pixel block are different.

[0293] Optionally, when the cost evaluation block is the first difference block, the cost evaluation element can be an element in the first difference block, such as the difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode. Method 12 can be that the cost calculation methods of the differences at at least two different positions in the same first difference block are different.

[0294] Optionally, when the cost evaluation block is the second difference block, the cost evaluation element can be an element in the second difference block, such as the difference between image blocks corresponding to at least two candidate encoding and / or decoding modes. Alternatively, method 12 can be that the cost calculation methods for the differences at at least two different positions in the same second difference block are different.

[0295] Optionally, when the cost evaluation block is the third difference block, the cost evaluation element can be an element in the third difference block, such as the difference between the two cost evaluation blocks in the current block. Method 12 can be that the cost calculation methods of the difference at at least two different positions in the same third difference block are different.

[0296] Optionally, when the cost evaluation block is the fourth difference block, the cost evaluation element can be an element in the fourth difference block, such as the difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode. Alternatively, the cost calculation methods for the differences at at least two different positions in the same fourth difference block can be different.

[0297] Optionally, when the cost evaluation block is a residual block, the cost evaluation element can be a residual in the residual block. Method 12 can be that the cost calculation methods of residuals at at least two different positions in the same residual block are different.

[0298] Optionally, when the cost evaluation block is an absolute value block of the difference, the cost evaluation element can be the absolute value of the difference at a certain position in the absolute value block of the difference. Method 12 can be that the absolute values ​​of the differences at at least two different positions in the absolute value block of the same difference correspond to different cost calculation methods.

[0299] Optionally, when the cost evaluation block is a transformation block of the difference, the cost evaluation element can be the value of at least one position in the transformation block of the difference. Option twelve can be that the cost calculation methods corresponding to at least two different positions in the transformation block of the same difference are different.

[0300] Optionally, when the cost evaluation block is the absolute value block of the difference transformation block, the cost evaluation element can be the value of at least one position in the absolute value block of the difference transformation block. Alternatively, method 12 can be that the cost calculation methods corresponding to at least two different positions in the absolute value block of the same difference transformation block are different.

[0301] Optionally, when the cost evaluation block is a quantization block of the difference, the cost evaluation element can be the value at least one position in the quantization block of the difference. Option twelve can be that the cost calculation methods corresponding to the values ​​at at least two different positions in the quantization block of the same difference are different.

[0302] Optionally, when the cost evaluation block is the absolute value block of the quantization block of the difference, the cost evaluation element can be the value of at least one position in the absolute value block of the quantization block of the difference. Alternatively, method 12 can be that the cost calculation methods corresponding to at least two different positions in the absolute value block of the quantization block of the same difference are different.

[0303] Optionally, when determining the first cost of the cost evaluation block of the current block, the cost can be calculated according to the rule that the cost calculation methods of cost evaluation elements at least two different positions in the same cost evaluation block are different, so as to obtain the first cost of the current block, and the encoding and / or decoding modes of the current block can be selected and / or sorted according to the first cost.

[0304] Optionally, the cost calculation method may include at least one of the following cost calculation methods one through five:

[0305] Cost calculation method one: determine or obtain the first cost of the cost evaluation block based on the second cost of at least one cost evaluation element within the cost evaluation block;

[0306] The second method of cost calculation involves non-uniform processing of the cost evaluation block to obtain a non-uniform block, and determining or obtaining the first cost of the cost evaluation block based on the third cost of at least one non-uniform block.

[0307] Cost calculation method three: determine or obtain the first cost of the cost evaluation block based on the fourth cost of at least one element of the transformed block after the cost evaluation block has been transformed;

[0308] Cost calculation method four: Determine or obtain the first cost of the cost evaluation block based on the fifth cost of at least one element in the quantized block after quantizing the cost evaluation block.

[0309] Cost calculation method five: determine or obtain the first cost of the cost evaluation block based on at least one neural network and / or cost lookup table.

[0310] Alternatively, the cost calculation can be performed in other ways, such as directly using a function calculation formula, etc., without any restrictions.

[0311] Optionally, Method Twelve may also include: the weights of cost evaluation elements at at least two different locations within the same cost evaluation block are calculated in different ways.

[0312] For example, when calculating the cost of a cost evaluation block, the weight calculation method for the cost evaluation element at one position in the same cost evaluation block is determined by using a lookup table, while the weight calculation method for the cost evaluation element at another position is determined by using a neural network.

[0313] Optionally, the weights of cost evaluation elements at at least two different positions in the same cost evaluation block are different, and / or the weights of cost evaluation elements at at least two different positions in the same cost evaluation block are the same.

[0314] In this approach, by comprehensively considering cost calculation rules when determining the first cost of a cost evaluation block, such as the different cost calculation methods for cost evaluation elements at at least two different positions in the same cost evaluation block, the cost evaluation elements can be taken into account when determining the cost calculation, thereby improving the accuracy of the determined first cost.

[0315] Method 13: The cost calculation methods for at least two cost evaluation blocks with different components are different;

[0316] Optionally, the cost evaluation block of at least two different components can be a cost evaluation block of at least two different components among the Y component, U component and V component.

[0317] For example, the calculation method corresponding to the cost evaluation block of the current block of the Y component can be different from the calculation method corresponding to the cost evaluation block of the current block of the U component, or it can be different from the calculation method corresponding to the cost evaluation block of the current block of the V component.

[0318] Optionally, the cost calculation methods for two different cost evaluation blocks of the same component can be different.

[0319] Optionally, when determining the first cost of the cost evaluation block of the current block, the cost can be calculated according to the rule that the cost calculation methods of at least two cost evaluation blocks of different components are different, to obtain the first cost of the current block, and the encoding and / or decoding modes of the current block can be selected and / or sorted according to the first cost.

[0320] Optionally, Method 13 may include a cost calculation method of at least one component of the cost evaluation block being cost calculation method A, and a cost calculation method of at least another component of the cost evaluation block being cost calculation method B.

[0321] Optionally, cost calculation method A and cost calculation method B are different, and cost calculation method A and / or cost calculation method B can be at least one of cost calculation methods one to five described in method twelf above.

[0322] Optionally, method thirteen may also include: at least two cost evaluation blocks of different components having different weight calculation methods.

[0323] For example, the weight calculation method for the cost evaluation block of at least one component can be determined by using a lookup table, while the weight calculation method for the cost evaluation block of another component can be determined by using a neural network.

[0324] Optionally, the weights corresponding to at least two different cost evaluation blocks can be the same or different.

[0325] In this approach, by comprehensively considering cost calculation rules when determining the first cost of a cost evaluation block, such as the different cost calculation methods for at least two cost evaluation blocks with different components, the impact of different block components on cost calculation can be taken into account when determining the cost calculation, thereby improving the accuracy of the determined first cost.

[0326] Method Fourteen: The cost calculation methods differ for at least two cost evaluation blocks with different size parameters;

[0327] Optionally, the size parameters may include the width, height, perimeter, and area of ​​the cost evaluation block.

[0328] Optionally, when determining the first cost of the cost evaluation block for the current block, the cost can be calculated based on the rule that the cost calculation methods of at least two cost evaluation blocks with different size parameters are different, to obtain the first cost of the current block, and the encoding and / or decoding modes of the current block can be selected and / or sorted based on the first cost.

[0329] Optionally, Method Fourteen may include a cost calculation method of at least one cost evaluation block of a size parameter that can be cost calculation method C, and a cost calculation method of at least another cost evaluation block of a size parameter that can be cost calculation method D.

[0330] Optionally, cost calculation method C and cost calculation method D are different, and cost calculation method C and / or cost calculation method D can be at least one of cost calculation methods one to five described in method twelf above.

[0331] Optionally, method fourteen may also include: the weights of at least two cost evaluation blocks with different size parameters are calculated in different ways.

[0332] For example, the weight calculation method for the cost evaluation block of at least one size parameter can be determined by using a lookup table, while the weight calculation method for the cost evaluation block of another size parameter can be determined by using a neural network.

[0333] Optionally, the weights corresponding to at least two cost evaluation blocks with different size parameters can be the same or different.

[0334] In this approach, by comprehensively considering cost calculation rules when determining the first cost of a cost evaluation block, such as the different cost calculation methods for at least two cost evaluation blocks with different size parameters, the influence of different block size parameters on cost calculation can be taken into account when determining the cost calculation, thereby improving the accuracy of the determined first cost.

[0335] Method 15: The cost calculation methods for at least two cost evaluation blocks with different texture features are different;

[0336] Optionally, texture features can reflect the microscopic properties of the cost evaluation block surface, such as roughness, smoothness, spots, lines, etc.

[0337] Optionally, when determining the first cost of the cost evaluation block of the current block, the cost can be calculated according to the rule that the cost calculation methods of at least two cost evaluation blocks with different texture features are different, to obtain the first cost of the current block, and the encoding and / or decoding modes of the current block can be selected and / or sorted according to the first cost.

[0338] Optionally, method 15 may include a cost calculation method of at least one cost evaluation block of a texture feature that can be cost calculation method E, and a cost calculation method of at least another cost evaluation block of a texture feature that can be cost calculation method F.

[0339] Optionally, cost calculation method E and cost calculation method F are different, and cost calculation method E and / or cost calculation method F can be at least one of cost calculation methods one to five described in method twelf above.

[0340] Optionally, method 15 may also include: the weights of at least two cost evaluation blocks with different texture features are calculated in different ways.

[0341] For example, the weight calculation method for at least one cost evaluation block of a texture feature can be determined by using a lookup table, while the weight calculation method for another cost evaluation block of a texture feature can be determined by using a neural network.

[0342] Optionally, the weights corresponding to the cost evaluation blocks of at least two different texture features can be the same or different.

[0343] In this approach, by comprehensively considering cost calculation rules when determining the first cost of a cost evaluation block, such as the different cost calculation methods for at least two cost evaluation blocks with different texture features, the influence of different texture features of the blocks on the cost calculation can be taken into account when determining the cost calculation, thereby improving the accuracy of the determined first cost.

[0344] Method 16: Classify the cost assessment blocks, with at least two different categories of cost assessment blocks having different cost calculation methods.

[0345] Optionally, cost evaluation blocks can be classified according to different classification rules. The cost calculation methods corresponding to different categories of cost evaluation blocks can be different, that is, the cost calculation methods of at least two different categories of cost evaluation blocks are different.

[0346] Optionally, the image processing method further includes classifying the cost evaluation blocks according to at least one of the following methods 1 to 6:

[0347] Method 1: The sum of the absolute values ​​of the cost evaluation elements in the cost evaluation block;

[0348] Optionally, cost evaluation blocks can be classified based on the sum of the absolute values ​​of the cost evaluation elements in the cost evaluation block, and the cost calculation methods of at least two different categories of cost evaluation blocks are different.

[0349] For example, if the sum of the absolute values ​​of the cost evaluation elements in the cost evaluation block is within the first interval or greater than the first preset threshold, the cost evaluation block can be classified as the first category; and / or, if the sum of the absolute values ​​of the cost evaluation elements in the cost evaluation block is within the second interval, which is different from the first interval, or is less than or equal to the first preset threshold, the cost evaluation block can be classified as the second category.

[0350] Optionally, the first interval range can be a pre-set interval range, such as 0-100 or 0-50, etc.

[0351] Optionally, the second interval range can be a pre-set interval range that is different from the first interval range. For example, if the first interval range is 0-100, the second interval range can be 101-150, or 150-200, etc.

[0352] Optionally, the first preset threshold can be a pre-set threshold, such as a pixel value of 100 or a pixel value of 50.

[0353] Optionally, the first and second categories can be two different block categories.

[0354] In this approach, when classifying cost evaluation blocks, the classification can be based on the cumulative absolute values ​​of the cost evaluation elements within the cost evaluation block. This allows for comprehensive consideration of cost evaluation elements during the classification process, improving the accuracy of cost evaluation block classification. Consequently, when calculating the cost of different categories of cost evaluation blocks, the influence of cost evaluation elements can be comprehensively considered, thus improving the accuracy of the determined first cost.

[0355] Method 2: Feature complexity of the cost evaluation block;

[0356] Optionally, feature complexity includes at least one of texture density (e.g., the complexity of texture features in the image patch corresponding to the cost evaluation block) and residual variability.

[0357] Optionally, cost evaluation blocks can be classified according to their characteristic complexity, and at least two different categories of cost evaluation blocks can have different cost calculation methods.

[0358] For example, if the feature complexity of the cost evaluation block is greater than a preset feature complexity threshold, the cost evaluation block can be classified as the third category; and / or, if the feature complexity of the cost evaluation block is less than or equal to the preset feature complexity threshold, the cost evaluation block can be classified as the fourth category.

[0359] Optionally, the preset feature complexity threshold can be a pre-set feature complexity threshold, such as setting a moderate level of texture density as the preset feature complexity threshold, or setting a moderate level of residual fluctuation as the preset feature complexity threshold.

[0360] Optionally, the texture density of the cost evaluation block can be measured by the variance, standard deviation, or other parameters of the elements in the cost evaluation block. For example, a variance of 10 or a standard deviation of 10 can be set to approximate a moderate level of texture density and used as a preset feature complexity threshold.

[0361] Optionally, the gradient values ​​of all elements within the cost evaluation block, excluding boundary points, in the four directions of horizontal, vertical, and two diagonals can be calculated to obtain the maximum gradient. Then, a gradient matrix can be constructed based on the maximum gradient and the cost evaluation block, and the sum of all coefficients in the gradient matrix can be used as an evaluation index of texture density.

[0362] For example, if the sum of all coefficients in the gradient matrix is ​​200, it can be considered as a medium level of texture density and used as a preset feature complexity threshold.

[0363] Optionally, the residual fluctuation of the cost evaluation block can be approximated by the value of the residual element in the residual block corresponding to the cost evaluation block. For example, the closer the value of the residual element is to 0, the smaller the residual fluctuation.

[0364] Optionally, the value of the residual element in the residual block corresponding to the cost evaluation block can be set to 5 as a preset feature complexity threshold, or the value of the residual element in the residual block corresponding to the cost evaluation block can be set to 10 as a preset feature complexity threshold.

[0365] Optionally, the third and fourth categories can be two different block categories.

[0366] In this approach, when classifying cost evaluation blocks, the classification can be based on the feature complexity of the cost evaluation blocks. This allows for comprehensive consideration of feature complexity during the classification operation, improving the accuracy of cost evaluation block classification. Consequently, when calculating the cost of different categories of cost evaluation blocks, the impact of feature complexity can be comprehensively considered, thus improving the accuracy of the determined first cost.

[0367] Method 3, the first characteristic value of the cost evaluation block;

[0368] Optionally, the first characteristic value may include at least one of the following: the sign ratio of the cost evaluation elements in the cost evaluation block, the root mean, and the mean.

[0369] Optionally, cost evaluation blocks can be classified based on their first characteristic value, and the cost calculation methods for at least two different categories of cost evaluation blocks can be different.

[0370] For example, if the first feature value of the cost evaluation block is greater than the preset feature threshold, the cost evaluation block can be classified as the fifth category; and / or, if the first feature value of the cost evaluation block is less than or equal to the preset feature threshold, the cost evaluation block can be classified as the sixth category.

[0371] Optionally, when the first feature value is the positive and negative sign ratio of the cost evaluation element, the preset feature threshold can be a threshold for the positive and negative sign ratio, such as 50%, 30%, etc.; when the first feature value is the mean squared error, the preset feature threshold can be a mean squared error with a value of 10 or a mean squared error with a value of 20; when the first feature value is the mean, the preset feature threshold can be a mean with a value of 10 or a mean with a value of 16, etc.

[0372] Alternatively, the fifth and sixth categories can be two different block categories.

[0373] In this approach, when classifying cost evaluation blocks, the first characteristic value of the cost evaluation block can be used for classification. This allows the first characteristic value to be considered comprehensively during the classification operation, improving the accuracy of cost evaluation block classification. Consequently, when calculating the cost of different categories of cost evaluation blocks, the influence of the first characteristic value can be considered comprehensively, thus improving the accuracy of the determined first cost.

[0374] Method 4, size parameters of the cost evaluation block;

[0375] Optionally, cost evaluation blocks can be classified according to their size parameters, and the cost calculation methods for at least two different categories of cost evaluation blocks can be different.

[0376] For example, if the size parameters of the cost evaluation block are within the preset size parameter range, the cost evaluation block can be classified as category seven; and / or, if the size parameters of the cost evaluation block are not within the preset size parameter range, the cost evaluation block can be classified as category eight.

[0377] Optionally, the size parameters of the cost evaluation block may include the width, height, area, and perimeter of the cost evaluation block.

[0378] Optionally, the preset size parameter range can be a pre-set size parameter range for the cost evaluation block, such as a width range of 0-16 and a height range of 0-16.

[0379] Alternatively, the seventh and eighth categories can be two different block categories.

[0380] In this approach, when classifying cost evaluation blocks, the size parameters of the cost evaluation blocks can be used for classification. This allows for comprehensive consideration of size parameters during the classification process, improving the accuracy of cost evaluation block classification. Consequently, when calculating the cost of different categories of cost evaluation blocks, the influence of size parameters can be comprehensively considered, thus improving the accuracy of the determined first cost.

[0381] Method 5: The sum of the absolute values ​​of the transform elements in the transform block;

[0382] Optionally, cost evaluation blocks can be classified based on the sum of the absolute values ​​of the transformation elements in the transformation block corresponding to the cost evaluation block, and the cost calculation methods of at least two different categories of cost evaluation blocks are different.

[0383] For example, if the sum of the absolute values ​​of the transformation elements in the transformation block corresponding to the cost evaluation block is within the third interval or greater than the second preset threshold, the cost evaluation block can be classified as the ninth category; and / or, if the sum of the absolute values ​​of the transformation elements in the transformation block corresponding to the cost evaluation block is within the fourth interval, which is different from the third interval, or less than or equal to the second preset threshold, the cost evaluation block can be classified as the tenth category.

[0384] Optionally, the third interval range can be a pre-set range, such as 0-100 or 101-150.

[0385] Optionally, the fourth interval range can be a pre-set interval range that is different from the third interval range. For example, if the third interval range is 0-100, then the second interval range can be 101-150, or 170-200, etc.

[0386] Optionally, the second preset threshold can be a pre-set threshold, such as a pixel value of 100 or a pixel value of 50.

[0387] Alternatively, Class 9 and Class 10 can be two different block categories.

[0388] In this approach, when classifying cost evaluation blocks, the classification can be based on the cumulative absolute values ​​of the transformation elements in the transformation blocks of the cost evaluation blocks. This allows for comprehensive consideration of the cumulative absolute values ​​of the transformation elements in the transformation blocks during the classification operation, improving the accuracy of cost evaluation block classification. Consequently, when calculating the cost of different categories of cost evaluation blocks, the influence of the cumulative absolute values ​​of the transformation elements in the transformation blocks can be comprehensively considered, thus improving the accuracy of the determined first cost.

[0389] Method 6: The sum of the absolute values ​​of the quantized elements in the quantized block.

[0390] Optionally, cost evaluation blocks can be classified based on the sum of the absolute values ​​of the quantized elements in the quantized block corresponding to the cost evaluation block, and the cost calculation methods of at least two different categories of cost evaluation blocks are different.

[0391] For example, if the sum of the absolute values ​​of the quantized elements in the quantization block corresponding to the cost evaluation block is within the fifth interval or greater than the third preset threshold, the cost evaluation block can be classified as the eleventh category; and / or, if the sum of the absolute values ​​of the quantized elements in the quantization block corresponding to the cost evaluation block is within the sixth interval, which is different from the fifth interval, or less than or equal to the third preset threshold, the cost evaluation block can be classified as the twelfth category.

[0392] Optionally, the fifth interval range can be a pre-set range, such as 0-80 or 101-150.

[0393] Optionally, the sixth interval range can be a pre-set interval range that is different from the fifth interval range. For example, if the fifth interval range is 0-80, then the sixth interval range can be 81-150, or 170-200, etc.

[0394] Optionally, the third preset threshold can be a pre-set threshold, such as a pixel value of 80 or a pixel value of 50.

[0395] Alternatively, Class 11 and Class 12 can be two different block categories.

[0396] In this approach, when classifying cost evaluation blocks, the classification can be based on the cumulative absolute values ​​of the quantized elements in the quantized blocks. This allows for comprehensive consideration of the cumulative absolute values ​​of the quantized elements in the quantized blocks during the classification process, improving the accuracy of cost evaluation block classification. Consequently, when calculating the cost of different categories of cost evaluation blocks, the cumulative absolute values ​​of the quantized elements in the quantized blocks can be comprehensively considered, thus improving the accuracy of the determined first cost.

[0397] Optionally, the cost evaluation block can be classified according to at least one of methods 1 to 6, or it can be classified in other ways besides methods 1 to 6, such as using a preset classification function. No restrictions are imposed here.

[0398] Optionally, method sixteen may include classifying cost evaluation blocks, wherein the cost calculation method of at least one category of cost evaluation blocks may be cost calculation method G, and the cost calculation method of at least another category of cost evaluation blocks may be cost calculation method H.

[0399] Optionally, the cost calculation method G and the cost calculation method H are different, and the cost calculation method G and / or the cost calculation method H can be at least one of the cost calculation methods one to five described in method twelf above.

[0400] Optionally, method sixteen may also include: classifying cost evaluation blocks, with at least two different categories of cost evaluation blocks having different weight calculation methods.

[0401] For example, the weight calculation method for at least one category of cost evaluation blocks can be determined using a lookup table, and the weight calculation method for at least another category of cost evaluation blocks can be determined using a neural network.

[0402] Optionally, the weights corresponding to at least two different categories of cost evaluation blocks can be the same or different.

[0403] Optionally, when determining the first cost of the cost evaluation block of the current block, the cost can be calculated based on the rule that the cost evaluation blocks are classified and that the cost calculation methods of at least two different categories of cost evaluation blocks are different, so as to obtain the first cost of the current block, and the encoding and / or decoding modes of the current block can be selected and / or sorted based on the first cost.

[0404] In this embodiment, by comprehensively considering cost calculation rules when determining the first cost of a cost evaluation block, such as classifying cost evaluation blocks, and ensuring that the cost calculation methods differ for at least two different categories of cost evaluation blocks, the influence of different block categories on cost calculation can be taken into account when determining the cost calculation. This improves the accuracy of the determined first cost, thereby improving the accuracy of sorting or filtering multiple encoding and / or decoding modes, and ultimately supporting the improvement of video encoding and / or decoding efficiency.

[0405] Third Embodiment

[0406] Based on any of the above embodiments, a third embodiment of this application is proposed. In the third embodiment, the first cost of the cost evaluation block is determined or obtained according to at least one of the following methods seventeen to twenty-one:

[0407] Method 17: The second cost of at least one cost evaluation element within the cost evaluation block;

[0408] Optionally, when determining the first cost of the cost evaluation block of the current block, the cost calculation method corresponding to the cost evaluation block of the current block can be determined according to at least one of the methods twelve to sixteen described in the second embodiment. If the cost calculation method is cost calculation method one, the second cost of at least one cost evaluation element in the cost evaluation block of the current block can be determined first, and the first cost of the cost evaluation block can be determined or obtained according to the second cost of at least one cost evaluation element in the cost evaluation block. Based on the first cost of the cost evaluation block, the encoding and / or decoding modes of the current block can be selected and / or sorted.

[0409] Optionally, the image processing method further includes at least one of methods 7 to 13:

[0410] Method 7: If the cost evaluation element is within the first range, then the second cost is determined or obtained based on the cost evaluation element and the first weight.

[0411] Optionally, the first range can be a pre-set range, such as a pixel range, like 0-100.

[0412] Optionally, the first weight can be a weight that can be determined according to the weight calculation method, or it can be a default weight that is set in advance, such as 0.5, etc. There is no restriction here.

[0413] Optionally, the weight calculation method can be determined or obtained according to at least one of the methods 12 to 16 described in the second embodiment. For example, the weight calculation methods of cost evaluation elements at at least two different positions in the same cost evaluation block are different.

[0414] Optionally, a cost evaluation element at least one position within the cost evaluation block of the current block can be determined. If the cost evaluation element is located within a first range, a second cost can be determined or obtained based on the cost evaluation element and the first weight.

[0415] Optionally, if the absolute value of at least one cost evaluation element is within a first range, a second cost can be determined or obtained based on the cost evaluation element and the first weight.

[0416] Optionally, the cost evaluation element and the first weight can be input into the corresponding function calculation formula (such as the sum of absolute errors, the sum of absolute transformation errors, the sum of squared differences, etc.) to calculate the cost and obtain the second cost of at least one cost evaluation element.

[0417] In this approach, when the cost evaluation element is within the first range in the cost evaluation block, the second cost is determined or obtained based on the cost evaluation element and the first weight. This reflects that when calculating the cost of the cost evaluation element, the size of the cost evaluation element is comprehensively considered, which improves the effectiveness of the determined second cost and thus improves the effectiveness of the determined or obtained first cost of the cost evaluation block.

[0418] Method 8: If the cost evaluation element is located in a second range different from the first range, then the second cost is determined or obtained based on the second weight different from the first weight and the cost evaluation element.

[0419] Optionally, the second range can be a pre-set range that is different from the first range. For example, if the first range is 0-100, the second range can be 101-200.

[0420] Optionally, the second weight can be a weight calculated according to the weight calculation method, or it can be a default weight set in advance. There are no restrictions here. For example, if the first weight is 0.3, the second weight can be 0.7.

[0421] Optionally, the first weight and the second weight are different. The first weight and the second weight can be calculated using two different weight calculation methods. The two different weight calculation methods can be determined or obtained according to at least one of the methods 12 to 16 described in the second embodiment. For example, the weight calculation methods of cost evaluation elements at at least two different positions in the same cost evaluation block are different.

[0422] Optionally, a cost evaluation element at least one position within the cost evaluation block of the current block can be determined. If the cost evaluation element is located in a second range different from the first range, a second cost is determined or obtained based on the second weight different from the first weight and the cost evaluation element.

[0423] Optionally, if the absolute value of at least one cost evaluation element is within the second range, the second cost can be determined or obtained based on the second weight and the cost evaluation element.

[0424] Optionally, the cost evaluation element and the second weight can be input into the corresponding function calculation formula to calculate the cost, thereby obtaining the second cost corresponding to at least one cost evaluation element.

[0425] In this approach, when the cost evaluation element is located within the second range in the cost evaluation block, the second cost is determined or obtained based on the cost evaluation element and the second weight. This reflects that when calculating the cost of the cost evaluation element, the size of the cost evaluation element is comprehensively considered, which improves the effectiveness of the determined second cost and thus improves the effectiveness of the determined or obtained first cost of the cost evaluation block.

[0426] Method 9: If the cost evaluation element is located in a third range that is different from both the first and second ranges, then the second cost is determined or obtained based on at least one neural network and / or a cost lookup table, and the cost evaluation element.

[0427] Optionally, the third range can be a pre-set range that is different from both the first and second ranges. For example, the first range is 0-80, the second range is 81-150, and the third range can be 151-200.

[0428] Optionally, the cost lookup table includes the mapping between indexes and costs.

[0429] Optionally, a cost evaluation element at least one position within the cost evaluation block of the current block can be determined. If the cost evaluation element is located in the third range, the cost evaluation element can be input into the neural network to obtain the second cost.

[0430] Optionally, a cost evaluation element at least one position within the cost evaluation block of the current block can be determined. If the cost evaluation element is located in the third range, the index corresponding to the cost evaluation element can be determined and the index can be entered into the cost lookup table for searching to obtain the second cost.

[0431] Optionally, a cost evaluation element at least one position within the cost evaluation block of the current block can be determined. If the cost evaluation element is located in the third range, the cost evaluation element can be input into the neural network, the index corresponding to the output result of the neural network can be determined, and the index can be input into the cost lookup table for lookup to obtain the second cost.

[0432] Optionally, a cost evaluation element at least one position within the cost evaluation block of the current block can be determined. If the cost evaluation element is located in the third range, the index corresponding to the cost evaluation element can be input into the cost lookup table for searching. The search result in the cost lookup table is input into the neural network, and the second cost is output.

[0433] In this approach, when the cost evaluation element is located in the third range within the cost evaluation block, a second cost is determined or obtained based on at least one neural network and / or lookup table, and the cost evaluation element. This reflects that when calculating the cost of the cost evaluation element, the size of the cost evaluation element is comprehensively considered, thereby improving the effectiveness of the determined second cost and thus improving the effectiveness of the determined or obtained first cost of the cost evaluation block.

[0434] Method 10: If the cost evaluation element is in the fourth range, then the second cost is determined or obtained based on the absolute value of at least one cost evaluation element.

[0435] Optionally, the fourth range can be a pre-set range that is different from the first, second, and third ranges. For example, the first range is 0-80, the second range is 81-150, the third range is 151-200, and the fourth range can be 201-255.

[0436] Optionally, a cost evaluation element at least one position within the cost evaluation block of the current block can be determined. If the cost evaluation element is located in the fourth range, the absolute value of the cost evaluation element can be input into the corresponding function calculation formula to calculate the cost and obtain the second cost corresponding to the cost evaluation element.

[0437] In this method, when the cost evaluation element is located in the fourth range within the cost evaluation block, the second cost is determined or obtained based on the absolute value of at least one cost evaluation element. This reflects that when calculating the cost of the cost evaluation element, the size of the cost evaluation element is comprehensively considered, which improves the effectiveness of the determined second cost and thus improves the effectiveness of the determined or obtained first cost of the cost evaluation block.

[0438] Method 11: If the cost evaluation block satisfies the first condition, then the second cost is determined or obtained according to the first parameter table;

[0439] Optionally, the first parameter table may include at least one first parameter, which may be a parameter required to determine the second cost of the cost evaluation element, such as a weight when performing cost calculation.

[0440] Optionally, the first parameter table can be a weight table containing the mapping relationship between weight indices and weights.

[0441] Optionally, the first parameter table may be a weight table containing the mapping relationship between the cost evaluation block element values ​​and the weights.

[0442] Optionally, satisfying the first condition may be: the size parameter of the cost evaluation block is greater than a preset size parameter threshold (e.g., width x height is 16x16, etc.), the first feature value of the cost evaluation block is greater than a preset feature threshold (e.g., the positive and negative sign ratio is 50%, etc.), etc., without any restrictions.

[0443] Optionally, if the cost evaluation block satisfies the first condition, the second cost can be determined or obtained based on at least one cost evaluation element in the current cost evaluation block and the first parameter table.

[0444] For example, when the cost evaluation block satisfies the first condition, the weight index corresponding to at least one cost evaluation element can be determined, and the weight can be searched in the first parameter table according to the weight index to obtain the weight. The function calculation formula corresponding to at least one cost evaluation element can be determined, and the weight obtained from the weight search and at least one cost evaluation element can be input into the function calculation formula to calculate the cost and obtain the second cost.

[0445] In this approach, when the cost evaluation block meets the first condition, the second cost is determined or obtained according to the first parameter table. This means that when calculating the cost of the cost evaluation elements, the first parameter table needs to be used in conjunction with the cost calculation to make the determined second cost more accurate.

[0446] Method 12: If the cost evaluation block does not meet the first condition, then the second cost is determined or obtained according to the second parameter table, which is different from the first parameter table.

[0447] Optionally, the second parameter table may contain at least one second parameter, which may be a parameter required to determine the second cost of the cost evaluation element, such as a weight when performing cost calculation.

[0448] Optionally, the first parameter and the second parameter are different. For example, the first parameter is a weight, and the weight is at least one of 0.1-0.5, while the second parameter is a weight, and the weight can be at least one of 0.6-0.9.

[0449] Optionally, the first condition may not be met if: the size parameter of the cost evaluation block is less than or equal to a preset size parameter threshold (e.g., width x height is 16x16), or the first feature value of the cost evaluation block is less than or equal to a preset feature threshold (e.g., the positive and negative sign ratio is 50%), etc., without any restrictions.

[0450] Optionally, if the cost evaluation block does not meet the first condition, the second cost can be determined or obtained based on at least one cost evaluation element and the second parameter table in the current cost evaluation block.

[0451] For example, when the cost evaluation block does not meet the first condition, the weight index corresponding to at least one cost evaluation element can be determined, and the weight can be searched in the second parameter table according to the weight index to obtain the weight. The function calculation formula corresponding to at least one cost evaluation element can be determined, and the weight obtained from the weight search and at least one cost evaluation element can be input into the function calculation formula to calculate the cost and obtain the second cost.

[0452] In this approach, when the cost evaluation block does not meet the first condition, the second cost is determined or obtained according to the second parameter table. This means that when calculating the cost of the cost evaluation elements, the second parameter table needs to be used in conjunction with the cost calculation to make the determined second cost more accurate.

[0453] Method 13: Determine or obtain the second cost based on the third parameter table.

[0454] Optionally, the third parameter table may contain at least one third parameter, which may be a parameter required to determine the second cost of the cost evaluation element, such as a weight when performing cost calculation.

[0455] Optionally, the third parameter may include the first parameter and / or the second parameter.

[0456] Optionally, when calculating costs, the second cost of at least one cost evaluation element can be determined or obtained directly from the third parameter table.

[0457] Optionally, a weight index corresponding to at least one cost evaluation element can be determined, and a weight lookup can be performed in the third parameter table based on the weight index to obtain the weight. A function calculation formula corresponding to at least one cost evaluation element can be determined, and the weight obtained from the weight lookup and at least one cost evaluation element can be input into the function calculation formula to calculate the cost and obtain the second cost.

[0458] For example, as shown in Figure 10, if the cost evaluation block is a residual block R, the cost evaluation element is a residual element ri, and the parameter table is a weight table, such as the prediction residual block in video coding, which contains multiple residual elements ri.

[0459] Optionally, the residual block R can be a prediction residual block in video coding, containing residual elements ri and residual block size parameters, such as block size attributes, such as 8x8, 16x16, etc. The residual block R can be scanned line by line to extract the absolute value |ri| of each residual element ri, and at the same time obtain the size parameters of the residual block R, such as the block width x height being 4x4, 16x16 or 8x8.

[0460] Optionally, residual blocks R with width x height greater than 16x16 can be processed as large-size residual blocks, residual blocks R with size parameters greater than 8x8 and less than or equal to 16x16 can be processed as medium-size residual blocks, and residual blocks R with size parameters less than or equal to 8x8 can be processed as small-size residual blocks.

[0461] Optionally, the size parameter of the residual block R can be judged. If the size parameter of the residual block R is greater than 16x16, the weight parameter in weight table 1 is used for cost calculation; and / or, if not, the size parameter of the residual block R is further checked to see if it is greater than 8x8; if so, the weight parameter in weight table 2 is used for cost calculation; and / or, if not, the weight parameter in weight table 3 is used for cost calculation.

[0462] Optionally, for residual block R, the weight query module performs a weight query from weight table 1 / 2 / 3 based on the absolute value |ri| of residual element ri and / or the index (e.g., index i) / position of residual element in residual block R, and obtains the weight coefficient w. That is, based on the size parameter of residual block R, the weight query module queries the weight coefficient w corresponding to index i in weight table 1, weight table 2 or weight table 3. Index i is the index corresponding to residual element ri. Then, the weight is calculated by weighting w*|ri| to obtain the second cost of residual element. Finally, the results of processing all residual elements and block size are summed (∑) to obtain the final block cost (i.e., the first cost of cost evaluation block).

[0463] For example, as shown in Figure 11, if the cost evaluation block is a residual block R, the cost evaluation element is a residual element ri, and the parameter table is a weight table, such as the prediction residual block in video coding, which contains multiple residual elements ri.

[0464] Optionally, the residual block R can be a prediction residual block in video coding, containing residual elements ri. The residual block R can be scanned line by line to extract the absolute value |ri| of each residual element ri, and the mean square error of the residual block R can be calculated using a mean square error module. μ is the mean value of the residual block R, and B is the size parameter of the residual block R, such as width x height as H x W.

[0465] Optionally, residual blocks R with a mean square error greater than 100 can be processed as high-complexity blocks, which means that residual blocks R with dense textures and large residual fluctuations can be processed as high-complexity blocks; residual blocks R with a mean square error less than or equal to 100 and greater than 25 can be processed as medium-complexity blocks, which means that residual blocks R with edges and medium textures can be processed as medium-complexity blocks; and / or, residual blocks R with a mean square error less than or equal to 25 can be processed as low-complexity blocks, which means that residual blocks R in the equilibrium region with small and uniform residuals can be processed as low-complexity blocks.

[0466] Optionally, the mean squared error of the residual block R can be judged. If the mean squared error of the residual block R is greater than 100, the cost is calculated using the weight parameters in weight table 1; and / or, if not, the mean squared error of the residual block R is further checked to see if it is greater than 25. If so, the cost is calculated using the weight parameters in weight table 2; and / or, if not, the cost is calculated using the weight parameters in weight table 3.

[0467] Optionally, for residual block R, the weight query module performs a weight query from weight table 1 / 2 / 3 based on the absolute value |ri| of residual element ri and / or the index (e.g., index i) / position of residual element in residual block R. That is, based on the mean square error of residual block R, the weight query module queries the weight coefficient w corresponding to index i in weight table 1, weight table 2 or weight table 3. Index i is the index corresponding to residual element ri. Then, weighted processing is performed, i.e. w*|ri| to obtain the second cost of residual element. Then, the results of processing all residual elements and block size are summed (∑) to obtain the final block cost (i.e. the first cost of cost evaluation block).

[0468] In this embodiment, by determining or obtaining the second cost based on the third parameter table, it can be reflected that when calculating the cost of the cost evaluation elements, it is necessary to combine the third parameter table to calculate the cost, so that the determined second cost is more accurate.

[0469] Optionally, for method seventeen, the second cost of at least one cost evaluation element in the cost evaluation block can be determined or obtained according to at least one of methods 7 to 13, or the second cost of at least one cost evaluation element in the cost evaluation block can be determined or obtained according to other methods, without limitation.

[0470] Optionally, the first cost of the cost evaluation block can be determined or obtained based on the second cost of at least one cost evaluation element in the current block. For example, the second cost of at least one cost evaluation element in the current block can be accumulated to obtain the first cost of the cost evaluation block. Alternatively, the absolute value of the second cost of at least one cost evaluation element in the current block can be accumulated to obtain the first cost of the cost evaluation block. Or, the second cost of at least one cost evaluation element in the current block can be input into the neural network and the first cost of the cost evaluation block can be output. There are no restrictions on this.

[0471] For example, as shown in Figure 12, if the cost evaluation block is a residual block R, and the cost evaluation element is a residual element ri, such as the prediction residual block in video coding, which contains multiple residual elements ri.

[0472] Optionally, the residual block R can be scanned row by row to extract the absolute value |ri| of each residual element ri, and the absolute value |ri| of each residual element ri can be processed in turn to check whether |ri| is greater than T2. ​​If it is, a table lookup operation is performed. If not, it can be checked whether |ri| is greater than T1. If it is, an identity operation is performed. If not, a weighted operation is performed.

[0473] Optionally, T1 and T2 are two different thresholds set in advance, and T2 is greater than T1.

[0474] Optionally, a lookup table operation can be performed: the corresponding value (such as the second cost corresponding to the residual element) can be obtained through a predefined lookup table LUT(|ri|). For example, the absolute value |ri| of the residual element ri can be nonlinearly mapped to optimize the complexity or accuracy of cost calculation.

[0475] Optionally, the identity operation can be performed by directly using the value of |ri| without weighting, or by setting the weight to 1 to preserve the original residual contribution.

[0476] Optionally, a weighted operation is performed: w*|ri| is calculated, where w is the weight coefficient, which can be determined based on the actual optimization results.

[0477] Optionally, at least one of the following operations can be performed on all elements in the residual block R: table lookup, identity, and weighting, and then summed (∑) to obtain the final block cost, which is the first cost of the cost evaluation block.

[0478] In this method, the first cost of the cost evaluation block is determined or obtained based on the second cost of at least one cost evaluation element within the cost evaluation block. This allows for the comprehensive consideration of the second cost of more granular cost evaluation elements when determining the cost of the cost evaluation block, thereby improving the accuracy of the first cost of the determined or obtained cost evaluation block.

[0479] Method 18, the third cost of at least one non-uniform block;

[0480] Alternatively, the non-uniform block is obtained by non-uniformizing the cost evaluation block.

[0481] Optionally, the non-uniformity weights may be different at at least two different locations in the non-uniform block.

[0482] Optionally, when determining the first cost of the cost evaluation block of the current block, the cost calculation method corresponding to the cost evaluation block of the current block can be determined according to at least one of the methods twelve to sixteen described in the second embodiment. If the cost calculation method is cost calculation method two, the cost evaluation block can be non-uniformed to obtain a non-uniform block. The first cost of the cost evaluation block is determined or obtained according to the third cost of at least one non-uniform block. The encoding and / or decoding mode of the current block is selected and / or sorted according to the first cost of the cost evaluation block.

[0483] Optionally, the cost evaluation block can be non-uniformed by applying different non-uniform weights to at least two cost evaluation elements of the cost evaluation block to obtain a non-uniform block.

[0484] Optionally, the image processing method further includes at least one of the following methods 14 to 17:

[0485] Method 14: Based on the third weight, at least one cost evaluation element located in at least one column on the far right and / or at least one row on the far bottom of the cost evaluation block is weighted to obtain a first non-uniform element, and a non-uniform block is determined or obtained based on at least one first non-uniform element.

[0486] Optionally, the third weight can be a pre-set weight, which can be a weight greater than 1, such as 1.5.

[0487] Optionally, the first non-uniform element can be a non-uniform element in a non-uniform block, and the absolute value of the first non-uniform element can be greater than the absolute value of the cost evaluation element located at the same position as the element to be homogenized.

[0488] Optionally, at least one cost evaluation element in the edge region of the cost evaluation block can be weighted according to the third weight to obtain at least one non-uniform element, and the image block containing at least one non-uniform element is taken as a non-uniform block. For example, the point evaluation element in the edge region of the cost evaluation block is replaced with the corresponding non-uniform element to obtain a non-uniform block.

[0489] Optionally, the edge region may include at least one column on the far right and / or at least one row on the far bottom of the cost evaluation block, such as the rightmost column or the bottom row of the cost evaluation block.

[0490] By performing non-uniform processing on the cost evaluation block, at least one cost evaluation element in at least the rightmost column and / or at least the bottom row of the cost evaluation block is weighted according to the third weight to determine or obtain the non-uniform block, thereby making the elements at the edge of the non-uniform block more accurate, improving the reference value of the elements at the edge of the non-uniform block, and thus making the prediction based on the non-uniform block more accurate and effective.

[0491] Method 15: Based on the distance between at least one cost evaluation element and the last column or last row in the cost evaluation block, determine or obtain a fourth weight; perform weighted processing on at least one cost evaluation element based on the fourth weight to obtain a second non-uniform element; and determine or obtain a non-uniform block based on at least one second non-uniform element.

[0492] Optionally, the fourth weight can be a pre-set weight, which can be a weight greater than 1 or a weight less than 1, such as 1.6, 0.8, etc.

[0493] Optionally, the smaller the distance between at least one cost evaluation element and the last column or last row in the cost evaluation block, the larger the fourth weight. For example, the fourth weight corresponding to the cost evaluation element located in the last column or last row in the cost evaluation block is a weight greater than 1, and the fourth weight corresponding to the cost evaluation element located in the central area of ​​the cost evaluation block is a weight less than 1.

[0494] Optionally, the second non-uniform element can be a non-uniform element in a non-uniform block, and the absolute value of the second non-uniform element can be greater than, equal to, or less than the absolute value of the cost evaluation element located at the same position as the element to be homogenized.

[0495] Optionally, for any cost evaluation element within the cost evaluation block, the distance between the cost evaluation element and the last column or last row in the cost evaluation block can be determined. If the distance is less than a preset distance threshold, a weight greater than 1 can be selected as the fourth weight; and / or, if the distance is equal to the preset distance threshold, a weight equal to 1 can be selected as the fourth weight; and / or, if the distance is less than the preset distance threshold, a weight less than 1 can be selected as the fourth weight.

[0496] Optionally, at least one cost evaluation element in the cost evaluation block that needs to be non-uniformed can be determined, and a corresponding fourth weight can be determined. For any cost evaluation element that needs to be non-uniformed, the cost evaluation element can be weighted according to the fourth weight corresponding to the cost evaluation element to obtain a second non-uniform element. Then, at least one cost evaluation element in the cost evaluation block that needs to be non-uniformed can be replaced with the corresponding second non-uniform element to obtain a non-uniform block.

[0497] By performing non-uniform processing on the cost evaluation block, a fourth weight is determined or obtained based on the distance between at least one cost evaluation element and the last column or last row in the cost evaluation block. The at least one cost evaluation element is then weighted according to the fourth weight to determine or obtain the non-uniform block. This makes the elements at the edge of the non-uniform block more accurate, improves the referenceability of the elements at the edge of the non-uniform block, and makes the prediction based on the non-uniform block more accurate and effective.

[0498] Method 16: Based on a non-uniform weight matrix with at least two different weights, weight at least one cost evaluation element in the cost evaluation block is weighted to obtain a third non-uniform element, and a non-uniform block is determined or obtained based on the at least one third non-uniform element.

[0499] Optionally, a non-uniform weight matrix containing at least two different weights can be set in advance. For example, the non-uniform weight matrix may include a fifth weight greater than 1 (e.g., 1.5) and a sixth weight less than 1 (e.g., 0.7).

[0500] Optionally, the third non-uniform element can be a non-uniform element in a non-uniform block, and the absolute value of the third non-uniform element can be greater than, equal to, or less than the absolute value of the cost evaluation element located at the same position as the element to be homogenized.

[0501] Optionally, for any cost evaluation element in the cost evaluation block, a corresponding weight can be determined in the non-uniform weight matrix, and the cost evaluation element can be weighted according to the weight to obtain a third non-uniform weight.

[0502] Optionally, at least one cost evaluation element in the cost evaluation block that needs to be non-uniformed can be determined, and its corresponding weight (such as the fifth or sixth weight) can be determined in the non-uniform weight matrix. For any cost evaluation element that needs to be non-uniformed, the cost evaluation element can be weighted according to the weight corresponding to the cost evaluation element to obtain the third non-uniform element. Then, at least one cost evaluation element in the cost evaluation block that needs to be non-uniformed can be replaced with its corresponding third non-uniform element to obtain the non-uniform block.

[0503] By performing non-uniform processing on the cost evaluation block, at least one cost evaluation element in the cost evaluation block is weighted according to a non-uniform weight matrix with at least two different weights to obtain a third non-uniform element. The non-uniform block is determined or obtained based on the at least one third non-uniform element, thereby making the elements at the edge of the non-uniform block more accurate and improving the reference value of the elements at the edge of the non-uniform block. This makes the prediction based on the non-uniform block more accurate and effective.

[0504] Method 17: Weight at least one cost evaluation element in the cost evaluation block according to the neural network and / or the first lookup table to obtain a fourth non-uniform element, and determine or obtain a non-uniform block based on the at least one fourth non-uniform element.

[0505] Optionally, the fourth non-uniform element can be a non-uniform element in a non-uniform block, and the absolute value of the fourth non-uniform element can be greater than, equal to, or less than the absolute value of the cost evaluation element located at the same position as the element to be homogenized.

[0506] Optionally, the weight corresponding to at least one cost evaluation element in the cost evaluation block can be determined or obtained based on the neural network and / or the first lookup table, and the at least one cost evaluation element can be weighted according to the weight to obtain the fourth non-uniform element.

[0507] Optionally, the cost evaluation block can be input into a neural network, and at least one cost evaluation element in the cost evaluation block can be weighted in the neural network to obtain a non-uniform block containing at least one fourth non-uniform element.

[0508] Optionally, the index corresponding to at least one cost evaluation element in the cost evaluation block can be determined, and the index can be entered into the first lookup table to find the corresponding fourth non-uniform element. The fourth non-uniform element is then updated to the cost evaluation block to obtain a non-uniform block. For example, the cost evaluation element in the cost evaluation block that needs to be non-uniformized can be replaced with the corresponding fourth non-uniform element to obtain a non-uniform block.

[0509] By performing non-uniform processing on the cost evaluation block, at least one cost evaluation element in the cost evaluation block is weighted according to the neural network and / or the first lookup table to obtain a fourth non-uniform element. The non-uniform block is determined or obtained based on the at least one fourth non-uniform element, thereby making the elements at the edge of the non-uniform block more prominent, improving the visual perception of the rationality of cost calculation, and making the first cost of the cost evaluation block determined or obtained based on the third cost of the non-uniform block more accurate and effective.

[0510] Optionally, for method eighteen, at least one non-uniform block of cost evaluation can be determined or obtained according to at least one of methods 14 to 17, or other methods can be used to determine or obtain at least one non-uniform block of cost evaluation, without limitation.

[0511] Optionally, a third cost of at least one non-uniform block can be determined or obtained based on the cost of at least one non-uniform element within the non-uniform block.

[0512] Optionally, if the non-uniform element and / or cost evaluation element is located in a first range, the cost of the non-uniform element is determined or obtained based on the non-uniform element and a first weight; and / or, if the non-uniform element and / or cost evaluation element is located in a third range different from both the first and second ranges, the cost of the non-uniform element is determined or obtained based on at least one neural network and / or a cost lookup table, and the non-uniform element; and / or, if the non-uniform element and / or cost evaluation element is located in a fourth range, the cost of the non-uniform element is determined or obtained based on the absolute value of at least one non-uniform element; and / or, if the non-uniform price block and / or cost evaluation block satisfies a first condition, the cost of the non-uniform element is determined or obtained based on a first parameter table; and / or, if the non-uniform block and / or cost evaluation block does not satisfy the first condition, the cost of the non-uniform element is determined or obtained based on a second parameter table different from the first parameter table; and the cost of the non-uniform element is determined or obtained based on a third parameter table.

[0513] Optionally, the first cost of the current block can be determined or obtained based on the third cost of at least one non-uniform block.

[0514] For example, the third cost of at least one non-uniform block can be used as the first cost; or a function calculation formula for cost calculation can be determined or obtained based on the third cost of at least one non-uniform block, and the cost calculation of the current block's cost evaluation block can be performed based on the function calculation formula to obtain the first cost. For example, when the third cost is greater than a cost threshold, the absolute error and the cost calculation of the cost evaluation block can be selected to obtain the first cost; etc., without any restrictions.

[0515] For example, as shown in Figure 13, the first cost of the cost evaluation block can be calculated based on non-uniform weights. Non-uniform weights can be used to non-uniformize the cost evaluation block before cost calculation to obtain the first cost. The following example uses the cost evaluation block as a residual block for illustration.

[0516] Optionally, for the residual block R, such as a 4x4 matrix, the elements are r i , j The residual block R is subjected to non-uniform weighting matrix to obtain the weighted residual block R. / (i.e., non-uniform blocks).

[0517] Optionally, the size parameters of the non-uniform weight matrix are consistent with the size parameters of the residual block R. The elements in the non-uniform weight matrix can be 1 or 1.1. For example, the elements in the bottom row and the rightmost column are 1.1, and the elements in other positions are 1.

[0518] Alternatively, the non-homogenization process can be element-wise multiplication to obtain the weighted residual block R. / This enhances the weight of residuals at key locations, such as edge pixels which may be used to predict other neighboring blocks, or edge residuals which have a greater visual impact, thereby improving the visual perceptual rationality of cost calculation.

[0519] Optionally, the weighted residual block R can be... / Perform a line-by-line scan, traversing the weighted residual block R in row-major order. / Each element r in i For example, if there are 16 elements, they are processed sequentially. When performing traversal and judgment processing, the absolute value of the residual |ri| of each residual element ri can be extracted, and then the corresponding residual processing can be performed, such as large residual processing, medium residual processing and small residual processing.

[0520] Optionally, when the absolute value of the residual |ri| is greater than T2, large residual processing can be performed, and the corresponding cost LUT(r) can be obtained by looking up the LUT table. i ).

[0521] Optionally, the lookup table (LUT) pre-stores the optimization cost mapping for large residuals, where T2 is a pre-set threshold, such as 100. Large residual processing can simplify the cost calculation of large residuals, avoid complex calculations, and improve efficiency.

[0522] Optionally, if the absolute value of the residual |ri| is greater than T1 and less than or equal to T2, moderate residual processing, i.e., identity processing, can be performed, such as directly using |ri| as the corresponding cost. T1 is a pre-set threshold, such as 3.

[0523] Optionally, if the absolute value of the residual |ri| is less than or equal to T1, small residual processing can be performed, that is, weighted processing. The absolute value of the residual |ri| is multiplied by the weight coefficient w to obtain the corresponding cost, i.e., w*|ri|.

[0524] Optionally, non-uniform blocks (i.e., weighted residual blocks R) can be used. / The costs of all elements in the block are summed to obtain the third cost of the non-uniform block, which can be used as the first cost of the cost evaluation block.

[0525] In this method, the first cost of the cost evaluation block is determined or obtained based on the third cost of at least one non-uniform block. Since the non-uniform elements within the non-uniform block are of inconsistent sizes, such as the edge portion being more prominent, the importance of important regions, such as the edge region, can be highlighted more when determining or obtaining the first cost of the cost evaluation block, thereby improving the accuracy of the first cost of the determined or obtained cost evaluation block.

[0526] Method 19: The fourth cost of at least one element in the transformed block after transforming the cost evaluation block;

[0527] Optionally, the transform block element can be an element in the transform block after the cost evaluation block has been transformed, or it can be a value at a certain position in the transform block.

[0528] Optionally, when determining the first cost of the cost evaluation block of the current block, the cost calculation method corresponding to the cost evaluation block of the current block can be determined according to at least one of the methods twelve to sixteen described in the second embodiment. If the cost calculation method is cost calculation method three, the fourth cost of at least one transformation element in the transformed block after the cost evaluation block is transformed can be determined, and the first cost of the cost evaluation block can be determined or obtained according to the fourth cost of the at least one transformation block element. The encoding and / or decoding modes of the current block are selected and / or sorted according to the first cost of the cost evaluation block.

[0529] Optionally, if the cost evaluation element and / or the transform block element is located in a first range, a fourth cost is determined or obtained based on the transform block element and the first weight; and / or, if the cost evaluation element and / or the transform block element is located in a second range different from the first range, a fourth cost is determined or obtained based on the second weight different from the first weight and the transform block element; and / or, if the cost evaluation element and / or the transform block element is located in a third range different from both the first and second ranges, a fourth cost is determined or obtained based on at least one neural network and / or a cost lookup table and the transform block element; and / or, if the cost evaluation element and / or the transform block element is located in a fourth range, a fourth cost is determined or obtained based on the absolute value of at least one transform block element; and / or, if the cost evaluation block and / or the transform block satisfies a first condition, a fourth cost is determined or obtained based on a first parameter table; and / or, if the cost evaluation block and / or the transform block does not satisfy the first condition, a fourth cost is determined or obtained based on a second parameter table different from the first parameter table; and a fourth cost is determined or obtained based on a third parameter table.

[0530] Optionally, the first cost of the current block's cost evaluation block can be determined or obtained based on the fourth cost of at least one transform block element in the transformed block after the cost evaluation block has been transformed.

[0531] For example, the fourth cost of at least one element of the transformed block after transforming the cost evaluation block can be used as the first cost; or the function calculation formula for cost calculation can be determined or obtained based on the fourth cost, and the cost evaluation block of the current block can be calculated based on the function calculation formula to obtain the first cost. For example, when the fourth cost is greater than a cost threshold, the absolute error can be selected to calculate the cost of the cost evaluation block to obtain the first cost; etc., without limitation.

[0532] In this method, the first cost of the cost evaluation block is determined or obtained by the fourth cost of at least one transform block element in the transformed block after the cost evaluation block is transformed. This allows for comprehensive consideration of the influence of transform block elements within the transformed block when determining the cost of the cost evaluation block, thereby improving the accuracy of the first cost of the determined or obtained cost evaluation block.

[0533] Method 20: The fifth cost of at least one element in the quantized block after quantizing the cost evaluation block;

[0534] Optionally, the quantization block element can be an element in the quantization block after the cost evaluation block has been quantized, or it can be a value at a certain position in the quantization block.

[0535] Optionally, when determining the first cost of the cost evaluation block of the current block, the cost calculation method corresponding to the cost evaluation block of the current block can be determined according to at least one of the methods twelve to sixteen described in the second embodiment. If the cost calculation method is cost calculation method four, the fifth cost of at least one quantized element in the quantized block after the cost evaluation block is quantized can be determined, and the first cost of the cost evaluation block can be determined or obtained according to the fifth cost of the at least one quantized block element. The encoding and / or decoding mode of the current block can be selected and / or sorted according to the first cost of the cost evaluation block.

[0536] Optionally, if the cost evaluation element and / or quantization block element is located in a first range, a fifth cost is determined or obtained based on the quantization block element and the first weight; and / or, if the cost evaluation element and / or quantization block element is located in a second range different from the first range, a fifth cost is determined or obtained based on the second weight different from the first weight and the quantization block element; and / or, if the cost evaluation element and / or quantization block element is located in a third range different from both the first and second ranges, a fifth cost is determined or obtained based on at least one neural network and / or a cost lookup table and the quantization block element; and / or, if the cost evaluation element and / or quantization block element is located in a fourth range, a fifth cost is determined or obtained based on the absolute value of at least one quantization block element; and / or, if the cost evaluation block and / or quantization block satisfies a first condition, a fifth cost is determined or obtained based on a first parameter table; and / or, if the cost evaluation block and / or quantization block does not satisfy the first condition, a fifth cost is determined or obtained based on a second parameter table different from the first parameter table; and a fifth cost is determined or obtained based on a third parameter table.

[0537] Optionally, the first cost of the current block's cost evaluation block can be determined or obtained based on the fifth cost of at least one quantized block element in the quantized block after the cost evaluation block has been quantized.

[0538] For example, the fifth cost of at least one element in the quantized block after quantizing the cost evaluation block can be used as the first cost; or the function calculation formula for cost calculation can be determined or obtained based on the fifth cost, and the cost evaluation block of the current block can be calculated based on the function calculation formula to obtain the first cost. For example, when the fifth cost is greater than a cost threshold, the absolute error can be selected to calculate the cost of the cost evaluation block to obtain the first cost; etc., without any restrictions.

[0539] In this method, the first cost of the cost evaluation block is determined or obtained by the fifth cost of at least one element in the quantized block after quantizing the cost evaluation block. This allows for comprehensive consideration of the influence of the elements in the quantized block when determining the cost of the cost evaluation block, thereby improving the accuracy of the first cost of the determined or obtained cost evaluation block.

[0540] Method 21: At least one neural network and / or cost lookup table to determine or obtain the first cost of the cost evaluation block.

[0541] Alternatively, the cost evaluation block can be input into a neural network to output the first cost of the cost evaluation block.

[0542] Optionally, the index corresponding to the cost evaluation block can be entered into the cost lookup table for searching to determine or obtain the first cost of the cost evaluation block.

[0543] Optionally, the cost evaluation block can be input into a neural network to obtain a first intermediate result, and the index corresponding to the first intermediate result can be input into a cost lookup table to obtain the first cost of the cost evaluation block.

[0544] Optionally, the cost evaluation block can be input into a neural network to obtain a second intermediate result. The index corresponding to the second intermediate result can be input into a cost lookup table to obtain a third intermediate result. The third intermediate result can be input into a neural network to obtain the first cost of the cost evaluation block. Alternatively, the index corresponding to the third intermediate result can be input into another cost lookup table to obtain the first cost of the cost evaluation block.

[0545] Optionally, the index corresponding to the cost evaluation block can be input into a cost lookup table to obtain a fourth intermediate result. The fourth intermediate result can be input into a neural network to obtain a fifth intermediate result. The fifth intermediate result can be input into a neural network to obtain the first cost of the cost evaluation block. Alternatively, the index corresponding to the fifth intermediate result can be input into another cost lookup table to obtain the first cost of the cost evaluation block.

[0546] For example, as shown in Figure 14, block cost calculation based on a multi-level lookup table can be illustrated by taking the encoding and / or decoding mode of the current block as the prediction mode and the residual block R as the cost evaluation block.

[0547] Optionally, the residual block R (containing at least one residual element) can be scanned row by row, for example, by traversing all elements in the residual block R in row priority order to generate a one-dimensional sequence {..., ri, rj, ...}, thereby converting the two-dimensional residual data of the residual block R into a linear sequence, which facilitates element-by-element processing and unifies the calculation process.

[0548] Optionally, each residual element (such as ri and rj) in the one-dimensional sequence can be transformed to generate a representation value, for example, by using a custom function to extract key features of the residual (such as amplitude and energy) to adapt to the subsequent lookup table logic.

[0549] Optionally, an intermediate value 1 can be obtained by searching in lookup table 1 using at least one representation value. Lookup table 1 pre-stores a mapping between at least one representation value and intermediate value 1, such as a cost mapping based on a rate distortion model. This enables a fast mapping of representation values ​​to preliminary costs, simplifies the calculation of nonlinear costs with large residuals, and improves efficiency.

[0550] Optionally, an intermediate value 2 can be obtained by searching in lookup table 2 using at least one intermediate value 1. Lookup table 2 pre-stores a mapping between at least one intermediate value 1 and intermediate value 2, such as the lookup table of the second layer of the convolutional neural network.

[0551] Optionally, the intermediate value 2 corresponding to each residual element can be summed to obtain the block cost.

[0552] In this approach, a first cost of a cost evaluation block is determined or obtained based on at least one neural network and / or a cost lookup table. This allows for cost determination by leveraging the advantages of the neural network and / or the cost lookup table, thereby improving the accuracy of the first cost of the determined or obtained cost evaluation block. This, in turn, improves the accuracy of sorting or filtering multiple encoding and / or decoding modes, and ultimately supports improved efficiency in video encoding and / or decoding.

[0553] Fourth embodiment

[0554] Based on any of the above embodiments, a fourth embodiment of this application is proposed.

[0555] In the fourth embodiment, step S10 includes steps S11 and S12:

[0556] Step S11: Determine or obtain the first candidate list based on the first cost of the cost evaluation block;

[0557] Step S12: Select and / or sort the encoding and / or decoding modes of the current block according to the first candidate list.

[0558] Optionally, the shape of the cost evaluation block includes at least one of the following: rectangular, T-shaped, L-shaped, and irregular shape.

[0559] For example, a T-shaped block or an L-shaped block formed by combining multiple residual blocks can be used as a cost evaluation block.

[0560] Alternatively, the irregular shape can be a non-rectangular shape formed by combining multiple rectangles.

[0561] Optionally, the cost evaluation block contains at least one row of cost evaluation elements and / or at least one column of cost evaluation elements;

[0562] Optionally, the encoding and / or decoding modes include at least one of prediction mode, block partitioning mode, motion vector, block vector, and filter;

[0563] Optionally, the prediction mode can be an angle prediction mode, a cross-component prediction mode, a block vector-based prediction mode, or an inter-frame prediction mode such as motion prediction / motion estimation.

[0564] Optionally, if the encoding and / or decoding mode is a prediction mode for encoding and / or decoding, prediction processing can be performed on the current block for each prediction mode, and residual processing can be performed to obtain a cost evaluation block. Then, a first candidate list can be determined or obtained based on the first cost of the cost evaluation block corresponding to each prediction mode, and the encoding and / or decoding mode of the current block can be selected and / or sorted based on the first candidate list.

[0565] Optionally, a preset candidate list can be constructed or updated based on the first cost of the determined or obtained cost evaluation block to determine or obtain the first candidate list.

[0566] Optionally, the pattern corresponding to the cost evaluation block can be determined or obtained based on the first cost of the cost evaluation block. The pattern may include at least one of the following: prediction pattern, block partitioning pattern, motion vector, block vector, and filter.

[0567] For example, the current block is processed using the first candidate prediction mode to obtain a prediction block, and the residual block between the current block and the prediction block is determined. The residual block is then used as the cost evaluation block.

[0568] Optionally, the list elements in the first candidate list can be sorted according to the first cost of the cost evaluation block, for example, sorted in order of magnitude of the first cost.

[0569] Optionally, the list elements in the first candidate list include at least one of the following: prediction mode, block partitioning mode, motion vector, block vector, filter, and cost evaluation block.

[0570] Optionally, step S12 includes: determining or obtaining the cost of at least one encoding and / or decoding mode based on the first candidate list, and selecting and / or sorting the encoding and / or decoding modes of the current block according to the cost of the at least one encoding and / or decoding mode.

[0571] Optionally, for any encoding and / or decoding mode, a list element corresponding to the encoding and / or decoding mode in the first candidate list can be determined, and the first cost of the cost evaluation block corresponding to the list element can be used as the cost of the encoding and / or decoding mode.

[0572] Optionally, the cost of the encoding and / or decoding mode may include at least one of the following: the cost of the prediction mode, the cost of the block partitioning mode, the cost of the motion vector, the cost of the block vector, and the cost of the filter.

[0573] Optionally, for each prediction mode, the block to be predicted can be predicted according to the prediction mode to obtain the prediction block, and a cost evaluation block (such as a residual block) can be determined based on the original block and the prediction block. The first cost of the cost evaluation block can be determined and the first cost of the cost evaluation block can be used as the cost of the prediction mode.

[0574] Optionally, a first cost of a cost evaluation block can be calculated for each block partitioning pattern, and the calculated first cost of the cost evaluation block can be used as the cost of that block partitioning pattern.

[0575] Optionally, for each block vector, a prediction region determined based on the block vector can be determined, and a residual region can be determined based on the reconstruction template region and the prediction region. This residual region can be used as a cost evaluation block, and the first cost of the cost evaluation block can be used as the cost of the block vector.

[0576] Optionally, for each motion vector, a prediction region determined based on the motion vector can be determined, and a residual region can be determined based on the reconstruction template region and the prediction region. This residual region can be used as a cost evaluation block, and the first cost of the cost evaluation block can be used as the cost of the motion vector.

[0577] Optionally, for each filter, a filtering region determined based on the filter can be determined, and a residual region can be determined based on the reconstruction template region and the filtering region. This residual region can be used as a cost evaluation block, and the first cost of the cost evaluation block can be used as the cost of the filter.

[0578] Optionally, after determining the cost of at least one encoding and / or decoding mode, the list elements in the first candidate list can be sorted in descending order, and at least one encoding and / or decoding mode can be selected from the first candidate list based on the cost of at least one encoding and / or decoding mode.

[0579] Optionally, by determining the cost of at least one encoding and / or decoding mode based on the first cost of at least one cost evaluation block, and selecting and / or sorting the encoding and / or decoding modes based on the cost of at least one encoding and / or decoding mode, it is possible to determine the first cost of the current block based on the cost of the current block evaluation block for different encoding and / or decoding modes, such as prediction mode, block partitioning mode, motion vector, block vector and filter.

[0580] In this embodiment, by determining or obtaining a first candidate list based on the first cost of the cost evaluation block, and selecting and / or sorting the encoding and / or decoding modes of the current block according to the first candidate list, it is possible to comprehensively consider the first cost of the cost evaluation block of the current block when calculating the cost of entropy encoding at different stages of video encoding and / or decoding. Furthermore, multiple encoding and / or decoding modes can be sorted or filtered in a list manner, thereby improving the accuracy of sorting or filtering multiple encoding and / or decoding modes and thus supporting the improvement of video encoding and / or decoding efficiency.

[0581] Fifth embodiment

[0582] Based on any of the above embodiments, a fifth embodiment of this application is proposed.

[0583] In the fifth embodiment, the first candidate list includes at least one of the second candidate list, the third candidate list, the fourth candidate list, the fifth candidate list, the sixth candidate list, the seventh candidate list, the eighth candidate list, the ninth candidate list, the first candidate sublist, the second candidate sublist, the third candidate sublist, and the fourth candidate sublist.

[0584] Optionally, the image processing method further includes at least one of the following steps b10 to b19:

[0585] Step b10: Based on the first sorting result of sorting at least two candidate encoding and / or decoding modes of the current block according to the first cost, determine or obtain the second candidate list;

[0586] Optionally, the candidate encoding and / or decoding modes may include at least one of prediction modes, block partitioning modes, motion vectors, block vectors, and filters.

[0587] Optionally, after determining or obtaining the first cost of the current block, the costs of at least two candidate encoding and / or decoding modes of the current block can be determined based on the first cost. The at least two candidate encoding and / or decoding modes can be sorted in order from lowest to highest cost based on their costs to obtain a first sorting result. A second candidate list can be determined or obtained based on the first sorting result so that the list elements in the second candidate list are consistent with the first sorting result.

[0588] Optionally, the first cost of the cost evaluation block can be used as the cost of the candidate encoding and / or decoding modes corresponding to the cost evaluation block.

[0589] Optionally, the current block can be processed according to the candidate encoding and / or decoding modes to obtain a cost evaluation block, and the candidate encoding and / or decoding modes can be used as the candidate encoding and / or decoding modes corresponding to the cost evaluation block.

[0590] For example, an example is given using candidate encoding and / or decoding modes as prediction modes and cost evaluation blocks as residual blocks.

[0591] The current block is predicted based on at least one prediction pattern to obtain a first prediction block. The residual block is determined based on the original block (i.e., the current block) and the prediction block. The first cost of the residual block is determined, and the cost of at least one prediction pattern is determined based on the first cost of the residual block. The at least one prediction pattern is then sorted in ascending order based on its cost to obtain a first sorting result. The at least one prediction pattern is then stored in a preset candidate list based on the first sorting result to obtain a second candidate list containing at least one prediction pattern.

[0592] Optionally, the first candidate list can be determined or obtained based on the second candidate list, and the encoding and / or decoding modes of the current block can be selected and / or sorted based on the first candidate list.

[0593] By sorting at least two candidate encoding and / or decoding modes of the current block according to a first cost, a second candidate list is determined or obtained, and then a first candidate list is determined. The encoding and / or decoding modes of the current block are selected and / or sorted according to the first candidate list. Multiple encoding and / or decoding modes can be sorted or filtered in a list manner, thereby improving the accuracy of sorting or filtering multiple encoding and / or decoding modes and thus supporting the improvement of video encoding and / or decoding efficiency.

[0594] Step b11: Based on a second cost calculation method that is different from the cost calculation method used to determine the first cost, determine or obtain the sixth cost of at least one cost evaluation block; based on the second sorting result of sorting at least two candidate encoding and / or decoding modes according to the at least one sixth cost, update the second candidate list to obtain the third candidate list.

[0595] Optionally, the second cost calculation method may be a cost calculation method determined or obtained according to at least one of methods twelve to sixteen described in the second embodiment.

[0596] Optionally, the calculation method for the first cost may differ from that for the second cost.

[0597] Optionally, the function calculation formula used in the first cost calculation method is different from the function calculation formula used in the second cost calculation method.

[0598] Optionally, a second candidate list is determined or obtained based on a first sorting result of sorting at least two candidate encoding and / or decoding modes of the current block according to a first cost; a sixth cost of at least one cost evaluation block is determined or obtained based on a second cost calculation method different from the cost calculation method used to determine the first cost; a third candidate list is obtained by updating the second candidate list based on a second sorting result of sorting at least two candidate encoding and / or decoding modes according to at least one sixth cost; a first candidate list is determined or obtained based on the third candidate list; and the encoding and / or decoding modes of the current block are selected and / or sorted based on the first candidate list.

[0599] Optionally, after recalculating the cost of at least one cost evaluation block according to the second cost calculation method to obtain the sixth cost, at least two candidate encoding and / or decoding modes can be sorted in ascending order of cost according to the at least one sixth cost to obtain a second sorting result. The list elements in the second candidate list can be re-sorted according to the second sorting result to obtain a third candidate list, so that the sorting result of the list elements in the third candidate list matches the second sorting result.

[0600] By determining the sixth cost based on the second cost calculation method, and then sorting at least two candidate encoding and / or decoding modes again based on the sixth cost to update the second candidate list and obtain the third candidate list, it is possible to generate a list by combining multiple cost calculation methods. Then, the encoding and / or decoding modes of the current block are selected and / or sorted based on the final generated first candidate list, so as to improve the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby supporting the improvement of video encoding and / or decoding efficiency.

[0601] Step b12: When at least one cost evaluation block satisfies the second condition, determine or obtain a fourth candidate list based on the second candidate list;

[0602] Optionally, satisfying the second condition may be that the size parameter of the cost evaluation block is greater than a preset size parameter threshold (e.g., width x height is 8x8), or the first feature value of the cost evaluation block is greater than a preset feature threshold (e.g., the positive and negative sign ratio is 30%), etc., without any restrictions.

[0603] Optionally, satisfying the second condition can be the same as satisfying the first condition, or it can be different from satisfying the first condition.

[0604] Optionally, a second candidate list is determined or obtained based on a first sorting result of sorting at least two candidate encoding and / or decoding modes of the current block according to a first cost, and a fourth candidate list is determined or obtained based on the second candidate list when at least one cost evaluation block satisfies a second condition, a first candidate list is determined or obtained based on the fourth candidate list, and the encoding and / or decoding modes of the current block are selected and / or sorted based on the first candidate list.

[0605] Optionally, when at least one cost evaluation block satisfies the second condition, the second candidate list can be used as the fourth candidate list, and the fourth candidate list can be used as the first candidate list to select and / or sort the encoding and / or decoding modes of the current block. Alternatively, the first n (n is a positive integer) list elements in the second candidate list can be determined, and the fourth candidate list can be constructed using these n list elements. The fourth candidate list can then be used as the first candidate list to select and / or sort the encoding and / or decoding modes of the current block.

[0606] When at least one cost evaluation block satisfies the second condition, a fourth candidate list is determined or obtained based on the second candidate list, and a first candidate list is determined or obtained based on the fourth candidate list. The encoding and / or decoding modes of the current block are selected and / or sorted according to the first candidate list. This allows multiple encoding and / or decoding modes to be sorted or filtered in a list manner, thereby improving the accuracy of sorting or filtering multiple encoding and / or decoding modes and thus supporting the improvement of video encoding and / or decoding efficiency.

[0607] Step b13: When at least one cost evaluation block does not meet the second condition, a seventh cost of at least one cost evaluation block is determined or obtained according to a third cost calculation method that is different from the cost calculation method for determining the first cost. The second candidate list is updated to obtain a fifth candidate list based on a third sorting result of sorting at least two candidate encoding and / or decoding modes according to at least one seventh cost.

[0608] Optionally, the third cost calculation method may be a cost calculation method determined or obtained based on at least one of methods twelve to sixteen described in the second embodiment.

[0609] Optionally, the calculation method for the first cost may differ from that for the third cost, and the calculation method for the third cost may be the same as or different from that for the second cost.

[0610] Optionally, the function calculation formula used in the first cost calculation method is different from the function calculation formula used in the third cost calculation method.

[0611] Optionally, when the third cost calculation method is the same as the second cost calculation method, the seventh cost and the sixth cost of the same cost evaluation block are the same.

[0612] Optionally, the second condition may not be met if the size parameter of the cost evaluation block is less than or equal to a preset size parameter threshold (e.g., width x height is 8x8), or the first feature value of the cost evaluation block is less than or equal to a preset feature threshold (e.g., the positive and negative sign ratio is 30%), etc., which are not restricted here.

[0613] Optionally, a second candidate list is determined or obtained based on a first sorting result of sorting at least two candidate encoding and / or decoding modes of the current block according to a first cost. If at least one cost evaluation block does not meet the second condition, a seventh cost of at least one cost evaluation block is determined or obtained based on a third cost calculation method that is different from the cost calculation method used to determine the first cost. The second candidate list is then updated based on a third sorting result of sorting at least two candidate encoding and / or decoding modes according to at least one seventh cost to obtain a fifth candidate list.

[0614] Optionally, after recalculating the cost of at least one cost evaluation block according to the third cost calculation method to obtain the seventh cost, at least two candidate encoding and / or decoding modes can be sorted in ascending order of cost according to the at least one seventh cost to obtain the third sorting result. The list elements in the second candidate list can be re-sorted according to the third sorting result to obtain the fifth candidate list, so that the sorting result of the list elements in the fifth candidate list matches the second sorting result.

[0615] When at least one cost evaluation block does not meet the second condition, the seventh cost determined or obtained according to the third cost calculation method is used to sort at least two candidate encoding and / or decoding modes. Then, the second candidate list is updated according to the third sorting result to obtain the fifth candidate list. This enables the generation of a list by combining multiple cost calculation methods. Then, the encoding and / or decoding modes of the current block are selected and / or sorted according to the final generated first candidate list. This improves the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby supporting the improvement of video encoding and / or decoding efficiency.

[0616] Step b14: Based on the fourth cost calculation method, which is different from the cost calculation method for determining the first cost, calculate the cost of at least one list element in the head of the second candidate list to obtain the eighth cost. Update the second candidate list based on the eighth cost to obtain the sixth candidate list.

[0617] Optionally, the fourth cost calculation method may be a cost calculation method determined or obtained based on at least one of methods twelve to sixteen described in the second embodiment.

[0618] Optionally, the calculation method for the first cost is different from that for the fourth cost.

[0619] Optionally, the function calculation formula used in the first cost calculation method is different from the function calculation formula used in the fourth cost calculation method.

[0620] Optionally, the list elements in the second candidate list may include at least one of the following: prediction mode, block partitioning mode, motion vector, block vector, filter, and cost evaluation block.

[0621] Optionally, the head of the second candidate list can be the position of the first m elements in the second candidate list, where m is a positive integer, such as 5.

[0622] Optionally, based on the first sorting result of sorting at least two candidate encoding and / or decoding modes of the current block according to the first cost, a second candidate list is determined or obtained, and based on the fourth cost calculation method which is different from the cost calculation method used to determine the first cost, the cost of at least one list element in the header of the second candidate list is calculated to obtain an eighth cost, the second candidate list is updated based on the eighth cost to obtain a sixth candidate list, the first candidate list is determined or obtained based on the sixth candidate list, and the encoding and / or decoding modes of the current block are selected and / or sorted based on the first candidate list.

[0623] Optionally, after calculating the cost of at least one list element in the head of the second candidate list according to the fourth cost calculation method to obtain the eighth cost, the second candidate list can be updated according to the eighth cost corresponding to at least one list element in the head of the second candidate list. For example, the list elements can be reordered and updated according to the cost of the eighth cost from smallest to largest to obtain the sixth candidate list.

[0624] By calculating the cost of at least one list element in the head of the second candidate list according to the fourth cost calculation method, the eighth cost is obtained. The second candidate list is then updated according to the eighth cost to obtain the sixth candidate list, which is the first candidate list. This allows for the generation of a list using a combination of multiple cost calculation methods. The encoding and / or decoding modes of the current block are then selected and / or sorted based on the final generated first candidate list, thereby improving the accuracy of sorting or filtering multiple encoding and / or decoding modes and thus supporting improved efficiency in video encoding and / or decoding.

[0625] Step b15: Determine or obtain the seventh candidate list based on the decision factors of at least one candidate encoding and / or decoding mode and the decision factors of at least one list element in the second candidate list;

[0626] Optionally, the decision factors for at least one candidate encoding and / or decoding mode include the cost, weight, and priority of the candidate encoding and / or decoding mode.

[0627] Optionally, the decision factors for at least one list element include the list length of the second candidate list, the number of list elements that can be stored, etc.

[0628] For example, at least one candidate encoding and / or decoding mode can be obtained from neighboring blocks and non-neighboring blocks of the current block, and the decision factor of the at least one candidate encoding and / or decoding mode and the decision factor of at least one list element in the second candidate list can be calculated. Then, the seventh candidate list can be determined or obtained based on these two decision factors. For example, if the length of the second candidate list is less than 6 and the priority of the at least one candidate encoding and / or decoding mode is high, the at least one candidate encoding and / or decoding mode can be added to the second candidate list and sorted according to its corresponding cost. Then, the first six list elements in the second candidate list can be selected to regenerate a new list to obtain the seventh candidate list.

[0629] Optionally, based on the first sorting result of sorting at least two candidate encoding and / or decoding modes of the current block according to the first cost, a second candidate list is determined or obtained, and based on the decision factor of at least one candidate encoding and / or decoding mode and the decision factor of at least one list element in the second candidate list, a seventh candidate list is determined or obtained, a first candidate list is determined or obtained based on the seventh candidate list, and the encoding and / or decoding modes of the current block are selected and / or sorted based on the first candidate list.

[0630] By determining or obtaining a seventh candidate list based on the decision factors of at least one candidate encoding and / or decoding mode and the decision factors of at least one list element in the second candidate list, it is possible to comprehensively consider the decision factors of the candidate encoding and / or decoding mode and the decision factors of the list elements when generating the list, thereby improving the accuracy of the first candidate list finally obtained based on the sixth candidate list.

[0631] Step b16: Calculate the cost of at least one element in the seventh candidate list according to the fifth cost calculation method, which is different from the cost calculation method used to determine the first cost, to obtain the ninth cost. Update the seventh candidate list according to the at least one ninth cost to obtain the eighth candidate list.

[0632] Optionally, the fifth cost calculation method may be a cost calculation method determined or obtained based on at least one of methods twelve to sixteen described in the second embodiment.

[0633] Optionally, the calculation method for the first cost is different from that for the fifth cost.

[0634] Optionally, the function calculation formula used in the first cost calculation method is different from the function calculation formula used in the fifth cost calculation method.

[0635] Optionally, based on the first sorting result of sorting at least two candidate encoding and / or decoding modes of the current block according to the first cost, a second candidate list is determined or obtained, and based on the decision factor of at least one candidate encoding and / or decoding mode and the decision factor of at least one list element in the second candidate list, a seventh candidate list is determined or obtained, and based on the fifth cost calculation method (different from the cost calculation method used to determine the first cost), the cost of at least one list element in the seventh candidate list is calculated to obtain a ninth cost, and the seventh candidate list is updated based on at least one ninth cost to obtain an eighth candidate list, and based on the eighth candidate list, a first candidate list is determined or obtained, and the encoding and / or decoding modes of the current block are selected and / or sorted based on the first candidate list.

[0636] Optionally, when calculating the cost of at least one list element in the seventh candidate list according to the fifth cost calculation method to obtain the ninth cost, the cost of at least one list element in the head of the seventh candidate list can be calculated to obtain the corresponding ninth cost.

[0637] Optionally, the list elements in the seventh candidate list that contain the ninth cost are determined, and the list elements are reordered according to the cost of the ninth cost in ascending order, while the order of the list elements in the seventh candidate list that do not contain the ninth cost remains unchanged, thus obtaining the eighth candidate list.

[0638] By calculating the cost of at least one element in the seventh candidate list according to the fifth cost calculation method, a ninth cost is obtained. The seventh candidate list is then updated based on at least one ninth cost to obtain an eighth candidate list. The first candidate list can then be determined or obtained based on the eighth candidate list. This allows for the generation of a list using a combination of multiple cost calculation methods. The encoding and / or decoding modes of the current block are then selected and / or sorted based on the final generated first candidate list. This improves the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby supporting improved efficiency in video encoding and / or decoding.

[0639] Step b17: Split the second candidate list to obtain at least one first candidate sublist;

[0640] Optionally, the splitting rules for the second candidate list can be set by the user according to their needs, and there are no restrictions here.

[0641] For example, if the second candidate list contains 16 list elements, the top five list elements of the second candidate list can be selected to split into a new sublist, namely the first candidate sublist, and the remaining 11 list elements of the second candidate list can be selected to construct another first candidate sublist.

[0642] Optionally, based on the first sorting result of sorting at least two candidate encoding and / or decoding modes of the current block according to the first cost, a second candidate list is determined or obtained, and the second candidate list can be split to obtain at least one first candidate sublist. The first candidate list is determined or obtained based on the at least one first candidate sublist, and the encoding and / or decoding modes of the current block are selected and / or sorted according to the first candidate list.

[0643] By splitting the second candidate list to obtain at least one first candidate sublist, and then determining or obtaining the first candidate list based on the at least one first candidate sublist, the encoding and / or decoding mode of the current block can be confirmed based on the first candidate list. This allows the current block to be encoded with less signaling, thus improving encoding efficiency.

[0644] Step b18: Based on the sixth cost calculation method, which is different from the cost calculation method for determining the first cost, calculate the cost of at least one list element in at least one first candidate sublist to obtain the tenth cost. Update the at least one first candidate sublist based on the at least one tenth cost to obtain the at least one second candidate sublist.

[0645] Optionally, the sixth cost calculation method may be a cost calculation method determined or obtained based on at least one of methods twelve to sixteen described in the second embodiment.

[0646] Optionally, the method for calculating the cost of the first cost may differ from the method for calculating the cost of the sixth cost.

[0647] Optionally, the function calculation formula used in the first cost calculation method is different from the function calculation formula used in the sixth cost calculation method.

[0648] Optionally, based on the first sorting result of sorting at least two candidate encoding and / or decoding modes of the current block according to the first cost, a second candidate list is determined or obtained. The second candidate list can be split to obtain at least one first candidate sublist. Based on a sixth cost calculation method that is different from the cost calculation method used to determine the first cost, a cost is calculated on at least one list element in the at least one first candidate sublist to obtain a tenth cost. The at least one first candidate sublist is updated based on the at least one tenth cost to obtain at least one second candidate sublist. The first candidate list is determined or obtained based on the at least one second candidate sublist. The encoding and / or decoding modes of the current block are selected and / or sorted based on the first candidate list.

[0649] Optionally, the cost can be calculated based on at least one list element in at least one first candidate sublist according to the sixth cost calculation method to obtain the tenth cost corresponding to at least one list element.

[0650] Optionally, the list elements in at least one first candidate sublist can be sorted and updated according to the tenth cost, for example, by re-sorting them in order from minimum to maximum cost to obtain at least one second candidate sublist.

[0651] For example, as shown in Figure 15, the generation of the most likely list based on secondary sorting is first performed by the low-cost cost calculation algorithm 1 to roughly sort all the patterns or corresponding residual blocks (i.e. cost evaluation blocks) in the list to obtain the second candidate list. Then, the more complex and accurate cost calculation algorithm 2 performs secondary sorting on the list elements (such as prediction patterns) at the head of the second candidate list to finally obtain the first candidate list.

[0652] Optionally, the information of the block to be predicted in the current block can be obtained first to obtain multiple prediction modes, such as obtaining 16 different prediction modes, including prediction (mode 1) to prediction (mode 16), and generating 16 corresponding prediction blocks (prediction block 1 to prediction block 16). Each prediction mode calculates the prediction result independently.

[0653] Optionally, the 16 predicted blocks and the original blocks are input into the subtractor for processing to obtain 16 residual blocks. The original block can be the original data block in the encoding process or the reconstructed template at the decoding end, such as the current block. The original block and each predicted block are respectively processed by the subtractor to calculate the residual, resulting in 16 residual blocks (residual block 1 to residual block 16). The residual represents the difference between the original data and the predicted data of the current block and is the core input for subsequent cost calculation.

[0654] Optionally, the initial cost calculation can be performed using cost calculation one. Cost calculation can be performed on each of the 16 residual blocks. The function calculation formula used can be SAD, SATD, etc., and then 16 initial costs (i.e., first costs) can be calculated, such as cost 1 to cost 16 in the figure.

[0655] Optionally, the quality of the cost quantification prediction model can provide a basis for subsequent ranking.

[0656] Optionally, an initial cost sort can be performed to obtain a second candidate list, which is the first sorting result based on the first cost sorting of at least two candidate encoding and / or decoding modes (e.g., 16 prediction modes) of the current block, to determine or obtain the second candidate list (e.g., list L1).

[0657] Optionally, the 16 initial costs corresponding to the 16 prediction modes can be sorted to generate a list L1 containing 16 items: [mode, cost, residual block]. The prediction modes with smaller costs can be selected by sorting, and the candidate set can be initially optimized.

[0658] Optionally, a list splitting operation can be performed to divide the second candidate list into multiple first candidate sublists.

[0659] Optionally, the first three list elements in list L1 are extracted to obtain the first candidate sublist. The remaining 13 list elements are extracted in order to obtain multiple first candidate sublists, such as L1(1), residual block; L1(2), residual block; L1(3), residual block; L1(4)..., L1(16).

[0660] Optionally, a second cost calculation is performed on at least one first candidate sublist using cost calculation method 2. For example, the sixth cost calculation method is used to calculate the cost of at least one list element in at least one first candidate sublist to obtain the tenth cost (such as cost 1, cost 2, cost 3, etc. in the figure). Then, the sublists are reordered to obtain at least one second candidate sublist, such as list L2[pattern, cost] in the figure, which has 3 items. Then, multiple second candidate sublists are concatenated to obtain list L3[pattern, cost], which has 16 items, i.e., the first candidate list.

[0661] Optionally, the concatenated list retains the complete candidate set of 16 prediction modes, while the first 3 prediction modes are further optimized to balance global search and local fine-tuning, thereby improving coding efficiency and quality.

[0662] By calculating the cost of at least one element in at least one first candidate sublist according to the sixth cost calculation method, the tenth cost is obtained. Then, the at least one first candidate sublist is updated according to the at least tenth cost to obtain at least one second candidate sublist. The first candidate list can be determined or obtained based on the at least one second candidate sublist. The encoding and / or decoding mode of the current block can be confirmed based on the first candidate list, thereby enabling the use of less signaling to encode the current block and improving encoding efficiency.

[0663] Step b19: Based on at least two different seventh cost calculation methods and eighth cost calculation methods, calculate the cost of at least one list element in at least two first candidate sublists respectively to obtain the eleventh cost and the twelfth cost. Update at least one first candidate sublist based on the eleventh cost to obtain the third candidate sublist. Update at least one first candidate sublist based on the twelfth cost to obtain the fourth candidate sublist. Determine or obtain the ninth candidate list based on the third candidate sublist and the fourth candidate sublist.

[0664] Optionally, both the seventh cost calculation method and the eighth cost calculation method can be calculation methods determined or obtained based on at least one of methods twelve to sixteen described in the second embodiment.

[0665] Optionally, the calculation methods for the first cost, the seventh cost, and the eighth cost are different from each other.

[0666] Optionally, the function calculation formula used for the first cost calculation method, the function calculation formula used for the seventh cost calculation method, and the function calculation formula used for the eighth cost calculation method are different from each other.

[0667] Optionally, based on the first sorting result of sorting at least two candidate encoding and / or decoding modes of the current block according to the first cost, a second candidate list is determined or obtained. The second candidate list can be split to obtain at least one first candidate sublist. The cost of at least one list element in the at least one first candidate sublist can be calculated according to the seventh cost calculation method to obtain an eleventh cost. The at least one first candidate sublist is updated according to the eleventh cost (e.g., the sorting of list elements) to obtain a third candidate sublist. The first candidate list is determined or obtained based on the third candidate sublist. The encoding and / or decoding modes of the current block are selected and / or sorted according to the first candidate list.

[0668] Optionally, the cost can be calculated for at least one list element in at least one first candidate sublist according to the eighth cost calculation method to obtain the twelfth cost. The at least one first candidate sublist can be updated according to the twelfth cost (e.g., the sorting of list elements) to obtain the fourth candidate sublist. The first candidate list can be determined or obtained according to the fourth candidate sublist, and the encoding and / or decoding mode of the current block can be selected and / or sorted according to the first candidate list.

[0669] Optionally, a ninth candidate list can be determined or obtained based on at least one third candidate sublist and at least one fourth candidate sublist (e.g., merging at least one third candidate sublist and at least one fourth candidate sublist to obtain a ninth candidate list), and a first candidate list can be determined or obtained based on the ninth candidate list, and the encoding and / or decoding modes of the current block can be selected and / or sorted based on the first candidate list.

[0670] By calculating the cost of at least one element in at least two first candidate sublists according to the seventh and eighth cost calculation methods respectively to update the lists, a third candidate sublist and a fourth candidate sublist are obtained. Then, a ninth candidate list is determined or obtained based on the third and fourth candidate sublists. The first candidate list can be determined or obtained based on at least one of the third, fourth, and ninth candidate lists. The encoding and / or decoding mode of the current block can be confirmed based on the first candidate list, thereby enabling the encoding of the current block with less signaling and improving encoding efficiency.

[0671] The technical solution of this embodiment can achieve the first cost of the current block by comprehensively considering the cost of the current block when calculating the cost of entropy encoding at different stages of video encoding and / or decoding. This can improve the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby supporting the improvement of video encoding and / or decoding efficiency.

[0672] Referring to Figure 16, this application also provides a processing apparatus, which includes:

[0673] Processing module A10 is used to evaluate the first cost of the current block based on the cost of the current block, and to select and / or sort the encoding and / or decoding modes of the current block.

[0674] Optionally, the cost evaluation block of the current block is determined or obtained based on at least one of the following:

[0675] The pixel block of the current block; at least one first difference block; at least one second difference block; at least one third difference block; at least one fourth difference block; residual block; absolute value block of the difference; transform block of the difference; absolute value block of the transform block of the difference; quantization block of the difference; absolute value block of the quantization block of the difference.

[0676] Optionally, processing module A10 is configured to perform at least one of the following:

[0677] The first difference block is determined or obtained based on the difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode; the second difference block is determined or obtained based on the difference between the image blocks corresponding to at least two candidate encoding and / or decoding modes; the third difference block is determined or obtained based on the difference between the two cost evaluation blocks of the current block; and the fourth difference block is determined or obtained based on the difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode.

[0678] Optionally, processing module A10 is configured to perform at least one of the following:

[0679] The cost calculation methods are different for cost evaluation elements at at least two different positions in the same cost evaluation block; the cost calculation methods are different for cost evaluation blocks with at least two different components; the cost calculation methods are different for cost evaluation blocks with at least two different size parameters; the cost calculation methods are different for cost evaluation blocks with at least two different texture features; and the cost evaluation blocks are classified such that the cost calculation methods are different for cost evaluation blocks with at least two different categories.

[0680] Optionally, the first cost of the cost evaluation block is determined or obtained according to at least one of the following:

[0681] The cost evaluation block contains at least one cost evaluation element with a second cost; at least one non-uniform block with a third cost; at least one transform block element in the transform block after transforming the cost evaluation block with a fourth cost; at least one quantization block element in the quantization block after quantizing the cost evaluation block with a fifth cost; and at least one neural network and / or cost lookup table to determine or obtain the cost evaluation block with a first cost.

[0682] Optionally, the non-uniform block is obtained by non-uniformizing the cost evaluation block;

[0683] Optionally, processing module A10 is configured to perform at least one of the following:

[0684] If the cost evaluation element is within a first range, then the second cost is determined or obtained based on the cost evaluation element and the first weight; if the cost evaluation element is within a second range different from the first range, then the second cost is determined or obtained based on the second weight different from the first weight and the cost evaluation element; if the cost evaluation element is within a third range different from both the first and second ranges, then the second cost is determined or obtained based on at least one neural network and / or a cost lookup table and the cost evaluation element; if the cost evaluation element is within a fourth range, then the second cost is determined or obtained based on the absolute value of at least one cost evaluation element; if the cost evaluation block satisfies the first condition, then the second cost is determined or obtained based on the first parameter table; if the cost evaluation block does not satisfy the first condition, then the second cost is determined or obtained based on the second parameter table different from the first parameter table; the second cost is determined or obtained based on the third parameter table.

[0685] Optionally, processing module A10 is configured to perform at least one of the following:

[0686] Based on the third weight, at least one cost evaluation element located in at least one column on the far right and / or at least one row on the far bottom of the cost evaluation block is weighted to obtain the first non-uniform element, and the non-uniform block is determined or obtained based on the at least one first non-uniform element.

[0687] Based on the distance between at least one cost evaluation element and the last column or last row in the cost evaluation block, a fourth weight is determined or obtained. Based on the fourth weight, at least one cost evaluation element is weighted to obtain a second non-uniform element. Based on at least one second non-uniform element, a non-uniform block is determined or obtained.

[0688] Based on a non-uniform weight matrix with at least two different weights, at least one cost evaluation element in the cost evaluation block is weighted to obtain a third non-uniform element, and a non-uniform block is determined or obtained based on at least one third non-uniform element.

[0689] At least one cost evaluation element in the cost evaluation block is weighted according to the neural network and / or the first lookup table to obtain the fourth non-uniform element, and the non-uniform block is determined or obtained based on the at least one fourth non-uniform element.

[0690] Optionally, the cost evaluation blocks may be categorized based on at least one of the following:

[0691] The sum of the absolute values ​​of the cost evaluation elements in the cost evaluation block; the feature complexity of the cost evaluation block; the first eigenvalue of the cost evaluation block; the size parameter of the cost evaluation block; the sum of the absolute values ​​of the transformation elements in the transformation block; the sum of the absolute values ​​of the quantization elements in the quantization block.

[0692] Optionally, the processing module A10 is configured to perform: determining or obtaining a first candidate list based on the first cost of the cost evaluation block; and selecting and / or sorting the encoding and / or decoding modes of the current block based on the first candidate list.

[0693] Optionally, the first candidate list includes at least one of the second candidate list, the third candidate list, the fourth candidate list, the fifth candidate list, the sixth candidate list, the seventh candidate list, the eighth candidate list, the first candidate sublist, the second candidate sublist, the third candidate sublist, and the fourth candidate sublist;

[0694] Optionally, processing module A10 is configured to perform at least one of the following:

[0695] Based on the first sorting result of sorting at least two candidate encoding and / or decoding modes of the current block according to the first cost, a second candidate list is determined or obtained;

[0696] Based on a second cost calculation method that is different from the cost calculation method used to determine the first cost, a sixth cost of at least one cost evaluation block is determined or obtained. Based on the second sorting result of sorting at least two candidate encoding and / or decoding modes according to the at least one sixth cost, the second candidate list is updated to obtain a third candidate list.

[0697] When at least one cost evaluation block satisfies the second condition, a fourth candidate list is determined or obtained based on the second candidate list.

[0698] When at least one cost evaluation block does not meet the second condition, a seventh cost is determined or obtained based on a third cost calculation method that is different from the cost calculation method for determining the first cost. The second candidate list is updated based on a third sorting result of sorting at least two candidate encoding and / or decoding modes according to the at least one seventh cost to obtain a fifth candidate list.

[0699] Based on a fourth cost calculation method that differs from the cost calculation method used to determine the first cost, the cost of at least one list element in the head of the second candidate list is calculated to obtain the eighth cost. The second candidate list is then updated based on the eighth cost to obtain the sixth candidate list.

[0700] A seventh candidate list is determined or obtained based on the decision factors of at least one candidate encoding and / or decoding mode and the decision factors of at least one list element in the second candidate list.

[0701] The cost of at least one element in the seventh candidate list is calculated based on the fifth cost calculation method, which is different from the cost calculation method used to determine the first cost, to obtain the ninth cost. The seventh candidate list is then updated based on the at least one ninth cost to obtain the eighth candidate list.

[0702] Split the second candidate list to obtain at least one first candidate sublist;

[0703] Based on a sixth cost calculation method that is different from the cost calculation method for determining the first cost, the cost of at least one list element in at least one first candidate sublist is calculated to obtain a tenth cost. Based on the at least tenth cost, the at least one first candidate sublist is updated to obtain at least one second candidate sublist.

[0704] Based on at least two distinct seventh cost calculation methods and eighth cost calculation methods, the cost is calculated for at least one list element in at least two first candidate sublists to obtain the eleventh cost and the twelfth cost. The at least one first candidate sublist is updated based on the eleventh cost to obtain the third candidate sublist. The at least one first candidate sublist is updated based on the twelfth cost to obtain the fourth candidate sublist. The ninth candidate list is determined or obtained based on the third candidate sublist and the fourth candidate list.

[0705] Optionally, processing module A10 is configured to perform at least one of the following:

[0706] The list elements in the first candidate list include at least one of the following: prediction mode, block partitioning mode, motion vector, block vector, filter, and cost evaluation block;

[0707] The shape of the cost evaluation block includes at least one of the following: rectangular, T-shaped, L-shaped, and irregular shape;

[0708] The cost evaluation block contains at least one row of cost evaluation elements and / or at least one column of cost evaluation elements;

[0709] The encoding and / or decoding modes include at least one of prediction mode, block partitioning mode, motion vector, block vector, and filter;

[0710] The cost of at least one encoding and / or decoding mode is determined or obtained based on the first candidate list, and the encoding and / or decoding modes of the current block are selected and / or sorted according to the cost of at least one encoding and / or decoding mode.

[0711] The processing apparatus provided in this application embodiment can execute the technical solutions shown in the corresponding method embodiments above. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0712] This application also provides a processing device, including a memory and a processor. The memory stores an image processing program, and when the image processing program is executed by the processor, it implements the steps of the image processing method in any of the above embodiments.

[0713] This application also provides a storage medium storing an image processing program, which, when executed by a processor, implements the steps of the image processing method in any of the above embodiments.

[0714] In the embodiments of the smart terminal and storage medium provided in this application, all the technical features of any of the above-described image processing method embodiments may be included. The extended and explanatory content of the specification is basically the same as that of the embodiments of the above methods, and will not be repeated here.

[0715] This application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to perform the methods described in the various possible implementations above.

[0716] This application also provides a chip, including a memory and a processor. The memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that a device with the chip installed performs the methods described in the various possible implementations above.

[0717] It is understood that the above scenarios are merely examples and do not constitute a limitation on the application scenarios of the technical solutions provided in the embodiments of this application. The technical solutions of this application can also be applied to other scenarios. For example, as those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0718] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0719] The steps in the method of this application embodiment can be adjusted, combined, or deleted according to actual needs.

[0720] The units in the device of this application embodiment can be merged, divided, and deleted according to actual needs.

[0721] In this application, the same or similar terms, concepts, technical solutions and / or application scenario descriptions are generally described in detail only when they appear for the first time. When they appear again, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of this application, the same or similar terms, concepts, technical solutions and / or application scenario descriptions that are not described in detail later can be referred to their previous relevant detailed descriptions.

[0722] In this application, the descriptions of the various embodiments have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0723] The technical features of the present application can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present application.

[0724] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, controlled terminal, or network device, etc.) to execute the methods of each embodiment of this application.

[0725] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, storage disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0726] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An image processing method, wherein, Including the following steps: S10, evaluate the first cost of the block based on the cost of the current block, and select and / or sort the encoding and / or decoding modes of the current block.

2. The image processing method as described in claim 1, wherein, The cost evaluation block for the current block is determined or obtained based on at least one of the following: The current block's pixel block; At least one first difference block; At least one second difference block; At least one third interpolation block; At least one fourth interpolation block; Residual block; The absolute value block of the difference; Transformation block for the difference; The absolute value block of the difference transformation block; Quantization block for the difference; The absolute value block of the quantization block of the difference.

3. The image processing method as described in claim 2, wherein, It also includes at least one of the following: The first difference block is determined or obtained based on the difference between the template of the current block and the image block corresponding to at least one candidate encoding and / or decoding mode; The second difference block is determined or obtained based on the difference between image blocks corresponding to at least two candidate encoding and / or decoding modes; The third difference block is determined or obtained based on the difference between the two cost evaluation blocks of the current block; The fourth difference block is determined or obtained based on the difference between the current block and the image block corresponding to at least one candidate encoding and / or decoding mode.

4. The image processing method as described in claim 1, wherein, It also includes at least one of the following: The cost calculation methods are different for cost evaluation elements at at least two different positions in the same cost evaluation block; The cost calculation methods are different for at least two cost evaluation blocks with different components; The cost calculation methods differ for at least two cost evaluation blocks with different size parameters; The cost calculation methods differ for at least two cost evaluation blocks with different texture features; The cost evaluation blocks are categorized, and at least two different categories of cost evaluation blocks have different cost calculation methods.

5. The image processing method as described in claim 4, wherein, The first cost of the cost evaluation block is determined or obtained based on at least one of the following: The second cost of at least one cost evaluation element within the cost evaluation block; The third cost of at least one non-uniform block; The fourth cost of at least one element of the transformed block after transforming the cost evaluation block; The fifth cost of at least one element in the quantized block after quantizing the cost evaluation block; At least one neural network and / or cost lookup table are used to determine or obtain the first cost of the cost evaluation block.

6. The image processing method as described in claim 5, wherein, The non-uniform block is obtained by non-uniform processing of the cost evaluation block; and / or, it also includes at least one of the following: If the cost evaluation element is within the first range, then the second cost is determined or obtained based on the cost evaluation element and the first weight; If the cost evaluation element is located in a second range that is different from the first range, then the second cost is determined or obtained based on the second weight that is different from the first weight and the cost evaluation element. If the cost evaluation element is located in a third range that is different from both the first and second ranges, then the second cost is determined or obtained based on at least one neural network and / or a cost lookup table, and the cost evaluation element. If the cost evaluation element is in the fourth range, then the second cost is determined or obtained based on the absolute value of at least one cost evaluation element; If the cost evaluation block meets the first condition, then the second cost is determined or obtained according to the first parameter table; If the cost evaluation block does not meet the first condition, then the second cost is determined or obtained according to the second parameter table, which is different from the first parameter table; The second cost is determined or obtained based on the third parameter table.

7. The image processing method as described in claim 6, wherein, It also includes at least one of the following: Based on the third weight, at least one cost evaluation element located in at least one column on the far right and / or at least one row on the far bottom of the cost evaluation block is weighted to obtain the first non-uniform element, and the non-uniform block is determined or obtained based on the at least one first non-uniform element. Based on the distance between at least one cost evaluation element and the last column or last row in the cost evaluation block, a fourth weight is determined or obtained. Based on the fourth weight, at least one cost evaluation element is weighted to obtain a second non-uniform element. Based on at least one second non-uniform element, a non-uniform block is determined or obtained. Based on a non-uniform weight matrix with at least two different weights, at least one cost evaluation element in the cost evaluation block is weighted to obtain a third non-uniform element, and a non-uniform block is determined or obtained based on at least one third non-uniform element. At least one cost evaluation element in the cost evaluation block is weighted according to the neural network and / or the first lookup table to obtain the fourth non-uniform element, and the non-uniform block is determined or obtained based on the at least one fourth non-uniform element.

8. The image processing method as described in claim 1, wherein, The cost assessment blocks should be categorized based on at least one of the following: The sum of the absolute values ​​of the cost evaluation elements in the cost evaluation block; Feature complexity of the cost evaluation block; The first characteristic value of the cost evaluation block; Size parameters of the cost evaluation block; The sum of the absolute values ​​of the transform elements in the transform block; The sum of the absolute values ​​of the quantized elements in the quantized block.

9. The image processing method as described in claim 1, wherein, Step S10 includes the following steps: S11, determine or obtain the first candidate list based on the first cost of the cost evaluation block; S12, Select and / or sort the encoding and / or decoding modes of the current block according to the first candidate list.

10. The image processing method as described in claim 9, wherein, The first candidate list includes at least one of the second, third, fourth, fifth, sixth, seventh, eighth, and ninth candidate lists, a first sublist, a second sublist, a third sublist, and a fourth sublist, and / or includes at least one of the following: Based on the first sorting result of sorting at least two candidate encoding and / or decoding modes of the current block according to the first cost, a second candidate list is determined or obtained; Based on a second cost calculation method that is different from the cost calculation method used to determine the first cost, a sixth cost of at least one cost evaluation block is determined or obtained. Based on the second sorting result of sorting at least two candidate encoding and / or decoding modes according to the at least one sixth cost, the second candidate list is updated to obtain a third candidate list. When at least one cost evaluation block satisfies the second condition, a fourth candidate list is determined or obtained based on the second candidate list. When at least one cost evaluation block does not meet the second condition, a seventh cost is determined or obtained based on a third cost calculation method that is different from the cost calculation method for determining the first cost. The second candidate list is updated based on a third sorting result of sorting at least two candidate encoding and / or decoding modes according to the at least one seventh cost to obtain a fifth candidate list. Based on a fourth cost calculation method that differs from the cost calculation method used to determine the first cost, the cost of at least one list element in the head of the second candidate list is calculated to obtain the eighth cost. The second candidate list is then updated based on the eighth cost to obtain the sixth candidate list. A seventh candidate list is determined or obtained based on the decision factors of at least one candidate encoding and / or decoding mode and the decision factors of at least one list element in the second candidate list. The cost of at least one element in the seventh candidate list is calculated based on the fifth cost calculation method, which is different from the cost calculation method used to determine the first cost, to obtain the ninth cost. The seventh candidate list is then updated based on the at least one ninth cost to obtain the eighth candidate list. Split the second candidate list to obtain at least one first candidate sublist; Based on a sixth cost calculation method that is different from the cost calculation method for determining the first cost, the cost of at least one list element in at least one first candidate sublist is calculated to obtain a tenth cost. Based on the at least tenth cost, the at least one first candidate sublist is updated to obtain at least one second candidate sublist. Based on at least two distinct seventh and eighth cost calculation methods, cost calculations are performed on at least one list element in at least two first candidate sublists to obtain eleventh and twelfth costs. At least one first candidate sublist is updated based on the eleventh cost to obtain a third candidate sublist. At least one first candidate sublist is updated based on the twelfth cost to obtain a fourth candidate sublist. The ninth candidate sublist is determined or obtained based on the third and fourth candidate sublists.

11. The image processing method as described in claim 9, wherein, It also includes at least one of the following: The list elements in the first candidate list include at least one of the following: prediction mode, block partitioning mode, motion vector, block vector, filter, and cost evaluation block; The shape of the cost evaluation block includes at least one of the following: rectangular, T-shaped, L-shaped, and irregular shape; The cost evaluation block contains at least one row of cost evaluation elements and / or at least one column of cost evaluation elements; The encoding and / or decoding modes include at least one of prediction mode, block partitioning mode, motion vector, block vector, and filter; Step S12 includes: determining or obtaining the cost of at least one encoding and / or decoding mode based on the first candidate list, and selecting and / or sorting the encoding and / or decoding modes of the current block according to the cost of the at least one encoding and / or decoding mode.

12. A processing apparatus, wherein, include: The system includes a memory and a processor, wherein the memory stores an image processing program, and when the image processing program is executed by the processor, it implements the steps of the image processing method as described in claim 1.

13. A storage medium, wherein, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the image processing method as described in claim 1.

Images