Image processing method, processing device, and storage medium

Abstract

The present application provides an image processing method, a processing device, and a storage medium. The image processing method can be applied to the processing device and comprises: determining or obtaining a second cost of a residual block on the basis of a first cost of elements within a current block; and selecting from among and / or sorting encoding and / or decoding modes on the basis of the second cost. According to the technical solution of the present application, the accuracy in sorting or screening a plurality of encoding and / or decoding modes can be improved, thereby improving the efficiency of video encoding and / or decoding.

Description

Image processing methods, processing devices and storage media Technical Field

[0001] This application relates to the field of image processing technology, specifically to an image processing method, processing device, and storage medium. Background Technology

[0002] The existing high-efficiency video coding standard protocol (H.266 / VVC) proposes a video frame coding technique to improve coding performance without significantly increasing computational complexity. Specifically, when encoding and decoding video frames, the protocol divides each frame into different blocks and performs prediction, transformation and quantization processing before encoding and decoding.

[0003] In the process of conceiving and implementing this application, the inventors discovered at least the following problems:

[0004] At various stages of encoding and decoding, when sorting or filtering encoding and / or decoding modes (such as sorting or filtering partitioning modes, or sorting or filtering prediction modes), cost evaluation is used. For example, in the prediction stage, the cost of prediction modes needs to be estimated to sort or filter multiple prediction modes. Currently, cost evaluation functions such as SAD (Sum of absolute differences) or SATD (Sum of Absolute Transformed Difference) are used for evaluation. However, SAD does not consider the correlation between residuals, and SATD ignores the weights of DC and high-frequency variable amplitudes, resulting in insufficient accuracy in cost estimation of prediction modes. Consequently, the accuracy of sorting or filtering multiple encoding and / or decoding modes is insufficient, thus limiting the efficiency of video encoding and / or decoding.

[0005] The preceding description is intended to provide general background information and does not necessarily constitute prior art.

[0006] Summary of the Invention

[0007] To address the aforementioned technical problems, this application provides an image processing method, processing device, and storage medium, aiming to solve the technical problem of how to improve the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby improving the efficiency of video encoding and / or decoding.

[0008] This application provides an image processing method, applicable to a processing device, comprising the following steps:

[0009] S10, determine or obtain the second cost of the residual block based on the first cost of the elements in the current block;

[0010] S20, select and / or sort the encoding and / or decoding modes according to the second cost.

[0011] Optionally, the method for determining or obtaining the first cost of the elements within the current block includes at least one of the following:

[0012] Based on at least one of the original value, absolute value, and preset number of digits of any element in the current block, obtain the first cost from the first lookup table;

[0013] For any two elements in the current block, the first cost of one element is obtained by weighting it according to the first weight of the other element, and the first cost of the other element is obtained by weighting it according to the second weight of the other element.

[0014] The elements in the current block include the first element and the second element. The third cost is obtained from the second lookup table based on the original value or absolute value of the first element in the current block. The first cost of the first element is determined based on the third cost. The fourth cost is obtained from the second lookup table based on the preset number of radixes of the second element in the current block. The first cost of the second element is determined based on the fourth cost.

[0015] The third weight is obtained from the third lookup table based on any element in the current block. The weight is then applied to the third weight to obtain the first cost.

[0016] For any element in the current block, calculate the fifth cost based on the first cost function, calculate the sixth cost based on the second cost function, and determine the first cost based on the fifth cost and the sixth cost.

[0017] If the text information obtained in the bitstream is within the first range, the first cost is calculated using the first element calculation method, and / or, if the text information obtained in the bitstream is within the second range, the first cost is calculated using the second element calculation method, which is different from the first element calculation method.

[0018] Optionally, step S10 includes:

[0019] The second cost of the residual block is determined or obtained based on the first cost of at least one element within at least one sub-block of the current block.

[0020] Optionally, the method of determining or obtaining at least one sub-block of the current block includes at least one of the following:

[0021] Based on at least one sub-residual block obtained by dividing the residual block corresponding to the current block, determine or obtain at least one sub-block;

[0022] Based on the sub-transformed block obtained by transforming at least one sub-residual block, determine or obtain at least one sub-block;

[0023] At least one sub-block is determined or obtained based on the sub-quantization block obtained by quantizing at least one sub-transform block.

[0024] Optionally, step S10 includes at least one of the following:

[0025] The seventh cost of at least one sub-block is determined or obtained based on the first cost of at least one element within at least one sub-block, and the second cost of the residual block is determined or obtained based on the seventh cost of at least one sub-block.

[0026] The cost of at least one sub-residual block is determined based on the cost of at least one sub-transform block, and the second cost of the residual block is determined or obtained based on the cost of at least one sub-residual block.

[0027] The cost of at least one sub-residual block is determined based on the cost of at least one sub-quantized block, and a second cost of the residual block is determined or obtained based on the cost of at least one sub-residual block.

[0028] Input at least one sub-block and / or the downsampled portion corresponding to the sub-block into the neural network, output the cost of at least one sub-residual block, and determine or obtain the second cost of the residual block based on the cost of at least one sub-residual block;

[0029] Based on the partitioning costs of at least two partitions for the residual block, a second cost for the residual block is determined or obtained.

[0030] Optionally, determining or obtaining a seventh cost for at least one sub-block based on a first cost of at least one element within at least one sub-block includes at least one of the following:

[0031] Determine or obtain the seventh cost of at least one sub-block based on the first cost of at least two elements within at least one sub-block;

[0032] Determine or obtain the seventh cost of at least one sub-block based on the first cost of the symbol of the element within at least one sub-block;

[0033] The elements within at least one sub-block are scanned to obtain a scan sequence. Based on the cost of the sub-sequence in the scan sequence, the seventh cost of the sub-block is determined or obtained.

[0034] Optionally, the determination or acquisition of the second cost of the residual block includes at least one of the following:

[0035] The eighth cost of the residual block is calculated according to the first cost function, the ninth cost of the residual block is calculated according to the second cost function, and the second cost of the residual block is determined based on the minimum value of the eighth cost and the ninth cost.

[0036] The eighth cost of the residual block is calculated according to the first cost function, and the ninth cost of the residual block is calculated according to the second cost function. The eighth cost and the ninth cost are weighted to obtain the second cost of the residual block.

[0037] The residual block is input into the neural network, and the second cost of the residual block is output.

[0038] If the text information obtained in the bitstream is within the third range, then the second cost of the residual block is calculated using the first cost function;

[0039] If the text information obtained in the bitstream is located in the fourth range, then the second cost function is used to calculate the second cost of the residual block;

[0040] If the text information obtained in the bitstream is in the fifth range, then the second cost of the residual block is determined or obtained based on the cost of at least one sub-residual block.

[0041] If the text information obtained in the bitstream is in the sixth range, then the second cost of the residual block is determined or obtained based on the cost of the sub-transform block.

[0042] If the text information obtained in the bitstream is within the seventh range, then the second cost of the residual block is determined or obtained based on the cost of the sub-quantized block.

[0043] Optionally, the image processing method further includes at least one of the following:

[0044] The encoding and / or decoding modes include at least one of prediction mode, block partitioning mode, motion vector, block vector, and filter;

[0045] The second cost of the residual block includes at least one of the following: the cost of the prediction mode, the cost of the block partitioning mode, the cost of the motion vector, the cost of the block vector, and the cost of the filter.

[0046] Use the value of each position in the current block and / or sub-block as an element;

[0047] Step S20 includes: determining the cost of at least one encoding and / or decoding mode based on at least one second cost, and selecting and / or sorting the encoding and / or decoding modes based on the cost of at least one encoding and / or decoding mode.

[0048] This application also provides a processing apparatus, which includes a processing module:

[0049] The processing module is used to determine or obtain the second cost of the residual block based on the first cost of the elements in the current block; and to select and / or sort the encoding and / or decoding modes based on the second cost.

[0050] This application also provides a processing device, including: a memory and a processor, wherein the memory stores an image processing program, and when the image processing program is executed by the processor, it implements the steps of the image processing method described above.

[0051] This application also provides a storage medium storing a computer program that, when executed by a processor, implements the steps of the image processing method described above.

[0052] As described above, the image processing method of this application can be applied to a processing device, including: determining or obtaining a second cost of a residual block based on a first cost of elements within the current block, and selecting and / or sorting encoding and / or decoding modes based on the second cost. Through the technical solution of this application, when calculating the cost of entropy encoding at different stages of video encoding and / or decoding, the first cost of elements within the current block can be comprehensively considered, which can improve the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby improving the efficiency of video encoding and / or decoding. Attached Figure Description

[0053] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0054] Figure 1 is a schematic diagram of the hardware structure of a mobile terminal implementing various embodiments of this application;

[0055] Figure 2 is a communication network system architecture diagram provided in an embodiment of this application;

[0056] Figure 3 is a schematic diagram of the hardware structure of a controller 140 provided in this application;

[0057] Figure 4 is a schematic diagram of the hardware structure of a network node 150 provided in this application;

[0058] Figure 5 is a flowchart illustrating the image processing method according to the first embodiment;

[0059] Figure 6 is a schematic diagram of the coding framework of the image processing method according to the first embodiment;

[0060] Figure 7 is a schematic diagram of pattern filtering for the predicted pattern in the image processing method according to the first embodiment;

[0061] Figure 8 is a schematic diagram of vector filtering for vectors in the image processing method according to the first embodiment;

[0062] Figure 9 is a schematic diagram of filter selection for a filter in an image processing method according to the first embodiment;

[0063] Figure 10 is a schematic diagram of cost lookup using a lookup table in the image processing method according to the second embodiment;

[0064] Figure 11 is another schematic diagram of cost lookup using a lookup table in the image processing method according to the second embodiment;

[0065] Figure 12 is a schematic diagram of the second cost of determining the residual block by using two partitioning methods in the image processing method according to the third embodiment;

[0066] Figure 13 is a schematic diagram of the image processing method according to the third embodiment, which uses scanning to determine the residual block cost;

[0067] Figure 14 is a schematic diagram illustrating the determination of block cost using different methods based on the bitstream markers obtained from the bitstream in the image processing method according to the fourth embodiment;

[0068] Figure 15 is a schematic diagram of a processing device provided in an embodiment of this application.

[0069] The realization of the objectives, functional features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0070] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0071] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0072] It should be understood that although the terms first, second, third, etc., may be used herein to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word “if” as used herein may be interpreted as “when…” or “in response to determination”. Furthermore, as used herein, the singular forms “a,” “an,” and “the” are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms “comprising,” “including,” indicate the presence of the stated feature, step, operation, element, component, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms “or,” “and / or,” “including at least one of the following,” etc., as used in this application may be interpreted as inclusive, or mean any one or any combination thereof. For example, "including at least one of the following: A, B, C" means "any one of the following: A; B; C; A and B; A and C; B and C; A and B and C." Similarly, "A, B, or C" or "A, B, and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A and B and C." Exceptions to this definition only occur when the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0073] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0074] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0075] It should be noted that step designations such as S10 and S20 are used in this document for the purpose of more clearly and concisely describing the corresponding content, and do not constitute a substantial limitation on the order. In specific implementation, those skilled in the art may execute S20 first and then S10, etc., but these should all be within the protection scope of this application.

[0076] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0077] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0078] Image processing equipment can be a smart terminal or a server, and smart terminals can be implemented in various forms. For example, the smart terminals described in this application can include smart terminals such as mobile phones, tablets, laptops, handheld computers, personal digital assistants (PDAs), portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, etc., as well as fixed terminals such as digital TVs and desktop computers.

[0079] The following description will use a mobile terminal as an example. Those skilled in the art will understand that, apart from elements specifically designed for mobile purposes, the construction according to the embodiments of this application can also be applied to fixed-type terminals.

[0080] Please refer to Figure 1, which is a schematic diagram of the hardware structure of a mobile terminal implementing various embodiments of this application. The mobile terminal 100 may include: an RF (Radio Frequency) unit 101, a WiFi module 102, an audio output unit 103, an A / V (Audio / Video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111, etc. Those skilled in the art will understand that the mobile terminal structure shown in Figure 1 does not constitute a limitation on the mobile terminal. The mobile terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0081] The following section, with reference to Figure 1, provides a detailed description of each component of the mobile terminal:

[0082] The radio frequency unit 101 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 110; additionally, it transmits uplink data to the base station. Typically, the radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, and a duplexer. Furthermore, the radio frequency unit 101 can also communicate wirelessly with networks and other devices. The aforementioned wireless communications may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution), 5G, and 6G.

[0083] WiFi is a short-range wireless transmission technology. Mobile terminals using WiFi module 102 can help users send and receive emails, browse web pages, and access streaming media, providing wireless broadband internet access. Although Figure 1 shows WiFi module 102, it is understood that it is not an essential component of the mobile terminal and can be omitted as needed without altering the essence of the invention.

[0084] The audio output unit 103 can convert audio data received by the radio frequency unit 101 or the WiFi module 102 or stored in the memory 109 into audio signals and output them as sound when the mobile terminal 100 is in call signal receiving mode, call mode, recording mode, voice recognition mode, broadcast receiving mode, etc. Furthermore, the audio output unit 103 can also provide audio output related to specific functions performed by the mobile terminal 100 (e.g., call signal receiving sound, message receiving sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.

[0085] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos acquired by an image capture device (such as a camera) in video capture mode or image capture mode. The processed image frames can be displayed on the display unit 106. The image frames processed by the GPU 1041 can be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) in operating modes such as telephone call mode, recording mode, and voice recognition mode, and can process such sound into audio data. The processed audio (voice) data can be converted into a format that can be transmitted to a mobile communication base station via the radio frequency unit 101 in telephone call mode. The microphone 1042 can implement various types of noise cancellation (or suppression) algorithms to eliminate (or suppress) noise or interference generated during the reception and transmission of audio signals.

[0086] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Optionally, the light sensor includes an ambient light sensor and a proximity sensor. Optionally, the ambient light sensor can adjust the brightness of the display panel 1061 according to the ambient light level, and the proximity sensor can turn off the display panel 1061 and / or backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc. Other sensors that may be configured in the phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0087] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0088] User input unit 107 can be used to receive input numerical or character information, and generate key signal inputs related to user settings and function control of the mobile terminal. Optionally, user input unit 107 may include touch panel 1071 and other input devices 1072. Touch panel 1071, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near touch panel 1071), and drive corresponding connection devices according to a pre-set program. Touch panel 1071 may include a touch detection device and a touch controller. Optionally, the touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to processor 110, and can receive and execute commands sent by processor 110. In addition, touch panel 1071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may also include other input devices 1072. Optionally, other input devices 1072 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc., without being specifically limited here.

[0089] Optionally, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. Subsequently, the processor 110 provides corresponding visual output on the display panel 1061 according to the type of touch event. Although in FIG. 1, the touch panel 1071 and the display panel 1061 are implemented as two independent components to realize the input and output functions of the mobile terminal, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to realize the input and output functions of the mobile terminal. The specific implementation is not limited here.

[0090] Interface unit 108 serves as an interface through which at least one external device can connect to mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, and so on. Interface unit 108 may be used to receive input (e.g., data, power, etc.) from the external device and transmit the received input to one or more elements within mobile terminal 100, or it may be used to transmit data between mobile terminal 100 and the external device.

[0091] The memory 109 can be used to store software programs and various data. The memory 109 may primarily include a program storage area and a data storage area. Optionally, the program storage area may store the operating system, applications required for at least one function (such as sound playback, image playback, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory 109 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0092] The processor 110 is the control center of the mobile terminal. It connects various parts of the mobile terminal via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 109, and by calling data stored in the memory 109, it performs various functions and processes data of the mobile terminal, thereby providing overall monitoring of the mobile terminal. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor. Optionally, the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 110.

[0093] The mobile terminal 100 may also include a power supply 111 (such as a battery) that supplies power to various components. Preferably, the power supply 111 can be logically connected to the processor 110 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0094] Although not shown in Figure 1, the mobile terminal 100 may also include a Bluetooth module, etc., which will not be described in detail here.

[0095] To facilitate understanding of the embodiments of this application, the communication network system on which the mobile terminal of this application is based is described below.

[0096] Please refer to Figure 2, which is a communication network system architecture diagram provided in an embodiment of this application. The communication network system is an LTE system based on the universal mobile communication technology. The LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203, and the operator's IP services 204, which are connected in sequence.

[0097] Optionally, UE201 can be the aforementioned terminal 100, which will not be described in detail here.

[0098] E-UTRAN202 includes eNodeB2021 and other eNodeB2022, etc. Optionally, eNodeB2021 can connect to other eNodeB2022 via backhaul (e.g., X2 interface), and eNodeB2021 connects to EPC203, providing access from UE201 to EPC203.

[0099] EPC203 may include MME (Mobility Management Entity) 2031, HSS (Home Subscriber Server) 2032, other MMEs 2033, SGW (Serving Gateway) 2034, PGW (Packet Data Network Gateway) 2035, and PCRF (Policy and Charging Rules Function) 2036, etc. Optionally, MME2031 is the control node that handles signaling between UE201 and EPC203, providing bearer and connection management. HSS2032 is used to provide registers to manage functions such as the Home Location Register (not shown in the figure) and stores user-specific information such as service characteristics and data rates. All user data can be sent through SGW2034. PGW2035 can provide UE 201 IP address allocation and other functions. PCRF2036 is the policy and charging control decision point for service data flow and IP bearer resources. It selects and provides available policy and charging control decisions for the policy and charging enforcement function unit (not shown in the figure).

[0100] IP services 204 may include the Internet, intranet, IMS (IP Multimedia Subsystem), or other IP services.

[0101] Although the above description uses the LTE system as an example, those skilled in the art should know that this application is not only applicable to the LTE system, but also to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, 5G and future new network systems (such as 6G), etc., without limitation.

[0102] Figure 3 is a schematic diagram of the hardware structure of a controller 140 provided in this application. The controller 140 includes a memory 1401 and a processor 1402. The memory 1401 is used to store program instructions, and the processor 1402 is used to call the program instructions in the memory 1401 to execute the steps performed by the controller in the first embodiment of the above method. The implementation principle and beneficial effects are similar, and will not be described again here.

[0103] Optionally, the controller further includes a communication interface 1403, which can be connected to the processor 1402 via a bus 1404. The processor 1402 can control the communication interface 1403 to implement the receiving and sending functions of the controller 140.

[0104] Figure 4 is a schematic diagram of the hardware structure of a network node 150 provided in this application. The network node 150 includes a memory 1501 and a processor 1502. The memory 1501 is used to store program instructions, and the processor 1502 is used to call the program instructions in the memory 1501 to execute the steps performed by the first node in the above method embodiment. The implementation principle and beneficial effects are similar, and will not be described again here.

[0105] Optionally, the controller further includes a communication interface 1503, which can be connected to the processor 1502 via a bus 1504. The processor 1502 can control the communication interface 1503 to implement the receiving and sending functions of the network node 150.

[0106] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0107] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk, SSD), etc.

[0108] Based on the above-described mobile terminal hardware structure and communication network system, various embodiments of this application are proposed.

[0109] First Embodiment

[0110] Referring to Figure 5, which is a flowchart illustrating the image processing method according to the first embodiment, the image processing method of this application embodiment can be applied to a processing device and includes the following steps:

[0111] S10, determine or obtain the second cost of the residual block based on the first cost of the elements in the current block;

[0112] S20, select and / or sort the encoding and / or decoding modes according to the second cost.

[0113] In this embodiment, the processing device can be a smart terminal, such as a mobile phone or computer, or a server, such as a local server or a cloud server. This embodiment and this application primarily use a smart terminal as an example for illustration.

[0114] Optionally, the technical solution of this embodiment can be applied to the fields of image encoding and decoding, video encoding and decoding, hardware video encoding and decoding, dedicated circuit video encoding and decoding, real-time video encoding and decoding, etc.

[0115] Optionally, for ease of understanding, a brief introduction to the encoding and decoding architecture is provided first: Referring to Figure 6, in the encoding and decoding framework, the input video frame can be divided into blocks to obtain multiple blocks. Each block undergoes frame transformation processing through a frame transformation module. The transformed video frame is then input into modules such as transformation, quantization, intra-frame prediction, inter-frame coding, inverse quantization, inverse transform, and loop filtering. The video frame processed by the quantization module is entropy encoded, outputting bits 01011100, etc. Motion estimation is performed based on the video frame obtained after loop filtering and the video frame processed by frame transformation to obtain motion information. Inter-frame prediction is then performed based on the motion information.

[0116] Optionally, at the encoding end, video frames are input to a frame processing module. This module executes a certain selection rule and then performs one or more of the following operations on the frame: geometric transformation, downsampling, linear transformation, nonlinear transformation, and pre-filtering (in the case of filtering, mode information may not be available). The transformed frame is then input to subsequent transformation, quantization, intra-frame prediction, inter-frame coding, inverse quantization, and inverse transform modules. The results of these processes are input to the inverse frame transform module, where inverse geometric transformation, upsampling, and filtering are performed before outputting the decoded image.

[0117] Alternatively, loop filtering can be considered as part of the frame inverse transform module.

[0118] Optionally, at the decoding end, the bitstream is read and the transform mode information is parsed (in the case of filtering, there may be no mode information); the video stream is subjected to inverse quantization, transform, inter-frame prediction, intra-frame prediction, etc., to decode the video frames (after transformation at the encoding end); the video frames are subjected to inverse frame transform according to the mode information and the decoded image is output. The inverse frame transform includes inverse geometric transformation, upsampling, filtering, etc.

[0119] Alternatively, loop filtering can be considered as part of the frame inverse transform module.

[0120] Optionally, cost calculation is required at each stage of encoding and / or decoding to select and / or sort based on the calculated cost. For example, in the block partitioning stage where the input video frame is divided into blocks, the cost of at least one block partitioning mode can be calculated according to this embodiment for selection and / or sorting. Similarly, in the prediction stage, when determining the prediction mode used for intra-frame prediction and / or inter-frame prediction, the cost of candidate prediction modes is calculated for selection and / or sorting. Furthermore, the methods described in this embodiment can also be used for various stages such as block vectors, motion vectors, and filters.

[0121] Optionally, this embodiment can address the problems existing in the cost evaluation function, such as the absolute value of each element in SAD or SATD failing to express the true entropy coding cost of the coefficient block. This embodiment can circumvent this defect and further improve the accuracy of its estimation of the true entropy coding cost by controlling the complexity of the cost evaluation function.

[0122] Optionally, during the encoding and / or decoding process, the first cost of computing the elements within the current block can be determined first.

[0123] Optionally, an element can be a value at a position within the current block.

[0124] Optionally, when the current block is a residual block, the pixel value of each pixel in the residual block can be used as an element.

[0125] Optionally, when the current block is a transform block, the value of each position in the transform block can be used as an element, such as using the value in the transform block that corresponds to the position of the pixel in the residual block as an element.

[0126] Optionally, when the current block is a quantization block, the value of each position in the quantization block can be used as an element, such as the value of the position in the quantization block that corresponds to the position of the pixel in the residual block.

[0127] Optionally, when determining the first cost of an element within the current block, one can choose to calculate it using a pre-set function formula, or query it from a pre-set lookup table that includes the mapping relationship between elements and costs, or use other methods to determine it; no restrictions are placed here.

[0128] Optionally, the function calculation formula may include at least one of the following: Sum of absolute differences (SAD), Sum of absolute transformed differences (SATD), Sum of squared differences (SSD), Mean absolute difference (MAD), and Mean squared difference (MSD).

[0129] Optionally, the cost of the residual region can be determined or obtained based on the first cost of the elements in the current block.

[0130] Optionally, the residual region includes at least one residual block, for example, the residual region can be a T-shaped region formed by combining multiple residual blocks.

[0131] Optionally, the encoding and / or decoding modes can be selected and / or ordered based on the cost of the residual regions.

[0132] Since the method of using residual regions is similar to the method of using residual blocks, the following example will only use the method of residual blocks.

[0133] Optionally, when the current block is a residual block, the first cost of the elements within the residual block can be summarized or otherwise operated on to determine the second cost of the residual block. Alternatively, after obtaining the second cost of the residual block, the second costs of at least one residual block contained in the residual region can be summarized or otherwise operated on to determine or obtain the cost of the residual region.

[0134] Optionally, encoding and / or decoding modes can be selected and / or sorted based on the second cost of the residual block, for example, determining the second cost of the residual block corresponding to each encoding and / or decoding mode, and selecting and / or sorting based on the magnitude of the second cost.

[0135] Optionally, encoding and / or decoding modes can be selected and / or sorted based on the cost of the residual regions. For example, the cost of the residual regions corresponding to each encoding and / or decoding mode can be determined, and the residual regions can be sorted and / or selected based on the magnitude of their costs.

[0136] Optionally, the encoding and / or decoding modes include at least one of prediction modes, block partitioning modes, motion vectors, block vectors, and filters.

[0137] Optionally, the prediction mode may include prediction modes for angle prediction, prediction modes for cross-component prediction, prediction modes based on block vectors, and prediction modes for inter-frame prediction such as motion prediction / motion estimation.

[0138] Optionally, if the encoding and / or decoding mode is a prediction mode of encoding and / or decoding, then for each prediction mode, the second cost of the residual block can be determined or obtained based on the first cost of the elements in the current block, and at least one prediction mode can be selected and / or sorted based on the second cost corresponding to each prediction mode.

[0139] For example, as shown in Figure 7, the information of the block to be predicted is obtained, and K prediction modes are used for prediction to generate K prediction blocks respectively. That is, prediction (mode 1) is used to predict to obtain prediction block 1, and prediction (mode K) is used to predict to obtain prediction block K. K is an integer greater than 1.

[0140] Optionally, the subtractor in Figure 7 operates on K prediction blocks to obtain K residual blocks. That is, residual block 1 = original block - prediction block 1, ..., residual block K = original block - prediction block K. The cost calculation module in Figure 7 calculates the cost for each of the K residual blocks to obtain K cost values, such as cost 1 for residual block 1 and cost K for residual block K. The cost sorting module in Figure 7 sorts the K costs, for example, according to the size of cost 1 to cost K, to obtain an ordered list [mode, cost]. The mode filtering module in Figure 7 filters the ordered list to obtain a prediction mode [list]. For example, the prediction mode with the lowest cost is selected to obtain a prediction mode or a list of prediction modes composed of multiple prediction modes.

[0141] Optionally, at the encoding end, a rapid, low-complexity coarse screening of the prediction modes can be performed according to this embodiment. Then, the remaining modes after coarse screening are precisely transformed, quantized, and entropy encoded to obtain the optimal prediction mode. For example, at the encoding end, there are a total of 67 angle prediction modes, mode 1,..., mode 67. The framework shown in Figure 7 can be used to first calculate the cost of the 34 prediction modes with odd numbers, i.e., prediction blocks 1, 3,..., 67. Then, these 34 costs are sorted and filtered in ascending order to obtain the four prediction modes with the lowest costs, let them be prediction modes 61, 21, 35, and 3. Then, the costs of the neighboring prediction modes 62, 60, 22, 20, 36, 34, 4, and 2 of these five prediction modes are calculated, and then the prediction modes 61, 62, 60, 21, 22, 20, 35, 36, 34, 3, 4, and 2 are sorted by cost. Select 5 cost patterns from these, for example, prediction patterns 60, 21, 36, 34, 3. This will form the final list of prediction patterns.

[0142] Optionally, referring to Figure 7, the method applied to the decoding end can be modified. Taking the most likely candidate prediction mode list for angle prediction modes as an example, there are a total of 67 angle prediction modes, and the framework in Figure 7 can be used. First, obtain the prediction modes of the reconstructed neighboring blocks of the current block to obtain the prediction mode sequence, such as mode 61,21,35,3,18,36. Calculate the residual blocks for the prediction blocks of these 6 modes according to the process in Figure 7 to obtain the residual block sequence 61,21,35,3,18,36. Then, calculate the cost to obtain the cost sequence [61,1111],[21,2123],[35,926],[3,875],[18,7854],[36,3542]. After sorting by cost, the prediction mode sequence 3,35,61,21,36,18 is obtained. Select the top 5 lowest costs as the most likely mode list, i.e., 3,35,61,21,36.

[0143] Optionally, if the encoding and / or decoding mode is a block vector or motion vector (hereinafter referred to as vector) of encoding and / or decoding, then for each vector, the second cost of the residual block can be determined or obtained according to the first cost of the elements in the current block, and at least one vector can be selected and / or sorted according to the second cost corresponding to each vector.

[0144] For example, let's take the decoding end as an example. As shown in Figure 8, the template region information of the current block is obtained, and K types of vectors are used to predict the template region, generating K types of predicted regions respectively. That is, predicting vector 1 yields predicted region 1, ..., and predicting vector K yields predicted region K. Optionally, the vector can be a block vector or a motion vector. Optionally, the decoding end cannot obtain the original pixels of the current block. Therefore, the decoding end usually finds a similar block / template, an approximate block / template, or a neighboring block / template for the current block to calculate the cost, which is denoted as the template region in this example. This template region has been reconstructed.

[0145] Optionally, the K predicted regions are processed by the subtractor in Figure 8 to obtain K residual regions. That is, residual region 1 = reconstructed template region - predicted region 1, ..., residual region K = reconstructed template region - predicted region K. The cost calculation module in Figure 8 can be used to calculate the cost of each of the K residual regions to obtain K cost values, such as cost 1 corresponding to residual region 1 and cost K corresponding to residual region K. The cost sorting module in Figure 8 sorts the K costs, for example, according to the magnitude of cost 1 to cost K, to obtain an ordered list [vector, cost]. The vector filtering module in Figure 8 filters the ordered list to obtain a list of most likely vectors. For example, selecting the top 5 vectors with the smallest cost yields the list of most likely vectors.

[0146] Optionally, if the encoding and / or decoding mode is an encoding and / or decoding filter, then for each filter, the second cost of the residual block can be determined or obtained based on the first cost of the elements in the current block, and at least one filter can be selected and / or sorted based on the second cost corresponding to each filter.

[0147] For example, let's take the decoding end as an example. As shown in Figure 9, the template region information of the current block is obtained, and K filters are applied to the template region to generate K filtered regions. That is, filtering with filter 1 yields filtered region 1, ..., filtering with filter K yields filtered region K. Optionally, the decoding end cannot obtain the original pixels of the current block. Therefore, the decoding end usually finds a similar block / template, an approximate block / template, or a neighboring block / template for the current block to calculate the cost, which is denoted as the template region in this example. This template region has already been reconstructed.

[0148] Optionally, the K filtering regions are processed by the subtractor in Figure 9 to obtain K residual regions. That is, residual region 1 = reconstruction template region - filtering region 1, ..., residual region K = reconstruction template region - filtering region K. The cost calculation module in Figure 9 can be used to calculate the cost of each of the K residual regions to obtain K cost values, such as cost 1 corresponding to residual region 1 and cost K corresponding to residual region K. The cost sorting module in Figure 9 sorts the K costs, for example, according to the magnitude of cost 1 to cost K, to obtain an ordered list [filter, cost]. The filter filtering module in Figure 9 filters the ordered list to obtain a list of most likely filters. For example, selecting the top 5 filters with the lowest cost yields the list of most likely filters.

[0149] Optionally, step S20 includes: determining the cost of at least one encoding and / or decoding mode based on at least one second cost, and selecting and / or sorting the encoding and / or decoding modes based on the cost of at least one encoding and / or decoding mode.

[0150] Optionally, the cost of at least one encoding and / or decoding mode is determined based on the cost of at least one residual region, and the encoding and / or decoding modes are selected and / or ordered based on the cost of at least one encoding and / or decoding mode.

[0151] Optionally, after determining the second cost of the residual block, the cost of at least one encoding and / or decoding mode can be determined based on the second cost of at least one residual block. For example, the second cost of the residual block in each encoding and / or decoding mode can be determined and used as the cost of the encoding and / or decoding mode.

[0152] Optionally, the second cost of the residual block includes at least one of the cost of the prediction mode, the cost of the block partitioning mode, the cost of the motion vector, the cost of the block vector, and the cost of the filter.

[0153] Optionally, the cost of the encoding and / or decoding mode may include at least one of the following: the cost of the prediction mode, the cost of the block partitioning mode, the cost of the motion vector, the cost of the block vector, and the cost of the filter.

[0154] Optionally, for each prediction mode, the block to be predicted can be predicted according to the prediction mode to obtain the prediction block, the residual block can be determined based on the original block and the prediction block, the second cost of the residual block can be determined, and the second cost of the residual block can be used as the cost of the prediction mode.

[0155] Optionally, a second cost of the residual block can be calculated once for each block partitioning pattern, and the calculated second cost of the residual block can be used as the cost of that block partitioning pattern.

[0156] Optionally, for each block vector, a prediction region determined based on the block vector can be determined, and a residual region can be determined based on the reconstruction template region and the prediction region, and a second cost of the residual region can be determined as the cost of the block vector.

[0157] Optionally, for each motion vector, a prediction region determined based on the motion vector can be determined, and a residual region can be determined based on the reconstruction template region and the prediction region, and a second cost of the residual region can be determined as the cost of the motion vector.

[0158] Optionally, for each filter, a filtering region determined according to the filter can be determined, and a residual region can be determined based on the reconstruction template region and the filtering region, and a second cost of the residual region can be determined as the cost of the filter.

[0159] Optionally, after determining the cost of at least one encoding and / or decoding mode, the most likely modes can be sorted in descending order to obtain a list of most likely modes. Alternatively, at least one encoding and / or decoding mode can be selected from at least one encoding and / or decoding mode based on the cost of at least one encoding and / or decoding mode.

[0160] Optionally, by determining the cost of at least one encoding and / or decoding mode based on the second cost of at least one residual block, and selecting and / or sorting the encoding and / or decoding modes based on the cost of at least one encoding and / or decoding mode, it is possible to determine, for different encoding and / or decoding modes, such as prediction mode, block partitioning mode, motion vector, block vector and filter, at least one of them, based on the second cost of the residual block determined by the first cost of the elements in the current block or obtained.

[0161] In this embodiment, by determining or obtaining the second cost of the residual block based on the first cost of the elements within the current block, and selecting and / or sorting the encoding and / or decoding modes based on the second cost, it is possible to comprehensively consider the first cost of the elements within the current block when calculating the cost of entropy encoding at different stages of video encoding and / or decoding. This can improve the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby improving the efficiency of video encoding and / or decoding.

[0162] Second Embodiment

[0163] Based on the first embodiment, a second embodiment of this application is proposed. In the second embodiment, the method for determining or obtaining the first cost of the elements within the current block includes at least one of methods one to six.

[0164] Method 1: Obtain the first cost from the first lookup table based on at least one of the original value, absolute value, and preset number of digits of any element in the current block;

[0165] Optionally, the first lookup table can be a pre-set table or array containing the cost of elements, for example, the first lookup table can be [key, cost], where the key can be the original value, absolute value, number of digits, etc.

[0166] Optionally, when the current block is a residual block, the original value of an element can be the pixel value at the pixel position in the residual block. Therefore, for any element within the residual block, a query can be performed from the first lookup table to determine the cost corresponding to that element, and this cost can be used as the first cost.

[0167] Optionally, the absolute value of any element within the residual block, such as the absolute value of a pixel, can be determined, and a query can be performed from the first lookup table based on the absolute value to determine the cost corresponding to the matching element, which is then used as the first cost.

[0168] Optionally, the preset number of bits in the base of any element within the residual block can also be determined, such as converting from decimal to binary. For example, if the original value of an element in the residual block is 13, after binary conversion it becomes 1101, which is a 4-bit binary number. Therefore, the preset number of bits in the base for the original value 13 can be determined to be 4. Optionally, a lookup table can be performed to determine the cost corresponding to the preset number of bits, such as the cost corresponding to 4, thus obtaining the first cost.

[0169] Optionally, when the current block is a transform block, the original value can be the value at any position in the transform block, such as the value of the pixel at that element position after transformation.

[0170] Optionally, when the current block is a quantization block, the original value can be the value at any position in the quantization block, such as the value of the pixel at that element position after transformation quantization.

[0171] Optionally, when the current block is a transform block or a quantization block, the method of obtaining the first cost from the first lookup table is similar to the method of obtaining the first cost from the first lookup table when the current block is a residual block, as described above.

[0172] For example, as shown in Figure 10, the current block can be a residual block R, a transform block T, or a quantization block Q. The input to the lookup table is the absolute value or number of bits of an element, and the output is the first cost of the element. Optionally, a cost can be calculated for each element's original value, absolute value, number of bits, or preset number of bits, thus forming a lookup table from multiple [key, cost] pairs. As shown in Figure 10, the lookup table generation method can be selected by using switch 1 and switch 2.

[0173] Optionally, Figure 10 contains two transformation modules (i.e., Transformation 1 and Transformation 2) and two quantization modules (i.e., Quantization 1 and Quantization 2). Transformation 2 and Quantization 2 can be used to calculate the target cost to generate an overdetermined system of equations. For example, the quantization modules in the DCT transformation and protocol can be selected to obtain a more accurate target cost. Transformation 1 and Quantization 1 are used for the actual calculation of the block cost.

[0174] Optionally, as shown in Figure 10, taking the lookup table for generating the absolute value of transform block T as an example, the absolute value module is connected to transform 1 via selection switch 1, and the absolute value module is connected to the statistics module via selection switch 2. Optionally, selection switch 1 can also be connected to residual block R or quantization 1. Selection switch 2 can also be connected to the bit count module.

[0175] Optionally, the residual block R is subjected to block transformation 1 to obtain the transform block T. For example, the Hadamard transform can be used, where T = HRH, and H is the Hadamard matrix. The transform block T has the same size as R.

[0176] Optionally, as shown in Figure 10, all elements r1,...,ri,... of the residual block R can be transformed by Transform 1 to obtain Transform Block T, where all elements of Transform Block T are t1,...,ti,.... Then, Transform Block T can be quantized by Quantization 1 to obtain Quantization Block Q, where all elements of Quantization Block Q are q1,...,qi,....

[0177] Optionally, all elements t1,...,ti,... of the transformation block T can be scanned sequentially. The absolute values ​​of these elements are then processed using the absolute value module to obtain the non-negative integer sequence t1,...,ti,....

[0178] Optionally, the range of values ​​for elements in the transform block T is determined by the number of bits in the original pixel and the block size, width W and height H. Therefore, the non-negative integer sequence t1,...,ti,... is |t1|…|t WH |

[0179] Alternatively, if the pixel bit count is 10, then the maximum value of the element can be 2. 10 WH. The maximum value of the elements in the residual block R is 2. 10 The maximum value of the quantized block Q also depends on the scaling factor S of the quantization, which is a maximum value of 2. 10 WH / S.

[0180] Optionally, all elements r1,...,ri,... of the residual block R can be sequentially input into the Transform 2, Quantization 2, and Entropy Coding modules shown in Figure 10. Optionally, the Entropy Coding module can use context-based adaptive binary arithmetic coding to obtain the target cost y of the residual block. k.

[0181] Optionally, the values ​​of the non-negative integer sequence t1,...,ti,... can be counted using the statistics module, and its maximum value can be denoted as n. This yields the number of values ​​0, the number of values ​​n, etc., i.e., the number of values ​​0...n are C respectively. i0 ...C in Let the costs of elements 0...n be x0...xn respectively. n Then, by performing calculations using overdetermined linear equations, a lookup table [key, cost] is obtained.

[0182] Optionally, the number of bits for all elements t1,...,ti,... can be determined by the bit count module, for example, b(t1),...,b(ti),..., and then the number of bits can be counted by the statistics module to obtain the number of values ​​0...n.

[0183] Optionally, based on the results obtained from the statistical module, a linear equation is obtained, as shown in Formula (I) below. C i,0 x0+…+C i,n x n =y i Formula (1);

[0184] Optionally, different blocks can be selected (e.g., different residual blocks are transformed to obtain different transformed blocks), and the above process can be repeated multiple times to obtain formulas corresponding to multiple formulas (I), which constitute the overdetermined linear equation system. For example, if repeated m+1 times, and m is greater than n, where m and n are both integers, then the overdetermined linear equation system includes the following formula (II).

[0185] Alternatively, the overdetermined linear equation system can be solved using the least squares method to obtain x0……x n The non-negative optimal solution is obtained using n+1 data pairs [i, x]. i Generate a lookup table for 0 ≤ i ≤ n.

[0186] For example, as shown in Figure 11, the current block can be a residual block R, a transform block T, or a quantization block Q. The input to the lookup table is the absolute value or absolute value bit count of the element, and the output is the first cost of the element. Optionally, a cost can be calculated for each element's original value, absolute value, absolute value bit count, or preset number of bits, thus forming a lookup table from multiple [key, cost] pairs. As shown in Figure 11, the lookup table generation method can be selected by using switch 1 and switch 2.

[0187] Optionally, as shown in Figure 11, there exists a residual block R, and all elements of the residual block are r1,...,ri,... After transformation, a transform block T is obtained, and all elements of the transform block T are t1,...,ti,... After quantizing the transform block T, a quantization block Q is obtained, and all elements of the quantization block Q are q1,...,qi,...

[0188] Optionally, quantization can be performed using bit shift operations or table lookup operations. For example, qi = ti / 16 can be calculated using qi = (ti + 8) >> 4. Optionally, all elements q1...q of the quantization block Q can be scanned. WH That is, taking the absolute value of (q1,...,qi,... in Figure 11) yields the non-negative integer sequence |q1|...|q WH That is, (|q1|,...,|q in Figure 11) i |,..), then for the non-negative integer sequence |q1|…|q WH |To find the number of bits, we obtain a binary bit sequence b(q1)…b(q) wH That is, (b(q1),...,b(qi),...) in Figure 11, for the binary bit sequence b(q1)...b(q) WH For each element in the (), the accumulation module can obtain the cost value by querying the table and accumulate all the cost values ​​to obtain the block cost. For example, the cost of the quantization block is obtained by accumulating the cost, and then the second cost of the residual block is determined based on the cost of the quantization block.

[0189] Optionally, by obtaining the first cost of an element in the current block from the first lookup table based on at least one of the original value, absolute value, and preset number of digits of any element in the current block, the accuracy of the obtained first cost can be improved. This enables the subsequent determination of the second cost of the residual block based on the first cost, and the selection and / or sorting of encoding and / or decoding modes based on the second cost of the residual block. This improves the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby improving the efficiency of video encoding and / or decoding.

[0190] Method 2: For any two elements in the current block, perform weighted processing based on the first weight of one element to obtain the first cost of the element, and perform weighted processing based on the second weight of the other element to obtain the first cost of the other element.

[0191] Optionally, the first weight is not equal to the second weight.

[0192] Optionally, different weights can be set for any two elements within the current block, namely a first weight and a second weight. For example, the first weight can be used when the original value of an element is greater than or equal to a preset element threshold, and / or the second weight can be used when the original value of an element is less than the preset element threshold. Another example is that the first weight, with a value of 1 / 2, can be used for the element in the first position, and / or the second weight, with a value of 1, can be used for the element in the second position. Optionally, the preset element threshold can be a pre-set threshold; optionally, the first weight can be the weight of a specified subset of elements; optionally, the second weight can be the weight of another subset of elements; optionally, other methods can also be set to determine the weight corresponding to each element within the current block, without restriction.

[0193] Optionally, for any element in the current block, the cost value of the element can be calculated and the weight corresponding to the element can be determined. If it is the first weight, the cost value of the element can be weighted according to the first weight to obtain the first cost of the element. And / or, if it is the second weight, the cost value of the element can be weighted according to the second weight to obtain the first cost of the element.

[0194] Optionally, by weighting any two elements within the current block according to the first weight of one element to obtain the first cost, and by weighting the other element according to the second weight to obtain the first cost of the other element, the accuracy of the obtained first cost can be improved. This enables the subsequent determination of the second cost of the residual block based on the first cost, and the selection and / or sorting of encoding and / or decoding modes based on the second cost of the residual block. This improves the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby improving the efficiency of video encoding and / or decoding.

[0195] Method 3: The elements in the current block include the first element and the second element. The third cost is obtained from the second lookup table based on the original value or absolute value of the first element in the current block. The first cost of the first element is determined based on the third cost. The fourth cost is obtained from the second lookup table based on the preset number of radixes of the second element in the current block. The first cost of the second element is determined based on the fourth cost.

[0196] Optionally, the first element and the second element can be any two different elements within the current block.

[0197] Optionally, the second lookup table can be similar to the first lookup table; they can be the same or different.

[0198] Optionally, for the first element, the third cost can be obtained from the second lookup table based on the original value or absolute value of the first element in the current block. The specific method of obtaining the cost can be referred to Method 1.

[0199] Optionally, after obtaining the third cost of any first element, the third cost can be processed to obtain the first cost of the first element, or the third cost can be directly used as the first cost of the first element.

[0200] Optionally, when processing the third cost, the third cost obtained based on the original value and the third cost obtained based on the absolute value can be determined, and the maximum or minimum cost among the two third costs can be selected as the first cost. Alternatively, the two third costs can be averaged to obtain the first cost.

[0201] Alternatively, the cost of the first element can be calculated using a function calculation formula (such as SAD or SATD), and the first cost of the first element can be determined based on the calculated cost and the third cost, such as selecting the maximum or minimum cost as the first cost.

[0202] Optionally, for the second element, the fourth cost can be obtained from the second lookup table based on the preset number of radixes of the second element in the current block. The specific method of obtaining the cost can be referred to Method 1.

[0203] Optionally, after obtaining the fourth cost of any second element, this fourth cost can be directly used as the first cost of the second element. Alternatively, the fourth cost can be processed to obtain the first cost of the second element.

[0204] Optionally, when processing the fourth cost, the cost of the second element can be calculated using a function calculation formula, and the first cost of the second element can be determined based on the calculated cost and the fourth cost, such as selecting the maximum or minimum cost as the first cost.

[0205] Optionally, by obtaining a third cost from a second lookup table based on the original or absolute value of the first element in the current block, determining the first cost of the first element based on the third cost, obtaining a fourth cost from the second lookup table based on the preset number of digits of the second element in the current block, and determining the first cost of the second element based on the fourth cost, the accuracy of the obtained first cost can be improved. This enables the subsequent determination of the second cost of the residual block based on the first cost, and the selection and / or sorting of encoding and / or decoding modes based on the second cost of the residual block. This improves the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby improving the efficiency of video encoding and / or decoding.

[0206] Method 4: Obtain the third weight from the third lookup table based on any element in the current block, and perform weighted processing based on the third weight to obtain the first cost;

[0207] Optionally, the third lookup table can be a table containing the mapping relationship between elements and weights.

[0208] Optionally, for any element within the current block, a query can be performed in the third lookup table to obtain the weight corresponding to that element, and this weight can be used as the third weight.

[0209] Optionally, the cost of the element in the current block can be calculated using a function calculation formula, and then the cost of the element can be weighted according to the third weight corresponding to the element obtained from the third lookup table to obtain the first cost. For example, the first cost of the element can be obtained by multiplying the third weight by the cost of the element.

[0210] Optionally, by obtaining a third weight from a third lookup table based on any element in the current block, and performing weighted processing based on the third weight to obtain a first cost, the accuracy of the obtained first cost can be improved. This enables the subsequent determination of the second cost of the residual block based on the first cost, and the selection and / or sorting of encoding and / or decoding modes based on the second cost of the residual block. This improves the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby improving the efficiency of video encoding and / or decoding.

[0211] Method 5: For any element in the current block, calculate the fifth cost based on the first cost function, calculate the sixth cost based on the second cost function, and determine the first cost based on the fifth and sixth costs;

[0212] Optionally, for any element within the current block, at least two different cost functions can be used for cost calculation.

[0213] Optionally, the first cost function and the second cost function are different. For example, the first cost function can be at least one of the following: sum of absolute errors, sum of absolute transformation errors, sum of squared differences, mean absolute difference, mean squared error, etc. The second cost function can be different from the first cost function and can be at least one of the following: sum of absolute errors, sum of absolute transformation errors, sum of squared differences, mean absolute difference, mean squared error, etc.

[0214] Optionally, for any element within the current block, a fifth cost can be calculated using a first cost function, such as by using the sum of absolute errors. For example, the predicted and reconstructed values ​​of the element are input into the first cost function for calculation to obtain the fifth cost. Optionally, a sixth cost can be calculated using a second cost function, such as by using the sum of absolute transformation errors. For example, the predicted and reconstructed values ​​of the element are input into the second cost function for calculation to obtain the sixth cost.

[0215] Optionally, after determining the fifth and sixth costs of the same element within the current block, the maximum or minimum value of the fifth and sixth costs can be selected as the first cost of the element. Alternatively, the average of the fifth and sixth costs can be calculated to obtain the first cost of the element.

[0216] Alternatively, at least one cost for any element in the current block can be calculated based on different cost functions, and the first cost of that element can be determined or obtained based on the calculated at least one cost.

[0217] For example, for any element in the current block, a cost is calculated using the sum of absolute errors, a cost is calculated using the sum of absolute transformation errors, a cost is calculated using the sum of squared differences, a cost is calculated using the mean absolute difference, and a cost is calculated using the mean squared error. Based on these five costs, the first cost of the element is determined or obtained, such as selecting the minimum or maximum cost as the first cost.

[0218] Optionally, by calculating the fifth cost based on the first cost function and the sixth cost based on the second cost function for any element in the current block, and determining the first cost based on the fifth and sixth costs, the accuracy of the obtained first cost can be improved. This enables the subsequent determination of the second cost of the residual block based on the first cost, and the selection and / or sorting of encoding and / or decoding modes based on the second cost of the residual block. This improves the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby improving the efficiency of video encoding and / or decoding.

[0219] Method 6: If the text information obtained in the bitstream is within the first range, then the first cost is calculated using the first element calculation method, and / or, if the text information obtained in the bitstream is within the second range, then the first cost is calculated using the second element calculation method, which is different from the first element calculation method.

[0220] Optionally, the text information may include tagging information such as labels and indexes. These can be tags encoded into the bitstream during encoding at the encoding end.

[0221] Alternatively, the text information can be a text number, a binary character, or a mark represented in other forms.

[0222] Optionally, the first range can be a range associated with text information that is set in advance by the user. For example, the first range can be a range containing scattered marker values ​​such as 0, 2, 4, 6, etc., that is, the first range is {0, 2, 4, 6}. The first range can also be a range containing a single marker value, that is, the first range is {2}. The first range can also be a numerical interval, such as the numerical interval corresponding to 1-5, that is, the first range is {1, 2, 3, 4, 5}.

[0223] Optionally, the second range can be a range associated with the text information pre-defined by the user, and can be different from the first range. For example, if the first range is {1,2,3,4,5}, then the second range can be {6,7,8,9}.

[0224] Optionally, if the processing device is a decoding end, and the text information obtained by the decoding end in the bitstream is within the first range, it can be determined that the first cost is calculated using the first element calculation method for the elements of the current block.

[0225] Optionally, if the text information obtained by the decoding end in the bitstream is within the second range, it can be determined that the first cost is calculated using the second element calculation method for the elements of the current block.

[0226] Optionally, the calculation method for the first element may include at least one of the following: obtaining a first cost from a first lookup table based on at least one of the original value, absolute value, and preset number of radixes of any element in the current block; for any two elements in the current block, obtaining a first cost of one element by weighting it according to a first weight, and obtaining a first cost of the other element by weighting it according to a second weight; the elements in the current block include a first element and a second element, obtaining a third cost from a second lookup table based on the original value or absolute value of the first element in the current block, determining the first cost of the first element based on the third cost, obtaining a fourth cost from a second lookup table based on a preset number of radixes of the second element in the current block, and determining the first cost of the second element based on the fourth cost; obtaining a third weight from a third lookup table based on any element in the current block, and obtaining a first cost by weighting it according to the third weight; for any element in the current block, calculating a fifth cost based on a first cost function, calculating a sixth cost based on a second cost function, and determining the first cost based on the fifth and sixth costs.

[0227] Optionally, the calculation method for the second element may differ from that for the first element, and may include at least one of the following: obtaining a first cost from a first lookup table based on at least one of the original value, absolute value, and preset number of digits of any element in the current block; for any two elements in the current block, obtaining a first cost of one element by weighting it according to a first weight, and obtaining a first cost of the other element by weighting it according to a second weight; the elements in the current block include a first element and a second element, obtaining a third cost from a second lookup table based on the original value or absolute value of the first element in the current block, determining the first cost of the first element based on the third cost, obtaining a fourth cost from a second lookup table based on a preset number of digits of the second element in the current block, and determining the first cost of the second element based on the fourth cost; obtaining a third weight from a third lookup table based on any element in the current block, and obtaining a first cost by weighting it according to the third weight; for any element in the current block, calculating a fifth cost based on a first cost function, calculating a sixth cost based on a second cost function, and determining the first cost based on the fifth and sixth costs.

[0228] Optionally, the decoding end can determine the method of calculating the first cost based on the range of the text information obtained in the bitstream, which can improve the accuracy of the calculated first cost. This allows for the subsequent determination of the second cost of the residual block based on the first cost, and the selection and / or sorting of encoding and / or decoding modes based on the second cost of the residual block. This can improve the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby improving the efficiency of video encoding and / or decoding.

[0229] The technical solution of this embodiment can improve the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby improving the efficiency of video encoding and / or decoding.

[0230] Third Embodiment

[0231] Based on any of the above embodiments of this application, a third embodiment of this application is proposed. In the third embodiment, step S10 includes:

[0232] The second cost of the residual block is determined or obtained based on the first cost of at least one element within at least one sub-block of the current block.

[0233] Optionally, the cost of the residual region is determined or obtained based on the first cost of at least one element within at least one sub-block of the current block.

[0234] Alternatively, in one feasible embodiment, the value of each position in the current block and / or sub-block is used as an element.

[0235] Optionally, when calculating the first cost of at least one element within at least one sub-block of the current block, it can be calculated using any of the methods described in Methods 1 to 6 above.

[0236] Optionally, a sub-block can be a sub-residual block obtained by dividing a residual block, a sub-transform block obtained by dividing a transform block, or a sub-quantization block obtained by dividing a quantization block.

[0237] Optionally, when the current block is a residual block, at least one sub-residual block obtained after the residual block is divided can be determined, and a first cost of at least one element within the at least one sub-residual block can be determined, such as the first cost of at least one pixel. Then, the second cost of the residual block can be calculated and determined based on the first cost of the at least one element. For example, the cost of at least one sub-residual block can be determined first based on the first cost of at least one element within the at least one sub-residual block, and the second cost of the residual block can be determined based on the cost of the at least one sub-residual block. Optionally, the first costs of at least one element within the sub-residual block can be accumulated to obtain the cost of the sub-residual block. Alternatively, the first cost of at least one element within the sub-residual block can be calculated using an appropriate formula, such as taking the maximum or minimum value or using a weighted summation, to obtain the cost of the sub-residual block. Optionally, the cost of at least one sub-residual block can be calculated in the same way to obtain the second cost of the residual block.

[0238] Optionally, when the current block is a transform block, a sub-transformed block obtained by transforming at least one sub-residual block can be determined. A second cost of the residual block is calculated based on the first cost of at least one element within the at least one sub-transformed block. For example, the cost of at least one sub-transformed block can be determined based on the first cost of at least one element within the at least one sub-transformed block, the cost of at least one sub-residual block can be determined based on the cost of the at least one sub-transformed block, and the second cost of the residual block can be determined based on the cost of the at least one sub-residual block. Optionally, the first cost of at least one element within the at least one sub-transformed block can be calculated using the methods described above, such as accumulation, taking the maximum or minimum value, or weighted summation, to obtain the cost of at least one sub-transformed block.

[0239] Optionally, when the current block is a quantization block, a sub-quantization block obtained after transforming and quantizing at least one sub-residual block can be determined. A second cost of the residual block is calculated based on the first cost of at least one element within the at least one sub-quantization block. For example, the cost of at least one sub-quantization block can be determined based on the first cost of at least one element within the at least one sub-quantization block, the cost of at least one sub-residual block can be determined based on the cost of the at least one sub-quantization block, and the second cost of the residual block can be determined based on the cost of the at least one sub-residual block. Optionally, the first cost of at least one element within the at least one sub-quantization block can be calculated using the methods described above, such as accumulation, taking the maximum or minimum value, or weighted summation, to obtain the cost of at least one sub-quantization block.

[0240] Optionally, by determining or obtaining the second cost of the residual block based on the first cost of at least one element within at least one sub-block of the current block, it can be ensured that the second cost of the residual block is closely related to the first cost of the element within the sub-block, thereby improving the accuracy of the obtained second cost of the residual block. This facilitates the subsequent selection and / or sorting of encoding and / or decoding modes based on the second cost of the residual block, improving the accuracy of sorting or filtering multiple encoding and / or decoding modes, and thus improving the efficiency of video encoding and / or decoding.

[0241] Optionally, in one embodiment, the determination or acquisition of at least one sub-block of the current block includes at least one of methods seven to nine.

[0242] Method 7: Determine or obtain at least one sub-block based on at least one sub-residual block obtained by partitioning the residual block corresponding to the current block;

[0243] Optionally, when dividing the residual block corresponding to the current block, at least one division method can be used to divide the residual block to obtain at least one sub-residual block. Optionally, the sub-residual block obtained by division can be rectangular or non-rectangular, such as trapezoidal, pentagonal, etc.

[0244] Optionally, the residual block can be divided into at least one sub-residual block based on at least one dividing line. Alternatively, at least one location point within the residual block can be selected as a dividing point, and at least two non-coincident dividing lines can be extended from the dividing point to the edges of the residual block, dividing the residual block into at least one sub-residual block based on the dividing lines. Alternatively, the current block can be divided into at least one non-rectangular region consisting of at least two rectangular regions to obtain sub-residual blocks.

[0245] Optionally, after dividing the residual block into at least one sub-residual block, the sub-residual block can be directly used as the sub-block. Alternatively, the sub-residual block can be deformed, such as by pixel shifting, to obtain the sub-block.

[0246] Optionally, the second cost of the residual block can be determined or obtained based on the first cost of at least one element within at least one sub-residual block.

[0247] Optionally, at least one sub-block is determined or obtained based on at least one sub-residual region obtained by dividing the residual region corresponding to the current block. Optionally, the sub-residual region may include at least one sub-residual block.

[0248] Optionally, by determining or obtaining at least one sub-block based on at least one sub-residual block obtained by dividing the residual block corresponding to the current block, the accuracy and effectiveness of the obtained sub-block are improved. This facilitates the subsequent determination of the second cost of the residual block based on the sub-block, and the selection and / or sorting of encoding and / or decoding modes based on the second cost of the residual block. This can improve the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby improving the efficiency of video encoding and / or decoding.

[0249] Method 8: Determine or obtain at least one sub-block based on the sub-transformed block obtained by transforming at least one sub-residual block;

[0250] Optionally, after dividing the residual block to obtain at least one sub-residual block, the at least one sub-residual block can be transformed to obtain a sub-transform block, and the sub-transform block can be used as a sub-block.

[0251] Optionally, at least one transformation method can be used to transform at least one sub-residual block to obtain at least one sub-transformed block. Optionally, a portion of the sub-residual blocks can be transformed using one transformation method to obtain sub-transformed blocks, while another portion of the sub-residual blocks can be transformed using a different transformation method to obtain sub-transformed blocks.

[0252] Optionally, the second cost of the residual block can be determined or obtained based on the first cost of at least one element within at least one sub-transformation block.

[0253] Optionally, by determining or obtaining at least one sub-block based on the sub-transformed block obtained by transforming at least one sub-residual block, the accuracy and effectiveness of the obtained sub-block are improved. This facilitates the subsequent determination of the second cost of the residual block based on the sub-block, and the selection and / or sorting of encoding and / or decoding modes based on the second cost of the residual block. This can improve the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby improving the efficiency of video encoding and / or decoding.

[0254] Method 9: Determine or obtain at least one sub-block based on the sub-quantization block obtained by quantizing at least one sub-transform block.

[0255] Optionally, after dividing the residual block to obtain at least one sub-residual block, the at least one sub-residual block can be transformed to obtain a sub-transform block, and the at least one sub-transform block can be quantized to obtain a sub-quantized block, which can then be used as a sub-block. Optionally, the quantization process can include scaling all elements within the sub-transform block.

[0256] Optionally, at least one sub-transform block can be quantized using at least one quantization method to obtain at least one sub-quantized block. Optionally, some sub-transform blocks can be quantized using one quantization method to obtain sub-quantized blocks, and other sub-transform blocks can be quantized using another quantization method to obtain sub-quantized blocks.

[0257] Optionally, the second cost of the residual block can be determined or obtained based on the first cost of at least one element within at least one sub-quantized block.

[0258] Optionally, by determining or obtaining at least one sub-block based on the sub-quantized block obtained by quantizing at least one sub-residual block, the accuracy and effectiveness of the obtained sub-block are improved. This facilitates the subsequent determination of the second cost of the residual block based on the sub-block, and the selection and / or sorting of encoding and / or decoding modes based on the second cost of the residual block. This can improve the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby improving the efficiency of video encoding and / or decoding.

[0259] Optionally, in one embodiment, step S10 includes at least one of the following methods ten to fourteen.

[0260] Method 10: Determine or obtain the seventh cost of at least one sub-block based on the first cost of at least one element within at least one sub-block, and determine or obtain the second cost of the residual block based on the seventh cost of at least one sub-block.

[0261] Optionally, a sub-block can be at least one of a sub-residual block, a sub-transform block, and a sub-quantization block.

[0262] Optionally, the seventh cost of a sub-block can be at least one of the costs of a sub-residual block, a sub-transform block, and a sub-quantization block.

[0263] Optionally, the cost of at least one sub-residual block is determined or obtained based on the first cost of at least one element within at least one sub-residual block, and the second cost of the residual block is determined or obtained based on the cost of at least one sub-residual block.

[0264] Optionally, the cost of at least one sub-transform block is determined or obtained based on the first cost of at least one element within at least one sub-transform block, and the second cost of the residual block is determined or obtained based on the cost of at least one sub-transform block.

[0265] Optionally, the cost of at least one sub-quantization block is determined or obtained based on the first cost of at least one element within at least one sub-quantization, and the second cost of the residual block is determined or obtained based on the cost of at least one sub-quantization block.

[0266] Optionally, when determining or obtaining the seventh cost of at least one sub-block based on the first cost of at least one element within at least one sub-block, an accumulation method can be used to accumulate the first cost of at least one element within at least one sub-block to obtain the seventh cost of at least one sub-block.

[0267] Optionally, after determining the seventh cost of at least one sub-block, the seventh cost of at least one sub-block can be processed in the same way (e.g., by accumulation) to obtain the second cost of the residual block.

[0268] Optionally, a seventh cost of at least one sub-block is determined or obtained based on a first cost of at least one element within at least one sub-block, and a cost of the residual region is determined or obtained based on the seventh cost of at least one sub-block.

[0269] Optionally, a seventh cost of at least one sub-block can be determined or obtained based on a first cost of at least one element within at least one sub-block, and a second cost of the residual block can be determined or obtained based on the seventh cost of at least one sub-block. This ensures that the second cost of the residual block is closely related to the first cost of the elements within the sub-block, thereby improving the accuracy of the obtained second cost of the residual block. This facilitates the subsequent selection and / or sorting of encoding and / or decoding modes based on the second cost of the residual block, improving the accuracy of sorting or filtering multiple encoding and / or decoding modes, and thus improving the efficiency of video encoding and / or decoding.

[0270] Method 11: Determine the cost of at least one sub-residual block based on the cost of at least one sub-transform block, and determine or obtain the second cost of the residual block based on the cost of at least one sub-residual block;

[0271] Optionally, the cost of at least one sub-transform block can be determined or obtained based on the first cost of at least one element within the sub-transform block.

[0272] Optionally, the determination or acquisition of the first cost of at least one element within the sub-transform block can be made by referring to any one of methods one to six in the above embodiments, simply by replacing the current block in methods one to six with the sub-transform block.

[0273] Optionally, after determining the first cost of at least one element within the sub-transform block, the cost of the sub-transform block can be calculated by accumulating, taking the maximum or minimum value, or by weighted summation.

[0274] Optionally, the cost of at least one sub-transform block can be input into a lookup table containing the mapping relationship between transform block costs and residual block costs to obtain the cost of the corresponding sub-residual block. Optionally, certain rules can be set to determine the cost of at least one sub-residual block based on the set rules and the cost of at least one sub-transform block. For example, if the set rule is that the cost of a sub-transform block equals the cost of a sub-residual block, then the cost of at least one sub-transform block is directly used as the cost of at least one sub-residual block. Alternatively, if the set rule is to add a constant value to the cost of a sub-transform block to obtain the cost of a sub-residual block, then a constant value can be added to the cost of at least one sub-transform block to obtain the cost of at least one sub-residual block.

[0275] Optionally, after determining the cost of at least one sub-residual block, the costs of at least one sub-residual block can be accumulated to obtain the second cost of the residual block, or the costs of at least one sub-residual block can be maximized, minimized, or weighted to obtain the second cost of the residual block.

[0276] Optionally, the cost of at least one sub-residual block is determined based on the cost of at least one sub-transform block, and the cost of the residual region is determined or obtained based on the cost of at least one sub-residual block. Alternatively, the cost of the sub-residual region is determined or obtained based on the cost of at least one sub-residual block, and the cost of the residual region is determined or obtained based on the cost of at least one sub-residual region.

[0277] Optionally, by determining the cost of at least one sub-residual block based on the cost of at least one sub-transform block, and determining or obtaining the second cost of the residual block based on the cost of at least one sub-residual block, it can be ensured that the second cost of the residual block is closely related to the first cost of the elements within the sub-block, thereby improving the accuracy of the obtained second cost of the residual block. This facilitates the subsequent selection and / or sorting of encoding and / or decoding modes based on the second cost of the residual block, improving the accuracy of sorting or filtering multiple encoding and / or decoding modes, and thus improving the efficiency of video encoding and / or decoding.

[0278] Method 12: Determine the cost of at least one sub-residual block based on the cost of at least one sub-quantized block, and determine or obtain the second cost of the residual block based on the cost of at least one sub-residual block;

[0279] Optionally, the cost of at least one sub-quantization block can be determined or obtained based on the first cost of at least one element within the sub-quantization block.

[0280] Optionally, the determination or acquisition of the first cost of at least one element within the sub-quantization block can be made by referring to any one of methods one to six in the above embodiments, simply by replacing the current block in methods one to six with the sub-quantization block.

[0281] Optionally, after determining the first cost of at least one element within the sub-quantized block, the cost of the sub-quantized block can be calculated by accumulating, taking the maximum or minimum value, or by weighted summation.

[0282] Optionally, the cost of at least one sub-quantization block can be input into a lookup table containing the mapping relationship between quantization block costs and residual block costs to obtain the cost of the corresponding sub-residual block. Optionally, certain rules can be set to determine the cost of at least one sub-residual block based on the set rules and the cost of at least one sub-quantization block. For example, if the set rule is that the cost of a sub-quantization block equals the cost of a sub-residual block, then the cost of at least one sub-quantization block is directly used as the cost of at least one sub-residual block. Alternatively, if the set rule is to add a constant value to the cost of a sub-quantization block to obtain the cost of a sub-residual block, then a constant value can be added to the cost of at least one sub-quantization block to obtain the cost of at least one sub-residual block.

[0283] Optionally, after determining the cost of at least one sub-residual block, the costs of at least one sub-residual block can be accumulated to obtain the second cost of the residual block, or the costs of at least one sub-residual block can be maximized, minimized, or weighted to obtain the second cost of the residual block.

[0284] Optionally, the cost of at least one sub-residual block is determined based on the cost of at least one sub-quantization block, and the cost of the residual region is determined or obtained based on the cost of at least one sub-residual block. Alternatively, the cost of the sub-residual region is determined or obtained based on the cost of at least one sub-residual block, and the cost of the residual region is determined or obtained based on the cost of at least one sub-residual region.

[0285] Optionally, by determining the cost of at least one sub-residual block based on the cost of at least one sub-quantization block, and determining or obtaining the second cost of the residual block based on the cost of at least one sub-residual block, it can be ensured that the second cost of the residual block is closely related to the first cost of the elements within the sub-block, thereby improving the accuracy of the obtained second cost of the residual block. This facilitates the subsequent selection and / or sorting of encoding and / or decoding modes based on the second cost of the residual block, improving the accuracy of sorting or filtering multiple encoding and / or decoding modes, and thus improving the efficiency of video encoding and / or decoding.

[0286] Method 13: Input at least one sub-block and / or the downsampled sample corresponding to the sub-block into the neural network, output the cost of at least one sub-residual block, and determine or obtain the second cost of the residual block based on the cost of at least one sub-residual block;

[0287] Alternatively, the second cost of obtaining the residual block can be calculated by using neural networks or model algorithms.

[0288] Optionally, at least one sub-block and / or its corresponding downsampled sample can be directly input into a neural network for model training, outputting the cost of at least one sub-residual block. Alternatively, at least one sub-block and / or its corresponding downsampled sample can also be directly input into a neural network for model training, outputting a second cost of the residual block. Optionally, the neural network can be a deep neural network or a large model.

[0289] Optionally, after determining the cost of at least one sub-residual block, the costs of at least one sub-residual block can be accumulated to obtain the second cost of the residual block, or the costs of at least one sub-residual block can be maximized, minimized, or weighted to obtain the second cost of the residual block.

[0290] Optionally, at least one sub-block and / or the downsampled portion corresponding to the sub-block is input into the neural network, and the cost of at least one sub-residual block is output. The cost of the residual region is determined or obtained based on the cost of at least one sub-residual block, or the cost of the sub-residual region is determined or obtained based on the cost of at least one sub-residual block, and the cost of the residual region is determined or obtained based on the cost of at least one sub-residual region.

[0291] Optionally, by inputting at least one sub-block and / or the downsampled portion corresponding to the sub-block into a neural network, and outputting the cost of at least one sub-residual block, and determining or obtaining a second cost of the residual block based on the cost of the at least one sub-residual block, the accuracy of the obtained second cost can be improved. This facilitates the subsequent selection and / or sorting of encoding and / or decoding modes based on the second cost of the residual block, thereby improving the accuracy of sorting or filtering multiple encoding and / or decoding modes and ultimately improving the efficiency of video encoding and / or decoding.

[0292] Method 14: Determine or obtain the second cost of the residual block based on the partitioning cost of at least two partitions for the residual block.

[0293] Optionally, the partitioning cost for each partition is determined or obtained based on the cost of the sub-residual blocks obtained by partitioning the residual block.

[0294] Optionally, different partitioning methods can be used to partition the residual blocks to obtain sub-residual blocks. Optionally, at least two partitioning methods can be at least one of the following methods a-c. Optionally, the at least two partitioning methods are not limited to methods a-c, and can also be other partitioning methods, such as partitioning based on user input instructions. In this embodiment, only methods a-c are used as examples.

[0295] Method a: Divide the residual block into at least one sub-residual block based on at least one dividing line;

[0296] Method b: Select at least one location point within the residual block as a split point, and extend at least two non-overlapping split lines from the split point to the edge of the residual block, dividing the residual block into at least one sub-residual block according to the split lines.

[0297] Method c divides the current block into at least one non-rectangular region consisting of at least two rectangular regions, resulting in a sub-residual block.

[0298] Optionally, the residual block can be divided using method a to obtain at least one sub-residual block. The cost of the at least one sub-residual block is then determined, for example, based on the first cost of at least one element within the at least one sub-residual block, to determine or obtain the cost of the at least one sub-residual block. Then, the cost of the residual block is determined or obtained based on the cost of the at least one sub-residual block, and this cost is used as the partitioning cost of method a.

[0299] Optionally, method b can be used to divide the residual block to obtain at least one sub-residual block, and determine the cost of at least one sub-residual block. For example, based on the first cost of at least one element in at least one sub-residual block, the cost of at least one sub-residual block can be determined or obtained, and then the cost of the residual block can be determined or obtained based on the cost of at least one sub-residual block, and used as the partitioning cost of method b.

[0300] Optionally, the residual block can be divided using method c to obtain at least one sub-residual block, and the cost of at least one sub-residual block can be determined. For example, the cost of at least one sub-residual block can be determined or obtained based on the first cost of at least one element in at least one sub-residual block, and then the cost of the residual block can be determined or obtained based on the cost of at least one sub-residual block, and used as the division cost of method c.

[0301] Optionally, after determining the partitioning costs of at least two partitions, the maximum or minimum partitioning cost can be selected from the partitioning costs of the at least two partitions as the second cost of the residual block. Alternatively, the second cost of the residual block can be obtained by averaging the partitioning costs of the at least two partitions, or other methods can be used to select one of the partitioning costs of the at least two partitions as the second cost of the residual block, etc.

[0302] For example, if partitioning cost of method a is the lowest, then after determining the partitioning costs of method a, method b, and method c, the partitioning cost of method a can be selected as the second cost of the residual block.

[0303] For example, as shown in Figure 12, the residual block can be divided using two partitioning methods. For instance, partitioning method 1 divides the residual block into sub-residual blocks, such as dividing the residual block into 6x6 = 36 sub-residual blocks. Each sub-residual block is input to Transform 1 module for transformation processing, resulting in 36 corresponding sub-transform blocks. For each sub-transform block, a sub-block cost can be calculated, and the costs of each sub-transform block are accumulated to obtain cost 1, i.e., cost 1 of partitioning method 1. Alternatively, partitioning method 2 can be used to divide the residual block into sub-residual blocks, such as dividing the residual block into 3x3 = 9 sub-residual blocks. Each sub-residual block is input to Transform 1 module for transformation processing, resulting in 9 corresponding sub-transform blocks. For each sub-transform block, a sub-block cost can be calculated, and the costs of each sub-transform block are accumulated to obtain cost 2, i.e., cost 2 of partitioning method 2. The final cost of the residual block is determined based on cost 1 and cost 2. For example, the minimum cost among cost 1 and cost 2 can be selected as the residual block cost, and the maximum cost can be selected as the residual block cost. Alternatively, cost 1 and cost 2 can be weighted to obtain the residual block cost. The weighted calculation formula can be as follows (Formula III).

[0304] Optionally, w1 is the weight of cost 1, w2 is the weight of cost 2, cost1 is the cost of cost 1, cost2 is the cost of cost 2, and cost is the cost of the residual block, i.e., the second cost of the residual block.

[0305] Optionally, the cost of the residual region is determined or obtained based on the partitioning cost of at least two partitions of the residual region.

[0306] Optionally, by determining or obtaining a second cost of the residual block based on the partitioning cost of at least two partitions for the residual block, the accuracy of the obtained second cost can be improved, which facilitates the subsequent selection and / or sorting of encoding and / or decoding modes based on the second cost of the residual block. This can improve the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby improving the efficiency of video encoding and / or decoding.

[0307] Optionally, in one embodiment, for mode ten, determining or obtaining a seventh cost of at least one sub-block based on a first cost of at least one element within at least one sub-block includes at least one of modes fifteen to seventeen.

[0308] Method 15: Determine or obtain the seventh cost of at least one sub-block based on the first cost of at least two elements within at least one sub-block;

[0309] Optionally, for the same sub-block, the first cost of computing at least two elements within the sub-block can be determined.

[0310] Optionally, the first cost of each element in the sub-block can be calculated and determined by referring to any one of the methods one to six in the above embodiments, that is, replacing the current block of any one of the methods one to six with the sub-block, and then calculating the first cost of the elements in the sub-block.

[0311] Optionally, the calculation methods for the first cost of at least two elements within a sub-block can be the same or different, and there is no restriction on this.

[0312] Optionally, after determining the first cost of at least two elements within a sub-block, the seventh cost of that sub-block can be calculated by accumulating, taking the maximum or minimum value, or by weighted summation. Alternatively, the seventh cost of the sub-block can be determined by averaging the first costs of the at least two elements or by selecting the maximum cost.

[0313] Optionally, the cost of at least one sub-residual block can be determined or obtained based on the first cost of at least two elements within at least one sub-residual block.

[0314] Optionally, the cost of at least one sub-transform block is determined or obtained based on the first cost of at least two elements within at least one sub-transform block.

[0315] Optionally, the cost of at least one sub-quantization block is determined or obtained based on the first cost of at least two elements within at least one sub-quantization block.

[0316] Optionally, the accuracy and effectiveness of obtaining the seventh cost of at least one sub-block can be improved by determining or obtaining the seventh cost of at least one sub-block based on the first cost of at least two elements within at least one sub-block.

[0317] Method 16: Determine or obtain the seventh cost of at least one sub-block based on the first cost of the symbol of the element within at least one sub-block;

[0318] Optionally, the sign of the sub-block element can include both positive and negative signs. Optionally, when the sign of the sub-block element is positive, the first cost of the elements within the sub-block is positive. And / or, when the sign of the sub-block element is negative, the first cost of the elements within the sub-block is negative.

[0319] Optionally, the seventh cost of at least one sub-block can be determined or obtained based on the sign and first cost of the elements within at least one sub-block. For example, based on the first cost of positive numbers within at least one sub-block, a lookup table is performed to obtain the seventh cost of at least one sub-block. Similarly, based on the first cost of negative numbers within at least one sub-block, a lookup table is performed to obtain the seventh cost of at least one sub-block. Furthermore, based on both the first cost of negative numbers and the first cost of positive numbers within at least one sub-block, a lookup table is performed to obtain the seventh cost of at least one sub-block.

[0320] Optionally, the cost of at least one sub-residual block is determined or obtained based on a first cost of the symbols of the elements within at least one sub-residual block. The cost of at least one sub-transform block is determined or obtained based on a first cost of the symbols of at least one sub-quantization block.

[0321] Optionally, the accuracy and effectiveness of obtaining the seventh cost of the sub-block can be improved by determining or obtaining the seventh cost of at least one sub-block based on the first cost of the symbol of the element within at least one sub-block.

[0322] Method 17 involves scanning the elements within at least one sub-block to obtain a scan sequence, and determining or obtaining the seventh cost of the sub-block based on the cost of the sub-sequence in the scan sequence.

[0323] Optionally, different scanning methods can be used to scan the elements within at least one sub-block to obtain at least one scanning sequence. For example, scanning from top to bottom or from left to right. Or, scanning line by line from left to right or from top to bottom.

[0324] Optionally, after scanning the elements within at least one sub-block to obtain a scan sequence, the cost of the sub-sequences in the scan sequence is determined, which can be done by looking up a table. After determining the cost of at least one sub-sequence, the cost of the at least one sub-sequence is calculated accordingly, such as by accumulation, to obtain the seventh cost of the sub-block.

[0325] Optionally, the cost of a subsequence can be determined or obtained based on the first cost of the elements within the sub-block, such as by summing the first costs of the elements contained in the subsequence to obtain the cost of the subsequence.

[0326] Optionally, the subsequence can be a non-zero subsequence, or the longest non-zero subsequence.

[0327] Optionally, elements within at least one sub-residual block are scanned to obtain a scan sequence, and the cost of the sub-residual block is determined or obtained based on the cost of the sub-sequences in the scan sequence. Elements within at least one sub-transform block are scanned to obtain a scan sequence, and the cost of the sub-transform block is determined or obtained based on the cost of the sub-sequences in the scan sequence. Elements within at least one sub-quantization block are scanned to obtain a scan sequence, and the cost of the sub-quantization block is determined or obtained based on the cost of the sub-sequences in the scan sequence.

[0328] For example, as shown in Figure 13, the residual block is transformed to obtain the transformed block. Two different scans are performed on the transformed block: Scan 1 is a top-down, left-to-right scan, and Scan 2 is a left-to-right, top-to-bottom line-by-line scan.

[0329] Optionally, if the elements within the transform block are scanned using scan 1, a scan sequence of 16 elements can be obtained. A sequence truncation module can then be used to extract subsequences from this scan sequence, such as the longest non-zero subsequence. For example, the longest non-zero subsequence of the scan sequence [0,3,-1,0,4,-2,0,0] is [3,-1,0,4,-2]. The cost of the longest non-zero subsequence can be calculated using the cost calculation module. For instance, the cost of the longest non-zero subsequence, i.e., cost 3, can be obtained by querying the lookup table [key, cost].

[0330] Optionally, if the elements within the transform block are scanned using scan 2, a scan sequence can be obtained. A sequence truncation module can then be used to extract subsequences from this scan sequence, such as the longest non-zero subsequence. The cost of the longest non-zero subsequence can be calculated using the cost calculation module. For example, the cost of the longest non-zero subsequence, i.e., cost 4, can be obtained by querying the lookup table [key, cost].

[0331] The final cost of the residual block is determined based on cost 3 and cost 4. For example, the minimum cost among cost 3 and cost 4 can be selected as the residual block cost, and the maximum cost can be selected as the residual block cost. Alternatively, cost 3 and cost 4 can be weighted to obtain the residual block cost. The weighted calculation formula can be as follows (IV).

[0332] Optionally, w3 is the weight of cost 3, w4 is the weight of cost 4, cost3 is the cost of cost 3, and cost4 is the cost of cost 4.

[0333] Optionally, by scanning the elements within at least one sub-block to obtain a scan sequence, and determining or obtaining the seventh cost of the sub-block based on the cost of the sub-sequence in the scan sequence, the validity of the obtained seventh cost can be ensured, thereby improving the accuracy and validity of the second cost of the residual block subsequently determined or obtained based on the seventh cost of at least one sub-block, so as to facilitate the selection and / or sorting of encoding and / or decoding modes based on the second cost of the residual block.

[0334] The technical solution of this embodiment can improve the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby improving the efficiency of video encoding and / or decoding.

[0335] Fourth embodiment

[0336] Based on any of the above embodiments of this application, a fourth embodiment of this application is proposed. In the fourth embodiment, the determination or acquisition of the second cost of the residual block includes at least one of the following methods eighteen to twenty-five.

[0337] Method 18: Calculate the eighth cost of the residual block according to the first cost function, calculate the ninth cost of the residual block according to the second cost function, and determine the second cost of the residual block based on the minimum value of the eighth cost and the ninth cost;

[0338] Optionally, the first cost function can be at least one of the following: sum of absolute errors, sum of absolute transformation errors, sum of squared differences, mean absolute difference, mean squared error, etc.

[0339] Optionally, the second cost function can be at least one of the following: sum of absolute errors, sum of absolute transformation errors, sum of squared differences, mean absolute difference, mean squared error, etc.

[0340] Optionally, the first cost function and the second cost function can be different.

[0341] Optionally, the eighth cost of the residual block can be calculated using the first cost function, and the ninth cost of the residual block can be calculated using the second cost function. The eighth cost and the ninth cost are then compared, and the minimum value between the eighth cost and the ninth cost is selected as the second cost of the residual block.

[0342] Optionally, the cost of the residual block can be calculated based on the absolute error, the cost of the residual block can be calculated based on the absolute transformation error, the point of the residual block can be calculated based on the sum of squared differences, the cost of the residual block can be calculated based on the mean absolute difference, the cost of the residual block can be calculated based on the mean squared error, and the minimum value among the five calculated costs of the residual block can be selected as the second cost of the residual block.

[0343] Optionally, the cost of the residual region is determined or obtained based on the first region cost of the residual region calculated by the first cost function and the second region cost of the residual region calculated by the second cost function. For example, the minimum value between the first region cost and the second region cost can be selected as the cost of the residual region, or the first region cost and the second region cost can be weighted to obtain the cost of the residual region.

[0344] Optionally, by calculating the eighth cost of the residual block according to the first cost function, calculating the ninth cost of the residual block according to the second cost function, and determining the second cost of the residual block based on the minimum value of the eighth and ninth costs, the effectiveness of the determined second cost of the residual block can be improved. This facilitates the subsequent selection and / or sorting of encoding and / or decoding modes based on the second cost of the residual block, thereby improving the accuracy of sorting or filtering multiple encoding and / or decoding modes and ultimately improving the efficiency of video encoding and / or decoding.

[0345] Method 19: Calculate the eighth cost of the residual block according to the first cost function, calculate the ninth cost of the residual block according to the second cost function, and perform weighted processing on the eighth cost and the ninth cost to obtain the second cost of the residual block;

[0346] Optionally, the first cost function and the second cost function can be derived from Method 18, which will not be repeated here.

[0347] Optionally, the eighth cost of the residual block can be calculated based on the first cost function, the ninth cost of the residual block can be calculated based on the second cost function, and the weights corresponding to the eighth cost and the ninth cost can be determined. The second cost of the residual block can be obtained by weighting the eighth cost, the weights corresponding to the eighth cost, the ninth cost, and the weights corresponding to the ninth cost.

[0348] Optionally, different weights corresponding to different cost functions can be set in advance, and then the weights corresponding to the eighth cost and the ninth cost can be determined based on the weights corresponding to different cost functions.

[0349] Optionally, by calculating the eighth cost of the residual block according to the first cost function and the ninth cost of the residual block according to the second cost function, and then weighting the eighth cost and the ninth cost, a second cost of the residual block can be obtained. This can improve the effectiveness of the determined second cost of the residual block, so that the encoding and / or decoding modes can be selected and / or sorted according to the second cost of the residual block. This can improve the accuracy of sorting or filtering multiple encoding and / or decoding modes, and thus improve the efficiency of video encoding and / or decoding.

[0350] Method 20: Input the residual block into the neural network and output the second cost of the residual block;

[0351] Optionally, the second cost of the residual block can be determined based on a neural network or model algorithm. This involves inputting the residual block into the neural network for computation and outputting the second cost of the residual block. The neural network is a pre-trained neural network.

[0352] Optionally, the residual region can be input into a neural network, and the cost of the residual region can be output.

[0353] Optionally, by determining the second cost of the residual block based on the actual network, the effectiveness of the determined second cost of the residual block can be improved, so that the encoding and / or decoding modes can be selected and / or sorted according to the second cost of the residual block. This can improve the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby improving the efficiency of video encoding and / or decoding.

[0354] Method 21: If the text information obtained in the bitstream is located in the third range, then the second cost of the residual block is calculated using the first cost function;

[0355] Alternatively, the text information can refer to method six in the above embodiments.

[0356] Optionally, the text information in this embodiment can be of the same type as the text information in Method Six, such as both being indexes. Alternatively, they can be of different types, such as one being an index and the other a tag.

[0357] Optionally, the third range can be a range associated with text information that is pre-defined by the user.

[0358] Optionally, the third range may be the same as or different from the first or second range.

[0359] Optionally, when the processing device is a decoding end, and the text information obtained by the decoding end in the bitstream is located in the third range, the second cost of the residual block can be calculated using the first cost function.

[0360] Optionally, the first cost function can be at least one of the following: sum of absolute errors, sum of absolute transformation errors, sum of squared differences, mean absolute difference, mean squared error, etc.

[0361] Optionally, if the text information obtained in the bitstream is located in the third range, the first cost function is used to calculate the cost of the residual region.

[0362] Optionally, when the text information obtained by the decoding end in the bitstream is within the third range, the second cost of the residual block is calculated using the first cost function. This can improve the effectiveness of the determined second cost of the residual block, so that the encoding and / or decoding modes can be selected and / or sorted according to the second cost of the residual block. This can improve the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby improving the efficiency of video encoding and / or decoding.

[0363] Method 22: If the text information obtained in the bitstream is located in the fourth range, then the second cost function is used to calculate the second cost of the residual block;

[0364] Alternatively, the text information can be provided in accordance with method 12 above.

[0365] Optionally, the fourth range can be a range associated with text information that is pre-defined by the user.

[0366] Optionally, the fourth range is different from the third range, and the fourth range may be the same as or different from the first or second range.

[0367] Optionally, when the processing device is a decoding end, and the text information obtained by the decoding end in the bitstream is located in the fourth range, the second cost function can be used to calculate the second cost of the residual block.

[0368] Optionally, the second cost function can be at least one of the following: sum of absolute errors, sum of absolute transformation errors, sum of squared differences, mean absolute difference, mean squared error, etc.

[0369] Optionally, the second cost function may differ from the first cost function.

[0370] Optionally, if the text information obtained in the bitstream is located in the fourth range, the cost of the residual region is calculated using the second cost function.

[0371] Optionally, when the text information obtained by the decoding end in the bitstream is within the fourth range, the second cost function is used to calculate the second cost of the residual block. This can improve the effectiveness of the determined second cost of the residual block, so that the encoding and / or decoding modes can be selected and / or sorted according to the second cost of the residual block. This can improve the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby improving the efficiency of video encoding and / or decoding.

[0372] Method 23: If the text information obtained in the bitstream is located in the fifth range, then the second cost of the residual block is determined or obtained based on the cost of at least one sub-residual block.

[0373] Alternatively, the text information can be provided in accordance with method 12 above.

[0374] Optionally, the fifth range can be a range associated with text information that is pre-defined by the user.

[0375] Optionally, the fifth range may be different from or partially the same as the first to fourth ranges mentioned above.

[0376] Optionally, when the processing device is a decoding end, and the text information obtained by the decoding end in the bitstream is located in the fifth range, the second cost of the residual block can be determined or obtained by using the cost of at least one sub-residual block.

[0377] Optionally, the cost of at least one sub-residual block may be determined or obtained based on the first cost of at least two elements within the at least one sub-residual block, or it may be determined or obtained based on the first cost of the sign of the elements within the at least one sub-residual block, or it may be determined or obtained by scanning the elements within the at least one sub-residual block to obtain a scan sequence, based on the cost of the subsequence in the scan sequence, or it may be determined or obtained based on a neural network or lookup table, or it may be determined or obtained based on the cost of at least one sub-transform block, or it may be determined or obtained based on the cost of at least one sub-quantization block.

[0378] Optionally, the method for determining or obtaining at least one element within at least one sub-residual block can refer to methods one to six in the above embodiments, that is, replacing the current block in methods one to six with a sub-residual block. This will not be repeated here.

[0379] Optionally, after determining the cost of at least one sub-residual block, a second cost of the residual block can be determined or obtained based on the cost of the at least one sub-residual block. For example, the second cost of the residual block can be obtained by accumulating the costs of the at least one sub-residual block.

[0380] Optionally, if the text information obtained in the bitstream is located in the fifth range, the cost of the residual region is determined or obtained based on the cost of at least one sub-residual block, or the cost of the residual region is determined or obtained based on the cost of at least one sub-residual region.

[0381] Optionally, when the text information obtained by the decoding end in the bitstream is within the fifth range, the second cost of the residual block is determined based on the cost of at least one sub-residual block. This can improve the effectiveness of the determined second cost of the residual block, so that the encoding and / or decoding modes can be selected and / or sorted according to the second cost of the residual block. This can improve the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby improving the efficiency of video encoding and / or decoding.

[0382] Method 24: If the text information obtained in the bitstream is located in the sixth range, then the second cost of the residual block is determined or obtained based on the cost of the sub-transform block.

[0383] Alternatively, the text information can be provided in accordance with method 12 above.

[0384] Optionally, the sixth range can be a range associated with text information that is pre-defined by the user.

[0385] Optionally, the sixth range may be different from or partially the same as the first to fifth ranges mentioned above.

[0386] Optionally, when the processing device is a decoding end, and the text information obtained by the decoding end in the bitstream is in the sixth range, the second cost of the residual block can be determined or obtained by using the cost of at least one sub-transform block.

[0387] Optionally, the cost of at least one sub-transform block may be determined or obtained based on the first cost of at least two elements within the at least one sub-transform block, or it may be determined or obtained based on the first cost of the sign of the elements within the at least one sub-transform block, or it may be determined or obtained by scanning the elements within the at least one sub-transform block to obtain a scan sequence, based on the cost of the subsequence in the scan sequence, or it may be determined or obtained based on a neural network or lookup table, or it may be determined or obtained based on the cost of at least one sub-quantization block.

[0388] Optionally, the method for determining or obtaining at least one element within at least one sub-transform block can refer to methods one to six in the above embodiments, that is, replacing the current block in methods one to six with a sub-transform block. This will not be repeated here.

[0389] Optionally, after determining the cost of at least one sub-transform block, the cost of at least one sub-residual block can be determined or obtained based on the cost of at least one sub-transform block, and a second cost of the residual block can be determined or obtained based on the cost of at least one sub-residual block.

[0390] Optionally, if the text information obtained in the bitstream is located in the sixth range, the cost of the residual region is determined or obtained based on the cost of the sub-transform block.

[0391] Optionally, when the text information obtained by the decoding end in the bitstream is within the sixth range, the second cost of the residual block is determined based on the cost of at least one sub-transform block. This can improve the effectiveness of the determined second cost of the residual block, so that the encoding and / or decoding modes can be selected and / or sorted according to the second cost of the residual block. This can improve the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby improving the efficiency of video encoding and / or decoding.

[0392] Method 25: If the text information obtained in the bitstream is located in the seventh range, then the second cost of the residual block is determined or obtained based on the cost of the quantized sub-block.

[0393] Alternatively, the text information can be provided in accordance with method 12 above.

[0394] Optionally, the seventh range can be a range associated with text information that is pre-defined by the user.

[0395] Optionally, the seventh range may be different from or partially the same as the first to sixth ranges mentioned above.

[0396] Optionally, when the processing device is a decoding end, and the text information obtained by the decoding end in the bitstream is in the seventh range, the second cost of the residual block can be determined or obtained by using the cost of at least one sub-quantization block.

[0397] Optionally, the cost of at least one sub-quantization block may be determined or obtained based on the first cost of at least two elements within the at least one sub-quantization block, or it may be determined or obtained based on the first cost of the sign of the elements within the at least one sub-quantization block, or it may be determined or obtained by scanning the elements within the at least one sub-quantization block to obtain a scan sequence, and then determining or obtaining the cost based on the cost of the subsequence in the scan sequence, or it may be determined or obtained based on a neural network or a lookup table.

[0398] Optionally, the method for determining or obtaining at least one element within at least one sub-quantization block can refer to methods one to six in the above embodiments, that is, replacing the current block in methods one to six with a sub-quantization block. This will not be repeated here.

[0399] Optionally, after determining the cost of at least one sub-quantization block, the cost of at least one sub-residual block can be determined or obtained based on the cost of at least one sub-quantization block, and a second cost of the residual block can be determined or obtained based on the cost of at least one sub-residual block.

[0400] Optionally, if the text information obtained in the bitstream is located in the seventh range, the cost of the residual region is determined or obtained based on the cost of the quantized sub-block.

[0401] For example, as shown in Figure 14, if the text information includes a bitstream marker, for a residual block R, if the processing device obtains the bitstream marker from the bitstream, it checks whether the bitstream marker is a first value or belongs to a first range. If yes, the block cost is calculated from the original pixels of the residual block, and / or; if no, it determines whether the bitstream marker is a second value or belongs to a second range. If yes, the block cost is calculated based on the transform block, and / or; if no, the block cost is calculated based on the quantization block. Optionally, the block cost can be the second cost of the residual block.

[0402] Optionally, when calculating the block cost based on the original pixels of the residual block, the residual block R can be scanned to obtain the element sequence r1,...,ri,... Optionally, when calculating the block cost based on the quantization block, the residual block R can first be transformed and quantized to obtain the quantization block Q, and the quantization Q can be scanned to obtain the element sequence q1,...,qi,... Optionally, when calculating the block cost based on the transform block, the residual block R can first be transformed to obtain the transform block T, and the transform block T can be scanned to obtain the element sequence t1,...,ti,...

[0403] Optionally, if switch 2 is selected, the absolute value of the element sequence t1,...,ti,... can be taken to obtain a non-negative integer sequence. For the non-negative integer sequence, a binary bit sequence b(t1)...b(ti)... can be obtained. For each element in the binary bit sequence, the accumulation module obtains its cost value by looking up the table [key, cost], and accumulates all the cost values ​​to obtain the block cost.

[0404] Optionally, when the text information obtained by the decoding end in the bitstream is within the seventh range, the second cost of the residual block is determined based on the cost of at least one sub-quantization block. This can improve the effectiveness of the determined second cost of the residual block, so as to facilitate the selection and / or sorting of encoding and / or decoding modes based on the second cost of the residual block.

[0405] The technical solution of this embodiment can improve the accuracy of sorting or filtering multiple encoding and / or decoding modes, thereby improving the efficiency of video encoding and / or decoding.

[0406] Referring to Figure 15, this application also provides a processing apparatus, which includes:

[0407] Processing module A10 is used to determine or obtain the second cost of the residual block based on the first cost of the elements in the current block; and to select and / or sort the encoding and / or decoding modes based on the second cost.

[0408] Optionally, the method for determining or obtaining the first cost of the elements within the current block includes at least one of the following:

[0409] Based on at least one of the original value, absolute value, and preset number of digits of any element in the current block, obtain the first cost from the first lookup table;

[0410] For any two elements in the current block, the first cost of one element is obtained by weighting it according to the first weight of the other element, and the first cost of the other element is obtained by weighting it according to the second weight of the other element.

[0411] The elements in the current block include the first element and the second element. The third cost is obtained from the second lookup table based on the original value or absolute value of the first element in the current block. The first cost of the first element is determined based on the third cost. The fourth cost is obtained from the second lookup table based on the preset number of radixes of the second element in the current block. The first cost of the second element is determined based on the fourth cost.

[0412] The third weight is obtained from the third lookup table based on any element in the current block. The weight is then applied to the third weight to obtain the first cost.

[0413] For any element in the current block, calculate the fifth cost based on the first cost function, calculate the sixth cost based on the second cost function, and determine the first cost based on the fifth cost and the sixth cost.

[0414] If the text information obtained in the bitstream is within the first range, the first cost is calculated using the first element calculation method, and / or, if the text information obtained in the bitstream is within the second range, the first cost is calculated using the second element calculation method, which is different from the first element calculation method.

[0415] Optionally, the processing module A10 is also used for:

[0416] The second cost of the residual block is determined or obtained based on the first cost of at least one element within at least one sub-block of the current block.

[0417] Optionally, the method of determining or obtaining at least one sub-block of the current block includes at least one of the following:

[0418] Based on at least one sub-residual block obtained by dividing the residual block corresponding to the current block, determine or obtain at least one sub-block;

[0419] Based on the sub-transformed block obtained by transforming at least one sub-residual block, determine or obtain at least one sub-block;

[0420] At least one sub-block is determined or obtained based on the sub-quantization block obtained by quantizing at least one sub-transform block.

[0421] Optionally, the processing module A10 is also used for at least one of the following:

[0422] The seventh cost of at least one sub-block is determined or obtained based on the first cost of at least one element within at least one sub-block, and the second cost of the residual block is determined or obtained based on the seventh cost of at least one sub-block.

[0423] The cost of at least one sub-residual block is determined based on the cost of at least one sub-transform block, and the second cost of the residual block is determined or obtained based on the cost of at least one sub-residual block.

[0424] The cost of at least one sub-residual block is determined based on the cost of at least one sub-quantized block, and a second cost of the residual block is determined or obtained based on the cost of at least one sub-residual block.

[0425] Input at least one sub-block and / or the downsampled portion corresponding to the sub-block into the neural network, output the cost of at least one sub-residual block, and determine or obtain the second cost of the residual block based on the cost of at least one sub-residual block;

[0426] Based on the partitioning costs of at least two partitions for the residual block, a second cost for the residual block is determined or obtained.

[0427] Optionally, determining or obtaining a seventh cost for at least one sub-block based on a first cost of at least one element within at least one sub-block includes at least one of the following:

[0428] Determine or obtain the seventh cost of at least one sub-block based on the first cost of at least two elements within at least one sub-block;

[0429] Determine or obtain the seventh cost of at least one sub-block based on the first cost of the symbol of the element within at least one sub-block;

[0430] The elements within at least one sub-block are scanned to obtain a scan sequence. Based on the cost of the sub-sequence in the scan sequence, the seventh cost of the sub-block is determined or obtained.

[0431] Optionally, the determination or acquisition of the second cost of the residual block includes at least one of the following:

[0432] The eighth cost of the residual block is calculated according to the first cost function, the ninth cost of the residual block is calculated according to the second cost function, and the second cost of the residual block is determined based on the minimum value of the eighth cost and the ninth cost.

[0433] The eighth cost of the residual block is calculated according to the first cost function, and the ninth cost of the residual block is calculated according to the second cost function. The eighth cost and the ninth cost are weighted to obtain the second cost of the residual block.

[0434] The residual block is input into the neural network, and the second cost of the residual block is output.

[0435] If the text information obtained in the bitstream is within the third range, then the second cost of the residual block is calculated using the first cost function;

[0436] If the text information obtained in the bitstream is located in the fourth range, then the second cost function is used to calculate the second cost of the residual block;

[0437] If the text information obtained in the bitstream is in the fifth range, then the second cost of the residual block is determined or obtained based on the cost of at least one sub-residual block.

[0438] If the text information obtained in the bitstream is in the sixth range, then the second cost of the residual block is determined or obtained based on the cost of the sub-transform block.

[0439] If the text information obtained in the bitstream is within the seventh range, then the second cost of the residual block is determined or obtained based on the cost of the sub-quantized block.

[0440] Optionally, the encoding and / or decoding modes include at least one of prediction mode, block partitioning mode, motion vector, block vector, and filter;

[0441] The second cost of the residual block includes at least one of the following: the cost of the prediction mode, the cost of the block partitioning mode, the cost of the motion vector, the cost of the block vector, and the cost of the filter.

[0442] Optionally, the value of each position in the current block and / or sub-blocks can be used as an element;

[0443] Optionally, the processing module A10 is also used for at least one of the following:

[0444] The cost of at least one encoding and / or decoding mode is determined based on at least one second cost, and the encoding and / or decoding modes are selected and / or ordered based on the cost of at least one encoding and / or decoding mode.

[0445] This application also provides a processing device, including a memory and a processor. The memory stores an image processing program, and when the image processing program is executed by the processor, it implements the steps of the image processing method in any of the above embodiments.

[0446] This application also provides a storage medium storing an image processing program, which, when executed by a processor, implements the steps of the image processing method in any of the above embodiments.

[0447] In the embodiments of the smart terminal and storage medium provided in this application, all the technical features of any of the above-described image processing method embodiments may be included. The extended and explanatory content of the specification is basically the same as that of the embodiments of the above methods, and will not be repeated here.

[0448] This application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to perform the methods described in the various possible implementations above.

[0449] This application also provides a chip, including a memory and a processor. The memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that a device with the chip installed performs the methods described in the various possible implementations above.

[0450] It is understood that the above scenarios are merely examples and do not constitute a limitation on the application scenarios of the technical solutions provided in the embodiments of this application. The technical solutions of this application can also be applied to other scenarios. For example, as those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0451] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0452] The steps in the method of this application embodiment can be adjusted, combined, or deleted according to actual needs.

[0453] The units in the device of this application embodiment can be merged, divided, and deleted according to actual needs.

[0454] In this application, the same or similar terms, concepts, technical solutions and / or application scenario descriptions are generally described in detail only when they appear for the first time. When they appear again, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of this application, the same or similar terms, concepts, technical solutions and / or application scenario descriptions that are not described in detail later can be referred to their previous relevant detailed descriptions.

[0455] In this application, the descriptions of the various embodiments have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0456] The technical features of the present application can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present application.

[0457] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, controlled terminal, or network device, etc.) to execute the methods of each embodiment of this application.

[0458] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, storage disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)). The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An image processing method, wherein, Including the following steps: S10, determine or obtain the second cost of the residual block based on the first cost of the elements in the current block; S20, select and / or sort the encoding and / or decoding modes according to the second cost.

2. The method as described in claim 1, wherein, The method for determining or obtaining the first cost of the elements within the current block includes at least one of the following: Based on at least one of the original value, absolute value, and preset number of digits of any element in the current block, obtain the first cost from the first lookup table; For any two elements in the current block, the first cost of one element is obtained by weighting it according to the first weight of the other element, and the first cost of the other element is obtained by weighting it according to the second weight of the other element. The elements in the current block include the first element and the second element. The third cost is obtained from the second lookup table based on the original value or absolute value of the first element in the current block. The first cost of the first element is determined based on the third cost. The fourth cost is obtained from the second lookup table based on the preset number of radixes of the second element in the current block. The first cost of the second element is determined based on the fourth cost. The third weight is obtained from the third lookup table based on any element in the current block. The weight is then applied to the third weight to obtain the first cost. For any element in the current block, calculate the fifth cost based on the first cost function, calculate the sixth cost based on the second cost function, and determine the first cost based on the fifth cost and the sixth cost. If the text information obtained in the bitstream is within the first range, the first cost is calculated using the first element calculation method, and / or, if the text information obtained in the bitstream is within the second range, the first cost is calculated using the second element calculation method, which is different from the first element calculation method.

3. The method as described in claim 1, wherein, Step S10 includes: The second cost of the residual block is determined or obtained based on the first cost of at least one element within at least one sub-block of the current block.

4. The method of claim 3, wherein, The method by which at least one sub-block of the current block is determined or obtained includes at least one of the following: Based on at least one sub-residual block obtained by dividing the residual block corresponding to the current block, determine or obtain at least one sub-block; Based on the sub-transformed block obtained by transforming at least one sub-residual block, determine or obtain at least one sub-block; At least one sub-block is determined or obtained based on the sub-quantization block obtained by quantizing at least one sub-transform block.

5. The method of claim 4, wherein, Step S10 includes at least one of the following: The seventh cost of at least one sub-block is determined or obtained based on the first cost of at least one element within at least one sub-block, and the second cost of the residual block is determined or obtained based on the seventh cost of at least one sub-block. The cost of at least one sub-residual block is determined based on the cost of at least one sub-transform block, and the second cost of the residual block is determined or obtained based on the cost of at least one sub-residual block. The cost of at least one sub-residual block is determined based on the cost of at least one sub-quantized block, and a second cost of the residual block is determined or obtained based on the cost of at least one sub-residual block. Input at least one sub-block and / or the downsampled portion corresponding to the sub-block into the neural network, output the cost of at least one sub-residual block, and determine or obtain the second cost of the residual block based on the cost of at least one sub-residual block; Based on the partitioning costs of at least two partitions for the residual block, a second cost for the residual block is determined or obtained.

6. The method of claim 5, wherein, The determination or acquisition of the seventh cost of at least one sub-block based on the first cost of at least one element within at least one sub-block includes at least one of the following: Determine or obtain the seventh cost of at least one sub-block based on the first cost of at least two elements within at least one sub-block; Determine or obtain the seventh cost of at least one sub-block based on the first cost of the symbol of the element within at least one sub-block; The elements within at least one sub-block are scanned to obtain a scan sequence. Based on the cost of the sub-sequence in the scan sequence, the seventh cost of the sub-block is determined or obtained.

7. The method of claim 1, wherein, The method of determining or obtaining the second cost of the residual block includes at least one of the following: The eighth cost of the residual block is calculated according to the first cost function, the ninth cost of the residual block is calculated according to the second cost function, and the second cost of the residual block is determined based on the minimum value of the eighth cost and the ninth cost. The eighth cost of the residual block is calculated according to the first cost function, and the ninth cost of the residual block is calculated according to the second cost function. The eighth cost and the ninth cost are weighted to obtain the second cost of the residual block. The residual block is input into the neural network, and the second cost of the residual block is output. If the text information obtained in the bitstream is within the third range, then the second cost of the residual block is calculated using the first cost function; If the text information obtained in the bitstream is located in the fourth range, then the second cost function is used to calculate the second cost of the residual block; If the text information obtained in the bitstream is in the fifth range, then the second cost of the residual block is determined or obtained based on the cost of at least one sub-residual block. If the text information obtained in the bitstream is in the sixth range, then the second cost of the residual block is determined or obtained based on the cost of the sub-transform block. If the text information obtained in the bitstream is within the seventh range, then the second cost of the residual block is determined or obtained based on the cost of the sub-quantized block.

8. The method of claim 1, wherein, It also includes at least one of the following: The encoding and / or decoding modes include at least one of prediction mode, block partitioning mode, motion vector, block vector, and filter; The second cost of the residual block includes at least one of the following: the cost of the prediction mode, the cost of the block partitioning mode, the cost of the motion vector, the cost of the block vector, and the cost of the filter. Use the value of each position in the current block and / or sub-block as an element; Step S20 includes: determining the cost of at least one encoding and / or decoding mode based on at least one second cost, and selecting and / or sorting the encoding and / or decoding modes based on the cost of at least one encoding and / or decoding mode.

9. A processing apparatus, wherein, include: The system includes a memory and a processor, wherein the memory stores an image processing program, and when the image processing program is executed by the processor, it implements the steps of the image processing method as described in claim 1.

10. A storage medium, wherein, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the image processing method as described in claim 1.

Images