Image processing method, processing equipment and storage medium

Abstract

The invention provides an image processing method, processing equipment and a storage medium. The image processing method comprises the following steps: determining or generating a target image block according to a reference parameter of at least one image block and a filtering lookup table; according to the technical scheme, when the filtering lookup table is adopted for filtering processing, the complexity of video coding and / or decoding can be reduced through the technical scheme, and then the efficiency of video coding and / or decoding is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, processing device and storage medium. Background Art

[0002] Existing high-efficiency video coding frameworks, such as Neural Network Based Video Coding (NNVC) and / or Enhanced Compression Model (ECM), propose a video frame encoding technology to improve the encoding performance without significantly increasing the computational complexity.

[0003] In the process of conceiving and implementing the present application, the inventors found that there are at least the following problems: In the loop filtering stage of the encoding and decoding process, the high complexity of the neural network limits its practicality. For example, when a low complexity neural network loop filter (LC-NNLF) structure is used for loop filtering, due to the presence of a CP decomposition (Candecomp Parafac, tensor decomposition) step in the LC-NNLF structure, and the large number of parameters and operations of the LC-NNLF, the complexity of the filtering process is high, which in turn limits the efficiency of video encoding and / or decoding. The foregoing description is intended to provide general background information and does not necessarily constitute prior art. Summary of the invention

[0004] In response to the above technical problems, the present application provides an image processing method, a processing device and a storage medium, aiming to solve the technical problem of how to reduce the complexity of filtering processing and thereby improve the efficiency of video encoding and / or decoding.

[0005] The present application provides an image processing method, which can be applied to a processing device, comprising the steps of:

[0006] S10, determining or generating a target image block according to a reference parameter of at least one image block and a filter lookup table.

[0007] Optionally, the reference parameter includes at least one of the following:

[0008] Quantization parameter; boundary strength; characterization parameter; position information of pixels to be filtered; pixel value to be filtered; size information of image block; filtering information of neighboring blocks; filtering information of non-neighboring blocks; filtering information of cross-component blocks; filtering information of co-located blocks; filtering information of time domain blocks; filtering information of default blocks; filtering information of candidate blocks.

[0009] Optionally, the candidate blocks are determined by motion vectors and / or block vectors.

[0010] Optionally, step S10 includes at least one of the following:

[0011] Determine or generate at least one first intermediate value according to the neural network and at least one reference parameter, and determine or generate a target image block according to the at least one first intermediate value and at least one filter lookup table;

[0012] Searching in at least one filter lookup table according to at least one reference parameter to obtain at least one second intermediate value, and determining or generating a target image block according to the neural network and the at least one second intermediate value;

[0013] Determine or generate at least one index according to at least one reference parameter, and determine or generate a target image block according to the at least one index and at least one filter lookup table;

[0014] Determine or generate at least one third intermediate value based on the neural network and at least one reference parameter, determine or generate at least one fourth intermediate value based on the at least one third intermediate value and at least one filter lookup table, and determine or generate a target image block based on the neural network and the at least fourth intermediate value;

[0015] At least one fifth intermediate value is determined or generated based on at least one reference parameter and at least one filter lookup table, at least one sixth intermediate value is determined or generated based on at least one fifth intermediate value and a neural network, and a target image block is determined or generated based on at least one sixth intermediate value and at least one filter lookup table.

[0016] Optionally, the neural network includes at least one of the following:

[0017] Neural networks based on fully connected layers;

[0018] Neural networks based on convolutional layers;

[0019] Transformer-based neural networks;

[0020] Neural network based on hybrid convolutional and fully connected layers and Transformer.

[0021] Optionally, when the neural network includes a neural network based on a convolutional layer, determining or generating at least one first intermediate value according to the neural network and at least one reference parameter includes:

[0022] Processing at least one image block input to a neural network including a convolution module to obtain at least one second image block;

[0023] At least one second image block is processed according to the convolution module and the reference parameters to determine or generate at least one first intermediate value.

[0024] Optionally, the convolution layer of the convolution module includes at least one of the following:

[0025] Convolutional layers with asymmetric convolutions;

[0026] Convolutional layers for grouped convolutions;

[0027] Convolutional layers with partial convolutions;

[0028] Convolutional layer for depthwise separable convolution.

[0029] Optionally, the image processing method further includes at least one of the following:

[0030] The convolution templates corresponding to at least two convolution layers in the convolution module are the same;

[0031] The convolution templates corresponding to at least two convolution layers in the convolution module include different convolution templates;

[0032] The pixel positions of the convolution kernels in at least two convolution templates are different;

[0033] The number and position of pixels in the convolution template are determined based on the receptive field.

[0034] Optionally, determining or generating at least one index according to at least one reference parameter includes at least one of the following:

[0035] Determine or generate a first pixel parameter according to at least one reference parameter, and determine or generate at least one index according to the first pixel parameter;

[0036] Determine or generate at least one first characterization value according to at least one reference parameter and at least one characterization lookup table, and determine or generate at least one index according to the at least one first characterization value;

[0037] Determine or generate a second pixel parameter according to at least one reference parameter, determine or generate at least one second characterization value according to the second pixel parameter and at least one characterization lookup table, and determine or generate at least one index according to the at least one second characterization value;

[0038] Determine or generate at least one third characterization value according to at least one reference parameter and at least one characterization lookup table, determine or generate a third pixel parameter according to the at least one third characterization value, and determine or generate at least one index according to the third pixel parameter;

[0039] Determine or generate at least one fourth representation value according to the high-order significant bits of at least one pixel value to be filtered, and determine or generate at least one index according to the at least one fourth representation value;

[0040] Determine or generate at least one fifth characterization value according to the characterization parameter, and determine or generate at least one index according to the at least one fifth characterization value;

[0041] At least one non-characteristic value is determined or generated according to at least one reference parameter, and at least one index is determined or generated according to the at least one non-characteristic value.

[0042] The present application also provides a processing device, comprising:

[0043] The processing module is used to determine or generate a target image block according to a reference parameter of at least one image block and a filter lookup table.

[0044] The present application also provides a processing device, including: a memory and a processor, wherein an image processing program is stored in the memory, and when the image processing program is executed by the processor, the steps of any of the above-mentioned image processing methods are implemented.

[0045] The present application also provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned image processing methods are implemented.

[0046] As described above, the processing method of the present application can be applied to a processing device, including the steps of: determining or generating a target image block based on reference parameters of at least one image block and a filter lookup table. Through the technical solution of the present application, it is possible to reduce the complexity of the filter when filtering at least one image block through a filter lookup table; and / or because the lookup table can be constructed based on a neural network, the filtering effect of the filtering process can be guaranteed, and the filtering process can be avoided by directly using a neural network, thereby improving the efficiency of video encoding and / or decoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0048] Figure 1 A schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present application;

[0049] Figure 2 A communication network system architecture diagram provided for an embodiment of the present application;

[0050] Figure 3 A schematic diagram of the hardware structure of a controller 140 provided in this application;

[0051] Figure 4 A schematic diagram of the hardware structure of a network node 150 provided in this application;

[0052] Figure 5 is a flowchart of an image processing method according to a first embodiment;

[0053] Figure 6 It is a schematic diagram of the encoding and decoding process in the image processing method;

[0054] Figure 7 is a schematic diagram of a convolution template in an image processing method according to the first embodiment;

[0055] Figure 8 is a schematic diagram of using a filter lookup table in an image processing method according to the first embodiment;

[0056] Fig. 9 is a schematic diagram of using a neural network in an image processing method according to a fourth embodiment;

[0057] Fig.10 is a schematic diagram of convolution templates corresponding to each convolution layer in a convolution module in an image processing method according to the fourth embodiment;

[0058] Fig.11 is a schematic diagram of a convolution template using a 1x3 convolution kernel with a smaller receptive field in an image processing method according to the fourth embodiment;

[0059] Fig.12 is a schematic diagram of a convolution template corresponding to a 1x3 convolution kernel whose receptive field is approximately the same as a 3x3 convolution kernel in the image processing method shown in the fourth embodiment;

[0060] Fig.13 is a schematic diagram of a convolution template corresponding to a 1x2 convolution kernel whose receptive field is approximately 3x3 convolution kernel in the image processing method shown in the fourth embodiment;

[0061] Fig.14 is a schematic diagram of a convolution template corresponding to a 1x3 convolution kernel whose receptive field is approximately 4x4 convolution kernel in the image processing method shown in the fourth embodiment;

[0062] Fig.15 is another schematic diagram of using a neural network in an image processing method according to a fourth embodiment;

[0063] Fig.16 is a schematic diagram of filtering processing performed by a neural network in combination with a convolution template and reference parameters in an image processing method according to a fourth embodiment;

[0064] Fig.17 It is a schematic diagram of a processing module of a processing device.

[0065] The realization of the purpose, functional features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. The above-mentioned drawings have shown clear embodiments of this application, which will be described in more detail later. These drawings and textual descriptions are not intended to limit the scope of the concept of this application in any way, but to illustrate the concept of this application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0066] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0067] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0068] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in this article, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this article, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination". Furthermore, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising" and "including" indicate the presence of the described features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the presence, occurrence or addition of one or more other features, steps, operations, elements, components, projects, kinds, and / or groups. The terms "or", "and / or", "including at least one of the following" used in this application, etc. can be interpreted as inclusive, or mean any one or any combination. For example, “comprising at least one of the following: A, B, C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”, and for another example, “A, B or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”. An exception to this definition will only occur when a combination of elements, functions, steps or operations are inherently mutually exclusive in some manner.

[0069] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are displayed in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and it can be performed in other orders. Moreover, at least a portion of the steps in the figure may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0070] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0071] It should be noted that in this article, step codes such as S10 and S20 are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the sequence. When implementing the step, those skilled in the art may execute S20 first and then S10, etc., but these should all be within the scope of protection of this application.

[0072] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0073] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0074] The processing device in this application may be a smart terminal or a server. Optionally, the smart terminal may be implemented in various forms. For example, the smart terminal described in this application may include smart terminals such as mobile phones, tablet computers, laptop computers, PDAs, portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, and fixed terminals such as digital TVs and desktop computers.

[0075] The following description will be made by taking a mobile terminal as an example, and those skilled in the art will understand that, in addition to components specifically used for mobile purposes, the construction according to the embodiments of the present application can also be applied to fixed-type terminals.

[0076] See also Figure 1 , which is a schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present application, the mobile terminal 100 may include: RF (Radio Frequency, radio frequency) unit 101, WiFi module 102, audio output unit 103, A / V (audio / video) input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, processor 110, and power supply 111 and other components. Those skilled in the art can understand that Figure 1 The structure of the mobile terminal shown in the figure does not constitute a limitation on the mobile terminal. The mobile terminal may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0077] Combine the following Figure 1 The following is a detailed introduction to the various components of the mobile terminal:

[0078] The radio frequency unit 101 can be used for receiving and sending signals during information transmission or communication. Specifically, after receiving the downlink information of the base station, it is sent to the processor 110 for processing; in addition, the uplink data is sent to the base station. Generally, the radio frequency unit 101 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, the radio frequency unit 101 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communications can use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution), 5G and 6G, etc.

[0079] WiFi is a short-range wireless transmission technology. Mobile terminals can help users send and receive emails, browse web pages, and access streaming media through WiFi module 102. It provides users with wireless broadband Internet access. Figure 1 The WiFi module 102 is shown, but it is understandable that it is not an essential component of the mobile terminal and can be omitted as needed without changing the essence of the invention.

[0080] The audio output unit 103 can convert the audio data received by the RF unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output it as sound when the mobile terminal 100 is in a call signal reception mode, a talk mode, a recording mode, a voice recognition mode, a broadcast reception mode, etc. Moreover, the audio output unit 103 can also provide audio output related to a specific function performed by the mobile terminal 100 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.

[0081] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processor (GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 106. The image frame processed by the graphics processor 1041 can be stored in the memory 109 (or other storage medium) or sent via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) via the microphone 1042 in a telephone call mode, a recording mode, a voice recognition mode, and other operating modes, and can process such sound into audio data. The processed audio (voice) data can be converted into a format output that can be sent to a mobile communication base station via the radio frequency unit 101 in the case of a telephone call mode. The microphone 1042 can implement various types of noise elimination (or suppression) algorithms to eliminate (or suppress) noise or interference generated in the process of receiving and sending audio signals.

[0082] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Optionally, the light sensor includes an ambient light sensor and a proximity sensor. Optionally, the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 1061 and / or the backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured on the mobile phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0083] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0084] The user input unit 107 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile terminal. Optionally, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 1071 or near the touch panel 1071 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Optionally, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 110, and can receive and execute the command sent by the processor 110. In addition, the touch panel 1071 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may further include other input devices 1072. Optionally, the other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, a function key (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, etc., which are not specifically limited here.

[0085] Optionally, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. Then, the processor 110 provides corresponding visual output on the display panel 1061 according to the type of touch event. Figure 1 In the figure, the touch panel 1071 and the display panel 1061 are used as two independent components to implement the input and output functions of the mobile terminal. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to implement the input and output functions of the mobile terminal, which is not limited here.

[0086] The interface unit 108 serves as an interface through which at least one external device can be connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and an external device.

[0087] The memory 109 can be used to store software programs and various data. The memory 109 can mainly include a program storage area and a data storage area. Optionally, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 109 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0088] The processor 110 is the control center of the mobile terminal. It uses various interfaces and lines to connect various parts of the entire mobile terminal. It executes various functions of the mobile terminal and processes data by running or executing software programs and / or modules stored in the memory 109, and calling data stored in the memory 109, so as to monitor the mobile terminal as a whole. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor. Optionally, the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 110.

[0089] The mobile terminal 100 may also include a power supply 111 (such as a battery) for supplying power to various components. Preferably, the power supply 111 may be logically connected to the processor 110 via a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system.

[0090] although Figure 1 Not shown, the mobile terminal 100 may further include a Bluetooth module, etc., which will not be described in detail here.

[0091] In order to facilitate understanding of the embodiments of the present application, the communication network system on which the mobile terminal of the present application is based is described below.

[0092] See also Figure 2 , Figure 2 A communication network system architecture diagram is provided for an embodiment of the present application, wherein the communication network system is an LTE system of universal mobile communication technology, and the LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203 and an operator's IP service 204, which are connected in sequence for communication.

[0093] Optionally, UE201 may be the above-mentioned terminal 100, which will not be described in detail here.

[0094] E-UTRAN 202 includes eNodeB 2021 and other eNodeBs 2022 , etc. Optionally, eNodeB 2021 may be connected to other eNodeBs 2022 via a backhaul (eg, an X2 interface), and eNodeB 2021 is connected to EPC 203 , and eNodeB 2021 may provide UE 201 with access to EPC 203 .

[0095] EPC203 may include MME (Mobility Management Entity) 2031, HSS (Home Subscriber Server) 2032, other MMEs 2033, SGW (Serving Gate Way) 2034, PGW (PDN Gate Way) 2035 and PCRF (Policy and Charging Rules Function) 2036, etc. Optionally, MME 2031 is a control node that processes signaling between UE 201 and EPC 203, and provides bearer and connection management. HSS 2032 is used to provide some registers to manage functions such as home location register (not shown in the figure), and store some user-specific information such as service features and data rates. All user data can be sent through SGW2034, PGW2035 can provide IP address allocation and other functions for UE 201, PCRF2036 is the policy and charging control policy decision point for service data flow and IP bearer resources, which selects and provides available policy and charging control decisions for the policy and charging execution function unit (not shown in the figure).

[0096] The IP service 204 may include the Internet, an intranet, an IMS (IP Multimedia Subsystem) or other IP services.

[0097] Although the above introduction takes the LTE system as an example, those skilled in the art should know that the present application is not only applicable to the LTE system, but also to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, 5G and future new network systems (such as 6G), etc., without limitation here.

[0098] Figure 3The hardware structure diagram of a controller 140 provided in the present application is shown in FIG. The controller 140 includes a memory 1401 and a processor 1402. The memory 1401 is used to store program instructions, and the processor 1402 is used to call the program instructions in the memory 1401 to execute the steps executed by the controller in the first embodiment of the above method. The implementation principle and beneficial effects are similar and will not be described in detail here.

[0099] Optionally, the controller further includes a communication interface 1403, which can be connected to the processor 1402 via a bus 1404. The processor 1402 can control the communication interface 1403 to implement the receiving and sending functions of the controller 140.

[0100] Figure 4 The hardware structure diagram of a network node 150 provided in the present application is shown in FIG. The network node 150 includes: a memory 1501 and a processor 1502, the memory 1501 is used to store program instructions, and the processor 1502 is used to call the program instructions in the memory 1501 to execute the steps performed by the first node in the first embodiment of the above method, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0101] Optionally, the controller further includes a communication interface 1503, which can be connected to the processor 1502 via a bus 1504. The processor 1502 can control the communication interface 1503 to implement the receiving and sending functions of the network node 150.

[0102] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to perform some steps of the methods of various embodiments of the present application.

[0103] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. Computer instructions can be stored in a storage medium, or transmitted from one storage medium to another storage medium. For example, computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. Available media can be magnetic media, (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid state hard disk solid state disk, SSD), etc.

[0104] Based on the above-mentioned mobile terminal hardware structure and communication network system, various embodiments of the present application are proposed.

[0105] First embodiment

[0106] Reference Figure 5 , Figure 5 is a flowchart of an image processing method according to the first embodiment. The image processing method of the embodiment of the present application can be applied to a processing device, including step S10:

[0107] Step S10 , determining or generating a target image block according to a reference parameter of at least one image block and a filter lookup table.

[0108] In this embodiment, the processing device may be a smart terminal, such as a mobile phone, a computer, etc., or a server, such as a local server or a cloud server. In this embodiment and this application, the processing device is mainly described as a smart terminal.

[0109] Optionally, the technical solution of this embodiment can be applied to the fields of image coding and decoding, video coding and decoding, hardware video coding and decoding, dedicated circuit video coding and decoding, real-time video coding and decoding, etc.

[0110] Optionally, for ease of understanding, a brief introduction to the encoding and decoding process is given: Figure 6As shown, it includes modules such as general coding control, transformation and quantization, intra-frame estimation, intra-frame prediction, motion compensation, motion estimation, inverse quantization and inverse transformation, filter control analysis, deblocking filtering and SAO filtering (i.e., loop filtering), entropy coding, and decoding frame buffer. Optionally, the motion compensation module can perform intra-frame / inter-frame selection to determine specific compensation. Optionally, when entropy coding is performed, it is performed according to the general control data determined by the general coding control module, the variable quantization coefficient determined by the transformation and quantization module, the intra-frame prediction data determined by the filter control analysis, and the motion data determined by the filter control and decoding frame buffer, thereby obtaining the coding bit rate.

[0111] Optionally, the decoded video signal is output via a decoded frame buffer.

[0112] Optionally, the loop filtering may include two branches, a Deblocking Filter and an LC-NNLF, and then the branch results are fused, and SAO (Sample Adaptive Offset) and ALF (Adaptive Loop Filter) processing are performed.

[0113] Optionally, due to the use of LC-NNLF (Low Complexity Neural Networkbased Loop Filter) for filtering in the loop filtering stage, the complexity of the filtering stage is too high. In this embodiment, a filter lookup table is used for filtering in the loop filtering stage, thereby reducing the complexity of the filtering stage and reducing the complexity of video encoding and / or decoding.

[0114] Optionally, similar reference parameters may refer to the second embodiment.

[0115] Optionally, the filter lookup table can be a lookup table, which is a commonly used accelerated calculation method in embedded systems. For a complex function or a series of calculations, if the output value of the calculation is placed in a lookup table, then all that needs to be done afterwards is to retrieve the value without executing the calculation process. Therefore, the lookup table is effective when the calculation time is longer than the memory access time. Optionally, the filter lookup table can include at least one pixel value before filtering, and at least one pixel value after filtering (such as the filtered pixel value of the first filtered pixel), and the corresponding relationship between the two.

[0116] Optionally, the filter lookup table may be a lookup table including filter pixels. Optionally, the filter lookup table may include the correspondence between pixels, the correspondence between pixels to be filtered and filter pixels, the correspondence between representation values ​​and filter pixels, and the correspondence between intermediate values ​​and filter pixels.

[0117] Optionally, each data unit of the filter lookup table is a filtered pixel value.

[0118] Optionally, an index may be set in the filter lookup table so as to perform a search in the filter lookup table according to the index to determine the corresponding filtered pixel value.

[0119] Optionally, the target image block may be an image block that has been filtered. Optionally, the target image block may include at least one first filtered pixel, or may include pixels associated with the at least one first filtered pixel.

[0120] Optionally, the processing device may be a decoding end, and if it is at the decoding end, the target image block may be a decoded image block. Optionally, the processing device may be an encoding end, and if it is at the encoding end, the target image block may be an image block after filtering processing.

[0121] Optionally, in this embodiment, the pre-trained neural network can be converted into a filter lookup table, and then the target image block after filtering is determined according to the filter lookup table. Then, the complexity of the neural network video compression method is further reduced while maintaining basically no performance loss. And / or compared with non-neural network video compression, it has better compression performance. And / or because it is performed by means of a filter lookup table, there is no need to use a GPU for inference, and it can be widely used in various devices such as mobile phones, TVs, and cameras.

[0122] Optionally, the method in this embodiment may be used to perform filtering processing on loop filtering, post filtering, and other stages that may require filtering processing.

[0123] Optionally, before using a filter lookup table to determine or obtain a target image block after filtering, the following four stages can be displayed, and then at least one first filtering pixel is determined by searching at least one filter lookup table based on at least one pixel to be filtered, and the target image block is determined based on the at least one first filtering pixel.

[0124] Optionally, the four stages are:

[0125] (1) Training phase: training a neural network;

[0126] (2) performing bit depth sampling on the input pixel value of the trained neural network, and converting the sampled value and the output of the neural network into a filter lookup table;

[0127] (3) Fine-tune the filter lookup table to obtain all input and output results of the full bit depth;

[0128] (4) Using the filter lookup table to perform an inference test, the index of the filter lookup table is determined according to the pixel to be filtered, and the index is input into the filter lookup table for search to obtain the filtered pixel after filtering.

[0129] Optionally, during the training phase of the neural network, a symmetric convolution kernel or an asymmetric convolution kernel may be used to train the neural network, and a corresponding filter lookup table may be constructed for each convolution kernel in the neural network. The filter lookup tables corresponding to each convolution kernel may be the same or different.

[0130] Optionally, when training a neural network, multiple convolutional layers may be stacked in the neural network, and the size of the receptive field (Receptive Field) of the convolution kernel of each convolutional layer is directly related to the size of the filter lookup table that needs to be converted subsequently.

[0131] Table 1 shows the filter lookup table sizes for different receptive fields, where n is a positive integer greater than 1, such as 10. As shown in Table 1, for an 8-bit input pixel, there are 256 possible pixel values. Assuming that the receptive field of the convolution kernel is 2x2, there are 256 possible values. 4 There are possible pixel value combinations, so the size of the filter lookup table corresponding to the convolution kernel of this convolution layer is 256. 4 Bytes, that is, 4GB is required to store the filter lookup table.

[0132] Table 1. Filter lookup table sizes for different receptive fields

[0133] Receptive field of convolution kernel Filter Lookup Table Filter lookup table size 1 pixel 1D 256B 2 pixels 2D 32KB 3 pixels 3D 16MB 4 pixels 4D 4GB 5 pixels 5D 1TB n pixels nD <![CDATA[(2 8 ) n B]]>

[0134] Optionally, when training a neural network, an asymmetric convolution kernel may be used for the convolution layer of the neural network. Figure 7 As shown, there are three different types of convolution templates corresponding to the convolution kernels of the receptive field of 1x3, including pixels I0 to I8. Figure 7 The convolution template corresponding to (a) is the selected pixel I0, pixel I1 and pixel I2. Figure 7 The convolution template corresponding to (b) is the selected pixel I0, pixel I4 and pixel I5. Figure 7 The convolution template corresponding to (c) is the selected pixel I0, pixel I7 and pixel I8.

[0135] When training a neural network, the input image block can be rotated so that the 1x3 convolution kernel acts on different positions of the 3x3 pixels, which is similar to the convolution of the 3x3 convolution kernel and the 3x3 image block, that is, the receptive field of the 1x3 convolution kernel is increased. A deep neural network is formed by cascading M convolution layers of different templates, and the neural network is trained until convergence to obtain a filter-enhanced neural network. M is a positive integer greater than 1.

[0136] Optionally, since the filter-enhanced neural network uses convolution kernels of M different convolution templates during the training phase, each convolution template corresponds to an input and output relationship, and is converted into a filter lookup table for storage. That is, all possible inputs and outputs of the convolution kernels of M different convolution templates are stored in the form of a lookup table to obtain a filter lookup table. For example, for M different convolution kernels with a receptive field of 3, M filter lookup tables need to be stored.

[0137] Optionally, the input values ​​to the neural network (such as the pixels to be filtered in the input image block) can also be bit-sampled. For example, for an 8-bit input pixel, a 1x3 convolution kernel covers 256 pixel values. 3 possible pixel value combinations. Therefore, the input image block can be sampled according to the sampling interval, such as a sampling interval of 4 bits. For example, for 256 pixel values ​​in the range [0,1,2,...,255], the pixel values ​​after 4-bit sampling are [0,16,32,48,64,80,96,112,128,144,160,176,192,208,224,240]. There are 163 possible pixel values ​​after sampling. The filter lookup table can only store the values ​​of the sampling points and the corresponding outputs. At this time, the size of each table is 4KB. For example, as shown in Table 2 below.

[0138] Table 2

[0139]

[0140]

[0141] Optionally, after the filter lookup table is acquired according to the trained neural network, the filter lookup table may be used to perform filtering processing in the filtering stage of video encoding and / or video decoding.

[0142] For example, Figure 8As shown, if the pixels to be filtered of at least one image block are pixel I0, pixel I1 and pixel I2, the indexes corresponding to pixel I0, pixel I1 and pixel I2 can be determined and input into at least one filter lookup table (such as a multi-level filter lookup table) for filtering. If the multi-level filter lookup table is filter lookup table 1-filter lookup table 9, the index can be input into filter lookup table 1 for search, and the output result of filter lookup table 1 is input into filter lookup table 2, and filtering is performed in sequence until filter lookup table 9 outputs the filtered pixel corresponding to the pixel to be filtered, and the target image block V is determined according to the filtered pixel.

[0143] Optionally, the output of a previous filter lookup table in the multi-stage filter lookup table may be the input of a subsequent filter lookup table.

[0144] Optionally, each level of the multi-level filter lookup table may include a correspondence between pixels, and each correspondence may be set with a corresponding index. and filtered pixel or intermediate value [V (2) ], and a corresponding index is set for each corresponding relationship. For example, the filter lookup table 9 stores the pixels to be filtered. and the corresponding relationship between the filtered pixel or the intermediate value [V], and a corresponding index is set for each corresponding relationship.

[0145] Optionally, when constructing a multi-level filter lookup table, a neural network can be used for processing, and the input and output of the convolution layer in the convolution module in the neural network are stored in the form of a lookup table to obtain a filter lookup table, and the filter lookup tables are aggregated in the order of each convolution layer to obtain a multi-level filter lookup table.

[0146] Optionally, each convolution layer may perform convolution processing according to a convolution template when performing convolution processing. For example, one convolution layer performs convolution processing according to convolution template 1, and another convolution layer performs convolution processing according to convolution template 9, until an image block V after neural network filtering processing is output.

[0147] Optionally, since different convolutional layers in the neural network correspond to different filter lookup tables, in this embodiment, at least one filter lookup table to be applied this time can be determined from multiple filter lookup tables based on the reference parameters of at least one image block, and the reference parameters of at least one image block are input into the determined at least one filter lookup table for search, so as to determine and output the corresponding filter pixels, and obtain a target image block including the filter pixels. At this time, the target image block is an image block after filtering by the filter lookup table.

[0148] In this embodiment, by determining or generating a target image block based on reference parameters of at least one image block and a filter lookup table, it is possible to reduce the complexity of the filter when filtering at least one image block through the filter lookup table; and / or because the lookup table can be constructed based on a neural network, the filtering effect of the filtering process can be guaranteed, and the filtering process can be avoided by directly using a neural network, thereby improving the efficiency of video encoding and / or decoding.

[0149] Second embodiment

[0150] Based on the first embodiment, a second embodiment is proposed.

[0151] In this embodiment, the reference parameter includes at least one of the following methods 1 to 13:

[0152] Method 1: Quantization parameters;

[0153] Optionally, the quantization parameter may include a quantization step size. The quantization step size may be a parameter for controlling quantization accuracy calculated based on a quantization parameter QP. Optionally, the quantization step size may determine the degree of precision loss in the process of dividing the original signal (such as a pixel value or a transform coefficient) from a continuous value into a plurality of value intervals. The quantization process maps continuous values ​​to a smaller number of discrete values, and achieves data compression by discarding some details. The larger the quantization step size, the greater the difference between each quantization level, and therefore the more details are discarded, which leads to increased compression efficiency, but also introduces greater distortion, i.e., image quality degradation. Conversely, the smaller the quantization step size, the more details are retained, the higher the image quality, but the required storage space and transmission bandwidth also increase accordingly.

[0154] Optionally, the filter lookup table to be applied this time can be screened out from multiple filter lookup tables based on the quantization parameters of at least one image block, and the pixels to be filtered in at least one image block are input into the filter lookup table to be applied this time for filtering, and the target image block after filtering is output.

[0155] Optionally, the pixels to be filtered in at least one image block can be converted into indexes, and the indexes are input into a filter lookup table for search, so as to determine the corresponding filtered pixels for output, and the target image block is constructed and determined based on the output filtered pixels. Optionally, the index can be determined based on the quantization parameter and the pixels to be filtered in at least one image block. For example, the index corresponding to the pixels to be filtered in at least one image block is determined in a table containing a mapping relationship between the pixels to be filtered and the index, and whether the index needs to be updated (for example, when the quantization parameter is greater than a preset quantization parameter threshold, it is determined that the index needs to be updated) is determined based on the quantization parameter. If updated, the index can be updated (for example, a preset index value is increased or decreased), and then input into the filter lookup table for search and output to obtain the target image block. Optionally, the target image block includes the filtered pixels.

[0156] In this embodiment, when the reference parameters include quantization parameters, the target image block is determined or generated according to the quantization parameters and the filter lookup table, thereby avoiding directly using a neural network for filtering, thereby improving the efficiency of video encoding and / or decoding.

[0157] Method 2: boundary strength;

[0158] Alternatively, the boundary strength may be a measure for quantifying the clarity or strength of the edge between adjacent image blocks. Alternatively, it may be determined whether the boundary area pixel is a true boundary or a pseudo boundary such as a pseudo contour caused by compression based on the boundary strength of the current image block to be filtered.

[0159] Optionally, the filter lookup table to be applied this time can be screened out from multiple filter lookup tables based on the boundary strength of at least one image block, and the pixels to be filtered in at least one image block are input into the filter lookup table to be applied this time for filtering, and the target image block after filtering is output.

[0160] The pixels to be filtered in at least one image block can be converted into an index, and the index is input into a filter lookup table for search, so as to determine the corresponding filtered pixels for output, and the target image block is constructed and determined according to the output filtered pixels. Optionally, the index can be determined according to the boundary strength and the pixels to be filtered in at least one image block. For example, the index corresponding to the pixels to be filtered in at least one image block is determined in a table containing a mapping relationship between the pixels to be filtered and the index, and it is determined whether the index needs to be updated according to the boundary strength (for example, when the boundary strength is greater than a preset boundary strength threshold, it is determined that the index needs to be updated). If it is updated, the index can be updated (for example, increasing or decreasing a preset index value), and then input into the filter lookup table for search and output to obtain the target image block.

[0161] In this embodiment, when the reference parameters include boundary strength, the target image block is determined or generated according to the boundary strength and the filter lookup table, thereby avoiding directly using a neural network for filtering, thereby improving the efficiency of video encoding and / or decoding.

[0162] Method three, characterization parameters;

[0163] Optionally, the characterization parameter may be each parameter corresponding to the characterization value, for example, it may include a characterization interval, and the characterization interval may be the interval between characterization values. Optionally, the characterization value may be a numerical value used to describe or represent a certain object, phenomenon feature, attribute, etc., which is often processed or selected to reflect key characteristics, such as using average grades to represent the learning level of a class. Optionally, the characterization value may be a sampling value. When the characterization value is a sampling value, the characterization interval may be a sampling interval. Optionally, the sampling value may be a pixel value.

[0164] Optionally, the following only uses the characterization parameter as the characterization interval as an example for illustration.

[0165] Optionally, the filter lookup table to be applied this time can be screened out from multiple filter lookup tables based on the representation interval of at least one image block, and the pixels to be filtered in at least one image block are input into the filter lookup table to be applied this time for filtering, and the target image block after filtering is output.

[0166] Optionally, when the characterization parameter is the sampling interval, the filter lookup table that needs to be applied this time can be screened out from multiple filter lookup tables based on the sampling interval of at least one image block, and the pixels to be filtered in at least one image block are input into the filter lookup table that needs to be applied this time for filtering, and the target image block after filtering is output.

[0167] Optionally, the pixels to be filtered in at least one image block can be converted into indexes, and the indexes can be input into a filter lookup table for search, so as to determine the corresponding filtered pixels for output, and construct and determine the target image block according to the output filtered pixels. Optionally, the index can be determined according to the characterization interval and the pixels to be filtered in at least one image block. For example, the index corresponding to the pixels to be filtered in at least one image block is determined in a table containing a mapping relationship between the pixels to be filtered and the index, and whether the index needs to be updated according to the characterization interval (for example, when the characterization interval is greater than a preset characterization interval threshold, it is determined that the index needs to be updated). If an update is required, the index can be updated (for example, by increasing or decreasing a preset index value), and then input into the filter lookup table for search and output to obtain the target image block.

[0168] In this embodiment, by determining or generating a target image block based on the characterization parameters and the filter lookup table, it is possible to avoid directly using a neural network for filtering, thereby improving the efficiency of video encoding and / or decoding.

[0169] Method 4: position information of the pixel to be filtered;

[0170] Optionally, the pixels to be filtered may be pixels to be filtered in at least one image block.

[0171] Optionally, the position information of the pixel to be filtered may include the pixel position of the pixel to be filtered in the image block to be filtered. It may also include the positional relationship between the pixel to be filtered and a preset pixel in the image block. Optionally, the preset pixel may be a middle pixel in the image block, or may be a pixel in the image block determined by a user, etc. Optionally, the positional relationship may include the distance between the pixel to be filtered and the preset pixel, such as an interval of 5 pixels, an interval of 3 pixels, etc. It may also include the directional relationship between the pixel to be filtered and the preset pixel, such as the pixel to be filtered is located on the left side of the preset pixel, the pixel to be filtered is located on the right side of the preset pixel, etc.

[0172] Optionally, the filter lookup table to be applied this time can be screened out from multiple filter lookup tables based on the position information of the pixels to be filtered in at least one image block, and the pixels to be filtered in at least one image block to be filtered are input into the filter lookup table to be applied this time for filtering, and the target image block after filtering is output.

[0173] The pixels to be filtered in at least one image block can be converted into indexes, and the indexes are input into a filter lookup table for searching, so as to determine the corresponding filtered pixels for output, and to construct and determine the target image block according to the output filtered pixels. Optionally, the index can be determined according to the position information of the pixels to be filtered and the pixels to be filtered in at least one image block. For example, the index corresponding to the pixels to be filtered in at least one image block is determined in a table containing a mapping relationship between the pixels to be filtered and the index, and whether the index needs to be updated is determined according to the position information of the pixels to be filtered. For example, when the position information of the pixels to be filtered meets a preset position condition (for example, the number of pixels between the pixels to be filtered and the preset pixels is greater than a preset pixel number threshold, and / or the pixels to be filtered are located on the left and / or right of the preset pixels, etc.), it is determined that the index needs to be updated. If it needs to be updated, the index can be updated (for example, the preset index value is increased or decreased), and then it is input into the filter lookup table for searching and output to obtain the target image block.

[0174] In this embodiment, when the reference parameters include the position information of the pixels to be filtered, the target image block is determined or generated based on the position information of the pixels to be filtered and the filter lookup table, thereby avoiding directly using a neural network for filtering processing, thereby improving the efficiency of video encoding and / or decoding.

[0175] Method 5, pixel value to be filtered;

[0176] Optionally, a pixel to be filtered in at least one image block to be filtered can be determined, and the pixel value of the pixel to be filtered can be input as the pixel value to be filtered into a filter lookup table for search, so as to determine a filter pixel corresponding to the pixel to be filtered in the filter lookup table, and output the filter pixel, and determine or generate a target image block according to the output filter pixel. That is, the target image block includes at least one filter pixel corresponding to the pixel to be filtered.

[0177] Optionally, the pixel to be filtered may be a predicted pixel (e.g., a predicted pixel), a reconstructed pixel (e.g., a reconstructed pixel), or a reference pixel. Optionally, the predicted pixel, the reconstructed pixel, and / or the reference pixel may be an intra-frame pixel and / or an inter-frame pixel. It may be an intermediate value used or generated when the predicted pixel and / or the reconstructed pixel generate a reference pixel, and is not limited here.

[0178] Optionally, at least one pixel value to be filtered can be converted into at least one index, and the at least one index can be input into at least one filter lookup table for search to determine the corresponding filter pixel in the filter lookup table, and the filter pixel is output, and the target image block is determined or generated based on the output filter pixel.

[0179] In this embodiment, when the reference parameters include the pixel value to be filtered, the target image block is determined or generated based on the pixel value to be filtered and the filter lookup table, thereby avoiding directly using a neural network for filtering processing, thereby improving the efficiency of video encoding and / or decoding.

[0180] Method six, size information of image block;

[0181] Optionally, the size information of the image block may include width, height, area and perimeter of the image block.

[0182] Optionally, at least one filter lookup table that needs to be applied this time can be screened out from multiple filter lookup tables based on the size information of at least one image block, and the pixels to be filtered in at least one image block are input into at least one filter lookup table that needs to be applied this time for filtering, and the target image block after filtering is output.

[0183] Optionally, the pixels to be filtered in at least one image block can be converted into indexes, and the indexes can be input into a filter lookup table for search, so as to determine the corresponding filtered pixels for output, and construct and determine the target image block according to the output filtered pixels. Optionally, the index can be determined according to the size information of at least one image block and the pixels to be filtered in at least one image block. For example, the index corresponding to the pixels to be filtered in at least one image block is determined in a table containing a mapping relationship between the pixels to be filtered and the index, and whether the index needs to be updated is determined according to the size information of at least one image block. For example, when the size information of at least one image block meets a preset size condition (for example, if the size information of at least one image block is greater than a preset size information threshold, it is determined that the preset size condition is met), it is determined that the index needs to be updated. If it needs to be updated, the index can be updated (for example, a preset index value is increased or decreased), and then it can be input into the filter lookup table for search and output to obtain the target image block.

[0184] Optionally, the at least one image block may be screened in a plurality of filter lookup tables according to the image block attribute and / or image block type, and then the at least one image block is input into the screened filter lookup table for filtering, and the target image block is output. Optionally, the image block attribute may include the image texture of the image block. The image block type includes a natural image or a screen content image.

[0185] In this embodiment, when the reference parameters include the size information of the image block, the target image block is determined or generated based on the size information of the image block and a filter lookup table, thereby avoiding directly using a neural network for filtering processing, thereby improving the efficiency of video encoding and / or decoding.

[0186] Method seven, filtering information of neighbor blocks;

[0187] Optionally, the neighbor block may be an image block adjacent to at least one image block, and may be an image block that has been subjected to filtering processing.

[0188] Optionally, the filtering information of the neighboring block may include filtering pixels of the neighboring block, a filtering lookup table used by the neighboring block when performing filtering processing, and the like.

[0189] Optionally, at least one filter lookup table that needs to be applied this time can be screened out from multiple filter lookup tables based on the filter information of the neighboring blocks of at least one image block, and the pixels to be filtered in at least one image block are input into the at least one filter lookup table that needs to be applied this time for filtering, and the target image block after filtering is output.

[0190] Optionally, the pixels to be filtered in at least one image block can be converted into an index, and the index is input into a filter lookup table for search, so as to determine the corresponding filter pixel for output, and construct and determine the target image block according to the output filter pixel. Optionally, the index can be determined according to the filter information of the neighboring block and the pixels to be filtered in at least one image block. For example, the index corresponding to the pixels to be filtered in at least one image block is determined in a table containing a mapping relationship between the pixels to be filtered and the index, and the selected filter lookup table is determined in multiple filter lookup tables according to the filter information of the neighboring block. For example, the filter lookup table used by the neighboring block is used as the selected filter lookup table. The index is input into the selected filter lookup table for search, and the corresponding filter pixel is output, and the target image block is determined or generated according to the filter pixel. For example, the target image block includes the output filter pixel.

[0191] Optionally, it is also possible to filter in multiple filter lookup tables according to the image block attribute and / or image block type of at least one neighboring block, and then input at least one image block into the filtered filter lookup table for filtering processing to output the target image block.

[0192] In this embodiment, when the reference parameters include filtering information of neighboring blocks, the target image block is determined or generated based on the filtering information of the neighboring blocks and a filtering lookup table, thereby avoiding directly using a neural network for filtering processing, thereby improving the efficiency of video encoding and / or decoding.

[0193] Mode 8: filtering information of non-neighboring blocks;

[0194] Optionally, the non-neighbor block may be an image block that is not adjacent to at least one image block, and may be an image block that has been subjected to filtering processing.

[0195] Optionally, the filtering information of the non-neighboring block may include filtering pixels of the non-neighboring block, a filtering lookup table used by the non-neighboring block when performing filtering processing, and the like.

[0196] Optionally, at least one filter lookup table that needs to be applied this time can be screened out from multiple filter lookup tables based on the filter information of non-neighboring blocks of at least one image block, and the pixels to be filtered in at least one image block are input into at least one filter lookup table that needs to be applied this time for filtering, and the target image block after filtering is output.

[0197] Optionally, the pixels to be filtered in at least one image block can be converted into an index, and the index is input into a filter lookup table for search, so as to determine the corresponding filter pixel for output, and construct and determine the target image block according to the output filter pixel. Optionally, the index can be determined according to the filter information of the non-neighboring block and the pixels to be filtered in at least one image block. For example, the index corresponding to the pixels to be filtered in at least one image block is determined in a table containing a mapping relationship between the pixels to be filtered and the index, and the selected filter lookup table is determined in multiple filter lookup tables according to the filter information of the non-neighboring block. For example, the filter lookup table used by the non-neighboring block is used as the selected filter lookup table. The index is input into the selected filter lookup table for search, and the corresponding filter pixel is output, and the target image block including the filter pixel is determined or generated according to the filter pixel.

[0198] Optionally, it is also possible to filter in multiple filter lookup tables according to the image block attribute and / or image block type of at least one non-neighboring block, and then input at least one image block into the filtered filter lookup table for filtering processing to output a target image block.

[0199] In this embodiment, when the reference parameters include filtering information of non-neighboring blocks, the target image block is determined or generated based on the filtering information of the non-neighboring blocks and a filtering lookup table, thereby avoiding directly using a neural network for filtering processing, thereby improving the efficiency of video encoding and / or decoding.

[0200] Mode nine, filtering information across component blocks;

[0201] Optionally, the cross-component block may be an image block of a different component from the current at least one image block. For example, if the current at least one image block is an image block of the Y component, the cross-component block may be an image block of the U component and / or the V component. If the current at least one image block is an image block of the U component, the cross-component block may be an image block of the Y component and / or the V component. If the current at least one image block is an image block of the V component, the cross-component block may be an image block of the U component and / or the Y component.

[0202] Optionally, the filtering information across component blocks may include parameter information that needs to be applied when performing filtering processing across component blocks.

[0203] Optionally, at least one filter lookup table that needs to be applied this time can be screened out from multiple filter lookup tables based on the filtering information across component blocks of at least one image block, and the pixels to be filtered in at least one image block are input into at least one filter lookup table that needs to be applied this time for filtering, and the target image block after filtering is output.

[0204] Optionally, the pixels to be filtered in at least one image block can be converted into an index, and the index can be input into a filter lookup table for search to determine the corresponding filter pixel for output, and the target image block can be constructed and determined based on the output filter pixel. Optionally, the index can be determined based on the filter information across the component block and the pixels to be filtered in at least one image block. For example, the index corresponding to the pixels to be filtered in at least one image block is determined in a table containing a mapping relationship between the pixels to be filtered and the index, and the selected filter lookup table is determined in multiple filter lookup tables based on the filter information across the component block. For example, when the filter information across the component block meets certain conditions, at least one type of filter lookup table is selected, and when the filter information across the component block meets another condition, another type of filter lookup table is selected. The two types of filter lookup tables can be different. The index is input into the selected filter lookup table for search, and the corresponding filter pixel is output, and the target image block including the filter pixel is determined or generated based on the filter pixel.

[0205] Optionally, it is also possible to filter in multiple filter lookup tables according to the image block attribute and / or image block type of at least one cross-component block, and then input at least one image block into the filtered filter lookup table for filtering processing to output a target image block.

[0206] In this embodiment, when the reference parameters include filtering information across component blocks, the target image block is determined or generated based on the filtering information across component blocks and a filtering lookup table, thereby avoiding directly using a neural network for filtering processing, thereby improving the efficiency of video encoding and / or decoding.

[0207] Method 10: filtering information of the same-position block;

[0208] Optionally, the co-located block may be an image block in a co-located image, which has the same position and size as the current block. Optionally, the co-located image may be an image in the reference image that is closest to the current image in terms of time.

[0209] Optionally, the filtering information of the co-located blocks may include a filtering lookup table used when the co-located blocks are subjected to filtering processing, and may also include size information, position information, etc. of the co-located blocks.

[0210] Optionally, at least one filter lookup table that needs to be applied this time can be screened out from multiple filter lookup tables based on the filter information of the co-located block of at least one image block, and the pixels to be filtered in at least one image block are input into at least one filter lookup table that needs to be applied this time for filtering, and the target image block after filtering is output.

[0211] Optionally, the pixels to be filtered in at least one image block can be converted into an index, and the index is input into a filter lookup table for search, so as to determine the corresponding filter pixel for output, and construct and determine the target image block according to the output filter pixel. Optionally, the index can be determined according to the filter information of the same-position block and the pixels to be filtered in at least one image block. For example, the index corresponding to the pixels to be filtered in at least one image block is determined in a table containing a mapping relationship between the pixels to be filtered and the index, and the selected filter lookup table is determined in multiple filter lookup tables according to the filter information of the same-position block. For example, the filter lookup table used by the same-position block is used as the selected filter lookup table. The index is input into the selected filter lookup table for search, and the corresponding filter pixel is output, and the target image block including the filter pixel is determined or generated according to the filter pixel.

[0212] Optionally, it is also possible to filter in multiple filter lookup tables according to the image block attribute and / or image block type of at least one co-located block, and then input at least one image block into the filtered filter lookup table for filtering processing to output a target image block.

[0213] In this embodiment, when the reference parameters include filtering information of the co-located blocks, the target image block is determined or generated based on the filtering information of the co-located blocks and a filtering lookup table, thereby avoiding directly using a neural network for filtering processing, thereby improving the efficiency of video encoding and / or decoding.

[0214] Method 11, filtering information of time domain block;

[0215] Optionally, the time domain block can be a block distinguished from the time domain, such as an image block in the previous frame. For example, if there is video data containing three frames of images, the first frame of image is played in the first second, the second frame of image is played in the second second, and the third frame of image is played in the third second. If the image block predicted at the current moment is the image block after the second frame of image is divided, then the time domain block can be determined as the image block corresponding to it in other frame images except the second frame of image.

[0216] Optionally, the filtering information of the time domain block may include a filtering lookup table used when the time domain block is subjected to filtering processing, and may also include size information, position information, etc. of the time domain block.

[0217] Optionally, at least one filter lookup table that needs to be applied this time can be screened out from multiple filter lookup tables based on the filtering information of the time domain block of at least one image block, and the pixels to be filtered in at least one image block are input into at least one filter lookup table that needs to be applied this time for filtering, and the target image block after filtering is output.

[0218] Optionally, the pixels to be filtered in at least one image block can be converted into indexes, and the indexes are input into a filter lookup table for search, so as to determine the corresponding filter pixels for output, and the target image block is constructed and determined based on the output filter pixels. Optionally, the index can be determined based on the filter information of the time domain block and the pixels to be filtered in at least one image block. For example, the index corresponding to the pixels to be filtered in at least one image block is determined in a table containing a mapping relationship between the pixels to be filtered and the index, and the selected filter lookup table is determined in multiple filter lookup tables based on the filter information of the time domain block. For example, the filter lookup table used by the time domain block is used as the selected filter lookup table. The index is input into the selected filter lookup table for search, and the corresponding filter pixels are output, and the target image block including the filter pixels is determined or generated based on the filter pixels.

[0219] Optionally, it is also possible to filter in multiple filter lookup tables according to the image block attribute and / or image block type of at least one time domain block, and then input at least one image block into the filtered filter lookup table for filtering processing to output a target image block.

[0220] In this embodiment, when the reference parameters include filtering information of the time domain block, the target image block is determined or generated based on the filtering information of the time domain block and the filtering lookup table, thereby avoiding directly using a neural network for filtering processing, thereby improving the efficiency of video encoding and / or decoding.

[0221] Mode 12: filter information of the default block;

[0222] Optionally, the default block may be an image block set in advance, for example, an image block with typical pixel features set in advance by an encoder and / or a decoder.

[0223] Optionally, the filtering information of the default block may include a filtering lookup table used when the default block is subjected to filtering processing, and may also include size information, position information, etc. of the default block.

[0224] Optionally, at least one filter lookup table that needs to be applied this time can be screened out from multiple filter lookup tables based on the filter information of the default block, and the pixels to be filtered in at least one image block are input into at least one filter lookup table that needs to be applied this time for filtering, and the target image block after filtering is output.

[0225] Optionally, the pixels to be filtered in at least one image block can be converted into indexes, and the indexes are input into a filter lookup table for search, so as to determine the corresponding filter pixels for output, and the target image block is constructed and determined based on the output filter pixels. Optionally, the indexes can be determined based on the filter information of the default block and the pixels to be filtered in at least one image block. For example, the indexes corresponding to the pixels to be filtered in at least one image block are determined in a table containing a mapping relationship between the pixels to be filtered and the indexes, and the selected filter lookup table is determined in a plurality of filter lookup tables based on the filter information of the default block. For example, the filter lookup table used by the default block is used as the selected filter lookup table. The indexes are input into the selected filter lookup table for search, and the corresponding filter pixels are output, and the target image block including the filter pixels is determined or generated based on the filter pixels.

[0226] Optionally, it is also possible to filter in multiple filter lookup tables according to the image block attribute and / or image block type of at least one default block, and then input at least one image block into the filtered filter lookup table for filtering processing to output a target image block.

[0227] In this embodiment, when the reference parameters include filtering information of the default block, the target image block is determined or generated based on the filtering information of the default block and the filtering lookup table, thereby avoiding directly using a neural network for filtering processing, thereby improving the efficiency of video encoding and / or decoding.

[0228] Method thirteen, filtering information of candidate blocks.

[0229] Optionally, the candidate block may be an image block to be selected and determined.

[0230] Optionally, for the candidate block in the thirteenth method, the candidate block is determined by a motion vector and / or a block vector. Optionally, at least one candidate block can be determined or obtained according to a motion vector and / or a block vector corresponding to at least one pixel to be filtered in at least one image block.

[0231] Optionally, the filtering information of the candidate block may include a filtering lookup table used when the candidate block is filtered, and may also include size information, position information, etc. of the candidate block.

[0232] Optionally, at least one filter lookup table that needs to be applied this time can be screened out from multiple filter lookup tables based on the filter information of the candidate block, and the pixels to be filtered in at least one image block are input into at least one filter lookup table that needs to be applied this time for filtering, and the target image block after filtering is output.

[0233] Optionally, the pixels to be filtered in at least one image block can be converted into indexes, and the indexes are input into a filter lookup table for search, so as to determine the corresponding filter pixels for output, and the target image block is constructed and determined based on the output filter pixels. Optionally, the index can be determined based on the filter information of the candidate block and the pixels to be filtered in at least one image block. For example, the index corresponding to the pixels to be filtered in at least one image block is determined in a table containing a mapping relationship between the pixels to be filtered and the index, and the selected filter lookup table is determined in multiple filter lookup tables based on the filter information of the candidate block. For example, the filter lookup table used by the candidate block is used as the selected filter lookup table. The index is input into the selected filter lookup table for search, and the corresponding filter pixels are output, and the target image block including the filter pixels is determined or generated based on the filter pixels.

[0234] Optionally, it is also possible to screen in multiple filter lookup tables according to the image block attribute and / or image block type of at least one candidate block, and then input at least one image block into the screened filter lookup table for filtering processing to output a target image block.

[0235] In this embodiment, when the reference parameters include filtering information of the candidate block, the target image block is determined or generated based on the filtering information of the candidate block and the filtering lookup table, thereby avoiding directly using a neural network for filtering processing, thereby improving the efficiency of video encoding and / or decoding.

[0236] Third embodiment

[0237] Based on the first embodiment or the second embodiment, the third embodiment is proposed.

[0238] In this embodiment, step S10 includes at least one of the following methods 14 to 18.

[0239] Method 14, determining or generating at least one first intermediate value according to the neural network and at least one reference parameter, and determining or generating a target image block according to the at least one first intermediate value and at least one filter lookup table;

[0240] Optionally, the first intermediate value may be data generated by an intermediate processing process during the filtering process, may be a filtered pixel generated after at least one filtering process, or may be other values. The intermediate processing process may be a processing process of performing a pre-filtering process on the pixel to be filtered (for example, filtering through a neural network or a filter lookup table).

[0241] Optionally, at least one reference parameter may be input into the neural network for filtering, and the first intermediate value may be output. For example, at least one of the quantization parameter, boundary strength, characterization interval, position information of the pixel to be filtered, pixel value to be filtered, size information of the image block, filtering information of the neighboring block, filtering information of the non-neighboring block, filtering information of the cross-component block, filtering information of the co-located block, filtering information of the time domain block, filtering information of the default block, and filtering information of the candidate block may be input into the neural network for filtering, and the first intermediate value may be output.

[0242] Optionally, the first intermediate value may be directly used as a filtering pixel, and the target image block may be determined or generated according to the filtering pixel.

[0243] Optionally, the first intermediate value may be processed to determine or generate a target image block. For example, the first intermediate value is input into at least one filter lookup table for search and determination to obtain a filter pixel, and the target image block is determined or generated based on the at least one filter pixel. Optionally, the filter lookup table may include a correspondence between the first intermediate value and the filter pixel.

[0244] Optionally, at least one filter lookup table may be a multi-layer filter lookup table. The first intermediate value may be converted into an index and input into the multi-layer filter lookup table for parallel or serial search, and the filter pixel may be outputted, and the target image block may be determined or generated according to the at least one filter pixel.

[0245] In this embodiment, by determining or generating at least one first intermediate value based on a neural network and at least one reference parameter, determining or generating a target image block based on at least one first intermediate value and at least one filter lookup table, and performing filtering processing together using the neural network and the filter lookup table, the filtering effect can be improved, thereby improving the effect of video encoding and / or decoding.

[0246] Mode 15, searching in at least one filter lookup table according to at least one reference parameter to obtain at least one second intermediate value, and determining or generating a target image block according to the neural network and the at least one second intermediate value;

[0247] Optionally, the second intermediate value may be data generated during an intermediate process of the filtering process, may be a filtered pixel generated after at least one filtering process, or may be other values. Optionally, the second intermediate value may be the same as the first intermediate value or may be different from the first intermediate value.

[0248] Optionally, at least one reference parameter may be used to determine at least one selected filter lookup table from a plurality of filter lookup tables, and at least one reference parameter may be input into at least one filter lookup table to determine at least one second intermediate value. Optionally, the filter lookup table includes a correspondence between at least one reference parameter and the second intermediate value, and an index may be set for each correspondence in the filter lookup table.

[0249] Optionally, at least one filter lookup table may be a multi-layer filter lookup table. The pixel value to be filtered of at least one image block may be converted into an index and input into the multi-layer filter lookup table for parallel search or serial search, and at least one second intermediate value may be output. Optionally, based on at least one of the quantization parameter, boundary strength, characterization parameter, position information of the pixel to be filtered, size information of the image block, filtering information of neighboring blocks, filtering information of non-neighboring blocks, filtering information of cross-component blocks, filtering information of co-located blocks, filtering information of time domain blocks, filtering information of default blocks, and filtering information of candidate blocks, the index of the pixel value to be filtered of at least one image block may be updated (e.g., the index value is increased or decreased), and the updated index may be input into the multi-layer filter lookup table for parallel search or serial search, and at least one second intermediate value may be output.

[0250] Optionally, the filter lookup table can be screened based on at least one of the quantization parameter, boundary strength, characterization interval, position information of the pixel to be filtered, the pixel value to be filtered, size information of the image block, filtering information of neighboring blocks, filtering information of non-neighboring blocks, filtering information of cross-component blocks, filtering information of co-located blocks, filtering information of time domain blocks, filtering information of default blocks and filtering information of candidate blocks, and the pixel value to be filtered can be converted into an index and input into the screened filter lookup table for search to determine the filtered pixel after filtering processing, and output it as the second intermediate value.

[0251] Optionally, at least one second intermediate value may be input into a neural network, and a filtered pixel may be outputted, and a target image block may be determined or generated based on the filtered pixel.

[0252] Optionally, the neural network may be determined by screening based on at least one reference parameter.

[0253] Optionally, an image block including at least one second intermediate value may be input into a neural network for filtering, and a target image block including filtered pixels may be output.

[0254] In this embodiment, at least one second intermediate value is obtained by searching at least one filter lookup table based on at least one reference parameter, a target image block is determined or generated based on a neural network and the at least one second intermediate value, and filtering is performed together using the neural network and the filter lookup table. This can improve the filtering effect and thereby improve the effect of video encoding and / or decoding.

[0255] Mode 16, determining or generating at least one index according to at least one reference parameter, and determining or generating a target image block according to at least one index and at least one filter lookup table;

[0256] Optionally, at least one index may be determined by at least one reference parameter in at least one image block, and a method for determining the index may refer to at least one of methods 31 to 37 in the following embodiments.

[0257] Optionally, the index can be a number, an array, an identifier, a label, etc.

[0258] Optionally, it can be determined whether it is necessary to update the index of the pixel value conversion to be filtered of at least one image block based on at least one item of quantization parameters, boundary strength, characterization parameters, position information of pixels to be filtered, size information of image blocks, filtering information of neighboring blocks, filtering information of non-neighboring blocks, filtering information of cross-component blocks, filtering information of co-located blocks, filtering information of time domain blocks, filtering information of default blocks and filtering information of candidate blocks. If necessary, the converted index can be increased or decreased to obtain at least one updated index.

[0259] Optionally, after determining at least one index, the at least one index may be input into at least one filter lookup table for search to determine and output a corresponding filter pixel, and determine or generate a target image block based on the output of the at least one filter pixel.

[0260] Optionally, after determining or generating at least one index, such as a first index, based on at least one reference parameter, the first index is input into a first layer filter lookup table in a multi-layer filter lookup table for search, obtaining a first search result, and then determining or obtaining a second index based on the first search result, and inputting the second index into a second layer filter lookup table in the multi-layer filter lookup table for search, until the last layer filter lookup table outputs a filtered pixel or a target image block containing a filtered pixel. Optionally, the at least one filter lookup table may include a multi-layer filter lookup table.

[0261] In this embodiment, the target image block is determined or generated by at least one index determined or generated according to at least one reference parameter and at least one filter lookup table. By using the filter lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0262] Method 17, determining or generating at least one third intermediate value based on the neural network and at least one reference parameter, determining or generating at least one fourth intermediate value based on the at least one third intermediate value and at least one filter lookup table, and determining or generating a target image block based on the neural network and the at least fourth intermediate value;

[0263] Optionally, the third intermediate value and / or the fourth intermediate value may be data generated by an intermediate process during the filtering process, may be a filtered pixel generated after at least one filtering process, or may be other values. Optionally, the fourth intermediate value, and / or the third intermediate value, and / or the second intermediate value, and / or the first intermediate value may be the same or different.

[0264] Optionally, a neural network may be determined or selected based on at least one reference parameter, and at least one reference parameter may be input into the neural network for filtering, and the third intermediate value may be obtained as output. Optionally, at least one image block and at least one reference parameter of the image block may be input into the neural network for filtering, and the image block may be obtained as output after a filtering process. The pixels in the image block after a filtering process may be used as the third intermediate value.

[0265] Optionally, at least one third intermediate value may be input into at least one filter lookup table for search and determination to generate at least one fourth intermediate value. Optionally, at least one third intermediate value may be converted into an index (for example, the third intermediate value is directly used as the index, or the third intermediate value is deformed to obtain the index), and then the index is input into at least one filter lookup table for search, and at least one fourth intermediate value is output. Optionally, the filter lookup table includes a correspondence between the third intermediate value and the fourth intermediate value, and an index may be set in the filter lookup table, and each correspondence corresponds to an index.

[0266] Optionally, the at least one filter lookup table may be a multi-layer filter lookup table. The at least one third intermediate value may be input into the multi-layer filter lookup table for serial or parallel lookup, and the at least one fourth intermediate value may be output.

[0267] Optionally, at least one fourth intermediate value may be input into a neural network for model training, and a target image block including at least one filtered pixel may be output.

[0268] In this embodiment, at least one reference parameter is filtered based on a neural network, a filter lookup table, and a neural network architecture to determine or generate a target image block. The filtering effect can be improved by jointly performing filtering using a neural network and a filter lookup table, thereby improving the effect of video encoding and / or decoding.

[0269] Method 18, determining or generating at least one fifth intermediate value based on at least one reference parameter and at least one filter lookup table, determining or generating at least one sixth intermediate value based on at least one fifth intermediate value and a neural network, and determining or generating a target image block based on at least one sixth intermediate value and at least one filter lookup table.

[0270] Optionally, the fifth intermediate value and / or the sixth intermediate value may be data generated by an intermediate process during the filtering process, may be a filtered pixel generated after at least one filtering process, or may be other values. Optionally, the sixth intermediate value, and / or the fifth intermediate value, and / or the fourth intermediate value, and / or the third intermediate value, and / or the second intermediate value, and / or the first intermediate value may be the same or different.

[0271] Optionally, at least one reference parameter may be used to determine at least one selected filter lookup table from a plurality of filter lookup tables, and at least one reference parameter may be input into at least one filter lookup table to search and determine at least one fifth intermediate value. Optionally, an index may also be determined based on at least one reference parameter, and the index may be input into at least one filter lookup table to search and determine at least one fifth intermediate value.

[0272] Optionally, the filter lookup table includes a correspondence between at least one reference parameter and the fifth intermediate value, and an index may be set for each correspondence in the filter lookup table.

[0273] Optionally, at least one filter lookup table may be a multi-layer filter lookup table. The first reference parameter may be converted into an index and input into the multi-layer filter lookup table for parallel or serial search, and at least one fifth intermediate value may be output.

[0274] Optionally, at least one fifth intermediate value may be input into a neural network for filtering, and at least one sixth intermediate value may be output. An image block including at least one fifth intermediate value may also be input into a neural network for filtering, and at least one sixth intermediate value may be output.

[0275] Optionally, at least one sixth intermediate value may be input into at least one filter lookup table to search and determine the filter pixel, and the target image block may be determined or generated according to the filter pixel. An index may also be determined based on at least one sixth intermediate value and input into at least one filter lookup table to search and determine the filter pixel.

[0276] Optionally, the filter lookup table may further include a correspondence between at least one sixth intermediate value and a filter pixel, and an index may be set for each correspondence in the filter lookup table.

[0277] Optionally, at least one sixth intermediate value may be input into a multi-layer filtering lookup table for parallel or serial lookup, and a filtered pixel may be output, or a target image block containing the filtered pixel may be output.

[0278] In this embodiment, at least one reference parameter is filtered based on the architecture of a filter lookup table, a neural network, and a filter lookup table to determine or generate a target image block. The filtering effect can be improved by jointly performing filtering using a neural network and a filter lookup table, thereby improving the effect of video encoding and / or decoding.

[0279] Fourth embodiment

[0280] Based on any of the above embodiments, a fourth embodiment is proposed.

[0281] In this embodiment, the neural network includes at least one of the following methods 19 to 22.

[0282] Method 19: neural network based on fully connected layer;

[0283] Method 20: Neural network based on convolutional layer;

[0284] Method 21: Transformer-based neural network;

[0285] Method 22: Neural network based on hybrid convolutional and fully connected layers and Transformer.

[0286] Optionally, for method 14, at least one of a neural network based on a fully connected layer, a neural network based on a convolutional layer, a neural network based on a Transformer, and a neural network based on a hybrid convolutional layer, a fully connected layer, and a Transformer, and at least one reference parameter is used to determine or generate at least one first intermediate value, and the target image block is determined or generated based on the at least one first intermediate value and at least one filter lookup table.

[0287] Optionally, for method fifteen, at least one second intermediate value can be obtained by searching in at least one filter lookup table based on at least one reference parameter, and at least one of a neural network based on a fully connected layer, a neural network based on a convolutional layer, a neural network based on a Transformer, and a neural network based on a hybrid convolutional layer, a fully connected layer, and a Transformer, and at least one second intermediate value can be used to determine or generate a target image block.

[0288] Optionally, for method seventeen, at least one third intermediate value may be determined or generated based on a neural network based on a fully connected layer, a neural network based on a convolutional layer, a neural network based on a Transformer, and a neural network based on a hybrid convolutional layer, a fully connected layer, and a Transformer, and at least one reference parameter, and at least one fourth intermediate value may be determined or generated based on the at least one third intermediate value and at least one filter lookup table. And the target image block may be determined or generated based on at least one fourth intermediate value of a neural network based on a fully connected layer, a neural network based on a convolutional layer, a neural network based on a Transformer, and a neural network based on a hybrid convolutional layer, a fully connected layer, and a Transformer.

[0289] Optionally, for method 18, at least one fifth intermediate value may be determined or generated based on at least one reference parameter and at least one filter lookup table. At least one of a neural network based on a fully connected layer, a neural network based on a convolutional layer, a neural network based on a Transformer, and a neural network based on a hybrid convolutional layer, a fully connected layer, and a Transformer, and at least one fifth intermediate value, at least one sixth intermediate value is determined or generated, and a target image block is determined or generated based on at least one sixth intermediate value and at least one filter lookup table.

[0290] Optionally, at least one item can be selected from a neural network based on a fully connected layer, a neural network based on a convolutional layer, a neural network based on a Transformer, and a neural network based on a hybrid convolutional layer, a fully connected layer, and a Transformer according to at least one reference parameter, for executing any one of the fourteenth to eighteenth methods. For example, if the neural network used by the default block during filtering is a neural network based on a convolutional layer, then the selected neural network is determined to be a neural network based on a convolutional layer.

[0291] In this embodiment, when the neural network includes at least one of a neural network based on a convolutional layer, a neural network based on a Transformer, and a neural network based on a hybrid convolutional layer and a fully connected layer and a Transformer, at least one image block can be filtered, and filtering can also be performed in combination with at least one filter lookup table to further improve the filtering effect, thereby improving the effect of video encoding and / or decoding.

[0292] Optionally, when the neural network includes a neural network based on a convolutional layer, in method 14, determining or generating at least one first intermediate value according to the neural network and at least one reference parameter includes step S11 and step S12:

[0293] Step S11, processing at least one image block input into a neural network including a convolution module to obtain at least one second image block;

[0294] Optionally, processing the at least one image block may include rotation, flipping, scaling, and the like.

[0295] Optionally, the convolution module includes at least one convolution layer.

[0296] Optionally, step S11 may include rotating at least one image block input to the neural network including the convolution module to obtain at least one second image block. Optionally, the at least one image block may be rotated multiple times to obtain multiple second image blocks, and the angle of each rotation may be the same, such as 90 degrees.

[0297] Optionally, at least one image block and reference parameters may be input into a neural network including a convolution module, and at least one image block may be rotated to obtain at least one second image block.

[0298] Step S12: Process at least one second image block according to the convolution module and the reference parameters to determine or generate at least one first intermediate value.

[0299] Optionally, a convolution module and reference parameters may be used to perform convolution processing on each second image block to determine or generate an image block including at least one first intermediate value.

[0300] Optionally, when the neural network includes a neural network based on a hybrid convolutional and fully connected layer and a Transformer, step S11 and step S12 may also be performed.

[0301] In this embodiment, when the neural network is a convolutional layer-based neural network and the neural network is used for filtering, at least one input image block can be processed to obtain at least one second image block, the at least one second image block is processed according to the convolution module and reference parameters of the neural network to determine or generate a first intermediate value, and then the filtering process is performed in combination with at least one filter lookup table to determine or generate a target image block. Using the neural network and the filter lookup table together for filtering can improve the filtering effect, thereby improving the effect of video encoding and / or decoding.

[0302] Optionally, the convolution layer of the convolution module includes at least one of the following methods twenty-three to twenty-six.

[0303] Method 23, convolutional layer with asymmetric convolution;

[0304] Method 24: convolutional layer of grouped convolution;

[0305] Method 25, convolutional layer with partial convolution;

[0306] Method twenty-six, convolutional layer of depthwise separable convolution.

[0307] Optionally, for step S12, in the neural network, in the convolution module, if the convolution layer of the asymmetric convolution receives at least one second image block, the at least one second image block can be convolved in the convolution layer. Optionally, the convolution layer of the asymmetric convolution can include a convolution layer with a 1X3 convolution kernel, and can also include a convolution layer with a 1X2 convolution kernel.

[0308] Optionally, for each second image block, the input channels can be divided into multiple groups, convolution processing can be performed on the image block features of each group of input channels, and then the output results of each group of output channels can be merged to obtain the convolution result of the convolution layer of the grouped convolution on the second image block.

[0309] Optionally, the at least one second image block may be input into a partially convolutional convolutional layer, and the at least one second image block may be partially convolved according to the partially convolutional convolutional layer.

[0310] Optionally, the at least one second image block may be input into a convolutional layer of a depthwise separable convolution, and the depthwise separable convolution processing may be performed on the at least one second image block according to the convolutional layer of the depthwise separable convolution.

[0311] In this embodiment, when processing at least one second image block using the convolution module of the neural network and the reference parameters, processing can be performed according to at least one of the convolution layer of asymmetric convolution, the convolution layer of grouped convolution, the convolution layer of partial convolution and the convolution layer of depth-separable convolution to improve the filtering effect of the neural network. Then, the filtering process is performed in combination with at least one filter lookup table to determine or generate the target image block. The filtering process is performed together using the neural network and the filter lookup table to improve the filtering effect, thereby improving the effect of video encoding and / or decoding.

[0312] Optionally, the image processing method also includes at least one of methods twenty-seven to thirty.

[0313] Method twenty-seven: the convolution templates corresponding to at least two convolution layers in the convolution module are the same;

[0314] Mode 28: The convolution templates corresponding to at least two convolution layers in the convolution module include different convolution templates;

[0315] Mode twenty-nine, the pixel positions of the convolution kernels in at least two convolution templates are different;

[0316] Method 30, determining the number and position of pixels in the convolution template according to the receptive field.

[0317] Optionally, when performing convolution processing on at least one second image block according to the convolution module and the reference parameters in the neural network, the second image block can be convoluted according to each convolution layer in the convolution module, and each convolution layer can use a convolution template corresponding thereto to perform convolution processing on the second image block. The convolution template can be set and determined in advance. And when performing convolution template screening, it can be determined according to the receptive field of the convolution kernel of the convolution layer. For example, for a 1x3 convolution kernel, it can be determined that the number of pixels acted on by the convolution kernel in the convolution template corresponding to the 1x3 convolution kernel is 3. For example, for a 1x2 convolution kernel, it can be determined that the number of pixels acted on by the convolution kernel in the convolution template corresponding to the 1x2 convolution kernel is 2. Optionally, the pixel positions acted on by the convolution kernel in each convolution template can be different or the same, so there can be multiple convolution templates corresponding to the convolution kernel of the same receptive field. For example, for the receptive field of a 3x3 convolution kernel, there can be 84 1x3 convolution templates.

[0318] Optionally, the convolution module may include multiple convolution layers, so after the convolution module receives at least one second image block, different convolution layers may be used in sequence to perform convolution processing in a convolution order, and the convolution templates used by each convolution layer for convolution processing may be the same or different.

[0319] Optionally, the convolution layer in the convolution module may perform convolution processing on at least one second image block according to the convolution template and the reference parameters, and perform normalization output to obtain an image block including at least one first intermediate value. Optionally, the first intermediate value may be a filtered pixel after filtering processing by a neural network.

[0320] Alternatively, for example Fig. 9 As shown, after the neural network receives the normalized input image, the input image can be rotated. Optionally, the input image includes at least one image block, that is, the at least one image block is rotated to obtain at least one second image block. The rotation angle can be 90 degrees, 4 times clockwise or counterclockwise rotation.

[0321] When the convolution module in the neural network performs convolution processing on at least one second image block, at least one convolution layer in the convolution module can perform convolution processing on at least one second image block according to the convolution template. For example, the learnable parameters of the convolution layer of Conc1x3 include the convolution template. Therefore, the pixel extraction of at least one second image block can be performed in the convolution layer according to the convolution template to form a 1x3 image block, and then the convolution result with the 1x3 convolution kernel is averaged, and then at least one 1x1 convolution residual block (i.e., Conv1x1 block) is connected. For example, after connecting 3 1x1 convolution residual blocks, the output normalized image block (i.e., the normalized output) is sent to the next convolution module. Optionally, 9 convolution modules can be stacked to increase the network depth of the neural network, expand the receptive field, and enrich the extracted features. Optionally, in the 9 convolution modules, the convolution kernel of at least one convolution layer can use different convolution templates. For example, the convolution sum of the convolution layer in each convolution module can use the following: Fig.10 The convolution template shown. Optionally, the convolution template may include pixels I0 to I8. The number and position of pixels in the convolution template may be determined based on the receptive field of the convolution kernel. Therefore, for a 1x3 convolution kernel, it can be determined that the number of pixels acting in the corresponding convolution template is 3, and they may be at different positions. For a 1x2 convolution kernel, it can be determined that the number of pixels acting in the corresponding convolution template is 2, and they may be at different positions.

[0322] Alternatively, if Fig.10 As shown, in the neural network, the convolution module may include three levels, and each level of the convolution module may include at least one convolution layer, for example, the first level of the convolution module includes three convolution layers with 1x3 convolution kernels. The second level of the convolution module includes three convolution layers with 1x3 convolution kernels. The third level of the convolution module includes three convolution layers with 1x3 convolution kernels.

[0323] Optionally, in the first-level convolution module, the pixels (i.e., the selected pixels) in the convolution template corresponding to the convolution layer of the first 1x3 convolution kernel may be pixel I0, pixel I1, and pixel I2. The pixels in the convolution template corresponding to the convolution layer of the second 1x3 convolution kernel may be pixel I0, pixel I4, and pixel I5. The pixels in the convolution template corresponding to the convolution layer of the third 1x3 convolution kernel may be pixel I0, pixel I7, and pixel I8.

[0324] Optionally, in the second-level convolution module, the pixels (i.e., the selected pixels) in the convolution template corresponding to the convolution layer of the first 1x3 convolution kernel may be pixel I0, pixel I1, and pixel I2. The pixels in the convolution template corresponding to the convolution layer of the second 1x3 convolution kernel may be pixel I0, pixel I4, and pixel I7. The pixels in the convolution template corresponding to the convolution layer of the third 1x3 convolution kernel may be pixel I0, pixel I5, and pixel I8.

[0325] Optionally, in the third-level convolution module, the pixels (i.e., the selected pixels) in the convolution template corresponding to the convolution layer of the first 1x3 convolution kernel may be pixel I0, pixel I1, and pixel I2. The pixels in the convolution template corresponding to the convolution layer of the second 1x3 convolution kernel may be pixel I0, pixel I4, and pixel I8. The pixels in the convolution template corresponding to the convolution layer of the third 1x3 convolution kernel may be pixel I0, pixel I5, and pixel I7.

[0326] Alternatively, if Fig.10 As shown in , in the neural network, by rotating the input image block, the convolution layer of the 1x3 convolution kernel produces a 3x3 receptive field. For the receptive field of the 3x3 convolution kernel, there can be 84 1x3 convolution templates. Fig.11 As shown in FIG. 1 , the convolution template corresponding to the convolution kernel with a smaller receptive field is used to perform convolution processing on the at least one second image block after rotation, and the increase in the receptive field is not obvious. Fig.12 The asymmetric convolution layer shown in FIG. 1 performs convolution processing on at least one second image block, and can achieve a convolution effect of performing convolution processing on a receptive field of a convolution kernel approximately 3×3. Fig.13 As shown, for a 1x2 convolution kernel, use Fig.13 The convolution template shown in FIG. 1 can also achieve the convolution effect of convolution processing with a receptive field of approximately 3x3 convolution kernel. Fig.14 As shown, an expanded convolution kernel can be used, that is, the width and / or height covered by the 1x3 convolution kernel in the convolution layer exceeds 3, for example, covering the receptive field of the 4x4 convolution kernel. Then, the 1x3 convolution kernel performs convolution processing on at least one second image block after rotation, achieving a convolution effect similar to the receptive field of the 4x4 convolution kernel.

[0327] Optionally, based on Fig. 9 , the neural network can be optimized, such as Fig.15As shown, the normalized input image can be rotated in the neural network to obtain four second image blocks, and the Conv1x3 convolution layer uses learnable parameters (i.e., convolution templates) to perform convolution processing on each second image block, and uses 4 kinds of rotated learnable weights to perform weighted averaging in each direction corresponding to the convolution processing results, and then connects to the Conv1x1 convolution layer and performs normalized output.

[0328] Optionally, when the convolution module in the neural network performs convolution processing on the rotated second image block, the convolution processing can be performed in combination with reference parameters of at least one image block, such as taking the reference parameter as the position information of the pixel to be filtered as an example.

[0329] Optionally, the relative position code can be determined according to the position information of the pixel to be filtered, such as Fig.16 As shown, when the convolution layer of Conv1x3 performs convolution processing on the second image block after the rotation processing, the convolution processing can be performed together according to the convolution template and the relative position code. For example, at least one second image block is processed according to the convolution template in which the pixels acted by the convolution kernel are pixels I0, pixels I4 and pixels I5 and the corresponding relative position code, and then multiple layers of Conv1x1 convolution layers are connected until a normalized image block is output.

[0330] In this embodiment, when processing at least one second image block using the convolution module and reference parameters of the neural network, the convolution module can be set as follows, that is, the convolution templates corresponding to at least two convolution layers in the convolution module are the same, and / or, the convolution templates corresponding to at least two convolution layers include different convolution templates, and / or the pixel positions acted on by the convolution kernels in at least two templates are different, and / or the number and position of pixels in the convolution template are determined according to the receptive field of the convolution kernel, thereby improving the filtering effect of the neural network. Then, a filtering process is performed in combination with a filter lookup table to determine or generate a target image block. Using a neural network and a filter lookup table to perform filtering processing together can improve the filtering effect, thereby improving the effect of video encoding and / or decoding.

[0331] Fifth embodiment

[0332] Based on any of the above embodiments, a fifth embodiment is proposed.

[0333] In this embodiment, in method sixteen, determining or generating at least one index based on at least one reference parameter includes at least one of the following methods thirty-one to thirty-seven.

[0334] Method 31, determining or generating a first pixel parameter according to at least one reference parameter, and determining or generating at least one index according to the first pixel parameter;

[0335] Optionally, the pixel parameter may include a value or data obtained by mathematically calculating, deforming, or combining the pixel or a reference parameter related to the pixel. The first pixel parameter may include a value or data obtained by mathematically calculating, deforming, or combining at least one reference parameter.

[0336] Optionally, at least one pixel to be filtered in at least one image block can be input into a mathematical formula or model, or a function for calculation to obtain a first pixel parameter, and the first pixel parameter can be indexed, for example, directly using the first pixel parameter as an index, or deforming or otherwise processing the first pixel parameter to obtain the index, or inputting the first pixel parameter into a table containing a mapping relationship between pixel parameters and indices to obtain an index corresponding to the first pixel parameter.

[0337] Optionally, at least one index determined according to the first pixel parameter may be input into at least one filter lookup table to perform lookup to determine at least one filter pixel, and the target image block may be determined according to the at least one filter pixel.

[0338] Optionally, for method fourteen, at least one first intermediate value may be determined or generated based on the neural network and at least one reference parameter, a pixel parameter may be determined or generated based on the at least one first intermediate value, at least one index may be determined or generated based on the pixel parameter, and a corresponding filter pixel may be determined in at least one filter lookup table based on the at least one index, thereby determining or generating a target image block.

[0339] Optionally, for method fifteen, a first pixel parameter may be determined or generated based on at least one reference parameter, at least one index may be determined or generated based on the first pixel parameter, at least one filter lookup table may be searched based on the at least one index to obtain at least one second intermediate value, and a target image block may be determined or generated based on the neural network and the at least one second intermediate value.

[0340] Optionally, for method seventeen, at least one third intermediate value may be determined or generated based on the neural network and at least one reference parameter, a pixel parameter may be determined or generated based on the at least one third intermediate value, at least one index may be determined or generated based on the pixel parameter, at least one fourth intermediate value may be determined or generated by searching in at least one filter lookup table based on the at least one index, and a target image block may be determined or generated based on the neural network and the at least one fourth intermediate value.

[0341] Optionally, for method eighteen, pixel parameters may be determined or generated based on at least one reference parameter, at least one index may be determined or generated based on the pixel parameters, at least one fifth intermediate value may be searched in at least one filter lookup table based on the at least one index, at least one sixth intermediate value may be determined or generated based on the at least one fifth intermediate value and a neural network, new pixel parameters may be determined or generated based on the at least one sixth intermediate value (for example, the sixth intermediate value may be directly used as a new pixel parameter, or the sixth intermediate value may be deformed or otherwise processed to obtain a new pixel parameter), at least one index may be determined or generated based on the new pixel parameters, and at least one filter lookup table may be searched based on the at least one index to determine or generate a target image block.

[0342] In this embodiment, at least one index is determined or generated by determining or generating a first pixel parameter determined or generated according to at least one reference parameter, and a target image block is determined by searching in at least one filter lookup table according to the at least one index. By using the filter lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0343] Method 32, determining or generating at least one first characterization value according to at least one reference parameter and at least one characterization lookup table, and determining or generating at least one index according to the at least one first characterization value;

[0344] Optionally, the representation value can be a numerical value used to describe or represent a certain object, phenomenon feature, attribute, etc., which is often processed or selected to reflect key characteristics, such as using the average score to represent the learning level of a class. Optionally, the representation value can be a sampling value. For example, for 256 pixel values ​​in the range [0,1,2,…,255], the sampling value after sampling with a sampling interval of 4 bits is [0,16,32,48,64,80,96,112,128,144,160,176,192,208,224,240].

[0345] Optionally, the characterization lookup table may be a lookup table including characterization values, or a sampling lookup table. Optionally, the characterization lookup table may include a correspondence between pixels, a correspondence between characterization values ​​and non-characterization values, or a correspondence between characterization values ​​and pixel values. Optionally, an index may be set in the characterization lookup table so as to search the characterization lookup table according to the index to determine a corresponding characterization value.

[0346] Optionally, at least one reference parameter (such as the pixel value to be filtered) may be input into at least one characterization lookup table for search, so as to determine the characterization value corresponding to at least one pixel to be filtered, and output it to obtain the first characterization value. Optionally, at least one reference parameter may be converted into an index matching the characterization lookup table and then input into at least one characterization lookup table for search, so as to determine the first characterization value corresponding to at least one pixel value to be filtered.

[0347] Optionally, the index can be determined based on the first characterization value, for example, by directly using the first characterization value as the index, or by deforming or otherwise processing the first characterization value to obtain the index, or by inputting the first characterization value into a table containing a mapping relationship between characterization values ​​and indices to obtain an index corresponding to the first characterization value.

[0348] Optionally, at least one index determined according to the first representation value may be input into at least one filter lookup table to perform lookup to determine at least one filter pixel, and the target image block may be determined according to the at least one filter pixel.

[0349] Optionally, for method fourteen, at least one first intermediate value may be determined or generated based on the neural network and at least one reference parameter, at least one characterization value may be determined or generated based on at least one first intermediate value and at least one characterization lookup table, at least one index may be determined or generated based on the at least one characterization value, and a corresponding filtered pixel may be searched in at least one filter lookup table based on the at least one index, thereby determining or generating a target image block.

[0350] Optionally, for method fifteen, at least one characterization value can be determined or generated based on at least one reference parameter and at least one characterization lookup table, at least one index can be determined or generated based on the at least one characterization value, at least one second intermediate value can be obtained by searching in at least one filter lookup table based on the at least one index, and a target image block can be determined or generated based on the neural network and the at least one second intermediate value.

[0351] Optionally, for method seventeen, at least one third intermediate value may be determined or generated based on the neural network and at least one reference parameter, at least one characterization value may be determined or generated based on at least one third intermediate value and at least one characterization lookup table, at least one index may be determined or generated based on the at least one characterization value, at least one fourth intermediate value may be determined or generated by searching in at least one filter lookup table based on the at least one index, and a target image block may be determined or generated based on the neural network and the at least one fourth intermediate value.

[0352] Optionally, for the eighteenth method, at least one characterization value may be determined or generated based on at least one reference parameter and at least one characterization lookup table, at least one index may be determined or generated based on the at least one characterization value, at least one fifth intermediate value may be determined or generated based on searching in at least one filter lookup table based on the at least one index, at least one sixth intermediate value may be determined or generated based on the at least one fifth intermediate value and a neural network. At least one index may be determined or obtained based on the at least one sixth intermediate value and at least one characterization value determined or generated by the at least one characterization lookup table, and at least one index may be searched in at least one filter lookup table based on the at least one index to determine or generate a target image block.

[0353] In this embodiment, at least one first characterization value is determined or generated based on at least one reference parameter and at least one characterization lookup table, at least one index is determined or generated based on the at least one first characterization value, and the target image block is searched and determined in at least one filter lookup table based on the at least one index. By using the filter lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0354] Method 33, determining or generating a second pixel parameter according to at least one reference parameter, determining or generating at least one second characterization value according to the second pixel parameter and at least one characterization lookup table, and determining or generating at least one index according to the at least one second characterization value;

[0355] Optionally, the second pixel parameter may be a value or data obtained by mathematically calculating, deforming, or combining at least one reference parameter (such as the pixel value to be filtered). Optionally, the second pixel parameter may be the same as or different from the first pixel parameter.

[0356] Optionally, after determining or generating the second pixel parameter based on at least one reference parameter, the second pixel parameter may be input into at least one characterization lookup table for search, so as to determine and output the characterization value corresponding to the second pixel parameter, thereby obtaining at least one second characterization value. Optionally, the second pixel parameter may be converted into an index matching the characterization lookup table and then input into at least one characterization lookup table for search, so as to determine the second characterization value corresponding to the second pixel parameter.

[0357] Optionally, the index can be determined based on the second characterization value, for example, by directly using the second characterization value as the index, or by deforming or otherwise processing the second characterization value to obtain the index, or by inputting the second characterization value into a table containing a mapping relationship between characterization values ​​and indices to obtain an index corresponding to the second characterization value.

[0358] Optionally, at least one index determined according to the second representation value may be input into at least one filter lookup table to perform lookup to determine at least one filter pixel, and the target image block may be determined according to the at least one filter pixel.

[0359] Optionally, for method fourteen, at least one first intermediate value may be determined or generated based on a neural network and at least one reference parameter, a pixel parameter may be determined or generated based on the at least one first intermediate value, and at least one characterization value may be determined or generated, at least one index may be determined or generated based on the at least one characterization value, and a corresponding filtered pixel may be searched in at least one filter lookup table based on the at least one index, thereby determining or generating a target image block.

[0360] Optionally, for method fifteen, the pixel parameters determined or generated may be based on at least one reference parameter, and at least one characterization lookup table, to determine or generate at least one characterization value, to determine or generate at least one index based on the at least one characterization value, to search in at least one filter lookup table based on the at least one index to obtain at least one second intermediate value, and to determine or generate a target image block based on the neural network and the at least one second intermediate value.

[0361] Optionally, for method seventeen, at least one third intermediate value may be determined or generated based on the neural network and at least one reference parameter, pixel parameters may be determined or generated based on the at least one third intermediate value, and at least one characterization lookup table may be used to determine or generate at least one characterization value, at least one index may be determined or generated based on the at least one characterization value, at least one fourth intermediate value may be determined or generated by searching in at least one filter lookup table based on the at least one index, and a target image block may be determined or generated based on the neural network and the at least one fourth intermediate value.

[0362] Optionally, for the eighteenth method, at least one characterization value may be determined or generated based on the pixel parameter determined or generated by at least one reference parameter and at least one characterization lookup table, at least one index may be determined or generated based on the at least one characterization value, at least one fifth intermediate value may be determined or generated based on searching in at least one filter lookup table based on the at least one index, at least one sixth intermediate value may be determined or generated based on the at least one fifth intermediate value and a neural network. At least one characterization value may be determined or generated based on the pixel parameter determined or generated by the at least one sixth intermediate value and at least one characterization lookup table, at least one index may be determined or obtained based on the at least one characterization value determined or generated most recently, and at least one filter lookup table may be searched based on the at least one index to determine or generate a target image block.

[0363] In this embodiment, by determining or generating a second pixel parameter according to at least one reference parameter, determining or generating at least one second characterization value according to the second pixel parameter and at least one characterization lookup table, determining or generating at least one index according to at least one second characterization value, and searching and determining a target image block in at least one filter lookup table according to at least one index, the complexity of the filter processing can be reduced by using the filter lookup table for filter processing, thereby improving the efficiency of video encoding and / or decoding.

[0364] Method 34, determining or generating at least one third characterization value according to at least one reference parameter and at least one characterization lookup table, determining or generating a third pixel parameter according to the at least one third characterization value, and determining or generating at least one index according to the third pixel parameter;

[0365] Optionally, the third pixel parameter may be the same as or different from the first pixel parameter and / or the second pixel parameter.

[0366] Optionally, the reference parameter of at least one image block may be input into at least one representation lookup table for parallel or serial search to determine and output the representation value corresponding to the at least one reference parameter to obtain at least one third representation value. Optionally, the at least one reference parameter may be converted into an index matching the at least one representation lookup table and then input into the at least one representation lookup table for search to determine at least one third representation value.

[0367] Optionally, the third characterization value is processed to obtain a third pixel parameter, for example, the third characterization value is mathematically calculated, deformed, or combined to obtain a value or data as the third pixel parameter.

[0368] Optionally, the index can be determined based on the third pixel parameter, for example, the third pixel parameter can be directly used as the index, or the third pixel parameter can be deformed or otherwise processed to obtain the index, or the third pixel parameter can be input into a table containing a mapping relationship between pixel parameters and indices to obtain an index corresponding to the third pixel parameter.

[0369] Optionally, at least one index determined according to the third pixel parameter may be input into at least one filter lookup table to search and determine at least one filter pixel, and the target image block may be determined according to the at least one filter pixel.

[0370] Optionally, for method fourteen, at least one first intermediate value may be determined or generated based on a neural network and at least one reference parameter, at least one characterization value may be determined or generated based on at least one intermediate value and at least one characterization lookup table, pixel parameters may be determined or generated based on the at least one characterization value, at least one index may be determined or generated based on the pixel parameters, and a corresponding filtered pixel may be searched in at least one filter lookup table based on the at least one index, thereby determining or generating a target image block.

[0371] Optionally, for method fifteen, at least one characterization value may be determined or generated based on at least one reference parameter and at least one characterization lookup table, a pixel parameter may be determined or generated based on at least one characterization value, and at least one index may be determined or generated based on the pixel parameter, and at least one second intermediate value may be obtained by searching in at least one filter lookup table based on the at least one index, and a target image block may be determined or generated based on the neural network and the at least one second intermediate value.

[0372] Optionally, for method seventeen, at least one third intermediate value may be determined or generated based on the neural network and at least one reference parameter, at least one characterization value may be determined or generated based on at least one third intermediate value and at least one characterization lookup table, pixel parameters may be determined or generated based on the at least one characterization value, at least one index may be determined or generated based on the pixel parameters, at least one fourth intermediate value may be determined or generated by searching in at least one filter lookup table based on the at least one index, and a target image block may be determined or generated based on the neural network and the at least one fourth intermediate value.

[0373] Optionally, for method eighteen, at least one characterization value can be determined or generated based on at least one reference parameter and at least one characterization lookup table, a pixel parameter can be determined or generated based on at least one characterization value, at least one index can be determined or generated based on the pixel parameter, at least one fifth intermediate value can be determined or generated by searching in at least one filter lookup table based on the at least one index, and at least one sixth intermediate value can be determined or generated based on the at least one fifth intermediate value and a neural network.

[0374] At least one characterization value is determined or generated according to at least one sixth intermediate value and at least one characterization lookup table, a pixel parameter is determined or generated according to the at least one characterization value, at least one index is determined or obtained according to the pixel parameter, and at least one filter lookup table is searched according to the at least one index to determine or generate a target image block.

[0375] In this embodiment, at least one third characterization value is determined or generated according to at least one reference parameter and at least one characterization lookup table, a third pixel parameter is determined or generated according to at least one third characterization value, at least one index is determined or generated according to the third pixel parameter, and a target image block is determined by searching at least one filter lookup table according to the at least one index. By using the filter lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0376] Method 35, determining or generating at least one fourth representation value according to the high-order significant bits of at least one pixel value to be filtered, and determining or generating at least one index according to the at least one fourth representation value;

[0377] Optionally, the high significant bit may be a valid bit in a pre-set bit when the pixel value of the pixel is expressed in binary code. The pre-set bit may be the highest bit, the highest four bits, etc., and may be set according to user requirements. For example, for an 8-bit pixel, the first 4 valid bits of the pixel value of the pixel may be used as the high significant bit. If there is a pixel 1=36 (00100100), the high significant bit of pixel 1 is 2 (i.e., 0010).

[0378] Optionally, the high-order significant bits of at least one pixel value to be filtered in at least one image block may be determined, and at least one characterization value may be determined based on the high-order significant bits. For example, the high-order significant bits of each of the at least one pixel value to be filtered may be processed to obtain at least one fourth characterization value. For example, the high-order significant bits of the pixel value to be filtered may be directly used as the fourth characterization value. The high-order significant bits may also be deformed, weighted, or otherwise processed to obtain the fourth characterization value.

[0379] Optionally, the index can be determined based on at least one fourth characterization value, for example, by directly using the fourth characterization value as the index, or by deforming or otherwise processing the fourth characterization value to obtain the index, or by inputting the fourth characterization value into a table containing a mapping relationship between characterization values ​​and indices to obtain an index corresponding to the fourth characterization value.

[0380] Optionally, at least one index determined according to the fourth representation value may be input into at least one filter lookup table to perform lookup to determine at least one filter pixel, and the target image block may be determined according to the at least one filter pixel.

[0381] Optionally, for method fourteen, at least one first intermediate value may be determined or generated based on the neural network and at least one reference parameter, at least one representation value may be determined or generated based on the high-order significant bits of the at least one intermediate value, at least one index may be determined and generated based on the at least one representation value, and the corresponding filter pixel may be searched and determined in at least one filter lookup table based on the at least one index, thereby determining or generating a target image block.

[0382] Optionally, for method fifteen, at least one characterization value can be determined or generated based on the high-order significant bits of the pixel value to be filtered in at least one reference parameter, at least one index can be determined or generated based on the at least one characterization value, at least one filter lookup table can be searched based on the at least one index to obtain at least one second intermediate value, and the target image block can be determined or generated based on the neural network and the at least one second intermediate value.

[0383] Optionally, for method seventeen, at least one third intermediate value may be determined or generated based on the neural network and at least one reference parameter, at least one representation value may be determined or generated based on the high-order significant bits of the at least one third intermediate value, at least one index may be determined or generated based on the at least one representation value, at least one fourth intermediate value may be determined or generated by searching in at least one filter lookup table based on the at least one index, and a target image block may be determined or generated based on the neural network and the at least one fourth intermediate value.

[0384] Optionally, for method eighteen, at least one characterization value can be determined or generated based on the high-order significant bits of the pixel value to be filtered in at least one reference parameter, at least one index can be determined or generated based on the at least one characterization value, at least one fifth intermediate value can be determined or generated by searching in at least one filter lookup table based on the at least one index, and at least one sixth intermediate value can be determined or generated based on the at least one fifth intermediate value and a neural network.

[0385] At least one representation value is determined or generated according to the high-order significant bits of at least one sixth intermediate value, at least one index is determined or obtained according to the at least one representation value, and at least one filter lookup table is searched according to the at least one index to determine or generate a target image block.

[0386] In this embodiment, at least one fourth representation value is determined or generated according to the high-order significant bits of at least one pixel value to be filtered, at least one index is determined or generated according to the at least one fourth representation value, and the target image block is determined by searching at least one filter lookup table according to the at least one index. By using the filter lookup table to perform filtering processing, the complexity of the filtering processing can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0387] Mode 36, determining or generating at least one fifth characterization value according to the characterization parameter, and determining or generating at least one index according to the at least one fifth characterization value;

[0388] Optionally, the characterization parameter may be a parameter corresponding to the characterization value, such as an interval between characterization values ​​(ie, a characterization interval). Optionally, when the characterization value is a sampling value, the characterization parameter may be a sampling interval.

[0389] Optionally, at least one fifth characterization value may be determined based on at least one reference parameter (such as the pixel value to be filtered) and the characterization parameter. The pixel value of a pixel separated from the pixel to be filtered by a characterization interval is used as the fifth characterization value. The fifth characterization value may also be determined based on a multiple of the characterization interval and the pixel to be filtered. For example, the pixel value of a pixel separated from the pixel to be filtered by a preset multiple of the characterization interval is used as the fifth characterization value.

[0390] Optionally, the index can be determined based on the fifth characterization value, for example, the fifth characterization value can be directly used as the index, or the fifth characterization value can be deformed or otherwise processed to obtain the index, or the fifth characterization value can be input into a table containing a mapping relationship between characterization values ​​and indices to obtain an index corresponding to the fifth characterization value.

[0391] Optionally, at least one index determined according to the fifth representation value may be input into at least one filter lookup table to perform lookup to determine at least one filter pixel, and the target image block may be determined according to the at least one filter pixel.

[0392] Optionally, for method fourteen, at least one first intermediate value may be determined or generated based on the neural network and at least one reference parameter, a representation value corresponding to the at least one first intermediate value may be determined based on the representation parameter, at least one index may be determined and generated based on the at least one representation value, and a corresponding filter pixel may be searched and determined in at least one filter lookup table based on the at least one index, thereby determining or generating a target image block.

[0393] Optionally, for method fifteen, a characterization value corresponding to at least one reference parameter may be determined based on the characterization parameter, at least one index may be determined or generated based on the at least one characterization value, at least one second intermediate value may be obtained by searching in at least one filter lookup table based on the at least one index, and a target image block may be determined or generated based on the neural network and the at least one second intermediate value.

[0394] Optionally, for method seventeen, at least one third intermediate value may be determined or generated based on the neural network and at least one reference parameter, a representation value corresponding to at least one third intermediate value may be determined or generated based on the representation parameter, at least one index may be determined or generated based on at least one representation value, at least one fourth intermediate value may be determined or generated by searching in at least one filter lookup table based on the at least one index, and a target image block may be determined or generated based on the neural network and the at least one fourth intermediate value.

[0395] Optionally, for method eighteen, a characterization value corresponding to at least one reference parameter may be determined based on the characterization parameter, at least one index may be determined or generated based on at least one characterization value, at least one fifth intermediate value may be determined or generated based on searching in at least one filter lookup table based on the at least one index, and at least one sixth intermediate value may be determined or generated based on the at least one fifth intermediate value and a neural network.

[0396] Determine a characterization value corresponding to at least one sixth intermediate value according to the characterization parameter, determine or obtain at least one index according to the at least one characterization value, and search in at least one filter lookup table according to the at least one index to determine or generate a target image block.

[0397] In this embodiment, at least one fifth characterization value is determined or generated according to the characterization parameter, at least one index is determined or generated according to the at least one fifth characterization value, and the target image block is determined by searching at least one filter lookup table according to the at least one index. By using the filter lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0398] Method thirty-seven: Determine or generate at least one non-characteristic value based on at least one reference parameter, and determine or generate at least one index based on the at least one non-characteristic value.

[0399] Optionally, the non-representational value may be data or pixel values ​​other than the representational value. For example, when the representational value is a sampled value, the non-representational value may be a non-sampled value, that is, if each pixel (such as a pixel to be filtered) is sampled according to a certain rule (such as a sampling interval) in at least one image block to obtain each sampled value, then the non-sampled value is a pixel value that has not been sampled.

[0400] Optionally, at least one non-representational value may be determined or obtained by using a certain rule according to at least one pixel value to be filtered in at least one image block. Optionally, at least one non-representational value determined according to at least one filtered pixel value may be processed again according to at least one reference parameter, such as format conversion, to obtain at least one new non-representational value.

[0401] Optionally, at least one non-representational value is converted into at least one index (for example, the non-representational value is directly used as the index, or the non-representational value is deformed or otherwise processed to obtain the index), and the at least one index is input into at least one filter lookup table for search, at least one filter pixel is determined, and the target image block is determined or generated based on the at least one filter pixel.

[0402] Optionally, an interpolation method may be used to determine the nearest representation value to the pixel corresponding to at least one non-representation value, and the representation value may be converted into an index, which is then input into at least one filter lookup table corresponding to the representation value for search, thereby determining at least one filtered pixel, and determining or generating a target image block based on the at least one filtered pixel.

[0403] Optionally, for method fourteen, at least one first intermediate value may be determined or generated based on the neural network and at least one reference parameter, at least one non-representational value may be determined based on the at least one first intermediate value, at least one index may be determined and generated based on the at least one non-representational value, and a corresponding filtered pixel may be searched and determined in at least one filter lookup table based on the at least one index, thereby determining or generating a target image block.

[0404] Optionally, for method fifteen, at least one non-representational value can be determined or generated based on at least one reference parameter, at least one index can be determined or generated based on at least one non-representational value, at least one second intermediate value can be obtained by searching in at least one filter lookup table based on the at least one index, and the target image block can be determined or generated based on the neural network and the at least one second intermediate value.

[0405] Optionally, for method seventeen, at least one third intermediate value may be determined or generated based on the neural network and at least one reference parameter, at least one non-representation value may be determined or generated based on the at least one third intermediate value, at least one index may be determined or generated based on the at least one non-representation value, at least one fourth intermediate value may be determined or generated by searching in at least one filter lookup table based on the at least one index, and a target image block may be determined or generated based on the neural network and the at least one fourth intermediate value.

[0406] Optionally, for method eighteen, at least one non-characterization value can be determined or generated based on at least one reference parameter, at least one index can be determined or generated based on at least one non-characterization value, at least one fifth intermediate value can be determined or generated based on at least one fifth intermediate value and a neural network.

[0407] At least one non-representation value is determined or obtained according to at least one sixth intermediate value, at least one index is determined or generated according to the at least one non-representation value, and at least one filter lookup table is searched according to the at least one index to determine or generate a target image block.

[0408] In this embodiment, at least one non-representational value is determined or generated according to at least one reference parameter, at least one index is determined or generated according to at least one non-representational value, and the target image block is determined by searching at least one filter lookup table according to the at least one index. By using the filter lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0409] Sixth embodiment

[0410] The present application also provides a processing device, referring to Fig.17 , the processing device comprises:

[0411] The processing module A10 is used to determine or generate a target image block according to a reference parameter of at least one image block and a filter lookup table.

[0412] Optionally, the reference parameters include at least one of the following: quantization parameter; boundary strength; characterization interval; position information of pixels to be filtered; pixel value to be filtered; size information of image blocks; filtering information of neighboring blocks; filtering information of non-neighboring blocks; filtering information of cross-component blocks; filtering information of co-located blocks; filtering information of time domain blocks; filtering information of default blocks; filtering information of candidate blocks.

[0413] Optionally, the candidate blocks are determined by motion vectors and / or block vectors.

[0414] Optionally, the processing module A10 is configured to perform at least one of the following:

[0415] Determine or generate at least one first intermediate value according to the neural network and at least one reference parameter, and determine or generate a target image block according to the at least one first intermediate value and at least one filter lookup table;

[0416] Searching in at least one filter lookup table according to at least one reference parameter to obtain at least one second intermediate value, and determining or generating a target image block according to the neural network and the at least one second intermediate value;

[0417] Determine or generate at least one index according to at least one reference parameter, and determine or generate a target image block according to the at least one index and at least one filter lookup table;

[0418] Determine or generate at least one third intermediate value based on the neural network and at least one reference parameter, determine or generate at least one fourth intermediate value based on the at least one third intermediate value and at least one filter lookup table, and determine or generate a target image block based on the neural network and the at least fourth intermediate value;

[0419] At least one fifth intermediate value is determined or generated based on at least one reference parameter and at least one filter lookup table, at least one sixth intermediate value is determined or generated based on at least one fifth intermediate value and a neural network, and a target image block is determined or generated based on at least one sixth intermediate value and at least one filter lookup table.

[0420] Optionally, the neural network includes at least one of the following: a neural network based on a fully connected layer; a neural network based on a convolutional layer; a neural network based on a Transformer; a neural network based on a hybrid convolutional and fully connected layer and a Transformer.

[0421] Optionally, when the neural network includes a neural network based on a convolutional layer, the processing module A10 is used to:

[0422] Processing at least one image block input to a neural network including a convolution module to obtain at least one second image block;

[0423] At least one second image block is processed according to the convolution module and the reference parameters to determine or generate at least one first intermediate value.

[0424] Optionally, the convolution layer of the convolution module includes at least one of the following: a convolution layer of asymmetric convolution; a convolution layer of grouped convolution; a convolution layer of partial convolution; a convolution layer of depth-wise separable convolution.

[0425] Optionally, the processing module A10 further includes at least one of the following:

[0426] The convolution templates corresponding to at least two convolution layers in the convolution module are the same; the convolution templates corresponding to at least two convolution layers in the convolution module include different convolution templates; the pixel positions affected by the convolution kernels in at least two convolution templates are different; the number and position of pixels in the convolution template are determined according to the receptive field.

[0427] Optionally, the processing module A10 is configured to perform at least one of the following:

[0428] Determine or generate a first pixel parameter according to at least one reference parameter, and determine or generate at least one index according to the first pixel parameter;

[0429] Determine or generate at least one first characterization value according to at least one reference parameter and at least one characterization lookup table, and determine or generate at least one index according to the at least one first characterization value;

[0430] Determine or generate a second pixel parameter according to at least one reference parameter, determine or generate at least one second characterization value according to the second pixel parameter and at least one characterization lookup table, and determine or generate at least one index according to the at least one second characterization value;

[0431] Determine or generate at least one third characterization value according to at least one reference parameter and at least one characterization lookup table, determine or generate a third pixel parameter according to the at least one third characterization value, and determine or generate at least one index according to the third pixel parameter;

[0432] Determine or generate at least one fourth representation value according to the high-order significant bits of at least one pixel value to be filtered, and determine or generate at least one index according to the at least one fourth representation value;

[0433] Determine or generate at least one fifth characterization value according to the characterization parameter, and determine or generate at least one index according to the at least one fifth characterization value;

[0434] At least one non-characteristic value is determined or generated according to at least one reference parameter, and at least one index is determined or generated according to the at least one non-characteristic value.

[0435] The processing device provided in the embodiment of the present application has similar implementation principles and beneficial effects to the technical solutions shown in the above-mentioned corresponding method embodiments, and will not be described in detail here.

[0436] An embodiment of the present application also provides a processing device, including a memory and a processor. An image processing program is stored in the memory. When the image processing program is executed by the processor, the steps of the image processing method in any of the above embodiments are implemented.

[0437] An embodiment of the present application further provides a storage medium, on which an image processing program is stored. When the image processing program is executed by a processor, the steps of the image processing method in any of the above embodiments are implemented.

[0438] In the embodiments of the smart terminal and storage medium provided in the present application, all technical features of any of the above-mentioned image processing method embodiments may be included, and the expanded and explained contents of the specification are basically the same as those of the embodiments of the above-mentioned methods, and will not be repeated here.

[0439] The embodiment of the present application further provides a computer program product, which includes a computer program code. When the computer program code runs on a computer, the computer executes the methods in the above various possible implementation modes.

[0440] An embodiment of the present application also provides a chip, including a memory and a processor, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that a device equipped with the chip executes the methods in various possible implementation modes as described above.

[0441] It is understood that the above scenarios are only examples and do not constitute a limitation on the application scenarios of the technical solutions provided in the embodiments of the present application. The technical solutions of the present application can also be applied to other scenarios. For example, it is known to those skilled in the art that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0442] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0443] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0444] The units in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0445] In the present application, the same or similar terminology concepts, technical solutions and / or application scenario descriptions are generally described in detail only the first time they appear. When they appear again later, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of the present application, for the same or similar terminology concepts, technical solutions and / or application scenario descriptions that are not described in detail later, reference can be made to the previous related detailed descriptions.

[0446] In the present application, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0447] The various technical features of the technical solution of the present application can be arbitrarily combined. In order to make the description concise, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present application.

[0448] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, a controlled terminal, or a network device, etc.) to execute the method of each embodiment of the present application.

[0449] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. Computer instructions can be stored in a storage medium, or transmitted from one storage medium to another storage medium. For example, computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. Available media can be magnetic media, (e.g., floppy disk, storage disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state storage disk Solid State Disk (SSD)), etc.

[0450] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image processing method, characterized in that: Includes steps: S10, determining or generating a target image block according to a reference parameter of at least one image block and a filter lookup table.

2. The method according to claim 1, characterized in that The reference parameters include at least one of the following: Quantization parameters; Boundary strength; Characterization parameters; Position information of pixels to be filtered; The pixel value to be filtered; Image block size information; Filter information of neighbor blocks; Filter information of non-neighboring blocks; Filter information across component blocks; Filter information of the same-position block; Filter information of time domain blocks; Filter information of the default block; Filter information of candidate blocks.

3. The method according to claim 2, characterized in that The candidate block is determined by a motion vector and / or a block vector; and / or, step S10 includes at least one of the following: Determine or generate at least one first intermediate value according to the neural network and at least one reference parameter, and determine or generate a target image block according to the at least one first intermediate value and at least one filter lookup table; Searching in at least one filter lookup table according to at least one reference parameter to obtain at least one second intermediate value, and determining or generating a target image block according to the neural network and the at least one second intermediate value; Determine or generate at least one index according to at least one reference parameter, and determine or generate a target image block according to the at least one index and at least one filter lookup table; Determine or generate at least one third intermediate value based on the neural network and at least one reference parameter, determine or generate at least one fourth intermediate value based on the at least one third intermediate value and at least one filter lookup table, and determine or generate a target image block based on the neural network and the at least fourth intermediate value; At least one fifth intermediate value is determined or generated based on at least one reference parameter and at least one filter lookup table, at least one sixth intermediate value is determined or generated based on at least one fifth intermediate value and a neural network, and a target image block is determined or generated based on at least one sixth intermediate value and at least one filter lookup table.

4. The method according to claim 3, characterized in that A neural network includes at least one of the following: Neural networks based on fully connected layers; Neural networks based on convolutional layers; Transformer-based neural networks; Neural network based on hybrid convolutional and fully connected layers and Transformer.

5. The method according to claim 3, characterized in that When the neural network includes a neural network based on a convolutional layer, determining or generating at least one first intermediate value according to the neural network and at least one reference parameter includes: Processing at least one image block input to a neural network including a convolution module to obtain at least one second image block; At least one second image block is processed according to the convolution module and the reference parameters to determine or generate at least one first intermediate value.

6. The method according to claim 5, characterized in that The convolutional layer of a convolutional module includes at least one of the following: Convolutional layers with asymmetric convolutions; Convolutional layers for grouped convolutions; Convolutional layers with partial convolutions; Convolutional layer for depthwise separable convolution.

7. The method according to claim 5, characterized in that Also includes at least one of the following: The convolution templates corresponding to at least two convolution layers in the convolution module are the same; The convolution templates corresponding to at least two convolution layers in the convolution module include different convolution templates; The pixel positions of the convolution kernels in at least two convolution templates are different; The number and position of pixels in the convolution template are determined based on the receptive field.

8. The method according to claim 3, characterized in that Determining or generating at least one index according to at least one reference parameter includes at least one of the following: Determine or generate a first pixel parameter according to at least one reference parameter, and determine or generate at least one index according to the first pixel parameter; Determine or generate at least one first characterization value according to at least one reference parameter and at least one characterization lookup table, and determine or generate at least one index according to the at least one first characterization value; Determine or generate a second pixel parameter according to at least one reference parameter, determine or generate at least one second characterization value according to the second pixel parameter and at least one characterization lookup table, and determine or generate at least one index according to the at least one second characterization value; Determine or generate at least one third characterization value according to at least one reference parameter and at least one characterization lookup table, determine or generate a third pixel parameter according to the at least one third characterization value, and determine or generate at least one index according to the third pixel parameter; Determine or generate at least one fourth representation value according to the high-order significant bits of at least one pixel value to be filtered, and determine or generate at least one index according to the at least one fourth representation value; Determine or generate at least one fifth characterization value according to the characterization parameter, and determine or generate at least one index according to the at least one fifth characterization value; At least one non-characteristic value is determined or generated according to at least one reference parameter, and at least one index is determined or generated according to the at least one non-characteristic value.

9. A processing device, characterized in that: include: A memory and a processor, wherein an image processing program is stored in the memory, and when the image processing program is executed by the processor, the steps of the image processing method according to claim 1 are implemented.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the image processing method according to claim 1 are implemented.

Images