Image processing method, processing device and storage medium

Abstract

Provided in the present application are an image processing method, a processing device and a storage medium. The image processing method comprises: determining a target image block according to at least one first filtered pixel. By means of the technical solution of the present application, the filtering effect of filtering processing can be improved, thereby improving the efficiency of video coding and / or decoding.

Description

Image processing method, processing device and storage medium Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, processing device and storage medium. Background Art

[0002] Existing high-efficiency video coding frameworks, such as Neural Network Based Video Coding (NNVC) and / or Enhanced Compression Model (ECM), propose a video frame encoding technology to improve coding performance without significantly increasing computational complexity.

[0003] During the process of conceiving and implementing this application, the inventors discovered at least the following problems: During the loop filtering stage of the encoding and decoding process, for example, the low complexity neural network loop filter (LC-NNLF) structure introduced in NNVC for loop filtering processing increases the encoding and decoding complexity due to the high complexity of the neural network, thereby limiting the efficiency of video encoding and / or decoding.

[0004] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the Invention

[0005] In response to the above technical problems, the present application provides an image processing method, a processing device and a storage medium, aiming to solve how to improve the filtering effect of filtering processing, thereby improving the efficiency of video encoding and / or decoding.

[0006] This application provides an image processing method that can be applied to a processing device, including:

[0007] A target image block is determined according to at least one first filtered pixel.

[0008] Optionally, a method for determining at least one first filtered pixel includes at least one of the following:

[0009] Determining by searching in at least one filter lookup table according to at least one pixel to be filtered;

[0010] Determined based on at least one first intermediate value and at least one filter lookup table;

[0011] A first filtered pixel corresponding to at least one second intermediate value is determined according to the neural network.

[0012] Optionally, the image processing method further includes at least one of the following:

[0013] At least one pixel to be filtered is in at least one image block;

[0014] Determining at least one first intermediate value corresponding to at least one pixel to be filtered according to a neural network;

[0015] At least one second intermediate value is determined by searching at least one filtering lookup table according to at least one pixel to be filtered.

[0016] Optionally, the step of searching at least one filter lookup table for at least one pixel to be filtered comprises at least one of the following:

[0017] Determining a first pixel parameter according to at least one pixel to be filtered, and determining the first pixel parameter according to the first pixel parameter and at least one filter lookup table;

[0018] Determining at least one first characterization value based on at least one pixel to be filtered and at least one characterization lookup table, and determining at least one first characterization value based on the at least one first characterization value and at least one filtering lookup table;

[0019] Determining a second pixel parameter based on at least one pixel to be filtered, determining at least one second characterization value based on the second pixel parameter and at least one characterization lookup table, and determining based on the at least one second characterization value and at least one filtering lookup table;

[0020] Determining at least one third characterization value based on at least one pixel to be filtered and at least one characterization lookup table, determining a third pixel parameter based on the at least one third characterization value, and determining based on the third pixel parameter and the at least one filtering lookup table;

[0021] Determining at least one fourth representation value according to the most significant bits of at least one pixel to be filtered, and determining according to the at least one fourth representation value and at least one filter lookup table;

[0022] Determining at least one fifth characterization value corresponding to at least one pixel to be filtered according to the characterization parameter, and determining the value according to the at least one fifth characterization value and at least one filter lookup table;

[0023] Determine at least one index according to at least one pixel to be filtered, and search and determine in at least one filtering lookup table according to the at least one index;

[0024] At least one non-representational value is determined according to at least one pixel to be filtered, and is determined according to the at least one non-representational value and a filtering lookup table.

[0025] Optionally, determining at least one index according to at least one pixel to be filtered includes at least one of the following:

[0026] determining a fourth pixel parameter according to at least one pixel to be filtered, and determining at least one index according to the fourth pixel parameter;

[0027] Determine at least one sixth representation value according to at least one pixel to be filtered and a representation lookup table, and determine at least one index according to the at least one sixth representation value;

[0028] Determine a fifth pixel parameter according to at least one pixel to be filtered, determine at least one seventh representation value according to the fifth pixel parameter and a representation lookup table, and determine at least one index according to the at least one seventh representation value;

[0029] Determining at least one eighth characterization value based on at least one pixel to be filtered and a characterization lookup table, determining a sixth pixel parameter based on the at least one eighth characterization value, and determining at least one index based on the sixth pixel parameter;

[0030] Determine at least one ninth representation value according to the most significant bit of at least one pixel to be filtered, and determine at least one index according to the at least ninth representation value;

[0031] At least one tenth representation value corresponding to at least one pixel to be filtered is determined according to the representation parameter, and at least one index is determined according to the at least one tenth representation value.

[0032] Optionally, searching and determining in at least one filter lookup table according to at least one index includes: searching and determining in a multi-layer filter lookup table according to at least one index.

[0033] Optionally, the image processing method further includes at least one of the following:

[0034] Searching in a multi-layer filter lookup table according to at least one index is performed in a serial or parallel manner;

[0035] The input of the first filter lookup table in the multi-layer filter lookup table is at least one index;

[0036] The output of the last layer of filter lookup table is at least one first filtered pixel;

[0037] The output of the previous filter lookup table of the multi-layer filter lookup table is the input of the next filter lookup table.

[0038] Optionally, determining based on at least one non-characteristic value and a filter lookup table includes at least one of the following:

[0039] searching and determining in a filter lookup table an eleventh characterization value corresponding to at least one non-characterization value;

[0040] At least one fourth filtered pixel is determined according to an eleventh representation value corresponding to the at least one non-representation value and a filter lookup table, and is determined according to the representation parameter, the at least one fourth filtered pixel, and an interpolation weight.

[0041] Optionally, a method for determining the at least one eleventh characterization value includes at least one of the following:

[0042] determining based on at least one non-characteristic value and a characterization lookup table;

[0043] Determined based on the most significant bit of at least one non-representational value;

[0044] An eleventh characterization value corresponding to the at least one non-characterization value is determined according to the characterization parameter.

[0045] Optionally, the interpolation weight is determined by at least one of the following methods:

[0046] Determined based on at least one non-characteristic value and at least one adjacent characterizing value;

[0047] Determined based on the least significant bit of at least one non-representational value;

[0048] Determined according to the characterization parameters.

[0049] The present application also provides a processing device, comprising:

[0050] The processing module is configured to determine a target image block according to at least one first filtered pixel.

[0051] The present application also provides a processing device, comprising: a memory and a processor, wherein an image processing program is stored in the memory, and when the image processing program is executed by the processor, the steps of any of the above-mentioned image processing methods are implemented.

[0052] The present application also provides a storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-mentioned image processing methods.

[0053] As described above, the processing method of the present application can be applied to a processing device, including: determining a target image block based on at least one first filtered pixel. Through the technical solution of the present application, it is possible to directly determine the filtered target image block based on the first filtered pixel, thereby avoiding the need for a neural network to perform filtering during the loop filtering stage, improving the filtering effect of the target image block, and thereby improving the efficiency of video encoding and / or decoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for describing the embodiments. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without inventive work.

[0055] FIG1 is a schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present application;

[0056] FIG2 is a diagram of a communication network system architecture provided by an embodiment of the present application;

[0057] FIG3 is a schematic diagram of the hardware structure of a controller 140 provided in this application;

[0058] FIG4 is a schematic diagram of the hardware structure of a network node 150 provided in this application;

[0059] FIG5 is a schematic flow chart of an image processing method according to the first embodiment;

[0060] FIG6 is a schematic diagram of the encoding and decoding process in the image processing method;

[0061] FIG7 is a schematic diagram of the architecture of a neural network using the CP decomposition strategy;

[0062] FIG8 is a schematic diagram of a scene in which an interpolation operation is performed in an image processing method according to a seventh embodiment;

[0063] FIG9 is a schematic diagram of another scenario in which an interpolation operation is performed in an image processing method according to the seventh embodiment;

[0064] FIG10 is a schematic diagram of a processing module of a processing device.

[0065] The purpose of this application, its features, and advantages will be further described in conjunction with the embodiments and with reference to the accompanying drawings. The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and the accompanying text are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of this application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0066] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0067] It should be noted that, in this article, the term "comprises", "includes" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements. In addition, the components, features, and elements with the same names in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further in conjunction with the context in the specific embodiment. It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this article, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining". Furthermore, as used in this article, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It should be further understood that the terms "comprising" and "including" indicate the presence of the described features, steps, operations, elements, components, items, kinds, and / or groups, but do not exclude the presence, occurrence or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms "or", "and / or", "including at least one of the following" and the like used in this application may be interpreted as inclusive, or mean any one or any combination. For example, “comprising at least one of the following: A, B, C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”; and for another example, “A, B or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”. An exception to this definition will occur only when a combination of elements, functions, steps or operations are inherently mutually exclusive in some manner.

[0068] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence in the order indicated by the arrows. Unless clearly stated herein, the execution of these steps is not strictly limited in order, and they can be performed in other orders. Moreover, at least a portion of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0069] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0070] It should be noted that in this article, step codes such as S10 and S20 are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the order. When implementing the step, those skilled in the art may execute S20 first and then S10, etc., but these should all be within the scope of protection of this application.

[0071] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0072] In the subsequent description, the use of suffixes such as "module", "component" or "unit" to represent elements is only for the purpose of facilitating the description of the present application and has no specific meaning. Therefore, "module", "component" or "unit" can be used interchangeably.

[0073] The processing device in this application may be a smart terminal or a server. Optionally, the smart terminal may be implemented in various forms. For example, the smart terminal described in this application may include smart terminals such as mobile phones, tablet computers, laptop computers, PDAs, portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, and fixed terminals such as digital TVs and desktop computers.

[0074] The subsequent description will be made using a mobile terminal as an example. It will be understood by those skilled in the art that, in addition to components specifically used for mobile purposes, the configuration according to the embodiments of the present application can also be applied to fixed-type terminals.

[0075] Please refer to Figure 1, which is a schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present application. The mobile terminal 100 may include components such as an RF (Radio Frequency) unit 101, a WiFi module 102, an audio output unit 103, an A / V (Audio / Video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111. Those skilled in the art will understand that the mobile terminal structure shown in Figure 1 does not limit the mobile terminal. The mobile terminal may include more or fewer components than shown, or may combine certain components, or arrange the components differently.

[0076] The following is a detailed introduction to the various components of the mobile terminal in conjunction with Figure 1:

[0077] The RF unit 101 can be used to send and receive information or receive signals during calls. Specifically, it receives downlink information from the base station and transmits it to the processor 110 for processing. It also transmits uplink data to the base station. Typically, the RF unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, and more. Furthermore, the RF unit 101 can communicate with the network and other devices via wireless communication. The above-mentioned wireless communications can use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution), 5G and 6G, etc.

[0078] WiFi is a short-range wireless transmission technology. A mobile terminal, through WiFi module 102, enables users to send and receive emails, browse web pages, and access streaming media, providing wireless broadband Internet access. Although FIG1 illustrates WiFi module 102, it is understood that it is not a required component of the mobile terminal and can be omitted as needed without altering the essence of the invention.

[0079] The audio output unit 103 can convert audio data received by the RF unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output it as sound when the mobile terminal 100 is in a call signal reception mode, a talk mode, a recording mode, a voice recognition mode, a broadcast reception mode, or the like. Furthermore, the audio output unit 103 can also provide audio output related to a specific function performed by the mobile terminal 100 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 103 may include a speaker, a buzzer, or the like.

[0080] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos captured by an image capture device (e.g., a camera) in video capture mode or image capture mode. The processed image frames may be displayed on the display unit 106. The image frames processed by the GPU 1041 may be stored in the memory 109 (or other storage medium) or transmitted via the RF unit 101 or the WiFi module 102. The microphone 1042 may receive sound (audio data) in operating modes such as a phone call mode, a recording mode, and a voice recognition mode, and may process such sound into audio data. In the phone call mode, the processed audio (voice) data may be converted into a format that can be transmitted to a mobile communication base station via the RF unit 101. The microphone 1042 may implement various types of noise cancellation (or suppression) algorithms to eliminate (or suppress) noise or interference generated during the reception and transmission of audio signals.

[0081] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Optionally, the light sensor includes an ambient light sensor and a proximity sensor. Optionally, the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 1061 and / or the backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that recognize the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be described here.

[0082] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. The user input unit 107 may be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile terminal. Optionally, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also known as a touch screen, can collect user touch operations on or near it (such as operations performed by the user using any suitable object or accessory such as a finger, stylus, etc. on or near the touch panel 1071) and drive the corresponding connection device according to a pre-set program. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Optionally, the touch detection device detects the user's touch direction, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 110, and can receive the command sent by the processor 110 and execute it. In addition, the touch panel 1071 can be implemented using various types such as resistive, capacitive, infrared and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may also include other input devices 1072. Optionally, other input devices 1072 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc., and the specific details are not limited here.

[0083] Optionally, the touch panel 1071 may overlay the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. The processor 110 then provides a corresponding visual output on the display panel 1061 based on the type of touch event. Although in FIG1 , the touch panel 1071 and the display panel 1061 are shown as two separate components to implement the input and output functions of the mobile terminal, in some embodiments, the touch panel 1071 and the display panel 1061 may be integrated to implement the input and output functions of the mobile terminal, which is not limited to this specific embodiment.

[0084] The interface unit 108 serves as an interface through which at least one external device can be connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and an external device.

[0085] Memory 109 can be used to store software programs and various data. Memory 109 may primarily include a program storage area and a data storage area. Optionally, the program storage area may store an operating system and at least one application required for a function (such as a sound playback function or an image playback function); the data storage area may store data generated based on the use of the mobile phone (such as audio data, a phone book, etc.). Furthermore, memory 109 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0086] Processor 110 is the control center of the mobile terminal, connecting all components of the mobile terminal using various interfaces and circuits. By running or executing software programs and / or modules stored in memory 109 and accessing data stored in memory 109, it executes various functions of the mobile terminal and processes data, thereby providing overall monitoring of the mobile terminal. Processor 110 may include one or more processing units; preferably, processor 110 may integrate an application processor and a modem processor. Optionally, the application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 110.

[0087] The mobile terminal 100 may also include a power supply 111 (such as a battery) for supplying power to various components. Preferably, the power supply 111 may be logically connected to the processor 110 through a power management system, thereby managing functions such as charging, discharging, and power consumption through the power management system.

[0088] Although not shown in FIG. 1 , the mobile terminal 100 may further include a Bluetooth module, etc., which will not be described in detail here.

[0089] To facilitate understanding of the embodiments of the present application, the communication network system on which the mobile terminal of the present application is based is described below.

[0090] Please refer to Figure 2, which is a communication network system architecture diagram provided in an embodiment of the present application. The communication network system is an LTE system based on universal mobile communication technology. The LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203, and an operator's IP service 204, which are sequentially connected in communication. Optionally, UE 201 can be the above-mentioned terminal 100, which will not be repeated here.

[0091] E-UTRAN 202 includes eNodeB 2021 and other eNodeBs 2022 . Optionally, eNodeB 2021 may be connected to other eNodeBs 2022 via a backhaul (eg, an X2 interface). eNodeB 2021 is connected to EPC 203 , and eNodeB 2021 may provide access from UE 201 to EPC 203 .

[0092] EPC 203 may include an MME (Mobility Management Entity) 2031, an HSS (Home Subscriber Server) 2032, other MMEs 2033, an SGW (Serving Gate Way) 2034, a PGW (PDN Gate Way) 2035, and a PCRF (Policy and Charging Rules Function) 2036. Optionally, MME 2031 is a control node that processes signaling between UE 201 and EPC 203, providing bearer and connection management. HSS 2032 provides registers for managing functions such as the Home Location Register (not shown) and stores user-specific information such as service features and data rates. All user data can be sent through SGW2034, PGW2035 can provide IP address allocation and other functions for UE 201, PCRF2036 is the policy and charging control policy decision point for service data flow and IP bearer resources, and it selects and provides available policy and charging control decisions for the policy and charging execution function unit (not shown in the figure).

[0093] The IP service 204 may include the Internet, an intranet, an IMS (IP Multimedia Subsystem), or other IP services.

[0094] Although the above introduction takes the LTE system as an example, those skilled in the art should know that this application is not only applicable to the LTE system, but can also be applied to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, 5G and future new network systems (such as 6G), etc., which are not limited here.

[0095] FIG3 is a schematic diagram of the hardware structure of a controller 140 provided in this application. The controller 140 includes a memory 1401 and a processor 1402. The memory 1401 is used to store program instructions, and the processor 1402 is used to call the program instructions in the memory 1401 to execute the steps performed by the controller in the first embodiment of the above method. The implementation principles and beneficial effects are similar and will not be repeated here.

[0096] Optionally, the controller further includes a communication interface 1403, which can be connected to the processor 1402 via a bus 1404. The processor 1402 can control the communication interface 1403 to implement the receiving and sending functions of the controller 140.

[0097] Figure 4 is a schematic diagram of the hardware structure of a network node 150 provided in this application. Network node 150 includes: a memory 1501 and a processor 1502. Memory 1501 is used to store program instructions, and processor 1502 is used to call the program instructions in memory 1501 to execute the steps performed by the first node in the first embodiment of the above method. The implementation principles and beneficial effects are similar and will not be repeated here.

[0098] Optionally, the controller further includes a communication interface 1503, which can be connected to the processor 1502 via a bus 1504. The processor 1502 can control the communication interface 1503 to implement the receiving and sending functions of the network node 150.

[0099] The integrated modules implemented in the form of software function modules can be stored in a computer-readable storage medium. The software function modules stored in a storage medium include a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute some of the steps of the methods of various embodiments of the present application.

[0100] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive solid state disk, SSD), etc.

[0101] Based on the above-mentioned mobile terminal hardware structure and communication network system, various embodiments of the present application are proposed.

[0102] First embodiment

[0103] 5 , which is a flow chart of an image processing method according to a first embodiment, the image processing method of the embodiment of the present application can be applied to a processing device, including step S10:

[0104] In step S10 , a target image block is determined according to at least one first filtered pixel.

[0105] In this embodiment, the processing device can be a smart terminal, such as a mobile phone, a computer, etc., or a server, such as a local server or a cloud server. In this embodiment and this application, the processing device is mainly described as a smart terminal.

[0106] Optionally, the technical solution of this embodiment can be applied to the fields of image coding and decoding, video coding and decoding, hardware video coding and decoding, dedicated circuit video coding and decoding, real-time video coding and decoding, and so on.

[0107] Optionally, for ease of understanding, a brief introduction to the encoding and decoding process is given: as shown in Figure 6, it includes modules such as general encoding control, transformation and quantization, intra-frame estimation, intra-frame prediction, motion compensation, motion estimation, inverse quantization and inverse transformation, filter control analysis, deblocking filtering and SAO filtering (i.e., loop filtering), entropy coding, and decoding frame buffer. Optionally, the motion compensation module can perform intra-frame / inter-frame selection to determine specific compensation. Optionally, when performing entropy coding, it is based on the general control data determined by the general encoding control module, the variable quantization coefficient determined by the transformation and quantization module, the intra-frame prediction data determined by the filter control analysis, and the motion data determined by the filter control and decoding frame buffer, thereby obtaining the encoding bit rate.

[0108] Optionally, the decoded video signal is outputted via a decoded frame buffer.

[0109] Optionally, the loop filter may include two branches: a deblocking filter and an LC-NNLF. The branch results are then fused and SAO (Sample Adaptive Offset) and ALF (Adaptive Loop Filter) processing is performed.

[0110] Optionally, to facilitate understanding, a brief introduction is given to at least one neural network used in loop filtering during the encoding and decoding process.

[0111] Optionally, a LC-NNLF (Low Complexity Neural Network based Loop Filter) structure design is introduced into NNVC.

[0112] Optionally, this neural network is used as the original neural network. As shown in FIG7 , the original neural network input includes at least one of an image block, a boundary strength, and a quantization parameter. For example, it may include an NxN YUV image block, a boundary strength BS (Boundary Strength), and a quantization parameter, such as a quantization step size QS (Qstep). Optionally, the network input layer of the original neural network includes a convolutional layer, which may be a 3x3 convolutional layer with a shape of 3x3x10xM, which is the location of the 3x3 conv 3x3x10xM in the figure. It may also include an activation function, such as a Leaky ReLu function.

[0113] Optionally, at least one hidden layer (i.e., n Hidden layers) is set in the original neural network, and the network architecture of each hidden layer can be the same or different, for example, it can include a 1x1 convolutional layer, such as the position of 1x1 conv 1x1xMxM in the figure. Optionally, it can also include an activation function, such as a LeaKy ReLu function. Optionally, it can also include a 1x1 convolutional layer, such as the position of 1x1 conv 1x1xMxK in the figure. Optionally, it can also include a 3x3 convolutional layer, such as the position of 3x3 conv 3x3xKxK in the figure.

[0114] Optionally, CP decomposition can be performed in the 3x3 convolutional layer, which includes four convolutional layers, namely:

[0115] A 1x1 convolutional layer with a shape of 1x1xKxR, such as the location of 1x1 conv 1x1xKxR in the figure;

[0116] The convolutional layer corresponding to the 3x1xRxR separable convolution is shaped like this, such as the location of 3x1 Sep conv 3x1xRxR in the figure;

[0117] The convolutional layer corresponding to the 1x3xRxR separable convolution is shaped like this, such as the location of 1x3 Sep conv 1x3xRxR in the figure;

[0118] The convolutional layer with a shape of 1x1xRxK convolution corresponds to the location of 1x1 conv 1x1xRxK in the figure.

[0119] Optionally, two 1x1 convolutional layers can be fused, for example, the corresponding position area of ​​1x1 conv 1x1xKxR in the figure and the corresponding position area of ​​1x1 conv 1x1xMxK in the figure are convolutionally fused to obtain a new 1x1 convolutional layer with a shape of 1x1xMxR.

[0120] Optionally, two 1x1 convolutional layers can be fused, for example, the position of 1x1 conv 1x1xRxK and the position of 1x1 conv 1x1xKxM in the figure are convolutionally fused to obtain another new 1x1 convolutional layer with a shape of 1x1xRxM.

[0121] Optionally, the network output layer connected to the hidden layer includes a convolution layer, which can be a 3x3 convolution layer, and its location can refer to the location of 3x3 conv 3x3xKxL in the figure.

[0122] Optionally, the output part includes at least one image block after filtering processing.

[0123] Alternatively, the boundary strength BS may be a metric for quantifying the sharpness or strength of edges between adjacent image blocks.

[0124] Optionally, the quantization step size QS may be a parameter for controlling quantization accuracy calculated according to a quantization parameter QP.

[0125] Optionally, the quantization step size can determine the degree of precision loss in the process of dividing the original signal (such as pixel values ​​or transform coefficients) from continuous values ​​into multiple value intervals. The quantization process maps continuous values ​​to a smaller number of discrete values, achieving data compression by discarding some details. The larger the quantization step size, the greater the difference between each quantization level, and therefore the more details are discarded, which leads to increased compression efficiency, but also introduces greater distortion, that is, reduced image quality. Conversely, the smaller the quantization step size, the more details are retained and the image quality is higher, but the required storage space and transmission bandwidth also increase accordingly.

[0126] Optionally, the input size of the NNVC filtering process is 144x144, including the current CTU (Coding Tree Unit) and 8 adjacent samples on each side. The luminance samples are interleaved into four 72x72 blocks before being used as input to the filtering process. The output tensor corresponds to the filtered CTU samples, organized into 64x64 blocks, including 4 luminance blocks and 2 chroma blocks. Optionally, QP (Quantizer Parameter) and boundary strength (BS) information are used as additional input information to the network.

[0127] Optionally, in the network architecture of the original neural network, the network input layer includes a 3x3 convolution layer and an activation function, which can accept samples with 10 channel inputs, and the output feature dimension is M=72. Optionally, the network architecture also includes n=11 hidden layers. For each hidden layer, the input features first pass through a 1x1 convolution layer, and the output feature dimension is M=72, and then pass through an activation function. After that, the features pass through a second 1x1 convolution layer, and the output feature dimension is reduced to k=24. Finally, it passes through a separable 3x3 convolution layer combined with CP decomposition. Optionally, the separable 3x3 convolution layer combined with CP decomposition can be composed of 4 convolution layers, and the rank of the decomposition is set to R:

[0128] The first layer consists of 1x1xKxR convolution;

[0129] The second layer consists of 3x1xRxR separable convolutions;

[0130] The third layer includes 1x3xRxR separable convolution;

[0131] The fourth layer consists of 1x1xRxK convolutions.

[0132] Optionally, the 1x1 convolutions of the first and fourth layers can be fused with adjacent 1x1 convolutions to reduce the complexity of LC-NNLF. The final output layer contains a 3x3 convolution layer that outputs filtered samples of 6 features for the final residual scaling, such as 4 luminance and 2 chrominance features.

[0133] Alternatively, the original neural network architecture has the following defects: after the weight decomposition in the CP decomposition step, sometimes an initial value with an abnormally large absolute value is generated, resulting in unstable training in the subsequent training phase of the network, which in turn causes the network to not converge. The number of channels in the original neural network is not flexible enough. The number of channels of the NNVC module must be a multiple of 16, but the number of channels of the network with the best performance may not meet the multiple of 16. In addition, since NNVC is used for filtering during the encoding and decoding process, NNVC has a large number of parameters, many operations, and high complexity.

[0134] Therefore, in this embodiment, it is not necessary to directly use a neural network for filtering processing, and the target image block can be directly determined based on the determined at least one first filtered pixel.

[0135] Optionally, the first filtered pixel may be a pixel that has been filtered.

[0136] Optionally, the target image block may be an image block that has been filtered. The target image block may include at least one first filtered pixel, or may include pixels associated with the at least one first filtered pixel.

[0137] Optionally, the processing device may be a decoding end. If at the decoding end, the target image block may be a decoded image block.

[0138] Optionally, the processing device may be an encoding end. If at the encoding end, the target image block may be an image block that has been filtered.

[0139] In this embodiment, by determining the target image block after filtering through the first filtering pixel, the need to use a neural network for filtering in the loop filtering stage can be avoided, the filtering effect of the target image block can be improved, and the efficiency of video encoding and / or decoding can be improved.

[0140] Second embodiment

[0141] Based on the first embodiment, a second embodiment is proposed.

[0142] In this embodiment, the method for determining at least one first filtering pixel includes at least one of the following methods 1 to 3:

[0143] Method 1: determining by searching in at least one filter lookup table according to at least one pixel to be filtered;

[0144] Optionally, step S10 includes: searching at least one filtering lookup table to determine at least one first filtering pixel according to at least one pixel to be filtered, and determining a target image block according to the at least one first filtering pixel.

[0145] Optionally, the pixel to be filtered may be a predicted pixel (e.g., a predicted pixel), a reconstructed pixel (e.g., a reconstructed pixel), or a reference pixel. Optionally, the predicted pixel, the reconstructed pixel, and / or the reference pixel may be an intra-frame pixel and / or an inter-frame pixel. The predicted pixel and / or the reconstructed pixel may be an intermediate value used or generated when generating a reference pixel, without limitation herein.

[0146] Optionally, this embodiment can provide a low-complexity neural network video compression (NNVC) method based on a lookup table method, which can convert a pre-trained neural network into a filter lookup table, and then determine the target image block after filtering according to the filter lookup table. In this way, the complexity of the neural network video compression method is further reduced while maintaining basically no performance loss. And compared with non-neural network video compression, it has better compression performance. And because it is performed by means of a filter lookup table, there is no need to use a GPU for inference, and it can be widely used in various devices such as mobile phones, TVs, and cameras.

[0147] Optionally, the filter lookup table is a lookup table, which is a commonly used accelerated calculation method in embedded systems. For a complex function or a series of calculations, if the output value of the calculation is placed in a lookup table, then all that needs to be done later is to retrieve the value without executing the calculation process. Therefore, the lookup table is effective when the calculation time is longer than the memory access time. Optionally, the filter lookup table can include at least one pixel value before filtering and at least one pixel value after filtering (such as the filtered pixel value of the first filtered pixel), as well as the corresponding relationship between the two.

[0148] Optionally, the filter lookup table may be a lookup table that includes filtered pixels. Optionally, the filter lookup table may include correspondences between pixels, correspondences between pixels to be filtered and filtered pixels, correspondences between characterization values ​​and filtered pixels, correspondences between intermediate values ​​and filtered pixels, correspondences between pixels to be filtered and intermediate values, and correspondences between pixel parameters and filtered pixels and / or pixels to be filtered. Optionally, each data unit in the filter lookup table is a filtered pixel value. Optionally, an index may be set in the filter lookup table so that a corresponding filtered pixel value can be determined by searching the filter lookup table based on the index.

[0149] Optionally, before using a filter lookup table to determine or obtain a target image block after filtering, the following four stages may be performed, and then at least one first filtering pixel is determined by searching in at least one filter lookup table based on at least one pixel to be filtered, and the target image block is determined based on the at least one first filtering pixel.

[0150] Optionally, the four stages may be:

[0151] (1) Training phase: training a neural network;

[0152] (2) performing bit depth sampling on the input pixel values ​​of the trained neural network, and converting the sampled values ​​and the output of the neural network into a filter lookup table;

[0153] (3) Fine-tune the filter lookup table to obtain all input and output results at full bit depth;

[0154] (4) Use the filter lookup table to perform inference testing, determine the index of the filter lookup table according to the pixel to be filtered, input the index into the filter lookup table for search, and obtain the filtered pixel after filtering.

[0155] Optionally, during the training phase of the neural network, a symmetric convolution kernel can be used to train the neural network, or an asymmetric convolution kernel can be used to train the neural network, and a corresponding filter lookup table can be constructed for each convolution kernel in the neural network, and the filter lookup table corresponding to each convolution kernel can be the same or different.

[0156] Optionally, when determining at least one pixel to be filtered (e.g., to be subjected to a filtering process (e.g., a loop filtering process), the at least one pixel to be filtered may be input into at least one filtering lookup table for search. Since the filtering lookup table includes a plurality of correspondences between pixels to be filtered and filtering pixels, a filtering pixel corresponding to the input at least one pixel to be filtered is searched in the at least one filtering lookup table and determined, and the filtered pixel is output as a first filtering pixel. A target image block is then determined based on the output at least one first filtering pixel, for example, a filtered target image block is constructed based on the at least one first filtering pixel.

[0157] In this embodiment, at least one first filtering pixel is determined by searching in at least one filtering lookup table based on at least one pixel to be filtered, and a target image block is determined based on the at least one first filtering pixel, thereby avoiding the phenomenon of excessively high complexity of filtering processing caused by directly using a neural network. In addition, a neural network can be used to construct a filtering lookup table, that is, the input and output of the neural network during filtering processing are stored in the filtering lookup table, so that when the filtering lookup table is used to filter the pixels to be filtered, a better filtering effect can be achieved. Compared with directly using a neural network for filtering processing, the complexity of filtering processing can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0158] Method 2: determining based on at least one first intermediate value and at least one filter lookup table;

[0159] Optionally, step S10 includes: determining or obtaining at least one first filtering pixel according to at least one first intermediate value and at least one filtering lookup table, and determining a target image block according to the at least one first filtering pixel.

[0160] Optionally, the first intermediate value may be data generated by an intermediate process during the filtering process, may be a filtered pixel generated after at least one filtering process, or may be other values. The intermediate process may be a process of performing pre-filtering processing on the pixel to be filtered (e.g., filtering processing using a neural network or a filter lookup table).

[0161] Alternatively, the pixel to be filtered may be input into a neural network for filtering, and the first intermediate value may be output. Alternatively, the at least one first intermediate value may be determined or obtained based on the pixel to be filtered and at least one filter lookup table. Alternatively, the at least one first intermediate value may be determined or obtained based on other rule-based methods.

[0162] Alternatively, an index corresponding to the at least one first intermediate value may be determined and input into at least one filter lookup table for lookup and determination, or at least one first filtered pixel corresponding to the at least one first intermediate value may be obtained. Specifically, the at least one first intermediate value may be filtered again using the at least one filter lookup table to obtain the at least one first filtered pixel.

[0163] Alternatively, a target image block may be constructed or determined based on at least one first filtered pixel. Alternatively, the target image block may include at least one first filtered pixel. Alternatively, some pixels in the target image block may be first filtered pixels, and another part of the pixels may be first intermediate values.

[0164] Alternatively, at least one pixel to be filtered in at least one image block may be input into at least one filter lookup table, and an image block after a first filtering process may be output. Alternatively, a pixel in the image block after the first filtering process may be used as a first intermediate value, and at least one first filtering pixel may be determined based on the at least one first intermediate value and the at least one filter lookup table, and a target image block may be determined based on the at least one first filtering pixel. Alternatively, the filtered pixels in the image block after the first filtering process may be input into a neural network or a filter lookup table for a second filtering process, and the number of times the second filtering process is performed may be one or more, to obtain a second filtered image block, and the pixels in the image block after the second filtering process may be used as a first intermediate value, and at least one first filtering pixel may be determined based on the at least one first intermediate value and the at least one filter lookup table, and a target image block may be determined based on the at least one first filtering pixel.

[0165] In this embodiment, by determining at least one first filtering pixel based on at least one first intermediate value and at least one filtering lookup table, and determining the target image block based on at least one first filtering pixel, it is possible to avoid the phenomenon of excessively high filtering processing complexity caused by directly using a neural network, and a neural network can be used to construct a filtering lookup table, that is, the input and output of the neural network during filtering processing are stored in the filtering lookup table, so that when the first intermediate value is filtered again using the filtering lookup table, a better filtering effect can be achieved. Compared with directly using a neural network for filtering processing, the complexity of the filtering processing can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0166] Method three: determining a first filtered pixel corresponding to at least one second intermediate value according to a neural network.

[0167] Optionally, step S10 includes: determining at least one first filtered pixel corresponding to the second intermediate value according to a neural network, and determining a target image block according to the at least one first filtered pixel.

[0168] Optionally, the second intermediate value may be data generated during an intermediate process of the filtering process, may be a filtered pixel generated after at least one filtering process, or may be another value. Optionally, the second intermediate value may be the same as or different from the first intermediate value.

[0169] Optionally, the neural network in this embodiment can be a neural network that performs pixel filtering. Optionally, the neural network can be a neural network based on a fully connected layer; a neural network based on a convolutional layer; a neural network based on a Transformer; a neural network based on a hybrid convolutional layer, a fully connected layer, and a Transformer, etc.

[0170] Alternatively, the pixel to be filtered may be input into a neural network, and the output may be a second intermediate value. Alternatively, the at least one second intermediate value may be determined or obtained based on the pixel to be filtered and at least one filter lookup table. Alternatively, the at least one second intermediate value may be determined or obtained based on other rule-based methods.

[0171] Optionally, at least one second intermediate value can be input into a neural network, which then filters the at least one second intermediate value to obtain at least one first filtered pixel as output. A target image block is then constructed or determined based on the at least one first filtered pixel. Optionally, the target image block can include at least one first filtered pixel. Optionally, some pixels in the target image block can be first filtered pixels, while another portion of pixels can be filtered pixels corresponding to the first intermediate value and / or filtered pixels corresponding to the second intermediate value.

[0172] Optionally, at least one pixel to be filtered in at least one image block may be input into at least one filter lookup table, and the image block after the first filtering process may be output. Optionally, the pixels in the image block after the first filtering process may be used as the second intermediate value. Alternatively, the filtered pixels in the image block after the first filtering process may be input into a neural network or a filter lookup table for re-filtering, and the number of re-filtering processes may be one or more. The image block after the second filtering process is obtained, and the pixels in the image block after the second filtering process are used as the second intermediate value. The at least one second intermediate value is input into a neural network for filtering, and the at least one first filtered pixel is output, and the target image block is determined based on the at least one first filtered pixel.

[0173] Optionally, the neural network used for each filtering process may be the same or different, and the filter lookup table used for each filtering process may be the same or different.

[0174] In this embodiment, a first filtered pixel corresponding to at least one second intermediate value is determined based on a neural network, and then a target image block is determined based on the at least one first filtered pixel. This allows the high performance of the neural network to be leveraged to perform a second filtering process on the second intermediate value, thereby improving the filtering effect corresponding to the target image block. Furthermore, because the neural network is not directly used to filter the pixels to be filtered in the image block, but rather the filtered second intermediate value is filtered again to obtain the corresponding first filtered pixel, the phenomenon of channel redundancy affecting the image quality enhancement effect during the filtering process is avoided. This improves the filtering effect compared to a method that directly performs a single filtering process using a neural network, thereby increasing the efficiency of video encoding and / or decoding.

[0175] Third embodiment

[0176] Based on the first embodiment or the second embodiment, the third embodiment is proposed.

[0177] In this embodiment, the image processing method includes at least one of the fourth to sixth methods:

[0178] Mode 4: at least one pixel to be filtered is in at least one image block;

[0179] Optionally, method 4 is combined with method 1, and the scheme of step S10 can be: determining at least one pixel to be filtered in at least one image block, searching and determining at least one first filtering pixel in at least one filtering lookup table based on the at least one pixel to be filtered, and determining the target image block based on the at least one first filtering pixel.

[0180] Optionally, some or all of the pixels to be filtered in at least one image block can be input into at least one filter lookup table for search to determine the filtered pixels corresponding to the pixels to be filtered in the filter lookup table, and output them as first filtered pixels. Then, a target image block is constructed or determined based on the at least one first filtered pixel, for example, the at least one first filtered pixel is placed in the image block according to the pixel position of the pixel to be filtered to obtain the target image block.

[0181] Optionally, method four is combined with method two, and the scheme of step S10 can be: determining at least one pixel to be filtered in at least one image block, determining at least one first intermediate value based on the at least one pixel to be filtered, determining at least one first filtering pixel based on the at least one first intermediate value and at least one filtering lookup table, and determining the target image block based on the at least one first filtering pixel.

[0182] Optionally, some or all of the pixels to be filtered in at least one image block can be determined, and a neural network, and / or a filter lookup table, and / or other methods can be used to process the pixels to be filtered to obtain at least one first intermediate value, and then the at least one first intermediate value is input into at least one filter lookup table for search to determine or obtain at least one first filtered pixel, and then the target image block is determined based on the at least one first filtered pixel.

[0183] Optionally, method four is combined with method three, and the scheme of step S10 can be: determining at least one pixel to be filtered in at least one image block, determining at least one second intermediate value based on the at least one pixel to be filtered, determining a first filtering pixel corresponding to the at least one second intermediate value based on a neural network, and determining a target image block based on the at least one first filtering pixel.

[0184] Optionally, some or all of the pixels to be filtered in at least one image block can be determined, and a neural network, and / or a filter lookup table, and / or other methods can be used to process the pixels to be filtered to obtain at least one second intermediate value, and then the at least one second intermediate value is input into the neural network, and the at least one second intermediate value is filtered again according to the neural network to determine or obtain at least one first filtered pixel, and then the target image block is determined based on the at least one first filtered pixel.

[0185] In this embodiment, by determining that at least one pixel to be filtered is in at least one image block, a filter lookup table can be used to filter the pixel to be filtered to obtain at least one first filtered pixel, and the target image block can be determined based on the at least one first filtered pixel, thereby avoiding the phenomenon of excessively high complexity of filtering processing caused by directly using a neural network. A neural network can be used to construct a filter lookup table, that is, the input and output of the neural network during filtering processing are stored in the filter lookup table, so that when the filter lookup table is used to filter the first intermediate value again, a better filtering effect can be achieved. Compared with directly using a neural network for filtering processing, the complexity of filtering processing can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0186] Method five, determining at least one first intermediate value corresponding to at least one pixel to be filtered according to a neural network;

[0187] Optionally, method five is combined with method one, and the scheme of step S10 can be: based on at least one pixel to be filtered, searching in at least one filter lookup table to determine at least one first filtering pixel, determining at least one first intermediate value corresponding to at least one pixel to be filtered based on a neural network, and determining a target image block based on at least one first filtering pixel and at least one first intermediate value.

[0188] Alternatively, some or all of the pixels to be filtered in at least one image block may be input into at least one filter lookup table for search, to determine and output filtered pixels corresponding to the inputted some or all of the pixels to be filtered, and the output filtered pixels may be used as the first filtered pixels. Alternatively, some or all of the pixels to be filtered in at least one image block may be input into a neural network, so that the neural network performs filtering processing on the inputted pixels to be filtered and outputs a first intermediate value.

[0189] Optionally, the target image block can be determined based on at least one first filtering pixel and at least one first intermediate value. For example, different pixel positions in the target image block are provided with first filtering pixels or first intermediate values. Optionally, for the same pixel position in the target image block, the filtering pixel actually located at that pixel position can be determined based on the first filtering pixel and the first intermediate value corresponding to that pixel position. For example, a mathematical calculation can be performed on the first filtering pixel and the first intermediate value to obtain the filtering pixel actually located at that pixel position. Alternatively, a filtering pixel can be selected from the first filtering pixel and the first intermediate value as the filtering pixel actually located at that pixel position based on a certain rule, such as a random selection rule.

[0190] Optionally, method five is combined with method two, and the scheme of step S10 can be: determining at least one first intermediate value corresponding to at least one pixel to be filtered based on a neural network, determining at least one first filtering pixel based on the at least one first intermediate value and at least one filtering lookup table, and determining the target image block based on the at least one first filtering pixel.

[0191] Optionally, at least one pixel to be filtered in at least one image block can be input into a neural network for filtering processing, and at least one first intermediate value can be output. The at least one first intermediate value can be input into at least one filter lookup table for search to determine the corresponding filtered pixel for output, and at least one first filtered pixel can be obtained. Then, the target image block after filtering processing can be constructed or determined based on the at least one first filtered pixel.

[0192] Optionally, method five is combined with method three, and the scheme of step S10 can be: determining at least one first intermediate value corresponding to at least one pixel to be filtered based on a neural network, determining at least one second intermediate value based on the at least one first intermediate value, determining a first filtering pixel corresponding to the at least one second intermediate value based on the neural network, and determining the target image block based on the first filtering pixel.

[0193] Optionally, at least one pixel to be filtered in at least one image block can be input into a neural network for a first filtering process to obtain an image block after the first filtering process, and the pixels in the image block are used as a first intermediate value. The at least one first intermediate value is then input into a neural network or a filter lookup table for filtering process to obtain at least one second intermediate value. Alternatively, the at least one first intermediate value can be subjected to other processing (such as weighting or deformation) to obtain a second intermediate value. The at least one second intermediate value is then input into a neural network for a second filtering process to output a first filtered pixel. The target image block is then constructed or determined based on the first filtered pixel.

[0194] In this embodiment, at least one first intermediate value corresponding to at least one pixel to be filtered is determined using a neural network, at least one first filtering pixel is determined based on the at least one first intermediate value and at least one filtering lookup table, and then a target image block is determined based on the at least one first filtering pixel. This allows for the use of both the neural network and the filtering lookup table for combined filtering processing, thereby improving the filtering effect corresponding to the target image block and, in turn, improving video encoding and / or decoding performance.

[0195] Method six: determining at least one second intermediate value by searching in at least one filtering lookup table according to at least one pixel to be filtered.

[0196] Optionally, method six is ​​combined with method one, and the scheme of step S10 can be: based on at least one pixel to be filtered, determine at least one second intermediate value in at least one filter lookup table; based on at least one second intermediate value, determine at least one first filter pixel in at least one filter lookup table; and determine the target image block based on at least one first filter pixel.

[0197] Alternatively, at least one pixel to be filtered in at least one image block may be input into at least one filter lookup table for search, to determine a corresponding filtered pixel, which is then output to obtain at least one second intermediate value. The at least one second intermediate value is then further input into the at least one filter lookup table for search, to output a corresponding first filtered pixel. A target image block is then constructed or determined based on the at least one first filtered pixel.

[0198] Optionally, in combination with Method 2, Step S10 may include: determining or obtaining at least one first intermediate value based on at least one pixel to be filtered; determining or obtaining at least one first filtered pixel based on the at least one first intermediate value; and determining or obtaining at least one second intermediate value based on the at least one pixel to be filtered in at least one filter lookup table; determining or obtaining at least one first filtered pixel based on the at least one second intermediate value; and determining or obtaining the target image block based on the at least one first filtered pixel. Optionally, the first filtered pixels at different pixel positions in the target image block may be determined or obtained using different methods.

[0199] Optionally, method six is ​​combined with method three, and the scheme of step S10 can be: based on at least one pixel to be filtered, determine at least one second intermediate value in at least one filter lookup table, determine the first filtering pixel corresponding to the at least one second intermediate value based on the neural network, and determine the target image block based on the at least one first filtering pixel.

[0200] In this embodiment, at least one second intermediate value is determined by searching at least one filter lookup table based on at least one pixel to be filtered. A first filter pixel corresponding to the at least second intermediate value can then be determined using a neural network, and a target image block can be determined based on the at least one first filter pixel. This allows for combined filtering using the neural network and the filter lookup table, improving the filtering effect corresponding to the target image block and, in turn, enhancing video encoding and / or decoding performance.

[0201] Fourth embodiment

[0202] Based on any of the above embodiments, a fourth embodiment is proposed.

[0203] In this embodiment, the first method, which is to determine by searching at least one to-be-filtered pixel in at least one filter lookup table, includes at least one of the following methods:

[0204] Method seven, determining a first pixel parameter according to at least one pixel to be filtered, and determining the first pixel parameter according to the first pixel parameter and at least one filter lookup table;

[0205] Optionally, the pixel parameter may include a value or data obtained by performing mathematical calculations, deformation, or combination on the pixel. The first pixel parameter may include a value or data obtained by performing mathematical calculations, deformation, or combination on the pixel to be filtered.

[0206] Optionally, at least one pixel to be filtered in at least one image block can be input into a mathematical formula or model, or a function for calculation to obtain a first pixel parameter. For example, the pixels of at least two reference blocks can be aggregated according to an aggregation function to obtain the first pixel parameter. A preset pixel value can be added or subtracted from the pixels of at least one reference block to obtain the first pixel parameter. A preset weight and the pixels of at least one reference block can be input into a weighted model, and the pixels of at least one reference block can be weighted according to the preset weight using the weighted model to obtain the first pixel parameter. Optionally, the first pixel parameter can be input into at least one filter lookup table to search for the corresponding filter pixel, and the corresponding filter pixel can be output to obtain at least one first filter pixel, and the target image block can be determined based on the at least one first filter pixel.

[0207] Optionally, the second method may include determining or obtaining at least one first filtered pixel based on pixel parameters determined by at least one first intermediate value and at least one filter lookup table. Optionally, the sixth method may include determining at least one second intermediate value by searching in at least one filter lookup table based on pixel parameters determined by at least one pixel to be filtered.

[0208] In this embodiment, by determining a first pixel parameter based on at least one pixel to be filtered, determining at least one first filtering pixel based on the first pixel parameter and at least one filtering lookup table, and then determining a target image block based on the at least one first filtering pixel, the problem of excessively complex filtering processing caused by directly using a neural network can be avoided. That is, by using a filtering lookup table for filtering, the complexity of the filtering processing can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0209] Method eight, determining at least one first characterization value based on at least one pixel to be filtered and at least one characterization lookup table, and determining based on at least one first characterization value and at least one filtering lookup table;

[0210] Alternatively, a representation value can be a numerical value used to describe or represent a characteristic or attribute of an object, phenomenon, or the like. It is often processed or selected to reflect key characteristics, such as using average grades to represent the learning level of a class. Alternatively, a representation value can be a sampled value. For example, for 256 pixel values ​​in the range [0, 1, 2, …, 255], the sampled values ​​after sampling with a sampling interval of 4 bits are [0, 16, 32, 48, 64, 80, 96, 112, 128, 144, 160, 176, 192, 208, 224, 240].

[0211] Optionally, the representation lookup table may be a lookup table including representation values ​​or a sampling lookup table. Optionally, the representation lookup table may include correspondences between pixels, correspondences between representation values ​​and non-representation values, or correspondences between representation values ​​and pixel values. Optionally, an index may be set in the representation lookup table so that a corresponding representation value can be determined by searching the representation lookup table based on the index.

[0212] Optionally, the at least one pixel to be filtered may be input into at least one representation lookup table for search to determine a representation value corresponding to the at least one pixel to be filtered, and the representation value may be output to obtain a first representation value. Alternatively, the at least one pixel to be filtered may be converted into an index and then input into the at least one representation lookup table for search to determine the first representation value corresponding to the at least one pixel to be filtered.

[0213] Optionally, the at least one first representation value may be input into at least one filter lookup table for search, and at least one first filtered pixel may be output. Optionally, the at least one first representation value may be converted into an index, which may then be input into at least one filter lookup table for search, and at least one first filtered pixel may be output. The target image block may be determined based on the at least one first filtered pixel.

[0214] Optionally, at least one sampling value is determined based on at least one pixel to be filtered and at least one sampling lookup table, at least one first filtering pixel is determined based on at least one first sampling value and at least one filtering lookup table, and the target image block is determined based on the at least one first filtering pixel.

[0215] Optionally, the second method may include determining or obtaining at least one characterization value based on at least one first intermediate value and at least one characterization lookup table, and determining or obtaining at least one first filtered pixel based on the determined at least one characterization value and at least one filtering lookup table.

[0216] Optionally, the sixth method may include determining or obtaining at least one characterization value based on at least one pixel to be filtered and at least one characterization lookup table, and determining at least one second intermediate value by searching in at least one filtering lookup table according to the determined at least one characterization value.

[0217] In this embodiment, by determining at least one first representation value based on at least one pixel to be filtered and at least one representation lookup table, determining at least one first filtering pixel based on the at least one first representation value and at least one filtering lookup table, and determining a target image block based on the at least one first filtering pixel, the problem of excessively complex filtering processing caused by directly using a neural network can be avoided. That is, by using a filtering lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0218] Method nine, determining a second pixel parameter based on at least one pixel to be filtered, determining at least one second characterization value based on the second pixel parameter and at least one characterization lookup table, and determining based on the at least one second characterization value and at least one filtering lookup table;

[0219] Optionally, the second pixel parameter may include a value or data obtained by performing mathematical calculations, deformation, or combination on the pixel to be filtered. Optionally, the second pixel parameter may be the same as or different from the first pixel parameter.

[0220] Alternatively, the second pixel parameter may be input into at least one representation lookup table for lookup to determine a representation value corresponding to the second pixel parameter and output the resultant value to obtain at least one second representation value. The second representation value may be input into at least one filter lookup table for lookup to determine a corresponding filtered pixel to obtain at least one first filtered pixel, and the target image block may be determined based on the at least one first filtered pixel.

[0221] Optionally, a second pixel parameter is determined based on at least one pixel to be filtered, at least one second sampling lookup table is determined based on the second pixel parameter and at least one sampling lookup table, at least one first filtering pixel is determined based on the at least one second sampling lookup table and at least one filtering lookup table, and the target image block is determined based on the at least one first filtering pixel.

[0222] Optionally, the second method may include determining pixel parameters based on at least one first intermediate value and at least one characterization value based on at least one characterization lookup table, and determining at least one first filtered pixel based on the at least one characterization value and at least one filtering lookup table.

[0223] Optionally, the sixth method may include determining at least one characterization value based on pixel parameters determined for at least one pixel to be filtered and at least one characterization lookup table, and determining at least one second intermediate value by searching the at least one filtering lookup table based on the at least one characterization value.

[0224] In this embodiment, by determining a second pixel parameter based on at least one pixel to be filtered, determining at least one second representation value based on the second pixel parameter and at least one representation lookup table, determining at least one first filtering pixel based on the at least one second representation value and at least one filtering lookup table, and determining a target image block based on the at least one first filtering pixel, the problem of excessively complex filtering processing caused by directly using a neural network can be avoided. That is, by using a filtering lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0225] Method 10: determining at least one third characterization value based on at least one pixel to be filtered and at least one characterization lookup table, determining a third pixel parameter based on the at least one third characterization value, and determining based on the third pixel parameter and at least one filtering lookup table;

[0226] Optionally, the third pixel parameter may be the same as or different from the first pixel parameter and / or the second pixel parameter.

[0227] Optionally, at least one pixel to be filtered in at least one image block can be input into at least one representation lookup table for search, to determine and output a representation value corresponding to the at least one pixel to be filtered, thereby obtaining at least one third representation value. The third representation value can then be processed to obtain a third pixel parameter, for example, by performing a mathematical calculation, transforming, or combining the third representation value to obtain a value or data as the third pixel parameter. Optionally, the third pixel parameter can be input into at least one filter lookup table for search, to determine and output a corresponding filter pixel, thereby obtaining at least one first filter pixel, and determining the target image block based on the at least one first filter pixel.

[0228] Optionally, at least one third sampling value is determined based on at least one pixel to be filtered and at least one sampling lookup table, a third pixel parameter is determined based on the at least one third sampling value, at least one first filtering pixel is determined based on the third pixel parameter and at least one filtering lookup table, and a target image block is determined based on the at least one first filtering pixel.

[0229] Optionally, the second method may include determining at least one characterization value based on at least one first intermediate value and at least one characterization lookup table, determining pixel parameters based on the at least one characterization value, and determining at least one first filtered pixel based on the pixel parameters and at least one filtering lookup table.

[0230] Optionally, the sixth method may include determining at least one characterization value based on at least one pixel to be filtered and at least one characterization lookup table, determining a pixel parameter based on the at least one characterization value, and determining at least one second intermediate value by searching in at least one filtering lookup table based on the pixel parameter.

[0231] In this embodiment, by determining at least one third representation value based on at least one pixel to be filtered and at least one representation lookup table, determining a third pixel parameter based on the at least third representation value, determining at least one first filtering pixel based on the third pixel parameter and at least one filtering lookup table, and determining a target image block based on the at least one first filtering pixel, the problem of excessively complex filtering processing caused by directly using a neural network can be avoided. That is, by using a filtering lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0232] Method eleven, determining at least one fourth representation value according to the most significant bit of at least one pixel to be filtered, and determining the value according to the at least one fourth representation value and at least one filter lookup table;

[0233] Optionally, the most significant bit can be a valid bit in the first preset bit of the pixel value of the pixel when it is expressed in binary code. The first preset bit can be the most significant bit, the most significant four bits, etc., and can be set according to user needs. For example, for an 8-bit pixel, the first four significant bits of the pixel value of the pixel can be used as the most significant bit. If there is a pixel 1=36 (00100100), the most significant bit of pixel 1 is 2 (i.e., 0010).

[0234] Optionally, the most significant bits of at least one pixel to be filtered in at least one image block may be determined, and at least one representation value may be determined based on the most significant bits. For example, the most significant bits of each of the at least one pixel to be filtered may be processed to obtain at least one fourth representation value. For example, the most significant bits of the pixel to be filtered may be directly used as the fourth representation value. Alternatively, the most significant bits may be transformed, weighted, or otherwise processed to obtain the fourth representation value.

[0235] Optionally, the at least one fourth representation value may be input into at least one filter lookup table for search to determine a corresponding filter pixel, which is then output to obtain at least one first filter pixel. Alternatively, the at least one fourth representation value may be processed or transformed before being input into at least one filter lookup table for search, which is then output to obtain at least one first filter pixel, and the target image block is determined based on the at least one first filter pixel.

[0236] Optionally, at least one fourth sampling value is determined according to the high-order significant bit of at least one pixel to be filtered, at least one first filtering pixel is determined according to the at least one fourth sampling value and at least one filtering lookup table, and the target image block is determined according to the at least one first filtering pixel.

[0237] Optionally, the second approach may include determining at least one representation value according to a high-order significant bit of at least one first intermediate value, and determining at least one first filtered pixel according to the at least one representation value and at least one filter lookup table.

[0238] Optionally, the sixth method may include determining at least one characterization value according to a high-order significant bit of at least one pixel to be filtered, and determining at least one second intermediate value by searching at least one filter lookup table according to the at least one characterization value.

[0239] In this embodiment, by determining at least one fourth representation value based on the most significant bits of at least one pixel to be filtered, determining at least one first filtering pixel based on the at least one fourth representation value and at least one filtering lookup table, and determining a target image block based on the at least one first filtering pixel, the problem of excessively high filtering processing complexity caused by directly using a neural network can be avoided. That is, by using a filtering lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0240] Method 12: determining at least one fifth characterization value corresponding to at least one pixel to be filtered according to the characterization parameter, and determining the value according to the at least one fifth characterization value and at least one filter lookup table;

[0241] Optionally, the characterization parameter may be various parameters corresponding to the characterization value, such as the interval between characterization values ​​(ie, the characterization interval). Optionally, when the characterization value is a sampling value, the characterization parameter may be a sampling interval.

[0242] Optionally, the fifth characterization value can be determined based on at least one pixel to be filtered and a characterization parameter. The pixel value of a pixel separated from the pixel to be filtered by a characterization interval can be used as the fifth characterization value. The fifth characterization value can also be determined based on a multiple of the characterization interval and the pixel to be filtered. For example, the pixel value of a pixel separated from the pixel to be filtered by a preset multiple of the characterization interval can be used as the fifth characterization value.

[0243] Optionally, the at least one fifth representation value may be input into at least one filter lookup table for search to determine a corresponding filter pixel, which is then output to obtain at least one first filter pixel. Alternatively, the at least one fifth representation value may be processed or transformed before being input into at least one filter lookup table for search, which is then output to obtain at least one first filter pixel, and the target image block is determined based on the at least one first filter pixel.

[0244] Optionally, at least one fifth sampling value corresponding to at least one pixel to be filtered is determined according to the sampling parameters, at least one first filtering pixel is determined according to the at least one fifth sampling value and at least one filtering lookup table, and the target image block is determined according to the at least one first filtering pixel.

[0245] Optionally, the second approach may include determining at least one characterization value corresponding to at least one first intermediate value according to the characterization parameter, and determining at least one first filtered pixel according to the at least one characterization value and at least one filter lookup table.

[0246] Optionally, the sixth method may include determining a characterization value corresponding to at least one pixel to be filtered according to the characterization parameter, and determining at least one second intermediate value by searching in at least one filter lookup table according to the characterization value.

[0247] In this embodiment, by determining at least one fifth representation value corresponding to at least one pixel to be filtered based on the representation parameters, determining at least one first filtering pixel based on the at least one fifth representation value and at least one filtering lookup table, and determining the target image block based on the at least one first filtering pixel, the problem of excessively complex filtering processing caused by directly using a neural network can be avoided. That is, by using a filtering lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0248] Method 13, determining at least one index according to at least one pixel to be filtered, and searching and determining in at least one filtering lookup table according to the at least one index;

[0249] Optionally, at least one index may be determined by at least one pixel to be filtered in at least one image block. For example, at least one pixel to be filtered may be used as the index, or at least one index may be obtained by deforming or otherwise processing at least one pixel to be filtered.

[0250] Optionally, the index can be a number, an array, an identifier, a label, etc.

[0251] Optionally, after determining at least one index, the at least one index can be input into at least one filter lookup table for search to determine the corresponding filter pixel and output to obtain at least one first filter pixel, and the target image block is determined based on the at least one first filter pixel.

[0252] Optionally, the second method may include determining at least one index according to at least one first intermediate value, and determining at least one first filtered pixel by searching in a filter lookup table according to the at least one index.

[0253] Optionally, the sixth method may include searching and determining at least one second intermediate value in at least one filtering lookup table based on at least one index determined by at least one pixel to be filtered.

[0254] In this embodiment, by determining at least an index based on at least one pixel to be filtered, determining at least one first filtering pixel by searching in at least one filtering lookup table based on the at least one index, and determining a target image block based on the at least one first filtering pixel, it is possible to avoid the phenomenon of excessively high filtering processing complexity caused by directly using a neural network. That is, by using a filtering lookup table for filtering processing, the complexity of the filtering processing can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0255] A fourteenth method is to determine at least one non-representational value according to at least one pixel to be filtered, and the determination is based on the at least one non-representational value and a filtering lookup table.

[0256] Alternatively, the non-representational value may be data or pixel values ​​other than the representational value. For example, when the representational value is a sampled value, the non-representational value may be a non-sampled value. That is, if each pixel (e.g., the pixel to be filtered) is sampled according to a certain rule (e.g., a sampling interval) in at least one image block to obtain each sampled value, the non-sampled value is the pixel value that has not been sampled.

[0257] Optionally, at least one non-representational value can be determined or obtained using a certain rule based on at least one pixel to be filtered in at least one image block. At least one first filtering pixel can be determined based on the at least one non-representational value and a filtering lookup table. For example, the at least one non-representational value can be converted into a representative value, and then an index corresponding to the representative value can be determined. The index can be input into at least one filtering lookup table for search to determine the corresponding filtering pixel, which is then output to obtain at least one first filtering pixel. The target image block can then be determined based on the at least one first filtering pixel.

[0258] Optionally, at least one non-sampled value is determined according to at least one pixel to be filtered, at least one first filtered pixel is determined according to the at least one non-sampled value and a filter lookup table, and the target image block is determined according to the at least one first filtered pixel.

[0259] Optionally, the second method may include determining at least one non-representational value based on at least one first intermediate value and determining at least one first filtered pixel using at least one filter lookup table.

[0260] Optionally, the sixth method may include determining at least one second intermediate value by searching at least one filtering lookup table based on at least one non-representational value determined for at least one pixel to be filtered.

[0261] In this embodiment, by determining at least one non-representational value based on at least one pixel to be filtered, determining at least one first filtering pixel based on the at least one non-representational value and a filtering lookup table, and determining a target image block based on the at least one first filtering pixel, it is possible to avoid the phenomenon of excessively high filtering processing complexity caused by directly using a neural network. That is, by using a filtering lookup table for filtering processing, the complexity of the filtering processing can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0262] Fifth embodiment

[0263] Based on any of the above embodiments, a fifth embodiment is proposed.

[0264] In this embodiment, in the thirteenth method, determining at least one index according to at least one pixel to be filtered includes at least one of the fifteenth to the twenty methods:

[0265] Method 15, determining a fourth pixel parameter according to at least one pixel to be filtered, and determining at least one index according to the fourth pixel parameter;

[0266] Optionally, the concept of pixel parameters may refer to the seventh method in the above embodiment.

[0267] Optionally, the fourth pixel parameter may be the same as or different from any one of the first pixel parameter, the second pixel parameter and the third pixel parameter.

[0268] Optionally, at least one pixel to be filtered in at least one image block may be input into a mathematical formula, model, or function for calculation to obtain a fourth pixel parameter. For example, multiple pixels to be filtered may be input into an aggregation function for aggregation to obtain the fourth pixel parameter. Optionally, the fourth pixel parameter may be indexed, such as by directly using the fourth pixel parameter as an index, deforming or otherwise processing the fourth pixel parameter to obtain an index, or inputting the fourth pixel parameter into a table containing a mapping relationship between pixel parameters and indices to obtain an index corresponding to the fourth pixel parameter.

[0269] Optionally, at least one index determined according to the fourth pixel parameter may be input into at least one filter lookup table for lookup to determine at least one first filter pixel, and the target image block may be determined according to the at least one first filter pixel.

[0270] Optionally, for the second approach, pixel parameters may be determined based on at least one first intermediate value, at least one index may be determined based on the determined pixel parameters, and at least one first filtered pixel may be determined by searching in a filter lookup table based on the at least one index.

[0271] Optionally, for the sixth approach, at least one index may be determined based on a pixel parameter determined for at least one pixel to be filtered, and at least one second intermediate value may be determined by searching in at least one filter lookup table based on the at least one index.

[0272] In this embodiment, by determining a fourth pixel parameter based on at least one pixel to be filtered, determining at least one index based on the fourth pixel parameter, searching at least one filter lookup table based on the at least one index to determine at least one first filter pixel, and determining a target image block based on the at least one first filter pixel, the problem of excessively complex filtering processing caused by directly using a neural network can be avoided. That is, by using a filter lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0273] Method 16: determining at least one sixth representation value based on at least one pixel to be filtered and a representation lookup table, and determining at least one index based on the at least one sixth representation value;

[0274] Optionally, the concepts of the characterization value and the characterization lookup table may refer to the eighth embodiment described above.

[0275] Optionally, the at least one pixel to be filtered may be input into at least one representation lookup table for search to determine a representation value corresponding to the at least one pixel to be filtered, and the representation value may be output to obtain the sixth representation value. Alternatively, the at least one pixel to be filtered may be converted into an index that matches the representation lookup table and then input into the at least one representation lookup table for search to determine the sixth representation value corresponding to the at least one pixel to be filtered.

[0276] Optionally, the index can be determined based on the sixth characterization value, for example, the sixth characterization value can be directly used as the index, or the sixth characterization value can be deformed or otherwise processed to obtain the index, or the sixth characterization value can be input into a table containing a mapping relationship between characterization values ​​and indices to obtain the index corresponding to the sixth characterization value.

[0277] Optionally, at least one sixth sampling value is determined according to at least one pixel to be filtered and a sampling lookup table, and at least one index is determined according to the at least one sixth sampling value.

[0278] Optionally, at least one index may be input into at least one filter lookup table to perform a lookup to determine at least one first filter pixel, and the target image block may be determined according to the at least one first filter pixel.

[0279] Optionally, for the second method, at least one characterization value may be determined based on at least one first intermediate value and a characterization lookup table, at least one index may be determined based on the at least one characterization value, and at least one first filtered pixel may be determined by searching in a filtering lookup table based on the at least one index.

[0280] Optionally, for method six, at least one characterization value may be determined based on at least one pixel to be filtered and a characterization lookup table, at least one index may be determined based on the at least one characterization value, and at least one second intermediate value may be determined by searching in at least one filtering lookup table based on the at least one index.

[0281] In this embodiment, by determining at least one sixth representation value based on at least one pixel to be filtered and a representation lookup table, determining at least one index based on the at least one sixth representation value, searching at least one first filtering pixel in at least one filtering lookup table based on the at least one index, and determining a target image block based on the at least one first filtering pixel, the problem of excessively complex filtering processing caused by directly using a neural network can be avoided. That is, by using a filtering lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0282] Method 17, determining a fifth pixel parameter according to at least one pixel to be filtered, determining at least one seventh characterization value according to the fifth pixel parameter and a characterization lookup table, and determining at least one index according to the at least one seventh characterization value;

[0283] Optionally, the fifth pixel parameter may be a value or data obtained by performing mathematical calculations, deformation, or combination on the pixel to be filtered. Optionally, the fifth pixel parameter may be the same as or different from any one of the first to fourth pixel parameters.

[0284] Alternatively, the fifth pixel parameter may be input into at least one representation lookup table for lookup to determine a representation value corresponding to the fifth pixel parameter and output the resultant value to obtain at least one seventh representation value. Alternatively, the fifth pixel parameter may be converted into an index that matches the representation lookup table and then input into at least one representation lookup table for lookup to determine the seventh representation value corresponding to the fifth pixel parameter.

[0285] Optionally, the index can be determined based on the seventh characterization value, for example, the seventh characterization value can be directly used as the index, or the seventh characterization value can be transformed or otherwise processed to obtain the index, or the seventh characterization value can be input into a table containing a mapping relationship between characterization values ​​and indices to obtain the index corresponding to the seventh characterization value.

[0286] Optionally, a fifth pixel parameter is determined according to at least one pixel to be filtered, at least one seventh sampling value is determined according to the fifth pixel parameter and a characterization lookup table, and at least one index is determined according to the at least one seventh sampling value.

[0287] Optionally, at least one index may be input into at least one filter lookup table to perform a lookup to determine at least one first filter pixel, and the target image block may be determined according to the at least one first filter pixel.

[0288] Optionally, for method 2, a pixel parameter can be determined based on at least one first intermediate value, and at least one characterization value can be determined based on the pixel parameter and a characterization lookup table, and at least one index can be determined based on the at least one characterization value, and at least one first filtered pixel can be determined by searching in a filtering lookup table based on the at least one index.

[0289] Optionally, for method six, at least one characterization value can be determined based on pixel parameters determined for at least one pixel to be filtered and a characterization lookup table, and at least one index can be determined based on the at least one characterization value, and at least one second intermediate value can be determined by searching in at least one filtering lookup table based on the at least one index.

[0290] In this embodiment, by determining a fifth pixel parameter based on at least one pixel to be filtered, determining at least one seventh representation value based on the fifth pixel parameter and a representation lookup table, determining at least one index based on the at least one seventh representation value, and determining at least one first filtering pixel by searching at least one filtering lookup table based on the at least one index, and determining a target image block based on the at least one first filtering pixel, the problem of excessively complex filtering processing caused by directly using a neural network can be avoided. That is, by using a filtering lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0291] Method 18, determining at least one eighth characterization value based on at least one pixel to be filtered and a characterization lookup table, determining a sixth pixel parameter based on the at least one eighth characterization value, and determining at least one index based on the sixth pixel parameter;

[0292] Optionally, at least one pixel to be filtered in at least one image block may be input into at least one representation lookup table for search, to determine and output a representation value corresponding to the at least one pixel to be filtered, thereby obtaining at least one eighth representation value. Alternatively, the at least one pixel to be filtered may be converted into an index that matches the representation lookup table and then input into the at least one representation lookup table for search, thereby determining the eighth representation value.

[0293] Optionally, the eighth characterization value is processed to obtain the sixth pixel parameter, for example, the eighth characterization value is mathematically calculated, deformed, or combined to obtain a value or data as the sixth pixel parameter.

[0294] Optionally, the index can be determined based on the sixth pixel parameter, for example, by directly using the sixth pixel parameter as the index, or by deforming or otherwise processing the sixth pixel parameter to obtain the index, or by inputting the sixth pixel parameter into a table containing a mapping relationship between pixel parameters and indices to obtain an index corresponding to the sixth pixel parameter.

[0295] Optionally, at least one eighth sampling value is determined according to at least one pixel to be filtered and a sampling lookup table, a sixth pixel parameter is determined according to the at least one eighth sampling value, and at least one index is determined according to the sixth pixel parameter.

[0296] Optionally, at least one index may be input into at least one filter lookup table to perform a lookup to determine at least one first filter pixel, and the target image block may be determined according to the at least one first filter pixel.

[0297] Optionally, for method 2, at least one characterization value can be determined based on at least one first intermediate value and a characterization lookup table, a pixel parameter can be determined based on the at least one characterization value, and at least one index can be determined based on the pixel parameter, and at least one first filtered pixel can be determined by searching in a filtering lookup table based on the at least one index.

[0298] Optionally, for method six, at least one characterization value can be determined based on at least one pixel to be filtered and a characterization lookup table, a pixel parameter can be determined based on the at least one characterization value, and at least one index can be determined based on the pixel parameter, and at least one second intermediate value can be determined by searching in at least one filtering lookup table based on the at least one index.

[0299] In this embodiment, at least one eighth representation value is determined based on at least one pixel to be filtered and a representation lookup table, a sixth pixel parameter is determined based on the at least one eighth representation value, at least one index is determined based on the sixth pixel parameter, at least one first filtering pixel is determined based on the at least one index in at least one filtering lookup table, and a target image block is determined based on the first filtering pixel. This can avoid the problem of excessively complex filtering processing caused by directly using a neural network. That is, by using a filtering lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0300] Method 19, determining at least one ninth representation value according to the most significant bit of at least one pixel to be filtered, and determining at least one index according to the at least ninth representation value;

[0301] Optionally, the high-order significant bits may refer to the method eleven in the above embodiment.

[0302] Optionally, the most significant bits of at least one pixel to be filtered in at least one image block may be determined, and at least one representation value may be determined based on the most significant bits. For example, the most significant bits of each of the at least one pixel to be filtered may be processed to obtain at least one ninth representation value. For example, the most significant bits of the pixel to be filtered may be directly used as the ninth representation value. Alternatively, the ninth representation value may be obtained by deforming, weighting, or performing other processing on the most significant bits.

[0303] Optionally, the index can be determined based on at least one ninth characterization value, for example, the ninth characterization value can be directly used as the index, or the ninth characterization value can be deformed or otherwise processed to obtain the index, or the ninth characterization value can be input into a table containing a mapping relationship between characterization values ​​and indices to obtain the index corresponding to the ninth characterization value.

[0304] Optionally, at least a ninth sampling value is determined according to the most significant bits of at least one pixel to be filtered, and at least one index is determined according to the at least ninth sampling value.

[0305] Optionally, at least one index may be input into at least one filter lookup table to perform a lookup to determine at least one first filter pixel, and the target image block may be determined according to the at least one first filter pixel.

[0306] Optionally, for the second approach, at least one index may be determined based on at least one representation value determined based on the most significant bit of at least one first intermediate value, and at least one first filtered pixel may be determined by searching in a filtering lookup table based on the at least one index.

[0307] Optionally, for the sixth approach, at least one index may be determined based on at least one representation value determined based on the most significant bit of at least one pixel to be filtered, and at least one second intermediate value may be determined by searching in at least one filtering lookup table based on the at least one index.

[0308] In this embodiment, by determining at least one ninth representation value based on at least the most significant bit of a pixel to be filtered, determining at least one index based on the at least one ninth representation value, searching at least one first filtering pixel in at least one filtering lookup table based on the at least one index, and determining a target image block based on the at least one first filtering pixel, the problem of excessively high filtering processing complexity caused by directly using a neural network can be avoided. That is, by using a filtering lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0309] Method 20: Determine at least one tenth representation value corresponding to at least one pixel to be filtered according to the representation parameter, and determine at least one index according to the at least one tenth representation value.

[0310] Optionally, the characterization parameters may refer to the twelveth method in the above embodiment.

[0311] Optionally, the tenth characterization value can be determined based on at least one pixel to be filtered and a characterization parameter. The pixel value of a pixel separated from the pixel to be filtered by the characterization interval is used as the tenth characterization value. The tenth characterization value can also be determined based on a multiple of the characterization interval and the pixel to be filtered. For example, the pixel value of a pixel separated from the pixel to be filtered by a preset multiple of the characterization interval is used as the tenth characterization value.

[0312] Optionally, the index can be determined based on the tenth characterization value, for example, the tenth characterization value can be directly used as the index, or the tenth characterization value can be deformed or otherwise processed to obtain the index, or the tenth characterization value can be input into a table containing a mapping relationship between characterization values ​​and indices to obtain the index corresponding to the tenth characterization value.

[0313] Optionally, at least a tenth sampling value corresponding to at least one pixel to be filtered is determined according to the sampling parameters, and at least one index is determined according to the at least tenth sampling value.

[0314] Optionally, at least one index may be input into at least one filter lookup table to perform a lookup to determine at least one first filter pixel, and the target image block may be determined according to the at least one first filter pixel.

[0315] Optionally, for the second method, at least one characterization value corresponding to at least one first intermediate value may be determined based on the characterization parameter, at least one index may be determined based on the at least one characterization value, and at least one first filtered pixel may be determined by searching in a filter lookup table based on the at least one index.

[0316] Optionally, for method six, at least one characterization value corresponding to at least one pixel to be filtered may be determined based on the characterization parameter, at least one index may be determined based on the at least one characterization value, and at least one second intermediate value may be determined by searching in at least one filter lookup table based on the at least one index.

[0317] In this embodiment, by determining at least one tenth representation value corresponding to at least one pixel to be filtered based on the representation parameter, determining at least one index based on the at least one tenth representation value, searching at least one first filtering pixel in at least one filtering lookup table based on the at least one index, and determining a target image block based on the at least one first filtering pixel, the problem of excessively complex filtering processing caused by directly using a neural network can be avoided. That is, by using a filtering lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0318] Sixth embodiment

[0319] Based on any of the above embodiments, a sixth embodiment is proposed.

[0320] In this embodiment, in the thirteenth method, searching in at least one filter lookup table according to at least one index includes the twenty-first method:

[0321] Method 21: Search and determine in a multi-layer filter lookup table according to at least one index.

[0322] Optionally, the index in this embodiment may be the index in any of the above embodiments, which is not limited here.

[0323] Optionally, the at least one filter lookup table comprises a multi-layer filter lookup table. The multi-layer filter lookup table may comprise at least two layers of filter lookup tables.

[0324] Optionally, the filter lookup tables in different layers of the multi-layer filter lookup table may be the same or different.

[0325] Optionally, at least one index may be input into a first filter lookup table in a multi-layer filter lookup table for search until the last filter lookup table outputs at least one first filter pixel, and the target image block is determined based on the at least one first filter pixel.

[0326] Optionally, the characterization lookup table may also be similar to the filtering lookup table, that is, it may be a multi-layer characterization lookup table.

[0327] Alternatively, after determining or generating at least one index, such as a first index, based on at least one reference parameter, the first index may be input into a first-layer filter lookup table in a multi-layer filter lookup table for search, obtaining a first search result. A second index may then be determined or obtained based on the first search result, and the second index may be input into a second-layer filter lookup table in the multi-layer filter lookup table for search, until a final filter lookup table outputs a filtered pixel or a target image block containing the filtered pixel. Optionally, the at least one filter lookup table may include a multi-layer filter lookup table.

[0328] In this embodiment, by determining at least one index based on at least one pixel to be filtered, searching a multi-layer filter lookup table based on the at least one index to determine at least one first filter pixel, and determining a target image block based on the at least one first filter pixel, the problem of excessively high filtering processing complexity caused by directly using a neural network can be avoided. That is, by using a filter lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0329] Optionally, the image processing method further includes at least one of Method 22 to Method 25.

[0330] Method 22, searching in a multi-layer filter lookup table according to at least one index in a serial or parallel manner;

[0331] Mode 23, the input of the first layer filter lookup table in the multi-layer filter lookup table is at least one index;

[0332] Mode 24: The output of the last filter lookup table is at least one first filtered pixel;

[0333] Mode 25: The output of the previous filter lookup table of the multi-layer filter lookup table is the input of the next filter lookup table.

[0334] Optionally, the index in this embodiment may be the index in any of the above embodiments, which is not limited here.

[0335] Optionally, each layer of the filter lookup table in the multi-layer filter lookup table may be the same or different.

[0336] Optionally, the serial search method may be to search for an index in a multi-layer filter lookup table in sequence, and output a corresponding filtered pixel, ie, the first filtered pixel.

[0337] Optionally, the parallel search may be performed by searching in a multi-layer filter lookup table according to an index at the same time, and outputting a plurality of first filtered pixels.

[0338] Optionally, at least one index corresponding to at least one image block may be searched in a multi-layer filtering lookup table to obtain at least one first filtered pixel, and the target image block may be determined based on the at least one first filtered pixel.

[0339] Optionally, at least one index may be input into a multi-layer filter lookup table for search, and when searching for filtered pixels in the multi-layer filter lookup table, the search may be performed in parallel or serially. For example, at least two indexes may be simultaneously input into a first-layer filter lookup table for simultaneous search, and the output of the first-layer filter lookup table may be used as the input to a second-layer filter lookup table, until the output of the last-layer filter lookup table yields at least one first filtered pixel, and the target image block is then determined based on the at least one first filtered pixel.

[0340] In this embodiment, by performing a serial or parallel search in a multi-layer filter lookup table based on at least one index, different filtering methods, such as serial or parallel, can be used for different filtering scenarios when using the multi-layer filter lookup table for filtering, thereby improving filtering efficiency. Furthermore, when using the multi-layer filter lookup table for filtering, at least one index can be input into the first layer of the multi-layer filter lookup table for filtering. Furthermore, the multi-layer filter lookup table can be used for combined filtering, where the output of the previous layer of the filter lookup table serves as the input to the next layer of the filter lookup table, and the output of the last layer of the filter lookup table serves as at least one first filtered pixel. The target image block is then determined based on the at least one first filtered pixel. Using the multi-layer filter lookup table for combined filtering can improve filtering effectiveness, thereby increasing the efficiency of video encoding and / or decoding.

[0341] Seventh embodiment

[0342] Based on any of the above embodiments, a seventh embodiment is proposed.

[0343] In this embodiment, in the fourteenth method, the determination is made based on at least one non-representational value and a filter lookup table, including the twenty-sixth method and / or the twenty-seventh method:

[0344] Method 26: Determine by searching in a filter lookup table an eleventh characterization value corresponding to at least one non-characterization value.

[0345] Optionally, the non-characteristic value may refer to the fourteenth method in the above embodiment.

[0346] Optionally, at least one non-characterization value may be determined according to at least one pixel to be filtered, and a characterization value corresponding to the at least one non-characterization value may be determined according to a pre-set rule and used as the eleventh characterization value.

[0347] Alternatively, the eleventh representation value may be input into at least one filter lookup table for lookup to determine a corresponding first filter pixel, and the target image block may be determined or obtained based on the first filter pixel. Alternatively, an index corresponding to the eleventh representation value may be determined and input into a multi-layer lookup table for lookup to determine at least one first filter pixel, and the target image block may be determined or obtained based on the first filter pixel.

[0348] In this embodiment, at least one first filtered pixel is determined by searching a filter lookup table based on an eleventh representation value corresponding to at least one non-representation value, and a target image block is determined based on the at least one first filtered pixel. This avoids the problem of excessively high filtering processing complexity that would otherwise occur if a neural network were directly used. That is, by using a filter lookup table for filtering, filtering complexity can be reduced, thereby improving video encoding and / or decoding efficiency.

[0349] Method 27: Determine at least one fourth filtered pixel according to an eleventh representation value corresponding to at least one non-representation value and a filter lookup table, and determine according to the representation parameter, the at least one fourth filtered pixel, and an interpolation weight.

[0350] Optionally, the non-characterization value may refer to the method 14 in the above embodiment. The characterization parameter may refer to the method 12 in the above embodiment.

[0351] Optionally, at least one non-characterizing value can be determined based on at least one pixel to be filtered, and a characterizing value corresponding to the at least one non-characterizing value can be determined based on a pre-set rule and used as the eleventh characterizing value. For example, if the characterizing value is a sampled value and the non-characterizing value is a non-sampled value, the characterizing value closest to the at least one non-characterizing value pixel can be determined to be the seventh characterizing value. Alternatively, the at least one non-characterizing value can be processed based on a certain rule to obtain the at least seventh characterizing value.

[0352] Optionally, the interpolation weights may be set or determined in advance.

[0353] Optionally, a search may be performed in at least one filter lookup table according to the eleventh representation value to determine a corresponding filtered pixel and output the result to obtain at least one fourth filtered pixel.

[0354] Optionally, the characterization parameters, at least one fourth filter pixel and the interpolation weight may be input into a mathematical calculation formula (such as the following interpolation formula) to obtain a first filter pixel, and the target image block may be determined based on the first filter pixel.

[0355] Optionally, for non-representation values, an interpolation method can be used to determine the corresponding first filtering pixel. Optionally, if at least one pixel to be filtered includes I0 and I1. And the pixels to be filtered I0 and I1 are non-sampled values ​​(i.e., non-representation values), non-representation value I0 = 36 (00100100); non-representation value I1 = 54 (00110110). Then the eleventh representation value can be determined based on the high-order significant bits of the non-representation values ​​I0 and I1, and then the fourth filtering pixel corresponding to the eleventh representation value is determined according to the filter lookup table. For example, the high-order significant bits (such as the highest 4 significant bits) of the non-representation values ​​I0 and I1 are determined to be 2 (i.e., 0010) and 3 (i.e., 0011). Then the eleventh representation value is determined to be 2 and 3, and the pixel value of the fourth filtering pixel in the corresponding filter lookup table is determined, i.e., the pixel value corresponding to LUT [2] [3]. Optionally, LUT is a filter lookup table.

[0356] Alternatively, referring to FIG8 , two boundary vertices are positioned at P 00 =LUT[2][3] and P 11 =LUT[3][4], and the other vertex is compared with L x and L y Get, L x <L y , select P 01 =LUT[2][4], otherwise, select P 10 =LUT[3][3]. For the value V of the lookup table corresponding to I0 and I1, the interpolation formula is as follows: V=(ω0P 00 +ω1P 01 +ω2P 11 ) / W formula (1);

[0357] Optionally, V is the first filtering pixel, P 00 、P 01 、P 10 、P 11 is the vertex determined based on the fourth filtered pixel, ω0, ω1, ω2 are interpolation weights, and ω0=WL y ,ω1=L y -L x ,ω2=L x, W=2 4 is the sampling interval (i.e., characterization parameter).

[0358] Optionally, the interpolation weights may be determined based on the least significant bits of the non-representation values ​​I0 and I1. For example, the least significant bits of the non-representation values ​​I0 and I1 are L x =4 (i.e. 0100) and L y =6 (ie 0110). Therefore, as shown in FIG8 , when the non-representation value I0=36 and the non-representation value I1=54, the four values ​​determined based on the non-representation values ​​I0 and I1 can be at most P 00 =LUT[2][3];P 01 =LUT[2][4];P 10 =LUT[3][3];P 11 =LUT[3][4], interpolation weights ω0=10, ω1=2, ω2=4. Then, the above formula (1) is used to calculate and obtain the corresponding first filtered pixel.

[0359] Optionally, as shown in FIG9 , if it is necessary to perform an interpolation operation on three non-representational values ​​simultaneously to determine corresponding filtered pixels, the corresponding spatial cube may be divided into six non-overlapping tetrahedrons.

[0360] For the low-order significant bits of the non-representational values ​​I0, I1, and I2, for example, the lowest 4 significant bits are L x , L y , L z The different conditions of (I0, I1, I2) represent that (I0, I1, I2) are located in different tetrahedrons, and then according to L x , L y , L z Different interpolation weights are selected based on the value of . Interpolation operation can be performed according to the following formula (2) to determine the corresponding first filtered pixel.

[0361] Alternatively, the calculation can be performed by combining formula (II) and the following Table 1. i It can be any one of the interpolation weights ω0, ω1, ω2, and ω3. i It may be any one of the boundary vertices P1, P2, P3, and P0 determined based on the fourth filtered pixel.

[0362] Table 1

[0363] Optionally, P 000 、P 100 、P 110 、P 111 、P 101 、P 011 、P 001 and P010 They are all boundary vertices and can be determined or obtained based on different characterization values ​​determined by at least one non-characterization value.

[0364] In this embodiment, by determining at least one fourth list pixel based on an eleventh representation value corresponding to at least one non-representation value and a filter lookup table, determining at least one first filter pixel based on a representation parameter, at least one fourth filter pixel, and an interpolation weight, and determining a target image block based on the at least one first filter pixel, the problem of excessively high filtering processing complexity caused by directly using a neural network can be avoided. That is, by using a filter lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0365] Optionally, in Mode 26 or Mode 27, the method for determining the at least one eleventh characterization value includes at least one of Modes 28 to 30:

[0366] Mode 28, determining based on at least one non-representational value and a representational lookup table;

[0367] Alternatively, at least one non-characteristic value may be input into a characterization lookup table for lookup to determine a corresponding characterization value, which is then output to obtain an eleventh characterization value. Alternatively, the characterization lookup table may include a correspondence between pixels, such as a correspondence between non-characterization values ​​and characterization values.

[0368] Optionally, at least one non-representational value can be determined based on at least one pixel to be filtered, at least one eleventh representational value can be determined based on the at least one non-representational value and a representational lookup table, at least one first filtering pixel can be determined by searching the filtering lookup table based on the at least eleventh representational value, and the target image block can be determined based on the at least first filtering pixel.

[0369] Optionally, at least one non-representation value can be determined based on at least one pixel to be filtered, at least one eleventh representation value can be determined based on the at least one non-representation value and a representation lookup table, at least one fourth filtered pixel can be determined based on the at least one eleventh representation value and a filtering lookup table, at least one first filtered pixel can be determined based on the representation parameter, the at least fourth filtered pixel and an interpolation weight, and the target image block can be determined based on the at least first filtered pixel.

[0370] In this embodiment, by determining at least one eleventh representation value based on at least one non-representation value and a representation lookup table, then determining at least one first filtered pixel by searching a filter lookup table based on the at least eleventh representation value, or determining at least one fourth filtered pixel by searching a filter lookup table based on the at least eleventh representation value, then determining at least one first filtered pixel based on the representation parameter, the at least fourth filtered pixel, and an interpolation weight, and finally determining a target image block based on the at least first filtered pixel, the problem of excessively high filtering processing complexity caused by directly using a neural network can be avoided. That is, by using a filter lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0371] Mode 29, determining based on a high-order significant bit of at least one non-representational value;

[0372] Alternatively, the most significant bit of the at least one non-representation value may be directly used as the corresponding eleventh representation value. Alternatively, the most significant bit of the at least one non-representation value may be processed (such as weighted or other operations) to obtain the corresponding eleventh representation value.

[0373] Optionally, at least one non-representational value can be determined based on at least one pixel to be filtered, at least one eleventh representational value can be determined based on the most significant bits of the at least one non-representational value, at least one first filtering pixel can be determined by searching in a filtering lookup table based on the at least eleventh representational value, and the target image block can be determined based on the at least first filtering pixel.

[0374] Optionally, at least one non-representation value can be determined based on at least one pixel to be filtered, at least one eleventh representation value can be determined based on the most significant bit of the at least one non-representation value, at least one fourth filtered pixel can be determined based on the at least one eleventh representation value and a filtering lookup table, at least one first filtered pixel can be determined based on the representation parameter, the at least fourth filtered pixel and the interpolation weight, and the target image block can be determined based on the at least first filtered pixel.

[0375] In this embodiment, by determining at least a 11th representation value based on the most significant bits of at least one non-representation value, then determining at least a first filtered pixel based on the at least 11th representation value in a filter lookup table, or determining at least a fourth filtered pixel based on the at least 11th representation value in a filter lookup table, then determining at least a first filtered pixel based on the representation parameter, the at least fourth filtered pixel, and an interpolation weight, and then determining a target image block based on the at least first filtered pixel, the problem of excessively high filtering processing complexity caused by directly using a neural network can be avoided. That is, by using a filter lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0376] Method 30, determining an eleventh characterization value corresponding to at least one non-characterization value according to the characterization parameter;

[0377] Optionally, at least one non-characterizing value may be updated based on the characterizing parameter to obtain an eleventh characterizing value. For example, when the characterizing parameter is a sampling interval and the non-characterizing value is a non-sampled value, a multiple of the sampling interval may be added to the non-sampled value to obtain a sampled value, which is then used as the eleventh characterizing value.

[0378] Optionally, at least one non-representational value can be determined based on at least one pixel to be filtered, an eleventh representational value corresponding to the at least one non-representational value can be determined based on the representation parameter, at least one first filtering pixel can be determined by searching in a filtering lookup table based on the at least eleventh representational value, and a target image block can be determined based on the at least first filtering pixel.

[0379] Optionally, at least one non-representational value can be determined based on at least one pixel to be filtered, an eleventh representational value corresponding to the at least one non-representational value can be determined based on the representation parameter, at least one fourth filtered pixel can be determined based on the at least one eleventh representational value and a filter lookup table, at least one first filtered pixel can be determined based on the representation parameter, the at least one fourth filtered pixel and an interpolation weight, and the target image block can be determined based on the at least one first filtered pixel.

[0380] In this embodiment, by determining at least one eleventh representation value corresponding to at least one non-representation value based on the representation parameter, then determining at least one first filtered pixel by searching a filter lookup table based on the at least eleventh representation value, or determining at least one fourth filtered pixel based on the at least eleventh representation value, then determining at least one first filtered pixel based on the representation parameter, the at least fourth filtered pixel, and an interpolation weight, and then determining a target image block based on the at least first filtered pixel, the problem of excessively high filtering processing complexity caused by directly using a neural network can be avoided. That is, by using a filter lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0381] Optionally, in Method 27, the method for determining the interpolation weight includes at least one of Methods 31 to 33:

[0382] Method 31, determining based on at least one non-characteristic value and at least one adjacent characterizing value;

[0383] Alternatively, a characterization value adjacent to the image position of the non-characterization value may be determined and used as the adjacent characterization value. Alternatively, the pixel position corresponding to the non-characterization value and the pixel position corresponding to the adjacent characterization value may be adjacent.

[0384] Optionally, an interpolation weight can be obtained based on at least one adjacent characterization value and at least one non-characterization value. For example, when the adjacent characterization value is greater than the non-characterization value, a preset weight is set as the interpolation weight. When the non-characterization value is greater than the adjacent characterization value, another preset weight is set as the interpolation weight. Alternatively, the difference or absolute difference between at least one adjacent characterization value and at least one non-characterization value can be compared, and different interpolation weights can be determined based on the size of the difference or absolute difference.

[0385] Optionally, at least one non-characterization value may be determined based on at least one pixel to be filtered, at least one eleventh characterization value may be determined based on the at least one non-characterization value and a characterization lookup table, and / or at least one eleventh characterization value may be determined based on a high-order significant bit of the at least one non-characterization value, and / or the eleventh characterization value corresponding to the at least one non-characterization value may be determined based on a characterization parameter.

[0386] Optionally, at least one fourth filtered pixel is determined based on at least one eleventh representation value and a filter lookup table, at least one interpolation weight is determined based on at least one non-representation value and at least one neighboring representation value, at least one first filtered pixel is determined based on the representation parameter, the at least fourth filtered pixel, and the interpolation weight, and the target image block is determined based on the at least first filtered pixel.

[0387] Optionally, when determining the interpolation weight, the interpolation weight may be determined based on at least one non-characterization value and at least one non-adjacent characterization value, at least one characterization value across component blocks, at least one characterization value of a default block, at least one adjacent characterization value, at least one characterization value of a time-domain block, and at least one characterization value of a candidate block. The candidate block may be determined based on a motion vector and / or a block vector.

[0388] In this embodiment, by determining an interpolation weight based on at least one non-representational value and at least one neighboring representational value, and then determining at least one first filtered pixel based on the representation parameter, at least one fourth list pixel, and the interpolation weight, and then determining a target image block based on the at least one first filtered pixel, the problem of excessively complex filtering processing caused by directly using a neural network can be avoided. That is, by using a filter lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0389] Mode 32, determining based on the least significant bit of at least one non-representational value;

[0390] Alternatively, the less significant bits of the non-representational value may be directly used as interpolation weights, or the less significant bits of the non-representational value may be processed accordingly to obtain interpolation weights, such as by addition or subtraction, or by comparing the less significant bits with a preset threshold and determining the interpolation weights based on different comparison results.

[0391] Optionally, at least one non-characterization value may be determined based on at least one pixel to be filtered, at least one eleventh characterization value may be determined based on the at least one non-characterization value and a characterization lookup table, and / or at least one eleventh characterization value may be determined based on a high-order significant bit of the at least one non-characterization value, and / or the eleventh characterization value corresponding to the at least one non-characterization value may be determined based on a characterization parameter.

[0392] Optionally, at least one fourth filtering pixel is determined based on at least one eleventh representation value and a filtering lookup table, at least one interpolation weight is determined based on the least significant bit of at least one non-representation value, at least one first filtering pixel is determined based on the representation parameter, the at least one fourth filtering pixel, and the interpolation weight, and the target image block is determined based on the at least one first filtering pixel.

[0393] In this embodiment, by determining an interpolation weight based on the least significant bits of at least one non-representational value, further determining at least one first filtered pixel based on the representation parameter, at least one fourth list pixel, and the interpolation weight, and determining a target image block based on the at least one first filtered pixel, the problem of excessively complex filtering processing caused by directly using a neural network can be avoided. That is, by using a filter lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0394] Method 33: Determine based on characterization parameters.

[0395] Optionally, the characterization parameter can be directly used as the interpolation weight, or the characterization parameter can be processed accordingly to obtain the interpolation weight. For example, the characterization parameter is compared with a preset threshold value, and the interpolation weight is determined according to different comparison results. Optionally, the low-order significant bit of at least one non-characterization value can be determined based on the characterization parameter, such as determining the binary-coded low-order data of at least one non-characterization value based on the characterization parameter, and determining the low-order significant bit in the low-order data. The amount of low-order data can be consistent with the sampling interval. Optionally, the characterization parameter includes the sampling interval. For example, if the binary-coded non-characterization value is 36 (00100100) and the sampling interval is 4 bits, the low-order significant bit of the non-characterization value can be 4 (0100).

[0396] Optionally, the interpolation weight is determined or obtained according to the least significant bit of at least one non-representational value.

[0397] Optionally, at least one non-characterization value may be determined based on at least one pixel to be filtered, at least one eleventh characterization value may be determined based on the at least one non-characterization value and a characterization lookup table, and / or at least one eleventh characterization value may be determined based on a high-order significant bit of the at least one non-characterization value, and / or the eleventh characterization value corresponding to the at least one non-characterization value may be determined based on a characterization parameter.

[0398] Optionally, at least one fourth filtering pixel is determined based on at least one eleventh representation value and a filtering lookup table, at least one interpolation weight is determined based on the representation parameter, at least one first filtering pixel is determined based on the representation parameter, the at least one fourth filtering pixel and the interpolation weight, and the target image block is determined based on the at least one first filtering pixel.

[0399] Optionally, the non-representation value may be input into a table containing interpolation weights, and the corresponding interpolation weights may be output.

[0400] In this embodiment, by determining interpolation weights based on representation parameters, further determining at least one first filtered pixel based on the representation parameters, at least one fourth list pixel, and the interpolation weights, and determining a target image block based on the at least one first filtered pixel, the problem of excessively complex filtering processing caused by directly using a neural network can be avoided. That is, by using a filter lookup table for filtering, the complexity of the filtering process can be reduced, thereby improving the efficiency of video encoding and / or decoding.

[0401] Eighth embodiment

[0402] The present application also provides a processing device, referring to FIG10 , which includes:

[0403] The processing module A10 is configured to determine a target image block according to at least one first filtered pixel.

[0404] Optionally, a method for determining at least one first filtered pixel includes at least one of the following:

[0405] Determining by searching in at least one filter lookup table according to at least one pixel to be filtered;

[0406] Determined based on at least one first intermediate value and at least one filter lookup table;

[0407] A first filtered pixel corresponding to at least one second intermediate value is determined according to the neural network.

[0408] Optionally, the processing module A10 further includes at least one of the following:

[0409] At least one pixel to be filtered is in at least one image block;

[0410] Determining at least one first intermediate value corresponding to at least one pixel to be filtered according to a neural network;

[0411] At least one second intermediate value is determined by searching at least one filtering lookup table according to at least one pixel to be filtered.

[0412] Optionally, the step of searching at least one filter lookup table for at least one pixel to be filtered comprises at least one of the following:

[0413] Determining a first pixel parameter according to at least one pixel to be filtered, and determining the first pixel parameter according to the first pixel parameter and at least one filter lookup table;

[0414] Determining at least one first characterization value based on at least one pixel to be filtered and at least one characterization lookup table, and determining at least one first characterization value based on the at least one first characterization value and at least one filtering lookup table;

[0415] Determining a second pixel parameter based on at least one pixel to be filtered, determining at least one second characterization value based on the second pixel parameter and at least one characterization lookup table, and determining based on the at least one second characterization value and at least one filtering lookup table;

[0416] Determining at least one third characterization value based on at least one pixel to be filtered and at least one characterization lookup table, determining a third pixel parameter based on the at least one third characterization value, and determining based on the third pixel parameter and the at least one filtering lookup table;

[0417] Determining at least one fourth representation value according to the most significant bits of at least one pixel to be filtered, and determining according to the at least one fourth representation value and at least one filter lookup table;

[0418] Determining at least one fifth characterization value corresponding to at least one pixel to be filtered according to the characterization parameter, and determining the value according to the at least one fifth characterization value and at least one filter lookup table;

[0419] Determine at least one index according to at least one pixel to be filtered, and search and determine in at least one filtering lookup table according to the at least one index;

[0420] At least one non-representational value is determined according to at least one pixel to be filtered, and is determined according to the at least one non-representational value and a filtering lookup table.

[0421] Optionally, determining at least one index according to at least one pixel to be filtered includes at least one of the following:

[0422] determining a fourth pixel parameter according to at least one pixel to be filtered, and determining at least one index according to the fourth pixel parameter;

[0423] Determine at least one sixth representation value according to at least one pixel to be filtered and a representation lookup table, and determine at least one index according to the at least one sixth representation value;

[0424] Determine a fifth pixel parameter according to at least one pixel to be filtered, determine at least one seventh representation value according to the fifth pixel parameter and a representation lookup table, and determine at least one index according to the at least one seventh representation value;

[0425] Determining at least one eighth characterization value based on at least one pixel to be filtered and a characterization lookup table, determining a sixth pixel parameter based on the at least one eighth characterization value, and determining at least one index based on the sixth pixel parameter;

[0426] Determine at least one ninth representation value according to the most significant bit of at least one pixel to be filtered, and determine at least one index according to the at least ninth representation value;

[0427] At least one tenth representation value corresponding to at least one pixel to be filtered is determined according to the representation parameter, and at least one index is determined according to the at least one tenth representation value.

[0428] Optionally, searching and determining in at least one filter lookup table according to at least one index includes: searching and determining in a multi-layer filter lookup table according to at least one index.

[0429] Optionally, the processing module A10 further includes at least one of the following:

[0430] Searching in a multi-layer filter lookup table according to at least one index is performed in a serial or parallel manner;

[0431] The input of the first filter lookup table in the multi-layer filter lookup table is at least one index;

[0432] The output of the last layer of filter lookup table is at least one first filtered pixel;

[0433] The output of the previous filter lookup table of the multi-layer filter lookup table is the input of the next filter lookup table.

[0434] Optionally, determining based on at least one non-characteristic value and a filter lookup table includes at least one of the following:

[0435] searching and determining in a filter lookup table an eleventh characterization value corresponding to at least one non-characterization value;

[0436] At least one fourth filtered pixel is determined according to an eleventh representation value corresponding to the at least one non-representation value and a filter lookup table, and is determined according to the representation parameter, the at least one fourth filtered pixel, and an interpolation weight.

[0437] Optionally, a method for determining the at least one eleventh characterization value includes at least one of the following:

[0438] determining based on at least one non-characteristic value and a characterization lookup table;

[0439] Determined based on the most significant bit of at least one non-representational value;

[0440] An eleventh characterization value corresponding to the at least one non-characterization value is determined according to the characterization parameter.

[0441] Optionally, the interpolation weight is determined by at least one of the following methods:

[0442] Determined based on at least one non-characteristic value and at least one adjacent characterizing value;

[0443] Determined based on the least significant bit of at least one non-representational value;

[0444] Determined according to the characterization parameters.

[0445] The processing device provided in the embodiment of the present application has similar implementation principles and beneficial effects to the technical solutions shown in the above-mentioned corresponding method embodiments, and will not be described in detail here.

[0446] An embodiment of the present application further provides a processing device, including a memory and a processor. The memory stores an image processing program, and when the image processing program is executed by the processor, the steps of the image processing method in any of the above embodiments are implemented.

[0447] An embodiment of the present application further provides a storage medium on which an image processing program is stored. When the image processing program is executed by a processor, the steps of the image processing method in any of the above embodiments are implemented.

[0448] In the embodiments of the smart terminal and storage medium provided in this application, all technical features of any of the above-mentioned image processing method embodiments may be included. The expanded and explained contents of the specification are basically the same as those of the embodiments of the above-mentioned methods and will not be repeated here.

[0449] An embodiment of the present application further provides a computer program product, which includes computer program code. When the computer program code runs on a computer, the computer executes the methods in the various possible implementation modes described above.

[0450] An embodiment of the present application also provides a chip, including a memory and a processor, wherein the memory is used to store computer programs, and the processor is used to call and run the computer programs from the memory, so that a device equipped with the chip executes the methods in the various possible implementation modes as described above.

[0451] It is understood that the above scenarios are merely examples and do not limit the application scenarios of the technical solutions provided in the embodiments of this application. The technical solutions of this application can also be applied to other scenarios. For example, those skilled in the art will appreciate that with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application will also be applicable to similar technical problems.

[0452] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0453] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0454] The units in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0455] In this application, the same or similar terminology, technical solutions and / or application scenario descriptions are generally only described in detail the first time they appear. When they appear again later, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of this application, for the same or similar terminology, technical solutions and / or application scenario descriptions that are not described in detail later, you can refer to the previous relevant detailed descriptions.

[0456] In this application, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0457] The various technical features of the technical solution of this application can be combined arbitrarily. In order to make the description concise, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0458] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as mentioned above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, controlled terminal, or network device, etc.) to execute the method of each embodiment of the present application.

[0459] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a storage disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state storage disk Solid State Disk (SSD)).

[0460] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image processing method, characterized in that: include: A target image block is determined according to at least one first filtered pixel.

2. The method according to claim 1, characterized in that The method for determining at least one first filtering pixel includes at least one of the following: Determining by searching in at least one filter lookup table according to at least one pixel to be filtered; Determined based on at least one first intermediate value and at least one filter lookup table; A first filtering pixel corresponding to at least one second intermediate value is determined according to the neural network.

3. The method according to claim 2, characterized in that Also includes at least one of the following: At least one pixel to be filtered is in at least one image block; Determining at least one first intermediate value corresponding to at least one pixel to be filtered according to the neural network; At least one second intermediate value is determined by searching at least one filtering lookup table according to at least one pixel to be filtered.

4. The method according to claim 2, characterized in that The method is determined based on searching at least one to-be-filtered pixel in at least one filter lookup table, including at least one of the following: Determining a first pixel parameter according to at least one pixel to be filtered, and determining according to the first pixel parameter and at least one filter lookup table; Determine at least one first characterization value based on at least one pixel to be filtered and at least one characterization lookup table, and determine based on at least one first characterization value and at least one filtering lookup table; Determine a second pixel parameter according to at least one pixel to be filtered, determine at least one second characterization value according to the second pixel parameter and at least one characterization lookup table, and determine according to at least one second characterization value and at least one filtering lookup table; Determine at least one third characterization value based on at least one pixel to be filtered and at least one characterization lookup table, determine a third pixel parameter based on the at least one third characterization value, and determine based on the third pixel parameter and at least one filter lookup table; Determine at least one fourth characterization value according to the high-order significant bits of at least one pixel to be filtered, and determine according to the at least one fourth characterization value and at least one filter lookup table; Determining at least one fifth characterization value corresponding to at least one pixel to be filtered according to the characterization parameter, and determining according to the at least one fifth characterization value and at least one filter lookup table; Determine at least one index according to at least one pixel to be filtered, and determine by searching in at least one filter lookup table according to the at least one index; At least one non-characteristic value is determined according to at least one pixel to be filtered, and is determined according to the at least one non-characteristic value and a filter lookup table.

5. The method according to claim 4, characterized in that Determining at least one index according to at least one pixel to be filtered includes at least one of the following: Determine a fourth pixel parameter according to at least one pixel to be filtered, and determine at least one index according to the fourth pixel parameter; Determine at least one sixth characterization value according to at least one pixel to be filtered and the characterization lookup table, and determine at least one index according to the at least one sixth characterization value; Determine a fifth pixel parameter according to at least one pixel to be filtered, determine at least one seventh characterization value according to the fifth pixel parameter and a characterization lookup table, and determine at least one index according to the at least one seventh characterization value; Determine at least one eighth characterization value according to at least one pixel to be filtered and the characterization lookup table, determine a sixth pixel parameter according to the at least one eighth characterization value, and determine at least one index according to the sixth pixel parameter; Determine at least one ninth representation value according to the high-order significant bits of at least one pixel to be filtered, and determine at least one index according to the at least one ninth representation value; At least one tenth characterization value corresponding to at least one pixel to be filtered is determined according to the characterization parameter, and at least one index is determined according to the at least one tenth characterization value.

6. The method according to claim 4, characterized in that Searching and determining in at least one filter lookup table according to at least one index includes: searching and determining in a multi-layer filter lookup table according to at least one index.

7. The method according to claim 6, characterized in that Also includes at least one of the following: Searching in a multi-layer filter lookup table in a serial or parallel manner according to at least one index; An input of a first layer filter lookup table in the multi-layer filter lookup table is at least one index; The output of the last layer of filter lookup table is at least one first filtered pixel; The output of the previous filter lookup table of the multi-layer filter lookup table is the input of the next filter lookup table.

8. The method according to claim 4, characterized in that Determined based on at least one non-characteristic value and a filter lookup table, including at least one of the following: Determine by searching in a filter lookup table an eleventh characterization value corresponding to at least one non-characterization value; At least one fourth filtered pixel is determined according to an eleventh representation value corresponding to the at least one non-representation value and a filter lookup table, and is determined according to the representation parameter, the at least one fourth filtered pixel and an interpolation weight.

9. The method according to claim 8, characterized in that The method for determining at least one eleventh characterization value comprises at least one of the following: determining based on at least one non-characteristic value and a characterization lookup table; Determined based on the most significant bit of at least one non-representational value; An eleventh characterization value corresponding to the at least one non-characterization value is determined according to the characterization parameter.

10. The method according to claim 8, characterized in that The interpolation weight is determined by at least one of the following: Determined based on at least one non-characteristic value and at least one adjacent characterization value; Determined based on the least significant bit of at least one non-representational value; Determined according to the characterization parameters.

11. A processing device, characterized in that: include: A memory and a processor, wherein an image processing program is stored in the memory, and when the image processing program is executed by the processor, the steps of the image processing method according to claim 1 are implemented.

12. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the image processing method according to claim 1 are implemented.

Images