Image processing method, processing equipment and storage medium

Abstract

The invention provides an image processing method, processing equipment and a storage medium, and the image processing method comprises the steps: determining or generating a prediction block according to a derivative block corresponding to a reference block of at least one to-be-predicted block. Through the technical scheme of the invention, the prediction effect of prediction processing can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, processing device and storage medium. Background Art

[0002] Existing high-efficiency video coding frameworks, such as Neural Network Based Video Coding (NNVC) and / or Enhanced Compression Model (ECM), propose a video frame encoding technology to improve coding performance without significantly increasing computational complexity.

[0003] During the process of conceiving and implementing this application, the inventors discovered that there are at least the following problems: in the prediction processing stage during the encoding and decoding process, a neural network can be used for prediction processing. However, when using a neural network for prediction, the boundary overlapping pixel area information between two adjacent image blocks will be ignored, affecting the prediction effect.

[0004] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the Invention

[0005] In response to the above technical problems, the present application provides an image processing method, a processing device and a storage medium, aiming to solve the technical problem of how to improve the prediction effect of prediction processing.

[0006] The present application provides an image processing method, which can be applied to a processing device, comprising the steps of:

[0007] S1 , determining or generating a prediction block according to a derivative block corresponding to at least one reference block of a block to be predicted.

[0008] Optionally, the reference block of at least one block to be predicted includes a reference prediction block and / or a reference reconstruction block of the reference block.

[0009] Optionally, the reference block is determined or obtained according to at least one of the following:

[0010] At least one of the upper adjacent pixel, upper non-adjacent pixel, left adjacent pixel, left non-adjacent pixel, upper left adjacent pixel, upper left non-adjacent pixel, lower left adjacent pixel, lower left non-adjacent pixel, upper right adjacent pixel, and upper right non-adjacent pixel of the block to be predicted;

[0011] At least one of a neighbor block, a non-neighbor block, an inter-component block, a co-located block, a time domain block, and a default block corresponding to the block to be predicted;

[0012] At least one of the width, height, block size, and block area of ​​the block to be predicted;

[0013] A candidate block determined or generated by a candidate motion vector or a candidate block vector of the block to be predicted;

[0014] If the first information of the block to be predicted satisfies the first condition, the reference block is the first reference block;

[0015] If the first information of the block to be predicted meets the second condition, the reference block is the second reference block.

[0016] Optionally, the image processing method further includes at least one of the following:

[0017] The derivative block includes at least one of a luminance component derivative block, a chrominance component derivative block, and an inter-component derivative block;

[0018] The luminance component derivative block includes at least one of a first horizontal derivative block, a first vertical derivative block, and a first horizontal-vertical mixed derivative block;

[0019] The chroma component derivative block includes at least one of a second horizontal derivative block, a second vertical derivative block, and a second horizontal-vertical mixed derivative block;

[0020] The cross-component derivative block includes at least one of a third horizontal derivative block, a third vertical derivative block, and a third horizontal-vertical mixed derivative block.

[0021] Optionally, step S1 includes the steps of:

[0022] S11, determining or obtaining at least one derivative block according to a reference reconstructed block and / or a reference prediction block of at least one reference block;

[0023] S12, determining or generating a prediction block based on the neural network and / or the lookup table and at least one derivative block.

[0024] Optionally, size parameters of the derived block match size parameters of the reference reconstructed block and / or the reference prediction block.

[0025] Optionally, step S11 includes:

[0026] Cropping a partial area of ​​at least one reference reconstructed block and / or a reference prediction block;

[0027] At least one derivative block is determined or obtained according to a filling result of filling the at least one reference reconstructed block and / or the reference prediction block after being cropped.

[0028] Optionally, the partial area of ​​at least one reference reconstructed block and / or reference prediction block is determined or obtained according to at least one of the following:

[0029] at least one of at least one leftmost column, at least one rightmost column, at least one topmost row, and at least one bottommost row of at least one reference reconstructed block and / or reference prediction block;

[0030] Transform size parameters;

[0031] Crop size parameters;

[0032] The size of the sliding area in which the sliding window slides on at least one reconstructed block and / or prediction block and / or the preset sliding step size of the sliding window.

[0033] Optionally, the image processing method further includes at least one of the following:

[0034] The crop size parameter is less than or equal to the transform size parameter;

[0035] The transform size parameter includes at least one of a transform width and a transform height;

[0036] The transform width is greater than or equal to the number of columns for cropping and / or padding a partial area of ​​at least one reference reconstructed block and / or a reference prediction block;

[0037] The transformation height is greater than or equal to the number of rows for cropping and / or filling a partial area of ​​at least one reference reconstructed block and / or a reference prediction block.

[0038] Optionally, the filling method for filling the at least one reference reconstructed block and / or reference prediction block after being cropped includes at least one of the following:

[0039] Filling the leftmost or rightmost column of pixels of the at least one reference reconstructed block and / or reference prediction block after cropping with an even number of pixels;

[0040] The top or bottom of the at least one cropped reference reconstructed block and / or reference prediction block is padded with pixels in even rows.

[0041] Optionally, the image processing method further includes at least one of the following:

[0042] transforming at least one of the first horizontal derivative block, the first vertical derivative block, and the first horizontal and vertical mixed derivative block to obtain a first transformed feature;

[0043] transforming at least one of the second horizontal derivative block, the second vertical derivative block, and the second horizontal and vertical mixed derivative block to obtain a second transformed feature;

[0044] transforming at least one of the third horizontal derivative block, the third vertical derivative block, and the third horizontal and vertical mixed derivative block to obtain a third transformed feature;

[0045] The reconstructed image information and / or the predicted image information of at least one reference reconstructed block and / or reference predicted block is transformed to obtain a fourth transformed feature.

[0046] Optionally, step S1 includes at least one of the following:

[0047] Determine or obtain a first feature set based on the neural network and / or the lookup table and a result of channel splicing of the first transformed feature and the fourth transformed feature, and determine or generate a prediction block based on the first feature set;

[0048] Determine or obtain a first feature set based on the neural network and / or the lookup table and a result of channel splicing of the second transformed feature and the fourth transformed feature, and determine or generate a prediction block based on the first feature set;

[0049] Determine or obtain a first feature set based on the neural network and / or the lookup table and a result of channel splicing of the first transformation feature, the second transformation feature, and the fourth transformation feature, and determine or generate a prediction block based on the first feature set;

[0050] A first feature set is determined or obtained based on the neural network and / or the lookup table, and the result of channel splicing of the third transformation feature and the fourth transformation feature, and a prediction block is determined or generated based on the first feature set.

[0051] Optionally, determining or generating the prediction block according to the first feature set includes at least one of the following:

[0052] Convolving some features in the first feature set according to a convolution module of the neural network to obtain a first convolution feature, and determining or generating a prediction block based on a fusion result of fusing the first convolution feature with unconvolved features in the first feature set;

[0053] A first lookup feature is obtained by searching for some features in the first feature set according to the lookup table, and a prediction block is determined or generated according to a fusion result of the first lookup feature and features not searched in the first feature set.

[0054] The present application also provides a processing device, comprising:

[0055] The processing module is used to determine or generate a prediction block according to a derivative block corresponding to at least one reference block of a block to be predicted.

[0056] The present application also provides a processing device, comprising: a memory and a processor, wherein an image processing program is stored in the memory, and when the image processing program is executed by the processor, the steps of any of the above-mentioned image processing methods are implemented.

[0057] The present application also provides a storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-mentioned image processing methods.

[0058] As described above, the image processing method of the present application can be applied to a processing device and includes determining or generating a prediction block based on a derivative block of at least one reference block of the block to be predicted. The technical solution of the present application comprehensively considers the derivative blocks of the at least one reference block of the block to be predicted when determining or generating the prediction block, thereby improving the prediction effect of the prediction process. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for describing the embodiments. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without inventive work.

[0060] Figure 1 A schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present application;

[0061] Figure 2 A communication network system architecture diagram provided in an embodiment of the present application;

[0062] Figure 3 A schematic diagram of the hardware structure of a controller 140 provided in this application;

[0063] Figure 4 A schematic diagram of the hardware structure of a network node 150 provided in this application;

[0064] Figure 5 is a flowchart of an image processing method according to the first embodiment;

[0065] Figure 6 is a flowchart of an image processing method according to a third embodiment;

[0066] Figure 7 A schematic diagram of a horizontal derivative block provided for this application;

[0067] Figure 8 A schematic diagram of a vertical derivative block provided for this application;

[0068] Figure 9 A schematic diagram of a horizontal and vertical hybrid derivative block provided in this application;

[0069] Figure 10It is a schematic diagram of a processing module of a processing device.

[0070] The purpose of this application, its features, and advantages will be further described in conjunction with the embodiments and with reference to the accompanying drawings. The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and the accompanying text are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of this application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0071] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0072] It should be noted that, in this document, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0073] It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the term "if" as used herein may be interpreted as "when," "when," or "in response to a determination." Furthermore, as used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It should be further understood that the terms "comprising" and "including" indicate the presence of the described features, steps, operations, elements, components, items, types, and / or groups, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, types, and / or groups. The terms "or," "and / or," "including at least one of the following," etc., used herein, may be interpreted as inclusive, or mean any one or any combination. For example, “comprising at least one of the following: A, B, C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”; and for another example, “A, B or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”. An exception to this definition will occur only when a combination of elements, functions, steps or operations are inherently mutually exclusive in some manner.

[0074] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence in the order indicated by the arrows. Unless clearly stated herein, the execution of these steps is not strictly limited in order, and they can be performed in other orders. Moreover, at least a portion of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0075] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0076] It should be noted that in this article, step codes such as S11 and S12 are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the order. When implementing the step, those skilled in the art may execute S12 first and then S11, etc., but these should all be within the scope of protection of this application.

[0077] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0078] In the subsequent description, the use of suffixes such as "module", "component" or "unit" to represent elements is only for the purpose of facilitating the description of the present application and has no specific meaning. Therefore, "module", "component" or "unit" can be used interchangeably.

[0079] The processing device in this application can be a smart terminal or a server, etc., and the smart terminal can be implemented in various forms. For example, the smart terminal described in this application can include smart terminals such as mobile phones, tablet computers, laptop computers, PDAs, portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, etc., as well as fixed terminals such as digital TVs and desktop computers.

[0080] The subsequent description will be made by taking a mobile terminal as an example. It will be understood by those skilled in the art that, in addition to components specifically used for mobile purposes, the configuration according to the embodiments of the present application can also be applied to fixed-type terminals.

[0081] See also Figure 1 , which is a schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present application. The mobile terminal 100 may include: an RF (Radio Frequency) unit 101, a WiFi module 102, an audio output unit 103, an A / V (audio / video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111. Those skilled in the art will understand that Figure 1 The structure of the mobile terminal shown in the figure does not constitute a limitation to the mobile terminal. The mobile terminal may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0082] The following combination Figure 1 A detailed introduction to the various components of the mobile terminal:

[0083] The RF unit 101 can be used to send and receive information or receive signals during calls. Specifically, it receives downlink information from the base station and transmits it to the processor 110 for processing. It also transmits uplink data to the base station. Typically, the RF unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, and more. Furthermore, the RF unit 101 can communicate with the network and other devices via wireless communication. The above-mentioned wireless communications may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution), 5G and 6G, etc.

[0084] WiFi is a short-range wireless transmission technology. Mobile terminals can help users send and receive emails, browse web pages, and access streaming media through the WiFi module 102. It provides users with wireless broadband Internet access. Figure 1 The WiFi module 102 is shown, but it is understandable that it is not an essential component of the mobile terminal and can be omitted as needed without changing the essence of the invention.

[0085] The audio output unit 103 can convert audio data received by the RF unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output it as sound when the mobile terminal 100 is in a call signal reception mode, a talk mode, a recording mode, a voice recognition mode, a broadcast reception mode, or the like. Furthermore, the audio output unit 103 can also provide audio output related to a specific function performed by the mobile terminal 100 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 103 may include a speaker, a buzzer, or the like.

[0086] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos captured by an image capture device (e.g., a camera) in video capture mode or image capture mode. The processed image frames may be displayed on the display unit 106. The image frames processed by the GPU 1041 may be stored in the memory 109 (or other storage medium) or transmitted via the RF unit 101 or the WiFi module 102. The microphone 1042 may receive sound (audio data) in operating modes such as a phone call mode, a recording mode, and a voice recognition mode, and may process such sound into audio data. In the phone call mode, the processed audio (voice) data may be converted into a format that can be transmitted to a mobile communication base station via the RF unit 101. The microphone 1042 may implement various types of noise cancellation (or suppression) algorithms to eliminate (or suppress) noise or interference generated during the reception and transmission of audio signals.

[0087] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Optionally, the light sensor includes an ambient light sensor and a proximity sensor. Optionally, the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 1061 and / or the backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that recognize the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be described here.

[0088] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0089] The user input unit 107 can be used to receive input digital or character information, and generate key signal input related to user settings and function control of the mobile terminal. Optionally, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also known as a touch screen, can collect user touch operations on or near it (such as operations performed by the user using a finger, stylus, or any other suitable object or accessory on or near the touch panel 1071) and drive the corresponding connection device according to a pre-set program. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Optionally, the touch detection device detects the user's touch direction and detects the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device and converts it into touch point coordinates, which are then sent to the processor 110. It can also receive commands sent by the processor 110 and execute them. In addition, the touch panel 1071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may further include other input devices 1072. Optionally, the other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, power keys, etc.), a trackball, a mouse, a joystick, etc., and the specifics are not limited here.

[0090] Optionally, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. The processor 110 then provides a corresponding visual output on the display panel 1061 according to the type of touch event. Figure 1 In the embodiment, the touch panel 1071 and the display panel 1061 are two independent components to realize the input and output functions of the mobile terminal. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to realize the input and output functions of the mobile terminal, which is not limited here.

[0091] The interface unit 108 serves as an interface through which at least one external device can be connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and an external device.

[0092] Memory 109 can be used to store software programs and various data. Memory 109 may primarily include a program storage area and a data storage area. Optionally, the program storage area may store an operating system and at least one application required for a function (such as a sound playback function or an image playback function); the data storage area may store data generated based on the use of the mobile phone (such as audio data, a phone book, etc.). Furthermore, memory 109 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0093] Processor 110 is the control center of the mobile terminal, connecting all components of the mobile terminal using various interfaces and circuits. By running or executing software programs and / or modules stored in memory 109 and accessing data stored in memory 109, it executes various functions of the mobile terminal and processes data, thereby providing overall monitoring of the mobile terminal. Processor 110 may include one or more processing units; preferably, processor 110 may integrate an application processor and a modem processor. Optionally, the application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 110.

[0094] The mobile terminal 100 may also include a power supply 111 (such as a battery) for supplying power to various components. Preferably, the power supply 111 may be logically connected to the processor 110 through a power management system, thereby managing functions such as charging, discharging, and power consumption through the power management system.

[0095] although Figure 1 Not shown, the mobile terminal 100 may further include a Bluetooth module, etc., which will not be described in detail here.

[0096] To facilitate understanding of the embodiments of the present application, the communication network system on which the mobile terminal of the present application is based is described below.

[0097] See also Figure 2 , Figure 2 A communication network system architecture diagram is provided for an embodiment of the present application. The communication network system is an LTE system of universal mobile communication technology. The LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203 and an operator's IP service 204, which are connected in sequence.

[0098] Optionally, UE201 may be the above-mentioned terminal 100, which will not be described in detail here.

[0099] E-UTRAN 202 includes eNodeB 2021 and other eNodeBs 2022 . Optionally, eNodeB 2021 may be connected to other eNodeBs 2022 via a backhaul (eg, an X2 interface). eNodeB 2021 is connected to EPC 203 , and eNodeB 2021 may provide access from UE 201 to EPC 203 .

[0100] EPC 203 may include an MME (Mobility Management Entity) 2031, an HSS (Home Subscriber Server) 2032, other MMEs 2033, an SGW (Serving Gate Way) 2034, a PGW (PDN Gate Way) 2035, and a PCRF (Policy and Charging Rules Function) 2036. Optionally, MME 2031 is a control node that processes signaling between UE 201 and EPC 203, providing bearer and connection management. HSS 2032 provides registers for managing functions such as the Home Location Register (not shown) and stores user-specific information such as service features and data rates. All user data can be sent through SGW2034, PGW2035 can provide IP address allocation and other functions for UE 201, PCRF2036 is the policy and charging control policy decision point for service data flow and IP bearer resources, and it selects and provides available policy and charging control decisions for the policy and charging execution function unit (not shown in the figure).

[0101] The IP service 204 may include the Internet, an intranet, an IMS (IP Multimedia Subsystem), or other IP services.

[0102] Although the above introduction takes the LTE system as an example, those skilled in the art should know that this application is not only applicable to the LTE system, but can also be applied to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, 5G and future new network systems (such as 6G), etc., which are not limited here.

[0103] Figure 3This is a schematic diagram of the hardware structure of a controller 140 provided in this application. The controller 140 includes a memory 1401 and a processor 1402. The memory 1401 is used to store program instructions, and the processor 1402 is used to call the program instructions in the memory 1401 to execute the steps performed by the controller in the first embodiment of the above method. The implementation principles and beneficial effects are similar and will not be repeated here.

[0104] Optionally, the controller further includes a communication interface 1403, which can be connected to the processor 1402 via a bus 1404. The processor 1402 can control the communication interface 1403 to implement the receiving and sending functions of the controller 140.

[0105] Figure 4 This is a schematic diagram of the hardware structure of a network node 150 provided in this application. The network node 150 includes a memory 1501 and a processor 1502. The memory 1501 is used to store program instructions, and the processor 1502 is used to call the program instructions in the memory 1501 to execute the steps performed by the first node in the first embodiment of the above method. The implementation principles and beneficial effects are similar and will not be repeated here.

[0106] Optionally, the controller further includes a communication interface 1503, which can be connected to the processor 1502 via a bus 1504. The processor 1502 can control the communication interface 1503 to implement the receiving and sending functions of the network node 150.

[0107] The integrated modules implemented in the form of software function modules can be stored in a computer-readable storage medium. The software function modules stored in a storage medium include a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute some of the steps of the methods of various embodiments of the present application.

[0108] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive solid state disk, SSD), etc.

[0109] Based on the above-mentioned mobile terminal hardware structure and communication network system, various embodiments of the present application are proposed.

[0110] First embodiment

[0111] Reference Figure 5 , Figure 5 FIG. 1 is a flow chart of an image processing method according to a first embodiment. The image processing method according to the embodiment of the present application can be applied to a processing device, comprising step S1:

[0112] In step S1 , a prediction block is determined or generated according to a derivative block corresponding to at least one reference block of a block to be predicted.

[0113] In this embodiment, the processing device can be a smart terminal, such as a mobile phone, a computer, etc., or a server, such as a local server or a cloud server. In this embodiment and this application, the processing device is mainly described as a smart terminal.

[0114] Optionally, the technical solution of this embodiment can be applied to the fields of image coding and decoding, video coding and decoding, hardware video coding and decoding, dedicated circuit video coding and decoding, real-time video coding and decoding, and so on.

[0115] Optionally, the processing device may store various images and videos in advance and select an image to be predicted from each image as an image block, or segment the selected image and use the segmented image block as the image block to be predicted. Alternatively, a frame of image may be extracted from a video sequence as an image block, or the extracted frame of image may be segmented to obtain an image block. Alternatively, the processing device may receive an input image or video and extract a frame of image from the image or video as an image block, or segment the extracted frame of image to obtain an image block. Alternatively, the processing device may receive an image or video sent from another network device and extract a frame of image from the image or video as an image block, or segment the extracted frame of image to obtain an image block. In this case, the processing device may pre-establish a communication connection with a network device on the network side of the mobile communication system in which it is located. Thus, the network device can transmit the image or video to the terminal device via the communication connection, and the terminal device will then receive the image or video.

[0116] Optionally, the technical solution of this embodiment can be performed for intra-frame prediction or inter-frame prediction, without limitation herein.

[0117] Optionally, the reference block is an image block that has been predicted and / or reconstructed.

[0118] Optionally, the reference block of the at least one image block may be another image block used to assist in performing prediction processing on the at least one image block.

[0119] Optionally, the reference block may be a rectangular block, a non-rectangular block, an L-shaped block, or a T-shaped block.

[0120] Optionally, the reference block may include multiple rows and / or columns of pixels, or may include one row and / or one column of pixels.

[0121] Optionally, the reference block of at least one block to be predicted includes a reference prediction block and / or a reference reconstruction block of the reference block.

[0122] Optionally, the derivative block corresponding to the at least one reference block includes a derivative block corresponding to a reference prediction block and / or a derivative block corresponding to a reference reconstruction block.

[0123] Optionally, the reference block is determined or obtained according to at least one of the following methods 1 to 6:

[0124] Method 1: at least one of the upper adjacent pixel, upper non-adjacent pixel, left adjacent pixel, left non-adjacent pixel, upper left adjacent pixel, upper left non-adjacent pixel, lower left adjacent pixel, lower left non-adjacent pixel, upper right adjacent pixel, and upper right non-adjacent pixel of the block to be predicted;

[0125] Optionally, the pixel to be predicted in the block to be predicted may be used as the first pixel.

[0126] Optionally, the upper adjacent pixel may be a pixel in the same image that is located in the same image block obtained by final division together with the first pixel, and / or the pixel is located above the first pixel and adjacent to the first pixel.

[0127] Optionally, the upper non-adjacent pixel may be a pixel in the same image that is in the same image block obtained by finally dividing the same image as the first pixel. Although the pixel is located above the first pixel, the pixel is not adjacent to the first pixel.

[0128] Optionally, the left adjacent pixel may be a pixel in the same image that is located in the same image block obtained by final division together with the first pixel, and / or the pixel is located on the left side of the first pixel and adjacent to the first pixel.

[0129] Optionally, the left non-adjacent pixel may be a pixel in the same image that is in the same image block obtained by the final division as the first pixel. Although the pixel is located to the left of the first pixel, the pixel is not adjacent to the first pixel.

[0130] Optionally, the upper left adjacent pixel may be a pixel in the same image that is in the same image block obtained by final division together with the first pixel, and / or the pixel is located above and to the left of the first pixel and adjacent to the first pixel.

[0131] Optionally, the upper left non-adjacent pixel may be a pixel in the same image that is in the same image block obtained by the final division together with the first pixel. Although the pixel is located to the upper left of the first pixel, the pixel is not adjacent to the first pixel.

[0132] Optionally, the lower left adjacent pixel may be a pixel in the same image that is in the same image block obtained by final division as the first pixel, and / or the pixel is located below the left of the first pixel and adjacent to the first pixel.

[0133] Optionally, the lower left non-adjacent pixel may be a pixel in the same image that is in the same image block obtained by final division as the first pixel, and / or the pixel is located below the left of the first pixel and is not adjacent to the first pixel.

[0134] Optionally, the upper right adjacent pixel may be a pixel in the same image that is in the same image block obtained by final division as the first pixel, and / or the pixel is located above and to the right of the first pixel and adjacent to the first pixel.

[0135] Optionally, the upper right non-adjacent pixel may be a pixel in the same image that is in the same image block obtained by final division as the first pixel, and / or the pixel is located above and to the right of the first pixel and is not adjacent to the first pixel.

[0136] Optionally, at least one of the upper adjacent pixel, upper non-adjacent pixel, left adjacent pixel, left non-adjacent pixel, upper left adjacent pixel, upper left non-adjacent pixel, lower left adjacent pixel, lower left non-adjacent pixel, upper right adjacent pixel and upper right non-adjacent pixel can be a reconstructed pixel or a predicted pixel.

[0137] Optionally, at least one of the upper adjacent pixel, the upper non-adjacent pixel, the left adjacent pixel, the left non-adjacent pixel, the upper left adjacent pixel, the upper left non-adjacent pixel, the lower left adjacent pixel, the lower left non-adjacent pixel, the upper right adjacent pixel and the upper right non-adjacent pixel can be directly used as a pixel in the reference block, or the at least one acquired pixel can be deduced or calculated to obtain the pixel in the reference block, and / or the reference block can be determined or obtained based on the pixels in the reference block.

[0138] In this embodiment, a reference block is determined or generated based on at least one of the upper adjacent pixels, upper non-adjacent pixels, left adjacent pixels, left non-adjacent pixels, upper left adjacent pixels, upper left non-adjacent pixels, lower left adjacent pixels, lower left non-adjacent pixels, upper right adjacent pixels, and upper right non-adjacent pixels of the image block, and a prediction block is determined or generated based on a derivative block corresponding to at least one reference block. Specifically, when performing prediction processing, the corresponding derivative block can be determined based on an accurate and effective reference block, and prediction processing can be performed using the derivative block, thereby improving the prediction effect of the prediction processing.

[0139] Method 2: at least one of the neighboring blocks, non-neighboring blocks, cross-component blocks, co-located blocks, time domain blocks, and default blocks corresponding to the block to be predicted;

[0140] Optionally, the default block may be a block set in advance, for example, a block with typical pixel features pre-set by an encoder and / or a decoder.

[0141] Optionally, the neighbor block may be a block adjacent to the block to be predicted, and / or may be a block that has been predicted or reconstructed.

[0142] Optionally, the non-neighbor block may be a block that is not adjacent to the block to be predicted, and / or may be a block that has been predicted or reconstructed.

[0143] Optionally, the co-located block may be an image block in the co-located image, having the same position and size as the block to be predicted. Optionally, the co-located image may be an image in the reference image that is closest to the current image in terms of time.

[0144] Optionally, the time domain block can be a block distinguished from the time domain, such as an image block in the previous frame. For example, if there is video data containing three frames of images, the first frame of image is played in the first second, the second frame of image is played in the second second, and the third frame of image is played in the third second. If the image block predicted at the current moment (such as the block to be predicted) is the image block divided from the second frame of image, then the time domain block can be determined to be the image block corresponding to it in the first frame of image.

[0145] Optionally, the cross-component block may be an image block of a different component than the current at least one block to be predicted. For example, if the current at least one block to be predicted is an image block of the Y component, the cross-component block may be an image block of the U component and / or the V component. Optionally, if the current at least one block to be predicted is an image block of the U component, the cross-component block may be an image block of the Y component and / or the V component. Optionally, if the current at least one block to be predicted is an image block of the V component, the cross-component block may be an image block of the U component and / or the Y component.

[0146] Optionally, at least one pixel may be obtained from at least one of a neighboring block, a non-neighboring block, a cross-component block, a co-located block, a time-domain block, and a default block corresponding to the block to be predicted. The obtained at least one pixel may be used as a pixel in a reference block, or the obtained at least one pixel may be derived or calculated to obtain the pixel in the reference block. The reference block is determined or obtained based on the pixels in the reference block.

[0147] Alternatively, at least one of an upper adjacent pixel, an upper non-adjacent pixel, a left adjacent pixel, a left non-adjacent pixel, an upper left adjacent pixel, and an upper left non-adjacent pixel may be determined from at least one of a neighboring block, a non-neighboring block, a cross-component block, a co-located block, a time-domain block, and a default block corresponding to the block to be predicted, and used as a pixel of the reference block. Alternatively, an adjacent region and / or a non-adjacent region corresponding to the block to be predicted may be determined, and at least one pixel therefrom may be selected as a pixel of the reference block. The reference block is determined or obtained based on the pixels in the reference block.

[0148] In this embodiment, a reference block is determined or generated based on at least one of a neighboring block, a non-neighboring block, a cross-component block, a co-located block, a time domain block, and a default block corresponding to the block to be predicted, and a prediction block is determined or generated based on a derivative block corresponding to at least one reference block. Specifically, when performing prediction processing, the corresponding derivative block can be determined based on an accurate and effective reference block, and prediction processing can be performed using the derivative block, thereby improving the prediction effect of the prediction processing.

[0149] Method 3: at least one of the width, height, block size, and block area of ​​the block to be predicted;

[0150] Optionally, at least one reference block is determined or obtained according to at least one of width, height, block size and block area of ​​at least one block to be predicted.

[0151] Optionally, for example, if at least one of the width, height, block size and block area of ​​at least one block to be predicted is greater than a preset threshold, at least one image block is selected as a reference block from the neighboring blocks, non-neighboring blocks, cross-component blocks, co-located blocks, time domain blocks and default blocks corresponding to the at least one block to be predicted.

[0152] Optionally, at least one of the width, height, block size and block area of ​​at least one block to be predicted may be input into a neural network for determining a reference block, and a reference block may be output.

[0153] In this embodiment, a reference block is determined or generated based on at least one of the width, height, block size, and block area of ​​an image block, and a prediction block is determined or generated based on a derivative block corresponding to at least one reference block. Specifically, when performing prediction processing, the corresponding derivative block can be determined based on an accurate and effective reference block, and the prediction processing is performed using the derivative block, thereby improving the prediction effect of the prediction processing.

[0154] Method 4: a candidate block determined or generated by a candidate motion vector or a candidate block vector of a block to be predicted;

[0155] Optionally, the candidate motion vector or candidate block vector of the block to be predicted may include the motion vector or block vector corresponding to at least one of the neighboring blocks, non-neighboring blocks, cross-component blocks, co-located blocks, time domain blocks and default blocks corresponding to the block to be predicted, and may also include the motion vector or block vector corresponding to at least one of the upper adjacent pixels, upper non-adjacent pixels, left adjacent pixels, left non-adjacent pixels, upper left adjacent pixels and upper left non-adjacent pixels of the block to be predicted, and may also include the motion vector or block vector of the block to be predicted, etc. The following only takes the motion vector or block vector of the block to be predicted as an example.

[0156] Optionally, a block vector calculation is performed on the block to be predicted, and a candidate block corresponding to the block to be predicted is determined according to the block vector calculation result, for example, pixels corresponding to the block vector calculation result are used as pixels in the candidate block.

[0157] Optionally, a motion vector is calculated for the block to be predicted, and a candidate block corresponding to the block to be predicted is determined according to the motion vector calculation result, for example, pixels corresponding to the motion vector calculation result are used as pixels in the candidate block.

[0158] In this embodiment, a reference block is determined or generated by a candidate block determined or generated by a candidate motion vector or a candidate block vector of a block to be predicted, and a prediction block is determined or generated based on a derivative block corresponding to at least one reference block. Specifically, when performing prediction processing, the corresponding derivative block can be determined based on an accurate and effective reference block, and prediction processing can be performed using the derivative block, which can improve the prediction effect of the prediction processing.

[0159] Method 5: If the first information of the block to be predicted meets the first condition, the reference block is the first reference block;

[0160] Optionally, the first information of the block to be predicted may be at least one of the above-mentioned methods 1 to 7.

[0161] Optionally, satisfying the first condition may be any condition set in advance by the user, such as the width of the block to be predicted being greater than a preset width threshold, the height of the block to be predicted being greater than a preset height threshold, etc.

[0162] Optionally, the first reference block may be a reference block set and determined in advance, such as at least one of a neighbor block, a non-neighbor block, a cross-component block, a co-located block, a time domain block, a candidate block and a default block corresponding to the block to be predicted.

[0163] Optionally, when the first information of the block to be predicted meets the first condition, a first reference block of the reference block may be determined, and a prediction block may be determined or generated based on at least one derivative block corresponding to the first reference block of the block to be predicted.

[0164] In this embodiment, when the first information of the block to be predicted satisfies the first condition, the reference block is the first reference block, and the prediction block is determined or generated based on the derivative block corresponding to the first reference block. Specifically, when performing the prediction processing, the corresponding derivative block can be determined based on the accurate and effective reference block, and the prediction processing is performed using the derivative block, which can improve the prediction effect of the prediction processing.

[0165] Method six: if the first information of the block to be predicted meets the second condition, the reference block is the second reference block.

[0166] Optionally, satisfying the second condition may be any condition set in advance by the user, and / or may be different from the second condition. For example, if the first condition is that the width of the block to be predicted is greater than a preset width threshold, the second condition may be that the width of the block to be predicted is less than the preset width threshold.

[0167] Optionally, the second reference block may be a reference block set and determined in advance, for example, it may be a block other than the first reference block, including at least one of a neighbor block, a non-neighbor block, a cross-component block, a co-located block, a time domain block, a candidate block and a default block corresponding to the block to be predicted.

[0168] In this embodiment, when the first information of the block to be predicted satisfies the second condition, the reference block is the second reference block, and the prediction block is determined or generated based on the derivative block corresponding to the second reference block. Then, when performing the prediction processing, the corresponding derivative block can be determined based on the accurate and effective reference block, and the derivative block can be used to perform the prediction processing, which can improve the prediction effect of the prediction processing.

[0169] Optionally, in step S1 , the prediction block may be an image block that has undergone prediction processing.

[0170] Optionally, the prediction block may include at least one predicted pixel, and may also include pixels associated with the at least one predicted pixel.

[0171] Optionally, the predicted pixel may be a pixel predicted at the encoding side, which may be referred to as a predicted pixel, and the reconstructed pixel may be a pixel predicted at the decoding side, which may be referred to as a predicted pixel.

[0172] Optionally, the processing device may be a decoding end. If at the decoding end, the prediction block may be a decoded image block, and / or the prediction block may include reconstruction of pixels.

[0173] Optionally, the processing device may be an encoding end. If at the encoding end, the prediction block may be an image block that has undergone prediction processing, and / or the prediction block may include pixel predictions.

[0174] Optionally, for at least one block to be predicted, a derivative block corresponding to a reference block of at least one block to be predicted can be determined first, and a prediction module can be used in combination with the derivative block to predict the block to be predicted to determine or generate a prediction block. The prediction module can use a lookup table, a neural network, or a mathematical function, etc., which is not limited here.

[0175] In this embodiment, when determining or generating a prediction block, derivative blocks of at least one reference block of the block to be predicted are comprehensively considered, thereby improving the prediction effect of the prediction process.

[0176] Second embodiment

[0177] Based on the first embodiment, a second embodiment is proposed.

[0178] In this embodiment, the image processing method further includes at least one of the following methods 7 to 10:

[0179] Mode 7, the derivative block includes at least one of a luminance component derivative block, a chrominance component derivative block, and a cross-component derivative block;

[0180] Optionally, at least one reference block includes a reference prediction block of the reference block, and the derivative block of at least one reference prediction block includes a luminance component derivative block of the reference prediction block on the luminance component, a chrominance component derivative block of the reference prediction block on the chrominance component, and a cross-component derivative block of the reference prediction block. The cross-component derivative block may include derivative blocks of multiple components, for example, it may include at least one of the derivative blocks of the reference prediction block in the three components of Y component, U component and V component.

[0181] Optionally, at least one reference block includes a reference reconstruction block of the reference block, and the derivative block of at least one reference reconstruction block includes a luminance component derivative block of the reference reconstruction block on the luminance component, a chrominance component derivative block of the reference reconstruction block on the chrominance component, and a cross-component derivative block of the reference reconstruction block. The cross-component derivative block may include derivative blocks of multiple components, for example, it may include at least one of the derivative blocks of the reference reconstruction block in the three components of Y component, U component and V component.

[0182] Optionally, the reference prediction block and / or the reference reconstruction block may apply the chroma component derivative block of the chroma component to the luma component, and the reference prediction block and / or the reference reconstruction block may apply the luma component derivative block of the luma component to the chroma component.

[0183] In this embodiment, by determining that the derivative blocks corresponding to the reference prediction block and / or the reference reconstructed block of at least one reference block include at least one of a luminance component derivative block, a chrominance component derivative block and a cross-component derivative block, it is possible to obtain derivative blocks from different components. When determining or generating a prediction block, the derivative blocks of the reference block of at least one block to be predicted are comprehensively considered, which can improve the prediction effect of the prediction processing.

[0184] Mode 8, the luminance component derivative block includes at least one of a first horizontal derivative block, a first vertical derivative block, and a first horizontal and vertical mixed derivative block;

[0185] Optionally, corresponding derivative blocks may be obtained from different directions on the luminance component, such as the first horizontal derivative block in the horizontal direction, the first vertical derivative block in the vertical direction, and the first horizontal vertical mixed derivative block in the horizontal and vertical mixed directions.

[0186] Optionally, the luminance component derivative block of the reference prediction block may include a first horizontal derivative block of the reference prediction block in the horizontal direction on the luminance component, a first vertical derivative block in the vertical direction on the luminance component, and a first horizontal vertical mixed derivative block in the horizontal and vertical mixed directions on the luminance component.

[0187] Optionally, the luminance component derivative block of the reference reconstructed block may include a first horizontal derivative block of the reference reconstructed block in the horizontal direction on the luminance component, a first vertical derivative block in the vertical direction on the luminance component, and a first horizontal vertical mixed derivative block in the horizontal and vertical mixed direction on the luminance component.

[0188] In this embodiment, by determining that the luminance derivative block corresponding to the reference prediction block and / or the reference reconstruction block of at least one reference block includes at least one of a first horizontal derivative block, a first vertical derivative block, and a first horizontal-vertical mixed derivative block, it is possible to obtain derivative blocks in the horizontal direction, the vertical direction, and the horizontal-vertical mixed direction on the luminance component. When determining or generating a prediction block, the derivative blocks of the reference block of at least one block to be predicted are comprehensively considered, which can improve the prediction effect of the prediction processing.

[0189] Mode 9, the chrominance component derivative block includes at least one of a second horizontal derivative block, a second vertical derivative block, and a second horizontal and vertical mixed derivative block;

[0190] Optionally, corresponding derivative blocks may be obtained from different directions on the chrominance component, such as a second horizontal derivative block in the horizontal direction, a second vertical derivative block in the vertical direction, and a second horizontal vertical mixed derivative block in a horizontal and vertical mixed direction.

[0191] Optionally, the chroma component derivative block of the reference prediction block may include a second horizontal derivative block of the reference prediction block in the horizontal direction on the chroma component, a second vertical derivative block in the vertical direction on the chroma component, and a second horizontal and vertical mixed derivative block in the horizontal and vertical mixed directions on the chroma component.

[0192] Optionally, the chroma component derivative block of the reference reconstructed block may include a second horizontal derivative block of the reference reconstructed block in the horizontal direction on the chroma component, a second vertical derivative block in the vertical direction on the chroma component, and a second horizontal and vertical mixed derivative block in the horizontal and vertical mixed directions on the chroma component.

[0193] In this embodiment, by determining that the chrominance component derivative block corresponding to the reference prediction block and / or reference reconstruction block of at least one reference block includes at least one of a second horizontal derivative block, a second vertical derivative block, and a second horizontal and vertical mixed derivative block, it is possible to obtain derivative blocks in the horizontal direction, the vertical direction, and the horizontal and vertical mixed direction on the chrominance component. When determining or generating a prediction block, the derivative blocks of the reference block of at least one block to be predicted are comprehensively considered, which can improve the prediction effect of the prediction processing.

[0194] In a tenth embodiment, the cross-component derivative block includes at least one of a third horizontal derivative block, a third vertical derivative block, and a third horizontal-vertical mixed derivative block.

[0195] Optionally, corresponding derivative blocks may be obtained from different directions in the cross component, such as the third horizontal derivative block in the horizontal direction, the third vertical derivative block in the vertical direction, and the third horizontal vertical mixed derivative block in the horizontal and vertical mixed direction.

[0196] Optionally, the cross-component derivative block of the reference prediction block may include a third horizontal derivative block obtained from the horizontal direction, a third vertical derivative block obtained from the vertical direction, and a third horizontal and vertical mixed derivative block obtained from the horizontal and vertical mixed directions of the reference prediction block in the cross-component.

[0197] Optionally, the cross-component derivative block of the reference reconstructed block may include a third horizontal derivative block obtained from the horizontal direction, a third vertical derivative block obtained from the vertical direction, and a third horizontal and vertical mixed derivative block obtained from the horizontal and vertical mixed directions of the reference reconstructed block in the cross-component.

[0198] In this embodiment, by determining that the cross-component derivative block corresponding to the reference prediction block and / or reference reconstruction block of at least one reference block includes at least one of a third horizontal derivative block, a third vertical derivative block, and a third horizontal and vertical mixed derivative block, it is possible to obtain derivative blocks from the horizontal direction, the vertical direction, and the horizontal and vertical mixed direction across the component. When determining or generating a prediction block, the derivative block of the reference block of at least one block to be predicted is comprehensively considered, which can improve the prediction effect of the prediction processing.

[0199] Third embodiment

[0200] Based on any of the above embodiments, a third embodiment is proposed.

[0201] In this embodiment, referring to Figure 6 , step S1 includes step S11 and step S12:

[0202] Step S11, determining or obtaining at least one derivative block according to a reference reconstructed block and / or a reference prediction block of at least one reference block;

[0203] Optionally, based on at least one reference reconstruction block, a derivative block corresponding to at least one reference reconstruction block can be determined or obtained, and / or corresponding derivative blocks can be obtained from different components and different directions, such as at least one item of the luminance component derivative block, the chrominance component derivative block, and the cross-component derivative block of at least one reference reconstruction block, such as the first horizontal derivative block, the first vertical derivative block, and the first horizontal vertical mixed derivative block obtained from the horizontal direction, the vertical direction, and the horizontal and vertical mixed direction of the luminance component of at least one reference reconstruction block, the second horizontal derivative block, the second vertical derivative block, and the second horizontal and vertical mixed derivative block obtained from the horizontal direction, the vertical direction, and the horizontal and vertical mixed direction of the chrominance component of at least one reference reconstruction block, and the third horizontal derivative block, the third vertical derivative block, and the third horizontal and vertical mixed derivative block obtained from the horizontal direction, the vertical direction, and the horizontal and vertical mixed direction of the cross-component of at least one reference reconstruction block.

[0204] Optionally, based on at least one reference prediction block, a derivative block corresponding to at least one reference prediction block can be determined or obtained, and / or corresponding derivative blocks can be obtained from different components and different directions, such as at least one item of the luminance component derivative block, the chrominance component derivative block, and the cross-component derivative block of at least one reference prediction block, such as the first horizontal derivative block, the first vertical derivative block, and the first horizontal vertical mixed derivative block obtained from the horizontal direction, the vertical direction, and the horizontal and vertical mixed direction of the luminance component of at least one reference prediction block, the second horizontal derivative block, the second vertical derivative block, and the second horizontal and vertical mixed derivative block obtained from the horizontal direction, the vertical direction, and the horizontal and vertical mixed direction of the chrominance component of at least one reference prediction block, and the third horizontal derivative block, the third vertical derivative block, and the third horizontal and vertical mixed derivative block obtained from the horizontal direction, the vertical direction, and the horizontal and vertical mixed direction of the cross-component of at least one reference prediction block.

[0205] Optionally, the size parameters of the derived block match the size parameters of the reference reconstructed block and / or the reference prediction block, and / or step S11 includes step a1 and step a2:

[0206] Step a1, cropping a partial area of ​​at least one reference reconstructed block and / or a reference prediction block;

[0207] Optionally, a partial area of ​​at least one reference reconstructed block to be cropped may be determined and cropped. Optionally, a partial area of ​​at least one reference prediction block to be cropped may be determined and cropped.

[0208] Optionally, the size parameters of a partial area of ​​at least one reference reconstructed block and / or reference prediction block can be determined, and the partial area of ​​at least one reference reconstructed block and / or reference prediction block can be determined based on the size parameters of the partial area. For example, if there is a reference prediction block a with three rows of pixel areas, and the size parameters of the partial area of ​​the reference prediction block a are the size parameters of the top row of pixel areas, then the top row of pixel areas of the reference prediction block a can be cropped.

[0209] Optionally, before performing prediction processing on at least one reference prediction block, the reference reconstruction block and / or reference prediction block of the at least one reference block, and its corresponding derivative blocks may be transformed, such as at least one of DCT transform (Discrete Cosine Transform), DST transform (Discrete Sine Transform), KL transform (Karhunen–Loève Transform), wavelet transform, Hadamard transform, etc.

[0210] Optionally, a preset transform size parameter corresponding to at least one reference prediction block and / or reference reconstructed block may be determined, and a size parameter of a partial area of ​​at least one reference prediction block and / or reference reconstructed block may be determined or obtained based on the preset transform size parameter.

[0211] Optionally, a size parameter of the partial area is smaller than a preset transformation size parameter.

[0212] Optionally, the size parameters of the partial area may include the size parameters of the left partial area, may include the size parameters of the right partial area, may include the sum of the size parameters of the left and right partial areas, may include the size parameters of the upper partial area, may include the size parameters of the lower partial area, may include the sum of the size parameters of the upper and lower partial areas, may include the sum of the size parameters of the left, right, upper and lower partial areas, and so on.

[0213] Optionally, the size parameter may include at least one of width, height, area, and perimeter.

[0214] Optionally, the preset transformation size parameter may be dynamically changed or statically fixed, which is not limited here.

[0215] Optionally, the preset transformation size parameter may be a transformation size parameter, and the transformation size parameter includes at least one of a transformation width and a transformation height.

[0216] Optionally, the transform width is greater than or equal to the number of columns for cropping and / or padding a partial area of ​​at least one reference prediction block and / or a reference reconstructed block;

[0217] Optionally, the transformation height is greater than or equal to the number of rows for cropping and / or filling a partial area of ​​at least one reference prediction block and / or a reference reconstructed block.

[0218] Optionally, the preset transform size parameter may be the same as or different from the size parameter of the reference prediction block and / or the reference reconstruction block. Optionally, the preset transform size parameter may also be set in advance, which is not limited here.

[0219] Optionally, the partial area of ​​at least one reference reconstructed block and / or reference prediction block is determined or obtained according to at least one of the following methods 11 to 14:

[0220] Mode 11: at least one of at least one leftmost column, at least one rightmost column, at least one topmost row, and at least one bottommost row of at least one reference reconstructed block and / or reference prediction block;

[0221] Optionally, a partial area of ​​at least one reference reconstructed block may be determined or obtained based on at least one leftmost column of at least one reference reconstructed block.

[0222] Optionally, a partial area of ​​at least one reference reconstructed block may be determined or obtained based on at least one rightmost column of at least one reference reconstructed block.

[0223] Optionally, a partial area of ​​at least one reference reconstructed block may be determined or obtained based on at least one uppermost row of at least one reference reconstructed block.

[0224] Optionally, a partial area of ​​at least one reference reconstructed block may be determined or obtained based on at least one bottom row of at least one reference reconstructed block.

[0225] Optionally, a partial area of ​​at least one reference prediction block may be determined or obtained based on at least one leftmost column of at least one reference prediction block.

[0226] Optionally, a partial area of ​​at least one reference prediction block may be determined or obtained based on at least one rightmost column of at least one reference prediction block.

[0227] Optionally, a partial area of ​​at least one reference prediction block may be determined or obtained based on at least one top row of at least one reference prediction block.

[0228] Optionally, a partial area of ​​at least one reference prediction block may be determined or obtained based on at least one bottom row of at least one reference prediction block.

[0229] For example, when the reference reconstructed block and / or the reference prediction block is a 2x2 image block, the top row of the reference reconstructed block and / or the reference prediction block may be used as the partial area.

[0230] In this embodiment, by determining at least one of the leftmost column, rightmost column, topmost row, and bottommost row of at least one reference reconstructed block and / or reference prediction block as a partial area to be cropped, and then performing cropping, it is possible to ensure that the cropped partial area is located at the edge of at least one reference reconstructed block and / or reference prediction block, thereby ensuring the accuracy of the derivative blocks subsequently determined or obtained based on the cropped partial area, thereby improving the prediction effect.

[0231] Method 12: Change size parameters;

[0232] Optionally, the transformation size parameter includes at least one of a transformation width and a transformation height.

[0233] Optionally, the transform width is greater than or equal to the number of columns for cropping and / or padding a partial area of ​​at least one reference reconstructed block and / or a reference prediction block;

[0234] Optionally, the transformation height is greater than or equal to the number of rows for cropping and / or filling a partial area of ​​at least one reference reconstructed block and / or a reference prediction block.

[0235] Optionally, the size parameter of the partial area of ​​at least one reference reconstructed block and / or the reference prediction block is smaller than the transform size parameter. Therefore, the size parameter of the partial area of ​​at least one reference reconstructed block can be determined or obtained based on the transform size parameter of the at least one reference reconstructed block, and then the partial area of ​​at least one reference reconstructed block can be determined or obtained based on the size parameter of the partial area of ​​the at least one reference reconstructed block, such as at least one of the at least leftmost column, at least one rightmost column, at least one topmost row, and at least one bottommost row of the at least one reference reconstructed block.

[0236] Optionally, the size parameters of a partial area of ​​at least one reference prediction block may be determined or obtained based on the transform size parameters of the at least one reference prediction block, and then the partial area of ​​at least one reference prediction block may be determined or obtained based on the size parameters of the partial area of ​​the at least one reference prediction block, such as at least one of the leftmost column, the rightmost column, the topmost row, and the bottommost row of the at least one reference prediction block.

[0237] In this embodiment, by determining or obtaining a partial area of ​​at least one reference reconstructed block and / or a reference prediction block based on a transform size parameter, the accuracy of the determined partial area is guaranteed, thereby ensuring the accuracy of the derivative block subsequently determined or obtained based on the cropped partial area, thereby improving the prediction effect.

[0238] Method 13, cutting size parameters;

[0239] Optionally, the cropping size parameter may be a parameter used to crop a partial area of ​​at least one image block.

[0240] Optionally, a partial area of ​​at least one reference reconstructed block and / or a reference prediction block may be cropped according to a cropping size parameter.

[0241] Optionally, the cropping size parameter may be a dynamically changing parameter or a static fixed parameter.

[0242] Optionally, the cropping size parameter is determined or obtained by a preset transformation size parameter, or may be a parameter preset in advance.

[0243] Optionally, the crop size parameter is less than or equal to the transform size parameter.

[0244] Optionally, the cropping size parameter may include a cropping width and / or height, and may include the number of rows and / or columns for cropping at least one of a reference prediction block and a reference reconstructed block of the reference block.

[0245] Optionally, a partial area of ​​at least one reference reconstructed block and / or a reference prediction block may be determined or obtained based on the cropping size parameter, a partial area of ​​at least one reference prediction block may be determined or obtained based on the cropping size parameter, and a partial area of ​​at least one reference reconstructed block may be determined or obtained based on the cropping size parameter.

[0246] Optionally, the number of rows and / or columns required to crop at least one reference reconstructed block and / or reference prediction block in the cropping size parameter may be used as the number of rows and / or columns included in the partial region.

[0247] Optionally, the cropping size parameter may be the same as the size parameter of the partial area.

[0248] In this embodiment, by determining at least a portion of the region based on the cropping size parameter, the determination of the portion of the region is closely associated with the cropping action, thereby ensuring the accuracy of the determined at least portion of the region, and further ensuring the accuracy of the subsequent determination or acquisition of the derivative block based on the at least portion of the region. This allows the predictor to effectively capture transition features between image blocks based on the derivative block, thereby improving the prediction effect of the prediction process.

[0249] Mode 14: The sliding area size of the sliding window sliding on at least one reference reconstructed block and / or reference prediction block and / or the sliding preset step size of the sliding window.

[0250] Optionally, the sliding window may be a window in the processing device that can slide on at least one reference reconstructed block and / or reference prediction block, and may slide on the displayed reference reconstructed block and / or reference prediction block in response to a user's dragging trajectory.

[0251] Optionally, the preset sliding step size may be a step size that the sliding window needs to slide, and may be set in advance.

[0252] Optionally, the sliding area may be an effective area of ​​the sliding window. The effective area may be an area generated by sliding the sliding window on at least one image block, and / or a reference prediction block, and / or a reference reconstruction block.

[0253] Optionally, the sliding area size may be the size of the sliding area, such as width, height, area, perimeter, etc.

[0254] Optionally, the sliding area where the sliding window slides on at least one reference reconstruction block according to a preset sliding step size can be used as a partial area of ​​the reference reconstruction block, and the sliding area where the sliding window slides on at least one reference prediction block according to a preset sliding step size can be used as a partial area of ​​the reference prediction block.

[0255] Optionally, a sliding area corresponding to a size of a sliding area where the sliding window slides on at least one reference reconstructed block may be used as a partial area of ​​the reference reconstructed block.

[0256] Optionally, a sliding area corresponding to a size of a sliding area where the sliding window slides on at least one reference prediction block may be used as a partial area of ​​the reference prediction block.

[0257] Optionally, the sliding area where the sliding window slides on at least one reference reconstructed block and / or reference prediction block according to a preset sliding step size can be processed to obtain a partial area of ​​at least one reference reconstructed block and / or reference prediction block, for example, an area where a sub-image block with a high degree of overlap between the sliding area and the sub-image block in at least one reference reconstructed block is located is selected as the partial area, or for example, the middle part of the sliding area is removed, and the obtained area is used as the partial area.

[0258] Optionally, at least a portion of the region is determined or obtained based on a sliding area where a sliding window slides on at least one reference prediction block according to a preset step size, and the at least a portion of the region is cropped.

[0259] Optionally, at least a portion of the region is determined or obtained based on a sliding region in which a sliding window slides on at least one reference reconstructed block according to a preset step size, and the at least a portion of the region is cropped.

[0260] In this embodiment, at least a portion of the region is determined or obtained by using the size of the sliding region of the sliding window sliding on at least one reference reconstructed block and / or reference prediction block and / or the preset sliding step size of the sliding window, so that the determination of the portion of the region is closely related to user needs, thereby ensuring the validity of the determined at least portion of the region, and further ensuring the accuracy of the subsequent determination or acquisition of the derived block based on the at least portion of the region, so that the predictor can effectively capture the transition characteristics between image blocks based on the derived block, thereby improving the prediction effect of the prediction processing.

[0261] Step a2: determining or obtaining at least one derivative block according to a filling result of filling at least one reference reconstructed block and / or reference prediction block after being cropped.

[0262] Optionally, for the reference reconstructed block, if a partial area of ​​the reference reconstructed block is determined or obtained according to at least one of the above-mentioned methods 11 to 14, the partial area can be cropped, and then the cropped reference reconstructed block can be pixel-filled. Then, based on the reference reconstructed block after pixel filling, the derivative block corresponding to the reference reconstructed block can be determined or obtained, for example, the reference reconstructed block after pixel filling can be directly used as the derivative block corresponding to the reference reconstructed block.

[0263] Optionally, for the reference prediction block, if a partial area of ​​the reference prediction block is determined or obtained according to at least one of the above-mentioned methods 11 to 14, the partial area can be cropped, and then the cropped reference prediction block can be pixel-filled, and then based on the reference prediction block after pixel filling, the derivative block corresponding to the reference prediction block can be determined or obtained, for example, the reference prediction block after pixel filling can be directly used as the derivative block corresponding to the reference prediction block.

[0264] Optionally, a filling method for filling the cropped reference reconstruction block and / or reference prediction block includes at least one of zero filling and mirror symmetric filling.

[0265] Optionally, the zero padding may be performed by extending the size of at least one image block by adding zero values ​​to the edge of at least one reference reconstructed block and / or reference prediction block.

[0266] Optionally, the mirror-symmetric filling may be to use the mirror image of the edge pixels of at least one reference reconstructed block and / or reference prediction block as the filling pixels.

[0267] Optionally, the at least one reference reconstructed block after cropping may be padded according to a size parameter of the at least one reference reconstructed block, so that the size parameter of the padded reference reconstructed block is the same as the size parameter of the at least one reference reconstructed block.

[0268] Optionally, the cropped at least one reference prediction block may be padded according to a size parameter of the at least one reference prediction block, so that the size parameter of the padded reference prediction block is the same as the size parameter of the at least one reference prediction block.

[0269] Optionally, a size parameter of a derivative block corresponding to the reference reconstructed block is the same as a size parameter of the reference reconstructed block.

[0270] Optionally, a size parameter of a derivative block corresponding to a reference prediction block is the same as a size parameter of the reference prediction block.

[0271] Optionally, when a partial area of ​​the reference reconstructed block and / or the reference prediction block of at least one reference block is at least the leftmost column and / or at least the rightmost column of the at least one reference reconstructed block and / or the reference prediction block, the at least the leftmost column and / or the at least one rightmost column of the at least one reference reconstructed block and / or the reference prediction block can be cropped to obtain the at least one cropped reference reconstructed block and / or reference prediction block.

[0272] Optionally, the leftmost n1 columns and the rightmost m1 columns of at least one reference reconstructed block and / or reference prediction block are cropped, where n1+m1 is the transformation width.

[0273] Optionally, the n1 column is at least one column, and the m1 column is at least one column.

[0274] Optionally, at least one leftmost column and at least one rightmost column of at least one reference reconstructed block and / or reference prediction block may be cropped simultaneously, or may be cropped in a certain order, which is not limited here.

[0275] Optionally, at least one reference reconstructed block and / or reference prediction block may be cropped in the horizontal direction and / or the vertical direction.

[0276] Optionally, at least one leftmost column and / or at least one rightmost column of at least one reference prediction block may be cropped, and the cropped at least one reference prediction block may be padded, and a derivative block may be determined or obtained based on the padded reference prediction block, and prediction processing may be performed based on the at least one derivative block.

[0277] Optionally, at least one leftmost column and / or at least one rightmost column of at least one reference reconstructed block may be cropped, and the cropped at least one reference reconstructed block may be padded, and a derivative block may be determined or obtained based on the padded reference reconstructed block, and prediction processing may be performed based on the at least one derivative block.

[0278] Optionally, when a partial area of ​​the reference reconstruction block and / or the reference prediction block of at least one reference block is at least one top row and / or at least one bottom row of at least one reference reconstruction block and / or reference prediction block, the at least one top row and / or at least one bottom row of the at least one reference reconstruction block and / or reference prediction block can be cropped to obtain the cropped at least one reference reconstruction block and / or reference prediction block.

[0279] Optionally, the top n2 rows and the bottom m2 rows of at least one reference reconstructed block and / or reference prediction block are cropped, where n2+m2 is the transformation height.

[0280] Optionally, the n2 line is at least one line, and the m2 line is at least one line.

[0281] Optionally, at least one top row and at least one bottom row of at least one reference reconstructed block and / or reference prediction block may be cropped simultaneously, or may be cropped in a certain order, which is not limited here.

[0282] Optionally, at least one reference reconstructed block and / or reference prediction block may be cropped in the horizontal direction and / or the vertical direction.

[0283] Optionally, at least one top row and / or at least one bottom row of at least one reference prediction block may be cropped, and the cropped at least one reference prediction block may be padded, a derivative block may be determined or obtained based on the padded reference prediction block, and prediction processing may be performed based on the at least one derivative block.

[0284] Optionally, at least one top row and / or at least one bottom row of at least one reference reconstructed block may be cropped, and the cropped at least one reference reconstructed block may be padded, and a derivative block may be determined or obtained based on the padded reference reconstructed block, and prediction processing may be performed based on the at least one derivative block.

[0285] In this embodiment, by cropping and then filling a portion of at least one reference reconstructed block and / or reference prediction block to determine or obtain a derivative block, it is ensured that the derivative block is closely associated with the original reference reconstructed block and / or reference prediction block, thereby ensuring the accuracy of the derivative block. This allows the predictor to effectively capture the transition features between image blocks based on the derivative block, thereby improving the prediction effect of the prediction processing.

[0286] Optionally, the filling method for filling the at least one reference reconstructed block and / or reference prediction block after cropping includes at least one of steps b1 to b2:

[0287] Step b1, padding the leftmost or rightmost portion of at least one cropped reference reconstructed block and / or reference prediction block with pixels in even columns;

[0288] Optionally, the number of padding columns for pixel padding is the transform width.

[0289] Optionally, if at least one leftmost column and / or at least one rightmost column of at least one reference reconstructed block and / or reference prediction block are cropped, and / or the number of all cropped columns is an even number of columns, then the leftmost or rightmost columns of the at least one reference reconstructed block and / or reference prediction block after cropping may be padded with pixels in an even number of columns based on the size parameters of the at least one reference reconstructed block and / or reference prediction block, for example, the padded pixel values ​​are zero, so that the size parameters of the padded reference prediction block are the same as the size parameters of the at least one reference prediction block, and the size parameters of the padded reference reconstructed block are the same as the size parameters of the at least one reference reconstructed block.

[0290] For example, if the leftmost column and the rightmost column of at least one reference reconstructed block and / or reference prediction block are cropped, the rightmost column of the cropped reference reconstructed block and / or reference prediction block can be padded with two columns of pixels, and / or the pixel filling method can be zero padding or mirror-symmetric padding.

[0291] Optionally, based on the size parameters of at least one reference prediction block, the leftmost or rightmost side of the at least one cropped reference prediction block is padded with even-numbered columns of pixels to obtain a padded reference prediction block, a derivative block is determined or obtained based on the padded reference prediction block, and prediction processing is then performed based on the at least one derivative block.

[0292] Optionally, based on the size parameters of at least one reference reconstructed block, the leftmost or rightmost part of the cropped at least one reference reconstructed block is padded with an even number of columns of pixels to obtain a padded reference reconstructed block, a derivative block is determined or obtained based on the padded reference reconstructed block, and prediction processing is then performed based on the at least one derivative block.

[0293] For example, Figure 7As shown, the boundary area of ​​the reconstructed block of at least one image block may be cropped, and two columns of zero padding may be performed on the rightmost side to obtain a first horizontal derivative block. Figure 7 The area with a pixel value of 0 in the image block is the area after zero filling, and the pixel value is non-zero (such as 2, 63, etc.) is the area in the reconstructed block that is not cropped or filled.

[0294] In this embodiment, by padding the leftmost or rightmost side of at least one cropped reference reconstructed block and / or reference prediction block with even-numbered columns of pixels based on the size parameters of at least one reference reconstructed block and / or reference prediction block to generate a corresponding derivative block, the padded pixels can be taken into consideration as a whole during subsequent transformation processing of the derivative block, thereby improving the prediction effect.

[0295] Step b2: Filling the top or bottom of the at least one cropped reference reconstructed block and / or reference prediction block with pixels in even rows.

[0296] Optionally, the number of rows of pixel padding is the transform height.

[0297] Optionally, if at least one top row and / or at least one bottom row of at least one reference reconstructed block and / or reference prediction block are cropped, and / or all cropped rows are an even number of rows, then the top or bottom rows of the at least one reference reconstructed block and / or reference prediction block after cropping may be padded with pixels of an even number of rows based on the size parameters of the at least one reference reconstructed block and / or reference prediction block, for example, the padded pixel values ​​are zero, so that the size parameters of the padded reference prediction block are the same as the size parameters of the at least one reference prediction block, and the size parameters of the padded reference reconstructed block are the same as the size parameters of the at least one reference reconstructed block.

[0298] For example, if at least one top row and at least one bottom row of the reference reconstructed block and / or the reference prediction block are cropped, two columns of pixels on the rightmost side of the cropped reference reconstructed block and / or the reference prediction block can be padded, and / or the pixels can be padded with zeros or with mirror-symmetric filling.

[0299] Optionally, based on the size parameters of at least one reference prediction block, the top or bottom of the at least one reference prediction block after cropping is padded with even rows of pixels to obtain a padded reference prediction block, a derivative block is determined or obtained based on the padded reference prediction block, and then prediction processing is performed based on the at least one derivative block.

[0300] Optionally, based on the size parameters of at least one reference reconstructed block, the top or bottom of the at least one reference reconstructed block after cropping is padded with an even number of rows of pixels to obtain a padded reference reconstructed block, a derivative block is determined or obtained based on the padded image block, and then prediction processing is performed based on the at least one derivative block.

[0301] For example, Figure 8 As shown, the boundary area of ​​the reconstructed block of at least one image block may be cropped, and two rows of zero padding may be performed on the bottom row to obtain a first vertical derivative block. Figure 8 The area with a pixel value of 0 in the image block is the area after zero filling, and the pixel value is non-zero (such as 2, 63, etc.) is the area in the reconstructed block that is not cropped or filled.

[0302] For example, Figure 9 As shown, the boundary area of ​​the reconstructed block of at least one image block can be cropped, and two columns of mirror filling can be performed on the rightmost side, and two rows of mirror filling can be performed on the bottom row to obtain a first horizontal and vertical mixed derivative block. Figure 9 The white area in the middle is the area after mirror filling, and the gray area is the area without cropping filling.

[0303] In this embodiment, by padding the top or bottom of the at least one reference reconstructed block and / or reference prediction block with even rows of pixels based on the size parameters of the at least one reference reconstructed block and / or reference prediction block after cropping, so as to generate a corresponding derivative block, the padded pixels can be taken into consideration as a whole when the derivative block is subsequently transformed, thereby improving the prediction effect.

[0304] Step S12: determining or generating a prediction block based on the neural network and / or the lookup table and at least one derivative block.

[0305] Optionally, prediction processing is performed based on a neural network and / or a lookup table, and a derivative block corresponding to a reference prediction block and / or a reference reconstructed block of at least one reference block.

[0306] Optionally, at least one index can be determined or obtained by using at least one derivative block of at least one reference prediction block, at least one reference prediction block, at least one reference reconstructed block, and at least one derivative block of at least one reference reconstructed block, and then a lookup table is searched based on the at least one index, and the prediction block after prediction processing is determined or obtained based on the search result.

[0307] Optionally, at least one derivative block of at least one reference prediction block, at least one reference prediction block, at least one reference reconstruction block, and at least one derivative block of at least one reference reconstruction block can be input into a neural network for model training, and the prediction block after prediction processing can be determined or obtained based on the output result.

[0308] Optionally, at least one of the derivative blocks of at least one reference prediction block, at least one reference prediction block, at least one reference reconstruction block, and at least one derivative block of at least one reference reconstruction block can be transformed to obtain transformation features corresponding to each block, and then the transformation features corresponding to each block are input into a neural network, and the prediction block after prediction processing is determined or obtained based on the output result, such as directly outputting the prediction block, or outputting a prediction mode, performing prediction processing according to the prediction mode, and obtaining the prediction block, etc.

[0309] Optionally, it is also possible to determine the transformation features corresponding to at least one of the derivative blocks of at least one reference prediction block, at least one reference prediction block, at least one reference reconstructed block, and the derivative blocks of at least one reference reconstructed block, and determine the index corresponding to each transformation feature, so as to search in at least one lookup table according to the index, and determine or obtain the prediction block after prediction processing according to the search result, for example, if the search result is a prediction mode, prediction processing is performed according to the prediction mode to obtain the prediction block, etc. For example, if the search result is a predicted pixel after prediction processing, the prediction block is determined or obtained according to the predicted pixel.

[0310] Optionally, in this embodiment, the lookup table structure of the lookup table is used to approximate the neural network, and the channels in the lookup table structure have the same meaning as the channels in the original neural network.

[0311] Optionally, in this embodiment, the neural network can be a neural network including complete input and output, or a neural network module including only a part of the neural network. For example, a neural network module with only one convolutional layer is also a neural network.

[0312] In this embodiment, by determining or obtaining a prediction block based on a neural network and / or a lookup table, and at least one derivative block, the advantages of the neural network and / or the lookup table can be combined, and the transition characteristics between image blocks can be effectively captured through the derivative block, thereby improving the prediction effect of the prediction processing.

[0313] Fourth embodiment

[0314] Based on any of the above embodiments, a fourth embodiment is proposed.

[0315] In this embodiment, the image processing method further includes at least one of the following steps c1 to c4:

[0316] Step c1, transforming at least one of the first horizontal derivative block, the first vertical derivative block, and the first horizontal and vertical mixed derivative block to obtain a first transformation feature;

[0317] Optionally, on the luminance component, the transform process may be performed on at least one derivative block corresponding to the reference prediction block of at least one reference block, or the transform process may be performed on at least one derivative block corresponding to the reference reconstructed block of at least one reference block.

[0318] Optionally, the transformation method may be at least one of DCT transformation, DST transformation, KL transformation, wavelet transformation, Hadamard transformation, etc. For example, 2x2 DCT transformation is used for transformation processing.

[0319] Optionally, the luminance component derivative block of at least one reference prediction block may be transformed to determine or obtain a first transformation feature of the reference prediction block on the luminance component. The first transformation feature may be a pixel feature of the block after the luminance component derivative block of the reference prediction block is transformed, such as a pixel value, a pixel position, etc.

[0320] Optionally, the luminance component derivative block of at least one reference reconstructed block may be transformed to determine or obtain a first transformation feature of the reference reconstructed block on the luminance component. The first transformation feature may be a pixel feature of the block after the luminance component derivative block of the reference reconstructed block is transformed, such as a pixel value, a pixel position, etc.

[0321] Optionally, within the luminance component, at least one of the first horizontal derivative block, the first vertical derivative block and the first horizontal vertical mixed derivative block corresponding to the reference prediction block and / or the reference reconstruction block in the horizontal direction, the vertical direction, and the horizontal and vertical mixed direction can be transformed, and then the corresponding transformation feature is obtained and used as the first transformation feature.

[0322] In this embodiment, by transforming at least one of the first horizontal derivative blocks, first vertical derivative blocks, and first horizontal and vertical mixed derivative blocks of at least one reference reconstructed block and / or reference prediction block in different directions on the luminance component, a first transformation feature is obtained, thereby facilitating the effective implementation of subsequent prediction processing by a neural network and / or a lookup table based on the first transformation feature.

[0323] Step c2, transforming at least one of the second horizontal derivative block, the second vertical derivative block, and the second horizontal and vertical mixed derivative block to obtain a second transformed feature;

[0324] Optionally, on the chrominance component, the transform process may be performed on at least one derivative block corresponding to the reference prediction block of at least one reference block, or the transform process may be performed on at least one derivative block corresponding to the reference reconstructed block of at least one reference block.

[0325] Optionally, the transformation method may be at least one of DCT transformation, DST transformation, KL transformation, wavelet transformation, Hadamard transformation, etc. For example, 2x2 DCT transformation is used for transformation processing.

[0326] Optionally, the chrominance component derivative block of at least one reference prediction block can be transformed to determine or obtain a second transformation feature of the reference prediction block on the chrominance component. The second transformation feature can be a pixel feature of the block after the chrominance component derivative block of the reference prediction block is transformed, such as pixel value, pixel position, etc.

[0327] Optionally, the chrominance component derivative block of at least one reference reconstructed block can be transformed to determine or obtain a second transformation feature of the reference reconstructed block on the chrominance component. The second transformation feature can be a pixel feature of the block after the chrominance component derivative block of the reference reconstructed block is transformed, such as pixel value, pixel position, etc.

[0328] Optionally, within the chrominance component, at least one of the second horizontal derivative block, the second vertical derivative block, and the second horizontal vertical mixed derivative block corresponding to the reference prediction block and / or the reference reconstruction block in the horizontal direction, the vertical direction, and the horizontal and vertical mixed direction can be transformed, and then the corresponding transformation feature is obtained and used as the second transformation feature.

[0329] In this embodiment, by transforming at least one of the second horizontal derivative blocks, second vertical derivative blocks, and second horizontal and vertical mixed derivative blocks of at least one reference reconstructed block and / or reference prediction block in different directions on the chrominance component, a second transformation feature is obtained, thereby facilitating the effective implementation of prediction processing by subsequent neural networks and / or lookup tables based on the second transformation feature.

[0330] Step c3, transforming at least one of the third horizontal derivative block, the third vertical derivative block, and the third horizontal and vertical mixed derivative block to obtain a third transformed feature;

[0331] Optionally, a transform process may be performed on the cross-component derivative blocks of the reference reconstructed block and / or the reference prediction block.

[0332] Optionally, the transformation method may be at least one of DCT transformation, DST transformation, KL transformation, wavelet transformation, Hadamard transformation, etc. For example, 2x2 DCT transformation is used for transformation processing.

[0333] Optionally, the cross-component derivative blocks of at least one reference prediction block can be transformed, and the third transformation feature can be determined based on the characteristics of the transformed block. For example, if it is necessary to obtain the third transformation feature of the reference prediction block in the luminance component, the chrominance component derivative block obtained by the reference prediction block in the chrominance component can be determined, and the chrominance component derivative block can be transformed to extract the corresponding transformation feature, which is used as the third transformation feature of the reference prediction block in the luminance component; for example, if it is necessary to obtain the third transformation feature of the reference prediction block in the chrominance component, the luminance component derivative block obtained by the reference prediction block in the luminance component can be determined, and the luminance component derivative block can be transformed to extract the corresponding transformation feature, which is used as the third transformation feature of the reference prediction block in the chrominance component.

[0334] Optionally, a cross-component derivative block of at least one reference reconstructed block may be transformed, and a third transform feature may be determined based on the features of the transformed block. The specific implementation process is similar to determining the third transform feature of the reference prediction block.

[0335] Optionally, the third transformation feature may be a pixel feature of a block after the cross-component derivative block of the reference prediction block is transformed, such as a pixel value, a pixel position, etc.

[0336] Optionally, at least one of the third horizontal derivative block, the third vertical derivative block and the third horizontal vertical mixed derivative block corresponding to the reference prediction block and / or the reference reconstruction block in the horizontal direction, the vertical direction, and the horizontal and vertical mixed direction is transformed, and then the corresponding transformation feature is obtained and used as the third transformation feature.

[0337] In this embodiment, a third transformation feature is obtained by transforming at least one of the third horizontal derivative blocks, third vertical derivative blocks, and third horizontal and vertical mixed derivative blocks of at least one reference reconstructed block and / or reference prediction block in different directions across the cross-component, thereby facilitating the effective prediction processing of subsequent neural networks and / or lookup tables based on the third transformation feature.

[0338] Step c4: transform the reconstructed image information and / or the predicted image information of at least one reference reconstructed block and / or reference predicted block to obtain a fourth transformed feature.

[0339] Optionally, the reconstructed image information of the reference reconstructed block may be pixel information, texture information, etc. of the reference reconstructed block, such as pixel value, pixel position, etc.

[0340] Optionally, the predicted image information of the reference prediction block may be pixel information, texture information, etc. of the reference prediction block, such as pixel value, pixel position, etc.

[0341] Optionally, the reconstructed image information of at least one reference reconstructed block may be transformed, and a transformation feature may be determined or obtained based on the transformed reconstructed image information and used as a fourth transformation feature, such as using pixel information in the transformed reconstructed image information as the fourth transformation feature.

[0342] Optionally, the predicted image information of at least one reference prediction block can be transformed, and a transformation feature can be determined or obtained based on the transformed predicted image information and used as a fourth transformation feature, such as using pixel information in the transformed predicted image information as the fourth transformation feature.

[0343] Optionally, after obtaining at least one reference reconstructed block, predicted image information of at least one reference prediction block belonging to the same reference block can be obtained based on the at least one reference reconstructed block, and the predicted image information can be transformed. The transformation feature is determined or obtained based on the transformed predicted image information, and is used as the fourth transformation feature, for example, the pixel information in the transformed predicted image information is used as the fourth transformation feature.

[0344] Optionally, after obtaining at least one reference prediction block, reconstructed image information of at least one reference reconstruction block belonging to the same reference block can be obtained based on the at least one reference prediction block, and the reconstructed image information can be transformed. The transformation feature is determined or obtained based on the transformed reconstructed image information, and is used as the fourth transformation feature, for example, the pixel information in the transformed reconstructed image information is used as the fourth transformation feature.

[0345] In this embodiment, the reconstructed image information and / or predicted image information of at least one reference reconstruction block and / or reference prediction block is transformed to obtain a fourth transformation feature, thereby facilitating the effective prediction processing of subsequent neural networks and / or lookup tables based on the third transformation feature.

[0346] Fifth embodiment

[0347] Based on any of the above embodiments, a fifth embodiment is proposed.

[0348] In this embodiment, step S1 includes at least one of the following steps d1 to d4:

[0349] Step d1, determining or obtaining a first feature set based on the neural network and / or the lookup table and the result of channel splicing of the first transformed feature and the fourth transformed feature, and determining or generating a prediction block based on the first feature set;

[0350] Optionally, the first transformation feature can be determined or obtained based on the result of transforming the luminance component derivative block of the prediction block on the luminance component. Optionally, the first transformation feature can be determined or obtained based on the result of transforming the luminance component derivative block of the reconstructed block on the luminance component.

[0351] Optionally, the first transformation feature can be determined or obtained based on the result of transforming at least one of the first horizontal derivative block, the first vertical derivative block, and the first horizontal and vertical mixed derivative block on the luminance component of the prediction block and / or the reconstructed block.

[0352] Optionally, the reconstructed image information of at least one reconstructed block may be transformed to obtain the fourth transformation feature, and the predicted image information of at least one prediction block may be transformed to obtain the fourth transformation feature.

[0353] Optionally, the first feature set may include a plurality of channel-stitched multi-channel dimensional image block features, such as a channel-stitched first transformation feature and a channel-stitched fourth transformation feature.

[0354] Optionally, a channel is a component of a feature map of an image block in a depth dimension, which is used to describe the feature representation of the number of features in a specific dimension. Each channel represents a certain feature (such as texture, edge, derivative distribution) extracted from an image block (such as a prediction block or a reconstructed block). For example, one channel may be used to detect horizontal edges, and another channel may be used to detect vertical edges.

[0355] Optionally, the image block features of at least two channels (such as the first transformation feature and / or the fourth transformation feature) can be channel-spliced ​​to obtain a first multi-channel feature, and then based on the correspondence between the channel feature and the index set in advance, the index corresponding to the first multi-channel feature (such as a one-dimensional index, a two-dimensional index or a three-dimensional index, etc.) is determined or obtained, and input into the lookup table for search to determine or obtain the prediction block.

[0356] Optionally, channel splicing can be performed on at least one first transformation feature and at least one fourth transformation feature based on at least one convolutional layer in the neural network to obtain at least one multi-channel dimensional image block feature, such as the multi-channel dimensional image block feature corresponding to the prediction block and the multi-channel dimensional image block feature corresponding to the reconstruction block.

[0357] For example, at least one first transformation feature corresponding to a derivative block of at least one reconstructed block and a fourth transformation feature corresponding to at least one reconstructed block are input into a neural network for group convolution processing, and channel splicing is performed on the transformation features of multiple channels (such as the first transformation feature and / or the fourth transformation feature) in each convolution group to obtain a first feature set of image block features containing at least one multi-channel dimension, and then subsequent convolution processing is performed on at least one image block feature in the first feature set until an output result of the neural network is obtained, and a prediction block is determined or obtained based on the output result. For example, when the output result is a prediction mode, the prediction block is predicted according to the prediction mode to obtain a prediction block.

[0358] Optionally, channel splicing can be performed on at least one first transformation feature and at least one fourth transformation feature to obtain a first feature set containing image block features of at least one multi-channel dimension. The index corresponding to at least one image block feature in the first feature set can be determined, and a search can be performed in at least one lookup table based on the index to determine or obtain a prediction block.

[0359] In this embodiment, a first feature set is determined or obtained by performing channel splicing based on a neural network and / or a lookup table, as well as the results of first and fourth transformation features, and a prediction block is determined or generated based on the first feature set. Advantages of the neural network and / or the lookup table can be utilized, and prediction processing is performed on the reference prediction block and its corresponding derivative block in the luminance component in combination with the transformation features of the reference reconstructed block and its corresponding derivative block in the luminance component, thereby improving the accuracy of the obtained prediction block.

[0360] Step d2: determining or obtaining a first feature set based on the neural network and / or the lookup table and the result of channel splicing of the second transformed features and the fourth transformed features, and determining or generating a prediction block based on the first feature set;

[0361] Optionally, the second transformation feature can be determined or obtained based on the result of transforming the chroma component derivative block of the prediction block on the chroma component, and the second transformation feature can be determined or obtained based on the result of transforming the luminance component derivative block of the reconstructed block on the chroma component.

[0362] Optionally, the second transformation feature can be determined or obtained based on the result of transforming at least one of the second horizontal derivative block, the second vertical derivative block, and the second horizontal and vertical mixed derivative block on the chrominance component of the prediction block and / or the reconstructed block.

[0363] Optionally, the reconstructed image information of at least one reconstructed block may be transformed to obtain the fourth transformation feature, and the predicted image information of at least one prediction block may be transformed to obtain the fourth transformation feature.

[0364] Optionally, the first feature set may include a plurality of channel-stitched multi-channel dimensional image block features, such as the channel-stitched second transformation feature and the channel-stitched fourth transformation feature.

[0365] Optionally, channel splicing can be performed on at least one second transformation feature and at least one fourth transformation feature based on at least one convolutional layer in the neural network to obtain at least one multi-channel dimensional image block feature, such as the multi-channel dimensional image block feature corresponding to the prediction block and the multi-channel dimensional image block feature corresponding to the reconstruction block.

[0366] For example, at least one second transformation feature corresponding to a derivative block of at least one reconstructed block and a fourth transformation feature corresponding to at least one reconstructed block are input into a neural network for group convolution processing, and channel splicing is performed on the transformation features of multiple channels (such as the second transformation feature and / or the fourth transformation feature) in each convolution group to obtain a first feature set of image block features containing at least one multi-channel dimension, and then subsequent convolution processing is performed on at least one image block feature in the first feature set until an output result of the neural network is obtained, and a prediction block is determined or obtained based on the output result. For example, when the output result is a prediction mode, the prediction block is predicted according to the prediction mode to obtain a prediction block.

[0367] Optionally, channel splicing can be performed on at least one second transformation feature and at least one fourth transformation feature to obtain a first feature set containing image block features of at least one multi-channel dimension. The index corresponding to at least one image block feature in the first feature set can be determined, and a search can be performed in at least one lookup table based on the index to determine or obtain a prediction block.

[0368] In this embodiment, a first feature set is determined or obtained by performing channel splicing based on a neural network and / or a lookup table, as well as the results of second and fourth transformation features, and a prediction block is determined or generated based on the first feature set. Advantages of the neural network and / or the lookup table can be utilized, and prediction processing is performed on the reference prediction block and its corresponding derivative block on the chrominance component in combination with the transformation features of the reference reconstructed block and its corresponding derivative block on the chrominance component, thereby improving the accuracy of the obtained prediction block.

[0369] Step d3: determining or obtaining a first feature set based on the neural network and / or the lookup table, and the result of channel splicing of the first transformed feature, the second transformed feature, and the fourth transformed feature, and determining or generating a prediction block based on the first feature set;

[0370] Optionally, the first transformation feature can be determined or obtained based on the result of transforming the luminance component derivative block of the prediction block on the luminance component. Optionally, the first transformation feature can be determined or obtained based on the result of transforming the luminance component derivative block of the reconstructed block on the luminance component.

[0371] Optionally, the first transformation feature can be determined or obtained based on the result of transforming at least one of the first horizontal derivative block, the first vertical derivative block, and the first horizontal and vertical mixed derivative block on the luminance component of the prediction block and / or the reconstructed block.

[0372] Optionally, the second transformation feature can be determined or obtained based on the result of transforming the chroma component derivative block of the prediction block on the chroma component, and the second transformation feature can be determined or obtained based on the result of transforming the luminance component derivative block of the reconstructed block on the chroma component.

[0373] Optionally, the second transformation feature can be determined or obtained based on the result of transforming at least one of the second horizontal derivative block, the second vertical derivative block, and the second horizontal and vertical mixed derivative block on the chrominance component of the prediction block and / or the reconstructed block.

[0374] Optionally, the reconstructed image information of at least one reconstructed block may be transformed to obtain the fourth transformation feature, and the predicted image information of at least one prediction block may be transformed to obtain the fourth transformation feature.

[0375] Optionally, the first feature set may include multiple channel-stitched multi-channel dimensional image block features, such as a first transformation feature stitched together through channels, a second transformation feature stitched together through channels, and a fourth transformation feature stitched together through channels.

[0376] Optionally, channel splicing can be performed on at least one first transformation feature, at least one second transformation feature and at least one fourth transformation feature based on at least one convolutional layer in the neural network to obtain at least one multi-channel dimensional image block feature, such as the multi-channel dimensional image block feature corresponding to the prediction block and the multi-channel dimensional image block feature corresponding to the reconstruction block.

[0377] For example, at least one first transformation feature and at least one second transformation feature corresponding to a derivative block of at least one reconstructed block, and a fourth transformation feature corresponding to at least one reconstructed block are input into a neural network for group convolution processing, and channel splicing is performed on the transformation features of multiple channels (such as the first transformation feature, the second transformation feature and / or the fourth transformation feature) in each convolution group to obtain a first feature set of image block features containing at least one multi-channel dimension, and then subsequent convolution processing is performed on at least one image block feature in the first feature set until an output result of the neural network is obtained, and a prediction block is determined or obtained based on the output result. For example, when the output result is a prediction mode, the prediction block is predicted according to the prediction mode to obtain a prediction block.

[0378] Optionally, channel splicing can be performed on at least one first transformation feature, at least one second transformation feature, and at least one fourth transformation feature to obtain a first feature set containing image block features of at least one multi-channel dimension. The index corresponding to at least one image block feature in the first feature set can be determined, and a search can be performed in at least one lookup table based on the index to determine or obtain a prediction block.

[0379] In this embodiment, a first feature set is determined or obtained by performing channel splicing based on a neural network and / or a lookup table, as well as the results of first, second, and fourth transformation features. A prediction block is determined or generated based on the first feature set. Advantages of the neural network and / or the lookup table can be utilized, and prediction processing is performed on the reference prediction block and its corresponding derivative block on the luminance component and chrominance component in combination with the transformation characteristics of the reference reconstructed block and its corresponding derivative block on the luminance component and chrominance component, thereby improving the accuracy of the obtained prediction block.

[0380] Step d4: Determine or obtain a first feature set based on the neural network and / or the lookup table, and the result of channel splicing of the third transformation feature and the fourth transformation feature, and determine or generate a prediction block based on the first feature set.

[0381] Optionally, the third transformation feature may be determined or obtained based on the result of transforming the cross-component derivative blocks of the prediction block, and the third transformation feature may be determined or obtained based on the result of transforming the cross-component derivative blocks of the reconstructed block.

[0382] Optionally, a third transformation feature can be determined or obtained based on the result of transforming at least one of a third horizontal derivative block, a third vertical derivative block, and a third horizontal and vertical mixed derivative block on the cross-component of the prediction block and / or the reconstructed block.

[0383] Optionally, the reconstructed image information of at least one reconstructed block may be transformed to obtain the fourth transformation feature, and the predicted image information of at least one prediction block may be transformed to obtain the fourth transformation feature.

[0384] Optionally, the first feature set may include a plurality of channel-joined multi-channel dimensional image block features, such as the channel-joined third transformation feature and the channel-joined fourth transformation feature.

[0385] Optionally, channel splicing can be performed on at least one third transformation feature and at least one fourth transformation feature based on at least one convolutional layer in the neural network to obtain at least one multi-channel dimensional image block feature, such as the multi-channel dimensional image block feature corresponding to the prediction block and the multi-channel dimensional image block feature corresponding to the reconstruction block.

[0386] For example, at least one third transformation feature corresponding to a derivative block of at least one reconstructed block and at least one fourth transformation feature corresponding to a reconstructed block are input into a neural network for group convolution processing, and channel splicing is performed on the transformation features of multiple channels (such as the third transformation feature and / or the fourth transformation feature) in each convolution group to obtain a first feature set of image block features containing at least one multi-channel dimension, and then subsequent convolution processing is performed on at least one image block feature in the first feature set until an output result of the neural network is obtained, and a prediction block is determined or obtained based on the output result. For example, when the output result is a prediction mode, the prediction block is predicted according to the prediction mode to obtain a prediction block.

[0387] Optionally, channel splicing can be performed on at least one third transformation feature and at least one fourth transformation feature to obtain a first feature set containing image block features of at least one multi-channel dimension. The index corresponding to at least one image block feature in the first feature set can be determined, and a search can be performed in at least one lookup table based on the index to determine or obtain a prediction block.

[0388] In this embodiment, a first feature set is determined or obtained by performing channel splicing based on a neural network and / or a lookup table, as well as the results of third and fourth transform features, and a prediction block is determined or generated based on the first feature set. Advantages of the neural network and / or the lookup table can be utilized, and prediction processing can be performed in combination with the transform features of the reference reconstructed block and its corresponding derivative block on the cross-components, and the transform features of the reference prediction block and its corresponding derivative block on the chrominance component, thereby improving the accuracy of the obtained prediction block.

[0389] Optionally, determining or generating the prediction block according to the first feature set includes at least one of the following steps e1 to e2:

[0390] Step e1, convolving some features in the first feature set according to the convolution module of the neural network to obtain a first convolution feature, and determining or generating a prediction block based on a fusion result of fusing the first convolution feature with unconvolved features in the first feature set;

[0391] Optionally, the first feature set may be determined or obtained according to at least one of steps d1 to d4 above.

[0392] Optionally, some of the features in the first feature set may be at least one multi-channel dimensional image block feature in the first feature set.

[0393] Optionally, the unconvolved features in the first feature set may be multi-channel dimensional image block features in the first feature set that do not participate in the convolution process.

[0394] Optionally, the neural network may include a first branch including a short-circuit structure and a second branch including at least one convolution module. Part of the features in the first feature set may pass through the second branch and be convolved by the convolution module in the second branch (such as a 3x3 convolution layer, etc.) to obtain a first convolution feature. The remaining other part of the features in the first feature set passes through the first branch to obtain the unconvolved features in the first feature set. The first convolution feature output by the second branch and the unconvolved features in the first feature set output by the first branch are fused (such as channel splicing) to obtain a fusion result, and the fusion result may be subjected to subsequent convolution processing to determine or generate the output result of the neural network, and a prediction block is determined or generated based on the output result of the neural network. For example, if the output result is a prediction block and the output result is a prediction mode, the prediction block is predicted based on the prediction mode to determine or generate a prediction block.

[0395] In this embodiment, a first convolution feature is obtained by convolving some features in the first feature set according to the convolution module of the neural network. A prediction block is determined or generated based on a fusion result of the first convolution feature and the unconvolved features in the first feature set. The partial convolution characteristics of the neural network can be utilized to achieve the goal of reducing complexity while ensuring the prediction effect.

[0396] Step e2: search for some features in the first feature set according to the lookup table to obtain first lookup features, and determine or generate a prediction block based on a fusion result of the first lookup features and features not searched in the first feature set.

[0397] Optionally, the first feature set may be determined or obtained according to at least one of steps d1 to d4 above.

[0398] Optionally, some of the features in the first feature set may be at least one multi-channel image block feature in the first feature set. Optionally, the features not searched in the first feature set may be multi-channel image block features in the first feature set that do not participate in the lookup table search operation.

[0399] Optionally, the correspondence between the image block features and the index can be set in advance, and the indexes corresponding to some features in the first feature set can be determined based on the correspondence, so as to perform a search in the lookup table and obtain the search results. The search results include the first lookup table features, such as the prediction mode corresponding to the index, the predicted pixel, etc.

[0400] Optionally, the first lookup feature and the features not looked up in the first feature set may be fused to obtain a fusion result, and then a prediction block may be determined or generated based on the fusion result and a subsequent neural network and / or lookup table.

[0401] Optionally, when the fusion result is predicted pixels and unsearched features, model training can be performed on the unsearched features in the fusion result based on a neural network to determine the predicted pixels corresponding to the unsearched features. The predicted pixels corresponding to the unsearched features can also be determined or obtained based on a lookup table, and then the predicted block can be obtained based on all the predicted pixels corresponding to the block to be predicted.

[0402] Optionally, when the fusion result is a prediction mode and an unsearched feature, the prediction mode corresponding to the unsearched feature can be determined based on the lookup table and / or neural network, and the selected prediction mode (such as the prediction mode with the lowest cost) can be determined among the various prediction modes, and then the prediction block can be predicted based on the selected prediction mode to obtain a prediction block.

[0403] In this embodiment, the first lookup table feature is obtained by searching for some features in the first feature set according to the lookup table, and the prediction block is determined or generated based on the fusion result of the first lookup table feature and the features not searched in the first feature set, and then the characteristics of the lookup table can be used to improve the prediction effect.

[0404] Sixth embodiment

[0405] The present application also provides a processing device, referring to Figure 10 , the processing device includes:

[0406] The processing module A10 is configured to determine or generate a prediction block according to a derivative block corresponding to at least one reference block of the block to be predicted.

[0407] Optionally, the reference block of the at least one block to be predicted includes a reference prediction block and / or a reference reconstructed block of the reference block; and / or the reference block is determined or obtained according to at least one of the following:

[0408] At least one of the upper adjacent pixel, upper non-adjacent pixel, left adjacent pixel, left non-adjacent pixel, upper left adjacent pixel, upper left non-adjacent pixel, lower left adjacent pixel, lower left non-adjacent pixel, upper right adjacent pixel, and upper right non-adjacent pixel of the block to be predicted;

[0409] At least one of a neighbor block, a non-neighbor block, an inter-component block, a co-located block, a time domain block, and a default block corresponding to the block to be predicted;

[0410] At least one of the width, height, block size, and block area of ​​the block to be predicted;

[0411] A candidate block determined or generated by a candidate motion vector or a candidate block vector of the block to be predicted;

[0412] If the first information of the block to be predicted satisfies the first condition, the reference block is the first reference block;

[0413] If the first information of the block to be predicted meets the second condition, the reference block is the second reference block.

[0414] Optionally, the processing module A10 is configured to perform at least one of the following:

[0415] The derivative block includes at least one of a luminance component derivative block, a chrominance component derivative block, and an inter-component derivative block;

[0416] The luminance component derivative block includes at least one of a first horizontal derivative block, a first vertical derivative block, and a first horizontal-vertical mixed derivative block;

[0417] The chroma component derivative block includes at least one of a second horizontal derivative block, a second vertical derivative block, and a second horizontal-vertical mixed derivative block;

[0418] The cross-component derivative block includes at least one of a third horizontal derivative block, a third vertical derivative block, and a third horizontal-vertical mixed derivative block.

[0419] Optionally, the processing module A10 is configured to execute:

[0420] Determine or obtain at least one derivative block based on a reference reconstructed block and / or a reference prediction block of at least one reference block;

[0421] A prediction block is determined or generated based on a neural network and / or a lookup table and at least one derivative block.

[0422] Optionally, the size parameters of the derived block match the size parameters of the reference reconstructed block and / or the reference prediction block, optionally,

[0423] Processing module A10 is used to execute:

[0424] Cropping a partial area of ​​at least one reference reconstructed block and / or a reference prediction block;

[0425] At least one derivative block is determined or obtained according to a filling result of filling at least one reference reconstructed block and / or reference prediction block after being cropped.

[0426] Optionally, the partial area of ​​at least one reference reconstructed block and / or reference prediction block is determined or obtained according to at least one of the following:

[0427] at least one of at least one leftmost column, at least one rightmost column, at least one topmost row, and at least one bottommost row of at least one reference reconstructed block and / or reference prediction block;

[0428] Transform size parameters;

[0429] Cutting size parameters;

[0430] The size of the sliding area in which the sliding window slides on at least one reconstructed block and / or prediction block and / or the preset sliding step size of the sliding window.

[0431] Optionally, the processing module A10 is configured to perform at least one of the following:

[0432] The crop size parameter is less than or equal to the transform size parameter;

[0433] The transform size parameter includes at least one of a transform width and a transform height;

[0434] The transform width is greater than or equal to the number of columns for cropping and / or padding a partial area of ​​at least one reference reconstructed block and / or a reference prediction block;

[0435] The transformation height is greater than or equal to the number of rows for cropping and / or filling a partial area of ​​at least one reference reconstructed block and / or a reference prediction block.

[0436] Optionally, the filling method for filling the at least one reference reconstructed block and / or reference prediction block after being cropped includes at least one of the following:

[0437] Filling the leftmost or rightmost column of pixels of the at least one reference reconstructed block and / or reference prediction block after cropping with an even number of pixels;

[0438] The top or bottom of the at least one cropped reference reconstructed block and / or reference prediction block is padded with pixels in even rows.

[0439] Optionally, the processing module A10 is configured to perform at least one of the following:

[0440] transforming at least one of the first horizontal derivative block, the first vertical derivative block, and the first horizontal and vertical mixed derivative block to obtain a first transformed feature;

[0441] transforming at least one of the second horizontal derivative block, the second vertical derivative block, and the second horizontal and vertical mixed derivative block to obtain a second transformed feature;

[0442] transforming at least one of the third horizontal derivative block, the third vertical derivative block, and the third horizontal and vertical mixed derivative block to obtain a third transformed feature;

[0443] The reconstructed image information and / or the predicted image information of at least one reconstructed block and / or predicted block is transformed to obtain a fourth transformed feature.

[0444] Optionally, the processing module A10 is configured to perform at least one of the following:

[0445] Determine or obtain a first feature set based on the neural network and / or the lookup table and a result of channel splicing of the first transformed feature and the fourth transformed feature, and determine or generate a prediction block based on the first feature set;

[0446] Determine or obtain a first feature set based on the neural network and / or the lookup table and a result of channel splicing of the second transformed feature and the fourth transformed feature, and determine or generate a prediction block based on the first feature set;

[0447] Determine or obtain a first feature set based on the neural network and / or the lookup table and a result of channel splicing of the first transformation feature, the second transformation feature, and the fourth transformation feature, and determine or generate a prediction block based on the first feature set;

[0448] A first feature set is determined or obtained based on the neural network and / or the lookup table, and the result of channel splicing of the third transformation feature and the fourth transformation feature, and a prediction block is determined or generated based on the first feature set.

[0449] Optionally, determining or generating the prediction block according to the first feature set includes at least one of the following:

[0450] Convolving some features in the first feature set according to a convolution module of the neural network to obtain a first convolution feature, and determining or generating a prediction block based on a fusion result of fusing the first convolution feature with unconvolved features in the first feature set;

[0451] A first lookup feature is obtained by searching for some features in the first feature set according to the lookup table, and a prediction block is determined or generated according to a fusion result of the first lookup feature and features not searched in the first feature set.

[0452] The processing device provided in the embodiment of the present application has similar implementation principles and beneficial effects to the technical solutions shown in the above-mentioned corresponding method embodiments, and will not be described in detail here.

[0453] An embodiment of the present application further provides a processing device, including a memory and a processor. The memory stores an image processing program, and when the image processing program is executed by the processor, the steps of the image processing method in any of the above embodiments are implemented.

[0454] An embodiment of the present application further provides a storage medium on which an image processing program is stored. When the image processing program is executed by a processor, the steps of the image processing method in any of the above embodiments are implemented.

[0455] In the embodiments of the processing device and storage medium provided in this application, all technical features of any of the above-mentioned image processing method embodiments may be included. The expanded and explained contents of the specification are basically the same as those of the embodiments of the above-mentioned methods and will not be repeated here.

[0456] An embodiment of the present application further provides a computer program product, which includes computer program code. When the computer program code runs on a computer, the computer executes the methods in the various possible implementation modes described above.

[0457] An embodiment of the present application also provides a chip, including a memory and a processor, wherein the memory is used to store computer programs, and the processor is used to call and run the computer programs from the memory, so that a device equipped with the chip executes the methods in the various possible implementation modes as described above.

[0458] It is understood that the above scenarios are merely examples and do not limit the application scenarios of the technical solutions provided in the embodiments of this application. The technical solutions of this application can also be applied to other scenarios. For example, those skilled in the art will appreciate that with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application will also be applicable to similar technical problems.

[0459] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0460] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0461] The units in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0462] In this application, the same or similar terminology, technical solutions and / or application scenario descriptions are generally only described in detail the first time they appear. When they appear again later, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of this application, for the same or similar terminology, technical solutions and / or application scenario descriptions that are not described in detail later, you can refer to the previous relevant detailed descriptions.

[0463] In this application, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0464] The various technical features of the technical solution of this application can be combined arbitrarily. In order to make the description concise, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0465] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as mentioned above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, controlled terminal, or network device, etc.) to execute the method of each embodiment of the present application.

[0466] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a storage disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state storage disk Solid State Disk (SSD)).

[0467] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image processing method, characterized in that: Including steps: S1 , determining or generating a prediction block according to a derivative block corresponding to at least one reference block of a block to be predicted.

2. The image processing method according to claim 1, wherein: The reference block of at least one block to be predicted includes a reference prediction block and / or a reference reconstructed block of the reference block; and / or the reference block is determined or obtained according to at least one of the following: At least one of the upper adjacent pixel, upper non-adjacent pixel, left adjacent pixel, left non-adjacent pixel, upper left adjacent pixel, upper left non-adjacent pixel, lower left adjacent pixel, lower left non-adjacent pixel, upper right adjacent pixel, and upper right non-adjacent pixel of the block to be predicted; At least one of a neighbor block, a non-neighbor block, an inter-component block, a co-located block, a time domain block, and a default block corresponding to the block to be predicted; At least one of the width, height, block size, and block area of ​​the block to be predicted; A candidate block determined or generated by a candidate motion vector or a candidate block vector of the block to be predicted; If the first information of the block to be predicted satisfies the first condition, the reference block is the first reference block; If the first information of the block to be predicted meets the second condition, the reference block is the second reference block.

3. The image processing method according to claim 1, wherein: Also include at least one of the following: The derivative block includes at least one of a luminance component derivative block, a chrominance component derivative block, and an inter-component derivative block; The luminance component derivative block includes at least one of a first horizontal derivative block, a first vertical derivative block, and a first horizontal-vertical mixed derivative block; The chroma component derivative block includes at least one of a second horizontal derivative block, a second vertical derivative block, and a second horizontal-vertical mixed derivative block; The cross-component derivative block includes at least one of a third horizontal derivative block, a third vertical derivative block, and a third horizontal-vertical mixed derivative block.

4. The image processing method according to any one of claims 1 to 3, wherein: Step S1 includes the following steps: S11, determining or obtaining at least one derivative block according to a reference reconstructed block and / or a reference prediction block of at least one reference block; S12, determining or generating a prediction block based on the neural network and / or the lookup table and at least one derivative block.

5. The image processing method according to claim 4, wherein: The size parameters of the derived block match the size parameters of the reference reconstructed block and / or the reference prediction block; and / or, step S11 includes: Cropping a partial area of ​​at least one reference reconstructed block and / or a reference prediction block; At least one derivative block is determined or obtained according to a filling result of filling the at least one reference reconstructed block and / or the reference prediction block after being cropped.

6. The image processing method according to claim 5, wherein: The at least one reference reconstructed block and / or a partial area of ​​the reference prediction block is determined or obtained according to at least one of the following: at least one of at least one leftmost column, at least one rightmost column, at least one topmost row, and at least one bottommost row of at least one reference reconstructed block and / or reference prediction block; Transform size parameters; Cutting size parameters; The size of the sliding area in which the sliding window slides on at least one reconstructed block and / or prediction block and / or the preset sliding step size of the sliding window.

7. The image processing method according to claim 6, wherein: Also include at least one of the following: The crop size parameter is less than or equal to the transform size parameter; The transform size parameter includes at least one of a transform width and a transform height; The transform width is greater than or equal to the number of columns for cropping and / or padding a partial area of ​​at least one reference reconstructed block and / or a reference prediction block; The transformation height is greater than or equal to the number of rows for cropping and / or filling a partial area of ​​at least one reference reconstructed block and / or a reference prediction block.

8. The image processing method according to claim 5, wherein: The filling method for filling the at least one reference reconstructed block and / or reference prediction block after being cropped includes at least one of the following: Filling the leftmost or rightmost column of pixels of the at least one reference reconstructed block and / or reference prediction block after cropping with an even number of pixels; The top or bottom of the at least one cropped reference reconstructed block and / or reference prediction block is padded with pixels in even rows.

9. The image processing method according to claim 4, wherein: Also include at least one of the following: transforming at least one of the first horizontal derivative block, the first vertical derivative block, and the first horizontal and vertical mixed derivative block to obtain a first transformed feature; transforming at least one of the second horizontal derivative block, the second vertical derivative block, and the second horizontal and vertical mixed derivative block to obtain a second transformed feature; transforming at least one of the third horizontal derivative block, the third vertical derivative block, and the third horizontal and vertical mixed derivative block to obtain a third transformed feature; The reconstructed image information and / or the predicted image information of at least one reconstructed block and / or predicted block is transformed to obtain a fourth transformed feature.

10. The image processing method according to claim 9, wherein: Step S1 includes at least one of the following: Determine or obtain a first feature set based on the neural network and / or the lookup table and a result of channel splicing of the first transformed feature and the fourth transformed feature, and determine or generate a prediction block based on the first feature set; Determine or obtain a first feature set based on the neural network and / or the lookup table and a result of channel splicing of the second transformed feature and the fourth transformed feature, and determine or generate a prediction block based on the first feature set; Determine or obtain a first feature set based on the neural network and / or the lookup table, and a result of channel splicing of the first transform feature, the second transform feature, and the fourth transform feature, and determine or generate a prediction block based on the first feature set; A first feature set is determined or obtained based on the neural network and / or the lookup table, and the result of channel splicing of the third transformation feature and the fourth transformation feature, and a prediction block is determined or generated based on the first feature set.

11. The image processing method according to claim 10, wherein: Determining or generating a prediction block based on the first feature set includes at least one of the following: Convolving some features in the first feature set according to a convolution module of the neural network to obtain a first convolution feature, and determining or generating a prediction block based on a fusion result of fusing the first convolution feature with unconvolved features in the first feature set; A first lookup feature is obtained by searching for some features in the first feature set according to the lookup table, and a prediction block is determined or generated according to a fusion result of the first lookup feature and features not searched in the first feature set.

12. A processing device, characterized in that include: A memory and a processor, wherein an image processing program is stored in the memory, and when the image processing program is executed by the processor, the steps of the image processing method according to any one of claims 1 to 11 are implemented.

13. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the steps of the image processing method according to any one of claims 1 to 11.

Images