Image processing method, processing device, and storage medium

Abstract

The present application provides an image processing method, a processing device, and a storage medium. The image processing method can be applied to a processing device, and comprises the steps of: predicting a first pixel on the basis of a reference pixel of the first pixel in an image block and a prediction model. The technical solution of the present application can improve the prediction effect of intra prediction, thereby improving the prediction efficiency of video encoding and / or decoding.

Description

Image processing method, processing device and storage medium TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an image processing method, a processing device and a storage medium. BACKGROUND

[0002] The video coding technology proposed by the existing video coding standard aims to improve the coding performance without significantly increasing the computational complexity, specifically including: when video coding is performed, each frame is divided into different blocks, and then prediction, transformation and quantization processing, and entropy encoding or entropy decoding processing are performed.

[0003] During the conception and implementation of the present application, the inventors have found that at least the following problems exist:

[0004] In the prediction stage, the prediction effect of the existing prediction mode is not ideal, which limits the prediction efficiency in the video coding and / or decoding process.

[0005] The foregoing description is intended to provide general background information and does not necessarily constitute the prior art.

[0006] SUMMARY

[0007] In view of the above technical problems, the present application provides an image processing method, a processing device and a storage medium, aiming to solve the technical problem of how to improve the prediction effect of the prediction processing involved in the video coding and decoding process.

[0008] The present application provides an image processing method, which can be applied to a processing device, comprising the steps of:

[0009] S10, predicting the first pixel according to the reference pixel of the first pixel in the image block and the prediction model.

[0010] Optionally, the reference pixel includes adjacent pixels and / or neighboring pixels.

[0011] Optionally, the determination or obtaining manner of the reference pixel includes at least one of the following:

[0012] determining or obtaining the reference pixel according to at least one of the following: the upper adjacent pixel, the upper non-adjacent pixel, the left adjacent pixel, the left non-adjacent pixel, the upper left adjacent pixel and the upper left non-adjacent pixel of the image block;

[0013] determining or obtaining the reference pixel according to the adjacent region and / or the non-adjacent region of the image block;

[0014] determining or obtaining the reference pixel according to the first component image block and / or the second component image block of the image block;

[0015] The reference pixel is determined or obtained according to at least one of the default block, the neighbor block, the non-neighbor block, the collocated block and the temporal block corresponding to the image block;

[0016] The reference pixel is determined or obtained according to the neighboring pixel and / or the adjacent pixel of the first pixel combination obtained according to the at least one first pixel combination of the image block.

[0017] Optionally, the step S10 comprises at least one of the following:

[0018] The first prediction mode is determined or obtained from the prediction modes corresponding to the at least one prediction model, and the first pixel is predicted according to the first prediction mode and the reference pixel of the first pixel in the image block.

[0019] The first prediction model is determined according to the reference pixel in the at least one prediction model, and the first pixel is predicted according to the first prediction model.

[0020] The model parameter of the prediction model of the image block is determined or obtained according to the model parameter of the prediction model of the first reference block, and the first pixel is predicted according to the reference pixel and the model parameter of the prediction model of the image block.

[0021] Optionally, the first prediction mode is determined or obtained from the prediction modes corresponding to the at least one prediction model, which comprises at least one of the following:

[0022] The first prediction mode is determined or obtained from the at least one prediction mode according to the size parameter and / or the position parameter of the image block in which the first pixel is located.

[0023] The first prediction mode is determined or obtained from the candidate prediction mode list containing the at least one prediction mode according to the size parameter and / or the position parameter of the image block in which the first pixel is located.

[0024] The first prediction mode is determined or obtained from the prediction modes corresponding to the at least one prediction model and / or the candidate prediction mode list containing the prediction modes corresponding to the at least one prediction model according to the syntax element obtained from the bitstream.

[0025] Optionally, the image processing method further comprises at least one of the following:

[0026] The first prediction mode comprises a second prediction mode, and the second prediction mode comprises one prediction model.

[0027] The first prediction mode comprises a third prediction mode, and the third prediction mode comprises at least two prediction models, and the model parameters of the at least two prediction models are different.

[0028] The prediction model comprises at least one of a linear model, a nonlinear model and a gradient model.

[0029] Optionally, the predicting the first pixel according to the first prediction mode and the reference pixel within the image block comprises at least one of:

[0030] if the first prediction mode comprises a second prediction mode, predicting the first pixel of the image block according to a second prediction model;

[0031] if the first prediction mode comprises a third prediction mode, predicting a part of the first pixels of the image block according to a third prediction model, and predicting another part of the first pixels of the image block according to a fourth prediction model.

[0032] Optionally, the determining the first prediction model according to the reference pixel in the at least one prediction model comprises:

[0033] determining or deriving the first prediction model from the at least one prediction model according to a pixel value of a neighboring pixel in the reference pixel, and / or an average value of at least one pixel in a neighboring region of the image block, and / or an average value of the at least one reference pixel.

[0034] Optionally, the determining or deriving the first reference block comprises at least one of:

[0035] determining or deriving the first reference block according to a block vector and / or a motion vector of a neighboring block and / or a non-neighboring block of the image block;

[0036] determining or deriving the first reference block according to a block vector and / or a motion vector of a neighboring pixel and / or a non-neighboring pixel of the image block;

[0037] determining or deriving the first reference block according to a block vector and / or a motion vector of at least one of a collocated block of the image block, a neighboring block and / or a non-neighboring block of the collocated block, a temporal block, a neighboring block and / or a non-neighboring block of the temporal block, a first component image block, a neighboring block and / or a non-neighboring block of the first component image block, a second component image block, and a neighboring block and / or a non-neighboring block of the second component image block;

[0038] determining or deriving the first reference block according to a block vector and / or a motion vector of at least one of a neighboring pixel and / or a non-neighboring pixel of the collocated block of the image block, a neighboring pixel and / or a non-neighboring pixel of the temporal block, a neighboring pixel and / or a non-neighboring pixel of the first component image block, and a neighboring pixel and / or a non-neighboring pixel of the second component image block;

[0039] The first reference block is determined or obtained according to at least one of a block vector and / or a motion vector of a reference block of the image block, a reference block of a neighboring image block, a reference block of a non-neighboring image block, a reference block of a collocated block, a reference block of a temporal block, a reference block of a first component image block, and a reference block of a second component image block.

[0040] The present application also provides an image processing device, which comprises:

[0041] The processing module is configured to predict the first pixel according to a reference pixel of the first pixel in the image block and a prediction model.

[0042] The present application also provides a processing device, which comprises a memory and a processor, wherein the memory stores a processing program, and the processing program is executed by the processor to implement the steps of the above method.

[0043] The present application also provides a storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the above method.

[0044] As described above, the processing method of the present application can be applied to a processing device, and comprises the step of predicting a first pixel according to a reference pixel of the first pixel in an image block and a prediction model. Through the above technical solution, the prediction effect (e.g. accuracy) of intra prediction can be improved in the prediction stage of video coding. BRIEF DESCRIPTION OF DRAWINGS

[0045] The accompanying drawings, which are incorporated herein and constitute part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced as follows. Obviously, for those of ordinary skill in the art, other drawings can also be obtained without creative effort based on these drawings.

[0046] FIG. 1 is a schematic diagram of the hardware structure of a mobile terminal implementing various embodiments of the present application;

[0047] FIG. 2 is a schematic diagram of a communication network system architecture provided by an embodiment of the present application;

[0048] FIG. 3 is a schematic diagram of the flow of an image processing method according to a first embodiment;

[0049] FIG. 4 is a schematic diagram of the encoding flow of an encoder in the image processing method according to the first embodiment;

[0050] FIG. 5 is a schematic diagram of the decoding flow of a decoder in the image processing method according to the first embodiment;

[0051] Fig. 6 is a schematic diagram of a to-be-predicted luma pixel and a corresponding reference luma pixel in the image processing method according to the first embodiment;

[0052] Fig. 7 is a schematic diagram of a to-be-predicted pixel combination and a corresponding reference luma pixel in the image processing method according to the second embodiment;

[0053] Fig. 8 is a schematic diagram of neighboring pixels and / or adjacent pixels in the image processing method according to the second embodiment;

[0054] Fig. 9 is a schematic diagram of reference chroma pixels and reference luma pixels of a to-be-predicted chroma pixel in the image processing method according to the second embodiment;

[0055] Fig. 10 is a schematic diagram of a reference luma pixel and a to-be-predicted luma pixel when a gradient model is used in the image processing method according to the third embodiment;

[0056] Fig. 11 is a schematic diagram of an image block predicted using a third prediction mode in the image processing method according to the third embodiment;

[0057] Fig. 12 is a schematic diagram of a neighboring area of a to-be-predicted block in the image processing method according to the third embodiment;

[0058] Fig. 13 is a schematic diagram of determining a reference block by a block vector in the image processing method according to the third embodiment;

[0059] Fig. 14 is a schematic diagram of determining a reference block by a motion vector in the image processing method according to the third embodiment;

[0060] Fig. 15 is a schematic diagram of determining a reference block by a continuous block vector in the image processing method according to the third embodiment;

[0061] Fig. 16 is a schematic diagram of determining a reference block by a continuous motion vector in the image processing method according to the third embodiment;

[0062] Fig. 17 is a schematic diagram of a processing module in an image processing apparatus.

[0063] The purposes, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. The above drawings have shown the specific embodiments of the present application, and the following will have more detailed description. These drawings and the written description are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0064] The exemplary embodiments will be described in detail herein with reference to a few examples. The description herein, in connection with the appended drawings, is intended to convey principles of the exemplary embodiments, and not specific operation procedures or details, as some implement details are not known at this time. It will be appreciated that the description herein and disclosed elements and / or methods can be modified in various manners. Such modifications are intended to fall within the scope of the present application. Also, it will be appreciated that the

[0065] It has to be noted that, as used herein, the terms "includes" and / or "contains", etc., are used in the sense of "comprising", meaning "including, but not limited to", and not in the sense of "consisting only of, or "consisting exclusively of", such that the listed steps or options need not be treated as being ordered and treated as essential to implement the disclosed embodiments. It is intended that the application be construed as including all such steps and options.

[0066] It is to be understood that the terms first, second, third, etc. can be adopted herein to describe various information only and should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information, without departing from the scope of the present disclosure. Depending on the context, the word "if' as used herein can be interpreted as meaning "when" or "upon determining", or "in response to determining". Furthermore, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It will be further understood that the terms "comprises", "comprising", "includes" and / or "including", etc., mean that the processes, methods, articles or apparatuses described herein include, but are not limited to, those elements expressly listed. The terms "or", "and / or", "including at least one of", etc. as used herein, can be interpreted as an inclusive "or" or "and / or", etc., meaning "any one or any combination of the listed items". For example, "A, B or C" or "A, B, and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C". Only when a combination of elements, functions, steps or operations are in some way inherently mutually exclusive, will an exception to this definition apply.

[0067] It should be understood that, although each step in the flowchart in the embodiments of the present application is shown in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless explicitly stated herein, the execution of these steps is not strictly limited in sequence, and they can be executed in other sequences. Moreover, at least part of the steps in the figure can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence is not necessarily sequential, but can be alternately executed with at least part of other steps or sub-steps or stages of other steps.

[0068] Depending on the context, the word "if" as used herein can be interpreted as meaning "when" or "while" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "if it is determined" or "if (a stated condition or event) is detected" can be interpreted as meaning "when it is determined" or "in response to determining" or "when (a stated condition or event) is detected" or "in response to detecting (a stated condition or event)".

[0069] It should be noted that, in the present document, step codes such as S10, S20, etc. are used for the purpose of more clearly and briefly expressing the corresponding content, and do not constitute a substantial limitation in sequence. Those skilled in the art may, in specific implementation, perform S20 before performing S10, etc., but these should be within the scope of protection of the present application.

[0070] It should be understood that the specific embodiments described herein are merely intended to explain the present application, and are not intended to limit the present application.

[0071] In the following description, the suffixes such as "module", "part", or "unit" used for an element are merely used to facilitate the explanation of the present application, and have no specific meaning by themselves. Thus, "module", "part", or "unit" can be mixedly used.

[0072] The processing device in the present application can be a smart terminal or a server, etc. The smart terminal can be implemented in various forms. For example, the smart terminal described in the present application can include a smart terminal such as a mobile phone, a tablet computer, a notebook computer, a palmtop computer, a Personal Digital Assistant (PDA), a Portable Media Player (PMP), a navigation device, a wearable device, a smart bracelet, a pedometer, etc., and a fixed terminal such as a digital TV, a desktop computer, etc.

[0073] In the following description, a mobile terminal will be exemplified, and it will be understood by those skilled in the art that the configuration according to the embodiments of the present application can be applied to a terminal of a fixed type, except for elements particularly used for mobile purposes.

[0074] Referring to FIG. 1, a mobile terminal 100 according to an embodiment of the present application can include a Radio Frequency (RF) unit 101, a WiFi module 102, an audio output unit 103, an A / V (audio / video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111, etc. It will be understood by those skilled in the art that the mobile terminal structure shown in FIG. 1 does not constitute a limitation on the mobile terminal, and the mobile terminal can include more or less components than those shown, or some components can be combined, or different components can be arranged differently.

[0075] The components of the mobile terminal will be described in detail below with reference to FIG. 1.

[0076] The radio frequency unit 101 can be used for receiving and transmitting signals in the process of transmitting or receiving information or a call. Specifically, the radio frequency unit 101 receives downlink information from a base station and provides the received information to the processor 110 for processing. In addition, the radio frequency unit 101 transmits uplink data to the base station. Generally, the radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 101 can communicate with a network and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to GSM (Global System for Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution), 5G, and 6G, and the like.

[0077] WiFi belongs to a short-range wireless transmission technology. The WiFi module 102 can help a user to send and receive e-mails, browse web pages, and access streaming media, and the like. The WiFi module 102 provides the user with wireless broadband Internet access. Although the WiFi module 102 is shown in FIG. 1, it can be understood that it does not belong to the essential components of the mobile terminal, and can be omitted as needed without changing the essence of the application.

[0078] The audio output unit 103 can convert audio data, which is received by the radio frequency unit 101 or the WiFi module 102 or stored in the memory 109, into an audio signal and output the audio signal as sound when the mobile terminal 100 is in a call signal reception mode, a call mode, a recording mode, a voice recognition mode, a broadcast reception mode, and the like. In addition, the audio output unit 103 can provide audio output related to a particular function performed by the mobile terminal 100 (e.g., a call signal reception sound, a message reception sound, and the like). The audio output unit 103 can include a speaker, a buzzer, and the like.

[0079] The A / V input unit 104 is configured to receive audio or video signals. The A / V input unit 104 can include a graphics processor (GPU) 1041 and a microphone 1042. The graphics processor 1041 processes image data of a still picture or a video obtained by an image capture device (e.g., a camera) in a video capture mode or an image capture mode. Processed image frames can be displayed on the display unit 106. Processed image frames can be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) via the microphone 1042 in a telephone call mode, a recording mode, a voice recognition mode, or the like, and can process such sound into audio data. Processed audio (voice) data can be converted into a format that can be transmitted to a mobile communication base station via the radio frequency unit 101 in the case of the telephone call mode. The microphone 1042 can implement various types of noise cancellation (or suppression) algorithms to cancel (or suppress) noise or interference generated in the process of receiving and transmitting audio signals.

[0080] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. The light sensor includes an ambient light sensor and a proximity sensor, which can optionally adjust the brightness of the display panel 1061 according to the brightness of ambient light, and turn off the display panel 1061 and / or the backlight when the mobile terminal 100 is moved to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally, three axes), and detect the magnitude and direction of gravity when at rest, which can be used for applications that recognize the posture of the mobile terminal (such as switching between landscape and portrait screens, related games, magnetometer posture calibration), vibration recognition related functions (such as a pedometer, a knock), and the like. The mobile terminal can also be configured with a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and other sensors, which are not described herein.

[0081] The display unit 106 is configured to display information input by a user or information provided to the user. The display unit 106 can include a display panel 1061, which can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0082] The user input unit 107 can be used to receive input numerals or character information, and to generate key signal inputs related to user settings of the mobile terminal and control of functions. Optionally, the user input unit 107 can include a touch panel 1071 and other input devices 1072. The touch panel 1071, also called a touch screen, can collect touch operations of a user on or proximity thereto (such as operations of the user using a finger, a stylus, or any suitable object or accessory on or in proximity to the touch panel 1071), and drive corresponding connection devices according to a pre-set program. The touch panel 1071 can include two parts, a touch detecting device and a touch controller. The touch detecting device detects the user's touch position and detects signals resulting from touch operations, and transmits the signals to the touch controller; the touch controller receives touch information from the touch detecting device, and converts it into touch coordinates, and sends it to the processor 110, and can receive commands from the processor 110 and execute them. In addition, the touch panel 1071 can be implemented in various types such as a resistive type, a capacitive type, an infrared type, and a surface acoustic wave type. In addition to the touch panel 1071, the user input unit 107 can include other input devices 1072. Optionally, the other input devices 1072 can include one or more of, but are not limited to, a physical keyboard, function keys (such as volume control buttons, switch buttons, etc.), a trackball, a mouse, a joystick, etc.

[0083] Optionally, the touch panel 1071 can cover the display panel 1061, and when the touch panel 1071 detects a touch operation on or in proximity thereto, it transmits to the processor 110 to determine the type of touch event, and then the processor 110 provides corresponding visual output on the display panel 1061 according to the type of touch event. Although in FIG. 1, the touch panel 1071 and the display panel 1061 are implemented as two independent components to realize the input and output functions of the mobile terminal, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to realize the input and output functions of the mobile terminal, without being limited specifically herein.

[0084] The interface unit 108 serves as an interface through which at least one external device can be connected with the mobile terminal 100. For example, the external device can include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input / output (I / O) port, a video I / O port, an earphone port, and the like. The interface unit 108 can be used to receive input (e.g., data information, power, and the like) from an external device and to transmit the received input to one or more elements within the mobile terminal 100, or can be used to transmit data between the mobile terminal 100 and the external device.

[0085] The memory 109 can be used to store software programs and various data. The memory 109 can mainly include a program storage area and a data storage area, and the program storage area can store an operating system, application programs required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; and the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), and the like. In addition, the memory 109 can include a high-speed random access memory, and can also include a nonvolatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device.

[0086] The processor 110 is a control center of the mobile terminal, connects all parts of the mobile terminal through various interfaces and lines, executes various functions of the mobile terminal and processes data by running or executing software programs and / or modules stored in the memory 109 and calling data stored in the memory 109, and thus performs overall monitoring on the mobile terminal. The processor 110 can include one or more processing units; preferably, the processor 110 can integrate an application processor and a modem processor, and the application processor can mainly process an operating system, a user interface, and application programs, and the like, and the modem processor can mainly process wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 110.

[0087] The mobile terminal 100 can also include a power supply 111 (such as a battery) for supplying power to various components; preferably, the power supply 111 can be logically connected to the processor 110 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system.

[0088] Although not shown in FIG. 1, the mobile terminal 100 can also include a Bluetooth module, and the like, which will not be described here.

[0089] In order to facilitate understanding of the embodiments of the present application, the communication network system based on the mobile terminal of the present application is described below.

[0090] Referring to FIG. 2, FIG. 2 is a communication network system architecture diagram provided by an embodiment of the present application, and the communication network system is an LTE system of a general mobile communication technology, which includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203, and an operator's IP service 204 which are sequentially connected in communication.

[0091] Optionally, the UE 201 can be the terminal 100 described above, which will not be repeated here.

[0092] The E-UTRAN 202 includes eNode Bs 2021 and other eNode Bs 2022. Optionally, the eNode Bs 2021 can be connected to each other through backhaul (for example, X2 interface), the eNode B 2021 is connected to the EPC 203, and the eNode B 2021 can provide access for the UE 201 to the EPC 203.

[0093] The EPC 203 can include MME (Mobility Management Entity) 2031, HSS (Home Subscriber Server) 2032, other MMEs 2033, SGW (Serving Gate Way) 2034, PGW (PDN Gate Way) 2035, and PCRF (Policy and Charging Rules Function) 2036. Optionally, the MME 2031 is a control node for processing signaling between the UE 201 and the EPC 203, and provides bearer and connection management. The HSS 2032 is used to provide some registers to manage functions such as home location register (not shown in the figure), and save some user-specific information about service features, data rates, etc. All user data can be transmitted through the SGW 2034, the PGW 2035 can provide IP address allocation for the UE 201 and other functions, and the PCRF 2036 is a policy and charging control policy decision point for service data flow and IP bearer resources, which selects and provides available policy and charging control decisions for policy and charging execution function units (not shown in the figure).

[0094] The IP service 204 can include the Internet, an intranet, an IMS (IP Multimedia Subsystem), or other IP services.

[0095] Although the above describes the LTE system as an example, those skilled in the art should know that the present application is not only applicable to the LTE system, but also applicable to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, 5G, and future new network systems (such as 6G), etc., which are not limited here.

[0096] Based on the above mobile terminal hardware structure and communication network system, various embodiments of the present application are proposed.

[0097] First embodiment

[0098] Referring to FIG. 3, which is a flowchart of an image processing method according to the first embodiment, the image processing method of the present embodiment can be applied to a processing device, including the following steps:

[0099] S10, predicting the first pixel according to the reference pixel of the first pixel in the image block and a prediction model.

[0100] In the present embodiment, the processing device can be a smart terminal such as a mobile phone, a computer, etc., or a server such as a local server or a cloud server. In the present embodiment and the present application, the processing device is mainly illustrated as a smart terminal.

[0101] Optionally, the technical solution of the present embodiment can be applied to the fields of image coding, video coding, hardware video coding, special circuit video coding, real-time video coding, etc.

[0102] Optionally, the processing device can obtain video image data from a video source and segment each frame of image in the video image data to obtain a plurality of image blocks.

[0103] Optionally, the image block can be a to-be-predicted image block to be predicted in the encoding side and / or the decoding side.

[0104] Optionally, the first pixel can be a pixel to be predicted in the image block. The prediction model can be a model for predicting the pixel value of the first pixel, such as a linear model, a nonlinear model, a gradient model, etc.

[0105] Optionally, the first pixel and the reference pixel of the first pixel can both be pixels of the first component. Optionally, the first component can be a chroma component or a luminance component.

[0106] Optionally, when the processing device is an encoder at the encoding side, referring to FIG. 4, the encoder receives video source input video data, such as the encoder receives video pictures from the video source, determines a to-be-predicted picture in the video pictures, divides the to-be-predicted picture into a plurality of image blocks (including to-be-predicted luma blocks and to-be-predicted chroma blocks), performs prediction processing on each of the plurality of image blocks by using the temporal and / or spatial correlation between the video pictures, the prediction processing includes intra prediction processing and / or inter prediction processing, and the intra prediction processing and / or the inter prediction processing respectively includes a plurality of prediction modes, for the prediction modes, the encoder determines a prediction mode finally adopted by each of the plurality of image blocks by using, for example, rate-distortion optimization, such as calculating a rate-distortion cost corresponding to each prediction mode or a rate-distortion cost of a combination of several prediction modes in a certain manner, to determine a minimum rate-distortion cost from the plurality of rate-distortion costs, and the prediction mode corresponding to the minimum rate-distortion cost or the combination of the prediction modes is the prediction mode finally adopted by the image block.

[0107] Optionally, when the finally adopted prediction mode is an extrapolation filter-based intra prediction (EIP) mode provided in the embodiment, the prediction pixel value of a to-be-predicted pixel can be determined by the prediction value or the reconstructed value of a neighboring pixel of the to-be-predicted pixel, and the step flow of step S10 described above can be performed. Hereinafter, the extrapolation filter-based intra prediction mode is referred to as the EIP mode or the EIP prediction mode.

[0108] Optionally, after the pixel value of each to-be-predicted pixel (such as the first pixel) of the to-be-predicted image block (including the luma block and the chroma block) is obtained based on step S10, the pixel value of each pixel sample in the original image block corresponding to the to-be-predicted image block is subtracted by the prediction value of the corresponding pixel sample in the prediction block, to obtain a residual value of each pixel sample and a residual block corresponding to the original image block. Then, the residual block is subjected to transform and quantization processing, and is encoded by an entropy encoder to form an encoded bitstream. In addition, the encoded bitstream can further include prediction parameters corresponding to the determined prediction mode and related side information. Optionally, the prediction parameters are packaged into the encoded bitstream after being subjected to entropy encoding. Optionally, the prediction parameters include indication information of the prediction mode. Optionally, the transformed and quantized residual block is added to the corresponding prediction data (such as the prediction block) obtained by using the prediction mode after being subjected to inverse quantization and inverse transform to obtain a reconstructed block. After the reconstructed block is obtained, a loop filtering module performs loop filtering processing on the reconstructed block according to filter control parameters, to reduce distortion. Then, the reconstructed block after the loop filtering processing is stored according to an encoded picture buffer.

[0109] Optionally, when the processing device is a decoder at the decoding side, referring to FIG. 5, after receiving the coded bitstream, the entropy decoding unit of the decoder parses and decodes the coded bitstream to obtain the transform coefficients. The inverse transform unit and the inverse quantization unit of the decoder perform inverse transform and inverse quantization on the transform coefficients to obtain the residual block. Optionally, the entropy decoding unit of the decoder parses and decodes the coded bitstream to obtain the prediction data, such as the prediction parameters and the related auxiliary information. The prediction processing unit of the decoder performs prediction processing by using the prediction parameters to determine the prediction block corresponding to the residual block. Optionally, the prediction processing includes intra prediction processing and / or inter prediction processing, and the intra prediction processing and / or the inter prediction processing respectively include one or more prediction modes in combination. When the prediction parameters indicate that the corresponding prediction mode is the prediction mode provided in the embodiment, the prediction mode is the prediction mode corresponding to the obtained residual block.

[0110] Optionally, when the prediction mode in the embodiment is used, the step S10 is performed, and after determining all the pixel samples of the to-be-predicted block, the prediction result of the to-be-predicted block is obtained, the obtained residual block and the corresponding prediction block (including the predicted luma block and the predicted chroma block) are added to obtain the reconstructed block, and the loop filtering unit in the decoder performs loop filtering on the reconstructed block according to the filter control data to reduce distortion and improve video quality. The reconstructed block after the loop filtering is further combined into a decoded image and stored in a decoded image buffer or output as decoded video data.

[0111] Optionally, when the image block is intra-predicted and the prediction mode provided in the embodiment is selected for prediction, any to-be-predicted pixel in the image block can be taken as the first pixel, and the first pixel is predicted according to the reference pixels of the first pixel and the prediction model to obtain the predicted pixel value.

[0112] Optionally, in an available embodiment, the reference pixels include the adjacent pixels and / or the neighboring pixels.

[0113] Optionally, the neighboring pixels are the pixels close to the first pixel, and the pixel distance between the neighboring pixels and the first pixel is less than or equal to a preset threshold. For example, the pixel distance between the neighboring pixels and the first pixel can be less than or equal to 4 pixel distances. The pixel distance can be the number of pixels between two pixels. If two pixels are separated by 4 pixels, the pixel distance is 4. Optionally, the neighboring pixels can include the pixels with a pixel distance less than or equal to 4 from the first pixel.

[0114] For example, if the image block is a luma component image block, the first pixel is a luma pixel to be predicted, and the reference pixel of the first pixel is a reference luma pixel. As shown in FIG. 6, in (a), (b) and (c) of FIG. 6, the reference luma pixel of the luma pixel to be predicted is a pixel (e.g., within a distance of 2 to 5 pixels) close to the luma pixel to be predicted, such as a neighboring pixel and / or a neighboring pixel.

[0115] The prediction mode in the embodiment can utilize the neighboring pixel and / or the neighboring pixel adjacent to the position of the pixel to be predicted to determine the prediction value (i.e., the predicted pixel value) initially obtained by the pixel to be predicted.

[0116] Optionally, when the processing device is an encoder, the prediction value initially obtained can be a prediction value obtained in a corresponding prediction mode, which can be directly used in a rate-distortion optimization process.

[0117] Optionally, when the processing device is a decoder, the prediction value initially obtained can be a prediction value obtained in a prediction mode corresponding to a to-be-predicted block (i.e., an image block in the decoding end) indicated by a syntax element parsed from a bitstream.

[0118] Since the coding process is based on raster scanning, the prediction value or the reconstructed value of the pixel above or to the left of the pixel to be predicted is known when the pixel to be predicted is predicted, and therefore the neighboring pixel and / or the neighboring pixel can be the pixel to the left or above the pixel to be predicted.

[0119] In the embodiment, the first pixel is predicted according to the reference pixel of the first pixel in the image block and the determined prediction model. In the prediction stage of video coding, when extrapolation filter-based intra prediction is used, the entire image block can be predicted by a single prediction model, but the first pixel can be predicted according to the reference pixel of the first pixel in the image block and the determined prediction model, thereby improving the prediction effect of the first pixel, that is, improving the prediction effect of the extrapolation filter-based intra prediction, and improving the prediction efficiency in the video coding and / or decoding process.

[0120] Second Embodiment

[0121] Based on the first embodiment, the second embodiment is provided.

[0122] In the embodiment, the determination or obtaining manner of the reference pixel includes at least one of the first to fifth manners.

[0123] In the first manner, the reference pixel is determined or obtained according to at least one of the following: an upper neighboring pixel, an upper non-neighboring pixel, a left neighboring pixel, a left non-neighboring pixel, a top-left neighboring pixel and a top-left non-neighboring pixel of the image block.

[0124] Optionally, the upper neighboring pixel can be a pixel in the same image as the first pixel in the same image block obtained by the final division, and the position of the pixel is above the first pixel and adjacent to the first pixel.

[0125] Optionally, the upper non-neighboring pixel can be a pixel in the same image as the first pixel in the same image block obtained by the final division, and the position of the pixel is above the first pixel, but the pixel is not adjacent to the first pixel.

[0126] Optionally, the left neighboring pixel can be a pixel in the same image as the first pixel in the same image block obtained by the final division, and the position of the pixel is left of the first pixel and adjacent to the first pixel.

[0127] Optionally, the left non-neighboring pixel can be a pixel in the same image as the first pixel in the same image block obtained by the final division, and the position of the pixel is left of the first pixel, but the pixel is not adjacent to the first pixel.

[0128] Optionally, the upper left neighboring pixel can be a pixel in the same image as the first pixel in the same image block obtained by the final division, and the position of the pixel is above and left of the first pixel and adjacent to the first pixel.

[0129] Optionally, the upper left non-neighboring pixel can be a pixel in the same image as the first pixel in the same image block obtained by the final division, and the position of the pixel is above and left of the first pixel, but the pixel is not adjacent to the first pixel.

[0130] Optionally, at least one of the upper neighboring pixel, the upper non-neighboring pixel, the left neighboring pixel, the left non-neighboring pixel, the upper left neighboring pixel and the upper left non-neighboring pixel can be a reconstructed pixel or a predicted pixel.

[0131] Optionally, at least one of the upper neighboring pixel, the upper non-neighboring pixel, the left neighboring pixel, the left non-neighboring pixel, the upper left neighboring pixel and the upper left non-neighboring pixel can be directly used as a reference pixel, or the at least one pixel can be derived or calculated to obtain a reference pixel.

[0132] For example, for the first pixel in the image block, the corresponding reference pixel can be obtained by the reference pixel module according to the preset mapping / correspondence rule. For example, the reference pixel can be selected from the above neighboring pixels, the above non-neighboring pixels, the left neighboring pixels, the left non-neighboring pixels, the upper left neighboring pixels and the upper left non-neighboring pixels of the image block according to the prediction model; for example, 4 pixels in the left reference region of the first pixel are selected as the reference pixels.

[0133] As shown in FIG. 6, (a), (b) and (c) in FIG. 6 describe three different preset mapping / correspondence rules:

[0134] The preset mapping / correspondence rule corresponding to (a) in FIG. 6 is that the reference luminance pixel and the first pixel are pixels in a 3X3 rectangular region;

[0135] The preset mapping / correspondence rule corresponding to (b) in FIG. 6 is that the reference luminance pixel and the first pixel are pixels in a 2X4 rectangular region;

[0136] The preset mapping / correspondence rule corresponding to (c) in FIG. 6 is that the reference luminance pixel and the first pixel are pixels in a 4X2 rectangular region.

[0137] The above preset mapping / correspondence rule further includes that the pixels are at least adjacent to another pixel in the region, that is, each of the reference luminance pixel and the first pixel is adjacent to another pixel in them, and optionally, the first pixel includes the to-be-predicted luminance pixel in FIG. 6.

[0138] In the embodiment, the reference pixel is determined or obtained according to at least one of the above neighboring pixels, the above non-neighboring pixels, the left neighboring pixels, the left non-neighboring pixels, the upper left neighboring pixels and the upper left non-neighboring pixels of the image block. The pixel value of the first pixel is highly approximate to at least one of the above neighboring pixels, the above non-neighboring pixels, the left neighboring pixels, the left non-neighboring pixels, the upper left neighboring pixels and the upper left non-neighboring pixels of the image block. Based on this, the above neighboring pixels and / or non-neighboring pixels are taken as the reference pixels of the first pixel, which can improve the prediction effect of predicting the first pixel in the prediction process.

[0139] Optionally, the reference pixel is determined or obtained according to the neighboring region and / or the non-neighboring region of the image block.

[0140] Optionally, the neighboring region is an image region adjacent to the image region position where the first pixel is located, for example, an image region above or left of the image region where the first pixel is located, and the non-neighboring region is an image region not adjacent to the image region position where the first pixel is located.

[0141] Optionally, the neighboring region contains a neighbor block, and the non-neighboring region contains a non-neighbor block.

[0142] Optionally, the processing device can select, for each first pixel in the image block, at least one predicted pixel or reconstructed pixel in the neighboring region and / or the non-neighboring region corresponding to the first pixel as a reference pixel, and can derive or calculate a pixel value of the selected at least one predicted pixel or reconstructed pixel as a pixel value of the corresponding reference pixel.

[0143] Optionally, the reference pixels can be located in the image block where the first pixel is located, for example, a part of the reference pixels can be located in the image block where the first pixel is located, and another part of the reference pixels can be located in the neighboring region and / or the non-neighboring region of the image block where the first pixel is located.

[0144] In an embodiment of the present application, the prediction processing is performed for the first pixel, and thus the first pixel can also be referred to as a pixel to be predicted.

[0145] As shown in FIG. 8, the first pixel can be a pixel to be predicted in (a), (b), and (c) of FIG. 8, and optionally, the first pixel can be a luminance pixel to be predicted in (a), (b), and (c) of FIG. 10. The first pixel can also be a chrominance pixel to be predicted (not shown in the figure).

[0146] As shown in FIG. 8, the first pixel is a pixel to be predicted in a block (i.e., an image block) to be predicted, and the reference pixels are neighboring pixels and / or neighboring pixels of the pixel to be predicted.

[0147] Specifically, the reference pixels can be reconstructed pixels and / or predicted pixels, and the reference pixels can be located in the neighboring region of the block to be predicted in whole or in part.

[0148] Optionally, the reference pixels are at least one of a plurality of upper neighboring pixels and / or neighboring pixels, a plurality of left neighboring pixels and / or neighboring pixels, and a plurality of upper left neighboring pixels and / or neighboring pixels of the pixel to be predicted.

[0149] As shown in (a) of FIG. 8, the reference pixels are located in the neighboring region of the block to be predicted in whole, and optionally, the reference pixels are reconstructed pixels.

[0150] As shown in (b) of FIG. 8, the reference pixels are located in the neighboring region of the block to be predicted in part, and optionally, a part of the reference pixels are reconstructed pixels in the neighboring region of the pixel to be predicted, and another part of the reference pixels are predicted pixels in the block to be predicted.

[0151] As shown in (c) of FIG. 8, the reference pixels are located in the block to be predicted in whole or in part, and optionally, all of the reference pixels are the pixel to be predicted in the block to be predicted.

[0152] In other embodiments, the reference pixel can be located in a non-adjacent region, which is a region that is not spatially adjacent. The non-adjacent region can be determined by a block vector and / or a motion vector. For example, a reference block of the image block to be predicted can be determined by a plurality of block vectors and / or motion vectors, and the reference block can be taken as the non-adjacent region. An image region can also be determined by one or more reference blocks, and the image region can be taken as the non-adjacent region.

[0153] As shown in FIG. 15, the non-adjacent region 1 (i.e., the reference block 1) can be determined by the block vector BV1 of the image block (i.e., the image block to be predicted) in which the first pixel is located, then the non-adjacent region 2 (i.e., the reference block 2) can be determined by the block vector BV2 of the non-adjacent region (i.e., the reference block 1), and finally, the corresponding pixel in the non-adjacent region 2 can be taken as the reference pixel of the first pixel.

[0154] As shown in FIG. 16, the non-adjacent region 1 (i.e., the reference block a) can be determined by the motion vector Mva of the image block (i.e., the image block to be predicted) in which the first pixel is located, then the non-adjacent region 2 (i.e., the reference block b) can be determined by the motion vector Mvb of the non-adjacent region (i.e., the reference block a), and finally, the corresponding pixel in the non-adjacent region 2 can be taken as the reference pixel of the first pixel.

[0155] In the above-described method of determining or obtaining the reference pixel from the adjacent region and / or the non-adjacent region of the image block, the determined reference pixel has a high similarity to the first pixel, thereby improving the accuracy of predicting the first pixel.

[0156] Optionally, the first component image block can be a chroma component image block or a luma component image block.

[0157] Optionally, the second component image block can be a chroma component image block or a luma component image block.

[0158] Optionally, the second component image block can be a chroma component image block or a luma component image block.

[0159] Optionally, the first component image block and the second component image block are image blocks of different components.

[0160] As shown in FIG. 9, the first pixel is a chroma pixel to be predicted, the first component image block is a luma image block, and the second component image block is a chroma image block.

[0161] Optionally, the position of the first component image block in the image of the first component and the position of the second component image block in the image of the second component correspond to each other. For example, the pixels of an image for display in YUV format have three color components, i.e. a luminance component Y, a chroma component U and a chroma component V, each image block of the image to be displayed corresponds to a luminance image block and two chroma image blocks, which correspond to each other. Similarly, in YUV format, each image block of a video image captured by a camera also corresponds to a luminance image block and two chroma image blocks.

[0162] The reference luminance pixel and the reference chroma pixel shown in FIG. 9 can be the corresponding luminance image block (first component image block) and chroma image block (second component image block) in the image block.

[0163] Optionally, at least one of the above-mentioned upper neighboring pixel, upper non-neighboring pixel, left neighboring pixel, left non-neighboring pixel, upper left neighboring pixel and upper left non-neighboring pixel of the luminance pixel at the corresponding position of the to-be-predicted chroma pixel can be used as the reference pixel of the to-be-predicted chroma pixel.

[0164] In the embodiment, the method for determining or obtaining the reference pixel through the first component image block and / or the second component image block of the image block uses the pixels of the associated components as the reference pixels, which improves the accuracy of the prediction of the first pixel due to the high similarity between the pixels and the first pixel.

[0165] Optionally, the reference pixel is determined or obtained according to at least one of the corresponding default block, neighbor block, non-neighbor block, co-located block and time-domain block of the image block.

[0166] Optionally, the default block can be a block set in advance, for example, a block with typical pixel characteristics set in advance by the encoder and / or the decoder.

[0167] Optionally, the neighbor block can be a block adjacent to the first pixel and can be a block that has been predicted or reconstructed.

[0168] Optionally, the non-neighbor block can be a block that is not adjacent to the first pixel and can be a block that has been predicted or reconstructed.

[0169] Optionally, the co-located block can be an image block in a co-located image, which has the same position and size as the image block to be predicted. Optionally, the co-located image can be an image closest in time to the current image in the reference image.

[0170] Optionally, the time domain block can be a block distinguished in the time domain, such as an image block in the previous frame. For example, if there are three frames of image data, the first frame of image is played in the first second, the second frame of image is played in the second second, and the third frame of image is played in the third second. If the image block to be predicted at the current time is an image block divided from the second frame of image, the time domain block can be determined as the image block corresponding to the first frame of image.

[0171] Optionally, at least one pixel can be obtained in at least one of the default block, the neighbor block, the non-neighbor block, the co-located block and the time domain block corresponding to the image block. The obtained at least one pixel can be used as the reference pixel, or the obtained at least one pixel can be derived or calculated to obtain the reference pixel.

[0172] Optionally, at least one of the above-mentioned pixels can be determined in at least one of the default block, the neighbor block, the non-neighbor block, the co-located block and the time domain block corresponding to the image block, and used as the reference pixel. In addition, the corresponding adjacent region and / or non-adjacent region can be determined in at least one of the default block, the neighbor block, the non-neighbor block, the co-located block and the time domain block corresponding to the image block, and at least one pixel in the region can be selected as the reference pixel.

[0173] In the embodiment, the reference pixel is determined or obtained according to at least one of the default block, the neighbor block, the non-neighbor block, the co-located block and the time domain block corresponding to the image block, so that the effectiveness of the reference pixel is guaranteed, and the prediction accuracy of the first pixel is improved.

[0174] In the fifth mode, the reference pixel is determined or obtained according to the adjacent pixel and / or the neighboring pixel of the first pixel combination obtained according to the at least one first pixel combination of the image block.

[0175] Optionally, a plurality of first pixels to be predicted in the image block can be determined, and a plurality of adjacent first pixels can be combined to obtain a first pixel combination. Optionally, the first pixel combination includes at least two adjacent first pixels.

[0176] Optionally, the adjacent pixel and / or the neighboring pixel corresponding to the first pixel combination can be determined in the image block.

[0177] Optionally, at least one of the above-mentioned pixels of the first pixel combination can be used as the adjacent pixel and / or the neighboring pixel of the first pixel combination.

[0178] Optionally, a pixel region where the first pixel combination is located can be determined, and at least one pixel in a neighboring region and / or a non-neighboring region of the pixel region can be determined as the neighboring pixel and / or the adjacent pixel of the first pixel combination.

[0179] For example, as shown in FIG. 7, if the image block is a luminance component image block, the first pixel is a to-be-predicted luminance pixel, the reference pixel of the first pixel is a reference luminance pixel, and the first pixel combination is a to-be-predicted pixel combination. As shown in (a), (b), (c), (d), (e) and (f) of FIG. 7, the to-be-predicted pixel combination can be composed of four neighboring to-be-predicted luminance pixels. The reference pixel of the to-be-predicted pixel combination (i.e., the neighboring pixel and / or the adjacent pixel of the to-be-predicted pixel combination) is a reference luminance pixel, and the reference luminance pixel can be located above, left of, and above and left of the to-be-predicted pixel combination.

[0180] Optionally, at least one of the neighboring pixels and / or the adjacent pixels corresponding to the first pixel combination can be selected as the reference pixel of the first pixel being predicted at the current moment.

[0181] Optionally, the reference pixel can be determined according to a neighboring pixel of the to-be-predicted pixel or a neighboring pixel of a collocated pixel of the to-be-predicted pixel. Optionally, the neighboring pixel includes a neighboring reconstructed pixel and / or a neighboring predicted pixel.

[0182] Optionally, the reference pixel can be determined according to vector information, and the vector information includes a block vector and / or a motion vector.

[0183] Optionally, the reference pixel can be determined according to a coded image block, and the coded image block is an image block determined according to a rate-distortion optimization or an image matching algorithm.

[0184] In this embodiment, the reference pixel is determined or obtained according to the neighboring pixel and / or the adjacent pixel of the first pixel combination obtained according to the at least one first pixel combination of the image block. Since the neighboring pixel and / or the adjacent pixel has high similarity with the first pixel, the accuracy of predicting the first pixel is improved.

[0185] Third Embodiment

[0186] Based on the first embodiment or the second embodiment, the third embodiment is proposed.

[0187] In this embodiment, the image processing method further includes at least one of the following:

[0188] The first prediction mode includes a second prediction mode, and the second prediction mode includes one prediction model.

[0189] The first prediction mode includes a third prediction mode, the third prediction mode includes at least two prediction models, and model parameters of the at least two prediction models are different.

[0190] The prediction model comprises at least one of a linear model, a non-linear model and a gradient model.

[0191] Optionally, the number of prediction models in the second prediction mode is one, and the number of prediction models in the third prediction mode is multiple.

[0192] Optionally, the model parameters and / or the model types of the prediction models included in each third prediction mode are different.

[0193] Optionally, the model types of the prediction models include at least one of a linear model, a non-linear model and a gradient model. The third prediction model can include multiple different linear models, non-linear models and gradient models. Optionally, the third prediction model includes multiple prediction models of the same model type, but the model parameters of the prediction models are different. For example, the third prediction model includes a linear model LM1 and a linear model LM2. The model parameters of the linear model LM1 and the linear model LM2 can be different. In another embodiment, the third prediction model includes prediction models of different model types. For example, the third prediction model includes a linear model and a non-linear model.

[0194] The non-linear model can have a form as shown in Equation One, i.e. Y N+1 = c11*Y1+ c12*Y2+... + c1N*Y N + c21*P1+ c22*P2+ c23*P3+... + c2N*P N + c31*B (Equation One).

[0195] Wherein c11-c1N, c21-c2N, c31 are all model parameters, such as filter coefficients. Y N+1 is the output of the non-linear model, such as the prediction value of the first pixel. Y1-Y N are inputs of the non-linear model, such as the prediction pixel values and / or the reconstructed pixel values of the reference pixels of the first pixel. P1-P N are non-linear terms. Optionally, P1-P N may be the square values of Y1-Y N . B is a bias term.

[0196] The gradient model can have a form as shown in Equation Two, i.e. Y N+1 = c11*Y1+ c12*Y2+... + c1N*Y N + c21*GLX+ c22*GLY+ c31*B (Equation Two).

[0197] Wherein c11-c1N, c21-c22, c31 are all model parameters, such as filter coefficients. YN+1 is an output of the gradient model, such as a predicted value of the first pixel. Y1-Y N is an input of the gradient model, such as a predicted pixel value and / or a reconstructed pixel value of a reference pixel of the first pixel. GLXand GLYare horizontal gradient and vertical gradient of at least one pixel and multiple pixels adjacent to Y N+1 is a bias term. For example, as shown in (a) of FIG. 10, the first pixel is a to-be-predicted luma pixel, and the reference pixel of the first pixel is a reference luma pixel, GLX= a difference between pixel values of pixel 2 and pixel 1 in the reference luma pixel. GLY= a difference between pixel values of pixel 6 and pixel 3. As shown in (b) of FIG. 10, GLX= a difference between pixel values of pixel 5 and pixel 4. GLY= a difference between pixel values of pixel 5 and pixel 1. As shown in (c) of FIG. 10, GLX= a difference between pixel values of pixel 2 and pixel 6. GLY= a difference between pixel values of pixel 2 and pixel 5.

[0198] Optionally, the step S10 comprises at least one of the following modes six to eight.

[0199] Mode six, determining or obtaining the first prediction mode from prediction modes corresponding to the at least one prediction model, and predicting the first pixel according to the first prediction mode and the reference pixel within the image block.

[0200] Optionally, the prediction modes corresponding to the at least one prediction model can be multiple, and the number of prediction models, and / or the model type, and / or the model parameter contained in each prediction mode can be different. For example, the prediction mode includes a linear model, the prediction mode includes a nonlinear model, and the prediction mode includes a gradient model. Optionally, the prediction mode in the embodiment can be an EIP prediction mode for intra prediction.

[0201] Optionally, the processing device can determine the first prediction mode from the multiple prediction modes. For example, if the multiple prediction modes include a second prediction mode and a third prediction mode, and the number of the second prediction mode and / or the third prediction mode can be multiple. For example, one second prediction mode includes a linear model, and another second prediction mode includes a nonlinear model. One third prediction mode includes a linear model and a gradient model, and another third prediction mode includes two linear models with different model parameters. If the determined prediction mode is the second prediction mode from the multiple second prediction modes and the multiple third prediction modes, the determined second prediction mode can be taken as the first prediction mode, and if the determined prediction mode is the third prediction mode, the determined third prediction mode can be taken as the first prediction mode.

[0202] Optionally, a suitable prediction mode can be selected from the plurality of prediction modes as the first prediction mode. Then, based on the first prediction mode, prediction pixel values of each of the to-be-predicted pixels in the image block where the first pixel is located are determined.

[0203] Optionally, the processing device can calculate the cost values of the respective prediction models, and select the prediction model with the minimum cost value as the first prediction model. In an embodiment, a reference pixel corresponding to each prediction model is determined, the reference pixel is input into the corresponding prediction model, the first pixel is predicted to obtain the to-be-predicted pixel value of the first pixel, and thus the to-be-predicted pixel values of the to-be-predicted block are obtained. By comparing the prediction pixel values of the to-be-predicted block with the original pixel values of the original pixels of the to-be-predicted block, the cost value of the to-be-predicted block can be determined, and the prediction model with the minimum cost value is selected as the first prediction model.

[0204] Optionally, the first prediction model can be a linear model, a non-linear model, or a gradient model.

[0205] For example, the first prediction model can be Formula Three. N+1 = f(Y1, Y2, Y3, …, Y N ) (Formula Three).

[0206] wherein Y1, Y2, Y3, …, Y N are inputs of the first prediction model, which can be reference pixels of the first pixel, including neighboring pixels and / or neighboring pixels. For example, they can be reference luminance pixels of the first pixel, or reference chrominance pixels of the first pixel. N+1 may be an output of the first prediction model.

[0207] For example, if the first pixel and the corresponding reference pixels thereof are luminance pixels, the first prediction model can be Formula Four. N+1 = c1*Y1 + c2*Y2 + c3Y3 + c4Y4 + c5Y5 + … + cN*Y N (Formula Four).

[0208] wherein N can be a natural number, for example, 6. c1~cN are model parameters (such as filter coefficients or weight parameters) of the first prediction model. Y1~Y N are inputs of the first prediction model, which can be pixel values of the reference pixels of the first pixel, such as prediction pixel values and / or reconstructed pixel values of the neighboring pixels and / or neighboring pixels of the first pixel. When Y1~Y N are prediction pixel values and / or reconstructed pixel values of the neighboring pixels and / or neighboring pixels of the first pixel, these prediction pixel values can be determined by EIP prediction. N+1The output of the first prediction model can be the predicted value of the first pixel.

[0209] Optionally, the input corresponding pixel in the third formula and the fourth formula is different from the output corresponding pixel in the third formula and the fourth formula.

[0210] Optionally, the first prediction model corresponds to the first prediction mode. After the processing device determines the first prediction mode, the predicted pixel value obtained by the first prediction model in this prediction process is the determined predicted pixel value, and the determined predicted pixel value is used for subsequent processing.

[0211] In this embodiment, by determining or obtaining the first prediction mode from the prediction modes corresponding to the at least one prediction model, and predicting the first pixel according to the first prediction mode and the reference pixels of the first pixel in the image block, the prediction of the first pixel can be performed without being limited to a linear model as a prediction mode, and can be selected according to the actual scene, thereby improving the prediction effect of the prediction of the first pixel.

[0212] Optionally, in mode six, the determination or obtaining of the first prediction mode from the prediction modes corresponding to the at least one prediction model comprises at least one of modes 1 to 3.

[0213] Mode 1, determining or obtaining the first prediction mode from the at least one prediction mode according to a size parameter and / or a position parameter of the image block in which the first pixel is located.

[0214] Optionally, the processing device can determine the size parameter of the image block in which the first pixel is located, such as the length, width and / or area of the image block, and then detect whether the parameter value of the size parameter is greater than a first preset threshold. If the parameter value of the size parameter is greater than the first preset threshold, a second prediction mode can be selected as the first prediction mode from the plurality of prediction modes. If there are a plurality of second prediction modes and each second prediction mode includes different prediction models, the cost value generated when each second prediction mode is used for prediction can be calculated, and one of the plurality of second prediction modes can be selected as the first prediction mode according to the cost value, such as selecting the second prediction mode with the minimum cost as the second prediction mode.

[0215] Optionally, if the parameter value of the size parameter is less than or equal to the first preset threshold, a third prediction mode can be selected as the first prediction mode from the plurality of prediction modes. If there are a plurality of third prediction modes and each third prediction mode includes different prediction models, the cost value generated when each third prediction mode is used for prediction can be calculated, and one of the plurality of third prediction modes can be selected as the first prediction mode according to the cost value, such as selecting the third prediction mode with the minimum cost as the second prediction mode.

[0216] Optionally, a correspondence between different size parameters and different prediction modes can also be set in advance, and the first prediction mode can be determined or obtained from the plurality of prediction modes according to the size parameter and / or the position parameter of the image block in which the first pixel is located and the correspondence.

[0217] Optionally, the position parameter of the image block in which the first pixel is located can also be determined, which can be the position of the first pixel in the image block, or the position of the image block in which the first pixel is located in a frame of image, etc. The position parameter can be compared with a preset position interval, and if the two match, one of the second prediction mode and the third prediction mode can be selected as the first prediction mode from the plurality of prediction modes. Optionally, if the second prediction mode is selected as the first prediction mode, but there are a plurality of second prediction modes, one of the second prediction modes can be selected as the first prediction mode according to the generation value corresponding to each second prediction mode.

[0218] If the position parameter does not match the preset position interval, the other of the second prediction mode and the third prediction mode can be selected as the first prediction mode from the plurality of prediction modes. Optionally, if the third prediction mode is selected as the first prediction mode, but there are a plurality of third prediction modes, one of the third prediction modes can be selected as the first prediction mode according to the generation value corresponding to each third prediction mode.

[0219] Optionally, the preset position interval can be a position interval set in advance, such as the top-left position in the image block, the position with a human image in a frame of image, etc. A correspondence between different position parameters and different prediction modes can also be constructed, and then the first prediction mode can be determined or obtained from the plurality of prediction modes according to the correspondence. Optionally, the above-mentioned position parameter can be a position parameter determined by a block vector or a motion vector.

[0220] In this embodiment, by determining or obtaining the first prediction mode from at least one prediction mode according to the size parameter and / or the position parameter of the image block in which the first pixel is located, it can be ensured that the first prediction mode obtained is more suitable for the image block in which the first pixel is located, so that the prediction effect is more accurate when the first pixel is predicted according to the first prediction mode.

[0221] Optionally, the first prediction mode can be determined or obtained from the candidate prediction mode list containing at least one prediction mode according to the size parameter and / or the position parameter of the image block in which the first pixel is located.

[0222] Optionally, the size parameters and / or the position parameters are associated in the candidate prediction mode list, and each prediction mode is stored in the candidate prediction mode list in advance, and then at least one prediction mode is selected as the first prediction mode in the candidate prediction mode list according to the size parameter and / or the position parameter of the image block in which the first pixel is located and the associated size parameter and / or the position parameter in the candidate prediction mode list. Please refer to Table 1, which is an example of the association of the size parameters and the position parameters in the candidate prediction mode list.

[0223] Table 1

[0224] The size parameters 1-3 can be 16x16, 32x32, 64x64 respectively, and the position parameters 1-3 can be (X1, Y1), (X2, Y2), (X3, Y3) respectively. For example, if the size parameter of the image block in which the first pixel is located is the same as or closest to the size parameter 1, prediction mode 1 is selected. If the position parameter of the image block in which the first pixel is located is the same as or closest to the position parameter 3, prediction mode 3 is selected.

[0225] Optionally, the processing device can determine the size parameter of the image block in which the first pixel is located, and if the size parameter is greater than a first preset threshold, a second prediction mode can be selected as the first prediction mode in the candidate prediction mode list containing at least one prediction mode. Optionally, if there are multiple second prediction modes in the candidate prediction mode list, the cost value generated when each second prediction mode is used for prediction can be calculated, and one of the multiple second prediction modes can be selected as the first prediction mode according to the cost value. For example, the second prediction mode with the minimum cost is selected as the second prediction mode.

[0226] Optionally, if the size parameter is less than or equal to the first preset threshold, a third prediction mode can be selected as the first prediction mode in the candidate prediction mode list containing at least one prediction mode. Optionally, if there are multiple third prediction modes in the candidate prediction mode list, the cost value generated when each third prediction mode is used for prediction can be determined, and one of the multiple third prediction modes can be selected as the first prediction mode according to the cost value. For example, the third prediction mode with the minimum cost is selected as the second prediction mode.

[0227] Optionally, the processing device can determine a position parameter of the image block in which the first pixel is located, compare the position parameter with a preset position interval, and if the position parameter is within the preset position interval, select one of the second prediction mode and the third prediction mode as the first prediction mode in the candidate prediction mode list containing at least one prediction mode. For example, if the second prediction mode is selected as the first prediction mode, but there are multiple second prediction modes in the candidate prediction mode list, one of the second prediction modes can be selected as the first prediction mode according to the cost value corresponding to each second prediction mode.

[0228] Optionally, if the position parameter is not within the preset position interval, the other one of the second prediction mode and the third prediction mode can be selected as the first prediction mode in the candidate prediction mode list containing at least one prediction mode. For example, if the third prediction mode is selected as the first prediction mode, but there are multiple third prediction modes in the candidate prediction mode list, one of the third prediction modes can be selected as the first prediction mode according to the cost value corresponding to each third prediction mode.

[0229] In this embodiment, by determining or obtaining the first prediction mode from the candidate prediction mode list containing at least one prediction mode according to the size parameter and / or the position parameter of the image block in which the first pixel is located, it can be ensured that the obtained first prediction mode is more suitable for the image block in which the first pixel is located, so that the prediction effect is more accurate when the first pixel is predicted according to the first prediction mode.

[0230] Optionally, the processing device can determine a position parameter of the image block in which the first pixel is located, compare the position parameter with a preset position interval, and if the position parameter is within the preset position interval, select one of the second prediction mode and the third prediction mode as the first prediction mode in the candidate prediction mode list containing at least one prediction mode. For example, if the second prediction mode is selected as the first prediction mode, but there are multiple second prediction modes in the candidate prediction mode list, one of the second prediction modes can be selected as the first prediction mode according to the cost value corresponding to each second prediction mode.

[0231] Optionally, if the processing device is a decoder, the syntax element such as a flag can be obtained from the code stream, and at least one prediction mode is selected as the first prediction mode from the multiple prediction modes according to the flag.

[0232] Optionally, the corresponding relationship between different flags and different prediction modes can be set in advance, the corresponding flag can be determined according to the prediction mode selected by the encoder, and the corresponding code stream can be obtained by encoding. The decoder obtains the corresponding code stream for decoding, determines the prediction mode corresponding to the flag obtained by decoding from the multiple prediction modes, and uses the prediction mode as the first prediction mode to predict the first pixel according to the first prediction mode.

[0233] Optionally, at least one prediction mode can also be selected as the first prediction mode from the candidate prediction mode list containing at least one prediction mode according to the syntax element. For example, the second prediction mode can be selected as the first prediction mode in the candidate prediction mode list.

[0234] In the embodiment, the first prediction mode is determined or obtained from the prediction mode corresponding to the at least one prediction model and / or the candidate prediction mode list containing the prediction mode corresponding to the at least one prediction model according to the syntax elements obtained from the bitstream, so that the first prediction mode obtained by the decoder is consistent with that in the encoder, and the prediction effect is more accurate when the first pixel is predicted according to the first prediction mode.

[0235] Optionally, in the sixth manner, the first pixel is predicted according to the first prediction mode and the reference pixel of the first pixel in the image block, and the first pixel is predicted according to at least one of the fourth to sixth manners.

[0236] In the fourth manner, if the first prediction mode includes the second prediction mode, the first pixel of the image block is predicted according to the second prediction model.

[0237] Optionally, the second prediction mode includes one prediction model, and the prediction model includes at least one of a linear model, a nonlinear model and a gradient model.

[0238] Therefore, when it is determined to predict the first pixel according to the second prediction mode, the prediction model used by the second prediction mode can be used as the second prediction model, and the first pixel of the image block is predicted according to the second prediction model. For example, the first pixel of the image block is predicted according to the linear model, the nonlinear model or the gradient model to obtain the predicted pixel value. Optionally, the reference pixel of the first pixel can be input into the second prediction model, and the predicted value of the first pixel is output.

[0239] In the embodiment, when the first prediction mode includes the second prediction mode, the first pixel can be predicted according to the second prediction model, so that the accuracy and effectiveness of predicting the first pixel are guaranteed.

[0240] In the fifth manner, if the first prediction mode includes the third prediction mode, a part of the first pixels of the image block are predicted according to the third prediction model, and another part of the first pixels of the image block are predicted according to the fourth prediction model.

[0241] Optionally, the image block refers to the image block after division in the video coding process, and the image block is the smallest unit for prediction processing.

[0242] Optionally, the third prediction mode includes at least two prediction models, and the model parameters of the at least two prediction models are different. The prediction model includes at least one of a linear model, a nonlinear model and a gradient model.

[0243] Therefore, when the third prediction mode is determined to be used, each first pixel in the image block can be predicted according to at least two prediction models used in the third prediction mode. For example, when the third prediction mode includes a third prediction model and a fourth prediction model, a part of the first pixels in the image block can be predicted according to the third prediction model, and another part of the first pixels in the image block can be predicted according to the fourth prediction model. Optionally, the third prediction model can be a linear model, a nonlinear model, or a gradient model. The fourth prediction model can be a linear model, a nonlinear model, or a gradient model, and the third prediction model and the fourth prediction model are different.

[0244] For example, as shown in FIG. 11, the first pixels in the image block include a first to-be-predicted pixel and a second to-be-predicted pixel. When the first prediction mode is determined to include the third prediction mode, the first to-be-predicted pixel can be predicted using the third prediction model to obtain a predicted value of the first to-be-predicted pixel, and the second to-be-predicted pixel can be predicted using the fourth prediction model to obtain a predicted value of the second to-be-predicted pixel.

[0245] Optionally, the first prediction mode includes the second prediction mode and the third prediction mode at the same time. Optionally, the third prediction mode includes two prediction models (such as the third prediction model and the fourth prediction model), and the second prediction mode includes one prediction model (such as the second prediction model). For the second prediction model, the second prediction model in the second prediction mode can be Formula Three, that is: Y N+1 = h(Y1, Y2, Y3, …, Y N ) (Formula Three).

[0246] For the third prediction mode, the two prediction models in the third prediction mode can be Formula Four and Formula Five, that is: Y N+1 = f(Y1, Y2, Y3, …, Y N ) (Formula Four); Y N+1 = g(Y1, Y2, Y3, …, Y N ) (Formula Five).

[0247] Y1, Y2, Y3, …, Y N are inputs of the prediction model, that is, reference pixels of the first pixels, such as reference luminance pixels. Y N+1The output of the first prediction model can be a predicted value of the first pixel, for example. The functions corresponding to the three prediction models f(), g() and h() can have different expressions. For example, at least one of f(), g() and h() can be a linear model, and the others can be non-linear models. For another example, one of f(), g() and h() can be a gradient model, another can be a linear model, and the last one can be a non-linear model.

[0248] Optionally, for the third prediction mode, if condition 1 is satisfied, the prediction model corresponding to formula four can be selected for prediction, and if condition 2 is satisfied, the prediction model corresponding to formula five can be selected for prediction. Optionally, condition 1 can be any one of the above modes 1-3, and condition 2 can be any one of the above modes 1-3. And condition 1 and condition 2 are different.

[0249] Optionally, if f(), g() and h() are all linear models, formula four can have an expression as shown in formula six, and formula five can have an expression as shown in formula seven. That is, for the third prediction mode, if condition 1 is satisfied, the prediction of the first pixel is performed according to formula six. N+1 = f(Y1, Y2, Y3,..., YN) = c11*Y1 + c12*Y2 + c13Y3 + c14Y4 + c15Y5 +... + c1N*YN N N (Formula six).

[0250] If condition 2 is satisfied, the prediction of the first pixel is performed according to formula seven. N+1 = g(Y1, Y2, Y3,..., YN) = c21*Y1 + c22*Y2 + c23Y3 + c24Y4 + c25Y5 +... + c2N*YN N N (Formula seven).

[0251] For the second prediction mode, formula three can have an expression as shown in formula eight. N+1 = h(Y1, Y2, Y3,..., YN) = c31*Y1 + c32*Y2 + c33Y3 + c34Y4 + c35Y5 +... + c3N*YN N N (Formula eight).

[0252] Wherein, c11-c1N, c21-c2N, c31-c3N are model parameters (such as filter coefficients and weight parameters). And the model parameters of each prediction model can be different.

[0253] ​​​Optionally, the first prediction mode comprises a third prediction mode. The third prediction mode adopts a multi-linear model, such as two linear models. In this embodiment, the multi-linear model can be pixel-level, such as a part of the first pixels in the image block are predicted by the third prediction model, and another part of the first pixels are predicted by the fourth prediction model.

[0254] In this embodiment, by predicting a part of the first pixels in the image block according to the third prediction model and predicting another part of the first pixels in the image block according to the fourth prediction model when the first prediction mode comprises the third prediction mode, the flexibility of the prediction model is further used, and the accuracy of predicting all the first pixels in the image block is improved.

[0255] In the seventh aspect, the first prediction model is determined according to the reference pixels in the at least one prediction model, and the first pixels are predicted according to the first prediction model.

[0256] In the seventh aspect, the first prediction model is determined according to the reference pixels in the at least one prediction model, and the first pixels are predicted according to the first prediction model.

[0257] The first prediction model is determined or obtained from the at least one prediction model according to the pixel values of the adjacent pixels in the reference pixels, and / or the average values of the at least one pixel in the adjacent regions of the image block, and / or the average values of the at least one reference pixel.

[0258] Optionally, the first prediction model is determined or obtained from the at least one prediction model according to the pixel values of the adjacent pixels in the reference pixels.

[0259] Optionally, the first prediction model is determined or obtained from the at least one prediction model according to the average values of the at least one pixel in the adjacent regions of the image block.

[0260] Optionally, the first prediction model is determined or obtained from the at least one prediction model according to the average values of the at least one reference pixel.

[0261] Optionally, the first prediction model is determined or obtained from the at least one prediction model according to the pixel values of the adjacent pixels in the reference pixels and the average values of the at least one pixel in the adjacent regions of the image block.

[0262] Optionally, the first prediction model is determined or obtained from the at least one prediction model according to the pixel values of the adjacent pixels in the reference pixels and the average values of the at least one reference pixel.

[0263] Optionally, the first prediction model is determined or obtained from the at least one prediction model according to the average values of the at least one pixel in the adjacent regions of the image block and the average values of the at least one reference pixel.

[0264] Optionally, the first prediction model is determined or obtained from the at least one prediction model according to the pixel value of the neighboring pixel in the reference pixel, and the average value of at least one pixel in the neighboring region of the image block, and the average value of the at least one reference pixel.

[0265] Optionally, the at least one prediction model can be selected as the first prediction model according to the reference pixel in a table containing a plurality of prediction models. The pixel value of the neighboring pixel in the reference pixel can be determined, and / or the pixel value of the neighboring pixel, and then the first prediction model is determined or obtained from the at least one prediction model according to the pixel value of the neighboring pixel and / or the neighboring pixel. For example, the pixel value range corresponding to each prediction model can be set in advance, the pixel value of the neighboring pixel and / or the neighboring pixel is matched with the pixel value range, the matched pixel value range is obtained, and the prediction model corresponding to the matched pixel value range is taken as the first prediction model. For example, when the matched pixel value range is a gradient model, the gradient model is taken as the first prediction model.

[0266] Optionally, the preset pixel range corresponding to each prediction model can be set, the reference pixel is matched with the preset pixel range corresponding to each prediction model, the matched preset pixel range is determined, and the prediction model corresponding to the matched preset pixel range is taken as the first prediction model. For example, as shown in Table 2 below, if there are prediction model 0, prediction model 1 and prediction model 2, and the preset pixel range corresponding to the prediction model 0 is 0-100, the preset pixel range corresponding to the prediction model 1 is 100-200, and the preset pixel range corresponding to the prediction model 2 is 200-300. If the pixel value of the reference pixel of the first pixel is 150, the first prediction model can be determined as the prediction model 1. And / or, if the pixel value of the reference pixel of the first pixel is 50, the first prediction model can be determined as the prediction model 0.

[0267] Table 2

[0268] Optionally, the preset pixel value corresponding to each prediction model can be set, and the absolute difference value between the pixel value of the reference pixel and the preset pixel value is determined. The preset pixel value corresponding to the smallest absolute difference value is determined, and the prediction model corresponding to the preset pixel value is taken as the first prediction model. For example, as shown in Table 3 below, if there are prediction model 0, prediction model 1 and prediction model 2, and the preset pixel value corresponding to the prediction model 0 is 100, the preset pixel value corresponding to the prediction model 1 is 200, and the preset pixel value corresponding to the prediction model 2 is 300. If the pixel value of the reference pixel of the first pixel is 190, the first prediction model can be determined as the prediction model 1. And / or, if the pixel value of the reference pixel of the first pixel is 90, the first prediction model can be determined as the prediction model 0.

[0269] Table 3

[0270] Optionally, the average value of at least one pixel in the adjacent region and / or the non-adjacent region of the image block can be determined, and the first prediction model can be determined or obtained according to the average value. For example, the interval of pixel values matching the average value can be determined, and the prediction model corresponding to the interval is taken as the first prediction model.

[0271] Optionally, the average value of the at least one reference pixel can be determined, and a prediction model can be selected as the first prediction model according to the average value of the at least one reference pixel. For example, when the reference pixels include the prediction pixels and the reconstructed pixels, the average value of the at least one reference pixel is calculated by averaging the prediction pixels and the reconstructed pixels, and a prediction model can be selected as the first prediction model according to the average value of the at least one reference pixel. For example, the average value is compared with the preset pixel value and / or the preset pixel range corresponding to each prediction model, if there is a preset pixel range matching the average value, the prediction model corresponding to the preset pixel range is taken as the first prediction model. And / or, if there is a preset pixel value with the highest similarity to the average value, the prediction model corresponding to the preset pixel value is taken as the first prediction model.

[0272] Optionally, the average value of at least one pixel in the adjacent region of the image block can be taken as the first value, and the average value of the at least one reference pixel can be taken as the second value. The first prediction model for determining the prediction value of the first pixel can be determined from the at least one prediction model according to the pixel values of the adjacent pixels of the first pixel, the at least one first value, and / or the at least one second value.

[0273] Optionally, if the pixel values of the adjacent pixels of the first pixel and the first value satisfy the first condition, the prediction value of the first pixel can be determined by using the first prediction model. And / or, if the pixel values of the adjacent pixels of the first pixel and the first value do not satisfy the first condition, the prediction value of the first pixel can be determined by using the second prediction model. For example, if the pixel values of at least one prediction pixel and / or reconstructed pixel in the adjacent pixels of the first pixel are greater than or equal to the first value, the first pixel can be predicted by using the first prediction model to obtain the prediction value of the first pixel. If the pixel values of at least one prediction pixel and / or reconstructed pixel in the adjacent pixels of the first pixel are less than the first value, the first pixel can be predicted by using the second prediction model to obtain the prediction value of the first pixel. As shown in FIG. 12, the first value can be the reconstructed pixel value of the pixel in the adjacent region adjacent to the block (i.e., the image block) to be predicted, and the adjacent region includes the left region, the upper-left region, and the upper region.

[0274] Optionally, if the first value and the second value satisfy the first condition, the first prediction model can be used to determine the prediction value of the first pixel. And / or, if the first value and the second value do not satisfy the first condition, the second prediction model can be used to determine the prediction value of the first pixel. For example, if the second value is greater than or equal to the first value, the first prediction model can be used to predict the first pixel to obtain the prediction value of the first pixel. And / or, if the second value is less than the first value, the second prediction model can be used to predict the first pixel to obtain the prediction value of the first pixel.

[0275] Optionally, if the pixel value of the neighboring pixel of the first pixel and the second value satisfy the first condition, the first prediction model can be used to determine the prediction value of the first pixel. And / or, if the pixel value of the neighboring pixel of the first pixel and the second value do not satisfy the first condition, the second prediction model can be used to determine the prediction value of the first pixel. For example, if the pixel value of at least one of the predicted pixel and / or the reconstructed pixel in the neighboring pixel of the first pixel is greater than or equal to the second value, the first prediction model can be used to predict the first pixel to obtain the prediction value of the first pixel. And / or, if the pixel value of at least one of the predicted pixel and / or the reconstructed pixel in the neighboring pixel of the first pixel is less than the second value, the second prediction model can be used to predict the first pixel to obtain the prediction value of the first pixel.

[0276] Optionally, the first prediction model and / or the second prediction model can be any one of a gradient model, a linear model and a nonlinear model. Optionally, the first prediction model and the second prediction model are different.

[0277] Optionally, at least one prediction model can be selected as the first prediction model according to the first condition from a plurality of prediction models, and the reconstructed pixel value and / or the predicted pixel value of the reference pixel of the first pixel are input into the first prediction model for prediction to obtain the prediction value of the first pixel.

[0278] In the embodiment, by determining or obtaining the first prediction model from at least one prediction model according to the pixel value of the neighboring pixel in the reference pixel, and / or the average value of at least one pixel in the neighboring region of the image block, and / or the average value of at least one reference pixel, the accuracy and effectiveness of the determined first prediction model are guaranteed.

[0279] Optionally, the model parameters of the prediction model of the image block are determined or obtained according to the model parameters of the prediction model of the first reference block, and the first pixel is predicted according to the reference pixel and the model parameters of the prediction model of the image block.

[0280] Optionally, the first reference block can be determined or obtained according to any one of the following modes 9 to 13. When the first pixel is predicted by using the prediction model, the model parameters of the prediction model can be inherited from the model parameters of the prediction model of the first reference block. For example, the model parameters can be inherited by using a candidate block vector, by using a motion vector, by using a temporal neighboring motion vector, a non-neighboring motion vector, by using chained motion vector prediction (CMVP) and chained block vector prediction (CBVP).

[0281] Therefore, after the model parameters of the prediction model of the first reference block are determined, the model parameters can be used as the model parameters of the prediction model of the image block to be predicted. Optionally, default model parameters can also be obtained, and the default model parameters and / or the model parameters of the prediction model of the first reference block can be used as the model parameters of the prediction model of the image block to be predicted.

[0282] Optionally, the reference pixels of the first pixel can be determined according to any one of the following modes 1 to 5.

[0283] Optionally, the prediction model is input with the reference pixels of the first pixel and the model parameters of the prediction model of the image block to obtain the prediction value of the first pixel.

[0284] In this embodiment, the model parameters of the prediction model of the image block are determined or obtained according to the model parameters of the prediction model of the first reference block, and the first pixel is predicted according to the reference pixels and the model parameters of the prediction model of the image block. Therefore, the inheritance function of the model parameters can be realized, and the phenomenon of fixed model parameters can be avoided, thereby improving the accuracy and effectiveness of the prediction of the first pixel.

[0285] Optionally, the first reference block can be determined or obtained according to any one of the following modes 9 to 13.

[0286] Mode 9: the first reference block is determined or obtained according to the block vectors and / or motion vectors of the neighboring image blocks and / or non-neighboring image blocks of the image block.

[0287] Optionally, the processing device can store the used model parameters (such as filter coefficients) in the storage unit, so as to be used when selecting the corresponding prediction mode and / or prediction model for the image block to be predicted. For example, the model parameters of the image block to be predicted can be determined by using a candidate block vector. The candidate block vector can be the block vector corresponding to the neighboring image blocks and / or non-neighboring image blocks of the image block to be predicted.

[0288] For example, as shown in Fig. 13, in the current image, the reference block of the to-be-predicted image block (i.e., the first reference block) is determined by the candidate block vector. For example, the position of the reference block of the to-be-predicted image block at (x0, y0) is determined by the BV (block vector) to be at (x1, y1). If the reference block uses the first prediction mode in the EIP prediction mode of the embodiment of the present application, the to-be-predicted image block uses the first prediction mode in the EIP prediction mode of the embodiment of the present application. And the model parameter of the prediction model used by the reference block is used as the model parameter of the prediction model used by the to-be-predicted image block. If the reference block uses the third prediction mode, and the third prediction mode includes the model parameter 1 of the prediction model f() corresponding to when condition 1 is satisfied, and the model parameter 2 of the prediction model g() corresponding to when condition 2 is satisfied, the to-be-predicted image block can inherit the model parameter 1 and the model parameter 2, and use the model parameter 1 and the model parameter 2 as the respective model parameters of f() and g() of the third prediction mode of the to-be-predicted image block.

[0289] Optionally, in the current image, the first reference block of the to-be-predicted image block can be determined by the candidate motion vector. The candidate motion vector can include the motion vector of the adjacent image block and / or the non-adjacent image block. And the model parameter of the prediction model used by the first reference block is used to determine the model parameter of the to-be-predicted image block.

[0290] For example, as shown in Fig. 14, in the current image, the motion vector of the adjacent image block of the to-be-predicted image block is determined, for example, the position of the reference image block of the to-be-predicted image block at (x0, y0) in the current image is determined by the MV (motion vector) to be at (x1, y1) in the reference image. If the reference block uses the first prediction mode in the EIP prediction mode of the embodiment of the present application. And the model parameter of the prediction model used by the reference block is used as the model parameter of the to-be-predicted image block. If the reference block uses the third prediction mode, and the third prediction mode includes the model parameter 1 of the prediction model f() corresponding to when condition 1 is satisfied, and the model parameter 2 of the prediction model g() corresponding to when condition 2 is satisfied, the to-be-predicted image block can inherit the model parameter 1 and the model parameter 2, and use the model parameter 1 and the model parameter 2 as the respective model parameters of f() and g() of the third prediction mode of the to-be-predicted image block.

[0291] In the embodiment, the first reference block is determined or obtained according to the block vector and / or the motion vector of the adjacent image block and / or the non-adjacent image block of the image block, and the model parameter of the prediction model of the image block is determined or obtained according to the model parameter of the prediction model of the first reference block, so as to predict the first pixel according to the model parameter and the reference pixel, to guarantee the accuracy of predicting the first pixel.

[0292] The tenth mode is to determine or obtain the first reference block according to the block vector and / or motion vector of the neighboring pixels and / or non-neighboring pixels of the image block;

[0293] Optionally, the processing device can determine the neighboring pixels and / or non-neighboring pixels of the image block, and the pixel positions of the neighboring pixels and / or non-neighboring pixels are not limited, such as the neighboring pixels of the upper neighboring region of the image block, the neighboring pixels of the left neighboring region of the image block, the non-neighboring pixels of the upper non-neighboring region of the image block.

[0294] Optionally, the processing device can determine the neighboring image block in which the neighboring pixels of the image block are located, perform block vector and / or motion vector calculation on the neighboring image block, determine the reference block of the neighboring image block according to the calculation result, and use the reference block as the first reference block. Further, the model parameters of the prediction model used when the first prediction mode is used on the first reference block are used to determine the model parameters of the prediction model of the image block to be predicted.

[0295] Optionally, the processing device can determine the non-neighboring image block in which the non-neighboring pixels of the image block are located, perform block vector and / or motion vector calculation on the non-neighboring image block, determine the reference block of the non-neighboring image block according to the calculation result, and use the reference block as the first reference block. Further, the model parameters of the prediction model used when the first prediction mode is used on the first reference block are used to determine the model parameters of the prediction model of the image block to be predicted.

[0296] In this embodiment, by determining or obtaining the first reference block according to the block vector and / or motion vector of the neighboring pixels and / or non-neighboring pixels of the image block, and further determining or obtaining the model parameters of the prediction model of the image block according to the model parameters of the prediction model of the first reference block, the model parameters of the prediction model of the image block are determined or obtained, so as to predict the first pixel according to the model parameters and the reference pixels, so as to guarantee the accuracy of predicting the first pixel.

[0297] The eleventh mode is to determine or obtain the first reference block according to the block vector and / or motion vector of at least one of the following: the collocated block of the image block, the neighboring image block and / or non-neighboring image block of the collocated block, the temporal block, the neighboring image block and / or non-neighboring image block of the temporal block, the first component image block, the neighboring image block and / or non-neighboring image block of the first component image block, the second component image block, and the neighboring image block and / or non-neighboring image block of the second component image block;

[0298] Optionally, the collocated block, temporal block, first component image block, and second component image block of the image block can refer to the fourth mode in the above embodiments, which will not be described here.

[0299] Optionally, the processing device can determine at least one of the collocated block, the temporal block, the first component image block and the second component image block of the image block to be predicted, determine the block vector and / or the motion vector corresponding to the at least one of the collocated block, the temporal block, the first component image block and the second component image block of the image block to be predicted, determine the reference block of the image block to be predicted through the block vector and / or the motion vector, and take the reference block as the first reference block, determine the model parameter used when the first reference block uses the first prediction mode, and take the model parameter as the model parameter of the image block to be predicted.

[0300] Optionally, after determining the collocated block and / or the temporal block of the image block, the processing device can determine the neighboring image block and / or the non-neighboring image block of the collocated block and / or the temporal block, and perform the calculation of the block vector and / or the motion vector on the neighboring image block and / or the non-neighboring image block of the collocated block and / or the temporal block to obtain the first reference block, determine the model parameter of the prediction model used when the first reference block uses the first prediction mode, and take the model parameter as the model parameter of the image block to be predicted.

[0301] Optionally, after determining the first component image block and / or the second component image block of the image block, the neighboring image block and / or the non-neighboring image block of the first component image block and / or the second component image block can be determined. For example, if the first component image block is a luminance image block and the second component image block is a chrominance image block, the neighboring image block and / or the non-neighboring image block of the luminance image block in the luminance component can be determined, and the neighboring image block and / or the non-neighboring image block of the chrominance image block in the chrominance component can be determined.

[0302] Optionally, the neighboring image block and / or the non-neighboring image block of the first component image block and / or the second component image block can be subjected to the calculation of the block vector and / or the motion vector to obtain the first reference block, and the model parameter of the prediction model used when the first reference block uses the first prediction mode can be determined, and the model parameter is taken as the model parameter of the image block to be predicted.

[0303] Optionally, if the number of block vectors and / or motion vectors obtained through any method is multiple, the multiple block vectors and / or motion vectors can be subjected to clustering processing. The block vectors and / or motion vectors are divided into multiple groups or clusters. Then, the reference block of the image block to be predicted is preferably determined through the block vector and / or the motion vector in the group with the largest number of block vectors and / or motion vectors, and the determined reference block is taken as the first reference block.

[0304] In the embodiment, the first reference block is determined or obtained according to at least one of the block vector and / or the motion vector of the collocated block of the image block, the neighboring image block and / or the non-neighboring image block of the collocated block, the temporal block, the neighboring image block and / or the non-neighboring image block of the temporal block, the first component image block, the neighboring image block and / or the non-neighboring image block of the first component image block, the second component image block and the neighboring image block and / or the non-neighboring image block of the second component image block, and the model parameter of the prediction model of the image block is determined or obtained according to the model parameter of the prediction model of the first reference block, so that the first pixel is predicted according to the model parameter and the reference pixel, thereby ensuring the accuracy of the prediction of the first pixel.

[0305] In the twelfth mode, the first reference block is determined or obtained according to at least one of the block vector and / or the motion vector of the neighboring pixel and / or the non-neighboring pixel of the collocated block of the image block, the neighboring pixel and / or the non-neighboring pixel of the temporal block, the neighboring pixel and / or the non-neighboring pixel of the first component image block, and the neighboring pixel and / or the non-neighboring pixel of the second component image block.

[0306] Optionally, the processing device can determine the collocated block corresponding to the image block, determine the first neighboring image block in which the neighboring pixel of the collocated block is located, perform block vector and / or motion vector calculation on the first neighboring image block, determine the reference block of the first neighboring image block according to the calculation result, and take the reference block as the first reference block, and determine the model parameter of the image block to be predicted according to the model parameter of the prediction model used when the first prediction mode is used for the first reference block.

[0307] Optionally, the processing device can determine the collocated block corresponding to the image block, determine the first non-neighboring image block in which the non-neighboring pixel of the collocated block is located, perform block vector and / or motion vector calculation on the first non-neighboring image block, determine the reference block of the first non-neighboring image block according to the calculation result, and take the reference block as the first reference block, and determine the model parameter of the image block to be predicted according to the model parameter of the prediction model used when the first prediction mode is used for the first reference block.

[0308] The processing device can determine the temporal block corresponding to the image block, determine the second neighboring image block in which the neighboring pixel of the temporal block is located, perform block vector and / or motion vector calculation on the second neighboring image block, determine the reference block of the second neighboring image block according to the calculation result, and take the reference block as the first reference block, and determine the model parameter of the image block to be predicted according to the model parameter of the prediction model used when the first prediction mode is used for the first reference block.

[0309] Optionally, the processing device can determine a time-domain block corresponding to the image block, determine a second non-adjacent image block in which non-adjacent pixels of the time-domain block are located, perform block vector and / or motion vector calculation on the second non-adjacent image block, determine a reference block of the second non-adjacent image block according to the calculation result, and use the reference block as the first reference block. The model parameters of the prediction model used when the first prediction mode is used on the first reference block are used to determine the model parameters of the prediction model of the image block to be predicted.

[0310] The processing device can determine a first component image block or a second component image block corresponding to the image block, determine a third adjacent image block in which adjacent pixels of the first component image block or the second component image block are located, perform block vector and / or motion vector calculation on the third adjacent image block, determine a reference block of the third adjacent image block according to the calculation result, and use the reference block as the first reference block. The model parameters of the prediction model used when the first prediction mode is used on the first reference block are used to determine the model parameters of the prediction model of the image block to be predicted.

[0311] Optionally, the processing device can determine a first component image block or a second component image block corresponding to the image block, determine a third non-adjacent image block in which non-adjacent pixels of the first component image block or the second component image block are located, perform block vector and / or motion vector calculation on the third non-adjacent image block, determine a reference block of the third non-adjacent image block according to the calculation result, and use the reference block as the first reference block. The model parameters of the prediction model used when the first prediction mode is used on the first reference block are used to determine the model parameters of the prediction model of the image block to be predicted.

[0312] In this embodiment, the first reference block is determined or obtained according to the block vector and / or motion vector of at least one of the adjacent pixels and / or non-adjacent pixels of the collocated block of the image block, the adjacent pixels and / or non-adjacent pixels of the time-domain block, the adjacent pixels and / or non-adjacent pixels of the first component image block, the adjacent pixels and / or non-adjacent pixels of the second component image block, and the model parameters of the prediction model of the first reference block are used to determine or obtain the model parameters of the prediction model of the image block, so as to predict the first pixel according to the model parameters and the reference pixels, thereby ensuring the accuracy of predicting the first pixel.

[0313] Thirteenth, the first reference block is determined or obtained according to the block vector and / or motion vector of at least one of the reference block of the image block, the reference block of the adjacent image block, the reference block of the non-adjacent image block, the reference block of the collocated block, the reference block of the time-domain block, the reference block of the first component image block, and the reference block of the second component image block.

[0314] Optionally, the reference block of the image block, the reference block of the neighboring image block, the reference block of the non-neighboring image block, the reference block of the collocated block, the reference block of the temporal block, the reference block of the first component image block and the reference block of the second component image block are determined in the same way. The following is only exemplified by the way of determining the reference block of the image block.

[0315] Optionally, the reference block of the image block is determined by at least one of the following:

[0316] The reference block of the image block can be determined or obtained according to the block vector and / or motion vector of the neighboring image block and / or non-neighboring image block of the image block;

[0317] The reference block of the image block is determined or obtained according to the block vector and / or motion vector of the neighboring pixel and / or non-neighboring pixel of the image block;

[0318] The reference block of the image block is determined or obtained according to the block vector and / or motion vector of at least one of the following: the collocated block of the image block, the neighboring image block and / or non-neighboring image block of the collocated block, the temporal block, the neighboring image block and / or non-neighboring image block of the temporal block, the first component image block, the neighboring image block and / or non-neighboring image block of the first component image block, the second component image block and the neighboring image block and / or non-neighboring image block of the second component image block;

[0319] The reference block of the image block is determined or obtained according to the block vector and / or motion vector of at least one of the following: the neighboring pixel and / or non-neighboring pixel of the collocated block of the image block, the neighboring pixel and / or non-neighboring pixel of the temporal block, the neighboring pixel and / or non-neighboring pixel of the first component image block, the neighboring pixel and / or non-neighboring pixel of the second component image block.

[0320] Optionally, the block vector and / or motion vector calculation is performed on at least one of the following: the reference block of the image block, the reference block of the neighboring image block, the reference block of the non-neighboring image block, the reference block of the collocated block, the reference block of the temporal block, the reference block of the first component image block and the reference block of the second component image block, to obtain the block vector calculation result and / or the motion vector calculation result.

[0321] Optionally, the image block corresponding to the block vector calculation result and / or the motion vector calculation result is determined as the first reference block.

[0322] For example, as shown in Fig. 15, the reference block 1 is determined by the candidate block vector of the to-be-predicted image block. For example, the position of the reference block of the to-be-predicted image block in the current image is (x1, y1) by the block vector BV1, where the position of the to-be-predicted image block in the current image is (x0, y0). Then, the reference block 2 is determined by the candidate block vector of the reference block 1. For example, the position of the reference block b of the reference block a in the current image is (x2, y2) by BV1 (block vector). The model parameters of the to-be-predicted image block are determined by the model parameters of the reference block 2. That is, if the reference block 2 uses the first prediction mode in the EIP prediction mode of the embodiment of the present application, the to-be-predicted image block uses the first prediction mode in the EIP prediction mode of the embodiment of the present application. And the model parameters corresponding to the first prediction mode when the reference block 2 uses the first prediction mode are used as the model parameters of the to-be-predicted image block. If the reference block 2 uses the third prediction mode, and the third prediction mode includes the model parameter 1 of the prediction model f() corresponding to the condition 1 being satisfied, and the model parameter 2 of the prediction model g() corresponding to the condition 2 being satisfied. The to-be-predicted image block can inherit the model parameter 1 and the model parameter 2, and use the model parameter 1 and the model parameter 2 as the model parameters of f() and g() respectively in the third prediction mode of the to-be-predicted image block.

[0323] Optionally, the candidate BV of the to-be-predicted image block is the BV of the spatially adjacent / non-adjacent block of the to-be-predicted image block, or the BV of the collocated block of the to-be-predicted block or the BV of the spatially adjacent or non-adjacent block of the collocated block. The candidate block vector of the reference block 1 can be the BV of the spatially adjacent / non-adjacent block of the reference block 1, the BV of the collocated block or the BV of the spatially adjacent or non-adjacent block of the collocated block, or the BV of the reference block 1 itself.

[0324] For example, as shown in FIG. 16, the reference block a can be determined by the candidate motion vector of the to-be-predicted image block. For example, the position of the reference block a of the to-be-predicted image block in the current image is (xa, ya) and the position of the reference block a in the reference image a is (xb, yb) determined by MVa (motion vector). Then, the reference block b is determined by the candidate motion vector of the reference block a. For example, the position of the reference block b of the reference block a in the reference image b is (x0, y0) determined by MVb (motion vector). The model parameter of the to-be-predicted image block is determined by the model parameter of the reference block b. That is, if the reference block b uses the first prediction mode in the EIP prediction mode of the embodiment of the present application, the to-be-predicted image block uses the first prediction mode in the EIP prediction mode of the embodiment of the present application. And the model parameter corresponding to the use of the first prediction mode by the reference block b is used as the model parameter of the to-be-predicted image block. If the reference block b uses the third prediction mode, and the third prediction mode includes the model parameter 1 of f() corresponding to the satisfaction of condition 1 and the model parameter 2 of g() corresponding to the satisfaction of condition 2, the to-be-predicted image block can inherit the model parameter 1 and the model parameter 2 and use the model parameter 1 and the model parameter 2 as the model parameters of f() and g() respectively in the third prediction mode of the to-be-predicted image block.

[0325] In the embodiment, the model parameter of the prediction model of the image block is determined or obtained according to the model parameter of the prediction model of the first reference block, which is determined or obtained according to at least one of the block vector and / or the motion vector of the reference block of the image block, the reference block of the neighboring image block, the reference block of the non-neighboring image block, the reference block of the co-located block, the reference block of the temporal block, the reference block of the first component image block and the reference block of the second component image block, so that the accuracy of the prediction of the first pixel can be improved.

[0326] The embodiment of the present application further provides an image processing device, referring to FIG. 17, the image processing device comprises:

[0327] The processing module A10 is used for predicting the first pixel according to the reference pixel and the prediction model of the image block.

[0328] Optionally, the reference pixel comprises a neighboring pixel and / or a neighboring pixel, and / or the reference pixel is determined or obtained in the following manner:

[0329] The reference pixel is determined or obtained according to at least one of the upper neighboring pixel, the upper non-neighboring pixel, the left neighboring pixel, the left non-neighboring pixel, the upper left neighboring pixel and the upper left non-neighboring pixel of the image block;

[0330] The reference pixel is determined or obtained according to the neighboring region and / or the non-neighboring region of the image block;

[0331] determine or obtain the reference pixel according to at least one of the default block, the neighbor block, the non-neighbor block, the collocated block and the temporal block corresponding to the image block;

[0332] determine or obtain the reference pixel according to at least one of the default block, the neighbor block, the non-neighbor block, the collocated block and the temporal block corresponding to the image block;

[0333] determine or obtain the reference pixel according to at least one of the default block, the neighbor block, the non-neighbor block, the collocated block and the temporal block corresponding to the image block;

[0334] Optionally, the processing module A10 is further configured to perform at least one of the following:

[0335] determine or obtain the first prediction mode from the prediction modes corresponding to the at least one prediction model, and predict the first pixel according to the first prediction mode and the reference pixel in the image block;

[0336] determine the first prediction model according to the reference pixel in the at least one prediction model, and predict the first pixel according to the first prediction model;

[0337] determine or obtain the model parameter of the prediction model of the image block according to the model parameter of the prediction model of the first reference block, and predict the first pixel according to the reference pixel and the model parameter of the prediction model of the image block.

[0338] Optionally, the determining or obtaining the first prediction mode from the prediction modes corresponding to the at least one prediction model comprises at least one of the following:

[0339] determine or obtain the first prediction mode from the at least one prediction mode according to the size parameter and / or the position parameter of the image block in which the first pixel is located;

[0340] determine or obtain the first prediction mode from the candidate prediction mode list containing the at least one prediction mode according to the size parameter and / or the position parameter of the image block in which the first pixel is located;

[0341] determine or obtain the first prediction mode from the prediction modes corresponding to the at least one prediction model and / or the candidate prediction mode list containing the prediction modes corresponding to the at least one prediction model according to the syntax element obtained from the bitstream.

[0342] Optionally, the first prediction mode comprises a second prediction mode, and the second prediction mode comprises one prediction model;

[0343] Optionally, the first prediction mode comprises a third prediction mode, and the third prediction mode comprises at least two prediction models, and the model parameters of the at least two prediction models are different;

[0344] Optionally, the prediction model comprises at least one of a linear model, a non-linear model and a gradient model.

[0345] Optionally, the prediction of the first pixel according to the first prediction mode and the reference pixel within the image block comprises at least one of:

[0346] if the first prediction mode comprises a second prediction mode, the first pixel of the image block is predicted according to a second prediction model;

[0347] if the first prediction mode comprises a third prediction mode, a part of the first pixels of the image block is predicted according to a third prediction model, and another part of the first pixels of the image block is predicted according to a fourth prediction model.

[0348] Optionally, the determination of the first prediction model from the at least one prediction model according to the reference pixel comprises:

[0349] the first prediction model is determined or derived from the at least one prediction model according to a pixel value of a neighboring pixel of the reference pixel, and / or an average value of at least one pixel in a neighboring region of the image block, and / or an average value of the at least one reference pixel.

[0350] Optionally, the determination or derivation of the first reference block comprises at least one of:

[0351] the first reference block is determined or derived according to a block vector and / or a motion vector of a neighboring block and / or a non-neighboring block of the image block;

[0352] the first reference block is determined or derived according to a block vector and / or a motion vector of a neighboring pixel and / or a non-neighboring pixel of the image block;

[0353] the first reference block is determined or derived according to a block vector and / or a motion vector of at least one of a collocated block of the image block, a neighboring block and / or a non-neighboring block of the collocated block, a temporal block, a neighboring block and / or a non-neighboring block of the temporal block, a first component image block, a neighboring block and / or a non-neighboring block of the first component image block, a second component image block and a neighboring block and / or a non-neighboring block of the second component image block;

[0354] the first reference block is determined or derived according to a block vector and / or a motion vector of at least one of a neighboring pixel and / or a non-neighboring pixel of the collocated block of the image block, a neighboring pixel and / or a non-neighboring pixel of the temporal block, a neighboring pixel and / or a non-neighboring pixel of the first component image block, a neighboring pixel and / or a non-neighboring pixel of the second component image block;

[0355] The first reference block is determined or obtained according to a block vector and / or a motion vector of at least one of the following: the reference block of the image block, the reference block of the neighboring image block, the reference block of the non-neighboring image block, the reference block of the collocated block, the reference block of the temporal block, the reference block of the first component image block, and the reference block of the second component image block.

[0356] The image processing apparatus provided by the embodiments of the present application has similar implementation principles and beneficial effects to the technical solutions shown in the corresponding method embodiments described above, and thus will not be described here in detail.

[0357] The embodiments of the present application further provide a processing device, including a memory and a processor, and the memory stores a processing program, and the processing program is executed by the processor to implement the steps of the image processing method in any of the embodiments described above.

[0358] The embodiments of the present application further provide a storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the image processing method in any of the embodiments described above.

[0359] In the embodiments of the processing device and the storage medium provided by the present application, all the technical features of any of the above image processing method embodiments can be included, and the description and explanation contents are basically the same as those of the above method embodiments, and thus will not be described here in detail.

[0360] The embodiments of the present application further provide a computer program product, and the computer program product includes computer program code, which, when running on a computer, causes the computer to execute the method in various possible embodiments.

[0361] The embodiments of the present application further provide a chip, including a memory and a processor, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the device installed with the chip executes the method in various possible embodiments.

[0362] It can be understood that the above scenarios are only examples and do not constitute a limitation on the application scenarios of the technical solutions provided by the embodiments of the present application. The technical solutions provided by the embodiments of the present application are also applicable to other scenarios. For example, those skilled in the art can know that, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0363] The above sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0364] The steps in the embodiments of the present application can be adjusted, combined and reduced in sequence according to actual needs.

[0365] The units in the device of the embodiments of the present application can be combined, divided, and deleted according to actual needs.

[0366] In the present application, for the same or similar term concept, technical solution and / or application scenario description, generally only the first time is described in detail, and the repeated description is not repeated in order to be brief, and when understanding the technical solutions of the present application, the same or similar term concept, technical solution and / or application scenario description which is not described in detail can be referred to the previous related description.

[0367] In the present application, the description of each embodiment has its own focus, and the part not described or recorded in a certain embodiment can be referred to the related description of other embodiments.

[0368] The technical features of the technical solutions of the present application can be combined arbitrarily, in order to make the description simple, the above-mentioned technical features of each embodiment are not described all possible combinations, however, as long as the combination of these technical features does not exist contradictory, should be considered as the range recorded in the present application.

[0369] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment method can be realized by software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is the better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of software product, and the computer software product is stored in the above-mentioned storage medium (such as ROM / RAM, magnetic disc, optical disc), including a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, controlled terminal, or network device, etc.) execute the method of each embodiment of the present application.

[0370] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present application are generated. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in a storage medium or transferred from one storage medium to another storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) mode. The storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (such as floppy disk, storage disk, magnetic tape), optical media (such as DVD), or semiconductor media (such as solid state disk (SSD)) and the like.

[0371] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the specification and drawings of the present application, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

An image processing method, wherein, The method comprises the steps of: S10, predicting the first pixel according to a reference pixel of the first pixel in the image block and a prediction model. The method of claim 1, wherein, The reference pixel comprises a neighboring pixel and / or a neighboring pixel, and / or the reference pixel is determined or obtained in the following manner: The reference pixel is determined or obtained according to at least one of the following: an upper neighboring pixel, an upper non-neighboring pixel, a left neighboring pixel, a left non-neighboring pixel, an upper-left neighboring pixel and an upper-left non-neighboring pixel of the image block; The reference pixel is determined or obtained according to a neighboring region and / or a non-neighboring region of the image block; The reference pixel is determined or obtained according to a first component image block and / or a second component image block of the image block; The reference pixel is determined or obtained according to at least one of the following: a default block, a neighbor block, a non-neighbor block, a collocated block and a temporal block corresponding to the image block; The reference pixel is determined or obtained according to a neighboring pixel and / or a neighboring pixel of a first pixel combination obtained according to at least one first pixel combination of the image block. The method of claim 1, wherein, The step S10 comprises at least one of the following: A first prediction mode is determined or obtained from a prediction mode corresponding to at least one prediction model, and the first pixel is predicted according to the first prediction mode and the reference pixel of the first pixel in the image block; The first prediction model is determined according to the reference pixel in at least one prediction model, and the first pixel is predicted according to the first prediction model; The model parameters of the prediction model of the image block are determined or obtained according to the model parameters of the prediction model of the first reference block, and the first pixel is predicted according to the reference pixel and the model parameters of the prediction model of the image block. The method of claim 3, wherein, The first prediction mode is determined or obtained from a prediction mode corresponding to at least one prediction model, and the first prediction mode comprises at least one of the following: The first prediction mode is determined or obtained from at least one prediction mode according to a size parameter and / or a position parameter of the image block in which the first pixel is located; The first prediction mode is determined or obtained from a candidate prediction mode list containing at least one prediction mode according to a size parameter and / or a position parameter of the image block in which the first pixel is located; The first prediction mode is determined or obtained from a prediction mode corresponding to at least one prediction model and / or a candidate prediction mode list containing a prediction mode corresponding to at least one prediction model according to a syntax element obtained from a bitstream. The method of claim 3, wherein, Further comprising at least one of the following: The first prediction mode comprises a second prediction mode, and the second prediction mode comprises one prediction model; The first prediction mode comprises a third prediction mode, and the third prediction mode comprises at least two prediction models, and the model parameters of the at least two prediction models are different; The prediction model comprises at least one of the following: a linear model, a nonlinear model and a gradient model. The method of claim 5, wherein, The first pixel is predicted according to the first prediction mode and the reference pixel of the first pixel in the image block, and the prediction comprises at least one of the following: If the first prediction mode comprises a second prediction mode, the first pixel of the image block is predicted according to the second prediction model. If the first prediction mode comprises a third prediction mode, a part of the first pixels of the image block is predicted according to a third prediction model, and another part of the first pixels of the image block is predicted according to a fourth prediction model. The method of claim 3, wherein, The determining the first prediction model from the at least one prediction model according to the reference pixels comprises: The first prediction model is determined or derived from the at least one prediction model according to the pixel values of the neighboring pixels of the reference pixels, and / or the average values of the at least one pixel in the neighboring regions of the image block, and / or the average values of the at least one reference pixel. The method of claim 3, wherein, The determining or deriving the first reference block comprises at least one of: The first reference block is determined or derived according to the block vectors and / or motion vectors of the neighboring image blocks and / or non-neighboring image blocks of the image block; The first reference block is determined or derived according to the block vectors and / or motion vectors of the neighboring pixels and / or non-neighboring pixels of the image block; The first reference block is determined or derived according to the block vectors and / or motion vectors of at least one of the following: the collocated block of the image block, the neighboring image blocks and / or non-neighboring image blocks of the collocated block, the temporal block, the neighboring image blocks and / or non-neighboring image blocks of the temporal block, the first component image block, the neighboring image blocks and / or non-neighboring image blocks of the first component image block, the second component image block, and the neighboring image blocks and / or non-neighboring image blocks of the second component image block; The first reference block is determined or derived according to the block vectors and / or motion vectors of at least one of the following: the neighboring pixels and / or non-neighboring pixels of the collocated block of the image block, the neighboring pixels and / or non-neighboring pixels of the temporal block, the neighboring pixels and / or non-neighboring pixels of the first component image block, and the neighboring pixels and / or non-neighboring pixels of the second component image block; The first reference block is determined or derived according to the block vectors and / or motion vectors of at least one of the following: the reference block of the image block, the reference block of the neighboring image block, the reference block of the non-neighboring image block, the reference block of the collocated block, the reference block of the temporal block, the reference block of the first component image block, and the reference block of the second component image block. The image processing device comprises: A processing device, wherein A memory and a processor, wherein the memory stores a processing program, and the processing program is executed by the processor to implement the steps of the image processing method according to claim 1. The storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the image processing method according to claim 1. A storage medium, wherein, ​

Images