Processing method, processing device, and storage medium

Abstract

According to the technical solution of the present application, a second prediction value of a first image block is determined or obtained on the basis of a first prediction value of the first image block, solving the technical problem of low illumination compensation efficiency in image processing scenarios. By means of the technical solution of the present application, the prediction quality of the first image block can be improved, thereby improving video encoding and / or decoding performance.

Description

Processing method, processing device and storage medium TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a processing method, a processing device and a storage medium. BACKGROUND

[0002] The existing high-efficiency video coding standard protocol (H.266 / VVC) proposes a video frame coding technology to improve coding performance without significantly increasing computational complexity. For example, when coding a video frame, the protocol divides each frame into different blocks, performs prediction, transformation and quantization processing on different blocks, and then performs coding processing.

[0003] In the process of conceiving and implementing the present application, the inventors have found that at least the following problems exist: When coding, the efficiency of illumination compensation is low, which limits the efficiency of video encoding and / or decoding.

[0004] The foregoing description is intended to provide general background information and does not necessarily constitute prior art.

[0005] SUMMARY

[0006] The main purpose of the present application is to provide a processing method, a processing device and a storage medium to solve the problems existing in the picture processing field, which can improve the efficiency of illumination compensation and thus improve the performance of video encoding and / or decoding.

[0007] The present application provides a processing method, which can be applied to a processing device, comprising the steps of:

[0008] S10, determining or obtaining a second prediction value of the first image block according to a first prediction value of the first image block.

[0009] Optionally, the first prediction value is determined or obtained according to at least one of the following:

[0010] An inter prediction mode and / or an intra prediction mode;

[0011] At least one of a pixel value, a position, motion information and a prediction mode of the image block;

[0012] A first flag;

[0013] A first parameter;

[0014] A candidate element;

[0015] At least two prediction values and corresponding weight coefficients.

[0016] Optionally, the method further comprises at least one of the following:

[0017] The first flag is an illumination compensation flag;

[0018] The first parameter is an illumination compensation parameter;

[0019] The candidate element includes a first candidate element and a second candidate element;

[0020] The candidate element includes at least one of a motion vector, a block vector, a motion vector candidate, a block vector candidate, a motion vector predictor, a block vector predictor, a motion vector index, a block vector index;

[0021] The candidate element is a candidate element in a candidate list corresponding to a lowest matching cost between a current template and a reference template;

[0022] The candidate element is a candidate element in a candidate list corresponding to a lowest rate-distortion cost between a current block and a prediction block;

[0023] The candidate element is a candidate element in a candidate list corresponding to a lowest matching cost between a current template and a reference template processed by a first parameter;

[0024] The candidate element is a candidate element in a candidate list corresponding to a lowest rate-distortion cost between a current block and a prediction block processed by a first parameter;

[0025] The candidate element is determined or obtained from a candidate list after a reordering process;

[0026] The candidate list includes a merge candidate list or an AMVP candidate list of motion vectors or block vectors;

[0027] At least two prediction values are determined or obtained according to the candidate element and / or the first parameter corresponding to the candidate element.

[0028] Optionally, step S10 includes at least one of the following:

[0029] The second prediction value of the first image block is determined or obtained according to the first flag of the first image block and the first prediction value of the first image block;

[0030] The second prediction value of the first image block is determined or obtained according to the first parameter of the first image block and the first prediction value of the first image block;

[0031] The second prediction value of the first image block is determined or obtained according to the first flag of the first image block and the first parameter of the first image block, and the first prediction value of the first image block.

[0032] Optionally, the first parameter is determined or obtained according to at least one of the following:

[0033] The first parameter corresponding to the image block, the sub-image block or the pixel;

[0034] a first parameter corresponding to a candidate element in the candidate list;

[0035] a first parameter in the storage unit.

[0036] Optionally, the image block comprises at least one of:

[0037] a spatially or temporally neighboring image block of the first image block;

[0038] a spatially or temporally non-neighboring image block of the first image block;

[0039] an image block or sub-image block determined by motion offset;

[0040] a collocated image block of the first image block;

[0041] an image block or sub-image block determined by sub-block temporal motion vector prediction processing.

[0042] Optionally, the first flag is determined or obtained by at least one of:

[0043] a first flag corresponding to a candidate element with the lowest matching cost;

[0044] a first flag corresponding to candidate elements with the lowest and second lowest matching costs;

[0045] a first flag corresponding to a candidate element with the lowest rate-distortion cost;

[0046] a first flag corresponding to candidate elements with the lowest and second lowest rate-distortion costs.

[0047] Optionally, the method further comprises:

[0048] if the first parameter of the first image block does not exist, determining the first parameter of the first image block according to pixels in a neighboring region of the first image block.

[0049] The application further provides a processing apparatus, comprising:

[0050] a processing module configured to determine or obtain a second prediction value of the first image block according to a first prediction value of the first image block.

[0051] The application further provides a processing device, comprising a memory, a processor, and a processing program stored in the memory and executable on the processor, wherein the processing program, when executed by the processor, implements the steps of the processing method according to any one of the above.

[0052] The processing device in the application can be a smart terminal or a server.

[0053] The application further provides a computer readable storage medium, and a processing program is stored on the computer readable storage medium. The processing program is executed by a processor to implement the steps of the processing method according to any one of the above.

[0054] The technical scheme of the application determines or obtains the second prediction value of the first image block according to the first prediction value of the first image block, solves the technical problem of low efficiency of illumination compensation in the image processing scene, and can perform illumination compensation on the first prediction value of the first image block, improve the prediction quality of the first image block, and thus improve the video encoding and / or decoding performance. BRIEF DESCRIPTION OF DRAWINGS

[0055] The accompanying drawings, which are incorporated into and form a part of the specification, illustrate an embodiment consistent with the present application and, together with the description, serve to explain the principles of the application. In order to more clearly illustrate the technical scheme of the embodiments of the application, the drawings needed to be used in the embodiment description will be briefly introduced as follows. Obviously, for those of ordinary skill in the art, no creative work is needed to obtain other drawings from these drawings.

[0056] Fig. 1 is a schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the application;

[0057] Fig. 2 is a schematic diagram of a communication network system architecture provided by an embodiment of the application;

[0058] Fig. 3 is a schematic diagram of the hardware structure of a controller 140 provided by the application;

[0059] Fig. 4 is a schematic diagram of the hardware structure of a network node 150 provided by the application;

[0060] Fig. 5 is a flowchart of a processing method according to a first embodiment;

[0061] Fig. 6 is an encoding diagram of the processing method according to the first embodiment;

[0062] Fig. 7 is a decoding diagram of the processing method according to the first embodiment;

[0063] Fig. 8 is a diagram of a current template and a reference template in the processing method according to a second embodiment;

[0064] Fig. 9 is a diagram of reordering template matching cost in the processing method according to the second embodiment;

[0065] Fig. 10 is a diagram of a spatial domain candidate in the processing method according to the second embodiment;

[0066] Fig. 11 is a diagram of a temporal domain candidate in the processing method according to the second embodiment;

[0067] Fig. 12 is a schematic diagram of non-adjacent spatial candidates in a processing method according to a second embodiment;

[0068] Fig. 13 is a schematic diagram of the position relationship of pixels Y1-Y5 in a processing method according to a second embodiment;

[0069] Fig. 14 is a schematic diagram of the luminance prediction pixel values determined by a first prediction mode in a processing method according to a second embodiment;

[0070] Fig. 15 is a schematic diagram of a local illumination compensation module in a processing method according to a second embodiment;

[0071] Fig. 16 is a schematic diagram of another local illumination compensation module in a processing method according to a second embodiment;

[0072] Fig. 17 is a schematic diagram of a local illumination compensation module in a processing method according to a second embodiment;

[0073] Fig. 18 is a schematic diagram of a module of a processing apparatus according to an embodiment of the present application.

[0074] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. The above-described drawings have shown the specific embodiments of the present application, and the following detailed description will be more detailed. These drawings and the following detailed description are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0075] The exemplary embodiments will be described in detail herein with reference to the accompanying drawings. The following description is presented with reference to the drawings, in which the same numbers represent the same or similar elements throughout the several drawings. The implementations described in the following exemplary embodiments are not meant to be all inclusive of all implementations consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present application, as detailed in the appended claims.

[0076] It should be noted that, as used in this document, the terms "include," "includes," "including," "has," "have," "having," or the like, are used in the sense of "including" and not by way of rinse, such that a process, method, article, or apparatus that includes items does not include only those items but can include other items not expressly listed or inherent to such process, method, article, or apparatus. The term "or" as used herein is used to mean, at least one of the item being purchased by the user, for example, the item "or" means at least one of the items "A" or "B". All structural and functional equivalents to the items described herein are expressly incorporated in and made part of the claims. Moreover, it is to be understood that the foregoing description and the examples applied are intended to cover all possible modifications and variations of the disclosed concepts and methods. Unless otherwise noted, the words "comprise" (or comprise), "comprises" (or comprises) and "comprising" (or comprising) will be understood to encompass the terms "include", "includes" and "including".

[0077] It should be understood that, although the terms first, second, third, etc. can be used herein to describe various information, the information should not be limited to these terms. These terms are only used to differentiate one piece of information from another piece of information of the same type. For example, a first information can also be referred to as a second information, and similarly, a second information can also be referred to as a first information, without departing from the scope of the present document. Depending on the context, the word "if' as used herein can be interpreted as meaning "when" or "upon determination" or "in response to a determination". Furthermore, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprise", "comprises", "comprising", "include", "includes" and "including" mean that the mentioned features, steps, operations, elements, components, items, kinds and / or groups are present, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, items, kinds and / or groups. The terms "or", "and / or", "at least one of the following", etc. used in the present document can be interpreted as inclusive or as meaning any one or any combination. For example, "at least one of: A, B, C" means "any of the following: A; B; C; A and B; A and C; B and C; A and B and C". As another example, "A, B, or C" or "A, B, and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A and B and C". The exception to this definition will occur only when a combination of elements, functions, steps or acts are in some way inherently mutually exclusive.

[0078] It should be understood that, although each step in the flowchart in the embodiments of the present application is shown in sequence according to the arrow, the steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of the steps is not strictly limited in sequence, and can be executed in other sequences. Moreover, some steps in the figure include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence is not necessarily sequential, but can be alternately executed with other steps or sub-steps or stages of other steps.

[0079] Depending on the context, the word "if" as used herein can be interpreted as meaning "when" or "while" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "if it is determined" or "if (a stated condition or event) is detected" can be interpreted as meaning "when it is determined" or "in response to determining" or "when (a stated condition or event) is detected" or "in response to detecting (a stated condition or event)".

[0080] It should be noted that, in the present document, step codes such as S10, S20, etc. are used for the purpose of more clearly and briefly expressing the corresponding content, and do not constitute a substantial limitation in sequence. Those skilled in the art may, in specific implementation, perform S20 before performing S10, etc., but these should be within the scope of protection of the present application.

[0081] It should be understood that the specific embodiments described herein are merely intended to explain the present application, and are not intended to limit the present application.

[0082] In the following description, the suffixes such as "module", "part", or "unit" used for an element are merely used for convenience of explanation of the present application, and have no specific meaning by themselves. Thus, "module", "part", or "unit" can be mixedly used.

[0083] The processing device in the present application can be a smart terminal or a server. The smart terminal can be implemented in various forms. For example, the smart terminal described in the present application includes smart terminals such as a mobile phone, a tablet computer, a notebook computer, a palmtop computer, a Personal Digital Assistant (PDA), a Portable Media Player (PMP), a navigation device, a wearable device, a smart bracelet, a pedometer, and fixed terminal devices such as a digital TV, a desktop computer, etc.

[0084] In the following description, a mobile terminal will be exemplified, and it will be understood by those skilled in the art that the configuration according to the embodiments of the present application can be applied to a terminal device of a fixed type, except for elements particularly used for mobile purposes.

[0085] Referring to FIG. 1, a hardware structure of a mobile terminal according to an embodiment of the present application is illustrated. The mobile terminal 100 includes a Radio Frequency (RF) unit 101, a WiFi module 102, an audio output unit 103, an A / V (audio / video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111, etc. Those skilled in the art will appreciate that the mobile terminal structure illustrated in FIG. 1 does not limit the mobile terminal, and the mobile terminal includes more or less or different components, or a combination of some components, or a different arrangement of components.

[0086] The components of the mobile terminal will be described in detail with reference to FIG. 1.

[0087] The radio frequency unit 101 can be used for receiving and transmitting signals in the process of transmitting or receiving information or a call. Specifically, the radio frequency unit 101 receives downlink information from a base station and provides the received information to the processor 110 for processing. In addition, the radio frequency unit 101 transmits uplink data to the base station. Generally, the radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. Alternatively, the radio frequency unit 101 can also communicate with a network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to GSM (Global System for Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution), 5G, and 6G, etc.

[0088] WiFi belongs to a short-range wireless transmission technology. The WiFi module 102 can help a user to send and receive e-mails, browse web pages, and access streaming media, etc. It provides the user with wireless broadband Internet access. Although the WiFi module 102 is shown in FIG. 1, it can be understood that it does not belong to the essential components of the mobile terminal, and can be omitted according to needs without changing the essence of the application.

[0089] The audio output unit 103 can convert audio data, which is received by the radio frequency unit 101 or the WiFi module 102 or stored in the memory 109, into an audio signal and output the audio signal as sound when the mobile terminal 100 is in a call signal receiving mode, a call mode, a recording mode, a voice recognition mode, a broadcast reception mode, and the like. Moreover, the audio output unit 103 can provide audio output related to a particular function performed by the mobile terminal 100 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 103 includes a speaker, a buzzer, and the like.

[0090] The A / V input unit 104 is configured to receive audio or video signals. The A / V input unit 104 includes a graphics processing unit (GPU) 1041 and a microphone 1042. The graphics processing unit 1041 processes image data of a still picture or a video obtained by an image capture device (e.g., a camera) in a video capture mode or an image capture mode. Processed image frames can be displayed on the display unit 106. Processed image frames can be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) via the microphone 1042 in a telephone call mode, a recording mode, a voice recognition mode, and the like, and can process such sound into audio data. Processed audio (voice) data can be converted into a format transmittable to a mobile communication base station in the case of a telephone call mode, and outputted. The microphone 1042 can implement various types of noise cancellation (or suppression) algorithms to cancel (or suppress) noise or interference generated in the process of receiving and transmitting audio signals.

[0091] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. The light sensor includes an ambient light sensor and a proximity sensor, which can optionally adjust the brightness of the display panel 1061 according to the brightness of ambient light, and turn off the display panel 1061 and / or the backlight when the mobile terminal 100 is moved to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude and direction of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when at rest, and can be used for applications that recognize the posture of the mobile terminal (such as switching between landscape and portrait screens, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometers, tapping), and the like. The mobile terminal can also be configured with a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and other sensors, which are not described here.

[0092] The display unit 106 is configured to display information input by a user or information provided to the user. The display unit 106 can include a display panel 1061, which can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0093] The user input unit 107 can be used to receive input numerals or character information, and to generate key signal inputs related to user settings of the mobile terminal and control of functions. Optionally, the user input unit 107 can include a touch panel 1071 and other input devices 1072. The touch panel 1071, also called a touch screen, can collect touch operations of a user on or proximity thereto (such as operations of the user using a finger, a stylus, or any suitable object or accessory on or in proximity to the touch panel 1071), and drive corresponding connection devices according to a pre-set program. The touch panel 1071 can include two parts, a touch detecting device and a touch controller. The touch detecting device detects the user's touch position and detects signals resulting from touch operations, and transmits the signals to the touch controller; the touch controller receives touch information from the touch detecting device, and converts it into touch coordinates, and sends it to the processor 110, and can receive commands from the processor 110 and execute them. Optionally, the touch panel 1071 can be implemented in various types such as a resistive type, a capacitive type, an infrared type, and a surface acoustic wave type. In addition to the touch panel 1071, the user input unit 107 can include other input devices 1072. Optionally, the other input devices 1072 include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control buttons, switch buttons, etc.), a trackball, a mouse, a joystick, etc., without limitation.

[0094] Optionally, the touch panel 1071 can cover the display panel 1061, and when the touch panel 1071 detects a touch operation on or in proximity thereto, it transmits to the processor 110 to determine the type of touch event, and then the processor 110 provides corresponding visual output on the display panel 1061 according to the type of touch event. Although in FIG. 1, the touch panel 1071 and the display panel 1061 are implemented as two independent components to realize the input and output functions of the mobile terminal, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to realize the input and output functions of the mobile terminal, without limitation.

[0095] The interface unit 108 serves as an interface through which at least one external device can be connected with the mobile terminal 100. For example, the external devices include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting devices having an identification module, an audio input / output (I / O) port, a video I / O port, an earphone port, and the like. The interface unit 108 can be used to receive input (e.g., data information, power, and the like) from external devices and to transmit the received input to one or more elements within the mobile terminal 100, or can be used to transmit data between the mobile terminal 100 and external devices.

[0096] The memory 109 can be used to store software programs and various data. The memory 109 can mainly include a program storage area and a data storage area, and the program storage area can store an operating system, application programs required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; and the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), and the like. Optionally, the memory 109 includes a high-speed random access memory, and further includes a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device.

[0097] The processor 110 is a control center of the mobile terminal, connects all parts of the mobile terminal through various interfaces and lines, executes various functions of the mobile terminal and processes data by running or executing software programs and / or modules stored in the memory 109 and calling data stored in the memory 109, and thus performs overall monitoring on the mobile terminal. The processor 110 can include one or more processing units; preferably, the processor 110 can integrate an application processor and a modem processor, and optionally, the application processor mainly processes an operating system, a user interface, and application programs, and the like, and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 110.

[0098] The mobile terminal 100 further includes a power supply 111 (such as a battery) for supplying power to various components, and preferably, the power supply 111 can be logically connected to the processor 110 through a power management system, so as to realize functions of managing charging, discharging, and power consumption management, and the like through the power management system.

[0099] Although not shown in FIG. 1, the mobile terminal 100 further includes a Bluetooth module and the like, which will not be described herein.

[0100] In order to facilitate understanding of the embodiments of the present application, the communication network system based on the mobile terminal of the present application is described below.

[0101] Please refer to FIG. 2, which is a communication network system architecture diagram provided by an embodiment of the present application. The communication network system is an LTE system of a general mobile communication technology, and the LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203, and an operator's IP service 204, which are sequentially connected in communication.

[0102] Optionally, the UE 201 can be the terminal 100 described above, which will not be described herein.

[0103] The E-UTRAN 202 includes eNode Bs 2021 and other eNode Bs 2022. Optionally, the eNode Bs 2021 can be connected to each other through backhaul (e.g., X2 interface), the eNode Bs 2021 are connected to the EPC 203, and the eNode Bs 2021 can provide access to the EPC 203 for the UEs 201.

[0104] The EPC 203 includes MMEs (Mobility Management Entity) 2031, a HSS (Home Subscriber Server) 2032, other MMEs 2033, SGWs (Serving Gate Way) 2034, PGWs (PDN Gate Way) 2035, and a PCRF (Policy and Charging Rules Function) 2036. Optionally, the MMEs 2031 are control nodes that handle signaling between the UEs 201 and the EPC 203, provide bearer and connection management. The HSS 2032 is configured to provide some registers to manage functions such as a home location register (not shown in the figure), and save some user-specific information about service features, data rates, etc. All user data can be transmitted through the SGWs 2034, the PGWs 2035 can provide IP address allocation and other functions for the UEs 201, and the PCRF 2036 is a policy and charging control policy decision point for service data flow and IP bearer resources, which selects and provides available policy and charging control decisions for policy and charging enforcement function elements (not shown in the figure).

[0105] The IP services 204 include the Internet, an intranet, an IMS (IP Multimedia Subsystem), or other IP services.

[0106] Although the above describes the LTE system as an example, those skilled in the art should know that the present application is not only applicable to the LTE system, but also applicable to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, 5G, and future new network systems (such as 6G), etc., which are not limited here.

[0107] Figure 3 is a schematic diagram of a hardware structure of a controller 140 according to an embodiment of the present application. The controller 140 comprises a memory 1401 and a processor 1402. The memory 1401 is configured to store program instructions. The processor 1402 is configured to invoke the program instructions stored in the memory 1401 to perform the steps executed by the controller in the method embodiment one. The implementation principle and the beneficial effects are similar, and thus are not described here in detail.

[0108] Optionally, the controller further comprises a communication interface 1403. The communication interface 1403 can be connected with the processor 1402 through the bus 1404. The processor 1402 can control the communication interface 1403 to realize the receiving and sending functions of the controller 140.

[0109] Figure 4 is a schematic diagram of a hardware structure of a network node 150 according to an embodiment of the present application. The network node 150 comprises a memory 1501 and a processor 1502. The memory 1501 is configured to store program instructions. The processor 1502 is configured to invoke the program instructions stored in the memory 1501 to perform the steps executed by the first node in the method embodiment one. The implementation principle and the beneficial effects are similar, and thus are not described here in detail.

[0110] Optionally, the controller further comprises a communication interface 1503. The communication interface 1503 can be connected with the processor 1502 through the bus 1504. The processor 1502 can control the communication interface 1503 to realize the receiving and sending functions of the network node 150.

[0111] The integrated modules implemented in the form of software function modules can be stored in a computer readable storage medium. The software function modules are stored in a storage medium, and include a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to execute part of the steps of the method embodiments of the present application.

[0112] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a storage medium or transferred from one storage medium to another storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. containing one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)) and the like.

[0113] Based on the above mobile terminal hardware structure and communication network system, various embodiments of the present application are proposed.

[0114] First embodiment

[0115] Referring to FIG. 5, FIG. 5 is a flowchart of a processing method according to the first embodiment, and the processing method of the embodiments of the present application can be applied to a processing device, including the steps of:

[0116] S10: determining or obtaining the second prediction value of the first image block according to the first prediction value of the first image block.

[0117] Optionally, the processing device can be a smart terminal such as a mobile phone, a computer, etc., or a server such as a local server or a cloud server. In this embodiment and the present application, the processing device is mainly illustrated as a smart terminal.

[0118] Optionally, the technical solution of the present embodiment can be applied to the fields of image coding, video coding, hardware video coding, special circuit video coding, real-time video coding, etc.

[0119] Optionally, the input image is divided into image blocks at the encoding end, each image block is subtracted by a prediction block obtained by a prediction mode to obtain a residual block, the residual block is processed by transformation and quantization, and then encoded by an entropy encoder to form an encoded bitstream. Optionally, the transformed and / or quantized residual block is added to the prediction block obtained by the prediction mode to obtain a reconstructed block, and the reconstructed block can be subjected to loop filtering processing to reduce distortion.

[0120] Optionally, the prediction mode includes an intra prediction mode and / or an inter prediction mode.

[0121] Optionally, the intra prediction includes at least one prediction mode, such as an angular mode, a planar mode, and the like.

[0122] Optionally, the inter prediction module can use local illumination compensation (LIC) to compensate the first prediction value of the to-be-predicted pixel.

[0123] Optionally, if there is an illumination change between the current block and the reference block, and the change is linear, a linear function a*p[x]+b can be fitted by using the adjacent reconstructed pixels (such as the reference template and the current template) of the current block and the reference block to compensate the illumination change. Optionally, p[x] is the reference block, a is a scaling factor, and b is an offset. a and b can be derived by using the least square method:

[0124] Optionally, i can be an integer, and 0

[0125] Optionally, M can be the width of the current block, and N can be the height of the current block. M and N can also be other integers, for example, M and N can be any integer satisfying M+N

[0126] Optionally, the embodiment technical solution can be applied to the encoding end. Referring to FIG. 6, the encoder includes various module units, such as a transformation module, a quantization module, an entropy encoding module, an inverse quantization module, an inverse transformation module, an intra prediction module, an inter prediction module, an encoded image buffer, and a loop filtering module. After receiving a video image (i.e., input video data in FIG. 6) from a video source, the input image is divided into image blocks (the image blocks include luminance blocks and chrominance blocks). The image blocks are subjected to prediction processing by using the temporal and / or spatial correlation between the video images.

[0127] Optionally, the prediction processing includes intra prediction processing and / or inter prediction processing, and the intra prediction processing and / or the inter prediction processing includes at least one prediction mode. For the prediction modes, the encoder uses an optimization strategy (e.g., rate-distortion optimization) to determine the prediction mode finally adopted by the image block. Optionally, the rate-distortion cost corresponding to each prediction mode or the rate-distortion cost of the combination of several prediction modes is calculated to determine the minimum rate-distortion cost from the rate-distortion costs, and the prediction mode or the combination of prediction modes corresponding to the minimum rate-distortion cost is the prediction mode finally adopted by the image block. The prediction mode includes a local illumination compensation mode in this embodiment, that is, local illumination compensation processing is performed on the basis of the first prediction value obtained by the first prediction mode to determine the pixel value of the final to-be-predicted pixel (i.e., the second prediction value of the first pixel). That is, the first prediction value of the first to-be-predicted pixel in the to-be-predicted block is determined, and the second prediction value of the first pixel is determined according to the first prediction value. The prediction block (e.g., the prediction luminance block and / or the prediction chrominance block) corresponding to the image block is obtained according to the second prediction value, and the pixel value of the pixel sample in the image block is subtracted by the prediction value of the corresponding pixel sample in the prediction block to obtain the residual value of the pixel sample and the residual block corresponding to the image block. The residual block is subjected to transformation and quantization processing, and then encoded by the entropy encoder to form the encoded bitstream. Optionally, the encoded bitstream further includes the prediction parameters corresponding to the determined prediction mode and related auxiliary information (e.g., side information).

[0128] Optionally, the prediction parameters are packaged into the encoded bitstream after being subjected to entropy encoding.

[0129] Optionally, the prediction parameters further include indication information of the prediction mode.

[0130] Optionally, the reconstructed block is obtained by adding the transformed and / or quantized residual block and the prediction block obtained by using the prediction mode, and the reconstructed block is subjected to loop filtering processing by the loop filtering module and the filter control data to reduce distortion.

[0131] Optionally, the present embodiment can be applied to the decoding end. Referring to FIG. 7, the decoder includes various module units, such as entropy decoding, inverse quantization, inverse transformation, intra prediction, inter prediction, loop filtering, and decoded image buffer. After receiving the encoded bitstream, the decoding unit of the decoder analyzes and decodes the encoded bitstream to obtain the transform coefficient; and the inverse transformation unit and the inverse quantization unit of the decoder perform inverse transformation and inverse quantization processing on the transform coefficient to obtain the residual block.

[0132] Optionally, a decoding unit of the decoder parses and decodes the coded bitstream to obtain the prediction parameters and the related auxiliary information. A prediction processing unit of the decoder performs prediction processing using the prediction parameters to determine a prediction block corresponding to the residual block; optionally, the prediction processing includes intra prediction processing and / or inter prediction processing, and the intra prediction processing and / or the inter prediction processing respectively includes at least one prediction mode in a combined manner. Optionally, the prediction parameters indicate that the corresponding prediction mode is the prediction mode corresponding to the residual block.

[0133] Optionally, at the decoding end, the second prediction value of the first pixel can be determined or obtained according to the first prediction value. For example, the first pixel value of the first pixel sample of the first color component is obtained or determined; and the second pixel value of the second pixel sample of the first color component and / or the second color component is determined according to the first pixel of the first pixel sample. After determining all pixel samples in the to-be-predicted block, the prediction result of the to-be-predicted block is obtained. The obtained residual block and the corresponding prediction block (including the predicted luma block and / or the predicted chroma block) are added to obtain a reconstructed block. The loop filter unit of the decoder performs loop filtering processing on the reconstructed block according to the filter control data to reduce distortion and improve video quality. The reconstructed block subjected to the loop filtering processing is combined into a decoded image and stored in a decoded image buffer or output as a decoded video signal.

[0134] The embodiment can improve the prediction quality of the first image block by performing illumination compensation on the first prediction value of the first image block, thereby improving the video encoding and / or decoding performance.

[0135] Second embodiment

[0136] Based on the first embodiment, the processing method further includes determining or obtaining the second prediction value of the first image block based on the first flag, the first parameter, the candidate element, or the candidate list.

[0137] Optionally, the first flag is an illumination compensation flag.

[0138] Optionally, the first parameter is an illumination compensation parameter.

[0139] Optionally, the candidate element includes a first candidate element and a second candidate element.

[0140] Optionally, the candidate element includes at least one of a motion vector, a block vector, a motion vector candidate, a block vector candidate, a motion vector predictor, a block vector predictor, a motion vector index, and a block vector index.

[0141] Optionally, the candidate element is a candidate element in the candidate list corresponding to the lowest matching cost between the current template and the reference template.

[0142] Optionally, the candidate element is a candidate element in the candidate list corresponding to a lowest rate-distortion cost between the current block and the prediction block after the first parameter processing.

[0143] Optionally, the candidate element is a candidate element in the candidate list corresponding to a lowest matching cost between the current template and the reference template after the first parameter processing.

[0144] Optionally, the candidate element is a candidate element in the candidate list corresponding to a lowest rate-distortion cost between the current block and the prediction block after the first parameter processing.

[0145] Optionally, the candidate element is determined or obtained from the candidate list after the reordering processing.

[0146] Optionally, the candidate list includes a merge candidate list or an AMVP candidate list of motion vectors or block vectors.

[0147] Optionally, at least two prediction values are determined or obtained according to the candidate element and / or the first parameter corresponding to the candidate element.

[0148] In the embodiment, the determination or obtaining manner of the first prediction value includes at least one of the following manner one to manner six:

[0149] Manner one, an inter prediction mode and / or an intra prediction mode;

[0150] Optionally, the inter prediction mode includes an inter prediction mode corresponding to a luma component, an inter prediction mode corresponding to a chroma component. When the current block is a block of the luma component, the inter prediction mode corresponding to the luma component is selected to predict the first pixel in the current block to obtain the first prediction value. When the current block is a block of the chroma component, the inter prediction mode corresponding to the chroma component is selected to predict the first pixel in the current block to obtain the first prediction value.

[0151] Optionally, the inter prediction mode of the current block can be an intra prediction mode of another image block (for example, a corresponding luma block) in an image unit where the current block (for example, a chroma block) is located.

[0152] Optionally, the intra prediction mode includes an intra prediction mode corresponding to a luma component, an intra prediction mode corresponding to a chroma component. When the current block is a block of the luma component, the intra prediction mode corresponding to the luma component is selected to predict the first pixel in the current block to obtain the first prediction value. When the current block is a block of the chroma component, the intra prediction mode corresponding to the chroma component is selected to predict the first pixel in the current block to obtain the first prediction value.

[0153] Optionally, the intra prediction mode of the current block can be an intra prediction mode of another image block (for example, a corresponding luma block) in an image unit where the current block (for example, a chroma block) is located.

[0154] at least one of the pixel value, the position, the motion information, the prediction mode of the image block;

[0155] Optionally, the pixel value of the image block can be a pixel value of a pixel in the image block corresponding to a default block, a neighbor block, a non-neighbor block, a co-located block, a reference block of the current block. The prediction mode can be determined according to the pixel value of the image block, such as the prediction mode adopted by the image block, and then the first pixel in the current block is predicted according to the determined prediction mode to obtain the first prediction value.

[0156] Optionally, the first prediction value can be obtained by deforming the pixel value of the first image block, such as weighted calculation, range adjustment, etc.

[0157] Optionally, the position of the image block can be the position of the image block corresponding to a default block, a neighbor block, a non-neighbor block, a co-located block, a reference block of the current block. The prediction mode can be determined according to the position of the first image block, such as the position of the different image blocks corresponding to different prediction modes can be set in advance, the prediction mode is determined according to the position of the first image block, and the first pixel in the current block is predicted according to the determined prediction mode to obtain the first prediction value.

[0158] Optionally, the motion information of the image block can be the motion information corresponding to the image block in the default block, the neighbor block, the non-neighbor block, the co-located block, the reference block corresponding to the current block, such as the motion information related to the motion estimation, motion compensation and other processing. The prediction mode can be determined according to the motion information of the first image block, such as the motion information of different image blocks corresponding to different prediction modes can be set in advance, the prediction mode is determined according to the motion information of the first image block, and the first pixel in the current block is predicted according to the determined prediction mode to obtain the first prediction value.

[0159] The prediction mode can be an intra-component prediction mode or a cross-component prediction mode. For example, the first pixel in the current block is predicted according to the cross-component prediction mode to obtain the first prediction value. The first pixel in the current block is predicted according to the intra-component prediction mode.

[0160] Optionally, the encoding side and / or the decoding side can determine or obtain the first prediction value of the first pixel in the current block according to the pixel value, the position, the motion information and / or the prediction mode of the first image block, and determine or obtain the second prediction value of the first pixel according to the first prediction value and the first parameter.

[0161] Optionally, the image block can be a first image block spatially or temporally adjacent to the first image block.

[0162] Referring to FIG. 10 and FIG. 11, the first image block spatially adjacent image block can be an image block in which the pixel samples A0-A1, B0-B2 in FIG. 10 or FIG. 11 are located. The first image block temporally adjacent image block can be an image block in which the pixel samples C0-C1 in FIG. 10 or FIG. 11 are located.

[0163] Optionally, the image block can be a first image block spatially or temporally non-adjacent image block.

[0164] Referring to FIG. 12, if the image blocks in which the pixel samples A01-A04, A11-A13, B01-B04, B11-B13, B21-B24 in FIG. 12 are located are not adjacent to the first image block, the first image block spatially non-adjacent image block can be the image blocks.

[0165] Optionally, the image block can be a first image block spatially or temporally non-adjacent image block.

[0166] Optionally, the image block can be a first image block spatially or temporally non-adjacent image block.

[0167] Optionally, the image block can be a first image block spatially or temporally non-adjacent image block.

[0168] Optionally, the image block can be a first image block spatially or temporally non-adjacent image block.

[0169] Optionally, the image block can be a first image block spatially or temporally non-adjacent image block.

[0170] Optionally, the image block can be a first image block spatially or temporally non-adjacent image block.

[0171] Optionally, the first flag can be a light compensation flag corresponding to a candidate element in a candidate list of a motion vector or a block vector corresponding to the first image block.

[0172] Optionally, the candidate list includes a merge candidate list or an advanced motion vector prediction (AMVP) candidate list of the motion vector or the block vector.

[0173] Optionally, the candidate element includes at least one of a motion vector (MV), a block vector (BV), a motion vector candidate, a block vector candidate, a motion vector predictor, a block vector predictor, a motion vector index, or a block vector index.

[0174] Optionally, the candidate element can be a candidate element in the candidate list corresponding to a lowest matching cost between the current template and the reference template, and the first flag can be a first flag corresponding to the candidate element with the lowest matching cost in the candidate list; for example, as shown in FIG. 8, a reference template (a reference top template, a reference left template) corresponding to a current template (a current top template, a current left template) of the current block is determined according to each candidate mv or bv in the first list, and a matching cost between the current template and the reference template is determined. Optionally, the matching cost is calculated in a manner including SAD and / or SATD.

[0175] Optionally, the candidate element can be a candidate element in the candidate list corresponding to a lowest rate-distortion cost between the current block and the prediction block, and the first flag can be a first flag corresponding to the candidate element with the lowest rate-distortion cost in the candidate list.

[0176] Optionally, the candidate element can be a candidate element in the candidate list corresponding to a lowest matching cost between the current template and the reference template processed by the first parameter, and the first flag can be a first flag corresponding to the candidate element with the lowest matching cost processed by the first parameter in the candidate list.

[0177] Optionally, the candidate element can be a candidate element in the candidate list corresponding to a lowest rate-distortion cost between the current block and the prediction block processed by the first parameter, and the first flag can be a first flag corresponding to the candidate element with the lowest rate-distortion cost processed by the first parameter in the candidate list.

[0178] Optionally, the candidate elements include a first candidate element and a second candidate element, such as the first candidate element and the second candidate element can be candidate elements in the candidate list corresponding to a lowest and a second-lowest matching cost between the current template and the reference template, and the first flag can be a first flag corresponding to the candidate elements with the lowest and the second-lowest matching cost in the candidate list.

[0179] Optionally, the candidate elements include a first candidate element and a second candidate element, such as the first candidate element and the second candidate element can be candidate elements in the candidate list corresponding to a lowest and a second-lowest rate-distortion cost between the current block and the prediction block, and the first flag can be a first flag corresponding to the candidate elements with the lowest and the second-lowest rate-distortion cost in the candidate list.

[0180] Optionally, the candidate elements include a first candidate element and a second candidate element, such as the first candidate element and the second candidate element can be candidate elements in the candidate list corresponding to a lowest and a second-lowest matching cost between the current template and the reference template processed by the first parameter, and the first flag can be a first flag corresponding to the candidate elements with the lowest and the second-lowest matching cost processed by the first parameter in the candidate list.

[0181] Optionally, the candidate elements include a first candidate element and a second candidate element, such as the first candidate element and the second candidate element can be respectively a candidate element in the candidate list corresponding to a lowest rate-distortion cost and a second lowest rate-distortion cost between the current block and the prediction block processed by the first parameter, and the first flag can be a first flag corresponding to the candidate element in the candidate list processed by the first parameter with the lowest rate-distortion cost and the second lowest rate-distortion cost.

[0182] Optionally, the candidate element is determined or obtained from the candidate list after the reordering processing, such as, as shown in FIG. 9, all the candidates in the first list are sorted in ascending order of the matching cost according to the template matching cost to determine the candidate motion vector or block vector with the minimum template matching, and the candidate element is determined or obtained from the candidate list after the reordering processing.

[0183] Optionally, the first parameter is determined or obtained according to the image block, the sub-image block or the pixel.

[0184] Optionally, the first parameter is determined or obtained according to the image block, the sub-image block or the pixel.

[0185] Optionally, the first parameter is determined or obtained according to the image block, the sub-image block or the pixel.

[0186] Optionally, the first parameter can be an illumination compensation parameter corresponding to a candidate element in a candidate list of a motion vector or a block vector of the first image block.

[0187] Optionally, the candidate list includes a merge candidate list or an AMVP candidate list of the motion vector or the block vector.

[0188] Optionally, the candidate element includes at least one of a motion vector, a block vector, a motion vector candidate, a block vector candidate, a motion vector predictor, a block vector predictor, a motion vector index, and a block vector index.

[0189] Optionally, the candidate element can be a candidate element in the candidate list corresponding to a lowest matching cost between the current template and the reference template, and the first parameter can be a first parameter corresponding to the candidate element in the candidate list with the lowest matching cost.

[0190] For example, if the candidate motion vector mv3 in FIG. 9 is the candidate mv with the minimum template matching cost, the candidate motion vector mv3 is a spatial neighboring candidate mv, the candidate motion vector mv3 is the mv corresponding to the top-left sample B2 in FIG. 10, or the candidate motion vector mv3 is derived from the mv corresponding to the top-left sample B2 in FIG. 10, the value of the illumination compensation parameters used by the top-left sample B2 in FIG. 10 (or the illumination compensation parameters used by the coding block in which the top-left sample B2 is located) is used as the value of the illumination compensation parameters used by the current image block. Alternatively, if the value of the illumination compensation parameters used by the top-left sample B2 (or the illumination compensation parameters used by the coding block in which the top-left sample B2 is located) is a1, b1, the value of the illumination compensation parameters used by the current image block is a1, b1. If the value of the illumination compensation parameters used by the top-left sample B2 (or the LIC flag corresponding to the coding block in which the top-left sample B2 is located) is a2, b2, the value of the LIC flag used by the current image block is a2, b2.

[0191] For another example, if the candidate motion vector mv3 in FIG. 9 is the candidate mv with the minimum template matching cost, the candidate motion vector mv3 is a temporal candidate mv, and the candidate motion vector mv3 is the mv corresponding to the bottom-right sample C0 in FIG. 11, or the candidate motion vector mv3 is derived from the mv corresponding to the bottom-right sample C0 in FIG. 11, the value of the illumination compensation parameters used by the bottom-right sample C0 in FIG. 11 (or the illumination compensation parameters used by the coding block in which the bottom-right sample C0 is located) is used as the value of the illumination compensation parameters used by the current image block. Alternatively, if the value of the illumination compensation parameters used by the bottom-right sample C0 (or the illumination compensation parameters used by the coding block in which the bottom-right sample C0 is located) is a1', b1', the value of the illumination compensation parameters used by the current image block is a1', b1'. If the value of the illumination compensation parameters used by the bottom-right sample C0 (or the LIC flag corresponding to the coding block in which the bottom-right sample C0 is located) is a2', b2', the value of the LIC flag used by the current image block is a2', b2'.

[0192] For example, if the candidate motion vector mv3 in FIG. 9 is the candidate mv with the lowest matching cost, the candidate motion vector mv3 is a non-adjacent spatial candidate mv, and the candidate motion vector mv3 is the mv corresponding to the left sample A11 in FIG. 12, or the candidate motion vector mv3 is derived from the mv corresponding to the left sample A11 in FIG. 12, the value of the illumination compensation parameter used by the left sample A11 in FIG. 12 (or the value of the illumination compensation parameter used by the coding block in which the left sample A11 in FIG. 12 is located) is used as the value of the illumination compensation parameter used by the current image block. That is, if the value of the illumination compensation parameter used by the left sample A11 in FIG. 12 (or the value of the illumination compensation parameter used by the coding block in which the left sample A11 in FIG. 12 is located) is a1", b1", the value of the illumination compensation parameter used by the current image block is a1", b1". If the value of the illumination compensation parameter used by the left sample A11 in FIG. 12 (or the value of the illumination compensation parameter used by the coding block in which the left sample A11 in FIG. 12 is located) is a2", b2", the value of the LIC flag of the current image block is a2", b2".

[0193] Optionally, the first parameter can also be derived from the first parameter corresponding to the candidate element with the lowest matching cost in the candidate list. The first parameter can also be derived from the illumination compensation parameter corresponding to the candidate element with the lowest matching cost in the candidate list.

[0194] Optionally, the candidate element can be the candidate element corresponding to the lowest rate-distortion cost between the current block and the prediction block in the candidate list, and the first parameter can be the parameter corresponding to the candidate element with the lowest rate-distortion cost in the candidate list.

[0195] Optionally, the candidate element can be the candidate element corresponding to the lowest matching cost between the current template and the reference template processed by the first parameter in the candidate list, and the first parameter can be the parameter corresponding to the candidate element with the lowest matching cost processed by the first parameter in the candidate list.

[0196] Optionally, the candidate element can be the candidate element corresponding to the lowest rate-distortion cost between the current block and the prediction block processed by the first parameter in the candidate list, and the first parameter can be the parameter corresponding to the candidate element with the lowest rate-distortion cost processed by the first parameter in the candidate list.

[0197] Optionally, the candidate elements include a first candidate element and a second candidate element, such as the first candidate element and the second candidate element can be the candidate elements corresponding to the lowest and second-lowest matching costs between the current template and the reference template in the candidate list, and the first parameter can be the parameter corresponding to the candidate elements with the lowest and second-lowest matching costs in the candidate list.

[0198] Optionally, the candidate elements include a first candidate element and a second candidate element, such as the first candidate element and the second candidate element can be the candidate elements in the candidate list corresponding to the lowest and the second lowest rate-distortion cost between the current block and the prediction block processed by the first parameter, and the first parameter can be the parameter corresponding to the candidate element with the lowest and the second lowest rate-distortion cost in the candidate list processed by the first parameter.

[0199] Optionally, the candidate elements include a first candidate element and a second candidate element, such as the first candidate element and the second candidate element can be the candidate elements in the candidate list corresponding to the lowest and the second lowest matching cost between the current template and the reference template processed by the first parameter, and the first parameter can be the parameter corresponding to the candidate element with the lowest and the second lowest matching cost in the candidate list processed by the first parameter.

[0200] Optionally, the candidate elements include a first candidate element and a second candidate element, such as the first candidate element and the second candidate element can be the candidate elements in the candidate list corresponding to the lowest and the second lowest rate-distortion cost between the current block and the prediction block processed by the first parameter, and the first parameter can be the parameter corresponding to the candidate element with the lowest and the second lowest rate-distortion cost in the candidate list processed by the first parameter.

[0201] Optionally, the candidate elements are determined or obtained from the candidate list after the reordering processing, such as the matching cost or the rate-distortion cost of the candidate elements in the candidate list can be reordered, and the candidate elements are determined or obtained from the candidate list after the reordering processing.

[0202] Optionally, the first parameter is determined or obtained according to the first parameter in the storage unit.

[0203] Optionally, the first parameter corresponding to the candidate element of the candidate list can be stored in the storage unit, and the first parameter corresponding to the candidate element with the minimum template matching cost is read from the storage unit after the candidate element is determined. By reading the first parameter corresponding to the candidate element from the storage unit, the first parameter is used as the first parameter of the first image block, and the first parameter of the first image block does not need to be calculated additionally, thereby reducing the calculation complexity.

[0204] Optionally, the first parameter of the second image block (the coded image block) is determined, and it is judged whether the storage unit is full. If yes, the first stored illumination compensation parameter in the storage unit is discarded; and / or if not, the illumination compensation parameter of the second image block is stored in the storage unit.

[0205] Optionally, the position information of the second image block or the motion information of the second image block can also be stored in the storage unit. The illumination compensation parameter can be associated with the position information of the second image block. The illumination compensation parameter can be associated with the motion information of the second image block. The illumination compensation parameter can be associated with both the position information and the motion information of the second image block. Optionally, the position information of the second image block includes the position information of the top-left pixel of the second image block, the size of the second image block, the motion information of the second image block includes the motion vector of the second image block, the reference image index, or the block vector of the second image block.

[0206] Optionally, if the motion vector mv or the block vector bv and the illumination compensation parameter are stored in the storage unit at the same time, the motion vector mv or the block vector bv with the minimum template matching cost is determined, and the illumination compensation parameter associated with (or corresponding to) the motion vector mv or the block vector bv is determined.

[0207] Optionally, if the position information of the image block and the illumination compensation parameter are stored in the storage unit at the same time, the motion vector mv or the block vector bv with the minimum template matching cost is determined, the position information of the image block associated with (or corresponding to) the motion vector mv or the block vector bv is determined, and the illumination compensation parameter is determined through the position information.

[0208] Optionally, if the first parameter of the first image block does not exist, the first parameter of the first image block is determined according to the pixels in the adjacent area of the first image block. Optionally, if the illumination compensation parameter corresponding to the candidate mv_a / bv_a of the first list does not exist or the storage unit does not store the illumination compensation parameter corresponding to the candidate mv_a / bv_a of the first list, the illumination compensation parameter of the first image block is determined by sampling the pixels in the upper area and / or the left area of the first image block.

[0209] Optionally, all the stored illumination compensation parameters correspond to the image blocks in the image in which the first image block is located.

[0210] Optionally, the stored illumination compensation parameters correspond to the image blocks in the reference image of the image in which the first image block is located, in addition to the image blocks in the image in which the first image block is located.

[0211] Optionally, whether each sub-block of the current block adopts the local illumination compensation processing is determined according to whether each sub-block of the reference block adopts the local illumination compensation processing. For example, if the LIC flag of the center pixel sample of a sub-block of the reference block indicates that the LIC is adopted, the LIC flag of the corresponding sub-block of the current block also indicates that the LIC is adopted. Optionally, if the LIC flag of the center pixel sample of a sub-block of the reference block is 1, the LIC flag of the corresponding sub-block of the current block is also 1; and / or, if the LIC flag of the center pixel sample of a sub-block of the reference block is 0, the LIC flag of the corresponding sub-block of the current block is also 0. Illustratively, the LIC flag of 1 indicates that the LIC is adopted, and the LIC flag of 0 indicates that the LIC is not adopted; and / or, if the LIC flag of the first image block is not output in the bitstream, the LIC flag of the first image block is 0 by default.

[0212] Optionally, if each sub-block of the reference block adopts the local illumination compensation processing, the illumination compensation parameters corresponding to each sub-block are taken as the illumination compensation parameters of the corresponding sub-block in the current block, for example, the illumination compensation parameters corresponding to the center pixel sample of each sub-block are taken as the illumination compensation parameters of the corresponding sub-block in the current block.

[0213] Optionally, the candidate element includes at least one of a motion vector, a block vector, a motion vector candidate, a block vector candidate, a motion vector predictor, a block vector predictor, a motion vector index, and a block vector index.

[0214] Optionally, the candidate element includes a first candidate element and a second candidate element.

[0215] Optionally, the candidate element includes at least one of a motion vector, a block vector, a motion vector candidate, a block vector candidate, a motion vector predictor, a block vector predictor, a motion vector index, and a block vector index.

[0216] Optionally, the candidate element is a candidate element in the candidate list corresponding to the lowest matching cost between the current template and the reference template.

[0217] Optionally, the candidate element is a candidate element in the candidate list corresponding to the lowest rate-distortion cost between the current block and the prediction block.

[0218] Optionally, the candidate element is a candidate element in the candidate list corresponding to the lowest matching cost between the current template and the reference template processed by the first parameter.

[0219] Optionally, the candidate element is a candidate element in the candidate list corresponding to the lowest rate-distortion cost between the current block and the prediction block processed by the first parameter.

[0220] Optionally, the candidate element is determined or obtained from the candidate list after the reordering processing.

[0221] Optionally, the candidate list includes a merge candidate list or an AMVP candidate list of motion vectors or block vectors, for example, the first list can be a merge candidate list of motion vectors, can be an AMVP candidate list of motion vectors, can be a merge candidate list of block vectors, or can be an AMVP candidate list of block vectors. As shown in FIG. 10, the mv corresponding to the B0, B1, B2, A0, A1 samples (or the mv corresponding to the CU where the samples are located) is taken as the neighboring spatial candidate mv corresponding to the image block (the current image block). As shown in FIG. 11, the temporal candidate includes one candidate mv, which can be obtained by scaling the mv corresponding to the co-located CU of the C0 sample, or can be obtained by scaling the mv corresponding to the co-located CU of the C1 sample. For example, if the co-located CU of the C0 cannot be used, the co-located CU of the C1 is used instead. As shown in FIG. 12, the non-neighboring spatial merge candidate is inserted into the time motion vector prediction (TmvP) (i.e., the temporal candidate) in the conventional merge candidate list, and the mode of the non-neighboring spatial merge candidate is shown in FIG. 12. The distance between the non-neighboring spatial candidate and the current coding block is based on the width and height of the current coding block. The non-neighboring spatial candidate includes the mv corresponding to the B01-B04, B11-B13, B21-B24, A11-A13, A01-A04 samples (or the mv corresponding to the CU where the samples are located). Optionally, the AMVP candidate list includes spatial candidates, temporal candidates, non-neighboring spatial candidates, etc. The candidate list construction method of the AMVP mode is similar to that of the merge mode, which will not be described here. Optionally, the list related to the block vector includes an IBC merge candidate list (block vector merge candidate list) and an IBC AMVP candidate list (block vector AMVP candidate list). The IBC merge candidate list includes neighboring spatial candidates bv, historical block vector candidates bv, and non-neighboring spatial candidates bv. The neighboring spatial candidates include the bv corresponding to the samples at the similar positions in FIG. 10, such as the upper right, lower left, and upper left samples. Similarly, the non-neighboring spatial candidates include the bv corresponding to the samples at the similar positions in FIG. 12.

[0222] Sixth, at least two prediction values and corresponding weight coefficients.

[0223] Optionally, at least two prediction values are determined or obtained according to the candidate elements and / or the first parameters corresponding to the candidate elements.

[0224] Optionally, the candidate elements include first candidate elements and second candidate elements, and the first prediction value and the second prediction value can be determined or obtained according to the first candidate elements and the second candidate elements, and / or the first parameters corresponding to the candidate elements.

[0225] Optionally, the candidate elements include at least one of motion vectors, block vectors, motion vector candidates, block vector candidates, motion vector predictors, block vector predictors, motion vector indexes, and block vector indexes.

[0226] Optionally, the first candidate element and the second candidate element are candidate elements in the candidate list corresponding to the lowest and the second lowest matching cost between the current template and the reference template.

[0227] Optionally, the first candidate element and the second candidate element are candidate elements in the candidate list corresponding to the lowest and the second lowest rate-distortion cost between the current block and the prediction block.

[0228] Optionally, the first candidate element and the second candidate element are candidate elements in the candidate list corresponding to the lowest and the second lowest matching cost between the current template and the reference template processed by the first parameter.

[0229] Optionally, the first candidate element and the second candidate element are candidate elements in the candidate list corresponding to the lowest and the second lowest rate-distortion cost between the current block and the prediction block processed by the first parameter.

[0230] Optionally, the candidate elements are determined or obtained from the candidate list after the reordering process.

[0231] Optionally, the candidate list comprises a merge candidate list or an AMVP candidate list of motion vectors or block vectors.

[0232] Optionally, in the merge mode, determining the first flag of the first image block comprises: taking the LIC flag corresponding to the optimal candidate mv pair with the minimum template matching cost as the first LIC flag of the first image block (current block); and taking the LIC flag corresponding to the suboptimal candidate mv pair with the second minimum template matching cost as the second LIC flag of the first image block (current block). According to the LIC flag, the illumination compensation parameter of the first image block is determined, and the illumination compensation parameter comprises the illumination compensation parameter corresponding to the optimal candidate mv (or optimal candidate bv) and the suboptimal candidate mv (or suboptimal candidate bv) in the reordered first list. For example, if the first LIC flag indicates enabling LIC, the illumination compensation parameter corresponding to the optimal candidate mv (or optimal candidate bv) in the reordered first list is taken as the first illumination compensation parameter of the first image block (current block); if the second LIC flag indicates enabling LIC, the illumination compensation parameter corresponding to the suboptimal candidate mv (or suboptimal candidate bv) in the reordered first list is taken as the second illumination compensation parameter of the first image block (current block).

[0233] Optionally, for inter prediction processing, a first prediction value of the first image block is determined using the optimal candidate mv; a second prediction value of the first image block is determined using the suboptimal candidate mv. The first prediction value is subjected to illumination compensation using the first illumination compensation parameter to obtain an illumination compensated first prediction value, and the second prediction value is subjected to illumination compensation using the second illumination compensation parameter to obtain an illumination compensated second prediction value. The illumination compensated first prediction value and the illumination compensated second prediction value are fused to obtain a final prediction value of the first image block.

[0234] Optionally, for intra block copy processing, a first prediction value of the first image block is determined using the optimal candidate bv; a second prediction value of the first image block is determined using the suboptimal candidate bv. The first prediction value is subjected to illumination compensation using the first illumination compensation parameter to obtain an illumination compensated first prediction value, and the second prediction value is subjected to illumination compensation using the second illumination compensation parameter to obtain an illumination compensated second prediction value. The illumination compensated first prediction value and the illumination compensated second prediction value are fused to obtain a final prediction value of the first image block.

[0235] Optionally, in the AMVP mode, the first flag of the first image block is determined according to the first list sorted according to the matching costs, including: taking the LIC flag corresponding to the optimal candidate mv with the minimum template matching cost as the first LIC flag of the first image block (current block); and taking the LIC flag corresponding to the suboptimal candidate mv with the second minimum template matching cost as the second LIC flag of the first image block (current block).

[0236] Optionally, the illumination compensation parameter of the first image block is determined according to the LIC flag. The illumination compensation parameter includes the illumination compensation parameter corresponding to the optimal candidate motion vector predictor (MVP) or the optimal candidate block vector predictor (BVP) or the suboptimal candidate motion vector predictor or the suboptimal candidate block vector predictor (BVP) in the first list sorted according to the matching costs. For example, if the first LIC flag indicates that the LIC is enabled, the illumination compensation parameter corresponding to the optimal candidate mv (or the optimal candidate bv) in the first list sorted according to the matching costs is taken as the first illumination compensation parameter of the first image block (current block); if the second LIC flag indicates that the LIC is enabled, the illumination compensation parameter corresponding to the suboptimal candidate mv (or the suboptimal candidate bv) in the first list sorted according to the matching costs is taken as the second illumination compensation parameter of the first image block (current block).

[0237] Optionally, for inter prediction, a first prediction value of the first image block is determined using an optimal candidate motion vector predictor and a first motion vector difference (MVD), and a second prediction value of the first image block is determined using a suboptimal candidate motion vector predictor and a second motion vector difference. For example, the first prediction value is determined using the first motion vector difference and a first motion vector, and the second prediction value is determined using the second motion vector difference and a second motion vector. The first motion vector and the second motion vector are motion vector predictors in an AMVP candidate list. The first prediction value is subjected to illumination compensation using a first illumination compensation parameter to obtain an illumination compensated first prediction value, and the second prediction value is subjected to illumination compensation using a second illumination compensation parameter to obtain an illumination compensated second prediction value. The illumination compensated first prediction value and the illumination compensated second prediction value are fused to obtain a final prediction value of the first image block.

[0238] Optionally, for intra block copy, a first prediction value of the first image block is determined using an optimal candidate block vector predictor and a first block vector difference (BVD), and a second prediction value of the first image block is determined using a suboptimal candidate block vector predictor and a second block vector difference. For example, the first prediction value is determined using the first block vector difference and a first block vector predictor, and the second prediction value is determined using the second block vector difference and a second block vector predictor. The first block vector and the second block vector are block vector predictors in a BVP candidate list. The first prediction value is subjected to illumination compensation using a first illumination compensation parameter to obtain an illumination compensated first prediction value, and the second prediction value is subjected to illumination compensation using a second illumination compensation parameter to obtain an illumination compensated second prediction value. The illumination compensated first prediction value and the illumination compensated second prediction value are fused to obtain a final prediction value of the first image block.

[0239] Optionally, in the AMVP mode, in addition to obtaining an optimal candidate motion vector predictor (MVP) or block vector predictor (BVP) of the first image block, a suboptimal motion vector predictor or block vector predictor of the first image block is also obtained. Determining the first flag of the first image block includes: taking an LIC flag corresponding to the optimal candidate motion vector predictor as a first LIC flag of the first image block (current block); and taking an LIC flag corresponding to the suboptimal candidate motion vector predictor as a second LIC flag of the first image block (current block).

[0240] Optionally, according to the LIC flag, a light compensation parameter of the first image block is determined. The light compensation parameter includes a light compensation parameter corresponding to the optimal candidate motion vector predictor (or the optimal candidate block vector predictor) and the suboptimal candidate motion vector predictor (or the suboptimal candidate block vector predictor) in the sorted first list. If the first LIC flag indicates enabling LIC, the light compensation parameter corresponding to the optimal candidate motion vector predictor (or the optimal candidate block vector predictor) in the sorted first list is taken as the first light compensation parameter of the first image block (the current block); if the second LIC flag indicates enabling LIC, the light compensation parameter corresponding to the suboptimal candidate motion vector predictor (or the suboptimal candidate block vector predictor) in the sorted first list is taken as the second light compensation parameter of the first image block (the current block).

[0241] Optionally, for inter prediction processing, a first prediction value of the first image block is determined by using the optimal candidate motion vector predictor and the first motion vector difference mvd; a second prediction value of the first image block is determined by using the suboptimal candidate motion vector predictor and the second motion vector difference mvd. The first prediction value is subjected to light compensation by using the first light compensation parameter to obtain a light-compensated first prediction value, and the second prediction value is subjected to light compensation by using the second light compensation parameter to obtain a light-compensated second prediction value. The light-compensated first prediction value and the light-compensated second prediction value are fused to obtain a final prediction value of the first image block.

[0242] Optionally, for inter prediction processing, a first prediction value of the first image block is determined by using the optimal candidate motion vector predictor and the first motion vector difference mvd; a second prediction value of the first image block is determined by using the suboptimal candidate motion vector predictor and the second motion vector difference mvd. The first prediction value is subjected to light compensation by using the first light compensation parameter to obtain a light-compensated first prediction value, and the second prediction value is subjected to light compensation by using the second light compensation parameter to obtain a light-compensated second prediction value. The light-compensated first prediction value and the light-compensated second prediction value are fused to obtain a final prediction value of the first image block.

[0243] Optionally, the weight coefficients are determined or obtained according to template matching cost or rate-distortion cost. For example, the template matching cost COST0 of the optimal candidate mv or bv and the template matching cost COST1 of the suboptimal mv or bv are calculated respectively, and the weight w0 and w1 are determined according to the ratio between the template matching cost COST0 and the template matching cost COST1; if the weighted processing is performed on the first prediction value and the second prediction value, the weighted result is calculated by using the following formula: Predw = w0*Pred0 + w1*Pred1, and optionally, Pred0 is the first prediction value, Pred1 is the second prediction value, and Predw is the prediction value after the weighted fusion. For another example, the rate-distortion cost COST0' of the optimal candidate mv or bv and the rate-distortion cost COST1' of the suboptimal mv or bv are calculated respectively, and the weight w0 and w1 are determined according to the ratio between the rate-distortion cost COST0' and the rate-distortion cost COST1'; if the weighted processing is performed on the first prediction value and the second prediction value, the weighted result is calculated by using the following formula: Predw = w0*Pred0 + w1*Pred1, and optionally, Pred0 is the first prediction value, Pred1 is the second prediction value, and Predw is the prediction value after the weighted fusion.

[0244] In the embodiment, the determination or obtaining manner of the first prediction value includes at least one of the following manner one to manner three:

[0245] Manner one, the second prediction value of the first image block is determined or obtained according to the first flag of the first image block and the first prediction value of the first image block.

[0246] Optionally, for each coded image block, there is an LIC flag.

[0247] Optionally, the second prediction value of the first image block is determined or obtained according to the first flag of the first image block and the first prediction value of the first image block, for example, when the first flag of the first image block is to enable the light compensation, the first prediction value of the first image block is taken as the second prediction value of the first image block.

[0248] Optionally, the first prediction value is determined or obtained according to at least one of the following: an inter prediction mode and / or an intra prediction mode; at least one of pixel value, position, motion information and prediction mode of the image block; the first flag; the first parameter; the candidate element; and the at least two prediction values and the corresponding weight coefficients.

[0249] Manner two, the second prediction value of the first image block is determined or obtained according to the first parameter of the first image block and the first prediction value of the first image block.

[0250] Optionally, the second prediction value of the first image block is determined or obtained by performing illumination compensation on the first prediction value of the first image block according to the first parameter of the first image block and the first prediction value of the first image block.

[0251] Optionally, the first parameter is determined or obtained according to at least one of the following: the first parameter corresponding to the image block, the sub-image block or the pixel; the first parameter corresponding to the candidate element in the candidate list; and the first parameter in the storage unit.

[0252] Optionally, the second prediction value of the first image block is determined or obtained according to the first flag of the first image block and the first parameter of the first image block, and the first prediction value of the first image block.

[0253] Optionally, when the first flag of the first image block is enabled for illumination compensation, the second prediction value of the first image block is determined or obtained by performing illumination compensation on the first prediction value of the first image block according to the first parameter of the first image block.

[0254] Optionally, for the obtained candidate mv or bv with the minimum matching cost, it can be determined that the candidate mv or bv is one of the adjacent spatial domain candidate, the temporal domain candidate and the non-adjacent spatial domain candidate, and it can be determined that the candidate mv or bv corresponds to the encoded image block or the encoded pixel sample from which the candidate mv or bv is obtained. For example, as shown in Table 1 and Table 2 below, a mapping relationship table about the LIC flag of the candidate mv or bv in the first list and the encoded block or the encoded pixel sample related to the candidate mv or bv can be constructed:

[0255] Table 1: Mapping relationship table of mv and LIC

[0256] Table 2: Mapping relationship table of bv and LIC

[0257] Optionally, the layout illumination compensation technology strategy adopted by the current block is the same as the layout illumination compensation technology strategy adopted by the candidate mv with the minimum template matching cost (or the encoding block in which the candidate mv with the minimum template matching cost is located).

[0258] Optionally, if the candidate motion vector mv3 in FIG. 9 is a neighboring spatial candidate mv, the candidate motion vector mv3 is a corresponding mv of the top-left sample in FIG. 10, or the candidate motion vector mv3 is derived from a corresponding mv of the top-left sample B2 in FIG. 10, a value of a LIC flag corresponding to the top-left sample B2 in FIG. 10 (or a LIC flag corresponding to a coding block in which the top-left sample B2 is located) is taken as a value of a LIC flag of the current image block. Optionally, if the value of the LIC flag corresponding to the top-left sample B2 (or the LIC flag corresponding to the coding block in which the top-left sample B2 is located) is 0, the value of the LIC flag of the current image block is 0. If the value of the LIC flag corresponding to the top-left sample B2 (or the LIC flag corresponding to the coding block in which the top-left sample B2 is located) is 1, the value of the LIC flag of the current image block is 1.

[0259] Optionally, if the candidate motion vector mv3 in FIG. 9 is a temporal candidate mv, the candidate motion vector mv3 is a corresponding mv of the bottom-right sample C0 in FIG. 11, or the candidate motion vector mv3 is derived from a corresponding mv of the bottom-right sample C0 in FIG. 11, a value of a LIC flag corresponding to the bottom-right sample C0 in FIG. 11 (or a LIC flag corresponding to a coding block in which the bottom-right sample C0 is located) is taken as a value of a LIC flag of the current image block. Optionally, if the value of the LIC flag corresponding to the bottom-right sample C0 (or the LIC flag corresponding to the coding block in which the bottom-right sample C0 is located) is 0, the value of the LIC flag of the current image block is 0. If the value of the LIC flag corresponding to the bottom-right sample C0 (or the LIC flag corresponding to the coding block in which the bottom-right sample C0 is located) is 1, the value of the LIC flag of the current image block is 1.

[0260] Optionally, if the candidate motion vector mv3 in FIG. 9 is a non-neighboring spatial candidate mv, the candidate motion vector mv3 is a corresponding mv of the left sample A11 in FIG. 12, or the candidate motion vector mv3 is derived from a corresponding mv of the left sample A11 in FIG. 12, a value of a LIC flag corresponding to the left sample A11 in FIG. 12 (or a LIC flag corresponding to a coding block in which the left sample A11 is located) is taken as a value of a LIC flag of the current image block. Optionally, if the value of the LIC flag corresponding to the left sample A11 (or the LIC flag corresponding to the coding block in which the left sample A11 is located) is 0, the value of the LIC flag of the current image block is 0. If the value of the LIC flag corresponding to the left sample A11 (or the LIC flag corresponding to the coding block in which the left sample A11 is located) is 1, the value of the LIC flag of the current image block is 1.

[0261] Optionally, the value of the LIC flag is 1, indicating that the illumination compensation is adopted; and the value of the LIC flag is 0, indicating that the illumination compensation is not adopted.

[0262] Optionally, at least one of the first prediction value and the first parameter can be input into a local illumination compensation module, and a second prediction value is output.

[0263] Optionally, if a nonlinear local illumination compensation model is: PreY1'=f(PreY1, PreY2, PreY3,..., PreYN) (Formula 1)

[0264] In order to reduce the calculation amount, an offset value can be set in Formula 1, and a group of offset values can be selected from the upper template and the left template corresponding to the current block. The pixel value of the top-left corner pixel of the upper current template can be selected as the offset value CurOffset, and the pixel value of the top-left corner pixel of the upper reference template can be selected as the offset value RefOffset. Optionally, if the offset value is set, Formula (1) can be updated to Formula (9), that is: PreY1'=f(PreY1-RefOffset, PreY2-RefOffset, PreY3-RefOffset,..., PreYN-RefOffset)+CurOffset Formula (9)

[0265] Optionally, if the first parameter needs to be determined, it can be determined according to the following Formula (10), that is: CurY1=f(RefY11-RefOffset, RefY12-RefOffset, RefY13-RefOffset,..., RefY1N-RefOffset)+CurOffset Formula (10)

[0266] Optionally, if a nonlinear local illumination compensation model can be: PreY1'=f(PreY1, PreY2, PreY3,..., PreYN)

[0267] Optionally, the luminance component is exemplified. PreY1 is the predicted pixel value of the luminance pixel Y1 determined by the first prediction mode (or the first prediction mode). PreY2, PreY3,..., PreYN are the predicted pixel values of other luminance pixels Y2, Y3,..., YN around the luminance pixel Y1 determined by the first prediction mode (or the first prediction mode). By the first prediction mode, the predicted luminance pixel values of the luminance pixel Y1 and other luminance pixels Y2, Y3,..., YN around it are determined. By these predicted luminance pixel values, the local illumination compensation is further performed on the luminance pixel Y1 to obtain the predicted luminance pixel value PreY1' of the luminance pixel Y1 after the compensation processing. The first prediction mode can be a default prediction mode, an inter-frame prediction mode, an intra-frame prediction mode, a prediction mode in a candidate list, such as at least one of a prediction mode corresponding to motion estimation, motion compensation, sub-pixel interpolation, a Merge mode, a geometric partitioning mode, an advanced motion vector prediction mode, a sub-block-based inter-frame prediction mode, and the like.

[0268] Optionally, the nonlinear local illumination compensation model can also be formula (2), that is: PreY1' = c0*PreY1 + c1*PreY2 + c2*PreY3 + c3*PreY4 + c4*PreY5 + c5*P1 + c6*P2 + c7*P3 + c8*P4 + c9*P5 + c10*B formula (2)

[0269] It can also be obtained by deforming formula (2): PreY1' = c0*PreY1 + c1*PreY2 + c2*PreY3 + c3*PreY4 + c4*PreY5 + ca*diagonal pixel + cb*position information + cc*gradient information + c5*P1 + c6*P2 + c7*P3 + c8*P4 + c9*P5 + c10*B

[0270] Optionally, c0-c10, ca, cb, cc are model parameters, i.e. first parameters (such as filter coefficients or weight coefficients). PreY1-PreY5 are predicted pixel values determined by the first prediction mode (or the first prediction mode). Optionally, PreY1 is a predicted pixel value of the luma pixel Y1 determined by the first prediction mode (or the first prediction mode), and PreY2, PreY3,..., PreY5 are predicted pixel values of other luma pixels Y2, Y3,..., Y5 around the luma pixel Y1 determined by the first prediction mode. P1-P5 are non-linear terms. Optionally, P1-P5 are squares of PreY1-PreY5. In another embodiment, P1-P5 are squares of PreY1-PreY5 and are scaled to a bit depth range. That is, P1=(PreY1*PreY1+midVal)>>bitDepth; bitDepth is a bit depth of a sample, and ">>" is a right shift sign. For example, for 10-bit video content, P is calculated by the following formula: P=(PreY1*PreY1+512)>>10

[0271] Optionally, the bias value B is further included in the formula (2). In an embodiment, the bias value B can be 0. In another embodiment, it can be other predetermined values (for example, for 10-bit video, it can be set to 512).

[0272] Optionally, the non-linear term in the formula (2) can only include 1 term. That is, only the square of the pixel value of PreY1. At this time, the formula (2) has a form as shown in the formula (3): PreY1'=c0*PreY1+c1*PreY2+c2*PreY3+c3*PreY4+c4*PreY5+c5*P1+c6*B formula (3)

[0273] The positional relationship of the luma pixels Y1-Y5 can be as shown in FIG. 13, the luma pixel Y1 is located in the middle of the luma pixels Y2-Y5, wherein the luma pixel Y2 is located above the luma pixel Y1, the luma pixel Y3 is located on the left side of the luma pixel Y1, the luma pixel Y4 is located on the right side of the luma pixel Y1, and the luma pixel Y5 is located below the luma pixel Y1. Optionally, the luma predicted pixel values PreY1-PreY5 of the luma pixels Y1-Y5 determined by the first prediction mode (or the first prediction mode) are as shown in FIG. 14.

[0274] Optionally, the second prediction value of the first pixel is determined or obtained according to at least one of the first prediction value, the offset value, the non-linear term, the position information, the gradient information and the first parameter, so as to guarantee that the obtained second prediction value is more accurate and effective, and thus the efficiency of video encoding and / or decoding is improved.

[0275] Optionally, the local illumination compensation module of any one of the above-mentioned modes 1 to 3 of the present embodiment can refer to FIG. 15 and FIG. 16. As shown in FIG. 15, the first prediction value can be PreY1-PreY5, the bias value is B, the first parameter is C0-C9, and P1-P5 are nonlinear terms. The second prediction value PreY1' is calculated by the summation module. As shown in FIG. 16, the first prediction value can be PreY1-PreY5, the bias value is B, the first parameter is C0-C6, and P1 is a nonlinear term. The second prediction value PreY1' is calculated by the summation module.

[0276] Optionally, the connection relationship between the local illumination compensation module and the first prediction module in the inter-prediction module can refer to FIG. 17, which includes the pixel to be predicted, the inter-prediction module, and the predicted pixel value PreY1 after compensation. The first prediction module and the nonlinear local illumination compensation module are arranged in the inter-prediction module, and the prediction pixel value (such as the first prediction value) input into the inter-prediction module includes PreY1, PreY2, PreY3, PreY4, and PreY5.

[0277] Optionally, the first prediction module and the nonlinear local illumination compensation module are both in the inter-prediction module. Optionally, in other embodiments, at least one of the first prediction module and the nonlinear local illumination compensation module is located in the inter-prediction module, and both of them do not need to be located in the inter-prediction module; or at least one of the first prediction module and the nonlinear local illumination compensation module can be located in the intra-prediction module, or both of them are located in the intra-prediction module.

[0278] Optionally, the input of the subsequent non-linear local illumination compensation module is determined by a first prediction mode corresponding to the first prediction module. That is, at least part of the input of the non-linear local illumination compensation module is the predicted pixel value determined by the first prediction module. In an embodiment, the first prediction mode can be an inter prediction mode. The motion vector involved in the inter prediction mode can be a motion vector determined by motion estimation, a motion vector determined by a merge motion vector prediction list (MergeMVP list) in a merge mode, a motion vector determined by a motion vector prediction list (MVP list) in an advanced motion vector prediction mode, a motion vector obtained in a symmetric motion vector difference mode, a motion vector determined based on a sub-block based temporal motion vector prediction mode, a motion vector determined based on a sub-block based affine motion compensation mode, a motion vector determined in an affine merge mode, a motion vector obtained in an affine advanced motion vector prediction mode, a motion vector obtained by a decoder-side motion vector refinement (DMVR) technology, or a motion vector determined by a motion vector prediction list (MVP list) in a geometric partition mode. After the motion vector is determined in the first prediction mode, the predicted pixel value of the pixel in the to-be-predicted block is determined by using the motion vector, so as to further perform the non-linear local illumination compensation processing.

[0279] Optionally, the technical scheme of the present application can be applied to the encoding end. Referring to FIG. 6, various module units are included, such as a transform module, a quantization module, an entropy encoding module, an inverse quantization module, an inverse transform module, an intra prediction module, an inter prediction module, an encoded image buffer, and a loop filtering module. In the encoding end, after the encoder receives a video image (i.e., input video data in FIG. 6) from a video source, the input image is divided into image blocks (the image blocks include luminance blocks and chrominance blocks). The image blocks are predicted by using the temporal and / or spatial correlation between the video images.

[0280] Optionally, the prediction processing includes intra prediction processing and / or inter prediction processing, and / or the intra prediction processing and / or the inter prediction processing includes at least one prediction mode. For these prediction modes, the encoder uses an optimization strategy (e.g., rate-distortion optimization) to determine the prediction mode finally adopted by the image block. For example, the rate-distortion cost corresponding to each prediction mode or the rate-distortion cost of the combination of several prediction modes is calculated to determine the minimum rate-distortion cost from at least one rate-distortion cost. The prediction mode or the combination of prediction modes corresponding to the minimum rate-distortion cost is the prediction mode finally adopted by the image block. These prediction modes include the local illumination compensation mode in this embodiment, i.e., local illumination compensation processing is performed on the basis of the first prediction value obtained by the first prediction mode to determine the pixel value of the final to-be-predicted pixel (i.e., the second prediction value of the first pixel). That is, the first prediction value of the first to-be-predicted pixel in the to-be-predicted block is determined, and the second prediction value of the first pixel is determined according to the first prediction value and the first parameter. In the prediction block (such as a prediction luminance block and a prediction chrominance block) corresponding to the image block is obtained according to the second prediction value, the pixel value of the pixel sample in the image block is subtracted from the prediction value of the corresponding pixel sample in the prediction block to obtain the residual value of the pixel sample and the residual block corresponding to the image block, the residual block is subjected to transformation and quantization processing, and then is encoded by the entropy encoder to form the encoded bitstream. Optionally, the encoded bitstream further includes the prediction parameter corresponding to the determined prediction mode and related auxiliary information (e.g., side information).

[0281] Optionally, the prediction parameter is packaged into the encoded bitstream after being subjected to entropy encoding.

[0282] Optionally, the prediction parameter further includes indication information of the prediction mode.

[0283] Optionally, the transformed and quantized residual block is added to the corresponding prediction block obtained by using the prediction mode to obtain a reconstructed block, and then the reconstructed block is subjected to loop filtering processing by the loop filtering module and the filter control data, thereby reducing distortion.

[0284] Optionally, the present embodiment can be applied to a decoding end, and as shown in FIG. 7, includes various module units, such as entropy decoding, inverse quantization, inverse transformation, intra prediction, inter prediction, loop filtering, and a decoded image buffer. In the decoding end, after receiving the encoded bitstream, the decoding unit of the decoder parses and decodes the encoded bitstream to obtain the transform coefficient; the inverse transformation unit and the inverse quantization unit of the decoder perform inverse transformation and inverse quantization processing on the transform coefficient to obtain the residual block.

[0285] Optionally, a decoding unit of the decoder parses and decodes the coded bitstream to obtain the prediction parameters and the related auxiliary information. A prediction processing unit of the decoder performs prediction processing using the prediction parameters to determine a prediction block corresponding to the residual block; optionally, the prediction processing includes intra prediction processing and / or inter prediction processing, and the intra prediction processing and / or the inter prediction processing respectively include at least one prediction mode in a combined manner. Optionally, the prediction parameters indicate that the corresponding prediction mode is the prediction mode corresponding to the residual block.

[0286] Optionally, at the decoding end, the second prediction value of the first pixel can be determined or obtained according to the first prediction value and the first parameter. For example, the first pixel value of the first pixel sample of the first color component is obtained or determined; and the second pixel value of the second pixel sample of the first color component and / or the second color component is determined according to the first pixel of the first pixel sample. Then, after determining all pixel samples in the to-be-predicted block, the prediction result of the to-be-predicted block is obtained. Then, the obtained residual block and the corresponding prediction block (including the predicted luma block and the predicted chroma block) are added to obtain a reconstructed block. The loop filtering unit of the decoder performs loop filtering processing on the reconstructed block according to the filter control data to reduce distortion and improve video quality. The reconstructed block after the loop filtering processing is further combined into a decoded image and stored in a decoded image buffer or output as a decoded video signal.

[0287] The embodiment can improve the prediction quality of the first image block by performing illumination compensation on the first prediction value of the first image block, thereby improving the video encoding and / or decoding performance.

[0288] Third embodiment

[0289] The embodiment also provides a processing apparatus. Please refer to FIG. 18, which is a functional module schematic diagram of the processing apparatus. The processing apparatus can be arranged in or be a processing device. The processing apparatus includes:

[0290] The processing module A10 is configured to determine or obtain the second prediction value of the first image block according to the first prediction value of the first image block.

[0291] Optionally, the first prediction value is determined or obtained according to at least one of the following:

[0292] An inter prediction mode and / or an intra prediction mode;

[0293] At least one of a pixel value, a position, motion information, and a prediction mode of the image block;

[0294] A first flag;

[0295] A first parameter;

[0296] A candidate element;

[0297] at least two prediction values and corresponding weight coefficients.

[0298] Optionally, the processing method further comprises determining or obtaining a second prediction value of the first image block based on the first flag, the first parameter, the candidate element, or the candidate list.

[0299] Optionally, the first flag is an illumination compensation flag.

[0300] Optionally, the first parameter is an illumination compensation parameter.

[0301] Optionally, the candidate element comprises a first candidate element and a second candidate element.

[0302] Optionally, the candidate element comprises at least one of a motion vector, a block vector, a motion vector candidate, a block vector candidate, a motion vector predictor, a block vector predictor, a motion vector index, or a block vector index.

[0303] Optionally, the candidate element is a candidate element in the candidate list corresponding to a lowest matching cost between the current template and the reference template.

[0304] Optionally, the candidate element is a candidate element in the candidate list corresponding to a lowest rate-distortion cost between the current block and the prediction block.

[0305] Optionally, the candidate element is a candidate element in the candidate list corresponding to a lowest matching cost between the current template and the reference template processed by the first parameter.

[0306] Optionally, the candidate element is a candidate element in the candidate list corresponding to a lowest rate-distortion cost between the current block and the prediction block processed by the first parameter; and the candidate element is determined or obtained from the candidate list after the reordering processing.

[0307] Optionally, the candidate list comprises a merge candidate list or an AMVP candidate list of motion vectors or block vectors.

[0308] Optionally, the at least two prediction values are determined or obtained based on the candidate element and / or the first parameter corresponding to the candidate element.

[0309] Optionally, the processing module A10 is further configured to perform at least one of the following:

[0310] determine or obtain a second prediction value of the first image block based on a first flag of the first image block and a first prediction value of the first image block;

[0311] determine or obtain a second prediction value of the first image block based on a first parameter of the first image block and a first prediction value of the first image block;

[0312] The second prediction value of the first image block is determined or obtained according to the first flag of the first image block, the first parameter of the first image block, and the first prediction value of the first image block.

[0313] Optionally, the first parameter is determined or obtained according to at least one of the following:

[0314] The first parameter corresponding to the image block, the sub-image block, or the pixel;

[0315] The first parameter corresponding to the candidate element in the candidate list;

[0316] The first parameter in the storage unit.

[0317] Optionally, the image block comprises at least one of the following:

[0318] The image block spatially or temporally adjacent to the first image block;

[0319] The image block spatially or temporally non-adjacent to the first image block;

[0320] The image block or the sub-image block determined by the motion offset;

[0321] The co-located image block of the first image block;

[0322] The image block or the sub-image block determined by the sub-block temporal motion vector prediction processing.

[0323] Optionally, the first flag is determined or obtained according to at least one of the following:

[0324] The first flag corresponding to the candidate element with the lowest matching cost;

[0325] The first flag corresponding to the candidate element with the lowest and the second lowest matching cost;

[0326] The first flag corresponding to the candidate element with the lowest rate-distortion cost;

[0327] The first flag corresponding to the candidate element with the lowest and the second lowest rate-distortion cost.

[0328] Optionally, the processing module A10 is further configured to perform:

[0329] If the first parameter of the first image block does not exist, the first parameter of the first image block is determined according to the pixels in the adjacent region of the first image block.

[0330] The processing device provided by the embodiments of the present application has similar implementation principles and beneficial effects to the technical solutions shown in the above corresponding method embodiments, and will not be described here in detail.

[0331] The embodiment of the present application further provides an intelligent terminal, comprising a memory and a processor, and the memory stores a processing program which, when executed by the processor, implements the steps of the processing method in any of the above embodiments.

[0332] The embodiment of the present application further provides a computer readable storage medium, which stores a processing program, and the processing program, when executed by a processor, implements the steps of the processing method in any of the above embodiments.

[0333] In the embodiments of the intelligent terminal and the computer readable storage medium provided by the present application, all the technical features of any of the above processing method embodiments can be included, and the description and explanation content is basically the same as that of the above method embodiments, which will not be repeated here.

[0334] The embodiment of the present application further provides a computer program product, which comprises computer program code, and when the computer program code runs on a computer, the computer executes the method in various possible embodiments as above.

[0335] The embodiment of the present application further provides a chip, comprising a memory and a processor, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the device installed with the chip executes the method in various possible embodiments as above.

[0336] It can be understood that the above scenarios are only examples and do not constitute a limitation on the application scenarios of the technical solutions provided by the embodiments of the present application. The technical solutions provided by the present application can also be applied to other scenarios. For example, those skilled in the art can know that, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0337] The above sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0338] The steps in the method of the embodiments of the present application can be adjusted, combined and deleted according to actual needs.

[0339] The units in the device of the embodiments of the present application can be combined, divided and deleted according to actual needs.

[0340] In the present application, for the same or similar term concept, technical solution and / or application scenario description, generally only the first time is described in detail, and for the sake of brevity, the repeated description is generally not repeated, and for the understanding of the technical solutions of the present application, the same or similar term concept, technical solution and / or application scenario description which is not described in detail can refer to the previous related detailed description.

[0341] In the present application, the description of each embodiment is focused on, and the part not described or recorded in a certain embodiment can be referred to the relevant description of other embodiments.

[0342] The technical features of the technical solutions of the present application can be combined arbitrarily, and in order to make the description simple, the possible combinations of each technical feature in the above-mentioned embodiments are not described, however, as long as the combinations of the technical features do not exist contradictory, it should be considered as the range recorded in the present application.

[0343] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment method can be realized by software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of software product, which is stored in the above-mentioned storage medium (such as ROM / RAM, magnetic disk, optical disk), including a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, controlled terminal, or network device, etc.) execute the method of each embodiment of the present application.

[0344] In the above-mentioned embodiments, all or part of them can be realized by software, hardware, firmware or any combination thereof. When realized by software, it can be realized in the form of computer program product in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the flow or function according to the embodiments of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. Computer instructions can be stored in a storage medium or transferred from one storage medium to another, for example, computer instructions can be transferred from one website site, computer, server or data center to another website site, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) mode. The storage medium can be any available medium that can be accessed by the computer or the data storage device such as server, data center, etc. containing one or more available media. The available medium can be magnetic medium (such as floppy disk, storage disk, magnetic tape), optical medium (such as DVD), or semiconductor medium (such as solid state storage disk (SSD)) and the like.

[0345] The above merely preferred embodiments of the present application and are not intended to limit the patent scope of the present application, any equivalent structure or equivalent process transformation using the content of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A processing method, wherein: Including steps: S10. Determine or obtain a second prediction value of the first image block according to the first prediction value of the first image block.

2. The method according to claim 1, wherein The first prediction value is determined or obtained according to at least one of the following: inter prediction mode and / or intra prediction mode; At least one of the pixel value, position, motion information, and prediction mode of the image block; First Sign; First parameter; candidate elements; At least two predicted values ​​and corresponding weight coefficients.

3. The method according to claim 2, wherein: Also include at least one of the following: The first sign is the light compensation sign; The first parameter is the illumination compensation parameter; The candidate elements include a first candidate element and a second candidate element; The candidate element includes at least one of a motion vector, a block vector, a motion vector candidate, a block vector candidate, a motion vector predictor, a block vector predictor, a motion vector index, and a block vector index; The candidate element is the candidate element in the candidate list corresponding to the lowest matching cost between the current template and the reference template; The candidate element is the candidate element in the candidate list corresponding to the lowest rate-distortion cost between the current block and the predicted block; The candidate element is the candidate element in the candidate list corresponding to the lowest matching cost between the current template and the reference template processed by the first parameter; The candidate element is a candidate element in the candidate list corresponding to the lowest rate-distortion cost between the current block and the predicted block processed by the first parameter; The candidate elements are determined or obtained from the candidate list after the re-ranking process; The candidate list includes a merge candidate list or an AMVP candidate list of motion vectors or block vectors; At least two prediction values ​​are determined or obtained based on the candidate elements and / or the first parameters corresponding to the candidate elements.

4. The method according to claim 2, wherein: Step S10 includes at least one of the following: Determining or obtaining a second prediction value of the first image block according to the first flag of the first image block and the first prediction value of the first image block; Determine or obtain a second prediction value of the first image block according to the first parameter of the first image block and the first prediction value of the first image block; A second prediction value of the first image block is determined or obtained according to the first flag of the first image block, the first parameter of the first image block, and the first prediction value of the first image block.

5. The method according to claim 2, wherein: The first parameter is determined or obtained according to at least one of the following: A first parameter corresponding to an image block, a sub-image block or a pixel; The first parameter corresponding to the candidate element in the candidate list; The first parameter in the storage unit.

6. The method of claim 2, wherein: The image block includes at least one of the following: an image block spatially or temporally adjacent to the first image block; The first image block is not spatially or temporally adjacent to an image block; An image block or sub-image block determined by motion offset; a co-located image block of the first image block; The image block or sub-image block is determined by the sub-block temporal motion vector prediction process.

7. The method of claim 2, wherein: The first flag is determined or obtained by at least one of the following: Match the first flag corresponding to the candidate element with the lowest cost; Match the first flags corresponding to the candidate elements with the lowest and second lowest cost; The first flag corresponding to the candidate element with the lowest rate-distortion cost; The first flags corresponding to the candidate elements with the lowest and second lowest rate-distortion costs.

8. The method of claim 2, wherein: Also includes: If the first parameter of the first image block does not exist, the first parameter of the first image block is determined according to pixels in an adjacent area of ​​the first image block.

9. A processing device, characterized in that include: A memory, a processor, and a processing program stored in the memory and operable on the processor, wherein the processing program implements the steps of the processing method according to claim 1 when executed by the processor.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a processing program, which implements the steps of the processing method according to claim 1 when executed by a processor.

Images