Image processing method, processing equipment and storage medium

Abstract

The invention provides an image processing method, processing equipment and a storage medium. The image processing method comprises the following steps: S1, predicting image block chrominance information according to correlation between two chrominance components; in the image coding and decoding process, the correlation between one brightness component and two chroma components of the image can be considered in the process of performing chroma prediction on the video image, the accuracy of performing chroma prediction on the chroma components in the image can be improved, and / or the overall efficiency of video image coding and decoding can be improved.

Description

Image processing method, processing device and storage medium Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, processing device and storage medium. Background Art

[0002] In the development of video coding technology, improvements made by various video coding standards are all committed to improving the video coding effect from different aspects. For example, in video image encoding and decoding, how to accurately predict the image chrominance is a hot topic in current research.

[0003] During the process of conceiving and implementing the present application, the inventors discovered that there are at least the following problems: the accuracy of the results obtained by currently predicting the chrominance components of video images is relatively poor, and / or the overall efficiency of video image encoding is relatively low.

[0004] The preceding description is intended to provide general background information and does not necessarily constitute prior art.

[0005] Summary of the Invention

[0006] In response to the above technical problems, the present application provides an image processing method, a processing device and a storage medium. In the process of performing chrominance prediction on a video image, the correlation between a luminance component and two chrominance components of the image is considered, which can improve the accuracy of chrominance prediction for the chrominance components in the image and / or improve the overall efficiency of video image encoding and decoding.

[0007] This application provides an image processing method that can be applied to a processing device (such as a mobile phone, server, etc.), comprising the following steps:

[0008] S1: Predict the chrominance information of the image block based on the correlation between the two chrominance components.

[0009] Optionally, step S1 includes at least one of the following:

[0010] Predicting second chrominance information of the image block according to the first chrominance prediction information of the image block;

[0011] Predicting second chrominance information of the image block based on the first chrominance prediction information of the image block and the first chrominance prediction model;

[0012] The second chroma information of the image block is predicted according to the first chroma prediction information of the image block and the second chroma prediction model.

[0013] Optionally, predicting the second chrominance information of the image block according to the first chrominance prediction information of the image block includes at least one of the following:

[0014] Acquire or determine first chroma reconstruction information according to the first chroma prediction information, and predict second chroma information according to the first chroma reconstruction information;

[0015] predicting the second chrominance information based on the first chrominance reconstruction information and the second chrominance information of the adjacent image block;

[0016] Second chroma initial prediction information is obtained or determined according to the first chroma prediction information, the second chroma initial prediction information is determined as an initial prediction result, and a prediction of the second chroma information is obtained or determined according to the initial prediction result.

[0017] Optionally, the acquiring or determining first chroma reconstruction information according to the first chroma prediction information includes:

[0018] The first chroma reconstruction information is determined or derived according to the first chroma prediction information and the residual information.

[0019] Optionally, the method for obtaining the residual information includes at least one of the following:

[0020] Obtain or determine residual information based on the first chroma prediction information;

[0021] Obtain or determine residual information from an image coding stream.

[0022] Optionally, the method further includes:

[0023] An initial prediction result is obtained or determined according to brightness information of the image block and / or adjacent image blocks.

[0024] Optionally, the obtaining or determining the initial prediction result according to the brightness information of the image block and / or the adjacent image blocks includes at least one of the following:

[0025] Predicting the second chrominance information according to the luminance information, and using the predicted second chrominance initial prediction information as the initial prediction result;

[0026] Predicting the second chrominance information according to the adjacent image blocks, and using the predicted second chrominance initial prediction information as the initial prediction result;

[0027] The second chroma information is predicted according to the second chroma reconstruction information of the adjacent image block, and the second chroma initial prediction information obtained by the prediction is used as the initial prediction result.

[0028] Optionally, the method further includes:

[0029] The first chrominance information of the image block is predicted according to the luminance information of the image block to determine or obtain the first chrominance prediction information.

[0030] The present application also provides a processing device, comprising: a memory and a processor, wherein an image processing program is stored in the memory, and when the image processing program is executed by the processor, the steps of any of the above-mentioned image processing methods are implemented.

[0031] The present application also provides a storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-mentioned image processing methods.

[0032] As described above, the image processing method of the present application can be applied to a processing device terminal and includes the following steps: S1: predicting chrominance information of an image block based on the correlation between two chrominance components. Through the above technical solution, the correlation between a luminance component and two chrominance components of an image can be considered during chrominance prediction for a video image, thereby improving the accuracy of chrominance prediction for chrominance components in the image and / or improving the overall efficiency of video image encoding and decoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for describing the embodiments. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without inventive work.

[0034] FIG1 is a schematic diagram of the hardware structure of a processing device for implementing various embodiments of the present application;

[0035] FIG2 is a diagram of a communication network system architecture provided by an embodiment of the present application;

[0036] FIG3 is a schematic diagram of an application scenario involving a convolutional cross component model prediction mode CCCM;

[0037] FIG4 is a schematic diagram of another application scenario involving the convolutional cross component model prediction mode CCCM;

[0038] FIG5A is a schematic diagram of an image coding scenario involved in a processing method provided in an embodiment of the present application;

[0039] FIG5B is a schematic diagram of an image decoding scenario involved in the processing method provided in an embodiment of the present application;

[0040] FIG6 is a schematic flow chart of an image processing method according to the first embodiment;

[0041] FIG7 is a schematic diagram of a chroma prediction framework involved in an image processing method according to a second embodiment;

[0042] FIG8 is a schematic diagram of a chroma prediction scenario involved in an image processing method according to a third embodiment;

[0043] FIG9 is a schematic diagram of another chroma prediction scenario involved in an image processing method according to a third embodiment;

[0044] FIG10 is a schematic diagram of another chroma prediction scenario involved in the image processing method according to the third embodiment;

[0045] FIG11 is a schematic diagram of another chroma prediction scenario involved in the image processing method according to the third embodiment;

[0046] FIG12 is a schematic diagram of another chroma prediction scenario involved in the image processing method according to the third embodiment.

[0047] The purpose of this application, its features, and advantages will be further described in conjunction with the embodiments and with reference to the accompanying drawings. The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and the accompanying text are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of this application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0048] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0049] It should be noted that, in this document, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0050] It should be understood that although the terms "first," "second," "third," etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are used solely to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the term "if," as used herein, may be interpreted as "upon," "when," or "in response to a determination." Furthermore, as used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It should be further understood that the terms "comprising" and "including" indicate the presence of the recited features, steps, operations, elements, components, items, types, and / or groups, but do not preclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, types, and / or groups. The terms "or," "and / or," "including at least one of the following," etc., as used herein, may be interpreted as inclusive, meaning any one or any combination. For example, “comprising at least one of the following: A, B, C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”; and for another example, “A, B or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”. An exception to this definition will occur only when a combination of elements, functions, steps or operations are inherently mutually exclusive in some manner.

[0051] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence in the order indicated by the arrows. Unless clearly stated herein, the execution of these steps is not strictly limited in order, and they can be performed in other orders. Moreover, at least a portion of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0052] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0053] It should be noted that in this article, step codes such as S1 are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the order. When implementing the step, those skilled in the art may execute other steps before executing S1, etc., but these should all be within the scope of protection of this application.

[0054] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0055] In the subsequent description, the use of suffixes such as "module", "component" or "unit" to represent elements is only for the purpose of facilitating the description of the present application and has no specific meaning. Therefore, "module", "component" or "unit" can be used interchangeably.

[0056] The processing device can be implemented in various forms. For example, the processing device described in this application can be a server, or a smart terminal such as a mobile phone, tablet computer, laptop computer, PDA, portable media player (PMP), navigation device, wearable device, smart bracelet, pedometer, etc., or a fixed terminal such as a digital TV, desktop computer, etc.

[0057] The subsequent description will be made by taking a mobile terminal as an example. It will be understood by those skilled in the art that, in addition to components specifically used for mobile purposes, the configuration according to the embodiments of the present application can also be applied to fixed-type terminals.

[0058] Please refer to Figure 1, which is a schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present application. The mobile terminal 100 may include components such as an RF (Radio Frequency) unit 101, a WiFi module 102, an audio output unit 103, an A / V (Audio / Video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111. Those skilled in the art will understand that the mobile terminal structure shown in Figure 1 does not limit the mobile terminal. The mobile terminal may include more or fewer components than shown, or may combine certain components, or arrange the components differently.

[0059] The following is a detailed introduction to the various components of the mobile terminal in conjunction with Figure 1:

[0060] The RF unit 101 can be used to receive and transmit signals during information transmission or calls. Specifically, it receives downlink information from the base station and transmits it to the processor 110 for processing. Optionally, it can send uplink data to the base station. Typically, the RF unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, and more. Furthermore, the RF unit 101 can communicate with the network and other devices via wireless communication. The above-mentioned wireless communications can use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution) and 5G, etc.

[0061] WiFi is a short-range wireless transmission technology. A mobile terminal, through WiFi module 102, enables users to send and receive emails, browse web pages, and access streaming media, providing wireless broadband Internet access. Although FIG1 illustrates WiFi module 102, it is understood that it is not a required component of the mobile terminal and can be omitted as needed without altering the essence of the invention.

[0062] The audio output unit 103 can convert audio data received by the RF unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output it as sound when the mobile terminal 100 is in a call signal reception mode, a talk mode, a recording mode, a voice recognition mode, a broadcast reception mode, or the like. Furthermore, the audio output unit 103 can also provide audio output related to a specific function performed by the mobile terminal 100 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 103 may include a speaker, a buzzer, or the like.

[0063] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos captured by an image capture device (e.g., a camera) in video capture mode or image capture mode. The processed image frames may be displayed on the display unit 106. The image frames processed by the GPU 1041 may be stored in the memory 109 (or other storage medium) or transmitted via the RF unit 101 or the WiFi module 102. The microphone 1042 may receive sound (audio data) in operating modes such as a phone call mode, a recording mode, and a voice recognition mode, and may process such sound into audio data. In the phone call mode, the processed audio (voice) data may be converted into a format that can be transmitted to a mobile communication base station via the RF unit 101. The microphone 1042 may implement various types of noise cancellation (or suppression) algorithms to eliminate (or suppress) noise or interference generated during the reception and transmission of audio signals.

[0064] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Optionally, the light sensor includes an ambient light sensor and a proximity sensor. Optionally, the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 1061 and / or the backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that recognize the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be described here.

[0065] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0066] The user input unit 107 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile terminal. Optionally, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also known as a touch screen, can collect user touch operations on or near it (such as operations performed by the user using a finger, stylus, or any other suitable object or accessory on or near the touch panel 1071) and drive the corresponding connection device according to a pre-set program. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Optionally, the touch detection device detects the user's touch direction and detects the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device and converts it into touch point coordinates, which are then sent to the processor 110. It can also receive commands sent by the processor 110 and execute them. In addition, the touch panel 1071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may further include other input devices 1072. Optionally, the other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, power keys, etc.), a trackball, a mouse, a joystick, etc., and the specifics are not limited here.

[0067] Optionally, the touch panel 1071 may overlay the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. The processor 110 then provides a corresponding visual output on the display panel 1061 based on the type of touch event. Although in FIG1 , the touch panel 1071 and the display panel 1061 are shown as two separate components to implement the input and output functions of the mobile terminal, in some embodiments, the touch panel 1071 and the display panel 1061 may be integrated to implement the input and output functions of the mobile terminal, which is not limited to this specific embodiment.

[0068] The interface unit 108 serves as an interface through which at least one external device can be connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and an external device.

[0069] Memory 109 can be used to store software programs and various data. Memory 109 may primarily include a program storage area and a data storage area. Optionally, the program storage area may store an operating system and at least one application required for a function (such as a sound playback function or an image playback function); the data storage area may store data generated based on the use of the mobile phone (such as audio data, a phone book, etc.). Furthermore, memory 109 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0070] Processor 110 is the control center of the mobile terminal, connecting all components of the mobile terminal using various interfaces and circuits. By running or executing software programs and / or modules stored in memory 109 and accessing data stored in memory 109, it executes various functions of the mobile terminal and processes data, thereby providing overall monitoring of the mobile terminal. Processor 110 may include one or more processing units; preferably, processor 110 may integrate an application processor and a modem processor. Optionally, the application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 110.

[0071] The mobile terminal 100 may also include a power supply 111 (such as a battery) for supplying power to various components. Preferably, the power supply 111 may be logically connected to the processor 110 through a power management system, thereby managing functions such as charging, discharging, and power consumption through the power management system.

[0072] Although not shown in FIG. 1 , the mobile terminal 100 may further include a Bluetooth module, etc., which will not be described in detail here.

[0073] To facilitate understanding of the embodiments of the present application, the communication network system on which the processing device of the present application is based is described below using a mobile terminal as an example.

[0074] Please refer to Figure 2, which is a communication network system architecture diagram provided in an embodiment of the present application. The communication network system is an LTE system of universal mobile communication technology. The LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203 and an operator's IP service 204, which are connected in sequence.

[0075] Optionally, UE201 may be the above-mentioned terminal 100, which will not be described in detail here.

[0076] E-UTRAN 202 includes eNodeB 2021 and other eNodeBs 2022 . Optionally, eNodeB 2021 may be connected to other eNodeBs 2022 via a backhaul (eg, an X2 interface). eNodeB 2021 is connected to EPC 203 , and eNodeB 2021 may provide access from UE 201 to EPC 203 .

[0077] EPC 203 may include an MME (Mobility Management Entity) 2031, an HSS (Home Subscriber Server) 2032, other MMEs 2033, an SGW (Serving Gate Way) 2034, a PGW (PDN Gate Way) 2035, and a PCRF (Policy and Charging Rules Function) 2036. Optionally, MME 2031 is a control node that processes signaling between UE 201 and EPC 203, providing bearer and connection management. HSS 2032 provides registers for managing functions such as the Home Location Register (not shown) and stores user-specific information such as service features and data rates. All user data can be sent through SGW2034, PGW2035 can provide IP address allocation and other functions for UE 201, PCRF2036 is the policy and charging control policy decision point for service data flow and IP bearer resources, and it selects and provides available policy and charging control decisions for the policy and charging execution function unit (not shown in the figure).

[0078] The IP service 204 may include the Internet, an intranet, an IMS (IP Multimedia Subsystem), or other IP services.

[0079] Although the above introduction takes the LTE system as an example, those skilled in the art should know that this application is not only applicable to the LTE system, but can also be applied to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, 5G and future new network systems (such as 6G), etc., which are not limited here.

[0080] This application proposes an image processing method that can predict the chrominance information of an image block based on the correlation between two chrominance components. This method can consider the correlation between a luminance component and two chrominance components during the prediction of a video image, thereby improving the accuracy of the prediction results for the chrominance components in the image and / or improving the overall efficiency of video image encoding and decoding.

[0081] Based on the above overall concept, various embodiments of the present application are further proposed.

[0082] To facilitate understanding, the following first explains the professional terms that may be involved in the embodiments of this application.

[0083] (1) Inter-frame prediction

[0084] Predicting image blocks is an essential step in encoding or decoding an image. For example, the encoder predicts an image block to generate a prediction block, constructing a low-energy residual block to reduce transmission bits. The decoder then decodes the image by entropy decoding the residual block and / or combining the residual block with the prediction block predicted by the decoder to generate a decoded image block. The encoder or decoder can predict image blocks using a number of preset prediction modes, including inter-frame prediction and intra-frame prediction.

[0085] (2) Convolutional Cross-Component Model CCCM

[0086] The convolution cross component model prediction mode CCCM uses a filter to predict the chrominance of the current image block from the reconstructed luminance samples. In one embodiment, the filter used by CCCM consists of a 5-tap (or 7-tap) plus-shaped spatial component, a nonlinear term, and a bias term. Optionally, the input of the 5-tap plus-shaped spatial component of the filter includes a center (C) luminance sample, and an upper sample (also called a north sample, N), a lower sample (also called a south sample, S), a left sample (also called a west sample, W), and a right sample (also called an east sample, E) of the center (C) luminance sample.

[0087] The calculation formula used by CCCM to predict chrominance pixels is as follows: predChromaVal = c0*C+c1*N+c2*S+c3*E+c4*W+c5*P+c6*B

[0088] Among them, C represents the luma sample at the corresponding position of the current chroma sample, and N, S, E, and W are the adjacent samples of the current luma sample, as shown in Figure 3.

[0089] The nonlinear term P = (C*C + midVal) >> bitDepth, and the bias term B = midVal. The bias term B represents a scalar offset between the input and output (similar to the offset term in CCLM) and is set to the middle chroma value (for 10-bit video, B = 512).

[0090] In another embodiment, the filter used by CCCM consists of a center (C) luminance sample, a gradient component, a nonlinear term, and a bias term. Optionally, the gradient component includes a vertical gradient component and a horizontal gradient component. For example, the vertical gradient component is Gy and the horizontal gradient component is Gx. As shown in Figure 4, the calculation formulas for Gy and Gx are as follows: Gy = (2N + NW + NE) – (2S + SW + SE) Gx = (2W + NW + SW) – (2E + NE + SE)

[0091] In another embodiment, the filter used by CCCM consists of a center (C) luma sample, center luma sample position information, a nonlinear term, and a bias term. Optionally, the center luma sample position information includes a vertical position and a horizontal position. For example, the vertical position of the center luma sample is Y, and the horizontal position of the center luma sample is X, which are calculated relative to the upper left coordinate of the sample (also called the image block) for which chroma prediction is currently required.

[0092] Alternatively, please refer to Figures 5A and 5B. Figure 5A is a schematic diagram of the image encoding scenario involved in the processing method provided in this application, and Figure 5B is a schematic diagram of the image decoding process. The encoder at the encoding end typically divides the input video image into at least one image block per frame for processing. Each image block can be subtracted from the prediction block obtained by the prediction mode to obtain a residual block. The residual block and the relevant parameters of the prediction mode are then subjected to a series of processing to obtain the encoded bitstream. Subsequently, at the decoding end, after receiving the bitstream, the decoder can parse the bitstream to obtain the prediction mode parameters. Furthermore, the inverse transform unit and inverse quantization unit of the decoder perform inverse transform and inverse quantization on the transform coefficients to obtain the residual block. Optionally, the decoding unit of the decoder parses and decodes the encoded bitstream to obtain the prediction parameters and related auxiliary information. Next, the prediction processing unit of the decoder performs prediction processing using the prediction parameters to determine the prediction block corresponding to the residual block. In this way, the decoder can obtain a reconstructed block by adding the obtained residual block to the corresponding prediction block. Optionally, the loop filtering unit of the decoder performs loop filtering on the reconstructed block to reduce distortion and improve video quality. Thus, the reconstructed blocks that have undergone loop filtering are further combined into decoded images and stored in a decoded image buffer or output as decoded video signals.

[0093] Optionally, the image processing method provided in the embodiments of the present application can be applied to the scenario where chroma prediction and / or luminance prediction is performed on an image block during the above-mentioned video image encoding process (for example, the scenario where intra-frame prediction is performed during the video image encoding process). Optionally, the image processing method provided in the embodiments of the present application can also be used in the scenario where chroma prediction and / or luminance prediction is performed on an image block to be decoded during the video decoding process. For example, the scenario where intra-frame prediction is performed during the video image decoding process.

[0094] First embodiment

[0095] In this embodiment, the image processing method provided herein may be executed by the aforementioned processing device, or a cluster consisting of multiple aforementioned processing devices. The processing device may be a mobile terminal (such as the aforementioned mobile terminal 100) or a server. Here, the image processing method provided herein will be described using the processing device as the executing entity in the first embodiment of the image processing method provided herein.

[0096] As shown in FIG6 , in this embodiment, the image processing method includes the following steps:

[0097] S1: Predict the chrominance information of the image block based on the correlation between the two chrominance components.

[0098] Optionally, in this embodiment, the processing device uses an intra-frame prediction mode to perform chromaticity prediction on the image block to be processed in the image during the process of encoding or decoding the image, and when using the intra-frame prediction mode to perform chromaticity prediction on the current image block to be processed, the image block chromaticity information of the image block is predicted based on the correlation between the two chromaticity components of the image block, the blue chromaticity component Cb and the red chromaticity component Cr.

[0099] Optionally, in this embodiment, the image block chromaticity information of the image block is a chromaticity prediction result of a blue chromaticity component Cb or a chromaticity prediction result of a red chromaticity component Cr. When the image block chromaticity information is a chromaticity prediction result of the blue chromaticity component Cb, the processing device predicts the blue chromaticity component Cb of the image block using the chromaticity prediction information of the red chromaticity component Cr of the image block, and when the image block chromaticity information is a chromaticity prediction result of the red chromaticity component Cr, the processing device predicts the red chromaticity component Cr of the image block using the chromaticity prediction information of the blue chromaticity component Cb of the image block. Optionally, when the processing device predicts the image block chromaticity information of the image block based on the correlation between the blue chromaticity component Cb and the red chromaticity component Cr of the image block, the positions of the blue chromaticity component Cb and the red chromaticity component Cr are co-located or close to each other (e.g., they are adjacent chromaticity blocks).

[0100] Optionally, in this embodiment, the image block may be an image block in an input video image (i.e., a video frame) that is being encoded or decoded and thus requires chrominance prediction. Optionally, the image block may also be referred to as an image sample, which may be referred to as a current block, a current sample, or a block to be processed. Under the H.265 / High Efficiency Video Coding (HEVC) standard, the sample to be predicted may be a Coding Tree Unit (CTU) or a Coding Unit (CU) in the input video image. The embodiment of the present application does not impose any specific restrictions on the type of image block.

[0101] Optionally, in this embodiment, after receiving a video image from a video source, the processing device, acting as an encoder, segments the video image to obtain at least one image block. The processing device then performs prediction processing on each image block using temporal and / or spatial correlations between the video images. When the processing device uses an intra-frame prediction mode (particularly a cross-component intra-frame prediction mode) to perform chrominance prediction on the current image block, it can determine that the image block is the current image block to be processed. In one embodiment, the processing device can predict the image block chrominance information of the image block based on the correlation between the blue chrominance component Cb and the red chrominance component Cr of the image block.

[0102] Optionally, the processing device, as an encoder, can use, for example, a rate-distortion optimization method to determine the intra-frame prediction mode ultimately adopted by the current image block. For example, the processing device can calculate the rate-distortion cost corresponding to each prediction mode to determine the minimum rate-distortion cost from the rate-distortion costs corresponding to multiple prediction modes, and the prediction mode corresponding to the minimum rate-distortion cost is the prediction mode ultimately adopted by the current image block. That is, assuming that for the prediction processing of the image block currently to be subjected to chroma prediction, the prediction modes that can be used are chroma intra-frame prediction modes 0-N (including cross-component intra-frame prediction modes), and when the processing device calculates that the prediction mode corresponding to the minimum rate-distortion cost for chroma prediction is mode i, the processing device determines mode i as the intra-frame prediction mode ultimately used for chroma prediction of the image block currently to be processed. Optionally, i=0,...,N.

[0103] Optionally, after the processing device determines or obtains the image block chrominance information of the image block in a sample-by-sample manner according to the above process, the processing device as an encoder can further subtract the sample value of the pixel in the current image block from the predicted value of the corresponding pixel in the predicted image block (i.e., the image block chrominance information of the image block) to obtain the residual value of the pixel and the residual block corresponding to the image block. Afterwards, the residual block will be transformed and quantized, and then encoded by the entropy encoder to finally form an encoded bit stream. Optionally, the encoded bit stream may also include prediction parameters corresponding to the intra-frame prediction mode determined by the processing device through the above process (packaged into the encoded bit stream after entropy encoding) and related auxiliary information (side information). Optionally, if the processing device adopts a cross-component intra-frame prediction mode, the above-mentioned prediction parameters at least include indication information about using a cross-component intra-frame prediction model for prediction operations.

[0104] Optionally, the transformed and quantized residual block is added to the corresponding prediction block obtained using the prediction mode to obtain a reconstructed block. After obtaining the reconstructed block, the processing device further performs loop filtering on the reconstructed block to reduce distortion.

[0105] Optionally, when the processing device acts as a decoder, it can receive the encoded bitstream transmitted by the processing device acting as an encoder. After the processing device acts as a decoder and receives the bitstream encoded by the encoder, the decoding unit of the decoder parses and decodes the bitstream to obtain prediction parameters. Optionally, the inverse transform unit and inverse quantization unit of the decoder perform inverse transform and inverse quantization on the transform coefficients to obtain a residual block. Next, the prediction processing unit of the decoder can treat the residual block as the block to be processed and perform prediction processing using the prediction parameters to determine the prediction block corresponding to the residual block.

[0106] Optionally, when the decoder uses the same intra-frame prediction mode as that used by the encoder, it can use the prediction parameters obtained by parsing the bitstream to obtain or determine the prediction mode required for chrominance prediction of the image block to be processed (for example, when the prediction parameters indicate that the corresponding prediction mode is the cross-component intra-frame prediction mode, the decoder uses the cross-component intra-frame prediction mode as the intra-frame prediction mode for chrominance prediction of the decoded residual block). Thus, the decoder directly uses the intra-frame prediction mode to predict the image block chrominance information of the image block to be processed based on the correlation between the blue chrominance component Cb and the red chrominance component Cr of the image block to be processed.

[0107] Optionally, after determining or deriving the chrominance prediction result (the image block chrominance information of the image block) for the image block in a sample-by-sample manner, the processing device acting as a decoder may further add the residual block obtained through analysis to the predicted values ​​of the corresponding pixels in the predicted image block (the image block chrominance information) to obtain a reconstructed block. Finally, the processing device performs loop filtering on the reconstructed block via a loop filtering unit to reduce distortion and improve video quality. The loop-filtered reconstructed blocks are further combined into a decoded image, which is stored in a decoded image buffer or output as a decoded video signal.

[0108] Optionally, in this embodiment, the prediction mode used by the processing device as an encoder or decoder to predict the current image block (e.g., chroma block) may be a convolutional cross-component intra prediction model CCCM as shown in the following formula (1): predChromaVal=c0*C+c1*N+c2*S+c3*E+c4*W+c5*P+c6*B Formula (1)

[0109] Alternatively, taking chroma prediction as an example, predChromaVal is the chroma prediction value of the image block to be processed (in this case, the chroma sample to be predicted), c0~c6 are weight coefficients, C is the luminance value of the co-located luminance sample of the chroma sample to be predicted, N is the luminance value of the luminance sample above / north of the co-located luminance sample, S is the luminance value of the luminance sample below / south of the co-located luminance sample, E is the luminance value of the luminance sample to the right / east of the co-located luminance sample, W is the luminance value of the luminance sample to the left / west of the co-located luminance sample, and P is a nonlinear term. B is a bias term, which represents a scalar offset between the input and output (for 10-bit depth video, B is set to the chroma median, i.e. 512), P = (C*C+midVal)>>bitDepth (midVal is the chroma median of the chroma sample, bitDepth is the bit depth of the video content). The positional relationship between N, S, E, W and C is shown in Figure 3.

[0110] Optionally, in order to use the CCCM model shown in the above formula (1) to perform chroma intra-frame prediction processing, the processing device needs to determine the weight coefficients c0~c6 in formula (1) and the luminance values ​​of C, N, S, E, W, P, and B, and then obtain the chroma prediction result of the chroma sample to be predicted predChromaVal based on formula (1). Optionally, the processing device can determine the weight coefficient in formula (1) by obtaining or determining at least one reference area from the image frame where the image block currently needs to perform chroma prediction is located, and determining the weight coefficient based on the sampling value of the luminance / chroma sample in the at least one reference area. In one embodiment, when performing chroma prediction, at least one luminance reference area is obtained or determined. In another embodiment, when performing chroma prediction, at least one chroma reference area is obtained or determined.

[0111] In this embodiment, the technical solution of the present application predicts the chrominance information of an image block in an image being encoded or decoded based on the correlation between two chrominance components of the image block through a processing device. In this way, the technical solution of the present application can consider the correlation between a luminance component and two chrominance components of the image in the process of chrominance prediction for the video image, thereby improving the accuracy of the results of chrominance prediction for the chrominance components in the image, and / or improving the overall efficiency of video image encoding and decoding.

[0112] Second embodiment

[0113] Based on any of the above embodiments, step S1 may include at least one of the following:

[0114] Method 1: predicting the second chrominance information of the image block based on the first chrominance prediction information of the image block;

[0115] Optionally, in this embodiment, when the processing device predicts the image block chromaticity information of the image block currently to be processed in the image based on the correlation between the blue chromaticity component Cb and the red chromaticity component Cr in the image block during the process of encoding or decoding the image, the processing device can predict the second chromaticity information of the image block based on the first chromaticity prediction information obtained by pre-predicting the first chromaticity information of the image block.

[0116] Optionally, in this embodiment, the first chromaticity information and the second chromaticity information of the image block to be processed are the two chromaticity components of the image block. In one embodiment, if the first chromaticity information of the image block is the blue chromaticity component Cb of the image block, then the second chromaticity information of the image block is the red chromaticity component Cr of the image block.

[0117] Optionally, in this embodiment, the processing device may predict the first chrominance information of the image block to be currently processed based on the luminance information of the image block, thereby determining or deriving the first chrominance prediction information of the image block.

[0118] Method 2: predicting the second chrominance information of the image block based on the first chrominance prediction information of the image block and the first chrominance prediction model;

[0119] Optionally, in this embodiment, when the processing device predicts the image block chromaticity information of the image block currently to be processed in the image based on the correlation between the blue chromaticity component Cb and the red chromaticity component Cr in the image block, the processing device may also first obtain a first chromaticity prediction model used when predicting the first chromaticity information of the image block, and then predict the second chromaticity information of the image block based on the first chromaticity prediction information of the image block and the first chromaticity prediction model.

[0120] Optionally, in this embodiment, the first chroma prediction model may be an intra-frame chroma prediction model used by the processing device when predicting the first chroma information of the image block currently to be processed to obtain the first chroma prediction information of the image block. In this way, the processing device can also use the same intra-frame chroma prediction model as the first chroma prediction model when predicting the second chroma information of the image block. That is, the processing device uses the same intra-frame chroma prediction model when predicting the first chroma information of the image block and when predicting the second chroma information of the image block.

[0121] Method three: predicting the second chrominance information of the image block according to the first chrominance prediction information of the image block and the second chrominance prediction model.

[0122] Optionally, in this embodiment, when the processing device predicts the image block chromaticity information of the image block currently to be processed in the image based on the correlation between the blue chromaticity component Cb and the red chromaticity component Cr in the image block, the processing device may also predict the second chromaticity information of the image block based on the first chromaticity prediction information and the second chromaticity prediction model of the image block.

[0123] Optionally, in this embodiment, the second chroma prediction model is any intra-frame chroma prediction model that is different from the first chroma prediction model. In this way, the processing device uses an intra-frame chroma prediction model that is different from the first chroma prediction model when predicting the second chroma information of the image block. That is, the processing device uses different intra-frame chroma prediction models when predicting the first chroma information of the image block and when predicting the second chroma information of the image block.

[0124] Optionally, when the processing device predicts the second chrominance information of the image block to be processed based on the first chrominance prediction information of the image block currently to be processed, it can adopt the prediction framework shown in Figure 7. As shown in Figure 7, the processing device determines the image block for which chrominance information is currently required to be predicted as the current block, thereby inputting the reconstructed block Rec(Y) of the luminance component Y(Luma) of the current block, the neighbor information of the reconstructed block Rec(Y), and the neighbor information of the current block into the "intra-frame prediction 1" module, thereby predicting the first chrominance information Cb of the current block based on the reconstructed block Rec(Y), the neighbor information of the reconstructed block Rec(Y), and the neighbor information of the current block through the "intra-frame prediction 1" to obtain the first chrominance prediction information Pred(Cb). Then, the processing device obtains the residual Res(Cb) based on the first chrominance prediction information Pred(Cb), and the residual Res(Cb) is transformed and quantized (the quantization stage will produce signal loss) to obtain the transformed state Res'(Cb). The transformed state Res'(Cb) is then inversely transformed and inversely quantized to obtain the spatial domain Res'(Cb). The spatial domain Res'(Cb) and the first chrominance prediction information Pred(Cb) are added together to obtain the first chrominance reconstruction information - the reconstructed block Rec(Cb). Finally, the processing device inputs the reconstructed block Rec(Y), the neighbor information of the reconstructed block Rec(Y), the reconstructed block Rec(Cb), the neighbor information of the reconstructed block Rec(Cb), and the neighbor information of the current block into the "Intra-frame Prediction 2" module to predict the second chrominance information Cr of the current block. Alternatively, similar to the process for processing the first chrominance information Cb, the processing device can also obtain spatial domain Res'(Cr) and second chrominance reconstruction information—reconstructed block Rec(Cr). Thus, at the encoding end, the processing device can generate the final encoded bitstream using the frequency domain Rec'(Y), Res'(Cb), and Res'(Cr) through entropy coding.

[0125] Optionally, in this embodiment, before the processing device predicts the first chrominance information Cb and the second chrominance information Cr of the current block, the luminance information Y (Luma) of the current block has already been predicted and reconstructed (the reconstructed block Rec(Y) is generated from the original pixel block Org(Y)). Optionally, Rec = Pred + Res', and Res = Org - Pred.

[0126] Optionally, in this embodiment, the "intra-frame prediction 1" module in the processing device can be any intra-frame prediction mode such as the direct current mode (DC mode), angle prediction mode, cross-component linear mode CCLM, convolution cross-component model CCCM, generalized linear model GLM, matrix weighted intra-frame prediction mode MIP, etc., to perform chroma intra-frame prediction on the first chroma information Cb of the current block.

[0127] Optionally, in this embodiment, the “intra-frame prediction 2” module in the processing device may select the following scheme:

[0128] Solution 1: Taking the reconstructed block Rec(Y), the neighbor information of the reconstructed block Rec(Y), and the neighbor information of the current block as input, the second chrominance information Cr of the current block is predicted using, for example, the CCCM algorithm to obtain the final prediction result Cr1.

[0129] Solution 2: Taking the reconstructed block Rec(Cb), the neighbor information of the reconstructed block Rec(Cb), and the neighbor information of the current block as input, the second chrominance information Cr of the current block is predicted using, for example, the CCCM algorithm to obtain the final prediction result Cr2.

[0130] Solution 3: After executing the two predictions of Solution 1 and Solution 2 above to obtain the prediction results Cr1 and Cr2, Cr1 and Cr2 are weighted according to the following formula to assign weights W1 and W2 to Cr1 and Cr2 respectively, thereby calculating the final prediction result for the chrominance information of the current block:

[0131] Optionally, in this embodiment, the above-mentioned neighbor information may include: the upper, upper right, upper left, left, and lower left pixel areas of the reconstructed block Rec(Y) / Rec(Cb) / Rec(Cr) / current block. If the neighbor information does not exist, pixel filling is performed on it.

[0132] Third embodiment

[0133] Based on any of the foregoing embodiments, predicting the second chrominance information of the image block according to the first chrominance prediction information of the image block may include at least one of the following:

[0134] Method (1): obtaining or determining first chroma reconstruction information according to the first chroma prediction information, and predicting the second chroma information according to the first chroma reconstruction information;

[0135] Optionally, in this embodiment, when the processing device predicts the second chromaticity information of the image block based on the correlation between the two chromaticity components of the image block and according to the first chromaticity prediction information of the image block currently to be processed, it can first obtain or determine the first chromaticity reconstruction information based on the first chromaticity prediction information, and then predict the second chromaticity information of the image block based on the first chromaticity reconstruction information.

[0136] Optionally, the acquiring or determining the first chroma reconstruction information according to the first chroma prediction information may include:

[0137] The first chroma reconstruction information is determined or derived according to the first chroma prediction information and the residual information.

[0138] Optionally, in this embodiment, after the processing device predicts the first chromaticity information of the image block based on the brightness information of the image block to be currently processed, thereby obtaining the first chromaticity prediction information of the image block, the processing device can obtain or determine the residual information of the first chromaticity information of the image block, and thus determine or obtain the first chromaticity reconstruction information of the first chromaticity information of the image block based on the first chromaticity prediction information and the residual information.

[0139] Optionally, in this embodiment, when the processing device uses the first chroma reconstruction information Rec(Cb) of the first chroma information Cb of the image block to predict the second chroma information Cr of the image block, the residual of the two chroma components of the first chroma information Cb and the second chroma information Cr will not be jointly encoded (i.e., the chroma joint encoding mode JointCbCr is disabled). Optionally, if the processing device uses the first chroma prediction information Pred(Cb) of the first chroma information Cb to predict the second chroma information Cr, the residual of the two chroma components of the first chroma information Cb and the second chroma information Cr can be jointly encoded (i.e., the chroma joint encoding mode JointCbCr is enabled).

[0140] Optionally, the method for obtaining the residual information includes at least one of the following:

[0141] Method a: obtaining or determining residual information according to the first chroma prediction information;

[0142] Optionally, in this embodiment, after obtaining the first chrominance prediction information of the image block to be processed, the processing device at the encoding end can obtain the residual information of the first chrominance information of the image block by subjecting the first chrominance prediction information to a subtractor, transformation, quantization, inverse quantization, and inverse transformation.

[0143] Method b: Obtain or determine residual information from the image coding stream.

[0144] Optionally, in this embodiment, after the processing device at the encoding end obtains the residual information and other information of the first chrominance information of the current image block, it can encode the residual information and other information together into the image coding stream during the entropy coding process and transmit it to the decoding end. In this way, the processing device at the decoding end can directly obtain or determine the residual information of the first chrominance information of the current image block by parsing the image coding stream transmitted from the encoding end.

[0145] Optionally, as shown in FIG8 , in the process of predicting chrominance information for any image block, the processing device can first find its collocated pixel in the reconstructed block Rec(Y) of the luminance component Luma of the image block for the pixel point X1 of the current image block to be processed. As shown in FIG8 , the co-located pixel in Rec(Y) is C, and the surrounding pixels of the co-located pixel C are 8 pixels such as Wn, N, En, ... Then, the processing device uses the formula X1=f(C, N, S, W, E, Wn, En, Ws, Es) to represent the mapping of the co-located block to the first chrominance information Cb of the image block, such as: X1=f(C, N, S, W, E, Wn, En, Ws, Es) n ,E n ,W s ,E s ) = c0C+c1N+c2S+c3W+c4E+c5W n +c6E n +c7W s +c8E s +c9.

[0146] Afterwards, the processing device uses the reference areas TL, T (top), L (left), TB, and TR of the co-located pixels of the image block luminance component Luma, as well as the co-located pixels / blocks of the reference areas in the current image block to be processed as parameter instances to obtain several solutions to the above formula. For example: X1 = 234, C = 278, N = 290, S = 234, W = 236, E = 267, Wn = 239, En = 280, Ws = 230, Es = 275 are a set of values ​​for the above formula, so that the processing device can solve the values ​​of c0-c9 in the formula. Optionally, when more than ten solutions to the above formula are obtained, the processing device can obtain the optimal solution for c0-c9 by minimizing the error.

[0147] Based on this, the processing device finds the same-position pixel and eight neighboring pixels of any pixel X1 in the image block's luminance component Luma for any image block whose chrominance information needs to be predicted, and substitutes each pixel into the above formula to obtain the prediction of pixel Cb. In this way, the processing device can ultimately obtain the first chrominance prediction information Pred(Cb) of the image block's first chrominance information Cb. Optionally, when a pixel in the image block's luminance component Luma (such as any one of the eight neighboring pixels) does not exist, the processing device can fill it in using the pixel's neighbor information or a constant pixel value.

[0148] Finally, the processing device obtains the residual information Res'(Cb) by passing the obtained first chrominance prediction information Pred(Cb) through a subtractor, transformation, quantization, inverse quantization, and inverse transformation, and obtains the first chrominance reconstruction information Rec(Cb) by passing the residual information Res'(Cb) through an adder. Thus, the processing device can use a method similar to the one used to predict the first chrominance information Cb using the luminance reconstruction information Rec(Y) to predict the second chrominance information Cr of the current image block using the first chrominance reconstruction information Rec(Cb) using the following formula: X2 = f(C,N,S,W,E) = c0C+c1N+c2S+c3W+c4E+c5C 2 +c6N 2 +c7S 2 +c8W 2 +c9E 2 +c 10 .

[0149] Optionally, the processing device uses a formula similar to the above formula X1 to derive the values ​​of C0-C10 in the formula X2 for the second chrominance information Cr to be predicted for the current image block. The second chrominance prediction information Pred(Cr) for the second chrominance information Cr can then be derived using the solved formula X2 and the first chrominance reconstruction information Rec(Cb).

[0150] Optionally, as shown in FIG9 , when predicting chrominance information for any image block, the processing device may first use an angular prediction mode to predict the first chrominance information Cb of the image block currently being processed to obtain first chrominance prediction information Pred(Cb). For example, a new Cb block may be generated from adjacent image blocks of the image block currently being predicted for the first chrominance information Cb. The first chrominance prediction information Pred(Cb) is generated vertically from a reference block above the Cb block.

[0151] Then, the processing device adopts the same process as above to obtain or determine the first chrominance reconstruction information Rec(Cb) of the first chrominance information Cb based on the first chrominance prediction information Pred(Cb).

[0152] Finally, for any pixel X2 of the image block for which the second chrominance information Cr is to be predicted, the processing device can find its co-located pixel C1 in the reconstructed block of the first chrominance reconstruction information Rec(Cb). The processing device then predicts the pixel X2 using the model X2 = f1(C1, N, S, W, E). The constant term in the model X2 = f1(C1, N, S, W, E) is solved using a formula similar to that for X1, thereby obtaining the second chrominance prediction information Pred(Cr).

[0153] Method (2): predicting the second chrominance information based on the first chrominance reconstruction information and the second chrominance information of the adjacent image block;

[0154] Optionally, in this embodiment, when the processing device predicts the second chromaticity information of the image block based on the correlation between the two chromaticity components of the image block and according to the first chromaticity prediction information of the image block to be processed, it can also predict the second chromaticity information of the image block based on the first chromaticity reconstruction information obtained or determined according to the first chromaticity prediction information and the second chromaticity information of the adjacent image blocks of the image block.

[0155] Optionally, when predicting chromaticity information for any image block, the processing device may first use an angle prediction mode to predict the first chromaticity information Cb of the image block to be processed to obtain first chromaticity prediction information Pred(Cb), and then obtain or determine first chromaticity reconstruction information Rec(Cb) of the first chromaticity information Cb based on the first chromaticity prediction information Pred(Cb).

[0156] The processing device can then use the angular prediction mode to predict the second chrominance information Cr of the image block currently being processed to obtain the second chrominance prediction information Pred(Cr). For example, a new Cr block is generated from adjacent image blocks of the image block currently being predicted for the second chrominance information Cr. The second chrominance prediction information Pred(Cr) is generated vertically from a reference block above the Cr block.

[0157] Method (3): obtaining or determining second chroma initial prediction information based on the first chroma prediction information, determining the second chroma initial prediction information as the initial prediction result, and obtaining or determining a prediction of the second chroma information based on the initial prediction result.

[0158] Optionally, in this embodiment, when the processing device predicts the second chromaticity information of the image block based on the correlation between the two chromaticity components of the image block and according to the first chromaticity prediction information of the image block to be processed, it can also first obtain or determine the second chromaticity initial prediction information based on the first chromaticity prediction information, and then determine the second chromaticity initial prediction information as the initial prediction result, so as to obtain or determine the prediction result for predicting the second chromaticity information by performing weighted calculation on the initial prediction result.

[0159] Optionally, in this embodiment, the second chroma initial prediction information as the initial prediction result may be a prediction of pixel X2 in the second chroma prediction information Pred(Cr).

[0160] Optionally, as shown in FIG10 , for any pixel X1 of an image block whose first chrominance information Cb is to be predicted, the processing device can find the co-located pixel C of the pixel X1 and eight surrounding pixels, namely, Wn, N, En, etc., in the reconstructed block Rec(Y) of the luminance component Luma of the image block. The processing device then uses the formula X1 = f(C, N, S, W, E, Wn, En, Ws, Es) to represent the mapping of the co-located block to the first chrominance information Cb, and solves the parameters in the function f to predict the first chrominance information based on the luminance reconstruction information Rec(Y) to obtain the first chrominance prediction information Pred(Cb). The processing device then obtains the first chrominance reconstruction information Rec(Cb) through a reconstruction process involving a subtractor, transformation, quantization, inverse quantization, and inverse transformation.

[0161] Then, for any pixel X2 of the image block to be predicted for the second chrominance information Cr, the processing device can also find its co-located pixel C0 in the reconstructed block of the luminance component Luma, and use the model X2=f0(C0) to predict the pixel X2, for example: X2=f0(C0)=c0C0+c1C0 2 The processing device can solve the formula X2 = f0(C0) = c0C0 + c1C0 similar to the formula X1 = f(C, N, S, W, E, Wn, En, Ws, Es). 2 The constant terms c0 and c1 in are used to obtain pixel X2 in the second chrominance prediction information Pred(Cr) for predicting the second chrominance information Cr based on the luminance component Y. Optionally, the processing device uses the same process to obtain pixel X2 in the second chrominance prediction information Pred(Cr) for predicting the second chrominance information Cr based on the first chrominance information Cb. Optionally, the formula for pixel X2 in this prediction process is X2 = f1(C1, N, S, W, E).

[0162] Finally, the processing device weights the pixel X2 of the second chrominance prediction information Pred(Cr) to obtain the prediction of the pixel X2 as follows:

[0163] Optionally, as shown in FIG11 , the processing device uses an angular prediction mode to predict any image block for which first chroma information Cb is to be predicted, obtaining first chroma prediction information Pred(Cb). For example, a new Cb block is generated from adjacent image blocks of the image block for which first chroma information Cb is to be predicted. The first chroma prediction information Pred(Cb) is vertically generated from a reference block above the Cb block. Based on the first chroma prediction information Pred(Cb), first chroma reconstruction information Rec(Cb) for the first chroma information Cb is then obtained or determined.

[0164] Then, for the pixel point X2 of the image block to be predicted for the second chrominance information Cr, the processing device finds its co-located pixel C0 in the reconstructed block of the luminance component Luma, and predicts the pixel point X2 using the model X2=f0(C0), for example: X2=f0(C0)=c0C0+c1C0 2 The processing device can solve the formula X2 = f0(C0) = c0C0 + c1C0 similar to the formula X1 = f(C, N, S, W, E, Wn, En, Ws, Es). 2 The constant terms c0 and c1 in are used to obtain pixel X2 in the second chrominance prediction information Pred(Cr) for predicting the second chrominance information Cr based on the luminance component Y. Optionally, the processing device uses the same process to obtain pixel X2 in the second chrominance prediction information Pred(Cr) for predicting the second chrominance information Cr based on the first chrominance information Cb. Optionally, the formula for pixel X2 in this prediction process is X2 = f1(C1, N, S, W, E).

[0165] Finally, the processing device weights the pixel X2 of the second chrominance prediction information Pred(Cr) obtained by respectively predicting the second chrominance information Cr from the luminance component Y and the first chrominance information Cb, thereby obtaining the prediction of the pixel X2 as follows:

[0166] Optionally, as shown in FIG12 , the processing device uses an angular prediction mode to predict any image block for which first chrominance information Cb is to be predicted, obtaining first chrominance prediction information Pred(Cb). For example, a new Cb block is generated from adjacent image blocks of the image block for which first chrominance information Cb is to be predicted. The first chrominance prediction information Pred(Cb) is vertically generated from a reference block above the Cb block. Based on the first chrominance prediction information Pred(Cb), first chrominance reconstruction information Rec(Cb) for the first chrominance information Cb is then obtained or determined.

[0167] The processing device then uses the angular prediction mode to predict any image block for which the second chrominance information Cr is to be predicted, obtaining the second chrominance prediction information Pred(Cr). For example, a new Cr block is generated from adjacent image blocks of the image block for which the second chrominance information Cr is to be predicted. The second chrominance prediction information Pred(Cr) is then vertically generated from a reference block above the Cr block. Second chrominance reconstruction information Rec(Cr) for the second chrominance information Cr is then obtained or determined based on the second chrominance prediction information Pred(Cr).

[0168] Based on this, the processing device can find the co-located pixel C1 in the reconstructed block of the first chrominance reconstruction information Rec(Cb) for the pixel X2 of the image block to be predicted for the second chrominance information Cr. The model X2 = f1(C1, N, S, W, E) is then used to predict the pixel X2. The constant term in the model X2 = f1(C1, N, S, W, E) is solved using a formula similar to that for X1 to predict the pixel X2 in the second chrominance prediction information Pred(Cr). Furthermore, the processing device weights the pixel X2 in the second chrominance prediction information Pred(Cr) to obtain a prediction for the pixel X2.

[0169] Fourth embodiment

[0170] Based on any of the above embodiments, the image processing method may further include:

[0171] An initial prediction result is obtained or determined according to brightness information of the image block and / or adjacent image blocks.

[0172] Optionally, in this embodiment, in addition to using the above process to predict the pixel X2 in the second chrominance prediction information Pred(Cr) and thereby using the prediction of the pixel X2 as the initial prediction result, the processing device may also obtain or determine the prediction of the pixel X2 based on the brightness information Y of the current image block to be processed and / or the adjacent image blocks of the image block, so as to weight the prediction of the pixel X2 using it as the initial prediction result, and finally predict the second chrominance prediction information Pred(Cr).

[0173] Optionally, the obtaining or determining of the initial prediction result according to the brightness information of the image block and / or the adjacent image blocks may include at least one of the following:

[0174] Method 1: predict the second chrominance information based on the luminance information, and use the predicted second chrominance initial prediction information as the initial prediction result;

[0175] Optionally, in this embodiment, the processing device may use the luminance information Y of the image block whose chrominance information is currently to be predicted and any feasible intra-frame chrominance prediction mode, such as an angular prediction mode, a cross-component linear model (CCLM), or a convolutional cross-component model (CCCM), to directly predict the second chrominance information Cr of the image block, thereby obtaining the second chrominance initial prediction information of the second chrominance information Cr (i.e., the prediction of pixel X2 in the second chrominance prediction information Pred(Cr)). The processing device then uses the prediction of pixel X2 in the second chrominance prediction information Pred(Cr) as the initial prediction result to perform weighted calculations, thereby obtaining the second chrominance prediction information Pred(Cr) that ultimately predicts the second chrominance information Cr.

[0176] Method 2: predicting the second chrominance information based on adjacent image blocks, and using the predicted second chrominance initial prediction information as the initial prediction result;

[0177] Optionally, in this embodiment, the processing device may further use adjacent image blocks of the image block whose chrominance information is currently to be predicted to predict the second chrominance information Cr of the image block using a corresponding intra-frame chrominance prediction mode, thereby obtaining second chrominance initial prediction information of the second chrominance information Cr (i.e., the prediction of pixel X2 in the second chrominance prediction information Pred(Cr)). The processing device then uses the prediction of pixel X2 in the second chrominance prediction information Pred(Cr) as the initial prediction result to weight the prediction, thereby obtaining the second chrominance prediction information Pred(Cr) that ultimately predicts the second chrominance information Cr.

[0178] Mode 3: predict the second chrominance information according to the second chrominance reconstruction information of the adjacent image block, and use the predicted second chrominance initial prediction information as the initial prediction result.

[0179] Optionally, in this embodiment, the processing device may further use the second chroma reconstruction information Rec(Cr) of the adjacent image block to predict the second chroma information Cr of the image block, thereby obtaining second chroma initial prediction information of the second chroma information Cr (i.e., the prediction of pixel X2 in the second chroma prediction information Pred(Cr)). The processing device then uses the prediction of pixel X2 in the second chroma prediction information Pred(Cr) as the initial prediction result to weight the prediction, thereby obtaining the second chroma prediction information Pred(Cr) that ultimately predicts the second chroma information Cr.

[0180] In this embodiment, the technical solution of the present application obtains or determines an initial prediction result based on the luminance information of an image block and / or its adjacent image blocks, and then weights the initial prediction result to obtain the chrominance information of the image block. This can further improve the accuracy of the chrominance prediction results for the chrominance components in the image, thereby improving the overall efficiency of video image encoding and decoding.

[0181] Fifth embodiment

[0182] Based on any of the above embodiments, the image processing method may further include:

[0183] The first chrominance information of the image block is predicted according to the luminance information of the image block to determine or obtain the first chrominance prediction information.

[0184] Optionally, in this embodiment, during the process of encoding or decoding an image, when performing chrominance prediction for an image block in the image, the processing device may first predict the first chrominance information Cb of the image block based on the luminance information Y of the image block, thereby determining or deriving the first chrominance prediction information Pred(Cb) for the image block. In this manner, the processing device can subsequently perform the aforementioned process of predicting the second chrominance information Cr for the image block using the first chrominance prediction information Pred(Cb).

[0185] Optionally, the processing device may employ a CCCM algorithm to determine or derive a first chrominance prediction Pred(Cb) for the first chrominance information Cb of the current image block to be processed. For example, when predicting the first chrominance information of the current image block, the processing device first determines the luminance value (C) and position (X, Y) of the co-located luminance block of the current image block, as well as the luminance values ​​of four luminance blocks adjacent to the left, above, right, and below the co-located luminance block. The processing device then obtains a reference region within the processed region of the image frame in which the co-located luminance block is located, and determines weight coefficients c0 to c6 based on the reference region. Thus, after determining the luminance value and weight coefficients c0 to c6, the processing device applies a gradient filter (e.g., [1, 0, -1]) centered on the co-located luminance block to obtain gradient components G'y and G'x (as shown in FIG4 : G'y = N–S, G'x = W–E). Finally, the processing device substitutes the luminance value (C), position (X, Y), weight coefficients c0 to c6, and gradient components G'y and G'x of the above luminance block into the following formula for CCCM:

[0186] predChromaVal=c0*C+c1*G'y+c2*G'x+c3*Y+c4*X+c5*P+c6*B can determine or obtain the first chrominance prediction information Pred(Cb) of the current image block based on the above-mentioned at least one luminance block and at least one weight coefficient.

[0187] In this embodiment, the technical solution of the present application first predicts a prediction result of one of the chrominance components of the image block based on the brightness information of the image block, and then considers the correlation between the two chrominance components of the image block, so that in the process of chrominance prediction for the video image, the prediction result of the chrominance component that has been predicted is used to predict the other chrominance component. In this way, the accuracy of chrominance prediction for the chrominance components in the image can be improved, and / or the overall efficiency of video image encoding and decoding can be improved.

[0188] An embodiment of the present application further provides a processing device, including a memory and a processor. The memory stores an image processing program, and when the image processing program is executed by the processor, the steps of the image processing method in any of the above embodiments are implemented.

[0189] An embodiment of the present application further provides a storage medium on which an image processing program is stored. When the image processing program is executed by a processor, the steps of the image processing method in any of the above embodiments are implemented.

[0190] In the embodiments of the processing device and storage medium provided in this application, all technical features of any of the above-mentioned image processing method embodiments may be included. The expanded and explained contents of the specification are basically the same as those of the embodiments of the above-mentioned methods and will not be repeated here.

[0191] An embodiment of the present application further provides a computer program product, which includes computer program code. When the computer program code runs on a computer, the computer executes the methods in the various possible implementation modes described above.

[0192] An embodiment of the present application also provides a chip, including a memory and a processor, wherein the memory is used to store computer programs, and the processor is used to call and run the computer programs from the memory, so that a device equipped with the chip executes the methods in the various possible implementation modes as described above.

[0193] It is understood that the above scenarios are merely examples and do not limit the application scenarios of the technical solutions provided in the embodiments of this application. The technical solutions of this application can also be applied to other scenarios. For example, those skilled in the art will appreciate that with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application will also be applicable to similar technical problems.

[0194] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0195] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0196] The units in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0197] In this application, the same or similar terminology, technical solutions and / or application scenario descriptions are generally only described in detail the first time they appear. When they appear again later, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of this application, for the same or similar terminology, technical solutions and / or application scenario descriptions that are not described in detail later, you can refer to the previous relevant detailed descriptions.

[0198] In this application, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0199] The various technical features of the technical solution of this application can be combined arbitrarily. In order to make the description concise, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0200] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as mentioned above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, controlled terminal, or network device, etc.) to execute the method of each embodiment of the present application.

[0201] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a storage disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state storage disk Solid State Disk (SSD)).

[0202] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image processing method, characterized in that: Includes steps: S1: Predict the chrominance information of the image block based on the correlation between the two chrominance components.

2. The method according to claim 1, characterized in that The step S1 includes at least one of the following: Predicting second chrominance information of the image block according to the first chrominance prediction information of the image block; Predicting second chrominance information of the image block according to the first chrominance prediction information of the image block and the first chrominance prediction model; The second chrominance information of the image block is predicted according to the first chrominance prediction information and the second chrominance prediction model of the image block.

3. The method according to claim 2, characterized in that The predicting the second chrominance information of the image block according to the first chrominance prediction information of the image block comprises at least one of the following: Acquire or determine first chroma reconstruction information according to the first chroma prediction information, and predict second chroma information according to the first chroma reconstruction information; Predicting the second chromaticity information according to the first chromaticity reconstruction information and the second chromaticity information of the adjacent image block; The second chromaticity initial prediction information is obtained or determined according to the first chromaticity prediction information, the second chromaticity initial prediction information is determined as an initial prediction result, and a prediction of the second chromaticity information is obtained or determined according to the initial prediction result.

4. The method according to claim 3, characterized in that The obtaining or determining the first chroma reconstruction information according to the first chroma prediction information includes: The first chroma reconstruction information is determined or derived according to the first chroma prediction information and the residual information.

5. The method according to claim 4, characterized in that The method for obtaining the residual information includes at least one of the following: Acquire or determine residual information according to the first chrominance prediction information; Obtain or determine residual information from a coded image stream.

6. The method according to claim 3, characterized in that Also includes: An initial prediction result is obtained or determined according to brightness information of the image block and / or adjacent image blocks.

7. The method according to claim 6, characterized in that The obtaining or determining the initial prediction result according to the brightness information of the image block and / or the adjacent image blocks includes at least one of the following: Predicting the second chromaticity information according to the brightness information, and using the predicted second chromaticity initial prediction information as the initial prediction result; Predicting the second chromaticity information according to the adjacent image blocks, and using the predicted second chromaticity initial prediction information as the initial prediction result; The second chromaticity information is predicted according to the second chromaticity reconstruction information of the adjacent image block, and the second chromaticity initial prediction information obtained by the prediction is used as the initial prediction result.

8. The method according to any one of claims 2 to 7, characterized in that Also includes: The first chrominance information of the image block is predicted according to the brightness information of the image block to determine or obtain the first chrominance prediction information.

9. A processing device, characterized in that: include: A memory and a processor, wherein an image processing program is stored in the memory, and when the image processing program is executed by the processor, the steps of the image processing method according to any one of claims 1 to 8 are implemented.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 8 are implemented.

Images