Image processing method, image processing device, and storage medium

Abstract

This disclosure provides an image processing method, an image processing device, and a storage medium. The image processing method includes determining or generating a prediction result for a pixel to be predicted according to a first luminance value of at least one undownsampled luminance pixel. In this disclosure, the prediction of image chrominance information is performed according to the luminance value of an undownsampled luminance pixel that has not undergone downsampling, thereby reducing the distortion level of the predicted chrominance value and / or improving the prediction quality of chrominance pixels in an image block.

Description

Technical Field

[0001] Cross - reference to Related Applications This application claims priority to Chinese Patent Application No. 202310730087.7, titled "IMAGE PROCESSING METHOD, IMAGE PROCESSING DEVICE, AND STORAGE MEDIUM", filed with the China National Intellectual Property Administration on June 20, 2023, the entire content of which is incorporated herein by reference.

[0002] This application relates to the field of image processing technology, and in particular, to an image processing method, an image processing device, and a storage medium.

Background Art

[0003] In some current implementations, under the YUV (where Y represents luminance / gray value, U represents chrominance, and V represents saturation) color encoding mode, before performing intra - prediction on a video image having an image format of 4:2:0 or 4:2:2, downsampling processing of the luminance pixels within an image block is typically required.

[0004] During the conception and implementation of this application, the inventors found at least the following problems: Downsampling luminance pixels results in some loss of luminance information, leading to distortion of the predicted chrominance values of chrominance pixels and / or poor prediction quality of chrominance pixels within an image block.

[0005] The foregoing description is intended to provide general background information and is not necessarily prior art.

Summary of the Invention

[0006] In response to the above - mentioned technical problems, this application provides an image processing method, an image processing device, and a storage medium that can reduce the distortion level of prediction results obtained through prediction and / or improve the prediction quality of pixels within an image block.

[0007] This application provides an image processing method applicable to processing devices (e.g., smart terminals or servers), and this method is:

[0008] Step S10: Includes determining or generating a prediction result for a pixel that will be predicted, according to at least one first luminance value of at least one luminance pixel.

[0009] Optionally, step S10 is to determine or generate a prediction result for a pixel that will be predicted according to at least one first luminance value of at least one luminance pixel that has not been downsampled. Alternatively, step S10 is to determine or generate a prediction result for a pixel that will be predicted according to at least one first luminance value of at least one downsampled luminance pixel.

[0010] Optionally, step S100 includes at least one of the following:

[0011] First method: Determining or generating a prediction result according to at least one first luminance value, at least one luminance offset, and at least one chrominance offset.

[0012] Second method: Determining or generating prediction results according to at least one first luminance value and at least one gradient value of a luminance pixel.

[0013] Third method: Determining or generating an intermediate prediction result for a pixel to be predicted according to at least one first luminance value, and determining or generating a prediction result according to the intermediate prediction result.

[0014] A fourth method: Determining or generating a prediction result according to at least one first luminance value and at least one position information of an undownsampled luminance pixel, or

[0015] Fifth method: Determining or generating a prediction result according to at least one first luminance value and at least one second luminance value of a downsampled luminance pixel.

[0016] Optionally, step S101 includes at least one of the following:

[0017] First method: Determining or generating a prediction result according to at least one first luminance value, at least one luminance offset, and at least one chrominance offset.

[0018] Second method: Determining or generating a prediction result according to at least one first luminance value and at least one downsampled luminance pixel gradient value.

[0019] Third method: Determining or generating an intermediate prediction result for a pixel to be predicted according to at least one first luminance value, and determining or generating a prediction result according to the intermediate prediction result.

[0020] A fourth method: Determining or generating a prediction result according to at least one first luminance value and at least one downsampled luminance pixel location information, or

[0021] Fifth method: Determining or generating a prediction result according to at least one first luminance value and at least one second luminance value of another downsampled luminance pixel.

[0022] Optionally, the interim forecast results include the first interim forecast results and / or the second interim forecast results, and / or determine or generate the forecast results according to the interim forecast results. This includes performing a weighting process on a first intermediate prediction result and a second intermediate prediction result in order to determine or generate a prediction result.

[0023] Optionally, in this method, at least one of the following: The first intermediate prediction result is determined or generated according to at least one first luminance value, The second intermediate prediction result is determined or generated according to the gradient value of at least one luminance pixel, or The weight for the weighting process is determined or generated according to the strain levels corresponding to the first intermediate prediction result and the second intermediate prediction result.

[0024] Optionally, the gradient value of at least one luminance pixel is the gradient value of at least one non-downsampled luminance pixel, or the gradient value of at least one luminance pixel is the gradient value of at least one downsampled luminance pixel.

[0025] Optionally, this method further includes at least one of the following: The position information is determined or generated according to at least one position scaling information, or The downsampled luminance pixels are obtained or generated by downsampling the luminance pixels.

[0026] Optionally, the downsampled luminance pixels are obtained or generated by downsampling non-downsampled luminance pixels. Alternatively, the downsampled luminance pixels are second downsampled luminance pixels obtained or generated by downsampling downsampled luminance pixels and / or another downsampled luminance pixel.

[0027] Optionally, before step S10, this method further includes obtaining or determining a first pixel flag according to the size of the luminance block where the luminance pixel is located.

[0028] When the first pixel flag is the first value, step S10 includes directly determining or generating a prediction result according to the luminance pixel.

[0029] Alternatively, if the first pixel flag is a second value, step S10 includes performing a downsampling process on the luminance pixel to obtain a downsampled luminance pixel, and determining or generating a prediction result according to the second luminance value of the downsampled luminance pixel.

[0030] Optionally, the prediction mode used to carry out step S10 is determined according to the values ​​of the second pixel flag and the gradient flag.

[0031] Optionally, in this method, at least one of the following:

[0032] At least one luminance pixel that has not been downsampled is a luminance pixel that was not downsampled during the image coding or decoding process. The pixels that will be predicted are chrominance pixels, or The prediction result is the chrominance prediction result.

[0033] Optionally, a luminance pixel is either an undownsampled luminance pixel or a first downsampled luminance pixel.

[0034] This application further provides a processing device including memory and a processor. The processor stores an image processing program therein. When executed by the processor, the image processing program causes any one of the image processing methods described above to be performed.

[0035] This application further provides a storage medium in which a computer program is stored. When executed by a processor, the computer program causes any one of the image processing methods described above to be performed.

[0036] As described above, the image processing method in this application is applicable to a processing device and includes step S10: determining or generating a prediction result for a pixel to be predicted according to a first luminance value of at least one undownsampled luminance pixel. Through the technical solution described above, this application can reduce the distortion level of the prediction result obtained through prediction and / or improve the prediction quality of the pixel to be predicted in an image block.

[0037] The accompanying drawings herein are incorporated herein to form part of this specification, illustrate embodiments conforming to this application, and are used to illustrate the principles of this application together with this specification. To illustrate the technical solutions more clearly in the embodiments of this application, the accompanying drawings used to illustrate embodiments are briefly described below. Obviously, those skilled in the art can still derive other accompanying drawings from these accompanying drawings without creative effort. [Brief explanation of the drawing]

[0038] [Figure 1] This is a schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of this application. [Figure 2] This is an architectural diagram of a communication network system according to one embodiment of this application. [Figure 3] This is a schematic diagram of application scenarios involving the convolutional cross-component model (CCCM). [Figure 4] This is a schematic diagram illustrating another application scenario involving gradient and location-based convolutional component models (GL-CCCM) for intrachrominance prediction. [Figure 5A] This is a schematic diagram of an image coding scenario involved in an image processing method according to one embodiment of this application. [Figure 5B]This is a schematic diagram of an image decoding scenario involved in an image processing method according to one embodiment of this application. [Figure 6] This is a schematic flowchart of the image processing method according to the first embodiment. [Figure 7A] This is a schematic diagram of the downsampled input luminance sample in the predictive model involved in the image processing method according to the second embodiment. [Figure 7B] This is a schematic diagram of an undownsampled input luminance sample in the predictive model involved in the image processing method according to the second embodiment. [Figure 8] This is a schematic diagram of the location of luminance samples for gradient calculation involved in the image processing method according to the second embodiment. [Figure 9A] This is a schematic diagram of the vertical and horizontal positions of the luminance sample and chrominance sample in 4:2:0 format involved in the image processing method according to the second embodiment. [Figure 9B] This is a schematic diagram of the vertical and horizontal positions of the luminance sample and chrominance sample in 4:2:2 format involved in the image processing method according to the second embodiment. [Figure 10] This is a schematic diagram of the downsampling process using the average value within adjacent regions involved in the image processing method according to the second embodiment. [Figure 11] This is a schematic diagram of the downsampling process using the maximum value within an adjacent region, which is involved in the image processing method according to the second embodiment. [Figure 12] This is a schematic diagram showing how to determine a chrominance sample of a chrominance block using a fusion process involved in the image processing method according to the second embodiment. [Figure 13] This is a schematic diagram showing the dimensions of the luminance block and chrominance block within the image block involved in the image processing method according to the third embodiment. [Figure 14] This is a schematic diagram illustrating the determination of a chrominance sample of a chrominance block using a downsampled luminance block, which is involved in the image processing method according to the third embodiment. [Figure 15]This is a schematic diagram illustrating the determination of a chrominance sample of a chrominance block using an undownsampled luminance block involved in the image processing method according to the third embodiment. [Figure 16] This is a schematic diagram of the application process that determines the combination mode of inter-component intra-prediction during the decoding process involved in the image processing method according to the second embodiment. [Modes for carrying out the invention]

[0039] The implementation, functional features, and advantages of the object of this application will be further described in conjunction with embodiments and with reference to the accompanying drawings. The aforementioned accompanying drawings illustrate obvious embodiments of this application, followed by more detailed descriptions in the subsequent text. These accompanying drawings and textual descriptions are not intended to limit the scope of the concepts of this application in any way, but rather to clarify the concepts of this application for those skilled in the art by referring to specific embodiments.

[0040] Exemplary embodiments are described in detail herein with reference to the examples shown in the accompanying drawings. Where the following description relates to the accompanying drawings, unless otherwise noted, the same numbers in different accompanying drawings represent the same or similar elements. The practices described in the following exemplary embodiments do not represent all practices consistent with this application. On the contrary, they are merely examples of apparatuses and methods consistent with some aspects of this application, as described in detail in the accompanying claims.

[0041] In this specification, the terms “equipment,” “include,” or any other variation thereof are intended to include non-exclusive inclusion; therefore, it should be noted that a process, method, article, or apparatus that includes a set of elements includes not only those elements but also other elements not expressly enumerated, or elements specific to that process, method, article, or apparatus. Without further limitation, an element defined by the expression “includes” does not exclude the presence of additional identical elements in a process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements having the same name in different embodiments of this application may have the same or different meanings, and the specific meaning must be determined based on the interpretation within the particular embodiment, or further in conjunction with the context of the particular embodiment.

[0042] Terms such as "first," "second," and "third" may be used herein to describe various types of information, and it should be understood that such information should not be limited to these terms. These terms are merely used to distinguish information of the same type from one another. For example, without deviating from the scope of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the term "in the case of" may be interpreted herein as "when," "on the occasion of," or "depending on the decision." Furthermore, where herein used, the singular forms "one," "a / an," and "the" are intended to include the plural unless otherwise indicated by the context. It should be further understood that the terms "equip" and "include" indicate the presence of a feature, step, action, component, module, item, type, and / or group without excluding the presence, occurrence, or addition of one or more other features, steps, actions, components, modules, items, types, and / or groups. The terms “or,” “and / or,” and “including at least one of the following,” when used in this application, may be interpreted as encompassing or meaning any one or any combination thereof. For example, “including at least one of the following: A, B, and C” means “any of the following: A; B; C; A and B; A and C; B and C; and A, B, and C.” In another example, “A, B, or C” or “A, B, and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; and A, B, and C.” An exception to the provisions arises only when the combination of components, functions, steps, or operations is in any way essential and mutually exclusive of each other.

[0043] In the flowcharts of the embodiments of this application, each step is shown sequentially according to the arrows, but it should be understood that these steps are not necessarily performed in the order indicated by the arrows. Unless expressly stated herein, there are no strict order restrictions on the execution of these steps, and they may be performed in a different order. and / or, at least a portion of the steps in the figures may include multiple substeps or multiple stages, which may not necessarily be completed at the same moment, may be performed at different moments, may not necessarily be performed sequentially, and may be performed alternately or interchangeably with other steps or at least a portion of a substep or stage of other steps.

[0044] In this context, for example, the terms “if” and “in case of” as used herein may be explained as “during” or “when” or “depending on determining” or “depending on detecting.” Similarly, in this context, the expressions “when determining” or “when detecting (the condition or event being described)” may be explained as “when determining” or “depending on determining” or “when detecting (the condition or event being described)” or “depending on detecting (the condition or event being described).”

[0045] In this specification, step names such as step S10 are used to more clearly and accurately represent the corresponding content and do not constitute a substantial restriction on the order. During a particular implementation, a person skilled in the art may perform other steps before performing step S10, etc., all of which fall within the scope of protection of this application.

[0046] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit this application.

[0047] In the following description, suffixes such as “module,” “component,” or “unit” used to indicate elements are solely for the purpose of facilitating the description of this application and do not have any specific meaning in themselves. Therefore, “module,” “component,” or “unit” may be used synonymously.

[0048] Processing devices can be implemented in various forms. For example, the processing devices described in this application may be servers, or may include intelligent terminals such as mobile phones, tablets, laptops, palmtop computers, personal digital assistants (PDAs), portable media players (PMPs), navigation devices, wearable devices, smart bracelets, and pedometers, as well as fixed terminals such as digital TVs and desktop computers.

[0049] In the following description, a portable terminal will be used as an example for illustrative purposes. Those skilled in the art will understand that, in addition to components specifically designed for portable purposes, the configurations of the application may also be applicable to fixed-type terminals.

[0050] Referring to Figure 1, a schematic diagram of the hardware structure of a mobile terminal implementing various embodiments of this application, the mobile terminal 100 may include components such as a radio frequency (RF) unit 101, a WiFi module 102, an audio output unit 103, an A / V (audio / video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111. Those skilled in the art will understand that the structure of the mobile terminal shown in Figure 1 does not constitute a limitation on mobile terminals. A mobile terminal may include more or fewer components than illustrated, or certain components may be combined, or different component arrangements may be used.

[0051] The components of the mobile device are described in detail below, along with Figure 1.

[0052] The radio frequency unit 101 may be used to receive and transmit signals during information reception and transmission or calling processes, specifically receiving downlink information from a base station and providing the downlink information to the processor 110 for processing, and additionally transmitting uplink data to the base station. Generally, the radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, and the like. Optionally, the radio frequency unit 101 may further communicate with a network and other devices via wireless communication. The aforementioned wireless communications may use any communication standard or protocol, but are not limited to, the global system of mobile communications (GSM), General-Purpose Packet Radio Service (GPRS), Code Division Multiple Access 2000 (CDMA2000), Wideband Code Division Multiple Access (WCDMA), Time Division Synchronous Code Division Multiple Access (TD-SCDMA), Frequency Division Duplex Long-Term Evolution (FDD-LTE), Time Division Duplex Long-Term Evolution (TDD-LTE), 5G, 6G, etc.

[0053] WiFi is a short-range wireless transmission technology, and a mobile terminal can provide users with wireless broadband internet access through a WiFi module 102, enabling them to receive and send emails, browse web pages, access streaming media, and more. Figure 1 shows the WiFi module 102, but it should be understood that the WiFi module 102 does not constitute an essential component of the mobile terminal and may be omitted entirely as needed without altering the essence of this disclosure.

[0054] The audio output unit 103 converts audio data received by the radio frequency unit 101 or the WiFi module 102, or stored in the memory 109, into an audio signal, and can output the audio signal as sound when the mobile terminal 100 is in a mode such as call signal reception mode, call mode, recording mode, voice recognition mode, and broadcast reception mode. And / or, the audio output unit 103 may also provide audio output related to specific functions performed by the mobile terminal 100 (e.g., call signal reception sound and message reception sound). The audio output unit 103 may include a loudspeaker, a buzzer, and the like.

[0055] The A / V input unit 104 is configured to receive audio or video signals. The A / V input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The graphics processing unit 1041 processes still image or video image data acquired by an image acquisition device (e.g., a camera) in video acquisition mode or image acquisition mode. The processed image frames may be displayed on the display unit 106. The image frames processed by the graphics processing unit 1041 may be stored in memory 109 (or another storage medium) or transmitted by the radio frequency unit 101 or the WiFi module 102. The microphone 1042 may receive sound (voice data) via the microphone 1042 in operating modes such as telephone calling mode, recording mode, and voice recognition mode, and / or process such sound into voice data. The processed voice (speech) data may be converted into a format that can be transmitted to a mobile communication base station via the radio frequency unit 101 for output in telephone calling mode. The microphone 1042 may implement various types of noise cancellation (or suppression) algorithms to remove (or suppress) noise or interference generated in the audio signal reception and transmission processes.

[0056] The mobile terminal 100 further includes at least one sensor 105, such as an optical sensor, a motion sensor, and other sensors. Optionally, the optical sensor includes an ambient light sensor and a proximity sensor. Optionally, the ambient light sensor may adjust the brightness of the display panel 1061 according to the brightness of ambient light, and the proximity sensor may turn off the display panel 1061 and / or backlight when the mobile terminal 100 is moved to the ear. As one type of motion sensor, an accelerometer sensor may detect the magnitude of acceleration in various directions (typically three axes), and when stationary, it may detect the magnitude and direction of gravity, and may be used for applications that recognize the orientation of the mobile phone (e.g., screen landscape and portrait switching, related games, and magnetometer orientation calibration), vibration recognition-related functions (e.g., pedometer and tap). Details of other sensors that may also be configured on the mobile phone, such as a fingerprint sensor, pressure sensor, iris sensor, molecular sensor, gyroscope, barometer, hygrometer, thermometer, and infrared sensor, are not repeated herein.

[0057] The display unit 106 is configured to display information entered by the user or information provided for the user. The display unit 106 may include a display panel 1061. The display panel 1061 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or similar.

[0058] The user input unit 107 may be configured to receive input numerical or character information and generate important signal inputs related to user settings and function control of the mobile terminal. Optionally, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also called a touchscreen, may collect user touch actions on or near the touch panel 1071 (e.g., user actions on or near the touch panel 1071 using a finger, stylus, or any suitable object or attachment) and drive corresponding connected devices according to a preset program. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Optionally, the touch detection device detects the user touch position, detects the signal produced by the touch action, and transmits the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts this touch information into contact coordinates, and then transmits these contact coordinates to the processor 110, which receives and executes commands transmitted by the processor 110. In addition, the touch panel 1071 may be implemented by using various types, such as resistive, capacitive, infrared, and surface acoustic wave types. Besides the touch panel 1071, the user input unit 107 may further include other input devices 1072. Optionally, the other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, function keys (e.g., volume control buttons and power buttons), a trackball, a mouse, and a joystick.

[0059] Optionally, the touch panel 1071 may cover the display panel 1061. After detecting a touch operation on or near the touch panel 1071, the touch panel 1071 transmits the touch operation to the processor 110 to determine the type of touch event. The processor 110 then provides a corresponding visual output on the display panel 1061 according to the type of touch event. In Figure 1, the touch panel 1071 and the display panel 1061 are used as two separate parts to perform the input and output functions of the mobile terminal, but in some embodiments, the touch panel 1071 and the display panel 1061 may be integrated to perform the input and output functions of the mobile terminal, and specific details are not limited herein.

[0060] The interface unit 108 functions as an interface through which at least one external device can be connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power (or battery charger) port, a wired or wireless data port, a storage card port, a port configured to connect a device having an identification module, an audio input / output (I / O) port, a video I / O port, an earphone port, and the like. The interface unit 108 may be configured to receive input (e.g., data information and power) from the external device and transmit the received input to one or more elements within the mobile terminal 100, or to transmit data between the mobile terminal 100 and the external device.

[0061] Memory 109 may be configured to store software programs and various types of data. Memory 109 may mainly include a program storage area and a data storage area. Optionally, the program storage area may store an operating system, applications required for at least one function (e.g., sound playback function and image playback function), etc., and the data storage area may store data created according to the use of the mobile phone (e.g., voice data and phonebook), etc. In addition, memory 109 may include high-speed random-access memory and may further include at least one magnetic disk storage device, non-volatile memory such as a flash memory device, or other volatile solid-state storage devices.

[0062] The processor 110 is the control center of the mobile terminal and is connected to various parts of the entire mobile terminal by using various interfaces and lines. By operating or executing software programs and / or modules stored in memory 109, and by retrieving data stored in memory 109, the processor 110 performs various functions and data processing of the mobile terminal, thereby performing overall monitoring on the mobile terminal. The processor 110 may include one or more processing units. Preferably, the processor 110 may integrate an application processor and a modem processor. Optionally, the application processor mainly handles the operating system, user interface, applications, etc., and the modem processor mainly handles wireless communication. It should be understood that the above-mentioned modem processor does not have to be integrated into the processor 110.

[0063] The mobile terminal 100 may further include a power supply 111 (e.g., a battery) for supplying power to its components. The power supply 111 may be logically connected to the processor 110 through a power management system, thereby performing functions such as charging, discharging, and power consumption management by using the power management system.

[0064] Although not shown in Figure 1, the mobile terminal 100 may also include a Bluetooth module, and further details will not be repeated herein.

[0065] To facilitate understanding of the embodiments of this application, the communication network system on which the mobile terminal of this application is based is described below.

[0066] Referring to Figure 2, Figure 2 is an architecture diagram of a communication network system according to one embodiment of the present application. The communication network system is an LTE system for universal mobile communication technology. The LTE system includes user equipment (UE) 201, an evolved UMTS terrestrial radio access network (E-UTRAN) 202, an evolved packet core (EPC) 203, and operator IP services 204, which are continuously connected.

[0067] Optionally, UE201 may be the terminal 100 described above, and further details will not be repeated herein.

[0068] E-UTRAN202 includes eNodeB2021, other eNodeB2022, etc. Optionally, eNodeB2021 may be connected to other eNodeB2022 via backhaul (e.g., X2 interface), and eNodeB2021 may be connected to EPC203. eNodeB2021 may provide access to EPC203 for UE201.

[0069] EPC203 may include a Mobility Management Entity (MME) 2031, a Home Subscriber Server (HSS) 2032, another MME 2033, a Serving Gateway (SGW) 2034, a PDN Gateway (PGW) 2035, a Policy and Billing Rules Function (PCRF) 2036, and others. Optionally, MME2031 may be a control node that handles signaling between UE201 and EPC203, providing bearer and connection management. HSS2032 provides several registers for managing functions such as a home location register (not shown in the diagram) and / or is used to store some user-specific information related to service characteristics, data rate, etc. All user data may be transmitted through SGW2034. PGW2035 may provide IP address assignment and other functions for UE201. PCRF2036 is a policy and billing control decision point for service data flows and IP bearer resources that selects and provides available policy and billing control decisions for policy and billing enforcement function units (not shown in the diagram).

[0070] IP services 204 may include the Internet, intranets, IP multimedia subsystems (IMS), and other IP services.

[0071] While the LTE system is used as an example for the above-mentioned introduction, those skilled in the art will recognize that this application is applicable not only to LTE systems but also to other wireless communication systems such as GSM, CDMA2000, WCDMA, TD-SCDMA, 5G, and future new network systems (e.g., 6G), and no limitations are provided herein.

[0072] This application provides an image processing method capable of determining or generating predicted results for pixels to be predicted according to a first luminance value of at least one undownsampled luminance pixel. Predicting image chrominance information according to the luminance value of an undownsampled luminance pixel that has not undergone downsampling can reduce the distortion level of the predicted chrominance values ​​and / or improve the prediction quality of chrominance pixels within an image block.

[0073] To facilitate understanding, technical terms that may be relevant to the embodiments of this application are first explained below.

[0074] (I) Interframe prediction

[0075] Predicting image blocks is an essential step during the process of encoding or decoding an image. For example, an encoder acquires predicted blocks by predicting image blocks, constructs residual blocks with low energy, and reduces the number of transmitted bits, while a decoder obtains decoded image blocks by entropy decoding the residual blocks and / or by combining the predicted blocks acquired through prediction within the decoder with the residual blocks. Prediction of image blocks by an encoder or decoder can be performed through several preset prediction modes, which include inter-prediction modes and intra-prediction modes.

[0076] (II) Convolutional Component Intercomponent Model (CCCM)

[0077] The convolutional component-interconvolutional model (CCCM) uses filters to predict the chrominance of the current image block from reconstructed luminance samples. In one embodiment, the filters used by the CCCM consist of a 5-tap plus-shaped spatial component, a nonlinear term, and a bias term. Optionally, the input to the 5-tap plus-shaped spatial component of the filter includes a central (C) luminance sample, as specifically shown in Figure 3, as well as upper (C) samples (also known as the north sample N), lower (S) samples (also known as the south sample S), left (also known as the west sample W), and right (also known as the east sample E) of the central (C) luminance sample.

[0078] The nonlinear term P represents the square of the central (C) luminance sample and is scaled to the sample value range of the content: P = (C * C + midVal) >> bit depth. For example, for 10-bit content, P = (C * C + 5¹²) >> 10.

[0079] The bias term B represents the scalar offset between the input and output. Optionally, the bias term B is set to the intermediate chrominance value (512 for 10-bit content).

[0080] The filter output is calculated as a convolution between the filter coefficients ci and the input values, and clipped to a range of valid chrominance samples. predChromaVal=c0*C+c1*N+c2*S+c3*E+c4*W+c5*P+c6*B

[0081] (III) Intraframe chrominance prediction using gradient and position-based convolutional component model (GL-CCCM)

[0082] The gradient and position-based convolutional component model (GL-CCCM) replaces four spatially adjacent samples in a CCCM filter using gradient and position information. Optionally, the GL-CCCM filter used for prediction is as follows: predChrmaVal=c0C+c1Gy+c2Gx+c3Y+c4X+c5P+c6B In the formula, Gy and Gx represent the vertical and horizontal gradients, respectively. When combined with a spatial sample for GL-CCCM as shown in Figure 4, the respective formulas for Gy and Gx are: Gy=(2N+NW+NE)-(2S+SW+SE), and Gx = (2W + NW + SW) - (2E + NE + SE).

[0083] Optionally, the Y and X parameters are the vertical and horizontal positions of the central (C) luminance sample.

[0084] Optionally, the Y and X parameters can be calculated relative to the coordinates of the upper-left corner of the image block.

[0085] Referencing Figures 5A and 5B, Figure 5A is a schematic diagram of an image coding scenario involved in an image processing method, and Figure 5B is a schematic diagram of an image decoding process involved in an image processing method. As shown in Figure 5A, the encoder on the coding side typically divides the input video image frame by frame into at least one image block for processing. Each image block may be subtracted from a predicted block predicted through a prediction mode in order to obtain a residual block. A series of processes are then performed on the residual block and the relevant parameters of the prediction mode to generate an encoded bitstream. Subsequently, as shown in Figure 5B, on the decoding side, the decoder may obtain the prediction mode parameters by receiving the bitstream and analyzing the bitstream. Then, the inverse transform and inverse quantization units of the decoder perform inverse transform and inverse quantization operations on the transform coefficients to obtain a residual block. Optionally, the decoding unit of the decoder analyzes and decodes the encoded bitstream to obtain the prediction parameters and relevant side information. Next, the prediction processing unit of the decoder performs prediction processing using the prediction parameters, thereby determining the predicted block corresponding to the residual block. In this way, the decoder can obtain reconstructed blocks by adding the acquired residual blocks and corresponding predicted blocks. Optionally, the decoder's loop filtering unit performs loop filtering on the reconstructed blocks to reduce distortion and improve video quality. Thus, the loop-filtered reconstructed blocks are further combined into the decoded image, which is stored as the decoded video signal in the decoded image buffer or output.

[0086] Optionally, the image processing method provided in this embodiment of the present application may be applied to the above-described scenarios involving chrominance and / or brightness prediction of image blocks during a video image coding process (e.g., a scenario involving intra-prediction during video image coding). Optionally, the image processing method provided in this embodiment of the present application may also be used for scenarios involving chrominance and / or brightness prediction of image blocks to be decoded during a video decoding process, such as a scenario involving intra-prediction during video image coding.

[0087] First Embodiment

[0088] In this embodiment, the execution entity of the image processing method may be the processing device described above, or a cluster consisting of multiple processing devices described above. The processing device may be a smart terminal (e.g., the mobile terminal 100 described above) or a server. In this specification, the processing device is considered the execution entity in the first embodiment of the image processing method for the purpose of describing the image processing method.

[0089] As shown in Figure 6, in this embodiment, the image processing method includes the following steps.

[0090] Step S10: Determine or generate a prediction result for the pixel to be predicted according to at least one first luminance value of at least one luminance pixel.

[0091] Optionally, step S10 is step S100: determine or generate a prediction result for a pixel that will be predicted according to at least one first luminance value of at least one luminance pixel that has not been downsampled. Alternatively, step S10 is step S101: determine or generate a prediction result for a pixel that will be predicted according to at least one first luminance value of at least one downsampled luminance pixel.

[0092] Optionally, in this embodiment, during the image encoding or decoding process, the processing device uses an intra-prediction mode to perform chrominance prediction for the image block to be processed within the image. When using an intra-prediction mode for chrominance prediction for the current image block to be processed, the processing device uses at least one first luminance value of at least one undownsampled luminance pixel in the image block that has not undergone downsampling to predict the chrominance information of the pixel to be predicted within the image block, thereby determining or generating a prediction result for the pixel to be predicted.

[0093] Optionally, in this embodiment, when using an intra-prediction mode for chrominance prediction for the current image block to be processed, the processing device may also use at least one first luminance value of at least one downsampled luminance pixel in the downsampled image block to predict the chrominance information of the pixel to be predicted in the image block, thereby determining or generating a prediction result for the pixel to be predicted.

[0094] Optionally, in this embodiment and other embodiments described later, at least one undownsampled luminance pixel refers to a luminance pixel that was not downsampled during the image coding or decoding process, the predicted pixel is a chrominance pixel of the image block, and the prediction result is the chrominance prediction result of the chrominance pixel. Note that at least one undownsampled luminance pixel may have been downsampled in an image processing process other than the image coding or decoding process.

[0095] Optionally, in this embodiment, an image block may be an image block in an input video image (i.e., a video frame) that is currently being encoded or decoded and therefore requires chrominance prediction. Optionally, an image block may also be simply referred to as the current block, current unit, or block to be processed. A pixel to be predicted may also be referred to as a sample to be predicted. Under the H.265 / High Efficiency Video Coding (HEVC) standard, a pixel to be predicted may be a pixel or sample in a coding tree unit (CTU) or code unit (CU) in the input video image. Embodiments of this application do not impose any particular limitations on the type of image block referred to.

[0096] Optionally, in this embodiment, after receiving a video image from a video source, the processing device acting as an encoder divides the video image into at least one image block. When using an intra-prediction mode (in particular, an inter-component intra-prediction mode) for chrominance prediction for the current image block, the processing device determines that the image block is the current image block to be processed. Subsequently, the processing device treats the chrominance pixels in the image block having the predicted chrominance information as the pixels to be predicted. Then, the processing device directly uses at least one first luminance value of at least one undownsampled luminance pixel in the image block that has not undergone downsampling to predict the chrominance information of the pixels to be predicted, thereby obtaining a chrominance prediction result for the pixels to be predicted.

[0097] Optionally, a processing device acting as an encoder may use methods such as rate-strain optimization to determine the final intra-prediction mode to be adopted for the current image block. Optionally, the processing device may calculate the rate-strain cost corresponding to each prediction mode and determine the minimum rate-strain cost from the rate-strain costs corresponding to multiple prediction modes. The prediction mode corresponding to the minimum rate-strain cost is the final prediction mode to be adopted for the current image block. That is, assuming that the available prediction modes for the prediction processing of an image block currently undergoing chrominance prediction are chrominance intra-prediction modes 0-N (including inter-component intra-prediction modes), the processing device determines mode i as the final intra-prediction mode to be used for chrominance prediction of the current image block to be processed, when the processing device calculates that the prediction mode corresponding to the minimum rate-strain cost for chrominance prediction is mode i. Optionally, i = 0, ..., and N.

[0098] Optionally, after the processing device has determined or derived the predicted chrominance information for each pixel in the image block according to the process described above, the processing device acting as an encoder may further subtract the predicted value of the corresponding pixel in the image block (i.e., the image block chrominance information of the image block) from the sample value of the pixel in the current image block to obtain the residual value of the pixel and the residual block corresponding to the image block. At this point, the residual block has already undergone transformation and quantization and is then encoded by an entropy encoder to form an encoded bitstream. Optionally, the encoded bitstream may also include prediction parameters (packed into the encoded bitstream after entropy encoding) corresponding to the intra-prediction mode determined by the processing device through the process described above, as well as relevant side information. Optionally, if the processing device uses an inter-component intra-prediction mode, the prediction parameters include at least indication information regarding the use of an inter-component intra-prediction model for prediction operation.

[0099] Optionally, the transformed and quantized residual blocks are added to the corresponding predicted blocks, which are acquired using the prediction mode, in order to acquire the reconstructed blocks. After acquiring the reconstructed blocks, the processing device may also perform loop filtering on the reconstructed blocks to reduce strain.

[0100] Optionally, when acting as a decoder, the processing device may receive an encoded bitstream transmitted by the aforementioned processing device acting as an encoder. After the processing device acting as a decoder receives the bitstream encoded by the encoder, the decoder's decoding unit analyzes and decodes the bitstream to obtain prediction parameters. Optionally, the decoder's inverse transform and inverse quantization units perform inverse transform and inverse quantization operations on the transform coefficients to obtain residual blocks. Next, the decoder's prediction processing unit may treat the residual blocks as blocks to be processed that now need to be decoded, and use prediction parameters for prediction processing to determine the prediction blocks corresponding to the residual blocks.

[0101] Optionally, when the decoder uses the same intra-prediction mode as the encoder, the prediction parameters obtained by analyzing the bitstream may be used to obtain or determine the prediction mode that will be used for predicting the chrominance of the current image block to be processed (for example, when the prediction parameters indicate that the corresponding prediction mode is an inter-component intra-prediction mode, the decoder uses the inter-component intra-prediction mode as the intra-prediction mode for predicting the chrominance of the residual block obtained after decoding). Thus, the decoder directly uses the intra-prediction mode to predict the chrominance information of the pixels to be predicted in the image block using at least one first luminance value of at least one undownsampled luminance pixel in the image block that has not undergone downsampling.

[0102] Optionally, a processing device acting as a decoder may determine or derive chrominance information for an image block by determining or deriving a sample-by-sample chrominance prediction result for at least one pixel that will be predicted to be in the image block. The processing device may then further add the analyzed residual block and the predicted values ​​for the corresponding pixels in the image block (image block chrominance information of the image block) to obtain a reconstructed block. Finally, the processing device may also perform loop filtering on the reconstructed block through a loop filtering unit to reduce distortion and improve video quality. The loop-filtered reconstructed block is further combined into a decoded image, which is stored as a decoded video signal in a decoded image buffer or output.

[0103] Optionally, in this embodiment, the prediction mode used by the processing device acting as an encoder or decoder for predicting the current image block (e.g., a chrominance block) may be an interconvolutional component intra-prediction model (CCCM) as shown in equation (1). predChromaVal=c0*C+c1*N+c2*S+c3*E+c4*W+c5*P+c6*B Formula (1)

[0104] For example, taking chrominance prediction as an example, predChromaVal is the predicted chrominance value of the pixel to be predicted in the image block to be processed (in this case, the predicted chrominance pixel), c0 to c6 are weight coefficients, C is the luminance value of the luminance pixel at the same position as the predicted chrominance pixel, N is the luminance value of the luminance pixel adjacent to the luminance pixel at the same position from the top / north, S is the luminance value of the luminance pixel adjacent to the luminance pixel at the same position from the bottom / south, E is the luminance value of the luminance pixel adjacent to the luminance pixel at the same position from the right / east, W is the luminance value of the luminance pixel adjacent to the luminance pixel at the same position from the left / west, and P is a nonlinear term. B is a bias term representing the scalar offset between the input and output (for video with 10-bit depth, B is set to the median chrominance, i.e., 512). P = (C * C + midVal) >> bitDepth (where midVal is the median chrominance of the chrominance pixel, and bitDepth is the bit depth of the video content). The relative positions of N, S, E, W, and C are shown in Figure 3.

[0105] In practice, there may be multiple methods for determining the weight coefficients. Optionally, the aforementioned weight coefficients may be determined according to the pixels in the adjacent region of the predicted block where the predicted pixels are located. Optionally, the aforementioned weight coefficients may be determined according to the pixels in the luminance prediction block and chrominance prediction block of the predicted block, which are determined through interpretation.

[0106] Optionally, in order to perform chrominance intra-prediction processing using the CCCM shown in equation (1), the processing device may need to determine the weight coefficients c0 to c6 in equation (1) and the luminance values ​​of C, N, S, E, W, P, and B, and then obtain the chrominance prediction result predChromaVal of the chrominance samples to be predicted based on equation (1). Optionally, the method by which the processing device determines the weight coefficients in equation (1) may involve obtaining or determining at least one reference region from the image frame in which the image block currently requiring chrominance prediction is located, and determining the weight coefficients according to the sample values ​​of luminance / chrominance pixels within at least one reference region. In one embodiment, at least one luminance reference region is obtained or determined during chrominance prediction. In another embodiment, at least one chrominance reference region is obtained or determined during chrominance prediction.

[0107] In this embodiment, according to the technical solution of the present application, when a processing device performs inter-component intra-prediction on the chrominance information of an image block, at least one first luminance value of at least one undownsampled luminance pixel in the image block that has not undergone downsampling is used to predict the chrominance information of the pixel to be predicted in the image block. That is, according to the technical solution of the present application, the downsampling operation is not performed on the luminance image block, but the luminance image block is used directly to determine the chrominance value of the chrominance image block, thereby retaining the luminance information lost during the downsampling process, reducing the level of distortion in the prediction results obtained through prediction to improve the accuracy of chrominance prediction, and / or improving the quality of the chrominance prediction of the pixel to be predicted in the image block.

[0108] Optionally, the technical solutions of this application are more suitable for encoding and decoding operations on images or videos, with clear detail provided through a method for directly using a luminance image block to determine the chrominance value of a chrominance image block without performing a downsampling operation on the luminance image block.

[0109] Second Embodiment

[0110] Based on any one of the embodiments described above, during an encoding or decoding process for an image, the processing device treats chrominance pixels in the current image block to be processed that require chrominance information prediction as pixels to be predicted, and then predicts the chrominance information of the pixels to be predicted according to at least one first luminance value of at least one undownsampled luminance pixel in the current image block to be processed that has not undergone downsampling, at least one luminance offset of the image block, at least one chrominance offset of the image block, a gradient value of at least one undownsampled luminance pixel, position information of at least one undownsampled luminance pixel, and / or at least one second luminance value of at least one downsampled luminance pixel that has undergone downsampling, thereby determining or generating a chrominance prediction result for the pixels to be predicted.

[0111] Optionally, step S100 may include at least one of the following:

[0112] First method: Determine or generate a prediction result according to at least one first luminance value, at least one luminance offset, and at least one chrominance offset.

[0113] Optionally, in this embodiment, during an encoding or decoding process for an image, the processing device treats chrominance pixels in the current image block to be processed that require chrominance information prediction as pixels to be predicted, and then predicts the chrominance information of the pixels to be predicted according to at least one first luminance value of at least one undownsampled luminance pixel in the current image block to be processed that has not undergone downsampling, at least one luminance offset of the image block, and at least one chrominance offset of the image block, thereby determining or generating a chrominance prediction result for the pixels to be predicted. For example, at least one luminance offset and at least one chrominance offset may be the preset luminance offset value Lumaoffset and preset chrominance offset value Chromaoffset described below.

[0114] When chrominance prediction is optionally performed on image blocks that will be processed within the image based on inter-component intra-prediction, the processing device may employ a prediction model PreChorma=f(Luma1, Luma2, Luma3, ..., and LumaN) to calculate the chrominance values ​​of chrominance pixels in the image blocks that require chrominance information prediction. PreChorma represents the chrominance pixels to be predicted, and Luma1 to LumaN are input luminance pixels, as well as undownsampled luminance pixels that have not undergone downsampling. Luma1 is the luminance pixel at the same position as the chrominance pixel to be predicted, and Luma2, Luma3, ..., and LumaN are the luminance pixels around the luminance pixel at the same position.

[0115] Optionally, if the processing device uses downsampled luminance pixels to perform inter-component intra prediction on the chrominance information of the predicted chrominance pixels, Luma1 is the downsampled luminance pixel at the same location as the predicted chrominance pixel, while Luma2, Luma3, ..., and LumaN are downsampled luminance pixels around the downsampled luminance pixel at the same location. If the processing device directly uses undownsampled luminance pixels that have not been downsampled to perform inter-component intra prediction on the chrominance information of the predicted chrominance pixels, Luma1 is also the undownsampled luminance pixel at the same location as the predicted chrominance pixel that has not been downsampled, while Luma2, Luma3, ..., and LumaN are undownsampled luminance pixels around the undownsampled luminance pixels at the same location that have not been downsampled.

[0116] Optionally, as shown in Figure 7A, during the downsampled intercomponent intra-prediction process, the processing device first downsamples luminance block A to obtain luminance block A', and then uses the downsampled luminance pixels in luminance block A' as input luminance pixels in the prediction model. Optionally, during the downsampled intercomponent intra-prediction process, the input luminance pixels include Luma1, Luma2, Luma3, ..., and Luma9, where Chroma1 is the chrominance pixel to be predicted. Luma1 is the luminance pixel at the same position as the chrominance pixel to be predicted in luminance block A' (and is downsampled), and Luma2, Luma3, ..., and Luma9 are the luminance pixels around the luminance pixel at the same position in luminance block A' (and are also downsampled).

[0117] Optionally, as shown in Figure 7B, when the image format of the image currently being encoded and decoded by the processing device is 4:2:0, during the non-downsampled inter-component intra-prediction process, the processing device directly uses the luminance pixels within luminance block A as input luminance pixels in the prediction model without downsampling luminance block A. Optionally, during the non-downsampled inter-component intra-prediction process by the processing device, the input luminance pixels include L1, L2, L3, ..., and L9, where C1 is the chrominance pixel to be predicted. L1 is the luminance pixel at the same location as the chrominance pixel to be predicted (not downsampled), and L2, L3, ..., and L9 are the luminance pixels around the luminance pixel at the same location (not downsampled).

[0118] Optionally, the prediction model PreChorma=f(Luma1,Luma2,Luma3,...,LumaN) described above may also include a nonlinear term P. Optionally, the nonlinear term P is expressed as the square of the luminance pixel C corresponding to the predicted chrominance pixel, scaled up to the bit depth range, i.e., P=(Luma1*Luma1+midVal)>>bitDepth, where bitDepth is the bit depth corresponding to the sample, and ">>" is the right shift code. Optionally, for 10-bit video content, P=(Luma1*Luma1+512)>>10.

[0119] Optionally, the above prediction model PreChorma=f(Luma1,Luma2,Luma3,...,LumaN) may also include a bias term B. Optionally, bias term B may represent a scalar offset between the input and output. Optionally, in the video coating standard, for 10-bit video content, bias term B is set to a chrominance value of 512.

[0120] Arbitrarily, the above prediction model PreChorma=f(Luma1,Luma2,Luma3,...,LumaN) can be transformed into predChromaVal=c0*Luma1+c1*Luma2+c2*Luma3+c3*Luma4+c4*Luma5+c5*P+c6*B when combined with a nonlinear term P and a bias term B. Luma2, Luma3, and Luma4 are the luminance pixels to the left, right, above, or below the luminance pixel Luma1 at the same position, respectively, while c0~c6 are weights.

[0121] Optionally, if the processing device sets the luminance offset value and chrominance offset value within the prediction model PreChorma=f(Luma1,Luma2,Luma3,...,LumaN) described above, the prediction model can be transformed into PreChorma'=f(Luma1',Luma2',Luma3',...,LumaN'), where Luma1'=Luma1-Lumaoffset, Luma2'=Luma2-Lumaoffset, and LumaN'=LumaN-Lumaoffset. Lumaoffset is a preset luminance offset value. PreChorma' must be added along with a preset chrominance offset value Chromaoffset to obtain the PreChorma of the chrominance pixels that will be predicted.

[0122] Optionally, based on setting luminance offset values ​​and chrominance offset values ​​within the prediction model PreChorma=f(Luma1,Luma2,Luma3,...,LumaN) described above, the processing device further sets a nonlinear term P', in which case the nonlinear term P'=(Luma1'Luma1'+midVal)>>bitDepth.

[0123] Optionally, the processing device may use the sample value of a luminance pixel at a preset position outside the current image block to be processed (also known as an encoding unit) and adjacent to the image to be processed as the luminance offset value Lumaoffset, and / or use the sample value of a chrominance pixel at the same position as the luminance sample at the preset position as the chrominance offset value Chromaoffset. Optionally, the processing device may use the sample value of a luminance pixel in the upper reference sample column, located outside and adjacent to the current image block to be processed, as the luminance offset value Lumaoffset, or the sample value of a luminance pixel in the upper-left corner, located outside and adjacent to the current image block to be processed, as the luminance offset value Lumaoffset. and / or, the processing device uses the sample value of a chrominance pixel in the upper reference sample column that is outside and adjacent to the current image block to be processed as the chrominance offset value Chromaoffset, or uses the sample value of a chrominance pixel in the left reference sample column that is outside and adjacent to the current image block to be processed as the chrominance offset value Chromaoffset, or uses the sample value of a chrominance pixel in the upper left corner that is outside and adjacent to the current image block to be processed as the chrominance offset value Chromaoffset.

[0124] In this embodiment, during the encoding or decoding process for an image, the processing device predicts the chrominance information of the pixel to be predicted according to at least one first luminance value of at least one undownsampled luminance pixel in the current image block to be processed that has not undergone downsampling, at least one luminance offset of the image block, and at least one chrominance offset of the image block, thereby obtaining a predicted chrominance result for the pixel to be predicted. Accordingly, the magnitudes of the luminance and chrominance values ​​used in the process of determining the weight coefficients during intercomponent prediction can be reduced (for example, the magnitudes of the luminance and chrominance values ​​used in determining the weight coefficients c0 to c6 described above are reduced), thereby reducing the computational load. In other words, the processing device obtains a smaller luminance value by subtracting a luminance offset value from the luminance value, then performs a prediction process on the smaller luminance value to obtain a smaller chrominance value, and finally adds the chrominance offset value to the smaller chrominance value to obtain the final chrominance prediction value, thereby effectively reducing the computational load during the calculation process and / or reducing the memory space required to store intermediate values ​​generated during the calculation process.

[0125] Second method: Determine or generate a prediction result according to at least one first luminance value and at least one gradient value of a luminance pixel.

[0126] In this embodiment, during an encoding or decoding process for an image, the processing device treats chrominance pixels in the current image block to be processed that require chrominance information prediction as pixels to be predicted, and then predicts the chrominance information of the pixels to be predicted according to at least one first luminance value of at least one undownsampled luminance pixel in the current image block to be processed that has not undergone downsampling, and the gradient value of at least one undownsampled luminance pixel (if there are multiple first luminance values, the gradient value of each of the multiple first luminance values ​​is used), thereby determining or generating a chrominance prediction result for the pixels to be predicted.

[0127] In this way, directly performing gradient calculations based on the luminance values ​​of undownsampled luminance pixels, compared to calculating gradient values ​​of downsampled luminance pixels after downsampling luminance samples, can ensure higher accuracy of the obtained gradient values, thereby reducing the level of distortion in the prediction results obtained after prediction and / or improving the prediction quality of pixels in an image block.

[0128] Optionally, when a processing device predicts the chrominance information of pixels to be predicted according to the first luminance value and gradient value of the undownsampled luminance pixels in the current image to be processed that have not undergone downsampling, the above prediction model PreChorma=f(Luma1,Luma2,Luma3,...,LumaN) can be used to calculate the chrominance value of chrominance pixels in an image block that requires chrominance information prediction, in which case the prediction model PreChorma=f(Luma1,Luma2,Luma3,...,LumaN) can be transformed into PreChorma=f(Luma1,Gx,Gy), where Gx and Gy are the horizontal and vertical gradients obtained by applying a gradient filter centered on Luma1, respectively.

[0129] 、 In this embodiment, Luma1, Luma2, Luma3, ..., and LumaN are all luminance pixels (or luminance samples) that were not downsampled during the inter-component intra-prediction process.

[0130] Arbitrarily, Gx = (2Luma5 + Luma2 + Luma7) - (2Luma6 + Luma4 + Luma9) and Gy = (2Luma3 + Luma2 + Luma4) - (2Luma8 + Luma7 + Luma9). As shown in Figure 8, Luma2 is the upper left luminance pixel of Luma1, Luma3 is the luminance pixel above Luma1, Luma4 is the upper right luminance pixel of Luma1, Luma5 is the left luminance pixel of Luma1, Luma6 is the right luminance pixel of Luma1, Luma7 is the lower left luminance pixel of Luma1, Luma8 is the lower luminance pixel of Luma1, and Luma9 is the lower right luminance pixel of Luma1.

[0131] Optionally, the processing device may use a gradient flag to indicate whether to use gradients to calculate the predicted chrominance value of a pixel.

[0132] Optionally, if the gradient flag has a first value (1), the processing device calculates the predicted chrominance value of the pixel using the gradient, and / or if the gradient flag has a second value (0), the processing device calculates the predicted chrominance value of the pixel using the luminance value of the luminance pixel without using the gradient of the luminance value.

[0133] In this embodiment, compared to solutions in which gradient values ​​are not used, by utilizing the gradient values ​​of luminance pixels, the processing device can predict the chrominance information of pixels that will be predicted according to at least one first luminance value of at least one undownsampled luminance pixel in the current image block to be processed that has not undergone downsampling and the gradient value of at least one undownsampled luminance pixel, thereby obtaining chrominance prediction results and enabling the prediction model to incorporate information about pixel edge variations, and thus improving the prediction quality of image blocks when predicting pixel chrominance information for image blocks that include object edges.

[0134] Optionally, during the encoding or decoding process for an image, after treating chrominance pixels in the current image block to be processed that require chrominance information prediction as pixels to be predicted, the processing device may also predict the chrominance information of the pixels to be predicted according to at least one first luminance value of at least one undownsampled luminance pixel in the current image block to be processed that has not undergone downsampling, at least one luminance offset of the image block, at least one chrominance offset of the image block, and the gradient value of at least one undownsampled luminance pixel (for example, if there are multiple first luminance values, the gradient value of each of the multiple first luminance values ​​is used), thereby determining or generating a chrominance prediction result for the pixels to be predicted.

[0135] A third method: Determine or generate an intermediate prediction result for the pixels to be predicted according to at least one first luminance value, and determine or generate a prediction result according to the intermediate prediction result.

[0136] In this embodiment, during an encoding or decoding process for an image, the processing device uses chrominance pixels in the current image block to be processed that require chrominance information prediction as pixels to be predicted. The process for determining the pixels to be predicted includes first predicting the chrominance information of the pixels to be predicted according to at least one first luminance value of at least one undownsampled luminance pixel in the current image block to be processed that has not undergone downsampling, thereby obtaining an intermediate prediction result regarding the chrominance information of the pixels to be predicted, and then performing further computational processing according to the intermediate prediction result to determine or generate a chrominance prediction result for the pixels to be predicted.

[0137] Optionally, the above-described intermediate prediction results regarding the chrominance information of the pixels to be predicted include a first intermediate prediction result and / or a second intermediate prediction result. Optionally, the above-described step of determining or generating a prediction result according to the intermediate prediction results may include:

[0138] To determine or generate the prediction result, a weighting process is performed on the first and second intermediate prediction results.

[0139] Optionally, when predicting the chrominance information of a pixel to be predicted within the current image block to be processed, the processing device may also use different inter-component intra-prediction formulas (also known as inter-component intra-prediction models) to determine multiple chrominance values ​​for the pixel to be predicted, and then perform weighted fusion on the multiple chrominance values ​​to obtain the final chrominance value for the pixel to be predicted.

[0140] Optionally, the first intermediate prediction result described above is determined or generated according to at least one first luminance value, while the second intermediate prediction result is determined or generated according to the gradient value of at least one luminance pixel.

[0141] Optionally, the gradient value of at least one luminance pixel is the gradient value of at least one luminance pixel that has not been downsampled.

[0142] Optionally, the gradient value S of at least one of the undownsampled luminance pixels mentioned above. GThe gradient value S can be determined through a series of gradient filters. These gradient filters may include Sobel, Prewitt, Roberts, and Laplacian operators. In practice, the first gradient value of at least one undownsampled luminance pixel can be determined through a first gradient filter (e.g., one of the various gradient filters), and the second gradient value of at least one undownsampled luminance pixel can be determined through a second gradient filter (e.g., another of the various gradient filters). The gradient value S is then determined. G This is obtained by performing a weighting process on the first and second gradient values. In one embodiment, a suitable first gradient filter and a suitable second gradient filter may be selected from a plurality of gradient filters according to the usage conditions (e.g., past usage conditions) of the plurality of gradient filters during the encoding and decoding process.

[0143] Optionally, the processing device may use the above-described prediction model predChromaVall=c0*Luma1+c1*Luma2+c2*Luma3+c3*Luma4+c4*Luma5+c5*P+c6*B to determine or generate a first intermediate prediction result: chrominance value ChromaA1 for the pixel to be predicted, and may use the above-described prediction model predChroma=c0Luma1+c1Gx+c2Gy+c3P+c4B to determine a second intermediate prediction result: chrominance value ChromaA2 for the pixel to be predicted. Finally, a weighting process is performed on chrominance values ​​ChromaA1 and ChromaA2 to determine the final chrominance value ChromaFinal for the pixel to be predicted.

[0144] Optionally, weights for the weighting process are determined or generated according to the strain levels corresponding to the first and second intermediate prediction results.

[0145] Optionally, the processing device may perform a weighting process on the intermediate prediction results regarding the chrominance information of the pixels to be predicted, according to the formula: ChromaFinal = k1 * ChromaA1 + k2 * ChromaA2, where k1 and k2 are weights.

[0146] Optionally, when the processing device is an encoder, the processing device may calculate the rate-strain cost J1 of a chrominance block consisting of chrominance values ​​ChromaA1 obtained using the prediction model predChromaVal=c0*Luma1+c1*Luma2+c2*Luma3+c3*Luma4+c4*Luma5+c5*P+c6*B, and the rate-strain cost J2 of a chrominance block consisting of chrominance values ​​ChromaA2 obtained using the prediction model predChroma=c0*Luma1+c1*Gx+c2*Gy+c3*P+c4*B. The values ​​of k1 and k2 are then determined according to the ratio of the rate-strain cost J1 to the rate-strain cost J2. Optionally, if J1 / J2=1:1, then k1 / k2=1, or if J1 / J2=1:2, then k1 / k2=2:1.

[0147] In this embodiment, when predicting chrominance information for a pixel to be predicted, the processing device first predicts an intermediate prediction result for the chrominance information of the pixel to be predicted, then performs further weighting calculation processing according to a plurality of intermediate prediction results to determine or generate a final chrominance prediction result for the pixel to be predicted, thereby obtaining a better chrominance prediction result for the pixel to be predicted.

[0148] A fourth method: Determine or generate a prediction result according to at least one first luminance value and position information of at least one luminance pixel that has not been downsampled.

[0149] In this embodiment, during an encoding or decoding process for an image, the processing device treats chrominance pixels in the current image block to be processed that require chrominance information prediction as pixels to be predicted, and then predicts the chrominance information of the pixels to be predicted according to at least one first luminance value of at least one undownsampled luminance pixel in the current image block to be processed that has not undergone downsampling, and the position information of at least one undownsampled luminance pixel, thereby determining or generating a chrominance prediction result for the pixels to be predicted.

[0150] In this embodiment, if the video or image format of the current image block to be processed is 4:2:2 or 4:2:0, then during the image prediction process performed by the processing device, dimensional differences between luminance blocks and corresponding chrominance blocks may occur in the image prediction process if the luminance pixels have not been downsampled. Therefore, the processing device sets position scaling information to determine the position information of the predicted chrominance pixels in the current image block to be processed according to the position scaling information, thereby accurately positioning the luminance pixels corresponding to the predicted chrominance pixels when the dimensions of the luminance blocks and corresponding chrominance blocks do not match.

[0151] Optionally, the location information of luminance pixels that have not been downsampled may be determined or generated according to at least one location scaling information. Optionally, as shown in Figures 9A and 9B (wherein "X" represents the location of a luminance pixel / sample in the image and "O" represents the location of a chrominance pixel / sample in the image), when a processing device uses location scaling information to determine the location information of a predicted chrominance pixel in the current image block to be processed, if the location scaling information in the x-direction is stepX, the location scaling information in the y-direction is stepY, and the location of the predicted chrominance sample is in the i-th row and j-th column of the chrominance block, then the location of the luminance sample at the same position as the predicted chrominance sample is in the i*stepY row and j*stepX column of the luminance block. For example, if stepX is 2, stepY is 1, and the location of the predicted chrominance sample is in the 1st row and 2nd column of the chrominance block, then the location of the luminance sample at the same position is in the 1st row and 4th column of the luminance block. Step size information of 2 for stepX and 1 for stepY corresponds to the 4:2:2 video / image format. If stepX is 2 and stepY is 2, and the predicted location of the chrominance sample is in the first row and second column of the chrominance block, then the location of the luminance sample at the same position as the predicted chrominance sample is in the second row and fourth column of the luminance block. Step size information of 2 for stepX and 2 for stepY corresponds to the 4:2:0 video / image format.

[0152] Optionally, the processing device may obtain or generate downsampled luminance pixels by performing a downsampling process on luminance pixels that have not been downsampled. Optionally, as shown in Figure 10, the processing device may obtain the sample value of a downsampled luminance sample by averaging samples within adjacent regions, and downsampling by the processing device through averaging samples within adjacent regions may include more background information of the image block. Optionally, as shown in Figure 11, the processing device may also use the maximum value of samples within adjacent regions as the sample value of a downsampled luminance sample, and a processing device that uses the maximum value of samples within adjacent regions as the sample value of a downsampled luminance sample may retain more texture information.

[0153] In other embodiments, the downsampling process may also be carried out using other methods. According to the implementation of this application, the downsampled pixel SD can be obtained using various downsampling methods. Optionally, a first downsampled pixel is obtained by an averaging method, and a second downsampled pixel is obtained by taking the maximum value. Subsequently, the downsampled pixel SD is obtained by performing a weighting process on the first and second downsampled pixels. Using multiple downsampling methods to obtain downsampled pixels provides novel downsampling methods, thereby improving the flexibility of encoding and decoding.

[0154] In this embodiment, the predicted pixel chrominance information is predicted according to at least one first luminance value and at least one location information of at least one undownsampled luminance pixel in the current image block to be processed that has not undergone downsampling, thereby obtaining the predicted pixel chrominance prediction result. Since the location information of the luminance pixel is used to determine or generate the predicted chrominance result, the prediction quality of the image block can be further improved when the pixel chrominance information prediction is performed on an image block where the pixel chrominance information prediction is correlated with the pixel value and the pixel location.

[0155] Optionally, during the encoding or decoding process for an image, after treating chrominance pixels in the current image block to be processed that require chrominance information prediction as pixels to be predicted, the processing device may also predict the chrominance information of the pixels to be predicted according to at least one first luminance value of at least one undownsampled luminance pixel in the current image block to be processed that has not undergone downsampling, at least one luminance offset of the image block, at least one chrominance offset of the image block, the gradient value of at least one undownsampled luminance pixel (for example, if there are multiple first luminance values, the gradient value of each of the multiple first luminance values ​​is used), and the position information of at least one undownsampled luminance pixel, thereby determining or generating a chrominance prediction result for the pixels to be predicted.

[0156] A fifth method: Determine or generate a prediction result according to at least one first luminance value and a second luminance value of at least one downsampled luminance pixel.

[0157] In this embodiment, during an encoding or decoding process for an image, the processing device uses chrominance pixels in the current image block to be processed that require chrominance information prediction as pixels to be predicted. The process of determining the pixels to be predicted includes predicting the chrominance information of the pixels to be predicted according to at least one first luminance value of at least one undownsampled luminance pixel in the current image block to be processed that has not undergone downsampling and a second luminance value of at least one downsampled luminance pixel that has undergone downsampling, thereby determining or generating a chrominance prediction result for the pixels to be predicted.

[0158] Optionally, the processing device predicts the chrominance information of the pixel to be predicted according to at least one first luminance value of at least one undownsampled luminance pixel in the current image block to be processed that has not undergone downsampling, and obtains a chrominance prediction result. Optionally, after the undownsampled luminance pixel is downsampled to obtain a downsampled luminance pixel, the chrominance information of the pixel to be predicted is predicted according to at least one second luminance value of the downsampled luminance pixel, and obtains another chrominance prediction result. Subsequently, the processing device fuses the two chrominance prediction results obtained through two-component intra-prediction to derive a final chrominance prediction result for the pixel to be predicted.

[0159] Optionally, if the current image block to be processed is in the 4:2:2 or 4:2:0 video or image format described above, the processing device may perform two inter-component intra-predictions on the current image block to be processed using the inter-component intra-prediction method, and then fuse the results obtained through the two inter-component intra-predictions to derive a final prediction result. Optionally, as shown in Figure 12, on the one hand, the processing device downsamples luminance block A using the inter-component intra-prediction method to obtain a downsampled luminance block A', and then determines the chrominance prediction result for the chrominance pixels in chrominance block a1 according to the luminance pixels in the downsampled luminance block A'. On the other hand, the processing device directly determines the chrominance prediction result for the chrominance pixels in chrominance block a2 according to the luminance pixels in luminance block A. Subsequently, the processing device fuses chrominance block a1 and chrominance block a2 to obtain the chrominance prediction result for chrominance block a.

[0160] Optionally, when a processing device performs weighted fusion of chrominance prediction results (chrominance value a1) obtained through downsampled inter-component intra prediction and chrominance prediction results (chrominance value a2) obtained through non-downsampled inter-component intra prediction, on the one hand, the chrominance value a1 of the predicted chrominance sample is obtained using downsampled inter-component intra prediction, and the chrominance value a2 of the predicted chrominance sample is obtained using non-downsampled inter-component intra prediction. Then, weighted fusion is performed on chrominance value a1 and chrominance value a2 to obtain a weighted chrominance value a. Optionally, a = w1*a1 + w2*a2, where w1 and w2 are weights.

[0161] Optionally, when the processing device is an encoder, the processing device may calculate the rate-strain cost J1 of a chrominance block consisting of multiple chrominance values ​​a1 obtained through downsampled inter-component intra prediction, and the rate-strain cost J2 of a chrominance block consisting of multiple chrominance values ​​a2 obtained through undownsampled inter-component intra prediction, thereby determining the values ​​of w1 and w2 according to the ratio of the rate-strain costs J1 and J2. Optionally, if J2 / J1=1:1, then w1 / w2=1, and if J2 / J1=1:2, then w1 / w2=2:1.

[0162] In this embodiment, the processing device determines the chrominance value of a chrominance pixel using the undownsampled and downsampled luminance values ​​within a luminance image block. Although luminance information may be lost during the downsampling process, the coordinate ranges corresponding to the downsampled luminance values ​​and the luminance values ​​corresponding to the non-downsampled process differ in images having a 4:2:2 or 4:2:0 video format. Therefore, the coordinate range corresponding to the downsampled luminance values ​​is broader than the coordinate range corresponding to the luminance values ​​corresponding to the non-downsampled process. Thus, by fusing the chrominance prediction values ​​corresponding to the luminance values ​​obtained through both the downsampling and non-downsampling processes, the processing device can achieve a more accurate chrominance prediction.

[0163] Optionally, during an encoding or decoding process for an image, after treating chrominance pixels in the current image block to be processed that require chrominance information prediction as pixels to be predicted, the processing device may also predict the chrominance information of the pixels to be predicted according to at least one first luminance value of at least one undownsampled luminance pixel in the current image block to be processed that has not undergone downsampling, at least one luminance offset of the image block, at least one chrominance offset of the image block, and a second luminance value of at least one downsampled luminance pixel that has undergone downsampling, thereby determining or generating a chrominance prediction result for the pixels to be predicted.

[0164] Optionally, the processing device may also predict the chrominance information of a pixel to be predicted according to at least one first luminance value of at least one undownsampled luminance pixel to be processed in the current image block that has not undergone downsampling, the gradient value of at least one undownsampled luminance pixel (for example, if there are multiple first luminance values, the gradient value of each of the multiple first luminance values ​​is used), and a second luminance value of at least one downsampled luminance pixel that has undergone downsampling, thereby determining or generating a predicted chrominance result for the pixel to be predicted.

[0165] Optionally, the processing device may also predict the chrominance information of a pixel to be processed, according to at least one first luminance value of at least one undownsampled luminance pixel in the current image block to be processed that has not undergone downsampling, the position information of at least one undownsampled luminance pixel, and a second gradient value of at least one downsampled luminance pixel that has undergone downsampling, thereby determining or generating a predicted chrominance result for the pixel to be processed.

[0166] Optionally, the processing device may also predict the chrominance information of a pixel to be processed, based on at least one first luminance value of at least one undownsampled luminance pixel in the current image block to be processed that has not undergone downsampling, the gradient value of at least one undownsampled luminance pixel (for example, if there are multiple first luminance values, the gradient value of each of the multiple first luminance values ​​is used), the position information of at least one undownsampled luminance pixel, and the second luminance value of at least one downsampled luminance pixel that has undergone downsampling, thereby determining or generating a predicted chrominance result for the pixel to be processed.

[0167] Optionally, the processing device may also predict the chrominance information of a pixel to be processed according to at least one first luminance value of at least one undownsampled luminance pixel in the current image block to be processed that has not undergone downsampling, at least one luminance offset of the image block, at least one chrominance offset of the image block, the gradient value of at least one undownsampled luminance pixel (for example, if there are multiple first luminance values, the gradient value of each of the multiple first luminance values ​​is used), the position information of at least one undownsampled luminance pixel, and the second luminance value of at least one downsampled luminance pixel that has undergone downsampling, thereby determining or generating a predicted chrominance result for the pixel to be processed.

[0168] Optionally, when a processing device encodes or decodes an image, the chrominance pixels in the current image block to be processed that require chrominance information prediction are used as pixels to be predicted, and the chrominance information of the pixels to be predicted may also be predicted using the first, second, third, fourth, and / or fifth methods described above, thereby obtaining predicted results of the chrominance information of the pixels to be predicted, all of which are intermediate prediction results, and then a weighted fusion process is performed on the multiple intermediate prediction results to obtain the final chrominance prediction result of the pixels to be predicted.

[0169] Optionally, assume that a processing device uses the first, second, third, fourth, and / or fifth methods described above to predict the chrominance information of a pixel to be predicted, resulting in chrominance values ​​ChromaA1, ChromaA2, ChromaA3, ChromaA4, and ChromaA5, respectively. The processing device may further treat the five chrominance values ​​as intermediate prediction results and perform a weighting process on the five intermediate prediction results of chrominance values ​​ChromaA1 to ChromaA5 to determine the final chrominance value ChromaFinal of the pixel to be predicted.

[0170] Optionally, the processing device may perform a weighting process on the intermediate prediction results regarding the chrominance information of the pixels to be predicted, according to the formula: ChromaFinal = k1*ChromaA1 + k2*ChromaA2 + k3*ChromaA3 + k4*ChromaA4 + k5*ChromaA5, where k1 to k5 are weights.

[0171] Optionally, when the processing device is an encoder, the rate-strain costs J1, J2, J3, J4, and J5 for predicting the chrominance information of the pixels to be predicted can be calculated, respectively, using the first, second, third, fourth, and / or fifth methods described above. The processing device can then determine the values ​​of k1 to k5 according to the ratio of the rate-strain costs J1 to J5. Optionally, if J1 / J2 / J3 / J4 / J5=1:1:1:1:1, then k1 / k2 / k3 / k4 / k5=1:1:1:1:1, or if J1 / J2 / J3 / J4 / J5=1:2:2:1:1, then k1 / k2 / k3 / k4 / k5=2:1:1:2:2.

[0172] Optionally, step S101 includes at least one of the following:

[0173] First method: Determine or generate a prediction result according to at least one first luminance value, at least one luminance offset, and at least one chrominance offset.

[0174] Similar to the principle and intent of step S100 described above, in this embodiment, during the encoding or decoding process for an image by the processing device, the processing device predicts the chrominance information of the pixel to be predicted according to at least one first luminance value of at least one downsampled luminance pixel in the current image block to be processed, at least one luminance offset of the image block, and at least one chrominance offset of the image block, thereby obtaining a predicted chrominance result for the pixel to be predicted. Thus, the magnitude of the luminance and chrominance values ​​used in the process of determining the weight coefficients during intercomponent prediction can be reduced (for example, the magnitude of the luminance and chrominance values ​​used to determine the weight coefficients c0~c6 described above can be reduced), thereby reducing the computational load. In other words, the processing device obtains a smaller luminance value by subtracting a luminance offset value from the luminance value, then performs a prediction process on the smaller luminance value to obtain a smaller chrominance value, and finally adds the chrominance offset value to the smaller chrominance value to obtain the final chrominance prediction value, thereby effectively reducing the computational load during the calculation process and / or reducing the memory space required to store intermediate values ​​generated during the calculation process.

[0175] Second method: Determine or generate a prediction result according to at least one first luminance value and gradient value of at least one downsampled luminance pixel.

[0176] Similar to the principle and intent of step S100 described above, in this embodiment, by utilizing the gradient values ​​of luminance pixels, compared to solutions in which gradient values ​​are not used, the processing device can predict the chrominance information of the pixels to be predicted according to at least one first luminance value of at least one downsampled luminance pixel in the current image block to be processed, and the gradient value of at least one downsampled luminance pixel, thereby obtaining a chrominance prediction result and enabling the prediction model to incorporate information about pixel edge variations, and thus improving the prediction quality of image blocks when predicting pixel chrominance information for image blocks containing object edges.

[0177] A third method: Determine or generate an intermediate prediction result for the pixels to be predicted according to at least one first luminance value, and determine or generate a prediction result according to the intermediate prediction result.

[0178] Optionally, the gradient value of at least one luminance pixel is the gradient value of at least one downsampled luminance pixel.

[0179] Similar to the principle and intent of step S100 described above, in this embodiment, when predicting chrominance information for a pixel to be predicted, the processing device first predicts an intermediate prediction result for the chrominance information of the pixel to be predicted, then performs further weighting calculation processing according to a plurality of intermediate prediction results to determine or generate a final chrominance prediction result for the pixel to be predicted, thereby obtaining a better chrominance prediction result for the pixel to be predicted.

[0180] A fourth method: Determine or generate a prediction result according to at least one first luminance value and position information of at least one downsampled luminance pixel.

[0181] Optionally, a downsampled luminance pixel is a second downsampled luminance pixel obtained or generated by downsampling a downsampled luminance pixel and / or another downsampled luminance pixel.

[0182] Similar to the principle and intent of step S100 described above, in this embodiment, the chrominance information of the pixel to be predicted is predicted according to at least one first luminance value of at least one downsampled luminance pixel in the current image block to be processed that has undergone downsampling, and the position information of at least one downsampled luminance pixel, thereby obtaining the predicted chrominance prediction result of the pixel to be predicted. Since the position information of the luminance pixel is used to determine or generate the predicted chrominance result, the prediction quality of the image block can be further improved when the pixel chrominance information prediction is performed on an image block where the pixel chrominance information prediction is correlated with the pixel value and the pixel position.

[0183] A fifth method: Determine or generate a prediction result according to at least one first luminance value and a second luminance value of at least one other downsampled luminance pixel.

[0184] Similar to the principle and intent of step S100 described above, in this embodiment, the processing device may perform two different downsampling processes on a luminance image block to obtain a first luminance value of a first downsampled luminance pixel resulting from a first downsampling process and a second luminance value of a second downsampled luminance pixel resulting from a second downsampling process. Thus, by fusing the first and second chrominance prediction values ​​obtained from the first and second luminance values, respectively, the processing device may obtain a more accurate chrominance prediction value.

[0185] It should be noted that the various embodiments and models described in this application can be combined with one another without departing from the fundamental principles and spirit of this application.

[0186] Third Embodiment

[0187] Based on any one of the embodiments described above, prior to step S10, the image processing method in this application is: The process further includes obtaining or determining a first pixel flag according to the dimensions of the luminance block in which the luminance pixels (non-downsampled luminance pixels and / or downsampled luminance pixels) are located, If the first pixel flag is a first value, step S10 includes directly determining or generating prediction results according to the luminance pixels (luminance pixels that have not been downsampled and / or luminance pixels that have been downsampled). Alternatively, if the first pixel flag is the second value, step S10 includes performing a downsampling process on the luminance pixel to obtain a downsampled luminance pixel, and determining or generating a prediction result according to the second luminance value of the downsampled luminance pixel.

[0188] Optionally, if the first pixel flag is the second value, step S10 includes performing a downsampling process on the luminance pixels that have not been downsampled to obtain downsampled luminance pixels, and determining or generating a prediction result according to the second luminance value of the downsampled luminance pixels.

[0189] Optionally, the luminance pixel is a first downsampled luminance pixel, and the downsampled luminance pixel is a second downsampled luminance pixel. If the first pixel flag is the second value, step S10 includes performing a downsampling process on the first downsampled luminance pixel to obtain a second downsampled luminance pixel, and determining or generating a prediction result according to the second luminance value of the second downsampled luminance pixel.

[0190] In this embodiment, the processing device may acquire or determine a first pixel flag according to the dimensions of an image block, and then decide whether to perform downsampling on the luminance pixels based on a first pixel flag of a different value before predicting chrominance information based on the luminance value of the luminance pixels. Optionally, if the first pixel flag acquired or determined according to the dimensions of an image block is a first value, the processing device may directly determine or generate chrominance prediction results based on the undownsampled luminance pixels without performing downsampling on the luminance pixels.

[0191] And / or, if the first pixel flag is the second value, the processing device performs downsampling on the unevaluated luminance pixels to obtain downsampled luminance pixels, and then determines or generates a chrominance prediction result according to the second luminance value of the downsampled luminance pixels.

[0192] When it is optionally decided whether to perform downsampling on luminance pixels according to the dimensions of an image block, if the dimensions of the image block are greater than a threshold, the processing device performs downsampling and then determines or generates a chrominance prediction result according to the second luminance value of the downsampled luminance pixels. If the dimensions of the image block are less than a threshold, downsampling is not performed, and the chrominance prediction result is determined or generated directly according to the luminance pixels that were not downsampled.

[0193] In this embodiment, the processing device requires a memory device to store luminance pixels used for chrominance prediction during the encoding and decoding process of image blocks. Therefore, for larger luminance image blocks, a larger memory device is required to store these luminance pixels if downsampling is not performed on the larger luminance image blocks. Thus, downsampling larger luminance image blocks may allow the use of a relatively smaller memory device to store these luminance pixels for chrominance prediction. However, in block partitioning, larger regions are regions with flat pixel variation. Therefore, the quality of the chrominance prediction results obtained after downsampling luminance image blocks tends to be satisfactory.

[0194] Optionally, to reduce inter-component redundancy, video coding standards may allow processing devices to use inter-component models for predicting the chrominance of the current block to be processed. Optionally, processing devices may predict chrominance pixels based on reconstructed luminance samples and inter-component models from the same image block. Optionally, inter-component models include inter-component linear models (CCLMs) and convolutional inter-component models (CCCMs). As shown in Figure 13, if the video or image format of the current image block to be processed is 4:2:2 or 4:2:0, the dimensions of the luminance blocks and chrominance blocks contained within the image block are not equal.

[0195] Optionally, in the aforementioned image blocks in 4:2:2 or 4:2:0 video or image formats, if the processing device uses the luminance pixels of the luminance blocks within the image block to predict the chrominance pixels of the chrominance blocks, the processing device may first perform a downsampling operation on the luminance blocks to obtain downsampled luminance blocks, and then determine the chrominance pixels in the chrominance blocks according to the luminance pixels in the downsampled luminance blocks. Optionally, as shown in Figure 14, the processing device may first downsample luminance block A to obtain downsampled luminance block A', and then determine the chrominance pixels in chrominance block A according to the luminance pixels in the downsampled luminance block A'.

[0196] Optionally, the processing device may also determine the chrominance pixels in the chrominance block directly according to the undownsampled luminance pixels in the luminance block, without performing downsampling on the luminance block. Optionally, as shown in Figure 15, the processing device does not need to downsample luminance block A (i.e., does not need to first obtain luminance block A' in Figure 14), but instead directly determines the chrominance pixels in chrominance block a according to the undownsampled luminance pixels in luminance block A.

[0197] Optionally, in the aforementioned image blocks in 4:2:2 or 4:2:0 video or image formats, if the processing device uses the luminance pixels of the luminance blocks within the image block to predict the chrominance pixels of the chrominance blocks, then non-downsampled inter-component intra-prediction may be performed for some image blocks in the video image, and downsampled inter-component intra-prediction may be performed for other image blocks in the video image, thereby improving the flexibility of chrominance prediction for image blocks.

[0198] Optionally, the processing device may indicate whether to use downsampled inter-component intra predictions or undownsampled inter-component intra predictions by determining the first pixel flag described above. Optionally, if the first pixel flag has a first value (e.g., the first value is 1), the processing device uses undownsampled inter-component intra predictions without downsampling the luminance pixels, but directly determines or generates chrominance prediction results according to the undownsampled luminance pixels. and / or, if the first pixel flag has a second value (e.g., the second value is 0), the processing device uses downsampled inter-component intra predictions, downsamples the luminance pixels to obtain downsampled luminance pixels, and then determines or generates chrominance prediction results according to the second luminance value of the downsampled luminance pixels.

[0199] Optionally, the value of the first pixel flag can be determined by several methods. Optionally, when the processing device is an encoder, if the rate-distortion cost corresponding to a chrominance block obtained through downsampled inter-component intra-prediction is less than the rate-distortion cost corresponding to a chrominance block obtained through undownsampled inter-luminance component intra-prediction for a given image block, the processing device adopts the downsampled inter-luminance component intra-prediction for the image block. In this case, the value of the first pixel flag is set to the second value (for example, the second value is 0). Optionally, if the rate-distortion cost corresponding to a chrominance block obtained through downsampled inter-luminance component intra-prediction is greater than the rate-distortion cost corresponding to a chrominance block obtained through undownsampled inter-luminance component intra-prediction for a given image block, the processing device adopts the undownsampled inter-luminance component intra-prediction for the image block. In this case, the value of the first pixel flag is set to the first value (for example, the first value is 1).

[0200] Optionally, the processing device may determine the value of the first pixel flag described above according to the dimensions of the image block or the dimensions of the luminance blocks within the image block. Optionally, if the length and / or width of the image block is greater than 32, the value of the first pixel flag is set to a second value (e.g., the second value is 0), which means that the processing device employs downsampled interluminance component intra-prediction for the image block. If the length and / or width of the image block is 32 or less, the value of the first pixel flag is set to a first value (e.g., the first value is 1), which means that the processing device employs undownsampled interluminance component intra-prediction for the image block.

[0201] Optionally, in this embodiment, the image processing method in this application is: The process further includes determining the prediction mode used to carry out step S10 according to the values ​​of the second pixel flag and the gradient flag.

[0202] In this embodiment, the processing device may acquire or determine a second pixel flag and / or gradient flag, and then determine a prediction mode used to predict the chrominance information of the pixels to be predicted in the image block in step S10 above, according to the values ​​of the second pixel flag and gradient flag.

[0203] When performing step S10 described above to optionally predict the chrominance information of pixels to be predicted within an image block, the processing device may specifically employ the following four inter-component prediction modes.

[0204] Mode 1: Inter-component intra prediction using downsampling and gradient calculation.

[0205] Mode 2: Inter-component intra prediction using downsampling but without gradient calculation.

[0206] Mode 3: Inter-component intra-prediction using gradient calculation without downsampling (compared to Mode 1, Mode 3 directly performs gradient calculation based on luminance values ​​that have not been downsampled, resulting in higher accuracy in the calculated gradient values).

[0207] Mode 4: Inter-component intra prediction without downsampling or gradient calculation.

[0208] Optionally, a prediction method using downsampling may allow the luminance and chrominance blocks for prediction to have constant dimensions, thereby eliminating the need to consider dimensional mismatch issues in subsequent calculations and simplifying the computation. And / or, as a result of employing a downsampling method, the dimensions of the downsampled luminance blocks are reduced, thereby saving some memory space. Optionally, a prediction method without downsampling may result in dimensional mismatch between the luminance and chrominance blocks for prediction, requiring consideration of how to reconcile the dimensions in subsequent calculations. However, a prediction method without downsampling directly utilizes the undownsampled luminance image blocks for prediction, thereby avoiding the loss of luminance information during the downsampling process, making the prediction method without downsampling more suitable for encoding and decoding scenarios for images or videos requiring higher sharpness and detail, and / or scenarios where the impact of lost luminance information during downsampling on chrominance prediction needs to be reduced. Optionally, in a prediction method that utilizes the luminance value of a luminance sample without its gradient value, if the prediction model has the same number of terms, the prediction model can use the luminance values ​​of multiple adjacent luminance samples for prediction. Therefore, a prediction method that utilizes the luminance value of a luminance sample without its gradient value is suitable for image blocks where the pixel value of the pixel sample to be predicted has a stronger correlation with that of adjacent pixel samples. Optionally, compared to a prediction method that does not utilize gradient values, a prediction method that utilizes the gradient value of a luminance sample improves the prediction quality for image blocks containing object edges because the prediction model includes information about pixel edge variations.

[0209] In this embodiment, modes 1 to 4 described above can be obtained by combining whether or not to perform downsampling with whether or not to adopt the gradient value of the luminance sample for prediction, thereby enabling the coding and decoding processes to apply different modes to different scenarios and further improving the flexibility of predicting the chrominance of the pixels to be predicted.

[0210] Optionally, in Mode 1 described above, the prediction model PreChorma=f(Luma1,Gx,Gy) may be used to calculate the chrominance values ​​of chrominance pixels in an image block that require chrominance information prediction. PreChorma represents the chrominance pixel to be predicted, Luma1 represents the downsampled luminance pixel at the same location as the chrominance pixel to be predicted, and Gx and Gy represent the horizontal and vertical gradients obtained by applying a gradient filter centered on Luma1, respectively.

[0211] Arbitrarily, the predictive model PreChorma=f(Luma1,Gx,Gy) can also be transformed into predChroma=c0*Luma1+c1*Gx+c2*Gy+c3*P+c4*B, where c0~c4 are weights, P is a nonlinear term, and B is a bias term. Gx=(2Luma5+Luma2+Luma7)-(2Luma6+Luma4+Luma9) and Gy=(2Luma3+Luma2+Luma4)-(2Luma8+Luma7+Luma9). Luma2 is the top-left luminance pixel of Luma1, Luma3 is the luminance pixel above Luma1, Luma4 is the top-right luminance pixel of Luma1, Luma5 is the left luminance pixel of Luma1, Luma6 is the right luminance pixel of Luma1, Luma7 is the bottom-left luminance pixel of Luma1, Luma8 is the bottom luminance pixel of Luma1, and Luma9 is the bottom-right luminance pixel of Luma1. Luma1 through Luma9 can be arbitrarily represented as downsampled luminance pixels.

[0212] Optionally, in Mode 2, the prediction model PreChorma=f(Luma1,Luma2,Luma3,...,LumaN) may be used to calculate the chrominance values ​​of chrominance pixels in an image block that require chrominance information prediction. PreChorma represents the chrominance pixels to be predicted, and Luma1~LumaN are input luminance pixels, which are downsampled luminance pixels. Optionally, Luma1 is the luminance pixel at the same location as the chrominance pixel to be predicted, and Luma2, Luma3,..., and LumaN are luminance pixels around the luminance pixel at the same location.

[0213] Arbitrarily, the above prediction model PreChorma=f(Luma1,Luma2,Luma3,...,LumaN) can be transformed into predChromaVal=c0*Luma1+c1*Luma2+c2*Luma3+c3*Luma4+c4*Luma5+c5*Luma6+c6*Luma7+c7*Luma8+c8*Luma9+c9*P+c10*B when combined with a nonlinear term P and a bias term B, where c0~c 10 is the weight, Luma2 is the top-left luminance pixel of Luma1, Luma3 is the top luminance pixel of Luma1, Luma4 is the top-right luminance pixel of Luma1, Luma5 is the left luminance pixel of Luma1, Luma6 is the right luminance pixel of Luma1, Luma7 is the bottom-left luminance pixel of Luma1, Luma8 is the bottom luminance pixel of Luma1, and Luma9 is the bottom-right luminance pixel of Luma1.

[0214] Optionally, in Mode 3, the prediction model PreChorma=f(Luma1,Gx,Gy) may be used to calculate the chrominance values ​​of chrominance pixels in an image block that require chrominance information prediction. PreChorma represents the chrominance pixel to be predicted, Luma1 represents the luminance pixel at the same location as the chrominance pixel that will be predicted but has not been downsampled, and Gx and Gy represent the horizontal and vertical gradients obtained by applying a gradient filter centered on Luma1, respectively.

[0215] Arbitrarily, the predictive model PreChorma=f(Luma1,Gx,Gy) can also be transformed into predChroma=c0*Luma1+c1*Gx+c2*Gy+c3*P+c4*B, where c0~c4 are weights, P is a nonlinear term, B is a bias term, Gx=(2Luma5+Luma2+Luma7)-(2Luma6+Luma4+Luma9), and Gy=(2Luma3+Luma2+Luma4)-(2Luma8+Luma7+Luma9). Luma2 is the top-left luminance pixel of Luma1, Luma3 is the luminance pixel above Luma1, Luma4 is the top-right luminance pixel of Luma1, Luma5 is the left luminance pixel of Luma1, Luma6 is the right luminance pixel of Luma1, Luma7 is the bottom-left luminance pixel of Luma1, Luma8 is the bottom luminance pixel of Luma1, and Luma9 is the bottom-right luminance pixel of Luma1. Luma1 through Luma9 can be arbitrary, representing luminance pixels that have not been downsampled.

[0216] Optionally, in Mode 4, the prediction model PreChorma=f(Luma1,Luma2,Luma3,...,LumaN) may be used to calculate the chrominance values ​​of chrominance pixels in an image block that require chrominance information prediction. PreChorma represents the chrominance pixels that will be predicted, and Luma1~LumaN are input luminance pixels that are not downsampled. Optionally, Luma1 is the luminance pixel at the same location as the chrominance pixel that will be predicted, and Luma2, Luma3,..., and LumaN are the luminance pixels around the luminance pixel at the same location.

[0217] Arbitrarily, the above prediction model PreChorma=f(Luma1,Luma2,Luma3,...,LumaN) can be transformed into predChromaVal=c0*Luma1+c1*Luma2+c2*Luma3+c3*Luma4+c4*Luma5+c5*Luma6+c6*Luma7+c7*Luma8+c8*Luma9+c9*P+c10*B when combined with a nonlinear term P and a bias term B, where c0~c 10 is the weight, Luma2 is the top-left luminance pixel of Luma1, Luma3 is the top luminance pixel of Luma1, Luma4 is the top-right luminance pixel of Luma1, Luma5 is the left luminance pixel of Luma1, Luma6 is the right luminance pixel of Luma1, Luma7 is the bottom-left luminance pixel of Luma1, Luma8 is the bottom luminance pixel of Luma1, and Luma9 is the bottom-right luminance pixel of Luma1.

[0218] Optionally, when a processing device acting as an encoding device encodes an image, the processing device may select the optimal chrominance prediction mode from the four chrominance prediction modes described above: Mode 1 to Mode 4, according to rate-distortion optimization (RDO).

[0219] Optionally, if Rate-Distortion Optimization (RDO) indicates that Mode 1 of the four chrominance prediction modes described above provides the best prediction for the chrominance block in image block X, the value of the second pixel flag is set to "1" and the gradient flag is also set to "1".

[0220] Optionally, if the RDO or SATD algorithm indicates that mode 2 of the four chrominance prediction modes described above provides the best prediction for the chrominance block in image block X, the value of the second pixel flag is set to "1" and the gradient flag is set to "0".

[0221] Optionally, as shown in Figure 16, the processing device acting as a decoder may analyze a second pixel flag and a gradient flag. If the value of the second pixel flag is 1, the processing device decides to perform downsampling, and / or if the value of the second pixel flag is 0, the processing device decides not to perform downsampling.

[0222] Optionally, if the gradient flag value is 1, the processing device decides to use the gradient value of the luminance sample to determine the corresponding chrominance sample, and / or, if the gradient flag value is 0, the processing device decides not to use the gradient value of the luminance sample to determine the corresponding chrominance sample. That is, based on combinations such as "11", "01", "10", and "00", the processing device decides which particular mode to adopt from the modes 1, 2, 3, or 4 described above when performing step S10 described above to predict the chrominance information of the pixels that will be predicted in the image block.

[0223] Optionally, the processing device may also set the values ​​of the second pixel flag and gradient flag using the Sum of Absolute Differences (SAD) algorithm (i.e., calculating the sum of absolute errors between the predicted block and the source image) or the SATD algorithm (i.e., calculating the sum of absolute values ​​of coefficients after the residuals have been Hadamard transformed), instead of using the RDO algorithm described above.

[0224] In this embodiment, the four prediction modes are obtained by combining two factors: whether to perform downsampling and whether to adopt the gradient value of the luminance sample. Thus, the processing device can select the optimal chrominance prediction mode from the multiple chrominance prediction modes according to the RDO or SATD algorithm, thereby increasing the processing device's flexibility in predicting chrominance pixels within an image block.

[0225] One embodiment of this application further provides a processing device including memory and a processor. The memory stores an image processing program. When executed by the processor, the image processing program causes steps of an image processing method to be carried out by any one of the image processing methods described above.

[0226] One embodiment of this application further provides a storage medium in which an image processing program is stored. When executed by a processor, the image processing program causes the processor to perform a step of an image processing method according to any one of the image processing methods described above.

[0227] Embodiments of processing devices and storage media provided in this application may include all technical features of any one of the image processing method embodiments described above. The development and description herein are essentially the same as those in the various embodiments of the methods described above, and further details are not provided herein.

[0228] One embodiment of this application further provides a computer program product including computer program code. When executed on a computer, the computer program code causes the computer to implement the method in the possible embodiments described above.

[0229] One embodiment of this application further provides a chip including memory and a processor. The memory is configured to store a computer program. The processor is configured to call and execute a computer program from the memory, thereby enabling a device comprising the chip to carry out the methods described in the various possible embodiments above.

[0230] It should be understood that the scenarios described above are merely examples and do not constitute limitations on the application scenarios of the technical solutions provided in the embodiments of this application. The technical solutions of this application may be applicable to other scenarios as well. For example, those skilled in the art will understand that, with the evolution of system architectures and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0231] The sequential numbering of the embodiments described above in this application is for illustrative purposes only and does not indicate priority of the embodiments.

[0232] The steps of the method in the embodiments of this application may be subject to sequence adjustments, mergers, and deletions as required by actual needs.

[0233] The units within the device in the embodiments of this application may be combined, separated, and removed according to actual needs.

[0234] In this application, detailed descriptions of the same or similar terms, technical solutions, and / or application scenarios are generally provided only upon their initial appearance. For the sake of brevity, detailed descriptions are generally omitted in subsequent repetitions. When understanding the technical solutions and other contents of this application, references may be made to the relevant detailed descriptions provided earlier for the same or similar terms, technical solutions, and / or application scenarios that are not described in detail later. In this application, the descriptions of embodiments have their own focus. For parts that are not described or documented in detail in one embodiment, references may be made to the relevant descriptions in other embodiments.

[0235] The various technical features of the technical solutions of this application can be freely combined. For the sake of brevity of explanation, not all possible combinations of the various technical features of the embodiments described above will be described. However, these combinations of technical features should be considered within the scope recorded in this application, provided that they are consistent.

[0236] As described above, those skilled in the art will clearly understand that the methods in the embodiments described above can be implemented using software combined with a necessary general-purpose hardware platform, or, obviously, using hardware, but that in many cases the former is the preferred implementation. Based on such understanding, the technical solutions of this application, either essentially or contributing to the prior art, may be presented in the form of a software product. The computer software product is stored on a storage medium such as those described above (e.g., read-only memory (ROM) / random access memory (RAM), magnetic disks, and optical disks) and includes several instructions for causing a terminal device (e.g., a mobile phone, computer, server, controlled terminal, or network device) to implement the methods described in the embodiments of this application.

[0237] The embodiments described above may be implemented, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, the embodiments described above may be implemented, in whole or in part, in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded into a computer and executed, the processes or functions described in the embodiments of this application are generated, in whole or in part. The computer may be a general-purpose computer, a dedicated computer, a computer network, or another programmable device. Computer instructions may be stored in a storage medium or transmitted from one storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., coaxial cable, optical fiber, and digital subscriber line) or wirelessly (e.g., infrared, wireless, and microwave). The storage medium may be any available medium accessible by the computer, or it may be a server, data center, and another data storage device that integrates one or more available mediums. The available media may include magnetic media (e.g., floppy disks, memory disks, and magnetic tapes), optical media (e.g., digital versatile disks (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)). The foregoing are merely preferred embodiments of this application and are not intended to limit the scope of the patent of this application. Any equivalent structural or process transformations, or direct or indirect applications in other related technical fields, made using the contents of the specification and accompanying drawings of this application are similarly included within the scope of the patent protection of this application.

Claims

1. An image processing method, Step S10: An image processing method comprising determining or generating a prediction result for a pixel that will be predicted according to at least one first luminance value of at least one luminance pixel.

2. The aforementioned step S10 is, First method: Determining or generating the prediction result according to at least one first luminance value, at least one luminance offset, and at least one chrominance offset. Second method: Determining or generating the prediction result according to the at least one first luminance value and the gradient value of the at least one luminance pixel, A third method: determining or generating an intermediate prediction result for the pixels to be predicted according to at least one first brightness value, and determining or generating the prediction result according to the intermediate prediction result. A fourth method: Determining or generating the prediction result according to the at least one first luminance value and the position information of the at least one luminance pixel, or Fifth method: Determining or generating the prediction result according to the at least one first luminance value and the second luminance value of at least one downsampled luminance pixel. The processing method according to claim 1, comprising at least one of the following.

3. The aforementioned interim forecast results include the first interim forecast result and / or the second interim forecast result, and / or The processing method according to claim 2, wherein determining or generating the prediction result according to the intermediate prediction result includes performing a weighting process on the first intermediate prediction result and the second intermediate prediction result in order to determine or generate the prediction result.

4. below: The first intermediate prediction result is determined or generated according to the at least one first luminance value. The second intermediate prediction result is determined or generated according to the gradient value of the at least one luminance pixel, or The weights for the weighting process are determined or generated according to the strain levels corresponding to the first and second intermediate prediction results. The processing method according to claim 3, wherein at least one of the following is the processing method according to claim 3.

5. below: The aforementioned location information is determined or generated according to at least one location scaling piece of information, or The downsampled luminance pixels are obtained or generated by downsampling the luminance pixels. The processing method according to any one of claims 2 to 4, wherein the processing method is at least one of the following.

6. The above method, before step S10, The process further includes obtaining or determining a first pixel flag according to the dimensions of the luminance block in which the luminance pixel is located. If the first pixel flag is a first value, step S10 includes directly determining or generating the prediction result according to the brightness pixel, or The processing method according to any one of claims 1 to 4, wherein, if the first pixel flag is a second value, step S10 includes performing a downsampling process on the luminance pixel to obtain a downsampled luminance pixel, and determining or generating the prediction result according to the second luminance value of the downsampled luminance pixel.

7. The processing method according to any one of claims 1 to 4, wherein the prediction mode used to carry out step S10 is determined according to the values ​​of the second pixel flag and the gradient flag.

8. below: The at least one luminance pixel is a luminance pixel that did not undergo downsampling during the image coding or decoding process. The predicted pixels are chrominance pixels, or The aforementioned prediction results are chrominance prediction results. The processing method according to any one of claims 1 to 4, wherein the processing method is at least one of the following.

9. A processing device comprising memory and a processor, wherein an image processing program is stored in the memory, and the image processing program, when executed by the processor, causes the steps of the image processing method described in any one of claims 1 to 8 to be carried out.

10. A storage medium having a computer program stored therein, wherein the computer program, when executed by a processor, causes the processor to perform the steps of the image processing method described in any one of claims 1 to 8.

Images