Image processing method, processing device and storage medium

Abstract

The present application provides an image processing method, a processing device and a storage medium. The image processing method can be applied to the processing device, and comprises: on the basis of image block information of a current block, determining or obtaining at least one reference image region; and, on the basis of the at least one reference image region, determining or obtaining a prediction mode for the current block. The technical solution of the present application can improve the prediction effect of intra prediction, thereby improving the prediction effect of video encoding and / or decoding.

Description

Image processing methods, processing devices and storage media Technical Field

[0001] This application relates to the field of image processing technology, specifically to an image processing method, processing device, and storage medium. Background Technology

[0002] The video coding techniques proposed by existing video coding standards aim to improve coding performance without significantly increasing computational complexity. Specifically, these techniques include dividing each frame into different blocks during video encoding and decoding, followed by prediction, transformation, and quantization processing, as well as entropy coding or entropy decoding.

[0003] In the process of conceiving and implementing this application, the inventors discovered at least the following problems:

[0004] In the image patch prediction stage, the prediction mode used can be determined by derivation at the decoding end. However, because the accuracy of estimating the probability of each prediction mode by derivation at the decoding end is not high, the prediction performance of the prediction mode determined in this way is poor. Therefore, it is necessary to improve the accuracy of the derived prediction mode during video encoding and decoding.

[0005] The preceding description is intended to provide general background information and does not necessarily constitute prior art.

[0006] Summary of the Invention

[0007] To address the aforementioned technical problems, this application provides an image processing method, processing device, and storage medium, aiming to solve the technical problem of how to improve the accuracy of deduced prediction modes during video encoding and decoding.

[0008] This application provides an image processing method, applicable to a processing device, comprising the following steps:

[0009] S10, determine or obtain at least one reference image region based on the image block information of the current block;

[0010] S20, determine or obtain the prediction mode of the current block based on at least one reference image region.

[0011] Optionally, step S10 includes at least one of the following:

[0012] Determine or obtain at least one reference image region based on at least one of the following: width, height, block size, block area, image block attributes, and image block type of the current block.

[0013] Based on image block information from at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, and default blocks corresponding to the current block, determine or obtain at least one reference image region;

[0014] Based on the image block information of the first component image block of the current block, determine or obtain at least one reference image region of the second component image block of the current block;

[0015] Based on the candidate motion vector or candidate block vector of the current block, determine or obtain the image block information of the candidate block, and at least one reference image region is determined or obtained;

[0016] If the current block satisfies the first condition, then at least one reference image region is determined or obtained according to the first calculation strategy;

[0017] If the current block does not meet the first condition, then at least one reference image region is determined or obtained according to the second calculation strategy.

[0018] Optionally, the second computation strategy may differ from the first computation strategy.

[0019] Optionally, the current block satisfies a first condition, including at least one of the following:

[0020] The width of the current block is greater than or equal to the first preset width threshold;

[0021] The height of the current block is greater than or equal to the first preset height threshold;

[0022] The area of ​​the current block is greater than or equal to the first preset area threshold.

[0023] Optionally, step S20 includes the following steps:

[0024] S21, determine or obtain a first probability of occurrence of at least one predicted pattern based on pixels, image blocks or image regions in at least one reference image region;

[0025] S22, determine or obtain the prediction pattern of the current block based on at least one first occurrence probability.

[0026] Optionally, step S21 includes at least one of the following:

[0027] Based on pixels, image blocks, or image regions in at least one reference image region, determine or obtain a first amplitude value and a second amplitude value in the same prediction direction, and determine or obtain a first probability of occurrence based on the first amplitude value and the second amplitude value in the same prediction direction.

[0028] At least one first amplitude value is determined or obtained based on a pixel, image block, or image region in at least one reference image region, and a first probability of occurrence is determined or obtained based on the first amplitude value and a scaling weight factor.

[0029] Based on the gradient magnitude and gradient direction of the gradient of a pixel in at least one reference image region, determine or obtain a first probability of occurrence of the prediction direction of at least one prediction mode.

[0030] Based on the prediction pattern corresponding to an image patch or image region in at least one reference image region, and the area of ​​the image patch or image region, determine or obtain a first probability of occurrence of the prediction direction of at least one prediction pattern;

[0031] A first probability of occurrence is determined or obtained based on at least one of the following: position, horizontal distance, vertical distance, and shortest distance between a pixel, image block, or image region in at least one reference image region and the current block;

[0032] Based on the pixel type of the pixel, the image block type of the image block, and the image region type of the image region, determine or obtain the first probability of occurrence;

[0033] If the syntax element obtained in the bitstream satisfies the second condition, the first occurrence probability is determined or obtained according to the first occurrence probability calculation strategy and at least one pixel, image block or image region in the reference image region. If the syntax element obtained in the bitstream does not satisfy the second condition, the first occurrence probability is determined or obtained according to the second occurrence probability calculation strategy, which is different from the first occurrence probability calculation strategy, and at least one pixel, image block or image region in the reference image region.

[0034] Optionally, step S22 includes at least one of the following:

[0035] Based on the first occurrence probability and the scaling weight factor, determine the second occurrence probability, and based on the second occurrence probability, determine or obtain the prediction pattern of the current block;

[0036] Based on the first and third occurrence probabilities of the same prediction direction, the prediction pattern of the current block is determined or obtained;

[0037] The scaling weight factor for the first occurrence probability is determined based on the fourth occurrence probability, and the prediction mode for the current block is determined based on the first occurrence probability and the scaling weight factor.

[0038] Optionally, the method of determining or obtaining the scaling weight factor includes at least one of the following:

[0039] Based on the position of a pixel, an image block, and at least one of the elements in at least one reference image region relative to the current block, a scaling weight factor is determined or obtained.

[0040] The scaling weight factor is determined or obtained based on at least one of the first image region, first image block and first pixel that is adjacent and / or non-adjacent to at least one of the pixels, image blocks and at least one of the image regions in at least one reference image region;

[0041] Based on the image region type of at least one image region, and / or the image block type of at least one image block, and / or the pixel type of at least one pixel, determine or obtain the scaling weight factor;

[0042] Based on the horizontal distance, and / or vertical distance, and / or shortest distance between the current block and a pixel in at least one reference image region, an image block, and at least one item in the image region; determine or obtain the scaling weight factor.

[0043] If the syntax elements obtained from the bitstream meet the third condition, the first scaling weight factor calculation strategy is used to determine or obtain the scaling weight factor. If the syntax elements obtained from the bitstream meet the fourth condition, the second scaling weight factor calculation strategy, which is different from the first scaling weight factor calculation strategy, is used to determine or obtain the scaling weight factor.

[0044] Optionally, the image processing method further includes at least one of the following:

[0045] Based on at least one of the gradient of pixels in different reference image regions, the area of ​​image blocks or image regions, determine or obtain at least two different occurrence probabilities, and determine or obtain the prediction mode of the current block based on the at least two different occurrence probabilities.

[0046] The reference image region includes at least one of the left image region located to the left of the current block and the upper image region located above the current block;

[0047] The prediction pattern of the current block is determined or obtained based on the maximum probability of occurrence of at least one first probability.

[0048] This application also provides an image processing apparatus, which includes:

[0049] The processing module is used to determine or obtain at least one reference image region based on the image block information of the current block; and to determine or obtain the prediction mode of the current block based on at least one reference image region.

[0050] This application also provides a processing device, including: a memory and a processor, wherein the memory stores an image processing program, and when the image processing program is executed by the processor, it implements the steps of any of the image processing methods described above.

[0051] This application also provides a storage medium storing a computer program that, when executed by a processor, implements the steps of any of the image processing methods described above.

[0052] As described above, the processing method of this application can be applied to a processing device, including: determining or obtaining at least one reference image region based on the image block information of the current block, and determining or obtaining a prediction mode for the current block based on the at least one reference image region. Through the technical solution of this application, during the prediction stage of video encoding and / or decoding, such as intra-frame prediction, the prediction mode can be directly determined based on at least one reference image region determined from the image information of the current block. This takes into account the relationship between the reference image region and the current block, making the determined prediction mode more accurate. In other words, it improves the accuracy of deriving the prediction mode during video encoding and / or decoding, thereby improving the prediction effect for the current block, such as increasing prediction accuracy, and thus improving the prediction efficiency during video encoding and / or decoding. Attached Figure Description

[0053] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0054] Figure 1 is a schematic diagram of the hardware structure of a mobile terminal implementing various embodiments of this application;

[0055] Figure 2 is a communication network system architecture diagram provided in an embodiment of this application;

[0056] Figure 3 is a schematic diagram of the hardware structure of a controller 140 provided in this application;

[0057] Figure 4 is a schematic diagram of the hardware structure of a network node 150 provided in this application;

[0058] Figure 5 is a flowchart illustrating the image processing method according to the first embodiment;

[0059] Figure 6 is a schematic diagram of the encoder's encoding process in the image processing method according to the first embodiment;

[0060] Figure 7 is a schematic diagram of the decoding process of the decoder in the image processing method according to the first embodiment;

[0061] Figure 8 is a schematic diagram of a DIMD template in an image processing method according to the second embodiment;

[0062] Figure 9 is a schematic diagram of gradient magnitude in the image processing method according to the second embodiment;

[0063] Figure 10 is a schematic diagram of the encoded region corresponding to the block to be predicted in the image processing method according to the second embodiment;

[0064] Figure 11 is a schematic diagram of the decoded region corresponding to the block to be predicted in the image processing method according to the second embodiment;

[0065] Figure 12 is a schematic diagram of the expanded DIMD template in the image processing method according to the second embodiment;

[0066] Figure 13 is a schematic diagram of image region segmentation into image sub-regions in the image processing method according to the third embodiment;

[0067] Figure 14 is a schematic diagram of an image sub-region in the image processing method according to the third embodiment;

[0068] Figure 15 is a schematic diagram of pixels and adjacent pixels in an image processing method according to a third embodiment;

[0069] Figure 16 is a schematic diagram of a coding block and adjacent coding blocks in an image processing method according to a third embodiment;

[0070] Figure 17 is a schematic diagram of the distance between a pixel and the current block in the image processing method according to the third embodiment;

[0071] Figure 18 is a schematic diagram of the processing module in the image processing device.

[0072] The realization of the objectives, functional features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0073] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0074] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0075] It should be understood that although the terms first, second, third, etc., may be used herein to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word “if” as used herein may be interpreted as “when…” or “in response to determination”. Furthermore, as used herein, the singular forms “a,” “an,” and “the” are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms “comprising,” “including,” indicate the presence of the stated feature, step, operation, element, component, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms “or,” “and / or,” “including at least one of the following,” etc., as used in this application may be interpreted as inclusive, or mean any one or any combination thereof. For example, "including at least one of the following: A, B, C" means "any one of the following: A; B; C; A and B; A and C; B and C; A and B and C." Similarly, "A, B, or C" or "A, B, and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A and B and C." Exceptions to this definition only occur when the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0076] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0077] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0078] It should be noted that step designations such as S10 and S20 are used in this document for the purpose of more clearly and concisely describing the corresponding content, and do not constitute a substantial limitation on the order. In specific implementation, those skilled in the art may execute S20 first and then S10, etc., but these should all be within the protection scope of this application.

[0079] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0080] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustration and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0081] The processing device in this application can be a smart terminal or a server, and the smart terminal can be implemented in various forms. For example, the smart terminal described in this application can include smart terminals such as mobile phones, tablets, laptops, handheld computers, personal digital assistants (PDAs), portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, etc., as well as fixed terminals such as digital TVs and desktop computers.

[0082] The following description will use a mobile terminal as an example. Those skilled in the art will understand that, apart from elements specifically designed for mobile purposes, the construction according to the embodiments of this application can also be applied to fixed-type terminals.

[0083] Please refer to Figure 1, which is a schematic diagram of the hardware structure of a mobile terminal implementing various embodiments of this application. The mobile terminal 100 may include: an RF (Radio Frequency) unit 101, a WiFi module 102, an audio output unit 103, an A / V (Audio / Video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111, etc. Those skilled in the art will understand that the mobile terminal structure shown in Figure 1 does not constitute a limitation on the mobile terminal. The mobile terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0084] The following section, with reference to Figure 1, provides a detailed description of each component of the mobile terminal:

[0085] The radio frequency unit 101 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 110; additionally, it transmits uplink data to the base station. Typically, the radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, and a duplexer. Furthermore, the radio frequency unit 101 can also communicate wirelessly with networks and other devices. The aforementioned wireless communications may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution), 5G, and 6G.

[0086] WiFi is a short-range wireless transmission technology. Mobile terminals using WiFi module 102 can help users send and receive emails, browse web pages, and access streaming media, providing wireless broadband internet access. Although Figure 1 shows WiFi module 102, it is understood that it is not an essential component of the mobile terminal and can be omitted as needed without altering the essence of the invention.

[0087] The audio output unit 103 can convert audio data received by the radio frequency unit 101 or the WiFi module 102 or stored in the memory 109 into audio signals and output them as sound when the mobile terminal 100 is in call signal receiving mode, call mode, recording mode, voice recognition mode, broadcast receiving mode, etc. Furthermore, the audio output unit 103 can also provide audio output related to specific functions performed by the mobile terminal 100 (e.g., call signal receiving sound, message receiving sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.

[0088] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos acquired by an image capture device (such as a camera) in video capture mode or image capture mode. The processed image frames can be displayed on the display unit 106. The image frames processed by the GPU 1041 can be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) in operating modes such as telephone call mode, recording mode, and voice recognition mode, and can process such sound into audio data. The processed audio (voice) data can be converted into a format that can be transmitted to a mobile communication base station via the radio frequency unit 101 in telephone call mode. The microphone 1042 can implement various types of noise cancellation (or suppression) algorithms to eliminate (or suppress) noise or interference generated during the reception and transmission of audio signals.

[0089] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Optionally, the light sensor includes an ambient light sensor and a proximity sensor. Optionally, the ambient light sensor can adjust the brightness of the display panel 1061 according to the ambient light level, and the proximity sensor can turn off the display panel 1061 and / or backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc. Other sensors that may be configured in the phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0090] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0091] User input unit 107 can be used to receive input numerical or character information, and generate key signal inputs related to user settings and function control of the mobile terminal. Optionally, user input unit 107 may include touch panel 1071 and other input devices 1072. Touch panel 1071, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near touch panel 1071), and drive corresponding connection devices according to a pre-set program. Touch panel 1071 may include a touch detection device and a touch controller. Optionally, the touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to processor 110, and can receive and execute commands sent by processor 110. In addition, touch panel 1071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may also include other input devices 1072. Optionally, other input devices 1072 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc., without being specifically limited here.

[0092] Optionally, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. Subsequently, the processor 110 provides corresponding visual output on the display panel 1061 according to the type of touch event. Although in FIG. 1, the touch panel 1071 and the display panel 1061 are implemented as two independent components to realize the input and output functions of the mobile terminal, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to realize the input and output functions of the mobile terminal. The specific implementation is not limited here.

[0093] Interface unit 108 serves as an interface through which at least one external device can connect to mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, and so on. Interface unit 108 may be used to receive input (e.g., data, power, etc.) from the external device and transmit the received input to one or more elements within mobile terminal 100, or it may be used to transmit data between mobile terminal 100 and the external device.

[0094] The memory 109 can be used to store software programs and various data. The memory 109 may primarily include a program storage area and a data storage area. Optionally, the program storage area may store the operating system, applications required for at least one function (such as sound playback, image playback, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory 109 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0095] The processor 110 is the control center of the mobile terminal. It connects various parts of the mobile terminal via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 109, and by calling data stored in the memory 109, it performs various functions and processes data of the mobile terminal, thereby providing overall monitoring of the mobile terminal. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor. Optionally, the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 110.

[0096] The mobile terminal 100 may also include a power supply 111 (such as a battery) that supplies power to various components. Preferably, the power supply 111 can be logically connected to the processor 110 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0097] Although not shown in Figure 1, the mobile terminal 100 may also include a Bluetooth module, etc., which will not be described in detail here.

[0098] To facilitate understanding of the embodiments of this application, the communication network system on which the mobile terminal of this application is based is described below.

[0099] Please refer to Figure 2, which is a communication network system architecture diagram provided in an embodiment of this application. The communication network system is an LTE system based on the universal mobile communication technology. The LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203, and the operator's IP services 204, which are connected in sequence.

[0100] Optionally, UE201 can be the aforementioned terminal 100, which will not be described in detail here.

[0101] E-UTRAN202 includes eNodeB2021 and other eNodeB2022, etc. Optionally, eNodeB2021 can connect to other eNodeB2022 via backhaul (e.g., X2 interface), and eNodeB2021 connects to EPC203, providing access from UE201 to EPC203.

[0102] EPC203 may include MME (Mobility Management Entity) 2031, HSS (Home Subscriber Server) 2032, other MMEs 2033, SGW (Serving Gateway) 2034, PGW (Packet Data Network Gateway) 2035, and PCRF (Policy and Charging Rules Function) 2036, etc. Optionally, MME2031 is the control node that handles signaling between UE201 and EPC203, providing bearer and connection management. HSS2032 is used to provide registers to manage functions such as the Home Location Register (not shown in the figure) and stores user-specific information such as service characteristics and data rates. All user data can be sent through SGW2034. PGW2035 can provide UE 201 IP address allocation and other functions. PCRF2036 is the policy and charging control decision point for service data flow and IP bearer resources. It selects and provides available policy and charging control decisions for the policy and charging enforcement function unit (not shown in the figure).

[0103] IP services 204 may include the Internet, intranet, IMS (IP Multimedia Subsystem), or other IP services.

[0104] Although the above description uses the LTE system as an example, those skilled in the art should know that this application is not only applicable to the LTE system, but also to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, 5G and future new network systems (such as 6G), etc., without limitation.

[0105] Figure 3 is a schematic diagram of the hardware structure of a controller 140 provided in this application. The controller 140 includes a memory 1401 and a processor 1402. The memory 1401 is used to store program instructions, and the processor 1402 is used to call the program instructions in the memory 1401 to execute the steps performed by the controller in the first embodiment of the above method. The implementation principle and beneficial effects are similar, and will not be described again here.

[0106] Optionally, the controller further includes a communication interface 1403, which can be connected to the processor 1402 via a bus 1404. The processor 1402 can control the communication interface 1403 to implement the receiving and sending functions of the controller 140.

[0107] Figure 4 is a schematic diagram of the hardware structure of a network node 150 provided in this application. The network node 150 includes a memory 1501 and a processor 1502. The memory 1501 is used to store program instructions, and the processor 1502 is used to call the program instructions in the memory 1501 to execute the steps performed by the first node in the first embodiment of the above method. The implementation principle and beneficial effects are similar, and will not be described again here.

[0108] Optionally, the controller further includes a communication interface 1503, which can be connected to the processor 1502 via a bus 1504. The processor 1502 can control the communication interface 1503 to implement the receiving and sending functions of the network node 150.

[0109] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0110] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk, SSD), etc.

[0111] Based on the above-described mobile terminal hardware structure and communication network system, various embodiments of this application are proposed.

[0112] First Embodiment

[0113] Referring to Figure 5, which is a flowchart illustrating the image processing method according to the first embodiment, the image processing method of this application embodiment can be applied to a processing device, including steps S10 and S20:

[0114] S10, determine or obtain at least one reference image region based on the image block information of the current block;

[0115] S20, determine or obtain the prediction mode of the current block based on at least one reference image region.

[0116] In this embodiment, the processing device can be a smart terminal, such as a mobile phone or computer, or a server, such as a local server or a cloud server. This embodiment and this application primarily use a smart terminal as an example for illustration.

[0117] Optionally, the technical solution of this embodiment can be applied to fields such as image encoding and decoding, video encoding and decoding, hardware video encoding and decoding, dedicated circuit video encoding and decoding, and real-time video encoding and decoding.

[0118] Optionally, the processing device can acquire video image data from a video source, segment each frame of the video image data to obtain multiple image blocks, and determine the image block to be predicted among the multiple image blocks as the current block.

[0119] Optionally, the current block can be a block of image to be predicted in the encoder and / or decoder.

[0120] Optionally, the image patch information may include at least one of the following: patch size, patch area, image patch attributes, and image patch type, used to determine the intra-frame prediction mode or inter-frame prediction mode.

[0121] Optionally, the image block information may also include pixel-related information in the image block (e.g., the position of the pixel, the color information of the pixel (brightness information or chromaticity information)).

[0122] Optionally, the image patch used to determine the intra-frame prediction mode or inter-frame prediction mode can be a reconstructed or predicted image patch. The pixel used to determine the intra-frame prediction mode or inter-frame prediction mode can be a reconstructed or predicted pixel.

[0123] Optionally, the block size is the area of ​​the image block, which is the product of its length and width.

[0124] Optionally, image patch attributes may include the location of the image patch or image features.

[0125] Optionally, image features may include image texture.

[0126] Optionally, the image block type can be a natural image, a screen content image, etc.

[0127] Optionally, the reference image region can be an image region used to determine the prediction mode of the current block, or it can be an image region that is adjacent to or not adjacent to the current block.

[0128] Optionally, the reference image region may include at least one of pixels, image blocks, and image regions used to determine the prediction mode.

[0129] Optionally, the prediction mode can be an intra-frame prediction mode, such as an angle prediction mode.

[0130] Alternatively, the prediction mode can also be an inter-frame prediction mode, such as affine motion mode, sub-block-based temporal MV prediction (SbTMVP), merge mode, skip mode, etc.

[0131] Optionally, when the processing device is an encoder on the encoding side, referring to FIG6, the encoder receives video data input from a video source, such as receiving video images from a video source, determining the image to be predicted in the video images, dividing the image to be predicted into multiple image blocks (including luma blocks and chroma blocks), and using the temporal and / or spatial correlation between video images, performing prediction processing on each of the multiple image blocks. The prediction processing includes intra-frame prediction processing and / or inter-frame prediction processing, and the intra-frame prediction processing and / or inter-frame prediction processing each include multiple prediction modes. For these prediction modes, the encoder uses, for example, rate-distortion optimization to determine the prediction mode finally adopted for each of the multiple image blocks. For example, it calculates the rate-distortion cost corresponding to each prediction mode or the rate-distortion cost of combining several prediction methods to determine the minimum rate-distortion cost from multiple rate-distortion costs. The prediction mode or combination of prediction modes corresponding to the minimum rate-distortion cost is the prediction mode finally adopted for the image block.

[0132] Optionally, these prediction modes include intra-frame prediction modes and inter-frame prediction modes: the intra-frame prediction mode can be determined using the method of this embodiment, such as by the improved DIMD (Decoder side intra mode derivation) mode or the improved OBIC mode; the inter-frame prediction mode can also be determined using the method of this embodiment.

[0133] Alternatively, the basic principle of the improved DIMD mode can be to analyze the gradient magnitude and direction of adjacent pixels, while the basic principle of the improved OBIC can be to deduce the prediction mode of the current block by analyzing the use of intra-prediction modes of adjacent / non-adjacent image blocks. Since the improved OBIC is also a decoding-side intra-mode derivation, the improved OBIC is a special type of DIMD.

[0134] The encoder then uses a defined intra-frame prediction mode or inter-frame prediction mode to predict the predicted pixel value of the pixel to be predicted.

[0135] Optionally, the improved DIMD mode can be embodied by performing steps S10 and S20.

[0136] Optionally, the improved OBIC mode can be embodied by performing steps S10 and S20.

[0137] Optionally, the decoding-side inter-frame prediction mode derivation in the embodiments of this application can be embodied by performing steps S10 and S20.

[0138] Optionally, the event-based inter-frame coding mode derivation in the embodiments of this application can be embodied by performing steps S10 and S20.

[0139] Optionally, after determining the prediction mode of the image block to be predicted (i.e., the current block) based on steps S10 and S20, the pixel values ​​of all pixels to be predicted can be obtained using the determined prediction mode. The pixel values ​​sampled from the original image block corresponding to the image block to be predicted are subtracted from the predicted values ​​sampled from the corresponding pixels in the prediction block to obtain the residual values ​​of the pixel samples and the residual block corresponding to the original image block. The residual block then undergoes transformation and quantization processing, and is encoded by an entropy encoder. Finally, an encoded bitstream is formed. Additionally, the encoded bitstream may also include prediction parameters corresponding to the determined prediction mode and related auxiliary information (side information).

[0140] Optionally, the prediction parameters are entropy-encoded and then packed into the encoded bitstream.

[0141] Optionally, the relevant auxiliary information may include the intra-frame mode derivation at the decoding end in this embodiment or the indication information based on the intra-frame coding mode derivation of the event.

[0142] Optionally, the relevant auxiliary information may include the inter-frame mode derivation at the decoding end in this embodiment or the indication information based on the inter-frame coding mode derivation of the event.

[0143] Optionally, the transformed and quantized residual block is inversely quantized and inversely transformed, then added to the corresponding prediction data (e.g., the prediction block) obtained using the prediction mode to obtain the reconstructed block. After obtaining the reconstructed block, the loop filtering module performs loop filtering on the reconstructed block according to the filter control data to reduce distortion. After the loop filtering process, the reconstructed block after loop filtering is stored according to the encoded image buffer.

[0144] Optionally, the transformed and quantized residual block is added to the corresponding prediction block obtained using the prediction mode to obtain a reconstruction block, and then the reconstruction block is subjected to loop filtering to reduce distortion.

[0145] Optionally, when the processing device is a decoder on the decoding side, referring to Figure 7, after receiving the encoded bitstream, the decoder's entropy decoding unit parses and decodes the encoded bitstream to obtain transform coefficients. The decoder's inverse transform unit and inverse quantization unit perform inverse transform and inverse quantization processing on the transform coefficients to obtain residual blocks. Optionally, the decoder's entropy decoding unit parses and decodes the encoded bitstream to obtain prediction data, such as prediction parameters and related auxiliary information. The decoder's prediction processing unit uses the prediction parameters to perform prediction processing, thereby determining the prediction block corresponding to the residual block. Optionally, the prediction processing includes intra-frame prediction processing and / or inter-frame prediction processing, and the intra-frame prediction processing and / or inter-frame prediction processing each include one or more prediction modes combined. When the auxiliary information indicates the improved DIMD mode, OBIC mode, decoder-side inter-frame mode derivation mode, or event-based inter-frame coding mode derivation mode in this embodiment, the prediction mode corresponding to the residual block can be determined according to the above steps S10 and S20.

[0146] Optionally, if the prediction mode of the image block to be predicted (i.e., the current block) is determined based on steps S10 and S20, the pixel values ​​of all pixels to be predicted can be obtained using the determined prediction mode. After sampling all pixels in the image block to be predicted, the prediction result of the image block to be predicted is obtained.

[0147] Optionally, the obtained residual block and the corresponding prediction block (including the predicted luminance block and the predicted chrominance block) are added together to obtain the reconstructed block. The loop filtering unit in the decoder performs loop filtering on the reconstructed block according to the filter control data to reduce distortion and improve video quality. The reconstructed block after loop filtering is further combined into a decoded image and stored in the decoded image buffer or output as decoded video data.

[0148] Optionally, this embodiment can determine the prediction mode of the current block for at least four scenarios.

[0149] Optionally, in DIMD mode, at least one reference image region can be determined or obtained based on the image block information of the current block, and the prediction mode of the current block can be determined or obtained based on the at least one reference image region.

[0150] Optionally, in OBIC mode, at least one reference image region can be determined or obtained based on the image block information of the current block, and the prediction mode of the current block can be determined or obtained based on the at least one reference image region.

[0151] Optionally, in the inter-frame mode derivation mode at the decoding end, at least one reference image region can be determined or obtained based on the image block information of the current block, and the prediction mode of the current block can be determined or obtained based on the at least one reference image region.

[0152] Optionally, in the event-based inter-frame coding mode derivation mode, at least one reference image region can be determined or obtained based on the image block information of the current block, and the prediction mode of the current block can be determined or obtained based on the at least one reference image region.

[0153] In this application, the intra-frame mode derivation mode (DIMD mode, OBIC mode) is used as an example for illustration. However, the technical solutions mentioned in this application can also be used in inter-frame mode derivation mode (e.g., decoding-end inter-frame mode derivation mode, event-based inter-frame coding mode derivation).

[0154] Optionally, the processing device may first determine the image block information of the current block, and select a reference image region based on the image block information. For example, when the image block information meets certain conditions, it may select an image region, image block, or pixel adjacent to the current block, and use the selected image region adjacent to the current block as the reference image region, or use the image region containing the image block adjacent to the current block as the reference image region. Alternatively, it may use the image region containing the pixel adjacent to the current block as the reference image region.

[0155] Optionally, after determining at least one reference image region, prediction modes associated with the at least one reference image region, such as angle prediction modes, can be determined, and at least one prediction mode can be selected from them as the prediction mode for the current block. The prediction mode associated with the at least one reference image region can be the prediction mode used by the reference image region, the prediction mode used by the image block in the reference image region, or the prediction mode used by the pixel in the reference image region, etc., and is not limited here.

[0156] Optionally, the basic principle of the improved DIMD mode can be to deduce the intra-prediction mode of the current block by analyzing the gradient magnitude and direction of adjacent pixels, or the usage of intra-prediction modes in adjacent / non-adjacent image blocks. Its specific implementation process can include determining the gradient magnitude and direction of pixels in the DIMD template, determining the gradient histogram or statistical results of the DIMD template with respect to the intra-prediction direction, and determining the intra-prediction direction of the block to be predicted based on the gradient histogram.

[0157] For example, as shown in Figure 8, DIMD templates adjacent to the block to be predicted (i.e., the current block) are determined above and to the left of the block to be predicted. For instance, three lines composed of various image blocks are located to the left and above the block to be predicted. Pixels in the middle pixel line (e.g., pixel x 4) are taken as pixels for gradient calculation. By calculating the gradient direction of multiple pixels in the pixel line, as well as the magnitudes of the horizontal and vertical gradients, the gradient direction and corresponding gradient magnitude values ​​of multiple pixels can be obtained. The gradient magnitude value is the sum of the absolute values ​​of the horizontal and vertical gradient magnitudes. If the gradient magnitude values ​​with the same gradient direction in multiple pixels are added together, the sum of the gradient magnitude values ​​corresponding to that gradient direction can be obtained. A histogram of gradient magnitude values ​​for different gradient directions of multiple pixels can be constructed, and the prediction direction perpendicular to the gradient direction of the maximum gradient magnitude value can be used as the prediction direction of the intra-frame prediction mode for the current block.

[0158] Optionally, the horizontal gradient Gx and vertical gradient Gy can be calculated using the 3x3 horizontal Sober operator and the vertical Sober operator, respectively. For example, the horizontal gradient Gx and vertical gradient Gy for a pixel x 4 in a pixel line can be calculated according to the following formulas (I) and (II).

[0159] Optionally, A can be a matrix consisting of nine pixels, centered at pixel x4, and including the pixels above x4 (x1), to the left (x3), below x4 (x7), to the right (x5), to the top left (x0), to the bottom left (x6), to the top right (x2), and to the bottom right (x8), as shown in Formula (III) below.

[0160] Optionally, the magnitude of the gradient G is the sum of the absolute values ​​of the horizontal and vertical gradients, and its calculation formula is shown in Formula (IV). G = |Gx| + |Gy| Formula (IV);

[0161] Alternatively, the gradient direction corresponding to a pixel can be calculated using arctan(Gx / Gy).

[0162] Optionally, since each gradient direction corresponds to a specific gradient direction range, and each gradient direction range corresponds to the prediction direction of an intra-frame prediction mode, for multiple pixels in a pixel line, the gradient magnitude values ​​with the same gradient direction range among the multiple pixels can be added together to obtain the sum of the gradient magnitude values ​​corresponding to that gradient direction range.

[0163] Optionally, the sum of the gradient magnitude values ​​of the prediction directions for the corresponding intra-prediction modes can also be obtained. Referring to Figure 9, the sum of the gradient magnitude values ​​includes the gradient magnitudes corresponding to the prediction directions of each intra-prediction mode.

[0164] Optionally, the basic principle of the improved OBIC mode can be to determine the intra prediction mode of the current block by analyzing the usage of intra prediction modes by adjacent and / or non-adjacent coded blocks. Its specific implementation process may include determining the area magnitude values ​​and intra prediction directions of multiple image blocks, determining the area magnitude histogram or statistical results of the multiple image blocks with respect to the intra prediction directions, and determining the intra prediction direction of the block to be predicted based on the area magnitude histogram or statistical results.

[0165] Optionally, the intra-prediction mode usage of multiple image blocks may include first determining at least one coded region or at least one decoded region, and then determining the intra-prediction mode usage of coded blocks within that coded region. Alternatively, it may involve determining the intra-prediction mode usage of decoded blocks within that decoded region. For example, as shown in FIG10, the coded region contains coded blocks 4 and 6 adjacent to the block to be predicted, and coded blocks 1, 2, 3, 5, 7, and 8 not adjacent to the block to be predicted. As shown in FIG11, the decoded region contains decoded blocks 4 and 6 adjacent to the block to be predicted, and decoded blocks 1, 2, 3, 5, 7, and 8 not adjacent to the block to be predicted.

[0166] In this embodiment, during the prediction stage of video encoding and / or decoding, such as intra-frame prediction, the prediction mode can be determined directly based on at least one reference image region determined from the image information of the current block. This takes into account the relationship between the reference image region and the current block, making the determined prediction mode more accurate. In other words, the accuracy of deriving the prediction mode is improved during video encoding and / or decoding, thereby improving the prediction effect for the current block, such as improving prediction accuracy, and thus improving the prediction efficiency during video encoding and / or decoding.

[0167] Second Embodiment

[0168] Based on the first embodiment, a second embodiment is proposed.

[0169] In this embodiment, step S10 includes at least one of the following methods one to six.

[0170] Method 1: Determine or obtain at least one reference image region based on at least one of the following: width, height, block size, block area, image block attributes, and image block type of the current block.

[0171] Optionally, at least one reference image region can be determined or obtained based on the width and height of the current block.

[0172] Optionally, for DIMD mode, the number of pixels, image blocks, or image regions that need to be expanded in the first image region of the DIMD template can be determined based on at least one of the current block's width, height, block size, block area, image block attributes, and image block type. The first image region in the DIMD template can be the image region where the pixel line is located, as shown in Figure 8.

[0173] Optionally, the number of pixel lines in the DIMD template can be determined based on the block size or the sum of the width and height of the current block. For example, by expanding from one pixel line in Figure 8 to four pixel lines in Figure 12, a reference image region containing four pixel lines is obtained. Then, the pixel gradients of the four pixel lines in the reference image region are calculated, and the prediction direction of the prediction mode of the current block is determined based on the maximum gradient magnitude value among the gradient magnitude values ​​of multiple pixels in different directions (e.g., the gradient magnitude value of pixel A). The prediction mode of the current block is determined based on the determined prediction direction.

[0174] Alternatively, the number of pixel lines can be determined based on a first mapping table between block size and the number of pixel lines.

[0175] Optionally, the first lookup table can be as shown in Table 1 below:

[0176] Table 1

[0177] Optionally, the number of pixel lines can be determined based on a second mapping table that includes the sum of width and height and the number of pixel lines. Optionally, the first lookup table can be as shown in Table 2 below:

[0178] Table 2

[0179] Optionally, the number of pixel lines can be determined or obtained based on the block size or the sum of the width and height of the current block and a mapping table, and at least one reference image region can be determined or obtained based on the number of pixel lines. For example, the reference image region is the image region containing the number of pixel lines.

[0180] Optionally, a reference image region or at least one of pixels, image blocks, and image regions within a reference image region can be determined based on the block size or the sum of the width and height of the current block.

[0181] Optionally, the image region within the reference region or reference image region can be an encoded region or a decoded region. Optionally, the reference image region or image region within the reference image region can be determined based on the block size of the current block and the containing block size and a third mapping table.

[0182] Alternatively, the third mapping table can be as shown in Table 3 below:

[0183] Table 3

[0184] Optionally, the reference image region or the image region within the reference region can be an encoded region or a decoded region. Optionally, the reference region or the image region within the reference image region can be determined based on the sum of the width and height of the current block and a fourth mapping table containing the sum of the width and height and the image region.

[0185] Alternatively, the fourth mapping table can be as shown in Table 4 below:

[0186] Table 4

[0187] Optionally, the encoded or decoded region can be determined by the position of the top-left pixel, the height of the encoded or decoded region, and its width. For example, the position of the top-left pixel can be the position N (N greater than 1) times the height of the image block above the top-left corner of the current block and N times the width of the image block to the left. The width of the encoded or decoded region is an integer multiple of the width of the current block, and the height of the encoded or decoded region is an integer multiple of the height of the current block.

[0188] Optionally, the number of pixels in the pixel line used to count gradient direction and gradient magnitude values ​​can be determined based on the block size or the sum of the width and height of the current block, and then the size of the reference region or the image region within the reference region can be determined based on the number of pixels counted. For example, the number of pixels in the reference region or the image region within the reference region is equal to the number of pixels used to count gradient direction and gradient magnitude values.

[0189] Optionally, the number of pixels in the reference region or the image region of the reference region may also be greater than the number of pixels used to count the gradient direction and gradient magnitude values. In this embodiment, the number of pixels in the pixel line used to count the gradient direction and gradient magnitude values ​​can be determined based on the block size or the sum of the width and height of the current block. Then, when constructing a histogram of gradient magnitude values, if the number of pixels counted reaches the aforementioned number of pixels used for counting, the pixel counting process is stopped. Optionally, the pixels used for counting may be pixels in the first gradient direction or pixels in the first prediction direction.

[0190] Alternatively, the number of pixels used for statistical gradient direction and gradient magnitude can be determined based on a fifth mapping table that includes the mapping relationship between the block size and the number of pixels used for statistical gradient direction and gradient magnitude values.

[0191] Alternatively, the fifth mapping table can be as shown in Table 5 below:

[0192] Table 5

[0193] Alternatively, the number of pixels used to count gradient direction and gradient magnitude can be determined based on a sixth mapping table containing a mapping relationship between the sum of width and height and the number of pixels used to count gradient direction and gradient magnitude values.

[0194] Alternatively, the sixth mapping table can be as shown in Table 6 below:

[0195] Table 6

[0196] Optionally, the number of image blocks used for statistics in the pixel line can be determined based on the block size or the sum of the width and height of the current block, and then the size of an image region in at least one reference image region can be determined or obtained based on the statistically analyzed image blocks. For example, the number of image blocks in the reference region or the image region in the reference region is equal to the number of image blocks used for statistics.

[0197] Optionally, the number of image blocks in the reference region or the image region of the reference region may be greater than the number of image blocks used for statistics. In this embodiment, the number of image blocks used for statistics in at least one reference image region can be determined based on the block size or the sum of the width and height of the current block. Then, when constructing a histogram of area amplitude values, if the number of image blocks used for statistics reaches the aforementioned number of image blocks used for statistics, the processing of image block statistics is stopped. Optionally, the image blocks used for statistics may be image blocks adopting a first prediction direction.

[0198] Alternatively, the number of image blocks used for statistics can be determined based on a seventh mapping table that includes the mapping relationship between block size and the number of image blocks used for statistics.

[0199] Alternatively, the seventh mapping table can be as shown in Table 7 below:

[0200] Table 7

[0201] Optionally, the number of image blocks used for statistics can be determined based on an eighth mapping table that contains a mapping relationship between the sum of width and height and the number of image blocks used for statistics.

[0202] Alternatively, the eighth mapping table can be as shown in Table 8 below:

[0203] Table 8

[0204] Optionally, the image patch used for statistics is determined by pixels in the reference area.

[0205] Optionally, the pixels in the reference area are first determined, and then the coordinate position and size of the image block where the pixel is located are determined. The coordinate position of the image block is the coordinate position of the upper left corner pixel of the image block, which is used to determine that the pixels of the adjacent image blocks are located in the area above the adjacent image block, the area to the left of the adjacent image block, or the area to the upper left of the adjacent image block.

[0206] Optionally, the horizontal offset of the pixel's coordinates from the coordinates of the image block is 0 to 2x the width of the image block, and the vertical offset of the pixel's coordinates from the coordinates of the image block is 0 to 2x the height of the image block.

[0207] Optionally, the horizontal and vertical offsets can be preset.

[0208] Optionally, when a pixel used to determine an adjacent image block is located in the upper region adjacent to the image block, the vertical offset of the pixel's coordinates from the image block's coordinates is -1 or an integer multiple of -0.5x the image block height, and the horizontal offset ranges from 0 to 2x the image block width. When a pixel used to determine an adjacent image block is located in the left region adjacent to the image block, the horizontal offset of the pixel's coordinates from the image block's coordinates is -1 or an integer multiple of -0.5x the image block width, and the vertical offset ranges from 0 to 2x the image block height. The aforementioned pixels determine or obtain the regions where adjacent image blocks are adjacent to each other and where the regions formed by these adjacent image blocks are adjacent to the current block.

[0209] Optionally, for OBIC mode, a similar approach to DIMD mode can be used. For example, the number of image blocks in at least one reference image region or the number of image blocks in an image region used for statistics can be determined based on the block size or the sum of the width and height of the current block. For instance, the number of image blocks used for statistics in at least one reference image region can be determined based on a mapping table containing the mapping relationship between block size or the sum of the width and height and the image block area. Then, when constructing a histogram of area amplitude values, if the number of image blocks found reaches the number of image blocks used for statistics, the image block search process stops.

[0210] Optionally, the image patch used for statistics can be an image patch with a first prediction direction.

[0211] Optionally, X1~X3, X1'~X3', X1”~X3”, and X1”'~X3”' mentioned above are integers.

[0212] Optionally, the block size or the sum of the width and height of the current block can be determined, and at least one reference image region or at least one image region among at least one reference image region can be determined or obtained based on the size or the sum of the width and height. For example, a mapping table containing the mapping relationship between the size or the sum of the width and height of the current block and at least one reference image region can be used to determine or obtain at least one reference image region.

[0213] Optionally, for DIMD mode, at least one reference image region or at least one image region among at least one reference image region can be determined or obtained based on the size or the sum of the width and height of the current block. Furthermore, the positions of pixel lines in the DIMD template are determined based on the determined or obtained at least one reference image region or at least one image region among at least one reference image region, and a histogram of gradient magnitude values ​​is determined based on the pixels in the pixel lines. For OBIC mode, at least one reference image region or at least one image region among at least one reference image region can be determined or obtained based on the block area of ​​the current block. For example, this can be determined based on the block size and a mapping table containing the mapping relationship between the block size and image blocks or image regions. Alternatively, it can be determined based on the sum of the width and height and a mapping table containing the mapping relationship between the sum of the width and height and image blocks or image regions.

[0214] Optionally, for DIMD mode, the number of pixels used to statistically determine the gradient direction and gradient magnitude values ​​can be determined or obtained based on the size or the sum of the width and height of the current block. Furthermore, the positions of pixel lines in the DIMD template are determined based on at least one determined or obtained reference image region or at least one image region among at least one reference image region, and a histogram of gradient magnitude values ​​is determined based on the pixels in the pixel lines. For OBIC mode, at least one reference image region or at least one image region among at least one reference image region can be determined or obtained based on the block area of ​​the current block. For example, this can be determined based on the block size and a mapping table containing the mapping relationship between the block size and image blocks or image regions. Alternatively, it can be determined based on the sum of the width and height and a mapping table containing the mapping relationship between the sum of the width and height and image blocks or image regions.

[0215] Optionally, image block attributes of the current block can be determined, such as the position or image features of the current block (e.g., the image texture of the current block). Optionally, the position or image features of the current block can be determined, and at least one reference image region or at least one reference area image region can be determined based on the position or image features of the current block. For example, if the current block is located in a first position region, the size of at least one reference region or at least one reference area image region is set to be less than a first threshold; if the current block is located in a second position region, the size of at least one reference region or at least one reference area image region is set to be greater than the first threshold. As another example, if the current block is located in a first position region, the position of at least one reference region or at least one reference area image region is set to be in a third position region; if the current block is located in a second position region, the position of at least one reference region or at least one reference area is set to be in a fourth position region.

[0216] Optionally, when the image texture corresponding to the current block is consistent with the preset image texture, pixels, image blocks and / or image regions adjacent to the current block can be selected as pixels, image blocks and / or image regions in the reference image region.

[0217] Optionally, when determining or obtaining at least one reference image region based on the position or image features of the current block, it can also be determined according to other rules, such as determining according to the specified input instructions, such as selecting a reference image region within a certain image range.

[0218] Optionally, the image block type of the current block can be determined. For example, the image block type of the current block can be a natural image or a screen content image. Different reference image regions can be selected for different image block types. For example, when the image block type of the current block is a natural image, pixels, image blocks, and / or image regions adjacent to the current block are selected as pixels, image blocks, and / or image regions in the reference image region. When the image block type of the current block is a screen content image, pixels, image blocks, and / or image regions not adjacent to the current block are selected as pixels, image blocks, and / or image regions in the reference image region.

[0219] Optionally, by determining or obtaining at least one reference image region based on at least one of the current block's block size, block area, image block attributes, and image block type, it is ensured that the determined or obtained reference image region is closely related to the current block, thereby ensuring that the prediction mode determined or obtained based on at least one reference image region is more accurate and improving the prediction effect of prediction processing for the current block.

[0220] Method 2: Determine or obtain at least one reference image region based on image block information from at least one of the following: neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, and default blocks corresponding to the current block.

[0221] Optionally, based on image block information of at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, and default blocks corresponding to the current block, at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, and default blocks corresponding to the current block is used as at least one reference image region.

[0222] Optionally, the default block can be a pre-set block, such as a block with typical pixel characteristics pre-set by the encoder and / or decoder.

[0223] Optionally, a neighboring block can be a block adjacent to the current block, and can be a block that has already been predicted or reconstructed.

[0224] Optionally, a non-neighbor block can be a block that is not adjacent to the current block, and can be a block that has already been predicted or reconstructed.

[0225] Optionally, the co-position block can be an image block in the co-position image that has the same position and size as the current block. Optionally, the co-position image can be the image in the reference image that is closest to the current image in time.

[0226] Optionally, the temporal block can be a block that is distinguished in the time domain, such as an image block in the previous frame. For example, if there is video data containing three frames of images, the first frame is played in the first second, the second frame is played in the second second, and the third frame is played in the third second, if the image block predicted at the current moment is an image block after dividing the second frame, then the temporal block can be determined to be the image block corresponding to it in the other frames of images besides the second frame.

[0227] Optionally, the image block information of neighboring blocks may include at least one of the following: block size, block area, image block attributes, and image block type. The block size of a neighboring block includes its width and height. The image block attributes of a neighboring block include its position and its image texture. The image block type of a neighboring block may include natural images or screen content images, etc.

[0228] Optionally, the image block information of non-neighboring blocks may include at least one of the following: block size, block area, image block attributes, and image block type. The block size of a non-neighboring block includes its width and height. The image block attributes of a non-neighboring block include its location and its image texture. The image block type of a non-neighboring block may include natural images or screen content images, etc.

[0229] Optionally, the image block information of the corresponding block may include at least one of the following: block size, block area, image block attributes, and image block type. The block size of the corresponding block includes its width and height. The image block attributes of the corresponding block include its position and its image texture. The image block type of the corresponding block may include a natural image or a screen content image, etc.

[0230] Optionally, the image block information of the temporal block may include at least one of the following: block size, block area, image block attributes, and image block type. The block size of the temporal block includes the width and height of the temporal block. The image block attributes of the temporal block include the position of the temporal block and the image texture of the temporal block. The image block type of the temporal block may include natural images or screen content images, etc.

[0231] Optionally, the image block information of the default block may include at least one of the following: block size, block area, image block attributes, and image block type. The block size of the default block includes its width and height. The image block attributes of the default block include its position and its image texture. The image block type of the default block may include natural images or screen content images, etc.

[0232] Optionally, the specific implementation process of Method 2 can refer to Method 1, that is, the current block in Method 1 can be replaced with a neighboring block, a non-neighboring block, a co-occurring block, a temporal block or a default block, which will not be repeated here.

[0233] Optionally, at least one pixel can be obtained from at least one of the default block, neighboring block, non-neighboring block, co-located block and temporal block corresponding to the current block, and the obtained pixel can be used as the fifth pixel, and the fifth pixel can be used as the pixel for calculating the gradient direction or gradient magnitude value.

[0234] Optionally, at least one image block can be obtained from at least one of the default block, neighboring block, non-neighboring block, co-located block and time domain block corresponding to the current block, and the obtained image block can be used as the fifth image block and the fifth image block can be used as the image block for calculating the area amplitude value.

[0235] Optionally, at least one image region can be obtained from at least one of the default block, neighboring block, non-neighboring block, co-located block and time domain block corresponding to the current block, and the obtained image region can be used as the fifth image region and the fifth image region can be used as the image region for calculating the area amplitude value.

[0236] Optionally, a pixel may be selected as the fifth pixel from at least one corresponding neighboring block and / or non-neighboring block, and / or image block corresponding to the motion vector, and / or image block corresponding to the block vector among the default block, neighboring block, non-neighboring block, co-located block and temporal block, or an image block may be selected as the fifth image block, or an image region may be selected as the fifth image region.

[0237] Optionally, by determining or obtaining at least one reference image region based on image block information from at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, and default blocks corresponding to the current block, it is ensured that the determined or obtained reference image region is closely related to the current block, thereby ensuring that the prediction mode determined or obtained based on at least one reference image region is more accurate and improving the prediction effect of prediction processing for the current block.

[0238] Method 3: Based on the image block information of the first component image block of the current block, determine or obtain at least one reference image region of the second component image block of the current block;

[0239] Optionally, the first component image block can be an image block of the chroma component or an image block of the luminance component.

[0240] Optionally, the second component image block can be an image block of the chroma component or an image block of the luminance component.

[0241] Optionally, the first component image block and the second component image block are image blocks of different components.

[0242] Optionally, at least one reference image region of the second component image block can be determined or obtained based on at least one of the block size, block area, image block attributes, and image block type of the first component image block.

[0243] Optionally, at least one pixel, at least one image block, or at least one image region may be selected or specified in the first component image block, and the selected at least one pixel, at least one image block, or at least one image region may be used as at least one pixel, at least one image block, or at least one image region in at least one reference image region of the second component image block.

[0244] Optionally, by determining or obtaining at least one reference image region of the second component image block of the current block based on the image block information of the first component image block of the current block, it is ensured that the determined or obtained reference image region is closely related to the current block, thereby ensuring that the prediction mode determined or obtained based on at least one reference image region is more accurate and improving the prediction effect of prediction processing for the current block.

[0245] Method 4: Determine or obtain at least one reference image region based on the candidate motion vector or candidate block vector of the current block to determine or obtain the image block information of the candidate block;

[0246] Optionally, candidate blocks can be determined or obtained sequentially based on multiple candidate motion vectors or multiple candidate block vectors in the candidate list of the current block.

[0247] Optionally, at least one of the following can be determined: block size, block area, image block attributes, and image block type of at least one candidate block. At least one reference image region is determined or obtained based on at least one of the following: block size, block area, image block attributes, and image block type of at least one candidate block. For example, at least one of the following can be used as at least one reference image region: block size, block area, image block attributes, and image block type of at least one candidate block.

[0248] Optionally, the image block attributes of the candidate block include the position of the candidate block, and the image block attributes include the image texture of the candidate block. The image block type of the candidate block includes natural images or screen content images, etc.

[0249] Optionally, the specific implementation process of determining or obtaining at least one reference image region based on at least one of the candidate block's block size, block area, image block attributes, and image block type can refer to the scheme in Method 1 above, that is, simply replacing the current block in Method 1 with the candidate block, which will not be repeated here.

[0250] Optionally, by determining or obtaining image block information of candidate blocks based on the block vector and / or motion vector of the current block, at least one reference image region is determined or obtained. This ensures that the determined or obtained reference image region is closely related to the current block, thereby ensuring that the prediction mode determined or obtained based on at least one reference image region is more accurate and improving the prediction effect of prediction processing for the current block.

[0251] Method 5: If the current block satisfies the first condition, then at least one reference image region is determined or obtained according to the first calculation strategy;

[0252] Optionally, the first calculation strategy can be any one of methods one to four: that is, the first calculation strategy can be to determine or obtain at least one reference image region based on at least one of the block size, block area, image block attributes and image block type of the current block; it can be to determine or obtain at least one reference image region based on the image block information of at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks and default blocks corresponding to the current block; it can be to determine or obtain at least one reference image region based on the image block information of the first component image block of the current block; it can be to determine or obtain at least one reference image region based on the image block information of the candidate blocks determined or obtained based on the block vector and / or motion vector of the current block.

[0253] Optionally, the first condition can be a pre-set condition that can be set according to user needs, and the first condition is not fixed and can be adaptively adjusted according to different scenarios.

[0254] Optionally, if the processing device detects that the current block meets the first condition, it can select a first calculation strategy to perform calculations to determine or obtain at least one reference image region.

[0255] Optionally, the current block satisfies the first condition, including at least one of the following:

[0256] The width of the current block is greater than or equal to the first preset width threshold;

[0257] The height of the current block is greater than or equal to the first preset height threshold;

[0258] The area of ​​the current block is greater than or equal to the first preset area threshold.

[0259] Optionally, the first preset width threshold can be a pre-set width threshold for an image block, such as 128.

[0260] Optionally, the first preset height threshold can be a pre-set height threshold for an image block, such as 128.

[0261] Optionally, the first preset area threshold can be a pre-set area threshold for an image block.

[0262] Optionally, when the processing device detects that the current block meets the first condition, it may select a first calculation strategy to perform calculations to determine or obtain at least one reference image region.

[0263] Optionally, if there are two image blocks, namely a second image block and a third image block, and if the block size of the second image block is larger than the block size of the third image block, then the number of pixels included in the reference image region of the second image block is determined to be greater than the number of pixels included in the reference image region of the third image block; or, the number of image blocks included in the reference image region of the second image block is determined to be greater than the number of image blocks included in the reference image region of the third image block; or, the number of image regions included in the reference image region of the second image block is determined to be greater than the number of image regions included in the reference image region of the third image block. Optionally, the current block can be either the second image block or the third image block.

[0264] Optionally, this embodiment may determine more pixels, image blocks, or image regions for determining the prediction mode for image blocks with larger block sizes or areas.

[0265] Optionally, this embodiment can select different numbers of pixels, image blocks, or image regions for determining the prediction mode for image blocks of different sizes. By associating the block size (e.g., the width and height of the current block) and / or the area of ​​the current block with the number of pixels, and / or the number of image blocks, and / or the number of image regions in the reference image region, it can be ensured that the prediction accuracy of the DIMD mode or OBIC mode is not affected by changes in the size or area of ​​the current block, thereby improving the prediction effect of subsequent prediction processing for the current block.

[0266] Method 6: If the current block does not meet the first condition, then at least one reference image region is determined or obtained according to the second calculation strategy.

[0267] Optionally, the first calculation strategy in method five can be different from the second calculation strategy in method six.

[0268] Optionally, the second calculation strategy can be any one of the methods from one to four, excluding the first calculation strategy: that is, the second calculation strategy can be to determine or obtain at least one reference image region based on at least one of the block size, block area, image block attributes, and image block type of the current block; it can be to determine or obtain at least one reference image region based on the image block information of at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, and default blocks corresponding to the current block; it can be to determine or obtain at least one reference image region based on the image block information of the first component image block of the current block; or it can be to determine or obtain at least one reference image region based on the image block information of the candidate blocks determined or obtained based on the block vector and / or motion vector of the current block.

[0269] Optionally, the current block not satisfying the first condition may include at least one of the following:

[0270] The width of the current block is less than the first preset width threshold;

[0271] The height of the current block is less than the first preset height threshold;

[0272] The area of ​​the current block is less than the first preset area threshold.

[0273] Optionally, when the processing device detects that the width of the current block is less than a first preset width threshold, or that the height of the current block is less than a first preset height threshold, or that the area of ​​the current block is less than a first preset area threshold, it may select a second calculation strategy to perform calculations to determine or obtain at least one reference image region.

[0274] Optionally, by associating the block size (e.g., the width and height of the current block) and / or the area of ​​the current block with the number of pixels in the reference image region, and / or the number of image blocks, and / or the number of image regions, a second calculation strategy is adopted when the current block does not meet the first condition. At least one reference image region is determined or obtained, and the prediction mode of the current block is determined or obtained based on at least one reference image region. This ensures that the prediction accuracy of the DIMD mode or OBIC mode is not affected by changes in the size or area of ​​the current block, thereby improving the prediction effect of subsequent prediction processing for the current block.

[0275] Third Embodiment

[0276] Based on any of the above embodiments, a third embodiment is proposed.

[0277] In this embodiment, step S20 includes steps S21 and S22.

[0278] Step S21: Determine or obtain a first probability of occurrence of at least one predicted pattern based on pixels, image blocks or image regions in at least one reference image region;

[0279] Optionally, at least one reference image region in this embodiment may be the reference image region in the first embodiment, or at least one reference image region determined according to any one of methods one to six in the second embodiment.

[0280] Optionally, after determining at least one reference image region, the processing device may determine a first probability of occurrence of at least one prediction mode based on at least one reference image region, such as determining a first probability of occurrence of at least one angle prediction mode based on at least one reference image region.

[0281] Optionally, for any pixel in at least one reference image region, the horizontal gradient value and vertical gradient value of the pixel can be determined, and the sum of the horizontal gradient value and vertical gradient value of the same pixel can be used as the gradient magnitude value of that pixel. The gradient direction of the pixel can be determined, and the prediction direction of the pixel can be determined based on the gradient direction, such as the direction perpendicular to the gradient direction. Optionally, the prediction direction of at least one pixel in at least one reference image region can be determined, and the gradient magnitude values ​​of pixels belonging to the same prediction direction can be accumulated to obtain the sum of the gradient magnitude values ​​of the same prediction direction.

[0282] Optionally, when accumulating the gradient magnitude values ​​of pixels in the same prediction direction, different scaling weight factors can be set for different pixels belonging to the same prediction direction. The gradient magnitude values ​​of pixels in the same prediction direction are then weighted and summed according to the different scaling weight factors to obtain the sum of the gradient magnitude values ​​in the same prediction direction.

[0283] Optionally, a prediction pattern can correspond to a prediction direction.

[0284] Optionally, for any prediction direction, the sum of the gradient magnitude values ​​of that prediction direction can be used as the first probability of occurrence of the prediction mode in that prediction direction.

[0285] Optionally, for any image patch in at least one reference image region, the area of ​​the image patch and the prediction direction corresponding to the image patch can be determined. The area amplitude values ​​of image patches belonging to the same prediction direction can be accumulated to obtain the sum of the area amplitude values ​​for that prediction direction.

[0286] Optionally, the area amplitude value of the image patch can be determined based on the area of ​​the image patch. For example, the area of ​​the image patch can be directly used as the area amplitude value, or the area of ​​the image patch can be deformed, such as multiplying the area of ​​the image patch according to a preset scaling weight factor to obtain the area amplitude value.

[0287] Optionally, when the intra-prediction mode used to calculate the area amplitude value of the image patch determined in the reference image is one of DIMD, OBIC, or spatial geometric partitioning mode (SGPM), multiple intra-prediction modes exist simultaneously for that determined image patch. In this case, a scaling weight factor is determined based on the number of intra-prediction modes for the image patch. This results in an area amplitude value of S / n for each intra-prediction mode, where S is the area of ​​the image patch, and n is the number of intra-prediction modes used by the image patch or the number of angular prediction modes used by the image patch.

[0288] For example, if the image determined in the reference image for calculating the area amplitude value adopts DIMD mode, and DIMD mode is a fusion mode, the image patch can adopt up to 5 intra-frame angle prediction modes.

[0289] Optionally, when five intra-frame prediction modes are used, the corresponding area amplitude value for each of these five intra-frame prediction modes is S / 5. If the image determined in the reference image for calculating the area amplitude value adopts OBIC mode, and OBIC mode is a fusion mode, the image patch can use a maximum of five intra-frame angle prediction modes.

[0290] Optionally, when using 5 intra-prediction modes, the corresponding area amplitude value for each of these 5 intra-prediction modes is S / 5. If the image determined in the reference image for calculating the area amplitude value uses the SGPM mode, the image patch can use at most 2 intra-prediction modes, and the corresponding area amplitude value for each of these 2 intra-prediction modes is S / 2. Optionally, n can also be the total number of prediction modes used by the image patch. The total number of prediction modes mentioned above includes the number of non-intra-prediction modes and the number of intra-prediction modes.

[0291] Optionally, for any prediction direction, the sum of the area amplitude values ​​in that prediction direction can be used as the first probability of occurrence of the prediction pattern in that prediction direction.

[0292] Optionally, step S21 includes at least one of the following methods ten to sixteen.

[0293] Method 10: Based on pixels, image blocks, or image regions in at least one reference image region, determine or obtain a first amplitude value and a second amplitude value in the same prediction direction, and determine or obtain a first probability of occurrence based on the first amplitude value and the second amplitude value in the same prediction direction;

[0294] Optionally, the first amplitude value and the second amplitude value can be gradient amplitude values ​​determined according to the DIMD mode, or area amplitude values ​​determined according to the OBIC mode.

[0295] Optionally, the magnitude value corresponding to a pixel in at least one reference image region can be determined, and pixels belonging to the same prediction mode among multiple pixels can be determined. The gradient magnitude values ​​of pixels belonging to the same prediction mode are accumulated to obtain the first probability of occurrence of the prediction mode.

[0296] Optionally, different prediction modes correspond to different prediction directions. After determining the gradient magnitude values ​​of multiple pixels in at least one reference image region, pixels belonging to the same prediction direction can be identified, and at least two different pixels belonging to the same prediction direction can be identified. Optionally, the gradient magnitude values ​​of the at least two different pixels can be a first magnitude value and a second magnitude value.

[0297] Optionally, the first amplitude value and the second amplitude value in the same prediction direction can be accumulated to obtain the first probability of occurrence of the prediction pattern in that prediction direction.

[0298] Optionally, the second amplitude value can be deformed based on the first amplitude value in the same prediction direction, and the gradient amplitude values ​​of other pixels in the same prediction direction can be accumulated based on the deformed second amplitude value to obtain the first probability of occurrence of the prediction mode in the prediction direction.

[0299] Optionally, the deformation processing can be an accumulation process, or it can be done by adding or subtracting a fixed value from the second amplitude value.

[0300] For example, if at least one reference image region is a DIMD template as shown in Figure 8, the gradient magnitude value of each pixel within the pixel line in the DIMD template can be determined.

[0301] Optionally, for any pixel within a pixel line, a second pixel that is adjacent or not adjacent to that pixel can be determined. If the second pixel is not within the pixel line or is within another pixel line, then the gradient magnitude value of the second pixel can be calculated. When the prediction direction of the prediction mode of the second pixel is consistent with the prediction direction of the prediction mode of the pixel, the gradient magnitude value of the pixel and the gradient magnitude value of the second pixel can be determined to be a first magnitude value and a second magnitude value in the same prediction direction. After determining the first magnitude value and the second magnitude value in the same prediction direction, the first probability of occurrence of the prediction mode in that prediction direction is determined based on the first magnitude value and the second magnitude value.

[0302] Optionally, a first amplitude value and a second amplitude value for the same prediction direction can be determined or obtained based on an image block or image region in at least one reference image region according to the OBIC mode, and after determining the first amplitude value and the second amplitude value for the same prediction direction, a first probability of occurrence of the prediction mode for that prediction direction can be determined based on the first amplitude value and the second amplitude value.

[0303] Optionally, for the OBIC mode, taking image blocks as an example: if a first amplitude value is obtained through a first image block and a second amplitude value is obtained through a second image block, and both the first and second image blocks belong to the same prediction direction, then the first probability of occurrence of the prediction mode in that prediction direction is determined by the first amplitude value and the second amplitude value.

[0304] Optionally, the first amplitude value can be a gradient amplitude value determined according to the DIMD mode, and the second amplitude value can be an area amplitude value determined according to the OBIC mode. Alternatively, the first amplitude value can be an area amplitude value determined according to the OBIC mode, and the second amplitude value can be a gradient amplitude value determined according to the DIMD mode.

[0305] Optionally, by determining or obtaining a first amplitude value and a second amplitude value in the same prediction direction based on pixels, image blocks, or image regions in at least one reference image region, and then determining or obtaining a first occurrence probability based on these two amplitude values, and determining or obtaining the prediction mode of the current block based on the first occurrence probability, the first occurrence probability of the at least one prediction mode determined or obtained can be increased, thereby improving the accuracy of the determined prediction mode and improving the prediction effect of the prediction processing of the current block.

[0306] Method 11: Determine or obtain at least one first amplitude value based on a pixel, image block, or image region in at least one reference image region, and determine or obtain a first occurrence probability based on the first amplitude value and a scaling weight factor;

[0307] Optionally, the scaling weight factor can be a weight used to weight the magnitude value, or a scaling weight factor, or a weight ratio, etc.

[0308] Optionally, at least one gradient magnitude value can be determined or obtained based on the DIMD mode and at least one pixel, image block, or image region in the reference image region, and the at least one gradient magnitude value can be used as the first magnitude value. At least one area magnitude value can be determined or obtained based on the OBIC mode and at least one image block or image region in the reference image region, and the at least one area magnitude value can be used as the first magnitude value.

[0309] Optionally, for a pixel within at least one reference image region, the horizontal gradient value and the vertical gradient value of the pixel can be determined, and the sum of the horizontal gradient value and the vertical gradient value of the same pixel can be used as the gradient magnitude value of that pixel, i.e., the first magnitude value.

[0310] Optionally, after determining the first amplitude value of any pixel in at least one reference image region, it can be determined whether scaling processing of the pixel is required. If scaling processing is required, the scaling weight factor corresponding to the pixel can be determined, and the first amplitude value of the pixel can be scaled according to the scaling weight factor to obtain a new first amplitude value. For example, the scaling weight factor can be multiplied by the first amplitude value of the pixel to obtain a new first amplitude value. Then, the prediction direction corresponding to the pixel is determined, and the amplitude values ​​of all pixels belonging to the prediction direction are added together to obtain the first probability of occurrence of the prediction direction. For example, if there are pixels 1, 2, and 3 belonging to the same prediction direction prediction mode, and after determining the first amplitude value of pixel 3, the first amplitude value of pixel 3 is multiplied according to the scaling weight factor of pixel 3 to obtain a new first amplitude value. The gradient amplitude values ​​corresponding to pixel 1, the gradient amplitude values ​​corresponding to pixel 2, and the new first amplitude value corresponding to pixel 3 are accumulated to obtain the first probability of occurrence of the prediction mode in the prediction direction.

[0311] Optionally, the scaling weight factor for each pixel can be the same or different.

[0312] Optionally, for an image block or image region within at least one reference image region, an area amplitude value corresponding to the image block or image region can be determined, and the area amplitude value corresponding to at least one image block or image region can be used as a first amplitude value.

[0313] Optionally, it can be determined whether scaling processing is required for the first amplitude value corresponding to the image block or image region. If scaling processing is required, a scaling weight factor corresponding to the image block or image region can be obtained, and the first amplitude value can be scaled according to the scaling weight factor to obtain a new first amplitude value. All area amplitude values ​​belonging to the same prediction direction are then accumulated to obtain the first occurrence probability of the prediction mode in that prediction direction. Optionally, for example, the new first amplitude values ​​corresponding to at least one image block or image region belonging to the same prediction direction are accumulated. That is, the actual area amplitude values ​​of each image block in the same prediction direction are accumulated. Optionally, after calculating the area amplitude value of the image block, if the area amplitude value is scaled using a scaling weight factor to obtain a new first amplitude value, then the new first amplitude value is used as the actual area amplitude value. And / or, if the area amplitude value is not scaled using a scaling weight factor, then the area amplitude value is used as the actual area amplitude value, that is, the first amplitude value is directly used as the actual area amplitude value.

[0314] Optionally, the first probability of occurrence is determined or obtained based on the first amplitude value and the scaling weight factor determined according to the third amplitude value.

[0315] Optionally, the scaling weight factor corresponding to the first amplitude value can be determined or obtained based on the third amplitude value.

[0316] Optionally, at least one third amplitude value is determined or obtained based on pixels, image blocks or image regions in at least one reference image region, and a scaling weight factor is determined based on the third amplitude value. Then, a first probability of occurrence is determined or obtained based on the scaling weight factor and the first amplitude value.

[0317] Optionally, at least two amplitude values, namely a first amplitude value and a third amplitude value, can be determined or obtained based on pixels, image blocks or image regions in at least one reference image region.

[0318] Optionally, the scaling weight factor corresponding to the first amplitude value can be determined or obtained based on the third amplitude value. For example, when the third amplitude value is greater than a preset amplitude threshold, the scaling weight factor is determined to be 2. When the third amplitude value is less than the preset amplitude threshold, the scaling weight factor is determined to be 8.

[0319] Optionally, after determining the first amplitude value and the scaling weight factor corresponding to the first amplitude value based on the third amplitude value, the first amplitude value can be scaled according to the scaling weight factor to obtain a new first amplitude value, and the first probability of occurrence of the prediction pattern of the prediction direction can be determined based on the new first amplitude value.

[0320] Optionally, by determining or obtaining at least one first amplitude value based on pixels, image blocks, or image regions in at least one reference image region, obtaining a new first amplitude value based on the first amplitude value and a scaling weight factor, determining or obtaining a first probability of occurrence based on the new first amplitude value, and / or determining or obtaining the first probability of occurrence based on the first amplitude value and a scaling weight factor determined according to a third amplitude value, the first probability of occurrence of the determined or obtained at least one prediction pattern can be improved, thereby improving the accuracy of the determined prediction pattern and improving the prediction effect of the current block prediction processing.

[0321] Method 12: Determine or obtain a first probability of occurrence of the prediction direction of at least one prediction mode based on the gradient magnitude value and gradient direction of the gradient of a pixel in at least one reference image region.

[0322] Optionally, after determining or obtaining at least one reference image region based on the image block information of the current block, the processing device may employ a DIMD mode to determine the first probability of occurrence of the prediction direction of each prediction mode based on the pixels in the at least one reference image region.

[0323] Optionally, the image region containing the DIMD template can be determined within the reference image region, and at least one pixel line can be defined within the DIMD. For any pixel within the at least one pixel line, the horizontal and vertical gradient values ​​of that pixel can be calculated, and the horizontal and vertical gradient values ​​of the same pixel can be added together, for example, by adding the absolute values ​​of the horizontal and vertical gradients to obtain the gradient magnitude value of that pixel. Furthermore, the prediction direction of the prediction mode for that pixel can be determined based on the gradient direction of that pixel, for example, using the direction perpendicular to the gradient direction as the prediction direction of the prediction mode for that pixel.

[0324] Optionally, after calculating the gradient magnitude value of any pixel, the gradient magnitude value can be scaled according to the scaling weight factor corresponding to that pixel to obtain the actual gradient magnitude value of the pixel. Alternatively, after calculating the horizontal gradient value and / or vertical gradient value of a pixel, the scaling weight factor can be multiplied by the horizontal gradient value to obtain a new horizontal gradient value, and then the new horizontal gradient value can be added to the vertical gradient value of the pixel to obtain the gradient magnitude value of the pixel. Alternatively, the scaling weight factor can be multiplied by the vertical gradient value to obtain a new vertical gradient value, and then the new vertical gradient value can be added to the horizontal gradient value of the pixel to obtain the gradient magnitude value of the pixel.

[0325] Optionally, after determining the gradient magnitude value corresponding to a pixel in at least one reference image region, the prediction direction corresponding to that pixel can be determined, and different gradient magnitude values ​​belonging to the same prediction direction can be accumulated to obtain a first probability of occurrence of the prediction mode in that prediction direction. Optionally, when accumulating gradient magnitude values, if the gradient magnitude value of a pixel has been scaled according to a scaling weight factor, the scaled gradient magnitude value, i.e., the actual gradient magnitude value, is accumulated with the gradient magnitude values ​​of other pixels in the same prediction direction to obtain a first probability of occurrence of the prediction direction of that prediction mode.

[0326] Optionally, when accumulating the gradient magnitude values ​​of pixels in the same prediction direction to determine or obtain the first probability of occurrence of the prediction direction of the prediction mode, a probability threshold can be set. If it is detected that the probability of occurrence of the prediction direction of at least one prediction mode is greater than the probability threshold, the calculation of the gradient magnitude value of the pixel can be stopped. That is, the gradient magnitude value can be not calculated for all pixels in the reference image region, or for all pixels on the pixel line.

[0327] Optionally, for the overall process of DIMD mode, prediction modes for n prediction directions can be determined, and prediction processing can be performed based on the prediction modes for the n prediction directions. Alternatively, a threshold can be set. If the gradient magnitude value corresponding to the prediction direction of the n prediction modes is greater than the threshold, the calculation of the gradient magnitude value of the pixel can be stopped. n is an integer greater than 1.

[0328] Optionally, by determining or obtaining the first probability of occurrence of the prediction direction of at least one prediction mode based on the gradient magnitude value and gradient direction of pixels in at least one reference image region, the first probability of occurrence of the determined or obtained prediction mode can be improved, thereby improving the accuracy of the determined prediction mode and improving the prediction effect of the current block prediction processing.

[0329] Method 13: Based on the prediction pattern corresponding to an image patch or image region in at least one reference image region, and the area of ​​the image patch or image region, determine or obtain the first probability of occurrence of the prediction direction of at least one prediction pattern.

[0330] Optionally, after determining or obtaining at least one reference image region based on the image block information of the current block, the processing device may employ OBIC mode to determine the first occurrence probability of the prediction direction of each prediction mode based on the image block or image region in at least one reference image region.

[0331] Optionally, at least one image patch within a reference image region can be determined, such as an image patch adjacent to the current patch or an image patch not adjacent to the current patch. The prediction mode used by the image patch in the reference image region and the area of ​​the image patch can be determined. Optionally, the area of ​​the image patch can be directly used as the area magnitude value of the image patch. A scaling weight factor corresponding to the image patch can be determined, and the area of ​​the image patch is multiplied according to the scaling weight factor to obtain the area magnitude value of the image patch.

[0332] Optionally, the area amplitude values ​​of each image patch belonging to the same prediction direction can be accumulated to obtain the first probability of occurrence of the prediction direction of the prediction mode.

[0333] Optionally, the image region in at least one reference image region can also be processed in the same manner as the image block in at least one reference image region in order to determine or obtain the first probability of occurrence of the prediction direction of at least one prediction mode.

[0334] Optionally, when accumulating the area amplitude values ​​of image blocks or image regions with the same prediction direction to determine or obtain the probability of occurrence of the prediction direction of the prediction mode, an occurrence probability threshold can be set. If it is detected that the probability of occurrence of the prediction direction of at least one prediction mode is greater than the occurrence probability threshold, the calculation of the area amplitude values ​​of the image blocks or image regions can be stopped; that is, the area amplitude values ​​can be not calculated for all image blocks or image regions in the reference image region.

[0335] Optionally, during the overall process of OBIC mode, prediction modes for n prediction directions can be determined, and prediction processing can be performed based on the prediction modes for the n prediction directions. Alternatively, a threshold can be set. If the area amplitude value corresponding to the prediction direction of any of the n prediction modes is greater than the threshold, the calculation of the area amplitude value of the image block or image region can be stopped. n is an integer greater than 1.

[0336] Optionally, by determining or obtaining the first probability of occurrence of the prediction direction of at least one prediction mode based on the prediction mode corresponding to the image block or image region in at least one reference image region and the area of ​​the image block or image region, the first probability of occurrence of the determined or obtained prediction mode can be improved, thereby improving the accuracy of the determined prediction mode and improving the prediction effect of the current block prediction processing.

[0337] Method 14: Determine or obtain a first probability of occurrence based on at least one of the following: position, horizontal distance, vertical distance, and shortest distance between a pixel, image block, or image region in at least one reference image region and the current block;

[0338] Optionally, different strategies are employed based on the position of pixels, image blocks, or image regions in at least one reference image region relative to the current block to obtain a first probability of occurrence. For example, different scaling weight factors are used to calculate the magnitude value for pixels, image blocks, or image regions at different positions, thereby determining the first probability of occurrence of the prediction pattern in each prediction direction.

[0339] Optionally, different strategies are adopted based on the distance (i.e., horizontal distance, vertical distance, or shortest distance) between the image block or image region in at least one reference image region and the current block to determine or obtain the actual area amplitude value of the image block or image region. The area amplitude value of the image block or image region can be accumulated with the area amplitude values ​​of other image blocks or image regions in the same prediction direction to obtain the first probability of occurrence of the prediction pattern in the prediction direction.

[0340] Optionally, different strategies can be adopted based on the distance (i.e., horizontal distance, vertical distance, or shortest distance) between a pixel in at least one reference image region and the current block to determine or obtain the actual gradient magnitude value of the pixel. The actual gradient magnitude value of the pixel can be accumulated with the actual gradient magnitude values ​​of other pixels in the prediction mode of the same prediction direction to obtain the first probability of occurrence of the prediction mode in that prediction direction.

[0341] Optionally, if the distance between a pixel and the current block is greater than or equal to a first distance threshold (which may be a pre-set distance value), the actual gradient magnitude value of the pixel can be calculated using a first calculation magnitude strategy. And / or, if the distance between a pixel and the current block is less than the first distance threshold, the actual gradient magnitude value of the pixel can be calculated using a second calculation magnitude strategy.

[0342] Optionally, the first calculation magnitude strategy may be to determine a pixel that is n times the width and / or n times the width away from the edge of the current block, determine the area and intra-prediction mode of the encoded or decoded image block in which the pixel is located, and use the intra-prediction mode as a candidate intra-prediction mode for the block to be predicted and the area as the magnitude of the candidate intra-prediction mode for the block to be predicted.

[0343] Optionally, the second amplitude calculation strategy can be to divide the image region into multiple image sub-regions. For example, as shown in Figure 13, image region A and image region B are included, and image region B is divided into multiple image sub-regions 1 and multiple image sub-regions 2. Preferably, the size of the multiple image sub-regions is the same as that of the region to be predicted.

[0344] Preferably, a larger scaling factor is set for image sub-regions that are close to the block to be predicted (e.g., image sub-region 1), and a smaller scaling factor is set for image sub-regions that are far from the block to be predicted (e.g., image sub-region 2).

[0345] Alternatively, multiple image sub-regions can be divided into image sub-regions of fixed size (e.g., 4x4 image sub-regions in Figure 14).

[0346] Optionally, the size of an image sub-region closer to the image sub-region to be predicted (e.g., image sub-region 1) is smaller than the size of an image sub-region farther from the image sub-region to be predicted. For example, the size of an image sub-region in image sub-region 1 is 4x4, while the size of an image sub-region in image sub-region 2 is 8x8. Then, a pixel within the image sub-region (e.g., the pixel at the top-left corner of the image sub-region) is selected, and the intra-prediction mode of the encoded or decoded image block containing that pixel is determined, along with the magnitude of using that intra-prediction mode as a candidate intra-prediction mode for the image sub-region to be predicted and the area of ​​the image sub-region (e.g., 4x4 or 8x8) as the candidate intra-prediction mode magnitude for the image sub-region to be predicted.

[0347] Optionally, the closer the image block is to the current block, the higher the similarity between the current block and the current block. Compared to pixels that are farther away, the intra prediction mode of the closer image block is used as the intra prediction mode of the current block for intra prediction, which will have a better effect.

[0348] Optionally, in DIMD mode, the gradient magnitude of a pixel is the magnitude used to determine the intra-prediction mode of the current block. Since pixels at different distances / positions have different similarity probabilities to the current block, different strategies for calculating the magnitude are needed for pixels at different distances / positions. For example, a higher-precision calculation strategy is used for pixels that are closer, and a lower-precision calculation strategy is used for pixels that are farther away. In this way, encoding and decoding performance can be improved without significantly increasing the computational complexity during video encoding and decoding.

[0349] Optionally, in OBIC mode, the area of ​​an image patch is used as the magnitude for determining the intra-prediction mode of the current patch. Therefore, only the size of the image patch area affects the determination of the intra-prediction mode. Applying the same calculation magnitude strategy to image patches of the same size but different locations often results in lower accuracy of the determined intra-prediction mode. Based on this, a higher-precision calculation strategy is applied to image patches that are closer together, while a lower-precision strategy is applied to image patches that are farther apart. In this way, encoding and decoding performance can be improved without significantly increasing the computational complexity during video encoding and decoding.

[0350] Optionally, by determining or obtaining a first occurrence probability based on at least one of the position, horizontal distance, vertical distance, and shortest distance between a pixel, image block, or image region in at least one reference image region and the current block, the validity of the determined or obtained first occurrence probability can be ensured, thereby improving the effectiveness of subsequently determining or obtaining the prediction pattern based on the first occurrence probability and improving the prediction effect of prediction based on the prediction pattern.

[0351] Method 15: Determine or obtain the first occurrence probability based on the pixel type of the pixel, the image block type of the image block, and the image region type of the image region;

[0352] Optionally, the image region type may include the image region to the left of the current block, the image region above the current block, the image region above the left of the current block, the image region in the same coding tree unit as the current block, and the image region not in the same coding tree unit as the current block.

[0353] Optionally, the image block type may include the image block to the left of the current block, the image block above the current block, the image block to the upper left of the current block, the image block in the same coding tree unit as the current block, and the image block not in the same coding tree unit as the current block, etc.

[0354] Optionally, the pixel type may include pixels to the left of the current block, pixels above the current block, pixels above the left of the current block, pixels in the same coding tree unit as the current block, pixels not in the same coding tree unit as the current block, etc.

[0355] Optionally, for DIMD mode, when determining the pixel lines in the reference image region and calculating the gradient magnitude value for any pixel within those lines, it can be determined whether scaling of the gradient magnitude value for that pixel is required based on the pixel type (e.g., scaling the gradient magnitude value when the pixel type is a pixel to the left of the current block). If scaling is not required, the gradient magnitude value of that pixel is used as the actual gradient magnitude value. And / or, if scaling is required, the gradient magnitude value can be scaled according to a scaling weight factor to obtain the actual gradient magnitude value of that pixel. The actual gradient magnitude value is then accumulated with the actual gradient magnitude values ​​of other pixels in the prediction mode along the same prediction direction to obtain the first probability of occurrence of the prediction mode in that prediction direction.

[0356] Optionally, for OBIC mode, it can be determined whether the area amplitude value needs to be scaled based on the image block type of the image block and / or the image region type of the image region.

[0357] Optionally, when the image block type or image region type is the image block or image region to the left of the current block, the area amplitude value of the image block or image region is scaled. If no scaling is required, the area amplitude value of the image block or image region is used as the actual gradient amplitude value. And / or, if scaling is required, the area amplitude value can be scaled according to the scaling weight factor to obtain the actual area amplitude value of the pixel. The actual area amplitude value is then accumulated with the actual area amplitude values ​​of other image blocks or image regions of the prediction mode in the same prediction direction to obtain the first occurrence probability of the prediction mode in that prediction direction.

[0358] Optionally, by determining or obtaining the first occurrence probability based on the pixel type of the pixel, the image block type of the image block, and the image region type of the image region, the effectiveness of the determined or obtained first occurrence probability can be ensured, thereby improving the effectiveness of subsequently determining or obtaining the prediction pattern based on the first occurrence probability and improving the prediction effect of prediction based on the prediction pattern.

[0359] Method 16: If the syntax element obtained in the bitstream satisfies the second condition, the first occurrence probability is determined or obtained according to the first occurrence probability calculation strategy and at least one pixel, image block or image region in the reference image region. If the syntax element obtained in the bitstream does not satisfy the second condition, the first occurrence probability is determined or obtained according to the second occurrence probability calculation strategy, which is different from the first occurrence probability calculation strategy, and at least one pixel, image block or image region in the reference image region.

[0360] Optionally, syntax elements may include tagging information such as tags and indexes. These can be tags encoded into the bitstream during encoding at the encoding end.

[0361] Alternatively, a syntax element can be a text number, a binary character, or a markup in other forms.

[0362] Optionally, the first probability calculation strategy and the second probability calculation strategy can be at least one of methods ten to fifteen.

[0363] Optionally, the first probability of occurrence calculation strategy and the second probability of occurrence calculation strategy are different.

[0364] Optionally, when the processing device is a decoder, it can detect whether the syntax elements obtained in the bitstream satisfy the second condition. If the syntax elements satisfy the second condition, a first occurrence probability calculation strategy can be used to calculate the first occurrence probability for pixels, image blocks, or image regions in at least one reference image region. If the syntax elements do not satisfy the fifth condition, a second occurrence probability calculation strategy can be used to calculate the first occurrence probability for pixels, image blocks, or image regions in at least one reference image region.

[0365] Optionally, by using different calculation strategies to calculate the first occurrence probability based on the syntax elements obtained from the code stream, the validity of the determined or obtained first occurrence probability can be ensured, thereby improving the validity of the subsequent determination or acquisition of the prediction pattern based on the first occurrence probability and improving the prediction effect based on the prediction pattern.

[0366] Step S22: Determine or obtain the prediction pattern of the current block based on at least one first occurrence probability.

[0367] Optionally, after determining the first occurrence probability of at least one prediction pattern, a prediction pattern can be selected as the prediction pattern for the current block based on at least one first occurrence probability. When making this selection, the choice can be made according to user needs, or based on default rules and the first occurrence probability; for example, the prediction pattern corresponding to the highest first occurrence probability can be selected as the prediction pattern for the current block.

[0368] Optionally, by determining or obtaining the prediction mode of the current block based on the first occurrence probability of a pixel, image block, or image region in at least one reference image region or the obtained at least one prediction mode, the accuracy of the determined prediction mode is improved, thereby improving the prediction effect of the prediction processing of the current block.

[0369] Optionally, step S22 includes at least one of the following methods seventeen to nineteen.

[0370] Method 17: Determine the second occurrence probability based on the first occurrence probability and the scaling weight factor; determine or obtain the prediction pattern of the current block based on the second occurrence probability.

[0371] Optionally, after determining or obtaining a first probability of occurrence of the prediction direction of at least one prediction mode based on pixels, image blocks, or image regions in at least one reference image region, it can be determined whether the first probability of occurrence needs to be scaled. If scaling is required, the first probability of occurrence can be multiplied using a scaling weight factor to obtain a second probability of occurrence.

[0372] Optionally, if the first probability of occurrence of the prediction direction for all prediction modes is obtained, all first probabilities can be scaled using the same scaling weight factor to obtain a second probability of occurrence. For example, if the scaling weight factor is 0.5, 0.5 can be multiplied by all first probabilities of occurrence to obtain the second probability of occurrence. This makes the amount of data for the second probability of occurrence smaller than that for the first probability of occurrence, which reduces processing complexity and improves prediction efficiency when determining or obtaining the prediction mode for the current block based on the second probability of occurrence. And / or, using a smaller numerical value to represent the probability of occurrence can save storage space.

[0373] Method 18: Determine or obtain the prediction pattern of the current block based on the first and third occurrence probabilities of the same prediction direction;

[0374] Optionally, the first probability of occurrence and the third probability of occurrence can be two probabilities of occurrence determined or obtained using two different methods for the prediction direction of the same prediction model. The first probability of occurrence can be the same as or different from the third probability of occurrence.

[0375] Optionally, if a first probability of occurrence and a third probability of occurrence of the prediction direction of the same prediction mode are determined based on the DIMD pattern, a DIMD template can be determined based on at least one reference image region, and at least two pixel lines can be determined in the DIMD template. The first probability of occurrence of the prediction direction of the same prediction mode is determined based on at least one of the at least two pixel lines. And the third probability of occurrence of the prediction direction of the same prediction mode is determined based on at least one other pixel line of the at least two pixel lines.

[0376] For example, if the first and third probabilities of the prediction direction of the same prediction mode are determined based on the OBIC pattern, then the first and third probabilities of the prediction direction of the same prediction mode can be determined based on different image blocks or image regions within at least one reference image region. For instance, the first probability of the prediction direction of multiple prediction modes can be determined based on multiple image blocks adjacent to and / or not adjacent to the current block within at least one reference image region. Similarly, the third probability of the prediction direction of multiple prediction modes can be determined based on multiple image regions adjacent to and / or not adjacent to the current block within at least one reference image region. Furthermore, the first and third probabilities of the prediction direction belonging to the same prediction mode can be determined.

[0377] Optionally, a first probability of occurrence of the prediction direction of multiple prediction modes can be calculated and determined using the DIMD mode for at least one reference image region. A third probability of occurrence of the prediction direction of multiple prediction modes can be calculated and determined using the OBIC mode for at least one reference image region. The first and third probabilities of occurrence of prediction directions belonging to the same prediction mode are also determined.

[0378] Optionally, the first and third occurrence probabilities for the same prediction direction can be processed to obtain a fifth occurrence probability for that prediction direction. For example, the highest or lowest occurrence probability can be selected as the fifth occurrence probability, or a weighted sum of the first and third occurrence probabilities for the same prediction direction can be performed to obtain the fifth occurrence probability. Based on the fifth occurrence probability, a prediction mode for the prediction direction corresponding to that probability is selected as the prediction mode for the current block. For example, the prediction mode for the prediction direction corresponding to the highest or lowest occurrence probability can be selected.

[0379] Optionally, by determining or obtaining the prediction pattern of the current block based on the first and third occurrence probabilities of the same prediction direction, the accuracy of the determined prediction pattern is ensured, and the prediction effect of prediction based on the prediction pattern is improved.

[0380] Method 19: Determine the scaling weight factor of the first occurrence probability based on the fourth occurrence probability, and determine the prediction mode of the current block based on the first occurrence probability and the scaling weight factor.

[0381] Optionally, the fourth probability of occurrence may be the probability of occurrence of the predicted direction of the prediction pattern determined or obtained based on at least one reference image region using the DIMD mode or the OBIC mode.

[0382] Optionally, the prediction direction of the prediction mode corresponding to the fourth occurrence probability can be the same as the prediction direction of the prediction mode corresponding to the first occurrence probability.

[0383] Optionally, for at least one reference image region, the DIMD mode can be used to process the first occurrence probability of the prediction direction of multiple prediction modes, or the OBIC mode can be used to process the first occurrence probability of the prediction direction of multiple prediction modes.

[0384] Optionally, the need to scale the first probability of occurrence can be determined based on the fourth probability of occurrence. For example, it can be checked whether the fourth probability of occurrence is greater than a preset probability threshold. If it is, a first probability of occurrence in the same prediction direction as the fourth probability of occurrence is determined, and this first probability of occurrence is scaled according to a scaling weight factor to obtain a sixth probability of occurrence. For example, the scaling weight factor can be multiplied by the first probability of occurrence to obtain the sixth probability of occurrence. And / or, if the fourth probability of occurrence is less than or equal to the preset probability threshold, it can be determined that no scaling of the first probability of occurrence is required, or the scaling weight factor can be directly set to 1.

[0385] Optionally, the prediction mode for the current block can be determined or obtained based on the first or fourth probability of occurrence corresponding to the prediction direction of each prediction mode. For example, the prediction mode corresponding to the highest probability of occurrence can be selected as the prediction mode for the current block.

[0386] Optionally, by determining the scaling weight factor of the first occurrence probability based on the fourth occurrence probability, and determining the prediction mode of the current block based on the first occurrence probability and the scaling weight factor, the accuracy of the determined prediction mode is ensured, thereby improving the prediction effect of prediction based on the prediction mode.

[0387] Optionally, the scaling weight factor is determined or obtained in a manner that includes at least one of the following methods 20 to 24:

[0388] Method 20: Determine or obtain the scaling weight factor based on the position of a pixel, an image block, and at least one of the elements in the image region relative to the current block in at least one reference image region.

[0389] Optionally, a scaling weight factor can be set based on the position of a pixel, an image block, and at least one of the elements in at least one reference image region relative to the current block. The scaling weight factor varies depending on the position. For example, the farther the pixel is from the current block, the smaller the scaling weight factor; the closer the pixel is to the current block, the larger the scaling weight factor.

[0390] Optionally, a mapping table including the mapping relationship between position and scaling weight factor can be determined, and the scaling weight factor can be obtained by querying the mapping table based on the position of a pixel, an image block, and at least one item in the image region relative to the current block in at least one reference image region.

[0391] Optionally, for DIMD mode, a scaling weight factor can be determined based on the position of the pixel used to determine the prediction mode relative to the current block in at least one reference image region. The calculated gradient magnitude value of the pixel is then multiplied by the scaling weight factor to obtain the actual gradient magnitude value. The prediction mode for the prediction direction of the pixel is then determined, and the actual gradient magnitude value is accumulated to the gradient magnitude value of the prediction mode in the same prediction direction as the pixel to obtain the first probability of occurrence of the prediction mode in that prediction direction. This allows the magnitude of the amplitude value to be closely related to the pixel's position. For example, pixels closer to the current block have a higher similarity to the current block, and using the prediction mode used by the closer pixels as the prediction mode for the current block for intra-frame prediction yields better prediction results compared to pixels farther from the current block.

[0392] Optionally, in DIMD mode, the magnitudes of the horizontal and vertical gradient values ​​of pixels affect the determination of the prediction mode for the current block. Therefore, a larger scaling weight factor can be set for pixels closer to the current block, and a smaller scaling weight factor can be set for pixels farther away from the current block. This adjusts the gradient magnitude values ​​of pixels in the reference image region, thereby enhancing the influence of the gradient magnitude values ​​of pixels closer to the current block on the determination of the prediction mode for the current block, reducing the influence of the gradient magnitude values ​​of pixels farther away from the current block on the determination of the prediction mode for the current block, and thus improving the accuracy of the prediction for the current block.

[0393] Optionally, for OBIC mode, the scaling weight factor can be determined based on the position of the image block and / or image region relative to the current block in at least one reference image region used to determine the prediction mode.

[0394] Optionally, the following uses an image patch as an example for illustration: In OBIC mode, the position of the image patch affects the accuracy of the prediction mode determination of the current patch. Generally, the image patch that is closer to the current patch is more similar to the current patch. Compared with the image patch that is farther away from the current patch, the prediction effect of using the prediction mode used by the closer image patch as the prediction mode of the current patch for intra-frame prediction is better.

[0395] Optionally, different scaling weight factors can be set for multiple image blocks with different locations but the same area. For example, the scaling weight factor is set to a larger value for image blocks closer to the current block, and a smaller value is set to a smaller value for image blocks farther away from the current block. This adjusts the area amplitude values ​​of image blocks in the reference image region, thereby enhancing the influence of the area amplitude values ​​of image blocks closer to the current block on the determination of the prediction mode of the current block, reducing the influence of the area amplitude values ​​of image blocks farther away on the determination of the prediction mode of the current block, and thus improving the accuracy of the prediction of the current block.

[0396] Optionally, pixels in the reference image region can be classified based on their position relative to the current block within at least one pixel in the reference image region, and the scaling weight factor of the pixel can be determined or obtained based on the classification result. For example, if a DIMD template exists in the reference image region, and there are four pixel lines in the DIMD template, a different scaling weight factor can be set for each pixel line. For instance, pixel line 1 corresponds to scaling weight factor 1, pixel line 2 corresponds to scaling weight factor 2, pixel line 3 corresponds to scaling weight factor 3, and pixel line 4 corresponds to scaling weight factor 4. If there exists a pixel whose position is located in pixel line 2, then the scaling weight factor of that pixel is scaling weight factor 2. If the pixel's position is located in pixel line 3, then the scaling weight factor of that pixel is scaling weight factor 3.

[0397] Optionally, image blocks in a reference image region can be classified based on their position relative to the current block within at least one reference image region, and a scaling weight factor for the image blocks can be determined or obtained based on the classification result. For example, different scaling weight factors can be set for different classification results.

[0398] Optionally, by determining or obtaining the scaling weight factor based on the position of a pixel, an image block, and at least one of the elements in the image region relative to the current block in at least one reference image region, the effectiveness of the determined scaling weight factor is ensured, which makes the subsequent determination of the prediction mode more effective, thereby improving the prediction effect of prediction based on the prediction mode.

[0399] Method 21: Determine or obtain a scaling weight factor based on at least one of a first image region, a first image block, and a first pixel that is adjacent to and / or not adjacent to at least one of a pixel, an image block, and at least one of an image region in at least one reference image region.

[0400] Optionally, a scaling weight factor can be determined or obtained based on a first pixel that is adjacent to and / or not adjacent to a pixel in at least one reference image region. For example, when calculating the gradient magnitude value of pixel 1 in the reference image region, a pixel 2 that is adjacent to and / or not adjacent to pixel 1 can be determined. If the prediction direction of pixel 2 is consistent with the prediction direction of pixel 1, the scaling weight factor of pixel 1 can be set to be greater than 1, and / or, if the prediction direction of pixel 2 is inconsistent with the prediction direction of pixel 1, the scaling weight factor of pixel 1 can be set to be less than 1. The scaling weight factor can also be set based on the magnitude of the gradient magnitude value of the first pixel; for example, the larger the gradient magnitude value, the larger the scaling weight factor.

[0401] Optionally, adjacent and / or non-adjacent image blocks can be identified within the image block containing the pixel in the reference image region, and a scaling weight factor can be determined or obtained. For example, if the prediction direction of the prediction mode corresponding to the adjacent and / or non-adjacent image block is consistent with the prediction direction of the prediction mode for the pixel, a larger value, such as 3, can be set for the scaling weight factor of that pixel. And / or, if they are inconsistent, a smaller value, such as 0.5, can be set for the scaling weight factor of that pixel. The scaling weight factor can also be set based on the area amplitude value of the adjacent and / or non-adjacent image blocks; for example, the larger the area amplitude value, the larger the scaling weight factor.

[0402] Optionally, the image region containing the pixels in the reference image region can be determined, such as a T-shaped image region. Adjacent and / or non-adjacent image regions can also be determined, so that scaling weights can be determined or obtained based on these adjacent and / or non-adjacent image regions. For example, the scaling weight factor can be set based on whether the prediction directions are consistent, or it can be set based on the area amplitude values ​​corresponding to the adjacent and / or non-adjacent image regions; for example, the larger the area amplitude value, the larger the scaling weight factor.

[0403] Optionally, a scaling weight factor is determined or obtained based on a first image patch that is adjacent to and / or non-adjacent to an image patch in at least one reference image region. For example, a larger scaling weight factor, such as a scaling weight factor greater than 1, can be set when the prediction direction of the first image patch is consistent with the prediction direction of the image patch, and / or a smaller scaling weight factor, such as a scaling weight factor less than 1, can be set when the prediction direction of the first image patch is inconsistent with the prediction direction of the image patch. The scaling weight factor can also be determined based on the magnitude of the area amplitude value of the first image patch; for example, the larger the area amplitude value, the larger the scaling weight factor.

[0404] Optionally, a scaling weight factor is determined or obtained based on pixels or image regions that are adjacent to and / or non-adjacent to an image block in at least one reference image region. For example, different scaling weight factors can be set based on the gradient magnitude value of the pixel and / or the area magnitude value of the image region. For instance, the larger the gradient magnitude value or the larger the area magnitude value, the larger the scaling weight factor.

[0405] Optionally, a scaling weight factor is determined or obtained based on a first image region that is adjacent to and / or non-adjacent to an image region in at least one reference image region. For example, a larger scaling weight factor, such as a scaling weight factor greater than 1, can be set when the prediction direction of the first image region is consistent with the prediction direction of the reference image region; conversely, a smaller scaling weight factor, such as a scaling weight factor less than 1, can be set when the prediction direction of the first image region is inconsistent with the prediction direction of the reference image region. Alternatively, the scaling weight factor can be determined based on the magnitude of the area amplitude value of the first image region; for example, a larger area amplitude value corresponds to a larger scaling weight factor.

[0406] Optionally, a scaling weight factor is determined or obtained based on an image region and / or non-adjacent pixels or image blocks within at least one reference image region. For example, different scaling weight factors can be set based on the gradient magnitude value of the pixel and / or the area magnitude value of the image block. For instance, the larger the gradient magnitude value or the larger the area magnitude value, the larger the scaling weight factor.

[0407] Optionally, for DIMD mode, as shown in Figure 15, if pixels A and B exist in the reference image region of the block to be predicted (i.e., the current block), and if the neighboring pixels of pixel A include at least one of the pixels above, to the left, below, and to the right, and if one of these neighboring pixels has the same gradient direction or prediction direction as pixel A, then the scaling weight factor of pixel A can be set to be greater than 1; and / or, if one of these neighboring pixels has a different gradient direction (or prediction direction) than pixel A, then the scaling weight factor of pixel A can be set to be less than 1.

[0408] Optionally, if the neighboring pixels of pixel B include at least one of the following: the pixel above, the pixel to the left, the pixel below, the pixel to the right, the pixel to the upper left, the pixel to the lower left, the pixel to the upper right, and the pixel to the lower right. If one of these neighboring pixels has the same gradient direction or prediction direction as pixel B, then the scaling weight factor of pixel B can be set to be greater than 1; and / or, if one of these neighboring pixels has a different gradient direction (or prediction direction) than pixel B, then the scaling weight factor of pixel B can be set to be greater than 1.

[0409] Optionally, for OBIC mode, as shown in Figure 16, if the reference image region of the block to be predicted (i.e., the current block) includes an encoded region, and the encoded region contains coded block 1, and the adjacent coded blocks of coded block 1 include at least one of the following: upper image block, left image block, lower image block, right image block, upper left image block, lower left image block, upper right image block, and lower right image block. If one of these adjacent coded blocks has the same gradient direction or prediction direction as coded block 1, the scaling weight factor of coded block 1 can be set to be greater than 1; and / or, if one of these adjacent coded blocks has a different gradient direction (or prediction direction) than coded block 1, the scaling weight factor of coded block 1 can be set to be greater than 1.

[0410] Optionally, the coded block is an image block.

[0411] Optionally, by determining or obtaining a scaling weight factor based on at least one of a first image region, first image block, and first pixel that is adjacent to and / or not adjacent to at least one of a pixel, image block, and at least one of an image region in at least one reference image region, the effectiveness of the determined scaling weight factor is ensured, which makes it easier to determine a more effective prediction mode subsequently, thereby improving the prediction effect of prediction based on the prediction mode.

[0412] Method 22: Determine or obtain the scaling weight factor based on the image region type of at least one image region, and / or the image block type of at least one image block, and / or the pixel type of at least one pixel;

[0413] Optionally, the pixel type, image block type, and image region type can refer to method fifteen in the above embodiments, and will not be repeated here.

[0414] Optionally, a scaling weight factor is determined or obtained based on the image region type of at least one image region. A mapping table containing the correspondence between different image region types and scaling weight factors can be determined, and the scaling weight factor corresponding to the image region type can be retrieved from this mapping table. Alternatively, other methods can be used to determine the scaling weight factor based on the image region type.

[0415] Optionally, a scaling weight factor is determined or obtained based on the image patch type of at least one image patch. A mapping table containing the correspondence between image patch types and scaling weight factors can be determined, and the scaling weight factor corresponding to the image patch type can be obtained by querying this mapping table.

[0416] Optionally, if the position coordinates of an image patch are to the left of the current block, then the image patch type of that image patch is the image patch to the left of the current block; if the position coordinates of an image patch are above the current block, then the image patch type of that image patch is the image patch above the current block; and / or, if the position coordinates of an image patch are to the upper left of the current block, then the image patch type of that image patch is the image patch to the upper left of the current block. Area amplitude histograms can be constructed for these three types of image patches, with smaller scaling weight factors set for smaller area amplitude values ​​and larger scaling weight factors set for larger area amplitude values. The scaling weight factor can be set to 1 for the top two area amplitude values ​​in each area histogram, and to 0 for other area amplitude values. In this way, for each of the three area histograms, at most two intra-frame prediction direction area amplitude values ​​are obtained. Then, based on the six area amplitude values ​​in the three area histograms, the prediction mode for one or more intra-frame prediction directions to be used, i.e., the prediction mode for the current block, is derived.

[0417] Optionally, a scaling weight factor is determined or obtained based on the pixel type of at least one pixel. A mapping table containing the correspondence between pixel types and scaling weight factors can be determined, and the scaling weight factor corresponding to the pixel type of the pixel can be retrieved from this mapping table.

[0418] Optionally, if the position coordinates of a pixel are located to the left of the current block, then the pixel type of that pixel is determined to be a pixel to the left of the current block.

[0419] Optionally, if the position coordinates of a pixel are above the current block, then the pixel type of that pixel is determined to be the pixel above the current block.

[0420] Optionally, if the position coordinates of a pixel are located above the left side of the current block, then the pixel type of that pixel is determined to be the pixel above the left side of the current block.

[0421] Optionally, gradient magnitude histograms can be constructed separately for each of the three types. For each gradient magnitude histogram, a smaller scaling factor is set for smaller gradient magnitude values, and a larger scaling factor is set for larger gradient magnitude values. Optionally, the scaling factor for the top two gradient magnitude values ​​in each gradient histogram can be set to 1, and the scaling factor for other gradient magnitude values ​​can be set to 0. In this way, for each of the three gradient histograms, at most two intra-frame prediction direction gradient magnitude values ​​are obtained. Then, based on the six gradient magnitude values ​​from the three gradient histograms, the prediction mode for one or more intra-frame prediction directions to be used, i.e., the prediction mode for the current block, is derived.

[0422] Optionally, by determining or obtaining the scaling weight factor based on the image region type of at least one image region, and / or the image block type of at least one image block, and / or the pixel type of at least one pixel, the effectiveness of the determined scaling weight factor is ensured, which makes the subsequent determination of the prediction mode more effective, thereby improving the prediction effect of prediction based on the prediction mode.

[0423] Method 23: Determine or obtain the scaling weight factor based on the horizontal distance, and / or vertical distance, and / or shortest distance between the current block and at least one pixel in at least one reference image region, an image block, and at least one item in the image region.

[0424] Optionally, a scaling weight factor is determined or obtained based on the horizontal distance between pixels in at least one reference image region and the current block. Different scaling weight factors can be set for different horizontal distances. For example, the smaller the horizontal distance, the larger the scaling weight factor.

[0425] Optionally, a scaling weight factor is determined or obtained based on the vertical distance between pixels in at least one reference image region and the current block. Different scaling weight factors can be set according to different vertical distances. For example, the smaller the vertical distance, the larger the scaling weight factor.

[0426] Optionally, a scaling weight factor is determined or obtained based on the shortest distance between a pixel in at least one reference image region and the current block. Different scaling weight factors can be set based on different shortest distances. For example, the smaller the shortest distance, the larger the scaling weight factor.

[0427] Optionally, for DIMD mode, for example, as shown in Figure 17, for pixel A in the pixel line above the current block in the DIMD template, the scaling weight factor of pixel A can be determined based on the vertical distance between pixel A and the current block. The vertical distance between the pixel and the current block is the difference between the pixel's ordinate and the ordinate of the current block. For pixel B in the pixel line to the left of the current block in the DIMD template, the scaling weight factor of pixel B can be determined based on the horizontal distance between pixel B and the current block. The horizontal distance between the pixel and the current block is the difference between the pixel's abscissa and the abscissa of the current block.

[0428] Optionally, a scaling weight factor is determined or obtained based on the horizontal distance, and / or vertical distance, and / or shortest distance between an image block in at least one reference image region and the current block.

[0429] Optionally, a scaling weight factor is determined or obtained based on the horizontal distance, and / or vertical distance, and / or shortest distance between an image region in at least one reference image region and the current block. For example, the larger the horizontal distance, the smaller the scaling weight factor of the image block or image region. The larger the vertical distance, the smaller the scaling weight factor of the image block or image region. The larger the shortest distance, the smaller the scaling weight factor of the image block or image region.

[0430] Optionally, for OBIC mode, the scaling weight factor can be determined or obtained based on at least one of the horizontal distance, vertical distance and shortest distance between the image patch and the current patch.

[0431] Optionally, the scaling weight factor can be determined or obtained based on a first distance between the current block and a pixel in at least one reference image region, an image block, and at least one item in the image region. Optionally, the first distance can be determined using the following formula (V) and / or formula (VI): d=(x1-x2) 2 +(y1+y2) 2 Formula (V); d=|x1-x2|+|y1+y2| Formula (VI);

[0432] Optionally, x1 and y1 can be the x-coordinate and y-coordinate of the current block, and x2 and y2 can be the x-coordinate and y-coordinate of the image block. d can be the first distance.

[0433] Optionally, the smaller the first distance, the larger the scaling weight factor can be, or the first distance can be compared with a preset distance threshold. If the first distance is greater than the preset distance threshold, the scaling weight factor can be set to be less than 1. If the first distance is less than or equal to the preset distance threshold, the scaling weight factor can be set to be greater than 1.

[0434] Optionally, a ninth mapping table can be determined, which includes the horizontal distance between a pixel in at least one reference image region, an image block, and at least one item in the image region and the current block, and the corresponding scaling weight factor, to determine or obtain the scaling weight factor. For example, after determining the horizontal distance between a pixel and the current block, the scaling weight factor can be obtained by querying the ninth mapping table.

[0435] Optionally, the ninth mapping table can be set according to Table 9 below.

[0436] Table 9

[0437] Optionally, a tenth mapping table can be determined, which includes the vertical distance between a pixel in at least one reference image region, an image block, and at least one item in the image region and the current block, and the corresponding scaling weight factor, to determine or obtain the scaling weight factor. For example, after determining the vertical distance between a pixel and the current block, the scaling weight factor can be obtained by querying the tenth mapping table.

[0438] The tenth mapping table can be set as shown in Table 10 below.

[0439] Table 10

[0440] Optionally, an eleventh mapping table can be determined, which includes the shortest distance between a pixel in at least one reference image region, an image block, and at least one item in the image region and the current block, and the corresponding scaling weight factor, to determine or obtain the scaling weight factor. For example, after determining the shortest distance between a pixel and the current block, the scaling weight factor can be obtained by querying the eleventh mapping table.

[0441] The eleventh mapping table can be set as shown in Table 11 below.

[0442] Table 11

[0443] Optionally, a function formula can be set in advance to input the shortest distance, either horizontal or vertical, between the current block and a pixel in at least one reference image region, an image block, and at least one item in the image region. This will be used to calculate the scaling weight factor.

[0444] Alternatively, the function formula can be as shown in formula (VII). y = b - ax Formula (VII);

[0445] Optionally, y is the scaling weight factor, x is the horizontal distance, vertical distance, or shortest distance, and a and b are constants, such as a = 0.5 and b = 4.

[0446] Optionally, by determining or obtaining the scaling weight factor based on the horizontal distance, and / or vertical distance, and / or shortest distance between the current block and a pixel, image block, and at least one of the elements in at least one reference image region, the effectiveness of the determined scaling weight factor is ensured, which facilitates a more effective prediction mode subsequently determined, thereby improving the prediction effect of prediction based on the prediction mode.

[0447] Method 24: If the syntax elements obtained in the bitstream meet the third condition, the first scaling weight factor calculation strategy is used to determine or obtain the scaling weight factor. If the syntax elements obtained in the bitstream meet the fourth condition, the second scaling weight factor calculation strategy, which is different from the first scaling weight factor calculation strategy, is used to determine or obtain the scaling weight factor.

[0448] Optionally, the syntax elements can refer to method sixteen in the above embodiments, which will not be repeated here.

[0449] Optionally, the third and fourth conditions may differ.

[0450] Optionally, the third condition may be a pixel, image block, or image region in at least one reference image region that is in the same coding tree as the current block.

[0451] Optionally, the fourth condition may be a pixel, image block, or image region in at least one reference image region that is not in the same coding tree as the current block.

[0452] Alternatively, the third and fourth conditions can also be two different conditions set by the user based on their needs.

[0453] Optionally, the priority of the third condition can be higher than the priority of the fourth condition.

[0454] Optionally, the first scaling weight factor calculation strategy may include at least one of the following: determining or obtaining a scaling weight factor based on the position of a pixel, image block, and at least one of an image region in at least one reference image region relative to the current block; determining or obtaining a scaling weight factor based on at least one of a first image region, first image block, and first pixel that is adjacent to and / or non-adjacent to a pixel, image block, and at least one of an image region in at least one reference image region; determining or obtaining a scaling weight factor based on the image region type of at least one image region, and / or the image block type of at least one image block, and / or the pixel type of at least one pixel; and determining or obtaining a scaling weight factor based on the horizontal distance, and / or vertical distance, and / or the shortest distance between a pixel, image block, and at least one of an image region in at least one reference image region and the current block.

[0455] Optionally, the second scaling weight factor calculation strategy may include at least one of the following: determining or obtaining a scaling weight factor based on the position of a pixel, image block, and at least one of an image region in at least one reference image region with respect to the current block; determining or obtaining a scaling weight factor based on at least one of a first image region, first image block, and first pixel that is adjacent to and / or non-adjacent to a pixel, image block, and at least one of an image region in at least one reference image region; determining or obtaining a scaling weight factor based on the image region type of at least one image region, and / or the image block type of at least one image block, and / or the pixel type of at least one pixel; and determining or obtaining a scaling weight factor based on the horizontal distance, and / or vertical distance, and / or the shortest distance between a pixel, image block, and at least one of an image region in at least one reference image region and the current block.

[0456] Optionally, when the processing device is a decoder, it can detect whether the syntax elements obtained in the bitstream satisfy the third condition. If the syntax elements satisfy the third condition, the scaling weight factor can be determined or obtained using the first scaling weight factor calculation strategy. And / or, if the syntax elements do not satisfy the third condition, it can detect whether the syntax elements satisfy the fourth condition. If the syntax elements satisfy the fourth condition, the scaling weight factor can be determined or obtained using the second scaling weight factor calculation strategy.

[0457] Optionally, by selecting different calculation strategies to determine or obtain scaling weight factors based on the conditions satisfied by the syntax elements obtained in the code stream, the effectiveness of the determined scaling weight factors is ensured, which makes the subsequent determined prediction mode more effective and improves the prediction effect of prediction based on the prediction mode.

[0458] Fourth embodiment

[0459] Based on any of the above embodiments, a fourth embodiment is proposed.

[0460] In this embodiment, the image processing method further includes at least one of the following methods 25 to 27.

[0461] Method 25: Determine or obtain at least two different occurrence probabilities based on at least one of the gradient of pixels in different reference image regions, the area of ​​image blocks or image regions, and determine or obtain the prediction mode of the current block based on the at least two different occurrence probabilities.

[0462] Optionally, different reference image regions may include at least two reference image regions, such as a first reference image region and a second reference image region.

[0463] Optionally, for the first reference image region, the gradient magnitude value corresponding to each pixel can be determined or obtained based on the gradient of the pixels in the first reference image region, and the first probability of occurrence of the prediction mode of multiple prediction directions can be determined or obtained based on the gradient magnitude value corresponding to each pixel.

[0464] Optionally, for the second reference image region, the area amplitude value of each image block or image region can be determined or obtained based on at least one of the areas of the image blocks or image regions in the second reference image region, and the first probability of occurrence of the prediction pattern of multiple prediction directions can be determined or obtained based on the area amplitude value of each image block or image region.

[0465] Optionally, the area amplitude value of the image block or image region can be determined or obtained based on the area of ​​the image block or image region in the first reference image region, and then the first occurrence probability of the prediction mode of multiple prediction directions can be determined or obtained based on the area amplitude value of the image block or image region.

[0466] Optionally, the area amplitude value of the image block or image region can be determined or obtained based on the area of ​​the image block or image region in the second reference image region, and then the first occurrence probability of the prediction mode of multiple prediction directions can be determined or obtained based on the area amplitude value of the image block or image region.

[0467] Optionally, for two first occurrence probabilities of a prediction mode in the same prediction direction (i.e., the first occurrence probability corresponding to the first reference image region and the first occurrence probability corresponding to the second reference image region), the largest or smallest occurrence probability can be selected as the actual occurrence probability of the prediction mode in that prediction direction. Furthermore, from the actual occurrence probabilities of prediction modes in at least one prediction direction, the prediction mode corresponding to the largest actual occurrence probability is randomly selected or chosen as the prediction mode for the current block.

[0468] Optionally, it can also be possible to calculate two different occurrence probabilities of the prediction mode for the same prediction direction for different coding trees. For example, based on at least one of the pixels, image blocks, and image regions in the same coding tree as the current block, a first occurrence probability of the prediction mode for multiple prediction directions can be determined or obtained, and based on at least one of the pixels, image blocks, and image regions not in the same coding tree as the current block, a first occurrence probability of the prediction mode for multiple prediction directions can be determined or obtained.

[0469] Optionally, after determining two different occurrence probabilities of a prediction pattern for the same prediction direction, the largest or smallest occurrence probability can be selected as the actual occurrence probability of the prediction pattern for that prediction direction. Then, from the actual occurrence probabilities of prediction patterns for at least one prediction direction, either randomly or by selecting the prediction pattern corresponding to the largest actual occurrence probability, it can be used as the prediction pattern for the current block.

[0470] Optionally, by determining or obtaining at least two different occurrence probabilities based on at least one of the gradient of pixels in different reference image regions, the area of ​​image blocks or image regions, and determining or obtaining the prediction mode of the current block based on the at least two different occurrence probabilities, the accuracy of the determined prediction mode is ensured, thereby improving the prediction effect of prediction based on the prediction mode.

[0471] Method 26: The reference image region includes at least one of the left image region located to the left of the current block and the upper image region located above the current block;

[0472] Optionally, after determining that the reference image region is the left image region located to the left of the current block, steps S21 and S22 in the above embodiments can be performed based on the left image region. That is, a first probability of occurrence of at least one predicted mode is determined or obtained based on the pixels, image blocks, or image regions in the left image region, and the predicted mode of the current block is determined or obtained based on at least one first probability of occurrence.

[0473] Optionally, after determining that the reference image region is the upper image region located above the current block, steps S21 and S22 in the above embodiments can be performed based on the upper image region. That is, a first probability of occurrence of at least one prediction mode is determined or obtained based on the pixels, image blocks, or image regions in the upper image region, and the prediction mode of the current block is determined or obtained based on at least one first probability of occurrence.

[0474] Optionally, the left image region and / or the upper image region can be an image region in the same coding tree as the current block, or an image region not in the same coding tree as the current block.

[0475] Optionally, for DIMD mode, the left image region and the top image region can be the image region where the left pixel is located within the same pixel line in the DIMD template, and the image region where the top pixel is located, respectively.

[0476] Optionally, by determining that the reference image region includes at least one of the left image region located to the left of the current block and the upper image region located above the current block, the validity of the prediction pattern determined or obtained based on the reference image region can be ensured, thereby improving the prediction effect of prediction based on the prediction pattern.

[0477] Method 27: The prediction pattern of the current block is determined or obtained based on the maximum probability of at least one first occurrence probability.

[0478] Optionally, after determining or obtaining a first probability of occurrence of at least one prediction mode based on pixels, image blocks or image regions in at least one reference image region, the maximum probability of occurrence is selected from the at least one first probability of occurrence, and the prediction mode of the prediction direction corresponding to the maximum probability of occurrence is determined as the prediction mode of the current block.

[0479] Optionally, by determining or obtaining the prediction pattern of the current block based on the maximum occurrence probability of at least one first occurrence probability, the effectiveness of the determined or obtained prediction pattern can be ensured, thereby improving the prediction effect of prediction based on the prediction pattern.

[0480] Through the technical solution of this embodiment, during the prediction stage of video encoding and / or decoding, such as intra-frame prediction, the prediction mode can be determined directly based on at least one reference image region determined from the image information of the current block. This takes into account the relationship between the reference image region and the current block, making the determined prediction mode more accurate. In other words, it improves the accuracy of deriving the prediction mode during video encoding and / or decoding, thereby improving the prediction effect for the current block, such as improving prediction accuracy, and thus improving the prediction efficiency during video encoding and / or decoding.

[0481] Referring to Figure 18, this application embodiment also provides an image processing apparatus, which includes:

[0482] Processing module A10 is used to determine or obtain at least one reference image region based on the image block information of the current block; and to determine or obtain the prediction mode of the current block based on at least one reference image region.

[0483] Optionally, processing module A10 is configured to perform at least one of the following:

[0484] Determine or obtain at least one reference image region based on at least one of the following: width, height, block size, block area, image block attributes, and image block type of the current block.

[0485] Based on image block information from at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, and default blocks corresponding to the current block, determine or obtain at least one reference image region;

[0486] Based on the image block information of the first component image block of the current block, determine or obtain at least one reference image region of the second component image block of the current block;

[0487] Based on the candidate motion vector or candidate block vector of the current block, determine or obtain the image block information of the candidate block, and at least one reference image region is determined or obtained;

[0488] If the current block satisfies the first condition, then at least one reference image region is determined or obtained according to the first calculation strategy;

[0489] If the current block does not meet the first condition, then at least one reference image region is determined or obtained according to the second calculation strategy.

[0490] Optionally, the second computation strategy may differ from the first computation strategy.

[0491] Optionally, the current block satisfies a first condition, including at least one of the following:

[0492] The width of the current block is greater than or equal to the first preset width threshold;

[0493] The height of the current block is greater than or equal to the first preset height threshold;

[0494] The area of ​​the current block is greater than or equal to the first preset area threshold.

[0495] Optionally, the processing module A10 is used for:

[0496] A first probability of occurrence of at least one predicted pattern is determined or obtained based on a pixel, image block, or image region in at least one reference image region; and a predicted pattern of the current block is determined or obtained based on at least one first probability of occurrence.

[0497] Optionally, processing module A10 is configured to perform at least one of the following:

[0498] Based on pixels, image blocks, or image regions in at least one reference image region, determine or obtain a first amplitude value and a second amplitude value in the same prediction direction, and determine or obtain a first probability of occurrence based on the first amplitude value and the second amplitude value in the same prediction direction.

[0499] At least one first amplitude value is determined or obtained based on a pixel, image block, or image region in at least one reference image region, and a first probability of occurrence is determined or obtained based on the first amplitude value and a scaling weight factor.

[0500] Based on the gradient magnitude and gradient direction of the gradient of a pixel in at least one reference image region, determine or obtain a first probability of occurrence of the prediction direction of at least one prediction mode.

[0501] Based on the prediction pattern corresponding to an image patch or image region in at least one reference image region, and the area of ​​the image patch or image region, determine or obtain a first probability of occurrence of the prediction direction of at least one prediction pattern;

[0502] A first probability of occurrence is determined or obtained based on at least one of the following: position, horizontal distance, vertical distance, and shortest distance between a pixel, image block, or image region in at least one reference image region and the current block;

[0503] Based on the pixel type of the pixel, the image block type of the image block, and the image region type of the image region, determine or obtain the first probability of occurrence;

[0504] If the syntax element obtained in the bitstream satisfies the second condition, the first occurrence probability is determined or obtained according to the first occurrence probability calculation strategy and at least one pixel, image block or image region in the reference image region. If the syntax element obtained in the bitstream does not satisfy the second condition, the first occurrence probability is determined or obtained according to the second occurrence probability calculation strategy, which is different from the first occurrence probability calculation strategy, and at least one pixel, image block or image region in the reference image region.

[0505] Optionally, processing module A10 is configured to perform at least one of the following:

[0506] Based on the first occurrence probability and the scaling weight factor, determine the second occurrence probability, and based on the second occurrence probability, determine or obtain the prediction pattern of the current block;

[0507] Based on the first and third occurrence probabilities of the same prediction direction, the prediction pattern of the current block is determined or obtained;

[0508] The scaling weight factor for the first occurrence probability is determined based on the fourth occurrence probability, and the prediction mode for the current block is determined based on the first occurrence probability and the scaling weight factor.

[0509] Optionally, the method of determining or obtaining the scaling weight factor includes at least one of the following:

[0510] Based on the position of a pixel, an image block, and at least one of the elements in at least one reference image region relative to the current block, a scaling weight factor is determined or obtained.

[0511] The scaling weight factor is determined or obtained based on at least one of the first image region, first image block and first pixel that is adjacent and / or non-adjacent to at least one of the pixels, image blocks and at least one of the image regions in at least one reference image region;

[0512] Based on the image region type of at least one image region, and / or the image block type of at least one image block, and / or the pixel type of at least one pixel, determine or obtain the scaling weight factor;

[0513] Based on the horizontal distance, and / or vertical distance, and / or shortest distance between the current block and a pixel in at least one reference image region, an image block, and at least one item in the image region; determine or obtain the scaling weight factor.

[0514] If the syntax elements obtained from the bitstream meet the third condition, the first scaling weight factor calculation strategy is used to determine or obtain the scaling weight factor. If the syntax elements obtained from the bitstream meet the fourth condition, the second scaling weight factor calculation strategy, which is different from the first scaling weight factor calculation strategy, is used to determine or obtain the scaling weight factor.

[0515] Optionally, processing module A10 is configured to perform at least one of the following:

[0516] Based on at least one of the gradient of pixels in different reference image regions, the area of ​​image blocks or image regions, determine or obtain at least two different occurrence probabilities, and determine or obtain the prediction mode of the current block based on the at least two different occurrence probabilities.

[0517] The reference image region includes at least one of the left image region located to the left of the current block and the upper image region located above the current block;

[0518] The prediction pattern of the current block is determined or obtained based on the maximum probability of occurrence of at least one first probability.

[0519] The image processing apparatus provided in this application embodiment is similar in implementation principle and beneficial effect to the corresponding method embodiment described above, and will not be repeated here.

[0520] This application also provides a processing device, including a memory and a processor. The memory stores an image processing program, and when the image processing program is executed by the processor, it implements the steps of the image processing method in any of the above embodiments.

[0521] This application also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the image processing method in any of the above embodiments.

[0522] In the embodiments of the processing device and storage medium provided in this application, all the technical features of any of the above-described image processing method embodiments may be included. The extended and explanatory content of the specification is basically the same as that of the embodiments of the above methods, and will not be repeated here.

[0523] This application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to perform the image processing methods described in the various possible implementations above.

[0524] This application also provides a chip, including a memory and a processor. The memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that a device with the chip installed performs the image processing methods as described in the various possible implementations above.

[0525] It is understood that the above scenarios are merely examples and do not constitute a limitation on the application scenarios of the technical solutions provided in the embodiments of this application. The technical solutions of this application can also be applied to other scenarios. For example, as those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0526] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0527] The steps in the method of this application embodiment can be adjusted, combined, or deleted according to actual needs.

[0528] The units in the device of this application embodiment can be merged, divided, and deleted according to actual needs.

[0529] In this application, the same or similar terms, concepts, technical solutions and / or application scenario descriptions are generally described in detail only when they appear for the first time. When they appear again, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of this application, the same or similar terms, concepts, technical solutions and / or application scenario descriptions that are not described in detail later can be referred to their previous relevant detailed descriptions.

[0530] In this application, the descriptions of the various embodiments have different focuses. For parts that are not described in detail or in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0531] The technical features of the present application can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present application.

[0532] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, controlled terminal, or network device, etc.) to execute the methods of each embodiment of this application.

[0533] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, storage disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0534] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An image processing method, wherein, Including the following steps: S10, determine or obtain at least one reference image region based on the image block information of the current block; S20, determine or obtain the prediction mode of the current block based on at least one reference image region.

2. The method as described in claim 1, wherein, Step S10 includes at least one of the following: Determine or obtain at least one reference image region based on at least one of the following: width, height, block size, block area, image block attributes, and image block type of the current block. Based on image block information from at least one of the following: neighboring blocks, non-neighboring blocks, co-located blocks, temporal blocks, and default blocks, determine or obtain at least one reference image region. Based on the image block information of the first component image block of the current block, determine or obtain at least one reference image region of the second component image block of the current block; Based on the candidate motion vector or candidate block vector of the current block, determine or obtain the image block information of the candidate block, and at least one reference image region is determined or obtained; If the current block satisfies the first condition, then at least one reference image region is determined or obtained according to the first calculation strategy; If the current block does not meet the first condition, then at least one reference image region is determined or obtained according to the second calculation strategy.

3. The method as described in claim 2, wherein, The second computation strategy differs from the first computation strategy; and / or, the current block satisfies a first condition, including at least one of the following: The width of the current block is greater than or equal to the first preset width threshold; The height of the current block is greater than or equal to the first preset height threshold; The area of ​​the current block is greater than or equal to the first preset area threshold.

4. The method of claim 1, wherein, Step S20 includes the following steps: S21, determine or obtain a first probability of occurrence of at least one predicted pattern based on pixels, image blocks or image regions in at least one reference image region; S22, determine or obtain the prediction pattern of the current block based on at least one first occurrence probability.

5. The method of claim 4, wherein, Step S21 includes at least one of the following: Based on pixels, image blocks, or image regions in at least one reference image region, determine or obtain a first amplitude value and a second amplitude value in the same prediction direction, and determine or obtain a first probability of occurrence based on the first amplitude value and the second amplitude value in the same prediction direction. At least one first amplitude value is determined or obtained based on a pixel, image block or image region in at least one reference image region, and a first probability of occurrence is determined or obtained based on the first amplitude value and a scaling weight factor; Based on the gradient magnitude and gradient direction of the gradient of a pixel in at least one reference image region, determine or obtain a first probability of occurrence of the prediction direction of at least one prediction mode. Based on the prediction pattern corresponding to an image patch or image region in at least one reference image region, and the area of ​​the image patch or image region, determine or obtain a first probability of occurrence of the prediction direction of at least one prediction pattern; A first probability of occurrence is determined or obtained based on at least one of the following: position, horizontal distance, vertical distance, and shortest distance between a pixel, image block, or image region in at least one reference image region and the current block; Based on the pixel type of the pixel, the image block type of the image block, and the image region type of the image region, determine or obtain the first probability of occurrence; If the syntax element obtained in the bitstream satisfies the second condition, the first occurrence probability is determined or obtained according to the first occurrence probability calculation strategy and at least one pixel, image block or image region in the reference image region. If the syntax element obtained in the bitstream does not satisfy the second condition, the first occurrence probability is determined or obtained according to the second occurrence probability calculation strategy, which is different from the first occurrence probability calculation strategy, and at least one pixel, image block or image region in the reference image region.

6. The method of claim 4, wherein, Step S22 includes at least one of the following: Based on the first occurrence probability and the scaling weight factor, determine the second occurrence probability, and based on the second occurrence probability, determine or obtain the prediction pattern of the current block; Based on the first and third occurrence probabilities of the same prediction direction, the prediction pattern of the current block is determined or obtained; The scaling weight factor for the first occurrence probability is determined based on the fourth occurrence probability, and the prediction mode for the current block is determined based on the first occurrence probability and the scaling weight factor.

7. The method of claim 5 or 6, wherein, The method for determining or obtaining the scaling weight factor includes at least one of the following: Based on the position of a pixel, an image block, and at least one of the elements in at least one reference image region relative to the current block, a scaling weight factor is determined or obtained. The scaling weight factor is determined or obtained based on at least one of the first image region, first image block and first pixel that is adjacent and / or non-adjacent to at least one of the pixels, image blocks and at least one of the image regions in at least one reference image region; Based on the image region type of at least one image region, and / or the image block type of at least one image block, and / or the pixel type of at least one pixel, determine or obtain the scaling weight factor; Based on at least one pixel in a reference image region, an image block, and at least one of the elements in the image region, the horizontal distance between the current block and the reference image block is determined. Distance, and / or vertical distance, and / or shortest distance, determine or obtain the scaling weight factor; If the syntax elements obtained from the bitstream meet the third condition, the first scaling weight factor calculation strategy is used to determine or obtain the scaling weight factor. If the syntax elements obtained from the bitstream meet the fourth condition, the second scaling weight factor calculation strategy, which is different from the first scaling weight factor calculation strategy, is used to determine or obtain the scaling weight factor.

8. The method of claim 4, wherein, It also includes at least one of the following: Based on at least one of the gradient of pixels in different reference image regions, the area of ​​image blocks or image regions, determine or obtain at least two different occurrence probabilities, and determine or obtain the prediction mode of the current block based on the at least two different occurrence probabilities. The reference image region includes at least one of the left image region located to the left of the current block and the upper image region located above the current block; The prediction pattern of the current block is determined or obtained based on the maximum probability of occurrence of at least one first probability.

9. A processing apparatus, wherein, include: The system includes a memory and a processor, wherein the memory stores an image processing program, and when the image processing program is executed by the processor, it implements the steps of the image processing method as described in claim 1.

10. A storage medium, wherein, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the image processing method as described in claim 1.

Images