Processing method, processing device, and storage medium

Abstract

The application provides a processing method, a processing device and a storage medium. The processing method can be applied to the processing device, and the processing method comprises the following steps: determining or obtaining at least one prediction mode of a current block according to matching information related to the at least one candidate mode and the current block. The technical scheme of the application can match a suitable prediction mode for the current block, and thus can support improving the prediction accuracy of the current block.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to a processing method, processing device, and storage medium. Background Technology

[0002] The existing video coding standard (H.266 / VVC) proposes a video frame coding technique. For example, when encoding and decoding video frames, the protocol divides each frame into different blocks and performs prediction processing and encoding / decoding processing.

[0003] In the process of conceiving and implementing this application, the inventors discovered at least the following problems: in the process of intra-frame prediction and / or inter-frame prediction, it is difficult to match a suitable prediction mode for image blocks, resulting in unsatisfactory prediction results for image blocks.

[0004] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides a processing method, processing device, and storage medium that can match a suitable prediction mode for the current block from at least one candidate mode, thereby supporting the improvement of the prediction accuracy of the current block.

[0006] This application provides a processing method applicable to a processing device, comprising the following steps:

[0007] S10, Based on the matching information of at least one candidate pattern with the current block, determine or obtain at least one prediction pattern for the current block.

[0008] Optionally, candidate patterns are determined or obtained based on at least one of the following:

[0009] The current block corresponds to at least one gradient histogram;

[0010] At least one area histogram corresponding to the current block;

[0011] A histogram of at least one usage count of the predicted mode corresponding to the encoded image block corresponding to the current block;

[0012] At least one gradient operator;

[0013] First pattern list.

[0014] Optionally, the first pattern list is determined or obtained based on at least one of the following:

[0015] At least one reference region and neural network model for the current block;

[0016] Matching information related to at least one reference region of the current block;

[0017] Matching information related to at least one reference region of the current block's sub-blocks;

[0018] The prediction mode corresponding to at least one of the following: the sub-blocks, adjacent blocks, non-adjacent blocks, co-located blocks, and encoded image blocks at preset positions;

[0019] The prediction mode corresponding to at least one of the following: the neighboring blocks, non-neighboring blocks, co-located blocks, and the encoded image blocks at preset positions of the current block's sub-blocks;

[0020] The prediction mode corresponding to the sub-blocks of at least one of the following: neighboring blocks, non-adjacent blocks, co-located blocks, and pre-defined positions of the current block.

[0021] Optionally, the method of determining or obtaining the first pattern list based on at least one reference region and neural network model of the current block includes at least one of the following:

[0022] Based on the pixels of at least one reference region of the current block and the neural network model, determine or obtain a first pattern list;

[0023] Based on the gradient information of at least one reference region of the current block and the neural network model, a first pattern list is determined or obtained.

[0024] Optionally, the gradient histogram and / or area histogram are determined or obtained based on at least one of the following:

[0025] Gradient information of at least one reference region of the current block;

[0026] Area information of at least one reference region for the current block;

[0027] At least one gradient operator corresponding to at least one lookup table;

[0028] At least one pair of gradient operators corresponding to weight information;

[0029] The weight information corresponding to the gradient information of at least one reference region of the current block;

[0030] The current block has at least one reference region and at least one lookup table.

[0031] Optionally, at least one gradient operator includes at least one pair of mutually perpendicular gradient operators.

[0032] Optionally, the gradient information includes at least one of the following:

[0033] Gradient information of pixels in the reference region;

[0034] Overall gradient information for the reference region;

[0035] Gradient information, gradient information of pixels in the reference region, and / or overall gradient information of the reference region, including gradient magnitude and / or gradient direction.

[0036] Optionally, the processing method further includes at least one of the following:

[0037] The first gradient information is determined or obtained based on the first pair of mutually perpendicular gradient operators;

[0038] The second gradient information is obtained by determining or negating the second pair of mutually perpendicular gradient operators;

[0039] The third gradient information is determined or obtained based on the first gradient information and the second gradient information.

[0040] Optionally, at least one reference region of the current block is determined or obtained based on at least one of the following:

[0041] At least one of the following: the pixel above the current block, the non-adjacent pixel above the current block, the pixel to the left of the current block, the non-adjacent pixel to the left of the current block, the pixel above the left of the current block, and the non-adjacent pixel above the left of the current block;

[0042] The current block is selected from at least one of the following: neighboring block, non-neighboring block, sibling block, temporal block, and default block;

[0043] The current block's width, height, block size, and block area must be at least one of these.

[0044] The candidate motion vector or candidate block vector of the current block is determined or the candidate block is obtained.

[0045] This application also provides a processing device, including: a memory and a processor, wherein the memory stores a processing program, and when the processing program is executed by the processor, it implements the steps of any of the processing methods described above.

[0046] This application also provides a storage medium storing a computer program that, when executed by a processor, implements the steps of any of the processing methods described above.

[0047] As described above, the processing method of this application can be applied to a processing device, including: determining or obtaining at least one prediction pattern for the current block based on matching information related to at least one candidate pattern and the current block. Through the technical solution of this application, a suitable prediction pattern can be matched for the current block from at least one candidate pattern, thereby improving the prediction accuracy of the current block. Attached Figure Description

[0048] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0049] Figure 1 A schematic diagram of the hardware structure of a mobile terminal to implement the various embodiments of this application;

[0050] Figure 2 A communication network system architecture diagram provided in this application embodiment;

[0051] Figure 3 A schematic diagram of the hardware structure of a controller 140 provided in this application;

[0052] Figure 4 A schematic diagram of the hardware structure of a network node 150 provided in this application;

[0053] Figure 5 This is a flowchart illustrating the processing method according to the first embodiment;

[0054] Figure 6 This is a schematic diagram of the encoder's encoding process in the image processing method shown in the first embodiment;

[0055] Figure 7 This is a schematic diagram of the decoding process of the decoder in the image processing method according to the first embodiment;

[0056] Figure 8 This is a schematic diagram of the reference area of ​​the DIMD mode in the processing method shown according to the second embodiment;

[0057] Figure 9 This is a schematic diagram of the gradient magnitude in the processing method shown in the second embodiment;

[0058] Figure 10 This is a schematic diagram of the encoded region corresponding to the block to be predicted in the processing method shown in the second embodiment;

[0059] Figure 11 This is a schematic diagram of the decoded region corresponding to the block to be predicted in the processing method shown in the second embodiment;

[0060] Figure 12 This is a schematic diagram of the reference region of the neural network model in the processing method shown in the fourth embodiment;

[0061] Figure 13This is a schematic diagram of the structure of a neural network model based on a fully connected layer, as shown in the fourth embodiment;

[0062] Figure 14 This is a schematic diagram of the structure of a neural network model based on convolutional layers, as shown in the fourth embodiment;

[0063] Figure 15 This is a schematic diagram of the structure of a neural network model based on hybrid convolutional and fully connected layers, as shown in the fourth embodiment.

[0064] Figure 16 This is a schematic diagram of intra-frame prediction direction shown according to the sixth embodiment;

[0065] Figure 17 This is a schematic diagram of the reference region corresponding to the block to be predicted in the image processing method shown in the sixth embodiment;

[0066] Figure 18 This is a schematic diagram of the processing module of the processing device.

[0067] The realization of the objectives, functional features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0068] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0069] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0070] It should be understood that although the terms first, second, third, etc., may be used herein to describe various information, this information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another; for example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if," as used herein, can be interpreted as "when," "when," or "in response to determination." Furthermore, as used herein, the singular forms "a," "an," and "the" are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms "comprising," "including," indicate the presence of the stated feature, step, operation, element, component, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms “or,” “and / or,” and “including at least one of the following” as used in this application may be interpreted as inclusive, or mean any one or any combination thereof. For example, “including at least one of the following: A, B, C” means “any one of the following: A; B; C; A and B; A and C; B and C; A and B and C”, or “A, B or C” or “A, B and / or C” means “any one of the following: A; B; C; A and B; A and C; B and C; A and B and C”. Exceptions to this definition will only occur if the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0071] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0072] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0073] It should be noted that step designations such as S10 are used in this paper to more clearly and concisely describe the corresponding content, and do not constitute a substantial restriction on the order.

[0074] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0075] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0076] The processing device can be implemented in various forms. For example, the processing device described in this application may include processing devices such as mobile phones, servers, tablet computers, laptops, handheld computers, personal digital assistants (PDAs), portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, and fixed terminals such as digital TVs and desktop computers.

[0077] The following description will use a mobile terminal as an example. Those skilled in the art will understand that, apart from elements specifically designed for mobile purposes, the construction according to the embodiments of this application can also be applied to fixed-type terminals.

[0078] Please see Figure 1 This is a schematic diagram of the hardware structure of a mobile terminal implementing various embodiments of this application. The mobile terminal 100 may include: an RF (Radio Frequency) unit 101, a WiFi module 102, an audio output unit 103, an A / V (Audio / Video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111, etc. Those skilled in the art will understand that... Figure 1 The mobile terminal structure shown does not constitute a limitation on the mobile terminal. The mobile terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0079] The following is combined with Figure 1 A detailed introduction to each component of the mobile terminal:

[0080] The radio frequency unit 101 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 110; and / or transmits uplink data to the base station. Typically, the radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, etc. In addition, the radio frequency unit 101 can also communicate with networks and other devices wirelessly. The aforementioned wireless communications may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution), 5G, and 6G.

[0081] WiFi is a short-range wireless transmission technology. Mobile terminals, through the WiFi module 102, can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 1 WiFi module 102 is shown, but it is understood that it is not a necessary component of a mobile terminal and can be omitted as needed without changing the nature of the invention.

[0082] The audio output unit 103 can convert audio data received by the radio frequency unit 101 or the WiFi module 102 or stored in the memory 109 into audio signals and output them as sound when the mobile terminal 100 is in call signal receiving mode, call mode, recording mode, voice recognition mode, broadcast receiving mode, etc. Furthermore, the audio output unit 103 can also provide audio output related to specific functions performed by the mobile terminal 100 (e.g., call signal receiving sound, message receiving sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.

[0083] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos acquired by an image capture device (such as a camera) in video capture mode or image capture mode. The processed image frames can be displayed on the display unit 106. The image frames processed by the GPU 1041 can be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) in operating modes such as telephone call mode, recording mode, and voice recognition mode, and can process such sound into audio data. The processed audio (voice) data can be converted into a format that can be transmitted to a mobile communication base station via the radio frequency unit 101 in telephone call mode. The microphone 1042 can implement various types of noise cancellation (or suppression) algorithms to eliminate (or suppress) noise or interference generated during the reception and transmission of audio signals.

[0084] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Optionally, the light sensor includes an ambient light sensor and a proximity sensor. Optionally, the ambient light sensor can adjust the brightness of the display panel 1061 according to the ambient light level, and the proximity sensor can turn off the display panel 1061 and / or backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc. Other sensors that may be configured in the phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0085] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0086] User input unit 107 can be used to receive input numerical or character information, and generate key signal inputs related to user settings and function control of the mobile terminal. Optionally, user input unit 107 may include touch panel 1071 and other input devices 1072. Touch panel 1071, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near touch panel 1071), and drive corresponding connection devices according to a pre-set program. Touch panel 1071 may include a touch detection device and a touch controller. Optionally, the touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to processor 110, and can receive and execute commands sent by processor 110. In addition, touch panel 1071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may also include other input devices 1072. Optionally, other input devices 1072 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc., without being specifically limited here.

[0087] Optionally, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. Subsequently, the processor 110 provides corresponding visual output on the display panel 1061 based on the type of touch event. Although in Figure 1 In this embodiment, the touch panel 1071 and the display panel 1061 are two independent components to realize the input and output functions of the mobile terminal. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to realize the input and output functions of the mobile terminal. The specific implementation is not limited here.

[0088] Interface unit 108 serves as an interface through which at least one external device can connect to mobile terminal 100; for example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, and so on. Interface unit 108 may be used to receive input from external devices (e.g., data information, power, etc.) and transmit the received input to one or more elements within mobile terminal 100, or it may be used to transmit data between mobile terminal 100 and external devices.

[0089] The memory 109 can be used to store software programs and various data. The memory 109 may primarily include a program storage area and a data storage area. Optionally, the program storage area may store the operating system, applications required for at least one function (such as sound playback, image playback, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory 109 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0090] The processor 110 is the control center of the mobile terminal. It connects various parts of the mobile terminal via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 109, and by calling data stored in the memory 109, it performs various functions and processes data of the mobile terminal, thereby providing overall monitoring of the mobile terminal. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor. Optionally, the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 110.

[0091] The mobile terminal 100 may also include a power supply 111 (such as a battery) that supplies power to various components. Preferably, the power supply 111 can be logically connected to the processor 110 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0092] although Figure 1 As not shown, the mobile terminal 100 may also include a Bluetooth module, etc., which will not be described in detail here.

[0093] To facilitate understanding of the embodiments of this application, the communication network system on which the mobile terminal of this application is based is described below.

[0094] Please see Figure 2 , Figure 2 This application provides a communication network system architecture diagram. The communication network system is an LTE system based on the universal mobile communication technology. The LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203, and the operator's IP services 204, which are connected in sequence.

[0095] Optionally, UE201 can be the aforementioned terminal 100, which will not be described in detail here.

[0096] E-UTRAN202 includes eNodeB2021 and other eNodeB2022, etc. Optionally, eNodeB2021 can connect to other eNodeB2022 via backhaul (e.g., X2 interface), and eNodeB2021 connects to EPC203, providing access from UE201 to EPC203.

[0097] EPC203 may include MME (Mobility Management Entity) 2031, HSS (Home Subscriber Server) 2032, other MMEs 2033, SGW (Serving Gateway) 2034, PGW (Packet Data Network Gateway) 2035, and PCRF (Policy and Charging Rules Function) 2036, etc. Optionally, MME2031 is the control node that handles signaling between UE201 and EPC203, providing bearer and connection management. HSS2032 is used to provide registers to manage functions such as the Home Location Register (not shown in the figure) and stores user-specific information such as service characteristics and data rates. All user data can be sent through SGW2034. PGW2035 can provide UE 201 IP address allocation and other functions. PCRF2036 is the policy and charging control decision point for service data flow and IP bearer resources. It selects and provides available policy and charging control decisions for the policy and charging enforcement function unit (not shown in the figure).

[0098] IP services 204 may include the Internet, intranet, IMS (IP Multimedia Subsystem), or other IP services.

[0099] Although the above description uses the LTE system as an example, those skilled in the art should know that this application is not only applicable to the LTE system, but also to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, 5G and future new network systems (such as 6G), etc., without limitation.

[0100] Figure 3This is a schematic diagram of the hardware structure of a controller 140 provided in this application. The controller 140 includes a memory 1401 and a processor 1402. The memory 1401 is used to store program instructions, and the processor 1402 is used to call the program instructions in the memory 1401 to execute the steps performed by the controller in the first embodiment of the above method. The implementation principle and beneficial effects are similar, and will not be described again here.

[0101] Optionally, the controller further includes a communication interface 1403, which can be connected to the processor 1402 via a bus 1404. The processor 1402 can control the communication interface 1403 to implement the receiving and sending functions of the controller 140.

[0102] Figure 4 This application provides a schematic diagram of the hardware structure of a network node 150. The network node 150 includes a memory 1501 and a processor 1502. The memory 1501 is used to store program instructions, and the processor 1502 is used to call the program instructions in the memory 1501 to execute the steps performed by the first node in the first embodiment of the above method. The implementation principle and beneficial effects are similar, and will not be described again here.

[0103] Optionally, the controller further includes a communication interface 1503, which can be connected to the processor 1502 via a bus 1504. The processor 1502 can control the communication interface 1503 to implement the receiving and sending functions of the network node 150.

[0104] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0105] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk, SSD), etc.

[0106] First Embodiment

[0107] Reference Figure 5 , Figure 5 This is a flowchart illustrating the processing method according to the first embodiment. The processing method of this application embodiment can be applied to a processing device, including:

[0108] Step S10: Determine or obtain at least one predicted pattern for the current block based on the matching information related to at least one candidate pattern with the current block.

[0109] In this embodiment, the processing device can be a smart terminal, such as a mobile phone or computer, or a server, such as a local server or a cloud server. This embodiment and this application primarily use a smart terminal as an example for illustration.

[0110] Optionally, the technical solution of this embodiment can be applied to fields such as image encoding and decoding, video encoding and decoding, hardware video encoding and decoding, dedicated circuit video encoding and decoding, and real-time video encoding and decoding.

[0111] Optionally, the processing device can acquire video image data from a video source, segment each frame of the video image data to obtain at least one image block, and determine the image block to be predicted in the at least one image block as the current block.

[0112] Optionally, the candidate mode is the prediction mode that the current block may adopt during the prediction process. Optionally, in intra-frame prediction, the candidate mode may include: at least one angle prediction mode, at least one non-angle prediction mode, and / or a prediction mode determined or obtained using at least one derived mode.

[0113] Optionally, the derivation mode is a derivation mode used to derive intra-frame prediction modes.

[0114] Optionally, angle prediction mode is a technique for predicting the current pixel block. It generates the predicted block by propagating the values ​​of neighboring pixels along a specific direction. Angle prediction mode is mainly used to process directional textures in images, which can effectively reduce spatial redundancy and improve compression efficiency. In the H.265 / HEVC (High Efficiency Video Coding) standard, the intra-frame prediction mode includes 33 angle prediction modes, which cover different angles from horizontal to vertical, ensuring accurate prediction of various texture directions. In the H.266 / VVC (Versatile Video Coding) standard, the angle modes are expanded to 65, with more densely added directions to more accurately capture edges in natural video.

[0115] Optionally, the non-angle prediction mode can be a prediction mode other than the angle prediction mode. For example, the non-angle prediction mode includes at least one of the following: DC mode, Planar mode, and neural network-based prediction mode.

[0116] Optionally, the derivation mode is a mode used to determine or obtain the prediction mode and / or candidate mode. The derivation mode is a method of deriving the prediction mode matching the current block by analyzing the relevant information of the current block. For example, the derivation mode includes at least one of the following: Decoder side intra mode derivation (DIMD) mode, Occurrence-based Intra Coding (OBIC) mode, and template-based intra mode derivation (TIMD) mode.

[0117] Optionally, the current block is subjected to prediction processing based on at least one prediction pattern of the current block determined or obtained through matching information related to at least one candidate pattern.

[0118] Optionally, matching information refers to information used to evaluate the degree of matching between different prediction modes and the current block and its corresponding at least one reference region. Matching information can be based on various indicators, such as the rate-distortion cost of the prediction results, prediction error, texture similarity, etc., to indicate the degree of matching between different prediction modes and the reference region.

[0119] Optionally, the matching information includes: SAD (Sum of Absolute Differences), SATD (Sum of Absolute Transformed Differences), and / or MRSAD (Mean-Removed Sum of Absolute Differences). Optionally, the smaller the SAD, SATD, and / or MRSAD values, the higher the degree of matching; and / or, the larger the SAD, SATD, and / or MRSAD values, the lower the degree of matching.

[0120] Optionally, the matching information includes: the gradient magnitude value or the sum of the corresponding gradient magnitude values ​​corresponding to the predicted pattern of the gradient histogram. Optionally, the larger the gradient magnitude value, the higher the degree of matching; and / or, the smaller the gradient magnitude value, the lower the degree of matching.

[0121] Optionally, the matching information includes: confidence information, wherein the higher the confidence level, the higher the degree of matching; and / or, the lower the confidence level, the lower the degree of matching.

[0122] Optionally, the matching information between the prediction pattern and / or candidate pattern and the current block can be the matching information between the prediction pattern and / or candidate pattern and at least one reference region of the current block.

[0123] Optionally, based on the matching information of at least one candidate pattern with the current block, the candidate pattern with the highest degree of matching with the current block is determined or obtained, and used as the predicted pattern for the current block.

[0124] Optionally, step S10 includes: determining or obtaining at least one prediction pattern of the current block from the candidate patterns based on at least one candidate pattern of the current block determined or obtained using at least one derivation pattern and matching information related to at least one candidate pattern.

[0125] Optionally, at least one candidate mode of the current block is determined or obtained based on at least one derived mode and pixels in the first reference region of the current block, and at least one predicted mode of the current block is determined or obtained from the candidate modes based on matching information related to the candidate modes and the second reference region of the current block.

[0126] Optionally, the first reference region and the second reference region may be the same or different.

[0127] Optionally, the matching information is the matching information between the predicted pixel value of at least one candidate mode of the second reference region with respect to the current block and the reconstructed pixel value corresponding to the predicted mode adopted by the second reference region.

[0128] Optionally, the reconstructed pixel value corresponding to the prediction mode adopted by the second reference region is the reconstructed pixel value of the reconstructed pixel obtained during the encoding or decoding process of the second reference region itself.

[0129] Optionally, determining or obtaining at least one prediction pattern of the current block based on matching information related to at least one candidate pattern and the current block includes: determining or obtaining first matching information related to at least one prediction pattern and the current block based on at least one derived pattern; determining or obtaining at least one candidate pattern of the current block based on the first matching information; determining second matching information related to a second reference region of the current block based on at least one candidate pattern; and determining or obtaining at least one prediction pattern of the current block from at least one candidate pattern based on the first matching information and / or the second matching information.

[0130] Optionally, determining or obtaining at least one prediction pattern of the current block based on matching information related to at least one candidate pattern includes: determining or obtaining a first pattern list of the current block based on first matching information related to the prediction pattern; and determining or obtaining at least one prediction pattern of the current block based on the first matching information and / or the corresponding second matching information of the candidate patterns in the first pattern list.

[0131] Optionally, the first matching information and / or the second matching information include at least one of the gradient magnitude value or the sum of the corresponding gradient magnitude values ​​corresponding to the predicted mode of the gradient histogram, confidence information, SAD, SATD and / or MRSAD, and the first matching information and the second matching information are different.

[0132] Optionally, the first matching information is the gradient magnitude value or the sum of the corresponding gradient magnitude values ​​corresponding to the prediction mode of the gradient histogram, and the second matching information is at least one of SAD, SATD and / or MRSAD.

[0133] Optionally, the first matching information is confidence information, and the second matching information is at least one of SAD, SATD, and / or MRSAD.

[0134] Optionally, determining or obtaining at least one prediction mode for the current block based on the first matching information and / or the corresponding second matching information of the candidate modes in the first mode list includes: determining or obtaining at least one prediction mode for the current block based on the sorting results and / or comparison results of the first matching information and / or the second matching information.

[0135] Optionally, determining or obtaining the first pattern list for the current block based on the predicted pattern and the first matching information currently relevant includes: determining whether to include the predicted pattern corresponding to the first matching information as a candidate pattern in the first pattern list for the current block based on the sorting results and / or comparison results of the first matching information.

[0136] Optionally, determining or obtaining at least one candidate pattern for the current block based on the first matching information includes: determining whether to use the predicted pattern corresponding to the first matching information as at least one candidate pattern for the current block based on the sorting results and / or comparison results of the first matching information.

[0137] Optionally, the sorting result includes: sorting at least one prediction mode according to the magnitude of the first matching information or the second matching information of at least one prediction mode, determining or obtaining sorting information about the at least one prediction mode, for example, if the matching degree corresponding to the first matching information or the matching degree corresponding to the second matching information of the first prediction mode is higher than the matching degree corresponding to the first matching information or the second matching information of the second prediction mode, the first matching information or the second matching information of the prediction mode is sorted in descending order, then in the sorted mode list, the first prediction mode is located before the second prediction mode; and / or, if the matching degree corresponding to the first matching information or the matching degree corresponding to the second matching information of the first prediction mode is higher than the matching degree corresponding to the first matching information or the second matching information of the second prediction mode, the first matching information or the second matching information of the prediction mode is sorted in ascending order, then in the sorted mode list, the first prediction mode is located after the second prediction mode.

[0138] Optionally, the comparison results include: determining the N prediction modes with the largest or smallest values ​​of the first or second matching information among the prediction modes in the current block, based on the numerical values ​​of the first or second matching information of each prediction mode.

[0139] Optionally, based on the matching information of at least one candidate pattern with the current block, at least one prediction pattern of the current block is determined or obtained, and based on the at least one prediction pattern and at least one weight information, the target prediction result of the current block is determined or obtained.

[0140] Optionally, the weight information includes the weight information corresponding to the prediction model and / or the weight information corresponding to the prediction result.

[0141] Optionally, the weight information may be the proportion of contribution of at least one prediction result and / or at least one prediction mode to the target prediction result. For example, the weight information may be a weight coefficient. Optionally, the weight information may also be a weight vector and a weight matrix containing the weight coefficient, etc., representing indication information used to measure the proportion of different elements (e.g., predicted values) in the result or decision.

[0142] Optionally, the confidence level of at least one prediction result and / or the confidence level of at least one prediction pattern are positively proportional to and / or positively correlated with their corresponding weight information.

[0143] Optionally, the weight information may include at least one of the following: weight coefficients, weight vectors, and weight matrices.

[0144] Optionally, the weight information corresponding to at least one prediction mode is determined or obtained based on the matching information between at least one prediction mode and the current block.

[0145] Optionally, the weight information corresponding to at least one prediction result is determined or obtained based on the matching information between the at least one prediction pattern of the at least one prediction result and the current block.

[0146] Optionally, the weight information corresponding to at least one prediction mode is positively proportional and / or positively correlated with the matching information related to the current block, that is, the higher the degree of matching, the greater the weight value corresponding to the weight information.

[0147] Optionally, the current block is subjected to prediction processing based on at least one prediction mode of the current block and at least one reference region of the current block as determined or obtained.

[0148] Optionally, at least one prediction pattern can be determined or obtained based on the matching information of at least one reference region and at least one candidate pattern related to the current block.

[0149] Optionally, the reference area includes at least one of the following: a reference pixel, a reference block, and a reference template.

[0150] Optionally, the reference region used to determine or obtain matching information related to at least one prediction pattern may be the same as or different from the reference region used to determine or obtain at least one prediction pattern for the current block.

[0151] Optionally, at least one reference region of the current block can be determined or obtained based on the size of the current block.

[0152] Optionally, based on the first target derivation mode, at least one candidate mode is determined or obtained, the first target derivation mode including at least one of the following: DIMD mode, OBIC mode and TIMD mode, and based on the matching information of the at least one candidate mode with the current block, at least one prediction mode of the current block is determined or obtained from the at least one candidate mode.

[0153] Optionally, at least one candidate pattern of a first preset number is determined or obtained according to the first target derivation pattern (e.g., DIMD pattern), and the pattern with the highest matching degree is selected from the at least one candidate pattern according to the matching information related to the reference area of ​​the current block, and determined as the prediction pattern of the current block for prediction processing.

[0154] Optionally, at least one candidate pattern of a first preset number is determined or obtained according to the first target derivation pattern (e.g., DIMD pattern). Based on the matching information of the first preset number of candidate patterns and the reference area of ​​the current block, at least one prediction pattern of a second preset number and the weight information corresponding to each of the first preset number of prediction patterns are determined or obtained from the at least one candidate pattern. The current block is then predicted using at least one prediction pattern to determine or obtain the prediction result corresponding to each of the at least one prediction pattern. Based on the prediction result corresponding to each of the at least one prediction pattern and the weight information corresponding to each of the at least one prediction pattern, the target prediction result of the current block is determined or obtained.

[0155] Optionally, the first preset quantity is greater than or equal to the second preset quantity.

[0156] Optionally, the first preset quantity is 3 to 8, and the second preset quantity is 3.

[0157] Optionally, compared to determining the prediction pattern solely based on the magnitude of DIMD patterns, OBIC patterns, etc., the embodiments of this application first select at least one potentially valid candidate pattern by deriving patterns such as DIMD patterns and OBIC patterns, and then directly verify the actual performance of the pattern by using the matching information between the candidate pattern and the current block to perform secondary screening, so that the determined or obtained prediction pattern has higher accuracy.

[0158] Optionally, based on the first target derivation pattern (e.g., DIMD pattern) of the current block, at least one candidate pattern is determined or obtained; based on the second target derivation pattern (e.g., TIMD pattern), matching information related to at least one candidate pattern is determined or obtained; and based on the matching information related to at least one candidate pattern, at least one prediction pattern of the current block is determined or obtained.

[0159] Optionally, candidate patterns include patterns belonging to a first number of prediction patterns and / or patterns belonging to a second number of prediction patterns, the first number and the second number being the same or different. For example, candidate patterns include prediction patterns among 67 prediction patterns and / or prediction patterns among 131 prediction patterns.

[0160] Optionally, the first number is greater than the second number. By increasing the number of prediction patterns (e.g., angle prediction patterns), the candidate patterns are made to better fit the actual edge direction of the current block, thereby improving the prediction accuracy for the current block.

[0161] Optionally, at least one prediction result of the current block, determined or obtained by predicting the current block through at least one prediction mode, is fused to determine or obtain the target prediction result of the current block.

[0162] Optionally, at least one prediction result of the current block is weighted and fused to determine or obtain the target prediction result of the current block. Optionally, weighted fusion means assigning different weight information to at least one prediction result based on the confidence level of at least one prediction result and / or the confidence level of at least one prediction mode corresponding to at least one prediction result, and performing fusion based on the weight information. For example, if the confidence level of a prediction result and / or its corresponding prediction mode is higher, then the prediction result can be given higher weight information; and / or, if the confidence level of a prediction result and / or its corresponding prediction mode is lower, then lower weight information is given.

[0163] Optionally, at least one prediction result can be fused using weight information to combine the advantages of different prediction models and improve prediction accuracy.

[0164] Optionally, at least one candidate pattern is determined or obtained based on at least one target derivation pattern of the current block determined or obtained through the size of the current block.

[0165] Optionally, at least one candidate mode can be determined or obtained based on the prediction mode corresponding to at least one target derivation mode.

[0166] Optionally, in the intra-frame prediction and / or inter-frame prediction process, using the same derivation mode for image blocks of different sizes results in the candidate mode / prediction mode obtained based on the derivation mode having unsatisfactory prediction effects for image blocks of some sizes, which in turn leads to poor encoding and / or decoding quality in the video encoding and / or decoding process. However, through the technical solution of this application, a suitable target derivation mode can be matched for the current block of different sizes, and then a suitable candidate mode and / or prediction mode can be further matched for the current block based on the target derivation mode, thereby supporting the improvement of the prediction accuracy of the current block.

[0167] Optionally, based on the size of the current block, at least one target derivation pattern of the current block is determined or obtained, and based on the at least one target derivation pattern, at least one candidate pattern is determined or obtained.

[0168] Optionally, if the size of the current block satisfies the first condition, the target derivation mode is the first derivation mode, and at least one candidate mode is determined or obtained based on the first derivation mode.

[0169] Optionally, the size of the current block includes at least one of the following: width, height, aspect ratio, depth, area, resolution, and number of pixels.

[0170] Optionally, the first condition can be a pre-set condition. Optionally, the first condition is not fixed and can be adaptively adjusted according to different scenarios.

[0171] Optionally, the first condition can be set based on the value of the quantization parameter or quantization information.

[0172] Optionally, when the quantization parameter is the first quantization parameter (for example, the value of the quantization parameter is 37), the first condition is that the width × height of the current block is greater than the first threshold. The first threshold can be 1024, 512, etc. For example, the first condition is that the width × height of the current block is greater than 1024, or the first condition is that the width × height of the current block is greater than 512.

[0173] Optionally, when the quantization parameter is the second quantization parameter (for example, the value of the quantization parameter is 22), the first condition is that the width × height of the current block is greater than the second threshold. The second threshold can be 1024, 512, etc. For example, the first condition is that the width × height of the current block is greater than 1024, or the first condition is that the width × height of the current block is greater than 512.

[0174] Optionally, the first quantization parameter and the second quantization parameter can be the same or different, and the first threshold and the second threshold can be the same or different.

[0175] Optionally, the size of the current block satisfies a first condition, including at least one of the following:

[0176] The current block's size is set to the first value;

[0177] The current block size is within the first numerical range;

[0178] The size of the current block is greater than or equal to the first threshold.

[0179] Optionally, the first value can be the size of a pre-set image block, such as a value of at least one of width, height, block size, and block area.

[0180] Optionally, the first value can be a value of the width and / or height of the image patch, for example, the first value can be 32, 64, 128, 256, etc.

[0181] Optionally, the first value can be the area of ​​the image patch, for example, the first value can be 256, 512, 1024, etc.

[0182] Optionally, the first numerical range can be a pre-defined range of the size of the image block, such as a range of at least one of the following: width, height, block size, and block area.

[0183] Optionally, the first numerical range can be a numerical range of the width and / or height of the image patch, for example, the first numerical range is [32, 256].

[0184] Optionally, the first numerical range can be a numerical range of the block area of ​​the image patch, for example, the first numerical range is [256, 1024].

[0185] Optionally, the first threshold may be a pre-set size of the image patch, such as a threshold for at least one of width, height, patch size, and patch area.

[0186] Optionally, the first threshold can be a threshold for the width and / or height of the image patch, for example, the first threshold can be 32, 64, 128, 256, etc.

[0187] Optionally, the first threshold can be a threshold of the block area of ​​the image block, for example, the first threshold can be 256, 512, 1024, etc.

[0188] Optionally, if the size of the current block is greater than or equal to the first threshold, the target derivation mode is the first derivation mode.

[0189] Optionally, the number of prediction modes corresponding to the first derivation mode is a first number. If the size of the current block is greater than or equal to the first threshold, it indicates that the current block belongs to a large image block. Then the target derivation mode is the first derivation mode. Based on the first derivation mode, at least one candidate mode is determined or obtained.

[0190] Alternatively, the first quantity is 35, 67, or 131.

[0191] Optionally, the first quantity is greater than or equal to 35, 67 or 131.

[0192] Optionally, the first derivation mode includes at least one of the following: DIMD mode, OBIC mode, TIMD mode, improved DIMD mode, improved OBIC mode, and improved TIMD mode.

[0193] Optionally, the number of prediction modes corresponding to the improved DIMD mode is greater than the number of prediction modes corresponding to the DIMD mode, and / or the number of prediction modes corresponding to the improved OBIC mode is greater than the number of prediction modes corresponding to the OBIC mode, and / or the number of prediction modes corresponding to the improved TIMD mode is greater than the number of prediction modes corresponding to the TIMD mode.

[0194] Optionally, the number of prediction modes corresponding to each of the DIMD mode, OBIC mode and TIMD mode is 67, and the number of prediction modes corresponding to each of the improved DIMD mode, improved OBIC mode and improved TIMD mode is 131.

[0195] Optionally, the processing method for the improved DIMD mode is different from or at least partially the same as the processing method for the DIMD mode; the processing method for the improved OBIC mode is different from or at least partially the same as the processing method for the OBIC mode; and the processing method for the improved TIMD mode is different from or at least partially the same as the processing method for the TIMD mode.

[0196] Optionally, increasing the number of prediction modes can improve prediction accuracy, but it requires transmitting more mode signaling, leading to increased bitstream overhead. However, in the derivation mode, there is no need to explicitly transmit the mode index. Therefore, increasing the number of prediction modes corresponding to the derivation mode will not increase the signaling volume, but may increase computational complexity. Different sized image patches have different sensitivities to the number of prediction modes. For example, large image patches with a size greater than or equal to the first threshold have a wider range of pixels covered by their reference area. Therefore, they are more sensitive to the texture direction for the accuracy of angle prediction in the prediction mode. Increasing the number of angle prediction modes in the prediction mode can more precisely match the actual texture direction. On the other hand, small image patches with a size less than the first threshold have a limited number of pixels covered by their reference area and stronger local correlation between adjacent pixels. Therefore, they are less sensitive to the number of angle prediction modes in the prediction mode. Thus, using the first derivation mode with a larger number of corresponding prediction modes for large image patches with a size greater than or equal to the first threshold can achieve higher benefits.

[0197] Optionally, in the H.266 / VVC standard, the number of prediction modes is 67, which has low prediction accuracy for large image blocks. Therefore, in this embodiment, when the size of the current block is greater than a first threshold, the target derivation mode of the current block is determined to be the first derivation mode, and the number of prediction modes of the first derivation mode is a first number, which is greater than 67, for example, the first number is 131. Since the number of prediction modes corresponding to the first derivation mode is greater than 67, the angle prediction mode corresponding to the first derivation mode can better fit the actual edge direction of the large image block, thereby improving the prediction accuracy for the current block.

[0198] Optionally, at least one candidate mode can be determined or obtained based on the prediction mode corresponding to the first derivation mode.

[0199] Optionally, the prediction direction corresponding to the prediction mode corresponding to the first derivation mode at least partially covers the prediction directions corresponding to the 67 prediction modes in the H.266 / VVC standard, and increases the number of prediction modes by filling in the angle intervals between the angle prediction modes in the original prediction modes.

[0200] Optionally, the first derivation pattern and its corresponding prediction pattern are associated through a first lookup table.

[0201] Optionally, the first lookup table is a predefined static table that stores the mapping relationship between the first derivation pattern and its corresponding prediction pattern, avoiding real-time calculation. Optionally, the first lookup table can be implemented through storage units.

[0202] Optionally, in the first derivation mode, a prediction mode is determined based on the prediction mode index by mapping to the first lookup table, and is used as a candidate mode. Optionally, the number of the first lookup tables can be one or more.

[0203] Optionally, the first lookup table includes a prediction pattern lookup table and / or a prediction angle lookup table.

[0204] Optionally, the prediction pattern lookup table stores the mapping relationship between the index of the prediction pattern and the corresponding prediction direction.

[0205] Optionally, the predicted angle lookup table stores the specific geometric directions corresponding to the angle prediction pattern, for example, from 0° to +45°.

[0206] In this embodiment, when the size of the current block meets the first condition, the target derivation mode is determined as the first derivation mode. Then, at least one candidate mode is determined or obtained based on the first derivation mode, so as to match the appropriate target derivation mode for the current block of different sizes. Then, based on the target derivation mode, appropriate candidate modes and / or prediction modes can be further matched for the current block, thereby improving the prediction accuracy of the current block and improving the encoding and decoding quality in the video encoding and / or decoding process.

[0207] Optionally, if the size of the current block does not meet the first condition, the target derivation mode is the second derivation mode, and at least one candidate mode of the current block is determined or obtained based on the second derivation mode.

[0208] Optionally, the second derivation mode may be the same as or different from the first derivation mode.

[0209] Optionally, the number of prediction modes corresponding to the second derivation mode and the first derivation mode are different, and / or the derivation mode types to which the second derivation mode and the first derivation mode belong are different.

[0210] Optionally, the size of the current block includes at least one of the following: the width, height, area, and aspect ratio of the current block.

[0211] Optionally, the first condition can be a pre-set condition. Optionally, the first condition is not fixed and can be adaptively adjusted according to different scenarios.

[0212] Optionally, the first condition can be set based on the value of the quantization parameter or quantization information.

[0213] Optionally, when the quantization parameter is the first quantization parameter (for example, the value of the quantization parameter is 37), the first condition is that the width × height of the current block is greater than the first threshold.

[0214] Optionally, the first threshold can be 1024, 512, etc. For example, the first condition is that the width × height of the current block is greater than 1024, or the first condition is that the width × height of the current block is greater than 512.

[0215] Optionally, when the quantization parameter is the second quantization parameter (for example, the value of the quantization parameter is 22), the first condition is that the width × height of the current block is greater than the second threshold.

[0216] Optionally, the second threshold can be 1024, 512, etc. For example, the first condition is that the width × height of the current block is greater than 1024, or the first condition is that the width × height of the current block is greater than 512.

[0217] Optionally, the first quantization parameter and the second quantization parameter can be the same or different, and the first threshold and the second threshold can be the same or different.

[0218] Optionally, the size of the current block does not satisfy the first condition, including at least one of the following:

[0219] The current block's size is not the first value;

[0220] The current block's size is not within the first numerical range;

[0221] The current block size is less than the first threshold.

[0222] Optionally, the first value can be the size of a pre-set image block, such as a value of at least one of width, height, block size, and block area.

[0223] Optionally, the first value can be a value of the width and / or height of the image patch, for example, the first value can be 32, 64, 128, 256, etc.

[0224] Optionally, the first value can be the area of ​​the image patch, for example, the first value can be 256, 512, 1024, etc.

[0225] Optionally, the first numerical range can be a pre-defined range of the size of the image block, such as a range of at least one of the following: width, height, block size, and block area.

[0226] Optionally, the first numerical range can be a numerical range of the width and / or height of the image patch, for example, the first numerical range is [32, 256].

[0227] Optionally, the first numerical range can be a numerical range of the block area of ​​the image patch, for example, the first numerical range is [256, 1024].

[0228] Optionally, the first threshold may be a pre-set size of the image patch, such as a threshold for at least one of width, height, patch size, and patch area.

[0229] Optionally, the first threshold can be a threshold for the width and / or height of the image patch, for example, the first threshold can be 32, 64, 128, 256, etc.

[0230] Optionally, the first threshold can be a threshold of the block area of ​​the image block, for example, the first threshold can be 256, 512, 1024, etc.

[0231] Optionally, if the size of the current block is less than the first threshold, the target derivation mode is the second derivation mode.

[0232] Optionally, the number of prediction modes corresponding to the second derivation mode is the second number. If the size of the current block is less than the first threshold, it indicates that the current block belongs to a small-sized image block. Then the target derivation mode is the second derivation mode. Based on the second derivation mode, at least one prediction mode of the current block is determined or obtained.

[0233] Optionally, the number of prediction modes corresponding to the first derivation mode is the first quantity, and the number of prediction modes corresponding to the second derivation mode is the second quantity, wherein the first quantity and the second quantity are the same or different.

[0234] Optionally, the first quantity and the second quantity are the same, for example, the first quantity and the second quantity are both 35, 67 or 131.

[0235] Optionally, the second quantity is less than the first quantity.

[0236] Alternatively, the second quantity can be 35, 67, or 131.

[0237] Optionally, the second quantity is greater than or equal to 35, 67, or 131.

[0238] Optionally, the second derivation mode includes at least one of the following: DIMD mode, OBIC mode, TIMD mode, improved DIMD mode, improved OBIC mode, and improved TIMD mode.

[0239] Optionally, the number of prediction modes corresponding to the improved DIMD mode is greater than the number of prediction modes corresponding to the DIMD mode, and / or the number of prediction modes corresponding to the improved OBIC mode is greater than the number of prediction modes corresponding to the OBIC mode, and / or the number of prediction modes corresponding to the improved TIMD mode is greater than the number of prediction modes corresponding to the TIMD mode.

[0240] Optionally, the number of prediction modes corresponding to each of the DIMD mode, OBIC mode and TIMD mode is 67, and the number of prediction modes corresponding to each of the improved DIMD mode, improved OBIC mode and improved TIMD mode is greater than 67.

[0241] Optionally, the processing method for the improved DIMD mode is different from or at least partially the same as the processing method for the DIMD mode; the processing method for the improved OBIC mode is different from or at least partially the same as the processing method for the OBIC mode; and the processing method for the improved TIMD mode is different from or at least partially the same as the processing method for the TIMD mode.

[0242] Optionally, the number of predicted modes corresponding to the first derivation mode is a first number, and the number of predicted modes corresponding to the second derivation mode is a second number. The first number being greater than the second number indicates that the number of predicted modes corresponding to the first derivation mode is greater than the number of predicted modes corresponding to the second derivation mode. If the size of the current block satisfies a first condition, for example, if the size of the current block is greater than or equal to a first threshold, indicating that the current block belongs to a large image block, then the target derivation mode for the current block is the first derivation mode, which has a larger number of corresponding predicted modes. At least one candidate mode for the current block is determined or obtained based on the first derivation mode, and the candidate mode belongs to the first... One of the prediction modes under a certain number is used to meet the prediction requirements of large image blocks where the accuracy of angle prediction in the prediction mode is more sensitive to the texture direction. It can more finely match the actual texture direction of the current block and improve the prediction accuracy. If the size of the current block does not meet the first condition, for example, if the size of the current block is less than the first threshold, it indicates that the current block belongs to a small image block. Then the target derivation mode of the current block is the second derivation mode with fewer corresponding prediction modes. At least one candidate mode of the current block is determined or obtained based on the second derivation mode. The candidate mode belongs to one of the prediction modes under the second number to avoid increasing the computational complexity.

[0243] Optionally, at least one candidate mode can be determined or obtained based on the prediction mode corresponding to the second derivation mode.

[0244] Optionally, the first quantity is 35, 67 or 131, for example, the first quantity is 131.

[0245] Optionally, the first quantity is greater than 35, 67, or 131.

[0246] Alternatively, the second quantity can be 35, 67, or 131, for example, the second quantity can be 67.

[0247] Optionally, the second quantity is less than or equal to 35, 67, or 131.

[0248] Optionally, the first quantity and the second quantity are the same, that is, the number of prediction patterns corresponding to the first derivation pattern is the same as the number of prediction patterns corresponding to the second derivation pattern. For example, the number of prediction patterns corresponding to the first derivation pattern and the second derivation pattern is the same, but the types of prediction patterns corresponding to each are at least partially different.

[0249] Optionally, the first quantity is greater than the second quantity, and the second quantity is greater than or equal to 67.

[0250] Optionally, the number of prediction modes corresponding to the second derivation mode is the second number, which is 67, and the number of prediction modes corresponding to the first derivation mode is the first number, which is 131.

[0251] Optionally, the prediction direction corresponding to the prediction mode corresponding to the second derivation mode at least partially covers the prediction directions corresponding to the 67 prediction modes in the H.266 / VVC standard, and increases the number of prediction modes by filling in the angle intervals between the angle prediction modes in the original prediction modes.

[0252] Optionally, the second derivation pattern and its corresponding prediction pattern are associated through a second lookup table.

[0253] Optionally, the second lookup table is a predefined static table that stores the mapping relationship between the second derivation pattern and its corresponding prediction pattern, avoiding real-time calculation. Optionally, the first lookup table can be implemented through storage units.

[0254] Optionally, in the second derivation mode, a prediction mode is determined based on the prediction mode index by mapping to the prediction mode index through the second lookup table, and is used as a candidate mode.

[0255] Optionally, the second lookup table includes a prediction pattern lookup table and / or a prediction angle lookup table, and the number of second lookup tables can be one or more.

[0256] Optionally, the prediction pattern lookup table stores the mapping relationship between the index of the prediction pattern and the corresponding prediction direction.

[0257] Optionally, the predicted angle lookup table stores the specific geometric directions corresponding to the angle prediction pattern, for example, from 0° to +45°.

[0258] In this embodiment, when the size of the current block does not meet the first condition, the target derivation mode is determined to be the second derivation mode. Then, at least one candidate mode is determined or obtained based on the second derivation mode. This achieves the matching of a suitable target derivation mode for the current block of different sizes. Furthermore, based on the target derivation mode, a suitable candidate mode and / or prediction mode can be further matched for the current block. This can improve the prediction accuracy of the current block and improve the encoding and decoding quality in the video encoding and / or decoding process.

[0259] Reference Figure 6 When the processing device is an encoder on the encoding side, the encoder can receive video data from a video source, such as receiving video images from the video source, determining the image to be predicted in the video images, dividing the image to be predicted into at least one image block, and using the temporal and / or spatial correlation between video images, performing prediction processing on each of the at least one image block, including intra-frame prediction processing and / or inter-frame prediction processing, and the intra-frame prediction processing and / or inter-frame prediction processing includes at least one derivation mode of prediction mode and / or at least one prediction mode. For the prediction mode, the encoder uses, for example, rate-distortion cost to determine the prediction mode finally adopted by each of the at least one image block. For example, it calculates the rate-distortion cost corresponding to each prediction mode or the rate-distortion cost of combining several prediction methods to determine the minimum rate-distortion cost from at least one rate-distortion cost. The prediction mode or combination of prediction modes corresponding to the minimum rate-distortion cost is the prediction mode finally adopted by the image block. At least one prediction mode of the current block can be determined or obtained based on the matching information of at least one candidate mode related to the current block.

[0260] Optionally, after determining or obtaining at least one prediction mode for the image block to be predicted (i.e., the current block), the image block to be predicted is processed by the at least one prediction mode to determine or obtain the prediction block (i.e., the prediction result) of the image block to be predicted.

[0261] Optionally, a residual block between the predicted block and the current block can be calculated. The residual block can be transformed and quantized, and then encoded by an entropy encoder to form an encoded bit stream.

[0262] Optionally, the encoded bitstream may include prediction parameters corresponding to a defined prediction mode and related side information.

[0263] Optionally, the prediction parameters are entropy-encoded and then packed into the encoded bitstream.

[0264] Optionally, the prediction parameters include indication information of the prediction mode.

[0265] Optionally, the transformed and quantized residual block can be added to the corresponding prediction data (such as the prediction block) obtained using the prediction mode after inverse quantization and inverse transformation to obtain the reconstruction block. After obtaining the reconstruction block, the loop filtering module performs loop filtering on the reconstruction block according to the filter control parameters to reduce distortion.

[0266] Optionally, after performing loop filtering, the reconstructed block after loop filtering is stored according to the encoded image buffer.

[0267] Reference Figure 7 When the processing device is a decoder on the decoding side, after receiving the encoded bitstream, the decoder's entropy decoding unit parses and decodes the encoded bitstream to obtain transform coefficients. The decoder's inverse transform unit and inverse quantization unit perform inverse transform and inverse quantization processing on the transform coefficients to obtain residual blocks.

[0268] Optionally, the decoder's entropy decoding unit parses and decodes the encoded bitstream to obtain prediction data, such as prediction parameters and related auxiliary information.

[0269] Optionally, the decoder's prediction processing unit performs prediction processing using prediction parameters to determine the prediction block corresponding to the residual block.

[0270] Optionally, the prediction processing includes intra-frame prediction processing and / or inter-frame prediction processing, and the intra-frame prediction processing and / or inter-frame prediction processing includes a combination of at least one derivation mode and / or at least one prediction mode.

[0271] Optionally, the processing method includes: determining or obtaining at least one predicted pattern for the current block based on matching information related to at least one candidate pattern with the current block.

[0272] Optionally, the image block to be predicted is predicted according to at least one prediction mode to determine or obtain the prediction block of the image block to be predicted. The obtained residual block and the corresponding prediction block (including the predicted luminance block and the predicted chrominance block) are added together to obtain the reconstructed block. The loop filtering unit of the decoder performs loop filtering on the reconstructed block to reduce distortion and improve video quality.

[0273] Optionally, the processing method further includes: the reconstructed blocks after loop filtering are further combined into a decoded image and stored in a decoded image buffer or output as a decoded video signal.

[0274] Optionally, when the processing device is an encoder, the initially obtained prediction value can be the prediction value obtained in the corresponding prediction mode, which can be directly used in the rate-distortion cost process.

[0275] Optionally, when the processing device is a decoder, the initially obtained prediction value can be the prediction value obtained through the prediction mode corresponding to the block to be predicted (i.e., the image block located in the decoding end) indicated by the syntax elements parsed in the bitstream.

[0276] Optionally, the predicted block can be used as the target image block, the residual block between the target image block and the current block can be calculated, and then encoded by an entropy encoder through transformation and quantization to form an encoded bitstream. Alternatively, the predicted block can be processed accordingly, for example, by using other models, and the processed image block can be used as the target image block, and the steps of calculating the residual block between the target image block and the current block and subsequent steps can be performed.

[0277] In this embodiment, a primary screening is performed using candidate patterns to narrow down the selection range of prediction patterns for the current block, selecting at least one potentially effective candidate pattern. Then, the actual performance of the pattern is directly verified through the matching information between the candidate pattern and the current block, so as to perform a secondary screening and make the determined or obtained prediction pattern more accurate.

[0278] Second Embodiment

[0279] Based on the first embodiment described above, a second embodiment is proposed.

[0280] In this embodiment, the candidate pattern is determined or obtained according to at least one of the following methods one to five:

[0281] Method 1: At least one gradient histogram corresponding to the current block.

[0282] Optionally, step S10 includes: determining or obtaining at least one candidate mode based on at least one gradient histogram corresponding to the current block, and determining or obtaining at least one prediction mode for the current block based on the matching information related to the at least one candidate mode and the current block.

[0283] Optionally, a gradient histogram is a statistical tool used to describe the gradient magnitude distribution of the current block in different directions. It calculates the gradient magnitude value of each pixel in at least one reference region of the current block in the direction corresponding to the prediction mode in each frame, and statistically analyzes the distribution of the gradient magnitude values ​​to obtain a histogram containing the gradient magnitude values ​​corresponding to each prediction direction.

[0284] Optionally, in the intra-frame prediction prediction modes, different prediction modes correspond to different prediction directions. For example, the vertical mode corresponds to the vertical direction, the horizontal mode corresponds to the horizontal direction, and the diagonal mode corresponds to the diagonal direction. Each prediction mode has a specific direction to describe its prediction direction.

[0285] Optionally, for each prediction direction, the sum of the gradient magnitude values ​​of each pixel in at least one reference region of the current block in that direction is calculated. This sum reflects the overall gradient strength of the current block in that prediction direction. The gradient histogram records the sum of the gradient magnitude values ​​corresponding to each prediction direction.

[0286] Optionally, based on at least one gradient histogram corresponding to the current block, it is determined whether the sum of gradient magnitude values ​​related to at least one prediction direction corresponding to the prediction mode contained in the gradient histogram satisfies the second condition. If the preset condition is satisfied, at least one candidate mode is determined or obtained.

[0287] Optionally, the second condition may be whether the sum of gradient magnitudes is one of the sums of the N (e.g., N is 3) gradient magnitude values ​​with the largest gradient magnitudes in the gradient histogram. For example, at least one candidate mode may be determined or obtained based on whether the sum of gradient magnitude values ​​related to at least one prediction direction corresponding to the prediction mode contained in at least one gradient histogram is one of the sums of the N gradient magnitude values ​​with the largest gradient magnitudes in the gradient histogram.

[0288] Optionally, the sums of the gradient magnitude values ​​in the gradient histogram are sorted in ascending or descending order to determine or obtain the sums of the first N gradient magnitude values ​​in ascending order or the sums of the last N gradient magnitude values ​​in descending order. For example, after sorting in ascending order, the prediction modes corresponding to the prediction directions whose sums of gradient magnitude values ​​are in the top 3 are determined as candidate modes.

[0289] Optionally, the gradient magnitude values ​​in the gradient histogram are compared to determine or obtain the sum of the N gradient magnitude values ​​with the largest gradient magnitude in the gradient histogram. Based on the prediction mode corresponding to the sum of the N gradient magnitude values ​​with the largest gradient magnitude, at least one candidate mode for the current block is determined or obtained.

[0290] Optionally, a gradient histogram of at least one reference region with respect to the intra-prediction direction can be determined or obtained based on the gradient magnitude and gradient direction of the pixels in at least one reference region of the current block, the corresponding intra-prediction direction can be determined based on the gradient histogram, and at least one candidate mode can be determined or obtained.

[0291] Optionally, at least one reference region of the current block includes at least one of: reference pixels, reference blocks, and reference templates. The reference region refers to a set of neighboring pixels surrounding the current block. These neighboring pixels can provide important clues about the edges and texture orientation of the block to be predicted. Therefore, the target inference mode can be analyzed and inferred based on these pixels to derive the intra-frame prediction mode of the current block.

[0292] Optionally, the processing method in the embodiments of this application can be DIMD mode and / or improved DIMD mode.

[0293] Reference Figure 8 The processing steps of the DIMD mode, the processing steps of the improved DIMD mode, and / or the determination or acquisition method of the gradient histogram include: determining a reference template adjacent to the block to be predicted (the current block) above and to the left of the block to be predicted, namely three pixel lines / pixel rows / pixel columns on the left and above the block to be predicted; then taking a pixel in the middle line (e.g., pixel A) as the pixel for calculating the gradient; by calculating the gradient direction of at least one pixel in the middle line, as well as the magnitude of the horizontal and vertical gradients, the gradient direction of at least one pixel and the gradient magnitude value corresponding to the gradient can be obtained.

[0294] Optionally, the gradient magnitude value is the sum of the absolute values ​​of the horizontal gradient and the vertical gradient. If the gradient magnitude values ​​with the same gradient direction in at least one pixel are added together, the sum of the gradient magnitude values ​​corresponding to that gradient direction can be obtained.

[0295] Optionally, a histogram of gradient magnitude values ​​for different gradient directions of at least one pixel can be constructed, and the prediction direction perpendicular to the gradient direction of the maximum gradient magnitude value can be used as the prediction direction of the intra-prediction mode and / or the prediction direction of the candidate mode for the current block.

[0296] Optionally, the horizontal gradient Gx and vertical gradient Gy can be calculated using the 3x3 horizontal Sober operator and the vertical Sober operator. For example, the horizontal gradient Gx and vertical gradient Gy of a pixel x 4 in a pixel line can be calculated according to the following formulas (I) and (II).

[0297]

[0298] Optionally, A can be a matrix consisting of nine pixels, centered on pixel x4, and including the pixel x1 above it, the pixel x3 to its left, the pixel x7 below it, the pixel x5 to its right, the pixel x0 to its upper left, the pixel x6 to its lower left, the pixel x2 to its upper right, and the pixel x8 to its lower right, as shown in Formula (III) below.

[0299]

[0300] Optionally, the magnitude of gradient G is the sum of the absolute values ​​of the horizontal gradient and the vertical gradient, and its calculation formula is shown in formula (iv).

[0301] Formula (IV) is G = |Gx| + |Gy|.

[0302] Alternatively, the gradient direction of a pixel can be calculated using arctan(Gx / Gy) or arctan(Gy / Gx).

[0303] Optionally, since each gradient direction corresponds to a specific gradient direction range, and each gradient direction range corresponds to the prediction direction of an intra-frame prediction mode, for at least one pixel in the pixel line, the gradient magnitude values ​​with the same gradient direction range in at least one pixel can be added together to obtain the sum of the gradient magnitude values ​​corresponding to the gradient direction range.

[0304] Alternatively, the sum of the gradient magnitude values ​​of the prediction direction of the corresponding intra-frame prediction mode can also be obtained.

[0305] Reference Figure 9 It includes the gradient magnitude values ​​corresponding to the prediction directions of each intra-frame prediction mode, and optionally, based on... Figure 9 The final intra-frame prediction mode selected was mode 30.

[0306] Optionally, the number of prediction modes corresponding to the DIMD mode is 67. In this embodiment of the application, the number of prediction modes corresponding to the improved DIMD mode is greater than 67, for example, 131.

[0307] Optionally, the prediction direction corresponding to the prediction mode corresponding to the DIMD mode is improved to cover at least part of the prediction direction corresponding to the 67 prediction modes corresponding to the DIMD mode, and the number of prediction modes is increased by filling the angle interval between the angle prediction modes in the original prediction mode.

[0308] In this embodiment, irrelevant prediction modes are quickly eliminated by using at least one gradient histogram, thus achieving a first screening of prediction modes and determining or obtaining at least one candidate mode. Then, the actual performance of the mode is directly verified by the matching information of the candidate mode and the current block, so as to perform a second screening, making the determined or obtained prediction mode more accurate. The gradient histogram can accurately capture the dominant prediction direction (such as vertical, horizontal or diagonal) related to at least one reference region of the current block, thereby matching a suitable candidate mode and / or prediction mode for the current block.

[0309] Method 2: At least one area histogram corresponding to the current block.

[0310] Optionally, step S10 includes: determining or obtaining at least one candidate pattern based on at least one area histogram corresponding to the current block, and determining or obtaining at least one prediction pattern for the current block based on the matching information between the at least one candidate pattern and the current block.

[0311] Optionally, the area histogram is used to describe at least one reference region of the current block, such as the area magnitude distribution of adjacent and / or non-adjacent coded blocks in different frames in the prediction direction. Optionally, the reference region can be an encoded region or a decoded region.

[0312] Optionally, an area histogram and / or statistical results can be generated by analyzing the relationship between the area magnitude value of each coded block (e.g., the size of the block or the texture coverage) and the intra-frame prediction direction.

[0313] Optionally, the area amplitude value can represent the size of the coding block, the texture coverage, or other area-related features. It reflects the texture distribution intensity of the coding block in a specific direction. Each coding block has a corresponding intra-prediction direction (e.g., vertical, horizontal, diagonal, etc.). The area histogram statistically analyzes the area amplitude values ​​in these directions. By analyzing the distribution of area amplitude values ​​in different intra-prediction directions recorded on the area histogram, it can be determined which prediction directions are statistically more consistent with the texture features of the current block.

[0314] Optionally, the processing method in the embodiments of this application can be OBIC mode and / or improved OBIC mode.

[0315] Optionally, at least one candidate mode is determined or obtained based on the prediction mode corresponding to the prediction direction in which the area amplitude value in the area histogram satisfies the third condition.

[0316] Optionally, the third condition can be an area amplitude value that ranks high in position or order after the area amplitude value is sorted according to a preset sorting rule, and / or the third condition can be an area amplitude value that ranks in a preset position (e.g., the preset position is the 1st) or a preset order range (e.g., the preset order range is 1 to 3rd) after the area amplitude value is sorted according to a preset sorting rule. For example, the preset sorting can be a sorting rule that arranges the area amplitude values ​​from largest to smallest, and the prediction mode corresponding to the prediction direction where the area amplitude value in the area histogram is in the top 3 is determined as the candidate mode.

[0317] Optionally, the basic principle of the improved OBIC mode proposed in this application may be to determine at least one candidate mode by analyzing the use of intra-prediction modes in adjacent coding blocks and / or non-adjacent coding blocks.

[0318] Optionally, the implementation process of the improved OBIC mode may include: determining the intra-prediction mode usage of at least one reference region of the current block (including at least one adjacent coding block and / or non-adjacent coding block of the current block), calculating the area amplitude value and intra-prediction direction of at least one reference region, determining at least one area amplitude histogram or statistical result, and determining at least one prediction direction of the current block based on the area amplitude histogram or statistical result.

[0319] Optionally, determining the intra-prediction mode usage of a coded block includes: determining a coded region and / or a decoded region, determining the intra-prediction mode usage of a coded block in the coded region, and / or determining the intra-prediction mode usage of a decoded block in the decoded region.

[0320] Optionally, the encoding unit includes an encoding block of three color components, which include a luminance component and two chrominance components.

[0321] Optionally, determining the intra-prediction mode usage of at least one reference region of the current block (including at least one adjacent coded block and / or non-adjacent coded block of the current block) may include: determining at least one coded region or at least one decoded region, determining the intra-prediction mode usage of coded blocks in the coded region, and / or determining the intra-prediction mode usage of decoded blocks in the decoded region, for example, such as Figure 10 As shown, in the encoded region, there are coded blocks 4 and 6 adjacent to the block to be predicted (i.e., the current block), and coded blocks 1, 2, 3, 5, 7, and 8 that are not adjacent to the block to be predicted. Figure 11 As shown, in the decoded region there are decoded blocks 4 and 6 that are adjacent to the block to be predicted, and decoded blocks 1, 2, 3, 5, 7 and 8 that are not adjacent to the block to be predicted.

[0322] Optionally, the number of prediction modes corresponding to the OBIC mode is 67. In this embodiment of the application, the number of prediction modes corresponding to the improved OBIC mode is greater than 67, for example, 131.

[0323] Optionally, the prediction direction corresponding to the prediction mode corresponding to the OBIC mode is improved to cover at least part of the prediction direction corresponding to the 67 prediction modes corresponding to the OBIC mode, and the number of prediction modes is increased by filling the angle interval between the angle prediction modes in the original prediction mode.

[0324] In this embodiment, irrelevant prediction modes are quickly eliminated by using at least one area histogram, thus achieving a first screening of prediction modes and determining or obtaining at least one candidate mode. Then, the actual performance of the mode is directly verified by the matching information of the candidate mode and the current block, so as to perform a second screening, making the determined or obtained prediction mode more accurate. The area histogram can accurately capture the dominant prediction direction related to at least one reference region of the current block, thereby matching a suitable candidate mode and / or prediction mode for the current block.

[0325] Method 3: A histogram of at least one usage count of the predicted mode corresponding to the encoded image block corresponding to the current block.

[0326] Optionally, step S10 includes: determining or obtaining at least one candidate mode based on at least one histogram of the number of uses of the predicted mode corresponding to the encoded image block corresponding to the current block, and determining or obtaining at least one predicted mode of the current block based on the matching information of the at least one candidate mode and the current block.

[0327] Optionally, the encoded image block corresponding to the current block is an adjacent or reference image block that has been encoded (or decoded) before the current block. These blocks can provide spatial and / or temporal prediction information for the prediction of the current block.

[0328] Optionally, the prediction mode usage frequency histogram corresponding to the encoded image block records the usage frequency of different intra-frame prediction modes in the encoded image block. By using the usage frequency of different intra-frame prediction modes recorded in the frequency histogram, it is possible to determine which prediction modes are frequently used in the encoded region.

[0329] Optionally, step S10 includes: determining or obtaining at least one candidate pattern based on the prediction pattern corresponding to the prediction direction whose usage frequency satisfies the fourth condition in the usage frequency histogram.

[0330] Optionally, the fourth condition can be the number of times the usage count is ranked higher in position or order after sorting according to a preset sorting rule, and / or the fourth condition can be the number of times the usage count is ranked in a preset position (e.g., the preset position is the 1st place) or a preset order range (e.g., the preset order range is 1 to 3rd place) after sorting according to a preset sorting rule. For example, the preset sorting can be a sorting rule that arranges the usage count from largest to smallest, and the prediction mode corresponding to the prediction direction with the top 3 usage counts in the usage count histogram is determined as the candidate mode.

[0331] In this embodiment, irrelevant prediction modes are quickly eliminated by using at least one usage histogram, thus achieving a first screening of prediction modes and determining or obtaining at least one candidate mode. Then, the actual performance of the mode is directly verified by the matching information of the candidate mode and the current block, so as to perform a second screening, making the determined or obtained prediction mode more accurate. By using the usage histogram, the prediction modes with high usage frequency in the encoded image blocks related to the current block can be accurately captured, thereby matching the current block with appropriate candidate modes and / or prediction modes.

[0332] Method 4: At least one gradient operator.

[0333] Optionally, step S10 includes: determining or obtaining at least one candidate mode based on at least one gradient operator, and determining or obtaining at least one prediction mode for the current block based on the matching information of the at least one candidate mode and the current block.

[0334] Optionally, a gradient operator is a tool and / or method for determining or obtaining at least one gradient information, capable of extracting pixel gradient information from an image through convolution operations or other mathematical operations.

[0335] Optionally, gradient information includes: gradient direction and / or gradient magnitude.

[0336] Optionally, at least one gradient operator includes at least one pair of mutually perpendicular gradient operators.

[0337] In this embodiment, irrelevant prediction modes are quickly eliminated by at least one gradient operator, thereby achieving a first screening of prediction modes and determining or obtaining at least one candidate mode. Then, the actual performance of the mode is directly verified by the matching information of the candidate mode and the current block, so as to perform a second screening and make the determined or obtained prediction mode have higher accuracy.

[0338] Method 5, First Pattern List.

[0339] Optionally, step S10 includes: determining or obtaining at least one candidate pattern based on the first pattern list, and determining or obtaining at least one prediction pattern for the current block based on the matching information of the at least one candidate pattern and the current block.

[0340] Optionally, step S10 includes: determining or obtaining at least one candidate mode based on at least one gradient histogram corresponding to the current block, at least one area histogram corresponding to the current block, at least one usage count histogram of the prediction mode corresponding to the encoded image block corresponding to the current block, at least one gradient operator corresponding to at least one lookup table, and at least one item in the first mode list; and determining or obtaining at least one prediction mode of the current block based on the matching information related to the at least one candidate mode and the current block.

[0341] Optionally, the first pattern list is a set for storing at least one prediction pattern, providing multiple possible pattern options for the prediction of the current block. By comparing the prediction effects of different patterns in the first pattern list, a suitable prediction pattern can be matched for the current block to improve the prediction accuracy for the current block.

[0342] Optionally, the first pattern list is at least one pattern list, where the prediction patterns in the at least one pattern list are different or at least partially the same.

[0343] Optionally, the first mode list includes an MPM (Most Probable Modes) list. Optionally, the first mode list in this embodiment can be an improved MPM list.

[0344] Optionally, the MPM list is a collection of several prediction patterns that are considered to be the most likely prediction patterns to apply to the current block. The determination or acquisition of the MPM list is based on the neighborhood information of the current block (such as the prediction patterns of the blocks above and to the left) and statistical information (such as the probability of pattern occurrence). In this way, the MPM list can reduce the amount of pattern information that the encoder needs to transmit, while improving the prediction accuracy of the decoder for the prediction patterns.

[0345] Optionally, the first pattern list includes at least one candidate pattern.

[0346] Optionally, in intra-frame prediction, the candidate modes in the first mode list may include: at least one angular prediction mode, at least one non-angular prediction mode, and / or a derived mode of at least one prediction mode.

[0347] Optionally, the candidate patterns in the first pattern list include prediction patterns belonging to a first number of prediction patterns and / or prediction patterns belonging to a second number of prediction patterns.

[0348] Optionally, a first mode list can be determined or obtained based on gradient information of at least one reference region of the current block.

[0349] Optionally, a first pattern list can be determined or obtained based on at least one reference region of the current block and the neural network model.

[0350] Optionally, a first pattern list can be determined or obtained based on the pixels of at least one reference region of the current block and the neural network model.

[0351] Optionally, a first pattern list can be determined or obtained based on the pixels, gradient information, and neural network model of at least one reference region of the current block.

[0352] Optionally, by using at least one reference region of the current block, the pixels and / or gradient information of the at least one reference region as input to the neural network model, at least one prediction mode and / or the confidence level of at least one prediction mode under a first quantity and / or a second quantity are determined or obtained, and a first mode list is determined or obtained based on the confidence level of at least one prediction mode and / or at least one prediction mode output by the neural network model.

[0353] Optionally, based on the confidence level of at least one predicted pattern determined or obtained by the neural network model, at least one candidate pattern is determined or obtained from at least one predicted pattern under a first number and / or a second number of outputs of the neural network model, and a first pattern list is determined or obtained based on the at least one candidate pattern.

[0354] Optionally, confidence level is a quantitative indicator of the degree of certainty the model has about the current prediction result.

[0355] In this embodiment, the first pattern list can provide candidate patterns with good fit for the current block, effectively narrowing the search range of candidate patterns, improving processing efficiency, and supporting the improvement of the prediction effect of the current block.

[0356] Third Embodiment

[0357] Based on any of the above embodiments, a third embodiment is proposed.

[0358] In this embodiment, the gradient histogram and / or area histogram are determined or obtained according to at least one of the following methods A1 to A6:

[0359] Method A1: Gradient information of at least one reference region of the current block.

[0360] Optionally, step S10 includes: determining or obtaining at least one gradient histogram corresponding to the current block based on gradient information of at least one reference region of the current block; determining or obtaining at least one candidate mode based on the at least one gradient histogram; and determining or obtaining at least one prediction mode of the current block based on matching information related to the at least one candidate mode and the current block.

[0361] Optionally, the gradient information includes gradient direction and / or gradient magnitude. The gradient direction represents the direction in which the gray value changes the fastest at a pixel in the image (e.g., at least one reference region), i.e., the direction perpendicular to the edge. The gradient magnitude represents the intensity of the gray value change at that pixel, i.e., the salience of the edge.

[0362] Optionally, the gradient information of at least one reference region includes the gradient information of pixels in at least one reference region, and the gradient information of pixels in at least one reference region includes the quantized gradient direction.

[0363] Optionally, the initial gradient direction of a pixel in at least one reference region is determined or obtained, and the quantized gradient direction of the pixel in at least one reference region is determined or obtained based on the angle difference between the determined or obtained initial gradient direction and the gradient direction adjacent to the initial gradient direction in at least one lookup table.

[0364] Alternatively, the angle difference can be information representing the angular differences related to the gradient direction.

[0365] Optionally, the initial gradient direction is compared with the gradient directions adjacent to the left and right of the initial gradient direction in at least one lookup table. If the angle difference between the initial gradient direction and the left adjacent gradient direction is less than the angle difference between the initial gradient direction and the right adjacent gradient direction, then the quantized gradient direction of the pixel is the left adjacent gradient direction; and / or, if the angle difference between the initial gradient direction and the left adjacent gradient direction is greater than the angle difference between the initial gradient direction and the right adjacent gradient direction, then the quantized gradient direction of the pixel is the right adjacent gradient direction.

[0366] Optionally, the initial gradient direction of the pixel is determined or obtained, the initial gradient direction information related to the initial gradient direction of the pixel is determined or obtained, and the quantized gradient direction information of the pixel is determined or obtained based on the difference between the determined or obtained initial gradient direction information and the gradient direction information adjacent to the initial gradient direction information in at least one lookup table.

[0367] Optionally, the initial gradient direction information is compared with the gradient direction information that is adjacent to the initial gradient direction information on the left and right sides in at least one lookup table. If the difference between the initial gradient direction information and the gradient direction information adjacent to the left side is less than the difference between the initial gradient direction information and the gradient direction information adjacent to the right side, then the quantized gradient direction information of the pixel is the gradient direction information adjacent to the left side; and / or, if the difference between the initial gradient direction information and the gradient direction information adjacent to the left side is greater than the difference between the initial gradient direction information and the gradient direction information adjacent to the right side, then the quantized gradient direction information of the pixel is the gradient direction information adjacent to the right side.

[0368] Optionally, the gradient direction information is the tangent of the gradient direction or a scaled tangent.

[0369] Optionally, if the gradient direction X in the gradient information is located between the first gradient direction and the second gradient direction in at least one lookup table, then at least one gradient histogram is determined or obtained based on the angle difference or difference between the gradient direction information x and the first gradient direction information and the second gradient direction information.

[0370] Optionally, if the gradient direction information x in the gradient information is located between the first gradient direction information and the second gradient direction information in at least one lookup table, then the gradient direction X is decomposed into gradient direction A and gradient direction B according to the angle difference or difference between the gradient direction information and the first gradient direction information and the second gradient direction information, and at least one gradient histogram corresponding to the current block is determined or obtained according to the gradient direction A and gradient direction B, as well as the gradient magnitude MA and gradient magnitude MB.

[0371] Optionally, based on the angle difference or difference between the gradient direction information x corresponding to the gradient direction X and the first gradient direction information, a first weight for gradient direction A is determined or obtained; based on the angle difference or difference between the gradient direction information x corresponding to the gradient direction X and the second gradient direction information, a second weight for gradient direction B is determined or obtained; based on the first weight and the gradient magnitude M corresponding to the gradient direction X, the gradient magnitude MA of gradient direction A is determined or obtained; based on the second weight and the gradient magnitude M corresponding to the gradient direction X, the gradient magnitude MB of gradient direction B is determined or obtained. For example, the gradient magnitude MA is equal to the first weight * the gradient magnitude M, and the gradient magnitude MB is equal to the second weight * the gradient magnitude M.

[0372] Optionally, gradient direction A is the first gradient direction mentioned above, and gradient direction B is the second gradient direction mentioned above. Optionally, the first weight is equal to the difference between gradient direction information x and second gradient direction information, or the difference between the first gradient direction information and second gradient direction information. The second weight is equal to the difference between gradient direction information x and first gradient direction information, or the difference between the first gradient direction information and second gradient direction information.

[0373] Optionally, if the gradient direction in the gradient information is located between the first direction and the second direction in at least one lookup table, then the weight information corresponding to the first direction and the second direction is determined or obtained according to the angle difference between the gradient direction and the first direction and the second direction, and at least one gradient histogram is determined or obtained according to the gradient magnitude in the gradient information and the weight information corresponding to the first direction and the second direction.

[0374] Optionally, if the gradient direction in the gradient information is 'a', located between a first direction (e.g., a1) and a second direction (e.g., a2) in at least one lookup table, the weight information corresponding to a1 and a2 is determined or obtained based on the angle difference between a and a1 and a2 respectively. For example, a1 corresponds to weight w1, and a2 corresponds to weight w2. If the gradient magnitude in the gradient information is 'b', at least one gradient histogram is determined or obtained based on the gradient magnitude b and the weight information w1 and w2 corresponding to the first and second directions respectively. In the gradient histogram, the gradient magnitude corresponding to the first direction a1 is b*w1, and the gradient magnitude corresponding to the second direction a2 is b*w2. Optionally, the weight w1 is equal to the angle difference between a and a2 or the angle difference between a1 and a2, and the weight w2 is equal to the angle difference between a and a1 or the angle difference between a1 and a2.

[0375] In this embodiment, when the gradient direction is between two prediction directions, the gradient magnitude is allocated according to the angle difference between the gradient direction and the prediction direction, thereby avoiding errors caused by rigid classification and improving the accuracy of angle prediction.

[0376] Optionally, the gradient information of the reference region includes the gradient information of at least one pixel in the reference region and / or the overall gradient information of the reference region.

[0377] Optionally, at least one gradient histogram is determined or obtained based on the overall gradient information of at least one reference region of the current block and / or the gradient information of the pixels.

[0378] Optionally, the gradient information of a pixel is the gradient direction and / or gradient magnitude calculated independently for each pixel in the reference region, reflecting the local variation characteristics of that point.

[0379] Optionally, the overall gradient information of the reference region is obtained by statistically analyzing and / or aggregating the gradient information of all pixels within the reference region, reflecting the global change trend of the region.

[0380] Optionally, the gradient information of a pixel includes: a first pixel gradient component, a second pixel gradient component, a pixel gradient direction, and / or a pixel gradient magnitude.

[0381] Optionally, gradient information of at least one pixel in the reference region is determined or obtained based on at least one pair of mutually perpendicular gradient operators. The pair of mutually perpendicular gradient operators includes a first gradient operator (e.g., a horizontal gradient operator Gx) and a second gradient operator (e.g., a vertical gradient operator Gy) corresponding to different directions. The value determined or obtained based on the first gradient operator is the gradient component of the first pixel (e.g., the value of Gx), and the value determined or obtained based on the second gradient operator is the gradient component of the second pixel (e.g., the value of Gy).

[0382] Optionally, the pixel gradient direction and / or pixel gradient magnitude can be determined or obtained based on the first pixel gradient component and the second pixel gradient component.

[0383] Optionally, the gradient information includes: a first gradient component (e.g., a first pixel gradient component and / or a first global gradient component), a second gradient component (e.g., a second pixel gradient component and / or a second global gradient component), a gradient direction (e.g., a pixel gradient direction and / or a global gradient direction), and / or a gradient magnitude (e.g., a pixel gradient magnitude and / or a global gradient magnitude).

[0384] Optionally, the overall gradient information includes: a first overall gradient component, a second overall gradient component, an overall gradient direction, and / or an overall gradient magnitude.

[0385] Optionally, the determination or acquisition of the overall gradient information of the reference region includes: determining or acquiring the gradient information of at least one pixel in the reference region, wherein the gradient information of at least one pixel includes: a first pixel gradient component and a second pixel gradient component; determining or acquiring a first overall gradient component based on the sum of the first pixel gradient components corresponding to the reference region; determining or acquiring a second overall gradient component based on the sum of the second pixel gradient components corresponding to the reference region; determining or acquiring the overall gradient magnitude based on the sum of the absolute values ​​of the first overall gradient component and the second overall gradient component; and determining or acquiring the overall gradient direction based on the first overall gradient component and the second overall gradient component.

[0386] Optionally, the summation can be performed directly in the above manner to preserve the directional statistical properties, and the synthesized result can represent the overall gradient vector.

[0387] Optionally, gradient information of at least one pixel in the reference region is determined or obtained, including: the gradient components of the first pixel (Gx1, Gx2, ... Gx...). n ) and the second pixel gradient components (Gy1, Gy2, ...Gy n Based on the sum of the gradient components of the first pixel corresponding to the reference region, the first global gradient component, namely the first global gradient component Gx, is determined or obtained. sum =Gx1+Gx2+...+Gx n, determining or obtaining a second overall gradient component according to the sum of the second pixel gradient components corresponding to the reference region, that is, the second overall gradient component Gy sum =Gy1+Gy2+...+Gy n , determining or obtaining the overall gradient magnitude AMP according to the sum of the absolute values of the first overall gradient component and the second overall gradient component sum , that is, the overall gradient magnitude AMP sum =|Gx sum |+|Gy sum |, the overall gradient direction is calculated by arctan(Gx sum / Gy sum ) or arctan(Gy sum / Gx sum ).

[0388] Optionally, the manner of determining or obtaining the overall gradient information of the reference region includes: determining or obtaining gradient information of at least one pixel in the reference region, including: a first pixel gradient component, a second pixel gradient component, a pixel gradient direction, and a pixel gradient magnitude; determining or obtaining the overall gradient magnitude according to the sum of the absolute values of the first pixel gradient component and the second pixel gradient component corresponding to the reference region, and / or the sum of the absolute values of the first pixel gradient component and the second pixel gradient component corresponding to the reference region as well as the number of non-zero pixel gradient magnitudes corresponding to the pixel gradient directions of the reference region; determining or obtaining the first overall gradient component according to the sum of the first pixel gradient components corresponding to the reference region; determining or obtaining the second overall gradient component according to the sum of the second pixel gradient components corresponding to the reference region; and determining or obtaining the overall gradient direction according to the first overall gradient component and the second overall gradient component.

[0389] Optionally, determining or obtaining gradient information of at least one pixel in the reference region includes: first pixel gradient components (Gx1, Gx2, ...Gxn) and second pixel gradient components (Gy1, Gy2, ...Gyn), pixel gradient directions and pixel gradient magnitudes; determining or obtaining the overall gradient magnitude according to the sum of the absolute values of the first pixel gradient component and the second pixel gradient component corresponding to the reference region, and the number of non-zero pixel gradient magnitudes corresponding to the pixel gradient directions of the reference region, that is, the overall gradient magnitude AMP sum =|Gx1|+|Gx2|+...+|Gx n |+|Gy1|+|Gy2|+...+|Gy n |, or, the overall gradient magnitude AMP sum =(|Gx1|+|Gx2|+...+|Gx n |+|Gy1|+|Gy2|+...+|Gy n丨) / n, where n is the number of non-zero pixel gradient magnitudes corresponding to the pixel gradient directions in the reference region. For example, if there are 10 directions in the pixel gradient directions of the reference region with non-zero corresponding pixel gradient magnitudes, then n is 10. The first overall gradient component is determined or obtained according to the sum of the first pixel gradient components corresponding to the reference region, that is, the first overall gradient component Gx sum =Gx1+Gx2+...+Gx n , the second overall gradient component is determined or obtained according to the sum of the second pixel gradient components corresponding to the reference region, that is, the second overall gradient component Gy sum =Gy1+Gy2+...+Gy n , the overall gradient direction is calculated by arctan(Gx sum / Gy sum ) or arctan(Gy sum / Gx sum ).

[0390] Optionally, the average value of absolute values can be obtained through the above manner, which is resistant to direction interference and can stably measure the gradient strength.

[0391] Optionally, in the case where the overall gradient information of the reference region is determined or obtained according to the sum of the absolute values of the first pixel gradient component and the second pixel gradient component corresponding to the reference region, if the overall gradient information is directly fused with the gradient information of the pixels corresponding to the reference region, the overall gradient magnitude corresponding to the overall gradient information in the determined or obtained gradient histogram will be the maximum magnitude in the histogram, thereby interfering with the selection of candidate modes. To avoid the influence of the overall gradient magnitude information, the gradient histogram may be determined or obtained according to the overall gradient information, the weight information corresponding to the overall gradient magnitude, and the gradient information of at least one pixel in the reference region, so as to ensure fair competition of gradient information and balance the consistency requirements of strength and direction. And / or, the prediction mode corresponding to the overall gradient direction in the overall gradient information is determined as mode 1, the gradient histogram is determined or obtained according to the gradient information of at least one pixel in the reference region, at least one mode 2 is determined or obtained according to the gradient histogram, and mode 1 and the at least one mode 2 are determined as candidate modes. In the case that the at least one prediction mode of the current block includes mode 1 and the at least one mode 2, at least one prediction result 1 of the current block is determined or obtained according to mode 1, at least one prediction result 2 of the current block is determined or obtained according to the at least one mode 2, and the target prediction result of the current block is determined or obtained according to the at least one prediction result 1, the at least one prediction result 2, and the weight information corresponding to the at least one prediction result 1.

[0392] Optionally, the overall gradient information includes an overall gradient direction and an overall gradient magnitude, and the gradient histogram is determined or obtained according to the gradient information of at least one pixel corresponding to the reference region Figure 1 , the gradient histogram Figure 1 Replace the gradient magnitude corresponding to the direction that is the same as the global gradient direction with the global gradient magnitude to determine or obtain the gradient histogram. Figure 2 According to the gradient histogram Figure 2 At least one candidate pattern is identified or obtained.

[0393] Optionally, candidate modes include: prediction modes corresponding to overall gradient information, prediction modes determined or obtained based on gradient histograms, and / or non-angular prediction modes.

[0394] Optionally, if the prediction mode determined or obtained based on the gradient histogram includes the prediction mode corresponding to the overall gradient information, the weight information corresponding to at least one candidate mode is determined or obtained based on the weight information corresponding to the prediction mode corresponding to the overall gradient information. The candidate modes include: the prediction mode corresponding to the overall gradient information, the prediction mode determined or obtained based on the gradient histogram, and / or the non-angle prediction mode.

[0395] Optionally, if a candidate mode is determined as the prediction mode for the current block, the weight information corresponding to the candidate mode when it is used as the prediction mode can be determined or obtained based on the weight information corresponding to the candidate mode.

[0396] Optionally, if a candidate mode is determined as the prediction mode for the current block, the weight information corresponding to the candidate mode is determined as the weight information corresponding to the prediction mode.

[0397] Optionally, the weight information corresponding to the candidate mode may be the same as or different from the weight information corresponding to the candidate mode when it is determined as the prediction mode.

[0398] Optionally, the predicted modes determined or obtained based on the gradient histogram determined or obtained from the gradient information of at least one pixel in the reference region include: mode a1, mode a2, and mode a3. The weights of each of modes a1, a2, and a3 can be determined based on the sum of their respective gradient magnitudes in the gradient histogram. For example, based on the sum of their respective gradient magnitudes in the gradient histogram, the weights of modes a1, a2, and a3 are 0.4, respectively. Mode a1 is the predicted mode corresponding to the overall gradient information. Based on the weight of mode a1 (0.4), the weights of modes a2 and a3 are adjusted. For example, since the predicted mode corresponding to the overall gradient information is used as the candidate mode by default, the weight value of mode a1 can be modified from the weight obtained by the sum of the gradient magnitudes of each mode in the gradient histogram to the fixed weight corresponding to the overall gradient information. The value of the fixed weight is a preset value. Based on the fixed weight and the sum of the gradient magnitudes of modes a2 and a3 in the gradient histogram, the weights of modes a2 and a3 are determined. For example, if the fixed weight is 0.6, the sum of the gradient magnitudes of mode a2 is 50, and the sum of the gradient magnitudes of mode a3 is 50, then the weight of mode a2 is 0.2, and the weight of mode a3 is 0.2.

[0399] Optionally, the candidate modes include: a prediction mode determined or obtained based on the gradient histogram and a planar mode. The prediction modes determined or obtained based on the gradient histogram include the prediction modes corresponding to the overall gradient information. In this embodiment, the prediction modes determined or obtained based on the gradient information of at least one pixel in the reference area include: mode a1, mode a2 and mode a3. Mode a1 is the prediction mode corresponding to the overall gradient information. A fixed weight is used for the prediction mode corresponding to the overall gradient information. For example, the fixed weight is equal to 0.2. The planar mode uses a fixed weight of 0.3. The weights of mode a2 and mode a3 are determined by the sum of the gradient magnitudes corresponding to mode a2 and mode a3 in the gradient histogram. If the sum of the gradient magnitudes of mode a2 is 20 and the sum of the gradient magnitudes of mode a3 is 30, the weight of mode a2 is 0.2 and the weight of mode a3 is 0.3. The weight of mode a2 can be determined according to formula (v).

[0400] W a2 =(1-W) a1 -W planar )*(AMP a2 / (AMP a2 +AMP a3 Formula (5);

[0401] Wa2 is the weight of pattern a2, W a1 For the weights of pattern a1, AMP a2AMP is the sum of the gradient magnitudes of mode a2. a3 This is the sum of the gradient magnitude values ​​of mode a3.

[0402] The weight of pattern a3 can be determined according to formula (vi);

[0403] W a3 =(1-W) a1 -W planar )*(AMP a3 / (AMP a2 +AMP a3 Formula (VI);

[0404] Wa2 is the weight of pattern a2, W a1 For the weights of pattern a1, AMP a2 AMP is the sum of the gradient magnitudes of mode a2. a3 This is the sum of the gradient magnitude values ​​of mode a3.

[0405] Optionally, the candidate modes include: a predicted mode determined or obtained based on the gradient histogram and a planar mode. The predicted modes determined or obtained based on the gradient histogram do not include the predicted modes corresponding to the overall gradient information. In this embodiment, a fixed weight is used for the predicted modes corresponding to the overall gradient information, for example, a fixed weight of 0.2. A fixed weight of 0.3 is used for the planar mode. Based on this, the weights of modes a2 and a3 are determined by the sum of the gradient magnitudes corresponding to modes a2 and a3 in the gradient histogram. If the sum of the gradient magnitudes of mode a2 is 20 and the sum of the gradient magnitudes of mode a3 is 30, the weight of mode a2 is 0.2 and the weight of mode a3 is 0.3. Optionally, the weights corresponding to modes a2 and a3 can be determined or obtained by the above formulas (v) and (vi).

[0406] In this embodiment, by combining the gradient information of pixels and / or the overall gradient information, the determined or obtained gradient histogram can take into account both local details and global structure. As a result, at least one candidate pattern determined or obtained based on the gradient histogram can match the content of the current block, thereby improving the prediction accuracy of the prediction pattern determined or obtained based on the candidate pattern.

[0407] Method A2 requires at least one reference region and at least one lookup table for the current block.

[0408] Optionally, at least one lookup table includes: lookup table A and / or lookup table B, where the number of prediction patterns corresponding to lookup table A is a first number, and the number of prediction patterns corresponding to lookup table B is a second number, and the first number and the second number may be the same or different.

[0409] Optionally, the first quantity is 35, 67 or 131, for example, the first quantity is 131.

[0410] Optionally, the first quantity is greater than 35, 67, or 131.

[0411] Alternatively, the second quantity can be 35, 67, or 131, for example, the second quantity can be 67.

[0412] Optionally, the second quantity is less than or equal to 35, 67, or 131.

[0413] Optionally, the first quantity and the second quantity are the same, that is, the number of prediction patterns corresponding to lookup table A is the same as the number of prediction patterns corresponding to lookup table B. For example, the number of prediction patterns corresponding to lookup table A and lookup table B is the same, but the types of prediction patterns corresponding to each other are at least partially different.

[0414] Optionally, the prediction patterns corresponding to lookup table A and lookup table B are different, or the prediction patterns corresponding to lookup table A and lookup table B are at least partially the same.

[0415] Optionally, lookup table A can be a lookup table corresponding to 37 prediction modes or a lookup table corresponding to 131 prediction modes.

[0416] Optionally, lookup table B can be a lookup table corresponding to 37 prediction modes or a lookup table corresponding to 131 prediction modes.

[0417] Optionally, step S10 includes: determining or obtaining at least one gradient histogram and / or at least one area histogram based on at least one reference region and at least one lookup table of the current block, and determining or obtaining at least one candidate pattern based on at least one gradient histogram and / or at least one area histogram.

[0418] Optionally, the number of prediction patterns corresponding to lookup table A is a first number. Based on at least one reference region of the current block and lookup table A, at least one statistical histogram A is determined or obtained, and the prediction patterns corresponding to the statistical histogram A are the number of prediction patterns corresponding to lookup table A, i.e., the first number.

[0419] Optionally, statistical histograms may include: gradient histograms and / or area histograms.

[0420] Optionally, the number of prediction patterns corresponding to lookup table B is the second number. Based on at least one reference region of the current block and lookup table B, at least one statistical histogram B is determined or obtained, and the number of prediction patterns corresponding to the statistical histogram B is the number of prediction patterns corresponding to lookup table B, i.e., the second number.

[0421] Optionally, based on at least one reference region of the current block and lookup table A, at least one statistical histogram A is determined or obtained; based on at least one reference region of the current block and lookup table B, at least one statistical histogram B is determined or obtained; and based on at least one statistical histogram A and at least one statistical histogram B, at least one candidate pattern is determined or obtained.

[0422] Optionally, based on the gradient information of at least one reference region of the current block and at least one lookup table, at least one gradient histogram is determined or obtained. The gradient information includes: pixel gradient information and / or overall gradient information. The gradient information, pixel gradient information and / or overall gradient information include at least one of gradient magnitude and gradient direction.

[0423] Optionally, at least one lookup table includes: lookup table A, the number of prediction modes corresponding to lookup table A is a first number, for example, 131, and at least one gradient histogram A is determined or obtained based on the gradient information of at least one reference region of the current block and lookup table A, the number of prediction modes corresponding to gradient histogram A is a first number.

[0424] Optionally, at least one lookup table includes: lookup table B, the number of prediction modes corresponding to lookup table B is a second number, for example, 67, at least one gradient histogram B is determined or obtained based on the gradient information of at least one reference region of the current block and lookup table B, the number of prediction modes corresponding to gradient histogram B is a second number.

[0425] Optionally, at least one lookup table includes: lookup table A and lookup table B, where the number of prediction modes corresponding to lookup table A is a first number, for example, 131, and the number of prediction modes corresponding to lookup table B is a second number, for example, 67. Based on the gradient information of at least one reference region of the current block and lookup table A, at least one gradient histogram A is determined or obtained, where the number of prediction modes corresponding to gradient histogram A is the first number. Based on the gradient information of at least one reference region of the current block and lookup table B, at least one gradient histogram B is determined or obtained, where the number of prediction modes corresponding to gradient histogram B is the second number.

[0426] Optionally, the processing method of this application embodiment can be applied to the intra-prediction module of the codec. The upstream / downstream modules of the intra-prediction module, such as upstream modules like reference pixel management and mode candidate list generation, and downstream modules like residual transformation, entropy coding, and loop filtering, which need to adjust parameters according to the prediction mode, may have the same or different number of prediction modes as the intra-prediction module. When the number of prediction modes corresponding to the intra-prediction module is different from the number of prediction modes corresponding to its upstream and downstream modules, mode index remapping can be performed. For example, the indexes of the 131 prediction modes in the intra-prediction module can be mapped to the 67 prediction modes. The remapping of mode indexes can be achieved by looking up tables or deducing rules to achieve compatibility. Optionally, some modes may trigger specific residual corrections, and the 131 modes can be converted into compatible correction parameters before mapping.

[0427] In this embodiment, based on at least one reference region and at least one lookup table of the current block, gradient histograms and / or area histograms corresponding to different prediction modes can be determined or obtained. The gradient histograms and / or area histograms can quickly eliminate irrelevant prediction modes, improve prediction efficiency, and accurately capture the dominant prediction direction related to at least one reference region of the current block, thereby matching suitable candidate modes and / or prediction modes for the current block.

[0428] Method A3 requires at least one gradient operator corresponding to at least one lookup table.

[0429] Optionally, step S10 includes: determining or obtaining at least one gradient histogram based on at least one reference region and at least one gradient operator corresponding to at least one lookup table of the current block, and determining or obtaining at least one candidate mode based on the at least one gradient histogram.

[0430] Optionally, at least one lookup table includes: lookup table A and / or lookup table B, where the number of prediction patterns corresponding to lookup table A is a first number, and the number of prediction patterns corresponding to lookup table B is a second number, and the first number and the second number may be the same or different.

[0431] Optionally, lookup table A corresponds to gradient operators A1 and A2, and lookup table B corresponds to gradient operators B1 and B2. Gradient operators A1 to A2 and gradient operators B1 to B2 are all different.

[0432] Optionally, the prediction patterns corresponding to lookup table A and lookup table B are different, or the prediction patterns corresponding to lookup table A and lookup table B are at least partially the same.

[0433] Optionally, lookup table A can be a lookup table corresponding to 37 prediction modes or a lookup table corresponding to 131 prediction modes.

[0434] Optionally, lookup table B can be a lookup table corresponding to 37 prediction modes or a lookup table corresponding to 131 prediction modes.

[0435] Optionally, gradient information of at least one reference region of the current block is determined or obtained based on at least one gradient operator corresponding to at least one reference region of the current block and at least one lookup table, and at least one gradient histogram is determined or obtained based on the gradient information of at least one reference region of the current block.

[0436] Optionally, if the first quantity is greater than the second quantity, the prediction direction corresponding to the prediction mode corresponding to lookup table A at least partially covers the prediction direction corresponding to the prediction mode corresponding to lookup table B. Since the gradient operator needs to match different directions to accurately calculate the gradient information, when the first quantity and the second quantity are different, the gradient information of at least one reference region of the current block can be determined or obtained by at least one gradient operator corresponding to lookup table A and / or at least one gradient operator corresponding to lookup table B. Based on the gradient information of at least one reference region of the current block, at least one gradient histogram can be determined or obtained.

[0437] Optionally, the first number is 131. Based on at least one gradient operator corresponding to lookup table A, the gradient information A of at least one reference region of the current block is determined or obtained. Based on the gradient information A of the at least one reference region, at least one gradient histogram A is determined or obtained. The gradient histogram A includes the gradient magnitude change of the at least one reference region in the prediction direction corresponding to the 131 prediction modes.

[0438] Optionally, the second number is 67. Based on at least one gradient operator corresponding to lookup table B, the gradient information B of at least one reference region of the current block is determined or obtained. Based on the gradient information B of the at least one reference region, at least one gradient histogram B is determined or obtained. The gradient histogram B includes the gradient magnitude change of the at least one reference region in the prediction direction corresponding to the 67 prediction modes.

[0439] Optionally, at least one candidate pattern is determined or obtained based on at least one gradient histogram A and / or at least one gradient histogram B.

[0440] Optionally, a gradient operator is a tool and / or method for determining or obtaining at least one gradient information, capable of extracting pixel gradient information from an image through convolution operations or other mathematical operations, including: gradient direction and / or gradient magnitude.

[0441] Optionally, at least one gradient operator includes at least one pair of mutually perpendicular gradient operators.

[0442] Optionally, mutually perpendicular gradient operators include: a first pair of mutually perpendicular gradient operators and / or a second pair of mutually perpendicular gradient operators, wherein the first pair of gradient operators and the second pair of gradient operators are different.

[0443] Optionally, a first gradient information is determined or obtained based on a first pair of mutually perpendicular gradient operators, and at least one gradient histogram is determined or obtained based on the first gradient information.

[0444] Optionally, a second gradient information is determined or obtained based on a second pair of mutually perpendicular gradient operators, and at least one gradient histogram is determined or obtained based on the second gradient information.

[0445] Optionally, the first pair of mutually perpendicular gradient operators includes a 0° gradient operator and a 90° gradient operator. The horizontal gradient Gx and vertical gradient Gy can be calculated using a 3x3 horizontal (0°) Sober operator and a vertical (90°) Sober operator. For example, the horizontal (0°) gradient Gx and vertical (90°) gradient Gy for a pixel x4 in a pixel line can be calculated according to the following formulas (I) and (II).

[0446]

[0447] Optionally, A can be a matrix consisting of nine pixels, centered on pixel x4, and including the pixel x1 above it, the pixel x3 to its left, the pixel x7 below it, the pixel x5 to its right, the pixel x0 to its upper left, the pixel x6 to its lower left, the pixel x2 to its upper right, and the pixel x8 to its lower right, as shown in Formula (III) below.

[0448]

[0449] Optionally, the magnitude of gradient G is the sum of the absolute values ​​of the horizontal gradient and the vertical gradient, and its calculation formula is shown in formula (iv).

[0450] Formula (IV) is G = |Gx| + |Gy|.

[0451] Alternatively, the gradient direction of a pixel can be calculated using arctan(Gx / Gy) or arctan(Gy / Gx).

[0452] Optionally, the second pair of mutually perpendicular gradient operators includes a +45° gradient operator and a -45° gradient operator, and the +45° gradient G can be calculated using the Prewitt operator, the Scharr operator, and / or the Sobel operator. +45° and -45° gradient G -45° For example, for a pixel x 4 in a pixel line, G +45° gradient and G -45° The gradient can be calculated using any pair of operators from Equation (VII) to Equation (XII).

[0453] Formulas (VII) and (VIII) are Prevet operators that can directly respond to diagonal edges through a reference template.

[0454]

[0455] Formulas (IX) and (X) are Sobel operators, with a center pixel weight of 0 and symmetrical increases or decreases on both sides, which can enhance the response of the diagonal edge.

[0456]

[0457] Formulas (XI) and (XII) are Schar operators, which provide more accurate calculation of the diagonal gradient.

[0458]

[0459]

[0460] Optionally, applying only the gradient operators in the horizontal and vertical directions to determine the gradient histogram results in low prediction accuracy for the diagonal direction. Therefore, this embodiment introduces a ±45° gradient operator to effectively improve the prediction accuracy in the diagonal direction.

[0461] Optionally, based on at least one reference region of the current block and at least one lookup table corresponding to a first pair of mutually perpendicular gradient operators, at least one gradient histogram of the current block is determined or obtained. The at least one lookup table includes lookup table A and / or lookup table B, and the first pair of gradient operators includes a 0° (horizontal direction) gradient operator and a 90° (vertical direction) gradient operator.

[0462] Optionally, based on at least one reference region of the current block and at least one lookup table corresponding to a first pair of mutually perpendicular gradient operators, at least one gradient information of the current block is determined or obtained, and based on the at least one gradient information, at least one gradient histogram is determined or obtained, the at least one lookup table includes lookup table A and / or lookup table B, and the first pair of gradient operators includes a 0° (horizontal direction) gradient operator and a 90° (vertical direction) gradient operator.

[0463] Optionally, based on at least one reference region of the current block and at least one lookup table corresponding to a second pair of mutually perpendicular gradient operators, at least one gradient information of the current block is determined or obtained, and based on the at least one gradient information, at least one gradient histogram of the current block is determined or obtained, wherein the at least one lookup table includes lookup table A and / or lookup table B, and the second pair of gradient operators includes a +45° gradient operator and a -45° gradient operator.

[0464] Optionally, the prediction direction a corresponding to arctan(Gx / Gy) or arctan(Gy / Gx) is determined or obtained. Based on the angle difference between each prediction direction and prediction direction a in at least one lookup table, the prediction direction corresponding to arctan(Gx / Gy) or arctan(Gy / Gx) is determined or obtained. For example, the direction with the smallest absolute angle difference is determined as the prediction direction corresponding to arctan(Gx / Gy) or arctan(Gy / Gx).

[0465] Alternatively, the gradient direction can be determined or obtained by a lookup table. For example, using a lookup table for arctan(a), the two closest indices index1 and index2 of arctan(Gx / Gy) or arctan(Gy / Gx) can be determined, and the corresponding prediction direction of the pixel can be determined based on the angle difference between arctan(Gx / Gy) or arctan(Gy / Gx) and index1 and index2.

[0466] Optionally, if the gradient direction corresponding to arctan(Gx / Gy) or arctan(Gy / Gx) is determined or obtained based on the gradient information and is located between the first direction and the second direction in at least one lookup table, then at least one gradient histogram is determined or obtained based on the angle difference between the gradient direction corresponding to arctan(Gx / Gy) or arctan(Gy / Gx) and the first direction and the second direction.

[0467] Optionally, if the gradient direction corresponding to arctan(Gx / Gy) or arctan(Gy / Gx) is determined or obtained based on the gradient information and is located between the first direction and the second direction in at least one lookup table, then the weight information corresponding to the first direction and the second direction is determined or obtained based on the angle difference between the gradient direction corresponding to arctan(Gx / Gy) or arctan(Gy / Gx) and the first direction and the second direction, and at least one gradient histogram is determined or obtained based on the weight information corresponding to the first direction and the second direction.

[0468] Optionally, if the gradient direction corresponding to arctan(Gx / Gy) or arctan(Gy / Gx) is determined or obtained based on the gradient information and is located between the first direction and the second direction in at least one lookup table, then the weight information corresponding to the first direction and the second direction is determined or obtained based on the angle difference between the gradient direction corresponding to arctan(Gx / Gy) or arctan(Gy / Gx) and the first direction and the second direction, the gradient magnitude is determined or obtained based on the gradient information, and at least one gradient histogram is determined or obtained based on the gradient magnitude and the weight information corresponding to the first direction and the second direction.

[0469] Optionally, if the gradient direction corresponding to arctan(Gx / Gy) or arctan(Gy / Gx) is determined or obtained based on the gradient information, and it is located between the first direction (e.g., a1) and the second direction (e.g., a2) in at least one lookup table, the weight information corresponding to a1 and a2 is determined or obtained based on the angle difference between a and a1 and a2. For example, a1 corresponds to weight w1 and a2 corresponds to weight w2. The gradient magnitude b is determined or obtained based on the gradient information. Based on the gradient magnitude b and the weight information w1 and w2 corresponding to the first direction and the second direction, at least one gradient histogram is determined or obtained. In the gradient histogram, the gradient magnitude corresponding to the first direction a1 is b*w1 and the gradient magnitude corresponding to the second direction a2 is b*w2.

[0470] In this embodiment, applying only the gradient operators in the horizontal and vertical directions to determine the gradient histogram results in low prediction accuracy for the diagonal direction. Therefore, different direction gradient operators, such as ±45°, are introduced in this embodiment to effectively improve the prediction accuracy in the diagonal direction.

[0471] Method A4, at least one pair of gradient operators corresponding to weight information.

[0472] Optionally, step S10 includes: determining or obtaining at least one gradient histogram based on at least one reference region of the current block and at least one pair of gradient operators' weight information; determining or obtaining at least one candidate mode based on the at least one gradient histogram; and determining or obtaining at least one prediction mode of the current block based on the matching information of the at least one candidate mode and the current block.

[0473] Optionally, based on at least one reference region of the current block, at least one pair of gradient operators corresponding to at least one lookup table, and at least one weight information corresponding to at least one pair of gradient operators, at least one gradient histogram is determined or obtained, and at least one candidate mode is determined or obtained based on the at least one gradient histogram.

[0474] Optionally, based on the weight information corresponding to at least one reference region of the current block and at least one pair of gradient operators, the gradient information of at least one reference region of the current block is determined or obtained; based on the at least one gradient information, at least one gradient histogram is determined or obtained; and based on the at least one gradient histogram, at least one candidate mode is determined or obtained.

[0475] Optionally, based on at least one reference region of the current block, at least one pair of gradient operators corresponding to at least one lookup table, and weight information corresponding to at least one pair of gradient operators, the gradient information of at least one reference region of the current block is determined or obtained, at least one gradient histogram is determined or obtained based on at least one gradient histogram, and at least one candidate mode is determined or obtained based on at least one gradient histogram.

[0476] Optionally, at least one lookup table includes: lookup table A and / or lookup table B, where the number of prediction patterns corresponding to lookup table A is a first number, and the number of prediction patterns corresponding to lookup table B is a second number, and the first number and the second number may be the same or different.

[0477] Optionally, the first quantity is 35, 67 or 131, for example, the first quantity is 131.

[0478] Optionally, the first quantity is greater than 35, 67, or 131.

[0479] Alternatively, the second quantity can be 35, 67, or 131, for example, the second quantity can be 67.

[0480] Optionally, the second quantity is less than or equal to 35, 67, or 131.

[0481] Optionally, the first quantity and the second quantity are the same, that is, the number of prediction patterns corresponding to lookup table A is the same as the number of prediction patterns corresponding to lookup table B. For example, the number of prediction patterns corresponding to lookup table A and lookup table B is the same, but the types of prediction patterns corresponding to each other are at least partially different.

[0482] Optionally, the prediction patterns corresponding to lookup table A and lookup table B are different, or the prediction patterns corresponding to lookup table A and lookup table B are at least partially the same.

[0483] Optionally, lookup table A can be a lookup table corresponding to 37 prediction modes or a lookup table corresponding to 131 prediction modes.

[0484] Optionally, lookup table B can be a lookup table corresponding to 37 prediction modes or a lookup table corresponding to 131 prediction modes.

[0485] Optionally, the mutually perpendicular gradient operators include: a first pair of mutually perpendicular gradient operators and / or a second pair of mutually perpendicular gradient operators, wherein the first pair of gradient operators and the second pair of gradient operators are different. For example, the first pair of gradient operators includes a 0° (horizontal direction) gradient operator and a 90° (vertical direction) gradient operator, and the second pair of gradient operators includes a +45° gradient operator and a -45° gradient operator.

[0486] Optionally, a first gradient information is determined or obtained based on a first pair of mutually perpendicular gradient operators, and at least one gradient histogram is determined or obtained based on the first gradient information.

[0487] Optionally, a second gradient information is determined or obtained based on a second pair of mutually perpendicular gradient operators, and at least one gradient histogram is determined or obtained based on the second gradient information.

[0488] Optionally, the first pair of gradient operators and the second pair of gradient operators are different. Based on the first gradient information and the second gradient information, the third gradient information is determined or obtained, and at least one gradient histogram is determined or obtained based on the third gradient information.

[0489] Optionally, based on the first gradient information, the weight information corresponding to the first pair of gradient operators, the second gradient information, and the weight information corresponding to the second pair of gradient operators, the third gradient information is determined or obtained, and at least one gradient histogram is determined or obtained based on the third gradient information.

[0490] Optionally, the third gradient information can be determined or obtained by fusing the first gradient information and the second gradient information through the weight information corresponding to at least one pair of gradient operators, for example, including gradient direction and / or gradient magnitude.

[0491] Optionally, gradient information of at least one reference region of the current block is determined or obtained based on the gradient operators corresponding to at least one lookup table, at least one gradient histogram corresponding to the current block is determined or obtained based on the gradient information of the at least one reference region of the current block, and at least one candidate mode is determined or obtained based on the at least one gradient histogram. The at least one lookup table includes lookup table A and / or lookup table B, and the mutually perpendicular gradient operators include a first pair of mutually perpendicular gradient operators and / or a second pair of mutually perpendicular gradient operators, wherein the first pair of gradient operators and the second pair of gradient operators are different.

[0492] Optionally, based on at least one reference region of the current block, at least one lookup table, and a first pair of mutually perpendicular gradient operators, the first gradient information of at least one reference region of the current block is determined or obtained; based on at least one reference region of the current block, at least one lookup table, and a second pair of mutually perpendicular gradient operators, the second gradient information of at least one reference region of the current block is determined or obtained; based on the first gradient information of at least one reference region, the second gradient information of at least one reference region, the weight information corresponding to the first pair of mutually perpendicular gradient operators, and the weight information corresponding to the second pair of mutually perpendicular gradient operators, at least one gradient histogram is determined or obtained.

[0493] Optionally, based on the first pair of mutually perpendicular gradient operators corresponding to at least one lookup table, the first gradient information of at least one reference region of the current block is determined or obtained; based on the first gradient information of at least one reference region of the current block, the at least one gradient histogram C corresponding to the current block is determined or obtained; and based on the at least one gradient histogram C, the at least one candidate mode is determined or obtained.

[0494] Optionally, based on the second pair of mutually perpendicular gradient operators corresponding to at least one lookup table, the second gradient information of at least one reference region of the current block is determined or obtained; based on the second gradient information of at least one reference region of the current block, the at least one gradient histogram D corresponding to the current block is determined or obtained; and based on the at least one gradient histogram D, the at least one candidate mode is determined or obtained.

[0495] Optionally, based on at least one gradient histogram C and at least one gradient histogram D, at least one third gradient information of at least one reference region of the current block is determined or obtained, based on at least one third gradient information, at least one gradient histogram E is determined or obtained, and at least one candidate mode is determined or obtained based on at least one gradient histogram E.

[0496] Optionally, at least one candidate pattern is determined or obtained based on at least one gradient histogram C and at least one gradient histogram D.

[0497] Optionally, the step of determining or obtaining at least one candidate mode based on at least one gradient histogram C and at least one gradient histogram D includes: fusing at least one gradient histogram C and at least one gradient histogram D based on the weight information corresponding to at least one pair of gradient operators to determine or obtain at least one third gradient information of at least one reference region of the current block; determining or obtaining at least one gradient histogram E based on the at least one third gradient information; and determining or obtaining at least one candidate mode based on the at least one gradient histogram E. Optionally, the number of prediction modes corresponding to the fused gradient histogram C and gradient histogram D is the same.

[0498] Optionally, based on the weight information corresponding to at least one pair of gradient operators, at least one gradient histogram C and at least one gradient histogram D are fused to determine or obtain at least one gradient histogram E. At least one candidate mode is determined or obtained based on at least one gradient histogram E. The weight information corresponding to the gradient operators includes: first weight information, which is the weight information corresponding to the gradient magnitude of each gradient direction in the gradient histogram, including: the weight information corresponding to the gradient magnitude of at least one gradient histogram C and / or the weight information corresponding to the gradient magnitude of at least one gradient histogram D. Optionally, gradient histogram C and gradient histogram D are different.

[0499] Optionally, the first weight information includes the weight value corresponding to the gradient magnitude of each prediction direction (i.e., the direction perpendicular to the gradient direction) in the gradient histogram. The magnitude of the weight value corresponding to the gradient magnitude of each prediction direction in the gradient histogram is inversely proportional to and / or negatively correlated with the angle difference between the prediction direction and the direction of the gradient operator corresponding to the gradient histogram. For example, if the direction of the gradient operator corresponding to the gradient histogram is 0° and 90°, then the direction with the smaller angle difference with the horizontal or vertical direction in each prediction direction of the gradient histogram has a larger weight value; and / or, the direction with the larger angle difference with the horizontal or vertical direction has a smaller weight value.

[0500] Optionally, the first pair of mutually perpendicular gradient operators includes a 0° gradient operator and a 90° gradient operator. The weight value corresponding to the gradient magnitude of mode 3 in gradient histogram C is 0.9, and the gradient magnitude is C1. The weight value corresponding to the gradient magnitude of mode 3 in gradient histogram D is 0.1, and the gradient magnitude is D1. Then, gradient histogram C and gradient histogram D are fused to determine or obtain the third gradient information and / or gradient histogram E. The gradient direction of the third gradient information is the gradient direction corresponding to mode 3, and the gradient magnitude is C1*0.9+D1*0.1. And / or the gradient magnitude value corresponding to mode 3 in gradient histogram E is C1*0.9+D1*0.1.

[0501] Optionally, since the convolution response is maximized and the detection is most accurate when the operator design direction (i.e., the first direction and / or the second direction mentioned above) is consistent with the prediction edge direction, the accuracy of the obtained prediction pattern can be effectively improved by setting the first weight information mentioned above.

[0502] Optionally, the first pair of mutually perpendicular gradient operators includes a 0° gradient operator and a 90° gradient operator, and the second pair of mutually perpendicular gradient operators includes a +45° gradient operator and a -45° gradient operator. Based on the first pair of mutually perpendicular gradient operators, a gradient histogram C is determined or obtained. Based on the second pair of mutually perpendicular gradient operators, a gradient histogram D is determined or obtained. Since gradient histogram C has higher precision for horizontal and vertical edges, the weight information corresponding to the gradient magnitude in the horizontal and vertical directions in gradient histogram C is greater than the weight information corresponding to the gradient magnitude in the horizontal and vertical directions in gradient histogram D. However, gradient histogram D has higher precision for diagonal edges, so the weight information corresponding to the gradient magnitude in the diagonal direction in gradient histogram D is greater than the weight information corresponding to the gradient magnitude in the diagonal direction in gradient histogram C.

[0503] Optionally, the first weight information corresponding to the gradient magnitude of the predicted direction of the gradient histogram C is shown in Table 1 below:

[0504] Table 1

[0505] Predicting direction w12 w13 w14 ... w111 w112 ... w117 w118 Value 1 0.9 0.8 ... 0.1 0 ... 0 0 Predicting direction w119 ... w124 ... w130 w131 w132 w133 w134 Value 0 ... 0 ... 0.6 0.7 0.8 0.9 1 Predicting direction w135 w136 w137 .... w142 w143 ... w149 w150 Value 0.9 0.8 0.7 0.2 0.1 ... 0 0 Predicting direction w151 w152 ... w157 ... w163 w164 w165 w166 Value 0 0 0.1 ... 0.7 0.8 0.9 1

[0506] Optionally, the first weight information corresponding to the gradient magnitude of the predicted direction of the gradient histogram D is shown in Table 2 below:

[0507] Table 2

[0508] Predicting direction w22 w23 w24 ... w211 w212 ... w217 w218 Value 0 0.1 0.2 ... 0.9 1 ... 1 1 Predicting direction w219 ... w224 ... w230 w231 w232 w233 w234 Value 1 ... 1 ... 0.4 0.3 0.2 0.1 0 Predicting direction w235 w236 w237 .... w242 w243 ... w249 w250 Value 0.1 0.2 0.3 0.8 0.9 ... 1 1 Predicting direction w251 w252 ... w257 ... w263 w264 w265 w266 Value 1 1 0.9 ... 0.3 0.2 0.1 0

[0509] Optionally, the first pair of mutually perpendicular gradient operators includes a 0° gradient operator and a 90° gradient operator, and the second pair of mutually perpendicular gradient operators includes a +45° gradient operator and a -45° gradient operator. Based on the first pair of mutually perpendicular gradient operators, a gradient histogram C is determined or obtained. Based on the second pair of mutually perpendicular gradient operators, a gradient histogram D is determined or obtained. In gradient histogram D, the weight value corresponding to the gradient magnitude of the prediction direction whose angle difference with the horizontal or vertical direction is less than a first angle difference threshold is 0. The weight value corresponding to the gradient magnitude of the prediction direction whose angle difference with 45°, 135°, and -45° is less than a second angle difference threshold is 1. Optionally, the first angle difference threshold and the second angle difference threshold are different.

[0510] Optionally, applying only horizontal and vertical gradient operators to determine the gradient histogram results in low prediction accuracy for diagonal directions. Therefore, this embodiment introduces a ±45° gradient operator to effectively improve prediction accuracy in diagonal directions. Furthermore, since horizontal (e.g., horizon, buildings) and vertical (e.g., trees, people) structures dominate in natural scenes, while diagonal textures (e.g., 45° diagonal) are relatively rare, the gradient operators corresponding to 0° and 90° are more accurate for prediction directions that are close to 0° and 90°. Therefore, by setting the above weight values, the prediction accuracy of the obtained prediction pattern can be effectively improved.

[0511] Optionally, mutually perpendicular gradient operators include: a first pair of mutually perpendicular gradient operators and / or a second pair of mutually perpendicular gradient operators, wherein the first pair of gradient operators and the second pair of gradient operators are different.

[0512] Optionally, the first pair of mutually perpendicular gradient operators includes a 0° gradient operator and a 90° gradient operator, and the second pair of mutually perpendicular gradient operators includes a +45° gradient operator and a -45° gradient operator. Based on at least one reference region of the current block and the first pair of mutually perpendicular gradient operators, at least one first gradient information of at least one reference region of the current block is determined or obtained. Based on at least one reference region of the current block and the second pair of mutually perpendicular gradient operators, at least one second gradient information of at least one reference region of the current block is determined or obtained. The second gradient information includes the gradient component G corresponding to +45°. +45°The gradient component G corresponding to -45° -45° For gradient component G +45° and gradient component G -45° By projecting, the gradient components D in the directions of the 0° and 90° gradient operators are determined or obtained. x1 and D y1 D x1 and D y1 The following formulas (xiii) and (xiv) are used to calculate the following:

[0513]

[0514] Optionally, the first gradient information includes the gradient component D corresponding to 0°. x2 The gradient component D corresponding to 90° y2 Based on the weight information corresponding to the first pair of mutually perpendicular gradient operators, the weight information corresponding to the second pair of mutually perpendicular gradient operators, and the gradient component D x1 D y1 Gradient component D x2 and D y2 At least one fusion gradient information is determined or obtained, including: the fusion gradient component corresponding to 0° and the fusion gradient component corresponding to 90°. Based on the at least one fusion gradient information, at least one gradient histogram is determined or obtained.

[0515] Optionally, the target gradient component Gx corresponding to 0° and the target gradient component Gy corresponding to 90° are calculated according to the following formulas (XV) and (XVI):

[0516] Gx = w a *D x1 +w b *D x2 Formula (XV);

[0517] Gy=w a *D y1 +w b *D y2 Formula (XVI);

[0518] Optionally, w a For the weight information corresponding to the second pair of mutually perpendicular gradient operators, w b The weight information is for the first pair of mutually perpendicular gradient operators. Optionally, the weights corresponding to the above weight information are preset fixed values, and / or the above weight values ​​can be adaptively adjusted according to different scenarios.

[0519] Optionally, the weight information corresponding to the gradient operators includes: the weight information corresponding to the first pair of mutually perpendicular gradient operators and / or the weight information corresponding to the second pair of mutually perpendicular gradient operators.

[0520] Optionally, the fusion gradient information of each pixel can be calculated in the above manner, thereby determining the distribution of the fusion gradient components in the local area of ​​the reference region. Based on the distribution of the fusion gradient components in the local area of ​​the reference region, at least some pixels in the reference region are enhanced.

[0521] Optionally, weight information corresponding to the gradient information of at least one reference region can be determined or obtained based on the fused gradient information of at least one reference region.

[0522] In this embodiment, by fusing gradient operators in the horizontal, vertical and diagonal directions, all dominant edge directions in the current block can be accurately detected, avoiding the blind spots of single-direction operators, thereby effectively improving the prediction accuracy for the current block.

[0523] Method A5: Gradient information corresponding to the weight information of at least one reference region of the current block.

[0524] Optionally, step S10 includes: determining or obtaining at least one gradient histogram based on the weight information corresponding to the gradient information of at least one reference region of the current block; determining or obtaining at least one candidate mode based on the at least one gradient histogram; and determining or obtaining at least one prediction mode of the current block based on the matching information of the at least one candidate mode and the current block.

[0525] Optionally, at least one gradient histogram can be determined or obtained based on the gradient information of at least one reference region of the current block and the weight information corresponding to the gradient information.

[0526] Optionally, at least one gradient histogram is determined or obtained based on the weight information corresponding to the gradient information of at least one reference region of the current block and at least one lookup table.

[0527] Optionally, at least one gradient histogram is determined or obtained based on the weight information corresponding to the gradient information of at least one reference region of the current block and at least one gradient operator corresponding to at least one lookup table.

[0528] Optionally, at least one gradient histogram is determined or obtained based on the weight information corresponding to the gradient information of at least one reference region of the current block and the weight information corresponding to at least one pair of gradient operators.

[0529] Optionally, gradient information of at least one reference region of the current block is determined or obtained based on at least one gradient operator corresponding to at least one reference region of the current block and at least one lookup table, and at least one gradient histogram is determined or obtained based on the gradient information of at least one reference region of the current block and the weight information corresponding to the gradient information.

[0530] Optionally, gradient information of at least one reference region of the current block is determined or obtained based on at least one gradient operator corresponding to at least one lookup table and at least one pair of gradient operators, and weight information corresponding to at least one reference region of the current block. At least one gradient histogram is determined or obtained based on gradient information of at least one reference region of the current block and weight information corresponding to the gradient information.

[0531] Optionally, the weight information corresponding to the gradient information includes: second weight information and / or third weight information, wherein the second weight information is weight information based on spatial distance, and the third weight information is weight based on directional consistency.

[0532] Optionally, the second weight information (i.e., spatial distance-based weight) of a pixel in at least one reference region of the current block is inversely proportional to and / or negatively correlated with the distance between the pixel and the center of the current block. That is, the farther the distance between the pixel and the center of the current block, the smaller the weight value of the second weight information of the pixel; the closer the distance between the pixel and the center of the current block, the larger the weight value of the second weight information of the pixel. The above-mentioned spatial distance-based weight can suppress the interference of edge pixels and strengthen the dominance of the central region.

[0533] Optionally, the weight information corresponding to the gradient information includes: second weight information. The gradient information of at least one reference region of the current block includes: gradient direction 1 of pixel a, gradient magnitude x of pixel a, gradient direction 2 of pixel b, and gradient magnitude y of pixel a. Since the distance between pixel a and the center of the current block is less than the distance between pixel b and the center of the current block, the second weight information w1 of pixel a is greater than the second weight information w2 of pixel b. Based on the gradient information of at least one reference region of the current block and the weight information corresponding to the gradient information of at least one reference region of the current block, the gradient magnitude of pixel a in gradient direction 1 is x*w1, and the gradient magnitude of pixel b in gradient direction 2 is y*w2. Based on the gradient magnitude x*w1 of pixel a in gradient direction 1 and the gradient magnitude y*w2 of pixel b in gradient direction 2, at least one gradient histogram is determined or obtained.

[0534] Optionally, the third weight information (i.e., the weight based on orientation consistency) of a pixel in at least one reference region of the current block is positively proportional to and / or positively correlated with the gradient orientation consistency between that pixel and at least one pixel in its neighboring region. For example, the stronger the consistency between the gradient direction of pixel A in the reference region and the gradient direction of at least one pixel in the neighboring region of pixel A, the larger the weight value of the third weight information; and / or, the weaker the consistency between the gradient direction of pixel A in the reference region and the gradient direction of at least one pixel in the neighboring region of pixel A, the smaller the weight value of the third weight information. The above-mentioned weight based on orientation consistency can strengthen the statistical weight of the locally consistent texture direction and improve the salience of the dominant edge mode.

[0535] Optionally, the smaller the angle difference between the gradient direction of a pixel and the gradient direction of at least one pixel in its neighboring region, the stronger the gradient direction consistency; and / or, the larger the angle difference between the gradient direction of a pixel and the gradient direction of at least one pixel in its neighboring region, the weaker the gradient direction consistency.

[0536] Optionally, based on the fusion gradient information of at least one pixel in at least one reference region of the current block, the third weight information of the pixels in at least one reference region of the current block is determined or obtained.

[0537] Optionally, the weight information corresponding to the gradient information includes: third weight information. The gradient information of at least one reference region of the current block includes: gradient direction 1 of pixel a, gradient magnitude x of pixel a, gradient direction 2 of pixel b, and gradient magnitude y of pixel a. Since the gradient direction consistency between pixel a and at least one pixel in its neighboring region is stronger than that between pixel b and at least one pixel in its neighboring region, the third weight information w3 of pixel a is greater than the third weight information w4 of pixel b. Based on the gradient information of at least one reference region of the current block and the weight information corresponding to the gradient information of at least one reference region of the current block, the gradient magnitude of pixel a in gradient direction 1 is x*w3, and the gradient magnitude of pixel b in gradient direction 2 is y*w4. Based on the gradient magnitude x*w3 of pixel a in gradient direction 1 and the gradient magnitude y*w4 of pixel b in gradient direction 2, at least one gradient histogram is determined or obtained.

[0538] Optionally, the weight information corresponding to the gradient information includes: second weight information and third weight information. The gradient information of at least one reference region of the current block includes: gradient direction 1 of pixel a, gradient magnitude x of pixel a, gradient direction 2 of pixel b, and gradient magnitude y of pixel a. Since the angle difference between pixel a and the center of the current block is less than the angle difference between pixel b and the center of the current block, the second weight information w1 of pixel a is greater than the second weight information w2 of pixel b. Furthermore, the gradient direction consistency between pixel a and at least one pixel in its neighboring region is stronger than that between pixel b and at least one pixel in its neighboring region. Since the gradient direction is consistent between pixels, the third weight information w3 of pixel a is greater than the third weight information w4 of pixel b. Based on the gradient information of at least one reference region of the current block and the weight information corresponding to the gradient information of at least one reference region of the current block, the gradient magnitude of pixel a in gradient direction 1 is x*w1*w3, and the gradient magnitude of pixel b in gradient direction 2 is y*w2*w4. Based on the gradient magnitude x*w1*w3 of pixel a in gradient direction 1 and the gradient magnitude y*w2*w4 of pixel b in gradient direction 2, at least one gradient histogram is determined or obtained.

[0539] In this embodiment, at least one gradient histogram can be determined or obtained based on the second weight information (i.e., weight based on spatial distance) and / or the third weight information (i.e., weight based on directional consistency). The weight based on spatial distance can suppress the interference of edge pixels and strengthen the dominance of the central region, while the weight based on directional consistency can strengthen the statistical weight of the local consistent texture direction and improve the salience of the dominant edge pattern. This improves the accuracy of the candidate pattern and / or predicted pattern determined or obtained by at least one gradient histogram, and improves the prediction accuracy of the predicted pattern for the current block.

[0540] Method A6: Area information of at least one reference region of the current block.

[0541] Optionally, step S10 includes: determining or obtaining at least one area histogram based on the area information of at least one reference region of the current block; determining or obtaining at least one candidate pattern based on the at least one area histogram; and determining or obtaining at least one prediction pattern of the current block based on the matching information of the at least one candidate pattern and the current block.

[0542] Optionally, based on the area information of at least one reference region of the current block, at least one area histogram is determined or obtained, and based on the at least one area histogram and at least one gradient histogram, at least one candidate pattern is determined or obtained.

[0543] Optionally, based on the intra-prediction mode usage of at least one reference region in the current block (including at least one adjacent coding block and / or non-adjacent coding block in the current block), the area information of at least one reference region is determined or obtained. The area information includes: area magnitude value and intra-prediction direction. Based on the area information of at least one reference region, at least one area histogram is determined or obtained.

[0544] Optionally, at least one reference region includes at least one adjacent coded block and / or a non-adjacent coded block of the current block, and the area information of the at least one reference region includes: the area magnitude value and intra-prediction direction of at least one adjacent coded block, and / or the area magnitude value and intra-prediction direction of at least one non-adjacent coded block.

[0545] Reference Figure 10 The reference region for the block to be predicted (i.e., the current block) in the coded region includes: coded blocks 4 and 6 adjacent to the block to be predicted, and coded blocks 1, 2, 3, 5, 7, and 8 that are not adjacent to the block to be predicted, such as... Figure 11 As shown, in the decoded region there are decoded blocks 4 and 6 adjacent to the block to be predicted, and decoded blocks 1, 2, 3, 5, 7 and 8 that are not adjacent to the block to be predicted. Based on the area amplitude value and intra-frame prediction direction of at least one encoded block and / or at least one decoded block, at least one area histogram is determined or obtained.

[0546] Optionally, the number of prediction patterns corresponding to the area histogram is either a first number or a second number.

[0547] Optionally, at least one area histogram is determined or obtained based on at least one lookup table and area information of at least one reference region of the current block.

[0548] Optionally, at least one lookup table includes: lookup table A and / or lookup table B, where the number of prediction patterns corresponding to lookup table A is a first number, and the number of prediction patterns corresponding to lookup table B is a second number, and the first number and the second number may be the same or different.

[0549] Optionally, the first quantity is 35, 67 or 131, for example, the first quantity is 131.

[0550] Optionally, the first quantity is greater than 35, 67, or 131.

[0551] Alternatively, the second quantity can be 35, 67, or 131, for example, the second quantity can be 67.

[0552] Optionally, the second quantity is less than or equal to 35, 67, or 131.

[0553] Optionally, the first quantity and the second quantity are the same, that is, the number of prediction patterns corresponding to lookup table A is the same as the number of prediction patterns corresponding to lookup table B. For example, the number of prediction patterns corresponding to lookup table A and lookup table B is the same, but the types of prediction patterns corresponding to each other are at least partially different.

[0554] Optionally, if the prediction mode corresponding to at least one adjacent coding block and / or non-adjacent coding block of the current block is determined or obtained according to lookup table A, and the number of prediction modes corresponding to lookup table A is a first number, then the number of prediction modes corresponding to at least one area histogram determined or obtained from the area information of at least one reference region of the current block is the first number, for example, the first number is 131.

[0555] Optionally, if the prediction pattern corresponding to at least one adjacent coding block and / or non-adjacent coding block of the current block is determined or obtained according to lookup table B, and the number of prediction patterns corresponding to lookup table B is a second number, then the number of prediction patterns corresponding to at least one area histogram determined or obtained from the area information of at least one reference region of the current block is the second number, for example, the second number is 67.

[0556] In this embodiment, the possible edge structure of the current block is quickly inferred by utilizing the area and orientation information of the adjacent and / or non-adjacent blocks of the current block, and an area histogram is established. Then, candidate patterns are directly filtered through the area histogram, for example, high-frequency prediction directions are retained, thereby improving the matching accuracy of prediction patterns and improving the prediction accuracy for the current block.

[0557] Optionally, the method of determining or obtaining at least one candidate pattern based on at least one statistical histogram includes the following methods a1 and / or a2:

[0558] Method a1: Based on the first statistical histogram of the current block under a first number of prediction modes and / or the second statistical histogram of the current block under a second number of prediction modes, at least one candidate mode is determined or obtained.

[0559] Optionally, the statistical histogram includes at least one of the following: a gradient histogram, an area histogram, and a histogram of the number of times the predicted mode is used corresponding to the encoded image patch, as well as a histogram obtained by adjusting or merging the above histograms.

[0560] Optionally, step S10 includes: determining or obtaining at least one candidate pattern based on the first statistical histogram of the current block under a first number of prediction patterns and / or the second statistical histogram of the current block under a second number of prediction patterns; and determining or obtaining at least one prediction pattern of the current block based on the matching information of the at least one candidate pattern with the current block.

[0561] Optionally, the first statistical histogram and the second statistical histogram may belong to the same or different histogram types. For example, the first statistical histogram is a gradient histogram, and the second statistical histogram is an area histogram.

[0562] Optionally, at least one candidate pattern is determined or obtained based on the first statistical histogram of the current block under the first number of prediction patterns, and the obtained candidate pattern is one of the first number of prediction patterns.

[0563] Optionally, the first number is 131. A first gradient histogram of at least one reference region of the current block under 131 prediction modes is determined or obtained. The first gradient histogram records the gradient magnitude of at least one reference region of the current block in each gradient direction corresponding to the 131 prediction modes. Based on the gradient magnitude in each gradient direction in the first gradient histogram, at least one candidate mode is determined or obtained. For example, the modes corresponding to the top 3 gradient directions with the highest gradient magnitude are determined as candidate modes.

[0564] Optionally, the first number is 67. The first gradient histogram of at least one reference region of the current block under 67 prediction modes is determined or obtained. The first gradient histogram records the gradient magnitude of at least one reference region of the current block in the gradient direction corresponding to the 67 prediction modes. Based on the gradient magnitude in each gradient direction in the first gradient histogram, at least one candidate mode is determined or obtained. For example, the mode corresponding to the top 3 gradient directions with the highest gradient magnitude is determined as the candidate mode.

[0565] Optionally, at least one candidate pattern is determined or obtained based on the second statistical histogram of the current block under the second number of prediction patterns, and the obtained candidate pattern is one of the prediction patterns of the second number.

[0566] Optionally, the second number is 67. A second gradient histogram of at least one reference region of the current block under 67 prediction modes is determined or obtained. The second gradient histogram records the gradient magnitude of at least one reference region of the current block in the gradient direction corresponding to the 67 prediction modes. Based on the gradient magnitude in each gradient direction in the second gradient histogram, at least one candidate mode is determined or obtained. For example, the modes corresponding to the top 3 gradient directions with the highest gradient magnitude are determined as candidate modes.

[0567] Optionally, the second number is 131, and a second gradient histogram of at least one reference region of the current block under 131 prediction modes is determined or obtained. The second gradient histogram records the gradient magnitude of at least one reference region of the current block in the gradient direction corresponding to the 131 prediction modes. Based on the gradient magnitude in each gradient direction in the second gradient histogram, at least one prediction mode is determined or obtained. For example, the mode corresponding to the top 3 gradient directions with the highest gradient magnitude is determined as the prediction mode of the current block.

[0568] Optionally, the first quantity and the second quantity are different. For example, the first quantity is greater than the second quantity. If the size of the current block meets the first condition, then at least one candidate pattern is determined or obtained based on the first statistical histogram of the current block under the prediction pattern of the first quantity. If the size of the current block does not meet the first condition, then at least one candidate pattern is determined or obtained based on the second statistical histogram of the current block under the prediction pattern of the second quantity.

[0569] Optionally, the first quantity and the second quantity are different. For example, the first quantity is greater than the second quantity. If the size of the current block satisfies the first condition, then at least one candidate pattern is determined or obtained based on the first statistical histogram of the current block under the prediction mode of the first quantity and / or the second statistical histogram of the current block under the prediction mode of the second quantity. If the size of the current block does not satisfy the first condition, then at least one candidate pattern is determined or obtained based on the second statistical histogram of the current block under the prediction mode of the second quantity.

[0570] Optionally, the first quantity and the second quantity are different. If the size of the current block meets the first condition, then at least one candidate pattern is determined or obtained based on the first statistical histogram of the current block under the prediction mode of the first quantity. If the size of the current block does not meet the first condition, then at least one candidate pattern is determined or obtained based on the first statistical histogram of the current block under the prediction mode of the first quantity and / or the second statistical histogram of the current block under the prediction mode of the second quantity.

[0571] Optionally, the first quantity and the second quantity are different, and at least one candidate pattern is determined or obtained based on the first statistical histogram of the current block under the prediction pattern of the first quantity and the second statistical histogram of the current block under the prediction pattern of the second quantity.

[0572] Optionally, the first quantity and the second quantity are different. Based on the first statistical histogram of the current block under the prediction mode of the first quantity, at least one prediction mode A is determined or obtained. Based on the second statistical histogram of the current block under the prediction mode of the second quantity, at least one prediction mode B is determined or obtained. Based on the matching information of at least one prediction mode A and at least one prediction mode B related to at least one reference region of the current block, at least one candidate mode is determined or obtained from at least one prediction mode A and at least one prediction mode B.

[0573] Optionally, matching information refers to information used to evaluate the degree of matching between different prediction modes and the reference region and its corresponding current block. Matching information can be based on various indicators, such as the rate-distortion cost of the prediction results, prediction error, texture similarity, etc., to indicate the degree of matching between different prediction modes and the reference region.

[0574] Optionally, the matching information includes: SAD, SATD and / or MRSAD. Optionally, the smaller the SAD value, SATD value and / or MRSAD value, the higher the degree of matching, and the larger the SAD value, SATD value and / or MRSAD value, the lower the degree of matching.

[0575] In this embodiment, the prediction pattern matching requirements of current blocks of different sizes are met by using the first statistical histogram of the first number of prediction patterns and / or the second statistical histogram of the current block of the second number of prediction patterns. For example, large-sized image blocks are more sensitive to the texture direction for the accuracy of angle prediction in the prediction pattern. Therefore, the number of angle prediction patterns in the prediction pattern can be increased to more finely match the actual texture direction and improve prediction accuracy. On the other hand, small-sized image blocks are less sensitive to the number of angle prediction patterns in the prediction pattern. Therefore, a statistical histogram with fewer prediction patterns can be used to determine the prediction pattern to improve prediction efficiency.

[0576] Method a2: Determine or obtain at least one first pattern based on the second statistical histogram of the current block under the second number of prediction patterns, and determine or obtain at least one candidate pattern based on at least one second pattern of the current block under the first number of prediction patterns determined or obtained through the at least one first pattern.

[0577] Optionally, step S10 includes: determining or obtaining at least one first pattern based on the second statistical histogram of the current block under the second number of prediction patterns; determining or obtaining at least one candidate pattern based on at least one second pattern of the current block under the first number of prediction patterns determined or obtained through the at least one first pattern; and determining or obtaining at least one prediction pattern of the current block based on the matching information of the at least one candidate pattern and the current block.

[0578] Optionally, the statistical histogram includes at least one of the following: a gradient histogram, an area histogram, and a histogram of the number of times the predicted mode is used corresponding to the encoded image patch, as well as a histogram obtained by adjusting or merging the above histograms.

[0579] Optionally, if the first quantity is greater than the second quantity, at least one first pattern is determined or obtained based on the second statistical histogram of the current block under the prediction pattern of the second quantity, and the first pattern is one of the prediction patterns of the second quantity. At least one candidate pattern is determined or obtained based on at least one second pattern of the current block under the prediction pattern of the first quantity, determined or obtained through at least one first pattern.

[0580] Optionally, the first number is 131 and the second number is 67. The prediction modes under the first number completely cover the prediction modes under the second number. The number of prediction modes is increased by filling the angle intervals between the angle prediction modes in the original prediction modes under the second number. Step S10 includes: determining or obtaining the second gradient histogram of at least one reference region of the current block under 67 prediction modes. The second gradient histogram records the gradient magnitude of at least one reference region of the current block in the gradient direction corresponding to the 67 prediction modes. Based on the gradient magnitude in each gradient direction in the second gradient histogram, at least one first mode is determined or obtained. For example, the mode corresponding to the gradient direction of the top 3 gradient magnitudes is determined as the first mode. Then, the first mode is converted into the mode corresponding to the 131 prediction modes to obtain at least one second mode. For example, mode 23 (i.e., the first mode) in the 67 prediction modes corresponds to mode 44 (i.e., the second mode) in the 131 prediction modes. Then, at least one candidate mode is determined or obtained based on the at least one second mode.

[0581] Optionally, based on at least one mapping table and at least one first mode, at least one second mode of the current block in a first number of prediction modes is determined or obtained, wherein the mapping table includes: a first mapping table and / or a second mapping table.

[0582] Optionally, the first number is 131 and the second number is 67. The conversion from 67 prediction models to 131 prediction models can be determined or obtained based on the first mapping table, which is shown in Table 3 below:

[0583] Table 3

[0584]

[0585]

[0586] Optionally, the first number is 131 and the second number is 67. The conversion from 131 prediction modes to 67 prediction modes can be determined or obtained according to the second mapping table, which is shown in Table 4 below:

[0587] Table 4

[0588]

[0589]

[0590] Optionally, the step of determining or obtaining at least one candidate mode based on at least one second mode includes: determining or obtaining at least one candidate mode based on at least one second mode and / or neighboring modes of at least one second mode under a first number of prediction modes.

[0591] Optionally, in the first and / or second number of prediction modes, each mode corresponds to a discretized direction. For example, in the 67 prediction modes, mode 18 is 0° (horizontal direction) and mode 34 is 45° diagonal. Therefore, the adjacent modes can be at least one prediction mode that is closest to the angle value and / or index of the prediction mode. For example, in the 67 prediction modes, if the prediction mode is mode 34 (45°), then its adjacent modes can be mode 33 and mode 35.

[0592] Optionally, at least one first pattern, such as a1 and a2, is determined or obtained based on a second statistical histogram of the current block under 67 prediction patterns. At least one second pattern, such as b1 and b2, is determined or obtained based on the at least one first pattern. At least one neighboring pattern of the at least one second pattern is determined or obtained under the first number of prediction patterns. For example, b1 is adjacent to b1' and b1”, and b2 is adjacent to b2' and b2”. At least one candidate pattern is determined or obtained based on the at least one second pattern and the neighboring patterns of the at least one second pattern under the first number of prediction patterns. For example, at least one candidate pattern is determined or obtained from b1, b1', b1”, b2, b2', and b2” based on the matching information of the at least one second pattern and the neighboring patterns of the at least one second pattern under the first number of prediction patterns with at least one reference region of the current block.

[0593] Optionally, matching information refers to information used to evaluate the degree of matching between different prediction modes and the reference region and its corresponding current block. Matching information can be based on various indicators, such as the rate-distortion cost of the prediction results, prediction error, texture similarity, etc., to indicate the degree of matching between different prediction modes and the reference region.

[0594] Optionally, the matching information includes: SAD, SATD, and / or MRSAD.

[0595] In this embodiment, at least one first mode is determined or obtained based on the second statistical histogram of the current block under the second number of prediction modes. At least one candidate mode is determined or obtained based on at least one second mode of the current block under the first number of prediction modes determined or obtained through the at least one first mode. This can reduce the number of statistical histograms to be built, reduce computational complexity, and at the same time meet the prediction mode matching requirements of current blocks of different sizes. It can match suitable prediction modes for current blocks of different sizes, thereby supporting the improvement of prediction accuracy of the current block and thus supporting the improvement of encoding and / or decoding quality in the video encoding and / or decoding process.

[0596] Fourth embodiment

[0597] Based on any of the above embodiments, a fourth embodiment is proposed.

[0598] In this embodiment, the first pattern list is determined or obtained according to at least one of the following methods B1 to B6:

[0599] Method B1, at least one reference region and neural network model for the current block.

[0600] Optionally, step S10 includes: determining or obtaining a first pattern list based on at least one reference region and a neural network model of the current block; determining or obtaining at least one candidate pattern based on the first pattern list; and determining or obtaining at least one prediction pattern of the current block based on the matching information of the at least one candidate pattern and the current block.

[0601] Optionally, the first pattern list is a set for storing at least one prediction pattern, providing multiple possible pattern options for the prediction of the current block. By comparing the prediction effects of different patterns in the first pattern list, a suitable prediction pattern can be matched for the current block to improve the prediction accuracy for the current block.

[0602] Optionally, the first pattern list is at least one pattern list, where the prediction patterns in the at least one pattern list are different or at least partially the same.

[0603] Optionally, the first mode list includes an MPM (Most Probable Modes) list. Optionally, the first mode list in this embodiment can be an improved MPM list.

[0604] Optionally, the MPM list is a collection of several prediction patterns that are considered to be the most likely prediction patterns to apply to the current block. The determination or acquisition of the MPM list is based on the neighborhood information of the current block (such as the prediction patterns of the blocks above and to the left) and statistical information (such as the probability of pattern occurrence). In this way, the MPM list can reduce the amount of pattern information that the encoder needs to transmit, while improving the prediction accuracy of the decoder for the prediction patterns.

[0605] Optionally, in intra-frame prediction, the candidate modes in the first mode list may include: at least one angular prediction mode, at least one non-angular prediction mode, and / or a derived mode of at least one prediction mode.

[0606] Optionally, a first pattern list can be determined or obtained based on the pixels of at least one reference region of the current block and the neural network model.

[0607] Optionally, a first pattern list can be determined or obtained based on gradient information of at least one reference region of the current block and the neural network model.

[0608] Optionally, a first pattern list can be determined or obtained based on the pixels, gradient information, and neural network model of at least one reference region of the current block.

[0609] Optionally, by using the pixel and / or gradient information of at least one reference region of the current block as input to the neural network model, at least one prediction mode and / or the confidence of at least one prediction mode under a first number and / or a second number are determined or obtained, and a first mode list is determined or obtained based on the confidence of at least one prediction mode and / or at least one prediction mode output by the neural network model.

[0610] Optionally, the first quantity is 35, 67 or 131, for example, the first quantity is 131.

[0611] Optionally, the first quantity is greater than 35, 67, or 131.

[0612] Alternatively, the second quantity can be 35, 67, or 131, for example, the second quantity can be 67.

[0613] Optionally, the second quantity is less than or equal to 35, 67, or 131.

[0614] Optionally, based on the confidence level of at least one predicted pattern determined or obtained by the neural network model, a first pattern list is determined or obtained based on at least one predicted pattern under a first quantity and / or a second quantity output by the neural network model.

[0615] Optionally, confidence level is a quantitative indicator of the degree of certainty the model has about the current prediction result.

[0616] Optionally, a neural network model is a computational model that mimics how neurons in the human brain process information. It performs various tasks by learning complex patterns and features in data. A neural network model consists of at least one layer of neurons, including an input layer, at least one hidden layer, and an output layer. Each neuron receives an input signal, which is then weighted and summed before being passed through a nonlinear activation function to produce an output. The weights of these connected neurons determine the degree of influence of the input signal on the output, while the bias is used to adjust the activation threshold of the neurons. Activation functions, such as ReLU, Sigmoid, or Tanh, can give neural networks the ability to model nonlinearly, enabling them to solve problems that linear models cannot solve.

[0617] Reference Figure 12 The specific implementation process of determining or obtaining the first pattern list through a neural network model may include: determining or obtaining the neural network model corresponding to the current block, and determining or obtaining a reference template of the neural network model (e.g., based on at least one reference region of the current block). Figure 12The reference template 1 on the left, the reference template 2 above, and the reference template 3 on the upper left are used as inputs to the neural network model. The neural network model obtains the output of the neural network model by using the correlation between the current block and its reference templates learned by the neural network model. For example, the confidence of at least one candidate pattern and / or at least one candidate pattern under the first and / or second number of times.

[0618] Optionally, the reference area includes: a reference block, a reference pixel, a reference frame, and / or a reference template.

[0619] Optionally, the reference region can be an image region adjacent to the current block, or an image region not adjacent to the current block. Optionally, the reference region can be a region composed of encoded and reconstructed pixels or blocks from the same frame and / or different frames.

[0620] Optionally, since there is a high correlation between the reference region and the current block (e.g., temporal correlation, spatial correlation, etc.), the embodiments of this application can significantly improve the matching degree between the obtained candidate pattern and the current block by utilizing these correlations, and improve the accuracy of the prediction results of the current block determined or obtained by using the candidate pattern as the prediction pattern.

[0621] Optionally, the reference block can be a pixel block corresponding to the current block extracted from a reference frame (also known as a reference image) or the current frame (also known as the current image). The reference block can have the same size as the current block, and the reference block can be used as a prediction block for generating the current block. By using the reference block, the encoder can take advantage of temporal or spatial correlations to reduce redundant information in the current frame, thereby achieving efficient compression.

[0622] Optionally, at least one reference block of the current block is determined or obtained based on an index obtained from the bitstream.

[0623] Optionally, a reference frame refers to a frame (also called an image) that has been encoded and reconstructed. These frames can be forward reference frames (i.e., frames / images that are preceding the current frame in the playback order) or backward reference frames (i.e., frames / images that are following the current frame in the playback order).

[0624] Optionally, a neural network model may include an input layer, a hidden layer, a dropout layer, and an output layer.

[0625] Optionally, the input layer is the first layer that receives input data, in which each node (or neuron) typically represents a feature of the input data.

[0626] Optionally, hidden layers are layers located between the input layer and the output layer. There can be multiple hidden layers, which process the input data in a weighted manner.

[0627] Optionally, the hidden layer includes at least one of the following: a fully connected layer, a convolutional layer, a pooling layer, a recursive layer, and a dropout layer.

[0628] Optionally, each neuron in a dense or fully connected layer is connected to every neuron in the layer above it.

[0629] Optionally, a convolutional layer is used to extract local features from the input data and is commonly used in image processing.

[0630] Optionally, a pooling layer can be used for downsampling to reduce the amount of data and computation.

[0631] Alternatively, a recurrent layer, such as LSTM or GRU, can be used to process sequential data.

[0632] Optionally, a dropout layer is used to randomly "drop out" (i.e. temporarily set the output to zero) a portion of neurons during model training. This can reduce the complex co-adaptation relationships between neurons, improve the model's generalization ability, and reduce the risk of overfitting.

[0633] Optionally, dropout layers can be added between at least one fully connected layer to prevent neurons in these layers from becoming overly reliant on the specific output of the previous layer, thereby enhancing the model's generalization ability. Dropout layers can be added between convolutional layers and fully connected layers to help reduce overfitting introduced by the fully connected layers. The output layer is the layer that generates the final prediction result.

[0634] Optionally, the first layer of the neural network model is used to construct a learnable gradient convolution, which determines or obtains the gradient information of the current block in at least one reference region through the convolution operation.

[0635] Optionally, if the first layer of the neural network model is used to construct a learnable gradient convolution, the neural network model can determine or obtain the gradient information of the current block in at least one reference region through the convolution operation. Therefore, the pixels of at least one reference region of the current block can be used as the input of the neural network model.

[0636] Optionally, if a learnable gradient convolution is not constructed in the first layer of the neural network model, the gradient information of at least one reference region of the current block is used as the input to the neural network model.

[0637] Optionally, a first neural network model can be determined or obtained from a set of neural network models; a first pattern list can be determined or obtained based on the first neural network model and at least one reference region of the current block, wherein the first neural network model is at least one model.

[0638] Optionally, a first neural network model is determined or obtained from a set of neural network models based on at least one reference region of the current block and a preset model mapping / correspondence rule.

[0639] Optionally, the pre-training includes a set of neural network models, including at least one of the following: a neural network model based on fully connected layers, a neural network based on convolutional layers, and a neural network based on a mixture of convolutional and fully connected layers.

[0640] Optionally, a neural network model based on fully connected layers, also known as a multilayer perceptron (MLP), includes an input layer, one or more hidden layers (i.e., fully connected layers), and an output layer. Each layer contains a certain number of neurons, and all neurons between adjacent layers are interconnected, i.e., "fully connected" between layers. Optionally, dropout layers may be set between at least one set of fully connected layers in a neural network model based on fully connected layers to help reduce overfitting introduced by fully connected layers.

[0641] Reference Figure 13 A fully connected neural network model includes an input layer, at least one hidden layer (i.e., a fully connected layer), and an output layer. The fully connected layer includes a linear layer and an activation function layer (e.g., a LeakyReLU layer). Pixels and gradient information of at least one reference region of the current block are input into the fully connected neural network model. Since the pixels of the reference region may exist in the form of a two-dimensional or three-dimensional matrix, in order to feed the above data into the fully connected layer for processing, the input layer can be flattened to transform the pixels of at least one reference region into a one-dimensional vector f, and then a nonlinear transformation using the ReLU activation function is performed to determine or obtain f. i This operation introduces nonlinearity, allowing the model to learn more complex patterns, f i The input will be fed into a network composed of fully connected layers. Each layer will perform linear transformations and activation function processing, and output a new feature vector f. r This is to extract higher-level abstract features from the input, and then to transform the high-level features f r With low-level features f i Concatenate the data along the channel dimension to generate the feature vector f. c , the feature vector f cThe data is fed into a linear layer and output through an output layer to show at least one prediction pattern and / or the confidence level of at least one prediction pattern under a first quantity and / or a second quantity.

[0642] Optionally, a convolutional neural network (CNN) includes an input layer, at least one hidden layer, and an output layer. The hidden layer includes convolutional layers for feature extraction; each convolutional layer may be followed by a pooling layer to reduce spatial size and computational complexity; and each convolutional layer may also be followed by a non-linear activation function (e.g., ReLU) to introduce non-linearity, enabling the network to learn more complex patterns. As the depth of the convolutional neural network increases, it can automatically learn feature representations from low to high levels, improving prediction accuracy.

[0643] Reference Figure 14 A convolutional neural network (CNN) includes an input layer, at least one hidden layer, and an output layer. The hidden layer includes a convolutional layer, an activation function layer (e.g., a Leaky ReLU layer), and a linear layer. Pixels from a reference region of the current block are input into the CNN. At least one convolutional layer performs convolution operations on the pixels of at least one reference region, and a non-linear transformation is performed using the ReLU activation function after each convolutional layer to obtain the feature f. 1 f 2 and f 3 Features f1 and f 2 and f 3 The features are fused and concatenated along the channel dimension to determine or obtain a new merged feature f containing three features. This preserves information from different convolutional paths, allowing the model to combine low-level and high-level features and enhance its representational power. The fused feature f is then processed again through the ReLU activation function to obtain fi. i This additional activation step helps to further enhance the model's expressiveness and flexibility. (The last part, "f", appears to be a typo and can be left as is.) i The data is fed into a linear layer to obtain at least one prediction pattern and / or the confidence level of at least one prediction pattern under the first and / or second number of prediction patterns.

[0644] Optionally, a neural network model based on hybrid convolutional and fully connected layers is an architecture that combines the advantages of CNN and MLP. It includes convolutional layers and fully connected layers. Convolutional layers can automatically learn the spatial hierarchical structure of input data, such as features like edges, textures, and shapes in images. Convolutional operations can effectively capture local patterns and reduce model complexity through parameter sharing mechanisms. Fully connected layers can flatten the features extracted by the convolutional layers into one-dimensional vectors, which are then fed into a series of fully connected layers. These layers are responsible for combining low-level features to determine or obtain higher-level abstract representations. A neural network model based on hybrid convolutional and fully connected layers can combine the advantages of two different types of layers, enabling efficient feature extraction from input data and accurate predictions based on those features.

[0645] Optionally, the neural network model based on hybrid convolutional and fully connected layers includes: an input layer, at least one convolutional layer, at least one fully connected layer, and an output layer.

[0646] Reference Figure 15 A neural network model based on hybrid convolutional and fully connected layers includes an input layer, at least one hidden layer, and an output layer. The hidden layer includes a convolutional layer, a fully connected layer (i.e., a linear layer), and an activation function layer (e.g., a Leaky ReLU layer). Pixels of a reference region of the current block are input into this neural network model. At least one convolutional layer is used to convolve the pixels of at least one reference region. A non-linear transformation is then performed using the ReLU activation function after each convolutional layer to obtain the feature f. 1 f 2 and f 3 , will feature f 1 f 2 and f 3 By fusing and stitching along the channel dimension, a new merged feature f containing three features is determined or obtained. 4 To preserve information from different convolutional paths, the model combines low-level and high-level features, enhancing representational power, and obtains features f through at least one stacked linear layer. 5 , will feature f 4 and f 5 The data is spliced ​​along the channel dimension to form a fused feature f. i The fused features f are processed through at least one convolutional layer. i Perform convolution processing to obtain at least one prediction pattern and / or the confidence level of at least one prediction pattern under a first number and / or a second number.

[0647] Optionally, the construction of a neural network model mainly includes: dataset construction and neural network construction and training.

[0648] Optionally, a first pattern list can be determined or obtained based on the output of the neural network model.

[0649] In this embodiment, the powerful nonlinear modeling capability of the neural network model can automatically learn the complex mapping relationship between input and output from a large amount of data. Therefore, the neural network can identify and utilize the subtle relationship between the current block and its reference region: for example, effectively capture the temporal correlation between frames and the spatial correlation within frames, understand the continuity and change patterns between different frames or adjacent regions within the same frame; and / or, perceive the consistency of local features such as edges, textures, etc., even if these features are shifted or deformed between different frames. Through multi-level feature extraction, it ensures that the prediction results are not only accurate in local details, but also conform to the actual content in the overall layout, so as to improve the accuracy of the candidate modes in the determined or obtained first mode list for the prediction of the current block from at least one dimension, and improve the encoding and / or decoding quality in the video encoding and / or decoding process; and / or the neural network model can output the confidence of at least one prediction mode at a time through parallel inference, thereby improving the prediction efficiency.

[0650] Optionally, step S10 includes: determining or obtaining a first pattern list based on the first neural network model and at least one reference region of the current block; determining or obtaining at least one candidate pattern based on the first pattern list; and determining or obtaining at least one prediction pattern of the current block based on the matching information of the at least one candidate pattern and the current block.

[0651] Optionally, the first neural network model is determined or obtained according to the following methods b1 and / or b2:

[0652] Method b1, at least one of the following: width, height, block size, and block area of ​​the current block;

[0653] Optionally, a first neural network model corresponding to the target model structure is determined from at least one model structure based on at least one of the width, height, size, and area of ​​the current block.

[0654] Optionally, based on at least one of the geometric attributes of the current block (i.e., width, height, block size, and block area), the model structure most suitable for processing these features is selected, so that the first neural network model can more accurately reflect the characteristics of the input data and improve the model prediction effect.

[0655] Optionally, the structure of the first neural network model is determined based on at least one of the width, height, size, and area of ​​the current block.

[0656] Optionally, since at least one of the width, height, size, and area of ​​the current block is associated with at least one reference region of the current block, a first neural network model suitable for processing the reference region and / or reference block data of the current block can be determined or obtained by at least one of the width, height, size, and area of ​​the current block.

[0657] Optionally, the specific structure of the selected model can be adjusted based on at least one of the geometric attributes of the current block (i.e., width, height, block size, and block area). For example, the number of hidden layers (e.g., convolutional layers, fully connected layers), filter size, pooling strategy, etc. of the neural network model can be adjusted based on at least one of the width, height, size, and area of ​​the current block.

[0658] Optionally, the structure of the first neural network model can be determined based on the width, height, and third mapping table of the current block.

[0659] Alternatively, the third mapping table can be as shown in Table 5 below:

[0660] Table 5

[0661] Width and height First Neural Network Model <![CDATA[Any one of the width and height of the current block is less than X1]]> First Structure <![CDATA[Both the width and height of the current block are greater than or equal to X1, and either one of the width and the height is less than X2]]> Second structure <![CDATA[Both the width and the height of the current block are greater than or equal to X2]]> Third Structure <![CDATA[Both the width and height of the current block are greater than or equal to X2, and either one of the width and height is less than X3]]> Fourth Structure

[0662] Optionally, the structure of the first neural network model can be determined based on the block size of the current block and the fourth mapping table.

[0663] Alternatively, the fourth mapping table can be as shown in Table 6 below:

[0664] Table 6

[0665] Block size First Neural Network Model <![CDATA[the block size of the current block is smaller than X4]]> First Structure <![CDATA[the block size of the current block is greater than X4 and less than X5]]> Second Structure <![CDATA[the block size of the current block is greater than X5 and less than X6]]> Third Structure <![CDATA[the block size of the current block is greater than X6 and smaller than X7]]> Fourth Structure

[0666] Optionally, the structure of the first neural network model can be determined based on the block size of the current block and the fifth mapping table.

[0667] Alternatively, the fifth mapping table can be as shown in Table 7 below:

[0668] Table 7

[0669]

[0670]

[0671] Optionally, if the current block and at least one reference region of the current block do not match any of the neural network models, the reconstructed reference pixels of the current block and at least one reference region of the current block can be transposed and / or downsampled to match the geometric properties of the current block and / or at least one reference region of the current block with the neural network model (e.g., the same size). Then, the reconstructed reference pixels obtained by transposition and / or downsampling are input into the neural network model to obtain the prediction result of the model. The number of target models and / or target model structures is reduced as a whole by transposition or downsampling.

[0672] Optionally, the number of neural network models is N, for example, including neural network model NN0, neural network model NN1, neural network model NN2... neural network model NN N-1 Neural network model NN0, neural network model NN1, neural network model NN2, ..., neural network model NN N-1 Corresponding to block sizes W0xH0, W1xH1, W2xH2, ..., W N-1 xH N-1 .

[0673] In this embodiment, based on at least one of the width, height, block size, and block area of ​​the current block, the model most suitable for processing these features is determined or obtained, so that the first neural network model can more accurately reflect the characteristics of the input data, improve the accuracy of at least one candidate pattern in the first pattern list determined or obtained by the model for the current block prediction, and improve the encoding and / or decoding quality in the video encoding and / or decoding process.

[0674] Method b2, at least one of the width, height, size, and area of ​​at least one reference region of the current block;

[0675] Optionally, a first neural network model corresponding to the target model structure is determined from at least one model structure based on at least one of the width, height, size, and area of ​​at least one reference region of the current block.

[0676] Optionally, based on at least one of the geometric attributes (i.e., width, height, block size, and block area) of at least one reference region of the current block, the model structure most suitable for processing these features is selected, so that the first neural network model can more accurately reflect the characteristics of the input data and improve the model prediction effect.

[0677] Optionally, the structure of the first neural network model is determined based on at least one of the width, height, size, and area of ​​at least one reference region of the current block.

[0678] Optionally, the specific structure of the selected model can be adjusted based on at least one of the geometric properties (i.e., width, height, block size, and block area) of at least one reference region of the current block. For example, the number of hidden layers (e.g., convolutional layers, fully connected layers), filter size, pooling strategy, etc. of the neural network model can be adjusted based on at least one of the width, height, size, and area of ​​at least one reference region.

[0679] Optionally, the structure of the first neural network model can be determined based on the width, height, and sixth mapping table of at least one reference region of the current block.

[0680] Alternatively, the sixth mapping table can be as shown in Table 8 below:

[0681] Table 8

[0682] Width and height First Neural Network Model <![CDATA[Either of the width and height of the reference region is less than X1]]> First Structure <![CDATA[Both the width and the height of the reference region are greater than or equal to X1, and either of the width and the height is less than X2]]> Second structure <![CDATA[The width and height of the reference region are both greater than or equal to X2]]> Third Structure <![CDATA[The width and height of the reference region are both greater than or equal to X2, and either of the width and height is less than X3]]> Fourth Structure

[0683] Optionally, the structure of the first neural network model can be determined based on the block size of at least one reference region of the current block and the seventh mapping table.

[0684] Alternatively, the seventh mapping table can be as shown in Table 9 below:

[0685] Table 9

[0686] Block size First Neural Network Model <![CDATA[the block size of the reference region is smaller than X4]]> First Structure <![CDATA[the block size of the reference area is greater than X4 and smaller than X5]]> Second structure <![CDATA[the block size of the reference area is greater than X5 and smaller than X6]]> Third Structure <![CDATA[the block size of the reference area is greater than X6 and less than X7]]> Fourth Structure

[0687] Optionally, the block size includes at least one of the block's width, height, scale, depth, area, resolution, and number of pixels. Optionally, X4 to X7 can be a preset threshold corresponding to at least one of the block size's width, height, scale, depth, area, resolution, and number of pixels.

[0688] Optionally, the structure of the first neural network model can be determined based on the block size of at least one reference region of the current block and the eighth mapping table.

[0689] Alternatively, the eighth mapping table can be as shown in Table 10 below:

[0690] Table 10

[0691] Block area First Neural Network Model <![CDATA[the block area of the reference region is less than X8]]> First Structure <![CDATA[the block area of the reference region is greater than X8 and less than X9]]> Second Structure <![CDATA[The block area of the reference region is greater than X9 and less than X 10 > Third Structure <![CDATA[the block area of the reference region is greater than X 10 and less than X 11 > Fourth Structure

[0692] In this embodiment, based on at least one of the width, height, size, and area of ​​at least one reference region of the current block, the model most suitable for processing these features is determined or obtained, so that the first neural network model can more accurately reflect the characteristics of the input data, improve the accuracy of the prediction of at least one candidate mode in the first mode list determined or obtained by the model for the current block, and improve the encoding and decoding quality in the video encoding and / or decoding process.

[0693] Method B2: Matching information related to at least one reference region of the current block.

[0694] Optionally, step S10 includes: determining or obtaining a first pattern list based on matching information related to at least one reference region of the current block; determining or obtaining at least one candidate pattern based on the first pattern list; and determining or obtaining at least one prediction pattern of the current block based on matching information related to the at least one candidate pattern and the current block.

[0695] Optionally, at least one reference region of a block refers to a pixel region or pixel sample region located in the left, top, or upper left of the current block, which is adjacent or not adjacent to the reconstructed or encoded region. The pixel values ​​or features of these regions can be used for prediction or other processing of the current block. Optionally, the reference region may include: a reference template, a reference block, and / or a reference pixel.

[0696] Optionally, the matching information refers to the information used to evaluate the degree of matching between different prediction modes and reference regions and their corresponding current blocks. The higher the degree of matching between the prediction mode and the current block and / or at least one reference region of the current block, the higher the accuracy of the prediction result of the current block obtained by using the prediction mode.

[0697] Optionally, the matching information includes: SAD, SATD and / or MRSAD. The smaller the SAD, SATD and / or MRSAD values, the higher the degree of matching. The larger the SAD, SATD and / or MRSAD values, the lower the degree of matching.

[0698] Optionally, a reference region of the current block can be matched with at least one prediction pattern to determine or obtain at least one matching information related to the reference region.

[0699] In this embodiment, by quantifying the matching information, the degree of matching between the predicted pattern and at least one reference region of the current block can be quickly selected as candidate patterns in the first pattern list. This can improve the fit between the candidate patterns in the pattern list and the current block, and also improve the fit between the predicted pattern determined or obtained based on the first pattern list and the current block, thereby improving the prediction effect of the current block. Furthermore, the matching information filtering method can also save computing resources, reduce computational complexity, and improve coding efficiency.

[0700] Method B3: Matching information related to at least one reference region of the current block's sub-blocks.

[0701] Optionally, step S10 includes: determining or obtaining a first pattern list based on matching information related to at least one reference region of the current block; determining or obtaining at least one candidate pattern based on the first pattern list; and determining or obtaining at least one prediction pattern of the current block based on matching information related to the at least one candidate pattern and the current block.

[0702] Optionally, a sub-block is part of the current block and inherits the position and some characteristics of the current block in the image. Therefore, the reference area of ​​the sub-block can also provide a reference for the prediction mode of the current block. Thus, based on the matching information related to at least one reference area of ​​the sub-block of the current block, the candidate mode in the first mode list can be determined or obtained.

[0703] Optionally, at least one reference region of a sub-block refers to a pixel region or pixel sample region located in the left, top, and upper left of the sub-block of the current block, which are adjacent or not adjacent to the reconstructed or encoded regions. The pixel values ​​or features of these regions can be used for prediction or other processing of the current block and / or its sub-blocks.

[0704] Optionally, pattern matching information includes: SAD, SATD, and / or MRSAD.

[0705] Optionally, candidate modes from the first mode list are determined or obtained from each prediction mode based on mode matching information related to at least one reference region of the sub-block of the current block.

[0706] Optionally, a reference region of a sub-block of the current block can be matched with at least one prediction pattern to determine or obtain at least one pattern matching information related to a reference region of the sub-block.

[0707] In this embodiment, the matching degree between the predicted pattern and at least one reference region of the current block's sub-block is quantified by pattern matching information. This allows for the rapid selection of patterns that closely match the current block and its sub-blocks as candidate patterns in the first pattern list. This improves the fit between the candidate patterns in the pattern list and the current block, and also enhances the fit between the predicted pattern determined or obtained based on the first pattern list and the current block. This, in turn, improves the prediction performance of the current block. Furthermore, the matching information filtering method can save computational resources, reduce computational complexity, and improve coding efficiency.

[0708] Method B4 is the prediction mode corresponding to at least one of the following encoded image blocks: the sub-blocks of the current block, adjacent blocks, non-adjacent blocks, co-located blocks, and preset positions.

[0709] Optionally, step S10 includes: determining or obtaining a first mode list corresponding to at least one of the encoded image blocks of the current block's sub-blocks, adjacent blocks, non-adjacent blocks, co-located blocks, and preset positions; determining or obtaining at least one candidate mode based on the first mode list; and determining or obtaining at least one prediction mode of the current block based on the matching information related to the at least one candidate mode and the current block.

[0710] Optionally, the current block, adjacent blocks, non-adjacent blocks, co-position blocks, and encoded image blocks at the predicted position are image regions of the same partitioning level. The sub-blocks of the current block, adjacent blocks, co-position blocks, and encoded image blocks at the predicted position are image regions of the same partitioning level. Furthermore, the sub-blocks of the current block, adjacent blocks, co-position blocks, and encoded image blocks at the predicted position are image regions of the next partitioning level of the current block, adjacent blocks, non-adjacent blocks, co-position blocks, and encoded image blocks at the predicted position.

[0711] Optionally, in video encoding and decoding technology, it is necessary to divide the image into blocks. For example, the image can be divided into at least one stripe, the stripe can be divided into at least one coding unit, and the coding unit can be divided into at least one sub-block. Different stripes belong to the same division level, different coding units belong to the same division level, and different sub-blocks belong to the same division level, while stripes and coding units belong to different division levels, and coding units and sub-blocks belong to different division levels.

[0712] Optionally, the current block includes at least one sub-block, which is part of the current block and inherits the position and some characteristics of the current block in the image. Therefore, it can provide a reference for the prediction mode of the current block, and then the candidate modes in the first mode list can be determined or obtained according to the prediction modes corresponding to the sub-blocks of the current block.

[0713] Optionally, the prediction patterns corresponding to the sub-blocks of the current block can also provide a reference for the prediction patterns of other sub-blocks of the current block. Therefore, based on the prediction patterns corresponding to one sub-block of the current block, the candidate patterns in the first pattern list of another sub-block of the current block can be determined.

[0714] Optionally, the neighboring blocks of the current block refer to the image blocks that are directly adjacent to the current block in space. The neighboring blocks can be blocks that have already been predicted or reconstructed. The neighboring blocks are highly correlated with the current block in space. Therefore, the prediction patterns and texture features of the neighboring blocks are often similar to those of the current block. By analyzing the prediction patterns of the neighboring blocks, a reference can be provided for the prediction pattern of the current block, and the prediction accuracy of the current block can be further improved.

[0715] Optionally, adjacent blocks include image blocks located above, to the left, and / or to the upper left of the current block.

[0716] Optionally, a co-location block can be an image block in a co-location image that has the same position and size as the current block, and the co-location image can be the image in the reference image that is closest to the current image in time. Since there is a strong temporal correlation between the co-location block and the current block, using the prediction mode of the co-location block as a candidate mode can better capture the motion characteristics in the video sequence and improve the prediction accuracy.

[0717] Optionally, non-adjacent blocks of the current block refer to image blocks that are not directly adjacent to the current block in space, but may be within a certain range or have some correlation. Non-adjacent blocks can also be blocks that have been predicted or reconstructed. Although non-adjacent blocks are not directly adjacent to the current block, they may still have some correlation with the current block. Therefore, they have certain reference value for the selection of target prediction mode for the current block and the prediction of the current block.

[0718] Optionally, the pre-coded image block at the preset position refers to the pre-coded block at certain positions according to specific rules or algorithms. These positions can be fixed (such as the upper left corner block) or dynamically determined according to the image content. Although non-adjacent blocks and pre-coded image blocks at preset positions may be spatially far from the current block, they may contain some unique texture or pattern information. This information can supplement the information of adjacent blocks and co-located blocks, further optimizing the selection of prediction modes.

[0719] Optionally, candidate modes in the first mode list are determined or obtained based on the adjacent prediction modes of the prediction modes corresponding to the sub-blocks, adjacent blocks, non-adjacent blocks, co-position blocks and / or encoded image blocks at preset positions of the current block. Optionally, the adjacent prediction modes of the prediction modes can be prediction modes that are adjacent in mode and / or adjacent in value.

[0720] Optionally, pattern adjacency refers to other patterns that are adjacent to a particular pattern (e.g., the first prediction pattern) in terms of direction or type in the definition or arrangement of prediction patterns. These patterns are usually logically or functionally associated with the first prediction pattern and may have similar prediction directions or texture features.

[0721] Optionally, numerical adjacency refers to other patterns that are numerically adjacent to a specific pattern in the numerical representation or index of the patterns. These patterns typically have consecutive index values ​​in the coding standard, indicating that they are similar in function or direction. Optionally, in the HEVC standard, angular patterns are represented by index values. If the index of the first possible pattern is n, the numerically adjacent patterns could be n-1 and n+1.

[0722] In this embodiment, determining or obtaining candidate modes from the first mode list based on the prediction modes corresponding to the sub-blocks, adjacent blocks, non-adjacent blocks, co-position blocks, and / or encoded image blocks at preset positions of the current block can improve the fit between the mode and the current block, thereby improving the prediction effect of the current block. And / or compared to screening candidate modes across the entire range, this embodiment utilizes existing information, namely the prediction modes corresponding to adjacent blocks, non-adjacent blocks, co-position blocks, and / or encoded image blocks at preset positions, effectively narrowing the search range of candidate modes and reducing computational complexity to a certain extent.

[0723] Method B5 is the prediction mode corresponding to at least one of the encoded image blocks of the current block's sub-blocks, including adjacent blocks, non-adjacent blocks, co-located blocks, and preset positions.

[0724] Optionally, step S10 includes: determining or obtaining a first mode list corresponding to at least one of the coded image blocks of the sub-blocks of the current block, including neighboring blocks, non-adjacent blocks, co-located blocks, and preset positions; determining or obtaining at least one candidate mode based on the first mode list; and determining or obtaining at least one prediction mode of the current block based on the matching information of the at least one candidate mode related to the current block.

[0725] Optionally, the encoded image blocks at preset positions of the sub-blocks of the current block include: the adjacent blocks of the sub-blocks of the current block, the non-adjacent blocks of the sub-blocks of the current block, the co-occurring blocks of the sub-blocks of the current block, and / or the encoded image blocks at preset positions of the sub-blocks of the current block.

[0726] Optionally, the prediction modes corresponding to the adjacent blocks, non-adjacent blocks, co-located blocks, and / or encoded image blocks at preset positions of the current block's sub-blocks are determined as candidate modes in the first mode list.

[0727] Optionally, the neighboring blocks of the current block's sub-blocks refer to image blocks that are spatially adjacent to the sub-blocks of the current block, and the neighboring blocks can be blocks that have already been predicted or reconstructed.

[0728] Optionally, the co-occurring block of the current block's sub-block can be an image block in the co-occurring image that has the same position and size as the current block's sub-block.

[0729] Optionally, a non-adjacent block of a sub-block of the current block refers to an image block that is not directly adjacent to the sub-block in space, but may be within a certain range or have some correlation. The non-adjacent block can also be a block that has been predicted or reconstructed.

[0730] Optionally, the pre-coded image block at the preset position refers to the pre-coded block at certain positions according to specific rules or algorithms. These positions can be fixed (such as the upper left corner block) or dynamically determined according to the image content.

[0731] Optionally, candidate modes in the first mode list are determined or obtained based on the adjacent prediction modes of the prediction modes corresponding to the adjacent blocks, non-adjacent blocks, co-position blocks and / or encoded image blocks at preset positions of the current block's sub-blocks. Optionally, the adjacent prediction modes of the prediction modes can satisfy mode-adjacency and / or numerical adjacency.

[0732] In this embodiment, determining or obtaining candidate modes from the first mode list based on the prediction modes corresponding to at least one of the encoded image blocks of the current block's sub-blocks, including adjacent blocks, non-adjacent blocks, co-located blocks, and preset positions, can improve the fit between the mode and the current block, thereby improving the prediction effect of the current block. And / or compared to screening candidate modes across the entire range, this embodiment utilizes existing information, namely the prediction modes corresponding to the sub-blocks of the encoded image blocks of adjacent blocks, non-adjacent blocks, co-located blocks, and / or preset positions, to effectively narrow the search range of candidate modes and reduce computational complexity to a certain extent.

[0733] Method B6 is the prediction mode corresponding to the sub-blocks of at least one of the following: neighboring blocks, non-adjacent blocks, co-located blocks, and pre-set positions of the current block.

[0734] Optionally, step S10 includes: determining or obtaining a first mode list based on the prediction modes corresponding to sub-blocks of encoded image blocks at least one of the adjacent blocks, non-adjacent blocks, co-located blocks, and preset positions of the current block; determining or obtaining at least one candidate mode based on the first mode list; and determining or obtaining at least one prediction mode of the current block based on the matching information related to the at least one candidate mode and the current block.

[0735] Optionally, the sub-blocks of the current block's adjacent blocks, non-adjacent blocks, co-located blocks, and / or encoded image blocks at preset positions include: sub-blocks of the current block's adjacent blocks, non-adjacent blocks, co-located blocks, and / or encoded image blocks at preset positions.

[0736] Optionally, the current block's neighboring blocks, non-neighboring blocks, co-located blocks, and / or encoded image blocks at preset positions may also include at least one sub-block. The sub-block is part of the neighboring blocks, non-neighboring blocks, co-located blocks, and / or encoded image blocks at preset positions. It inherits the position and some characteristics of the neighboring blocks, non-neighboring blocks, co-located blocks, and / or encoded image blocks at preset positions in the image. Therefore, it can also provide a reference for the prediction mode of the current block.

[0737] Optionally, at least some of the prediction modes corresponding to the adjacent blocks, non-adjacent blocks, co-located blocks, and / or sub-blocks of the encoded image blocks at preset positions of the current block are determined as candidate modes in the first mode list.

[0738] Optionally, the prediction modes corresponding to the adjacent blocks, non-adjacent blocks, co-located blocks, and / or sub-blocks of the encoded image block at a preset position of the current block include at least one of the following: angle prediction mode, non-angle prediction mode, and derivation mode.

[0739] In this embodiment, determining or obtaining candidate modes in the first mode list based on the prediction modes corresponding to the sub-blocks of the coded image blocks at preset positions and the adjacent blocks, non-adjacent blocks, co-position blocks, and / or preset positions of the current block can improve the fit between the mode and the current block, thereby improving the prediction effect of the current block. And / or, compared to screening candidate modes across the entire range, this embodiment utilizes existing information, namely the prediction modes corresponding to the sub-blocks of the coded image blocks at preset positions and the adjacent blocks, non-adjacent blocks, co-position blocks, and / or preset positions, to effectively narrow the search range of candidate modes and reduce computational complexity to a certain extent.

[0740] Fifth embodiment

[0741] Based on any of the above embodiments, a fifth embodiment is proposed.

[0742] In this embodiment, the method of determining or obtaining at least one candidate mode based on at least one gradient operator includes at least one of the following methods C1 to C4:

[0743] Method C1: Determine or obtain at least one candidate mode based on at least one gradient information determined or obtained by at least one gradient operator.

[0744] Optionally, the gradient information includes gradient direction and / or gradient magnitude. The gradient direction represents the direction in which the gray value changes the fastest at a pixel in the image (e.g., at least one reference region), i.e., the direction perpendicular to the edge. The gradient magnitude represents the intensity of the gray value change at that pixel, i.e., the salience of the edge.

[0745] Optionally, step S10 includes: determining or obtaining at least one gradient information based on at least one gradient operator, determining or obtaining at least one gradient histogram and / or statistical results, determining or obtaining at least one candidate pattern based on at least one gradient histogram and / or statistical results, and determining or obtaining at least one prediction pattern for the current block based on matching information related to the at least one candidate pattern and the current block.

[0746] Optionally, at least one gradient information is determined or obtained based on at least one reference region and at least one gradient operator of the current block, and at least one candidate mode is determined or obtained based on the at least one gradient information.

[0747] Optionally, the predicted direction corresponding to the gradient direction is determined or obtained based on the gradient direction in the gradient information of at least one reference region of the current block and the angle difference between each predicted direction and the predicted direction in at least one lookup table (e.g., a first lookup table and / or a second lookup table). For example, the direction with the smallest absolute angle difference is determined as the predicted direction corresponding to the gradient direction, and the magnitude value of the gradient magnitude in the gradient information of at least one reference region is assigned to the predicted direction.

[0748] Optionally, the gradient information of at least one reference region includes the gradient information of pixels in at least one reference region, and the gradient information of pixels in at least one reference region includes the quantized gradient direction.

[0749] Optionally, the initial gradient direction of a pixel in at least one reference region is determined or obtained, and the quantized gradient direction of the pixel in at least one reference region is determined or obtained based on the angle difference between the determined or obtained initial gradient direction and the gradient direction adjacent to the initial gradient direction in at least one lookup table.

[0750] Optionally, the initial gradient direction is compared with the gradient directions adjacent to the left and right of the initial gradient direction in at least one lookup table. If the angle difference between the initial gradient direction and the left adjacent gradient direction is less than the angle difference between the initial gradient direction and the right adjacent gradient direction, then the quantized gradient direction of the pixel is the left adjacent gradient direction; and / or, if the angle difference between the initial gradient direction and the left adjacent gradient direction is greater than the angle difference between the initial gradient direction and the right adjacent gradient direction, then the quantized gradient direction of the pixel is the right adjacent gradient direction.

[0751] Optionally, the initial gradient direction of the pixel is determined or obtained, the initial gradient direction information related to the initial gradient direction of the pixel is determined or obtained, and the quantized gradient direction information of the pixel is determined or obtained based on the difference between the determined or obtained initial gradient direction information and the gradient direction information adjacent to the initial gradient direction information in at least one lookup table.

[0752] Optionally, the initial gradient direction information is compared with the gradient direction information that is adjacent to the initial gradient direction information on the left and right sides in at least one lookup table. If the difference between the initial gradient direction information and the gradient direction information adjacent to the left side is less than the difference between the initial gradient direction information and the gradient direction information adjacent to the right side, then the quantized gradient direction information of the pixel is the gradient direction information adjacent to the left side; and / or, if the difference between the initial gradient direction information and the gradient direction information adjacent to the left side is greater than the difference between the initial gradient direction information and the gradient direction information adjacent to the right side, then the quantized gradient direction information of the pixel is the gradient direction information adjacent to the right side.

[0753] Optionally, the gradient direction information is the tangent of the gradient direction or a scaled tangent.

[0754] Optionally, if the gradient direction X in the gradient information is located between the first direction (i.e., the first gradient direction) and the second direction (i.e., the second gradient direction) in the first lookup table or the second lookup table, then according to the angle difference or difference between the gradient direction information x and the first gradient direction information and the second gradient direction information, the gradient direction X is decomposed into gradient direction A and gradient direction B, and at least one candidate mode is determined or obtained based on gradient direction A and gradient direction B.

[0755] Optionally, based on the angle difference or difference between the gradient direction information x corresponding to the gradient direction X and the first gradient direction information, a first weight for gradient direction A is determined or obtained; based on the angle difference or difference between the gradient direction information x corresponding to the gradient direction X and the second gradient direction information, a second weight for gradient direction B is determined or obtained; based on the first weight and the gradient magnitude M corresponding to the gradient direction X, the gradient magnitude MA of gradient direction A is determined or obtained; based on the second weight and the gradient magnitude M corresponding to the gradient direction X, the gradient magnitude MB of gradient direction B is determined or obtained. For example, the gradient magnitude MA is equal to the first weight * the gradient magnitude M, and the gradient magnitude MB is equal to the second weight * the gradient magnitude M.

[0756] Optionally, gradient direction A is the first gradient direction mentioned above, and gradient direction B is the second gradient direction mentioned above. Optionally, the first weight is equal to the difference between gradient direction information x and second gradient direction information, or the difference between the first gradient direction information and second gradient direction information. The second weight is equal to the difference between gradient direction information x and first gradient direction information, or the difference between the first gradient direction information and second gradient direction information.

[0757] Optionally, if the gradient direction in the gradient information is located between the first direction and the second direction in at least one lookup table, then the weight information corresponding to the first direction and the second direction is determined or obtained according to the angle difference between the gradient direction and the first direction and the second direction, and at least one candidate mode is determined or obtained according to the weight information corresponding to the first direction and the second direction. Optionally, at least one gradient histogram and / or statistical results are determined or obtained according to the weight information corresponding to the first direction and the second direction, and at least one candidate mode is determined or obtained according to the at least one gradient histogram and / or statistical results.

[0758] Optionally, if the gradient direction in the gradient information is located between the first direction and the second direction in the first lookup table or the second lookup table, then the weight information corresponding to the first direction and the second direction is determined or obtained according to the angle difference between the gradient direction and the first direction and the second direction. At least one candidate mode is determined or obtained according to the gradient magnitude in the gradient information and the weight information corresponding to the first direction and the second direction. Optionally, at least one gradient histogram and / or statistical result is determined or obtained according to the gradient magnitude in the gradient information and the weight information corresponding to the first direction and the second direction. At least one candidate mode is determined or obtained according to the at least one gradient histogram and / or statistical result.

[0759] In this embodiment, when the gradient direction is between two prediction directions, the gradient magnitude is allocated according to the angle difference between the gradient direction and the prediction direction, thereby avoiding errors caused by rigid classification and improving the accuracy of angle prediction.

[0760] Method C2: Based on the overall gradient information corresponding to at least one reference region determined or obtained by at least one gradient operator, at least one candidate mode is determined or obtained.

[0761] Optionally, step S10 includes: determining or obtaining at least one gradient histogram and / or statistical results based on the overall gradient information corresponding to at least one reference region determined or obtained by at least one gradient operator; determining or obtaining at least one candidate mode based on the at least one gradient histogram and / or statistical results; and determining or obtaining at least one prediction mode for the current block based on the matching information related to the at least one candidate mode and the current block.

[0762] Optionally, the gradient information includes gradient information of at least one pixel in the reference region and / or the overall gradient information of the reference region.

[0763] Optionally, at least one candidate mode is determined or obtained based on the overall gradient information and / or pixel gradient information of at least one reference region of the current block. For example, at least one gradient histogram and / or statistical results are determined or obtained based on the overall gradient information and / or pixel gradient information of at least one reference region of the current block, and at least one candidate mode is determined or obtained based on the at least one gradient histogram and / or statistical results.

[0764] Optionally, the gradient information of a pixel is the gradient direction and / or gradient magnitude calculated independently for each pixel in the reference region, reflecting the local variation characteristics of that point.

[0765] Optionally, the overall gradient information of the reference region is obtained by statistically analyzing and / or aggregating the gradient information of all pixels within the reference region, reflecting the global change trend of the region.

[0766] Optionally, the gradient information of a pixel includes: a first pixel gradient component, a second pixel gradient component, a pixel gradient direction, and / or a pixel gradient magnitude.

[0767] Optionally, gradient information of at least one pixel in the reference region is determined or obtained based on mutually perpendicular gradient operators. A pair of mutually perpendicular gradient operators includes: a first gradient operator (e.g., a horizontal gradient operator Gx) and a second gradient operator (e.g., a vertical gradient operator Gy) corresponding to different directions. The value determined or obtained based on the first gradient operator is the gradient component of the first pixel (e.g., the value of Gx), and the value determined or obtained based on the second gradient operator is the gradient component of the second pixel (e.g., the value of Gy).

[0768] Optionally, the pixel gradient direction and / or pixel gradient magnitude can be determined or obtained based on the first pixel gradient component and the second pixel gradient component.

[0769] Optionally, the gradient information includes: a first gradient component (e.g., a first pixel gradient component and / or a first overall gradient component), a second gradient component (e.g., a second pixel gradient component and / or a second overall gradient component), a gradient direction (e.g., a pixel gradient direction and / or an overall gradient direction) and / or a gradient magnitude (e.g., a pixel gradient magnitude and / or an overall gradient magnitude).

[0770] Optionally, the overall gradient information includes: a first overall gradient component, a second overall gradient component, an overall gradient direction and / or an overall gradient magnitude.

[0771] Optionally, the manner of determining or obtaining the overall gradient information of the reference region includes: determining or obtaining gradient information of at least one pixel in the reference region, where the gradient information of the at least one pixel includes a first pixel gradient component and a second pixel gradient component, determining or obtaining the first overall gradient component according to the sum of the first pixel gradient components corresponding to the reference region, determining or obtaining the second overall gradient component according to the sum of the second pixel gradient components corresponding to the reference region, determining or obtaining the overall gradient magnitude according to the sum of the absolute values of the first overall gradient component and the second overall gradient component, and determining or obtaining the overall gradient direction according to the first overall gradient component and the second overall gradient component.

[0772] Optionally, edge directions usually tend to be consistent in a local region, so direct summation in the above manner can preserve directional statistical characteristics, and the synthesized result can represent the overall gradient vector.

[0773] Optionally, determining or obtaining the gradient information of at least one pixel in the reference region includes: the first pixel gradient components (Gx1, Gx2, ...Gx n ) and the second pixel gradient components (Gy1, Gy2, ...Gy n ), determining or obtaining the first overall gradient component according to the sum of the first pixel gradient components corresponding to the reference region, that is, the first overall gradient component Gx sum =Gx1+Gx2+...+Gx n , determining or obtaining the second overall gradient component according to the sum of the second pixel gradient components corresponding to the reference region, that is, the second overall gradient component Gy sum =Gy1+Gy2+...+Gy n , determining or obtaining the overall gradient magnitude AMP according to the sum of the absolute values of the first overall gradient component and the second overall gradient component sum , that is, the overall gradient magnitude AMP sum =丨Gx sum 丨+丨Gy sum 丨, the overall gradient direction is obtained by arctan(Gx sum / Gy sum ) or arctan(Gysum / Gx sum ) is calculated.

[0774] Optionally, the manner of determining or obtaining the overall gradient information of the reference region includes: determining or obtaining the gradient information of at least one pixel in the reference region, including a first pixel gradient component, a second pixel gradient component, a pixel gradient direction, and a pixel gradient magnitude; determining or obtaining the overall gradient magnitude according to the sum of the absolute values of the first pixel gradient component and the second pixel gradient component corresponding to the reference region, and / or the sum of the absolute values of the first pixel gradient component and the second pixel gradient component corresponding to the reference region as well as the number of non-zero pixel gradient magnitudes corresponding to the pixel gradient directions of the reference region; determining or obtaining a first overall gradient component according to the sum of the first pixel gradient components corresponding to the reference region; determining or obtaining a second overall gradient component according to the sum of the second pixel gradient components corresponding to the reference region; and determining or obtaining the overall gradient direction according to the first overall gradient component and the second overall gradient component.

[0775] Optionally, determining or obtaining the gradient information of at least one pixel in the reference region includes: a first pixel gradient component (Gx1, Gx2, ...Gxn) and a second pixel gradient component (Gy1, Gy2, ...Gyn), a pixel gradient direction, and a pixel gradient magnitude; and determining or obtaining the overall gradient magnitude, that is, the overall gradient magnitude AMP, according to the sum of the absolute values of the first pixel gradient component and the second pixel gradient component corresponding to the reference region, as well as the number of non-zero pixel gradient magnitudes corresponding to the pixel gradient directions of the reference region. sum =|Gx1|+|Gx2|+...+|Gx n |+|Gy1|+|Gy2|+...+|Gy n |, or, the overall gradient magnitude AMP sum =(|Gx1|+|Gx2|+...+|Gx n |+|Gy1|+|Gy2|+...+|Gy n |) / n, where n is the number of non-zero pixel gradient magnitudes corresponding to the pixel gradient directions of the reference region, for example, if there are 10 directions in the pixel gradient directions of the reference region whose corresponding pixel gradient magnitudes are not zero, then n is 10; the first overall gradient component, that is, the first overall gradient component Gx, is determined or obtained according to the sum of the first pixel gradient components corresponding to the reference region. sum =Gx1+Gx2+...+Gx n , the second overall gradient component, that is, the second overall gradient component Gy, is determined or obtained according to the sum of the second pixel gradient components corresponding to the reference region. sum =Gy1+Gy2+...+Gy n , the overall gradient direction is obtained via arctan(Gx sum / Gy sum ) or arctan(Gy sum / Gx sum ) Calculated.

[0776] Optionally, the average absolute value can be obtained through the above method, which can resist directional interference and stably measure the gradient strength.

[0777] Optionally, when the overall gradient information of the reference region is determined or obtained based on the sum of the absolute values ​​of the first and second pixel gradient components corresponding to the reference region, if this overall gradient information is directly fused with the gradient information of the corresponding pixel in the reference region, the overall gradient magnitude corresponding to the overall gradient information in the determined or obtained gradient histogram and / or statistical results will be the maximum magnitude in the histogram and / or statistical results, thus interfering with the selection of the prediction mode. To avoid the influence of the overall gradient magnitude information, the gradient histogram and / or statistical results can be determined or obtained based on the overall gradient information, the weight information corresponding to the overall gradient magnitude, and the gradient information of at least one pixel in the reference region, thereby ensuring fair competition of gradient information, balancing the consistency requirements of intensity and direction, and / or... The prediction mode corresponding to the overall gradient direction in the overall gradient information is determined as mode 1. Based on the gradient information of at least one pixel in the reference region, a gradient histogram and / or statistical results are determined or obtained. Based on the gradient histogram and / or statistical results, at least one mode 2 is determined or obtained. Mode 1 and at least one mode 2 are determined as candidate modes. If at least one prediction mode of the current block includes mode 1 and at least one mode 2, mode 1 and mode 2 are determined as at least one prediction mode of the current block. Based on mode 1, at least one prediction result 1 of the current block is determined or obtained. Based on at least one mode 2, at least one prediction result 2 of the current block is determined or obtained. Based on the weight information corresponding to at least one prediction result 1, at least one prediction result 2, and at least one prediction result 1, the target prediction result of the current block is determined or obtained.

[0778] Optionally, the overall gradient information includes the overall gradient direction and the overall gradient magnitude. Based on the gradient information of at least one pixel corresponding to the reference region, the gradient histogram is determined or obtained. Figure 1 And / or statistical result 1, the gradient histogram Figure 1 Replace the gradient magnitude corresponding to the direction that is the same as the overall gradient direction in Statistical Result 1 with the overall gradient magnitude to determine or obtain the gradient histogram. Figure 2 And / or statistical result 2, based on gradient histogram Figure 2 And / or statistical result 2, to identify or obtain at least one candidate pattern.

[0779] Optionally, the target prediction result of the current block is determined or obtained based on at least one candidate mode and the weight information corresponding to the candidate mode. The candidate modes include: the prediction mode corresponding to the overall gradient information, the prediction mode determined or obtained based on the gradient histogram, and / or the non-angle prediction mode.

[0780] Optionally, if the prediction mode determined or obtained based on the gradient histogram and / or statistical results includes the prediction mode corresponding to the overall gradient information, the weight information corresponding to at least one candidate mode is determined or obtained based on the weight information corresponding to the prediction mode corresponding to the overall gradient information. The candidate modes include: the prediction mode corresponding to the overall gradient information, the prediction mode determined or obtained based on the gradient histogram, and / or the non-angle prediction mode.

[0781] Optionally, the predicted modes determined or obtained based on the gradient histogram determined or obtained from the gradient information of at least one pixel in the reference region include: mode a1, mode a2, and mode a3. The weights of each of modes a1, a2, and a3 can be determined based on the sum of their respective gradient magnitudes in the gradient histogram. For example, based on the sum of their respective gradient magnitudes in the gradient histogram, the weights of modes a1, a2, and a3 are 0.4, respectively. Mode a1 is the predicted mode corresponding to the overall gradient information. Based on the weight of mode a1 (0.4), the weights of modes a2 and a3 are adjusted. For example, since the predicted mode corresponding to the overall gradient information is used as the candidate mode by default, the weight value of mode a1 can be modified from the weight obtained by the sum of the gradient magnitudes of each mode in the gradient histogram to the fixed weight corresponding to the overall gradient information. The value of the fixed weight is a preset value. Based on the fixed weight and the sum of the gradient magnitudes of modes a2 and a3 in the gradient histogram, the weights of modes a2 and a3 are determined. For example, if the fixed weight is 0.6, and the sum of the gradient magnitudes of modes a2 and a3 is 50, then the weight of mode a2 is 0.2, and the weight of mode a3 is 0.2.

[0782] Optionally, if a candidate mode is determined as the prediction mode for the current block, the weight information corresponding to the candidate mode when it is used as the prediction mode can be determined or obtained based on the weight information corresponding to the candidate mode.

[0783] Optionally, if a candidate mode is determined as the prediction mode for the current block, the weight information corresponding to the candidate mode is determined as the weight information corresponding to the prediction mode.

[0784] Optionally, the weight information corresponding to the candidate mode may be the same as or different from the weight information corresponding to the candidate mode when it is determined as the prediction mode.

[0785] Optionally, the candidate modes include: a prediction mode determined or obtained based on the gradient histogram and a planar mode. The prediction modes determined or obtained based on the gradient histogram include the prediction modes corresponding to the overall gradient information. In this embodiment, the prediction modes determined or obtained based on the gradient information of at least one pixel in the reference area include: mode a1, mode a2 and mode a3. Mode a1 is the prediction mode corresponding to the overall gradient information. A fixed weight is used for the prediction mode corresponding to the overall gradient information. For example, the fixed weight is equal to 0.2. The planar mode uses a fixed weight of 0.3. The weights of mode a2 and mode a3 are determined by the sum of the gradient magnitudes corresponding to mode a2 and mode a3 in the gradient histogram. If the sum of the gradient magnitudes of mode a2 is 20 and the sum of the gradient magnitudes of mode a3 is 30, the weight of mode a2 is 0.2 and the weight of mode a3 is 0.3. The weight of mode a2 can be determined according to formula (v).

[0786] W a2 =(1-W) a1 -W planar )*(AMP a2 / (AMP a2 +AMP a3 Formula (5);

[0787] Wa2 is the weight of pattern a2, W a1 For the weights of pattern a1, AMP a2 AMP is the sum of the gradient magnitudes of mode a2. a3 This is the sum of the gradient magnitude values ​​of mode a3.

[0788] The weight of pattern a3 can be determined according to formula (vi);

[0789] W a3 =(1-W) a1 -W planar )*(AMP a3 / (AMP a2 +AMP a3 Formula (VI);

[0790] Wa2 is the weight of pattern a2, W a1 For the weights of pattern a1, AMP a2 AMP is the sum of the gradient magnitudes of mode a2. a3 This is the sum of the gradient magnitude values ​​of mode a3.

[0791] Optionally, the candidate modes include: a predicted mode determined or obtained based on the gradient histogram and a planar mode. The predicted modes determined or obtained based on the gradient histogram do not include the predicted modes corresponding to the overall gradient information. In this embodiment, a fixed weight is used for the predicted modes corresponding to the overall gradient information, for example, a fixed weight of 0.2. A fixed weight of 0.3 is used for the planar mode. Based on this, the weights of modes a2 and a3 are determined by the sum of the gradient magnitudes corresponding to modes a2 and a3 in the gradient histogram. If the sum of the gradient magnitudes of mode a2 is 20 and the sum of the gradient magnitudes of mode a3 is 30, the weight of mode a2 is 0.2 and the weight of mode a3 is 0.3. Optionally, the weights corresponding to modes a2 and a3 can be determined or obtained by the above formulas (v) and (vi).

[0792] In this embodiment, by combining the gradient information of pixels and / or the overall gradient information, the determined or obtained candidate mode can take into account both local details and global structure. As a result, at least one prediction mode determined or obtained based on the candidate mode can match the content of the current block, thereby improving the prediction accuracy.

[0793] Method C3 determines or obtains at least one candidate mode based on the gradient operator corresponding to at least one lookup table.

[0794] Optionally, step S10 includes: determining or obtaining at least one candidate mode based on the gradient operator corresponding to at least one lookup table; determining or obtaining at least one prediction mode of the current block based on the matching information of the at least one candidate mode and the current block; and determining or obtaining at least one prediction mode of the current block based on the matching information of the at least one candidate mode and the current block.

[0795] Optionally, at least one lookup table includes: lookup table A and / or lookup table B, where the number of prediction patterns corresponding to lookup table A is a first number, and the number of prediction patterns corresponding to lookup table B is a second number, and the first number and the second number may be the same or different.

[0796] Optionally, the first quantity is 35, 67 or 131, for example, the first quantity is 131.

[0797] Optionally, the first quantity is greater than 35, 67, or 131.

[0798] Alternatively, the second quantity can be 35, 67, or 131, for example, the second quantity can be 67.

[0799] Optionally, the second quantity is less than or equal to 35, 67, or 131.

[0800] Optionally, the first quantity and the second quantity are the same, that is, the number of prediction patterns corresponding to lookup table A is the same as the number of prediction patterns corresponding to lookup table B. For example, the number of prediction patterns corresponding to lookup table A and lookup table B is the same, but the types of prediction patterns corresponding to each other are at least partially different.

[0801] Optionally, the first quantity and the second quantity are different; for example, the first quantity is 131 and the second quantity is 67.

[0802] Optionally, lookup table A corresponds to gradient operators A1 and A2, and lookup table B corresponds to gradient operators B1 and B2. Gradient operators A1 to A2 and gradient operators B1 to B2 are all different.

[0803] Optionally, the prediction patterns corresponding to lookup table A and lookup table B are different, or the prediction patterns corresponding to lookup table A and lookup table B are at least partially the same.

[0804] Optionally, lookup table A can be a lookup table corresponding to 37 prediction modes or a lookup table corresponding to 131 prediction modes.

[0805] Optionally, lookup table B can be a lookup table corresponding to 37 prediction modes or a lookup table corresponding to 131 prediction modes.

[0806] Optionally, gradient information of at least one reference region of the current block is determined or obtained based on at least one gradient operator corresponding to at least one reference region of the current block and at least one lookup table, and at least one gradient histogram is determined or obtained based on the gradient information of at least one reference region of the current block.

[0807] Optionally, if the first quantity is greater than the second quantity, the prediction direction corresponding to the prediction mode corresponding to lookup table A at least partially covers the prediction direction corresponding to the prediction mode corresponding to lookup table B. Since the gradient operator needs to match different directions to accurately calculate the gradient information, if the first quantity and the second quantity are different, at least one candidate mode can be determined or obtained by at least one gradient operator corresponding to lookup table A and / or at least one gradient operator corresponding to lookup table B.

[0808] Optionally, the first number is 131. Based on at least one gradient operator corresponding to lookup table A, the gradient information A of at least one reference region of the current block is determined or obtained. Based on the gradient information A of the at least one reference region, at least one gradient histogram A and / or statistical results are determined or obtained. The gradient histogram A and / or statistical results include the gradient magnitude change of the at least one reference region in the prediction direction corresponding to the 131 prediction modes.

[0809] Optionally, the second number is 67. Based on at least one gradient operator corresponding to lookup table B, the gradient information B of at least one reference region of the current block is determined or obtained. Based on the gradient information B of the at least one reference region, at least one gradient histogram B and / or statistical results are determined or obtained. The gradient histogram B and / or statistical results include the gradient magnitude change of the at least one reference region in the prediction direction corresponding to the 67 prediction modes.

[0810] Optionally, at least one candidate pattern is determined or obtained based on at least one gradient histogram A and / or at least one gradient histogram B.

[0811] Optionally, at least one gradient operator includes at least one pair of mutually perpendicular gradient operators.

[0812] Optionally, mutually perpendicular gradient operators include: a first pair of mutually perpendicular gradient operators and / or a second pair of mutually perpendicular gradient operators, wherein the first pair of gradient operators and the second pair of gradient operators are different.

[0813] Optionally, a first gradient information is determined or obtained based on a first pair of mutually perpendicular gradient operators, and at least one gradient histogram is determined or obtained based on the first gradient information.

[0814] Optionally, a second gradient information is determined or obtained based on a second pair of mutually perpendicular gradient operators, and at least one gradient histogram is determined or obtained based on the second gradient information.

[0815] Optionally, the first pair of mutually perpendicular gradient operators includes a 0° gradient operator and a 90° gradient operator. The horizontal gr...

Claims

1. A processing method, characterized in that, Including the following steps: S10, based on the gradient magnitude values ​​of each pixel in at least one first reference region of the current block in the directions corresponding to multiple intra-prediction modes, statistically analyze the distribution of gradient magnitude values, and determine or obtain at least one gradient histogram corresponding to the current block, wherein the gradient histogram includes the sum of gradient magnitude values ​​corresponding to multiple intra-prediction modes respectively. Based on the sorting result of the sum of gradient magnitude values ​​corresponding to multiple intra-prediction modes in the gradient histogram, a first preset number of candidate modes is determined or obtained from multiple intra-prediction modes, wherein the first preset number is multiple. Based on the sorting results of matching information related to the second reference region of the current block through candidate patterns, at least one prediction pattern of the current block is determined or obtained from a first preset number of candidate patterns, wherein the matching information is the matching information between the predicted pixel value of the second reference region determined or obtained based on the candidate patterns and the reconstructed pixel value of the second reference region, and the first reference region and the second reference region are different.

2. The processing method as described in claim 1, characterized in that, Candidate patterns are also determined or obtained based on at least one of the following: The current block corresponds to at least one area histogram; A histogram of at least one usage count of the predicted mode corresponding to the encoded image block corresponding to the current block; At least one gradient operator; First pattern list.

3. The processing method as described in claim 2, characterized in that, The first pattern list is determined or obtained based on at least one of the following: At least one reference region and neural network model for the current block; Matching information related to at least one reference region of the current block; Matching information related to at least one reference region of the current block's sub-blocks.

4. The processing method as described in claim 2, characterized in that, The first pattern list is determined or obtained based on at least one of the following: The prediction mode corresponding to at least one of the following: the sub-blocks, adjacent blocks, non-adjacent blocks, co-located blocks, and encoded image blocks at preset positions; The prediction mode corresponding to at least one of the following: the neighboring blocks, non-neighboring blocks, co-located blocks, and the encoded image blocks at preset positions of the current block's sub-blocks; The prediction mode corresponding to the sub-blocks of at least one of the following: neighboring blocks, non-adjacent blocks, co-located blocks, and pre-defined positions of the current block.

5. The processing method as described in claim 3, characterized in that, The method of determining or obtaining the first pattern list based on at least one reference region and neural network model of the current block includes at least one of the following: Based on the pixels of at least one reference region of the current block and the neural network model, determine or obtain a first pattern list; Based on the gradient information of at least one reference region of the current block and the neural network model, a first pattern list is determined or obtained.

6. The processing method as described in claim 2, characterized in that, The gradient histogram is also determined or obtained based on at least one of the following: At least one gradient operator corresponding to at least one lookup table; At least one pair of gradient operators corresponding to weight information; The weight information corresponding to the gradient information of at least one reference region of the current block; The current block has at least one reference region and at least one lookup table.

7. The processing method as described in claim 2, characterized in that, An area histogram is also determined or obtained based on at least one of the following: Area information of at least one reference region for the current block; The current block has at least one reference region and at least one lookup table.

8. The processing method as described in claim 6, characterized in that, At least one gradient operator includes at least one pair of mutually perpendicular gradient operators.

9. The processing method as described in claim 6, characterized in that, Gradient information includes at least one of the following: Gradient information of pixels in the reference region; Overall gradient information for the reference region; Gradient information, gradient information of pixels in the reference region, and / or overall gradient information of the reference region, including gradient magnitude and / or gradient direction.

10. The processing method as described in claim 6, characterized in that, The gradient information includes at least one of the following: first gradient information, second gradient information, and third gradient information.

11. The processing method as described in claim 10, characterized in that, It also includes at least one of the following: The first gradient information is determined or obtained based on the first pair of mutually perpendicular gradient operators; The second gradient information is obtained by determining or negating the second pair of mutually perpendicular gradient operators; The third gradient information is determined or obtained based on the directions of the first and second gradients.

12. The processing method according to any one of claims 3 to 7, characterized in that, At least one reference region of the current block is determined or obtained based on at least one of the following: At least one of the following: the pixel above the current block, the non-adjacent pixel above the current block, the pixel to the left of the current block, the non-adjacent pixel to the left of the current block, the pixel above the left of the current block, and the non-adjacent pixel above the left of the current block; The current block is selected from at least one of the following: neighboring block, non-neighboring block, sibling block, temporal block, and default block; The current block's width, height, block size, and block area must be at least one of these. The candidate motion vector or candidate block vector of the current block is determined or the candidate block is obtained.

13. A processing apparatus, characterized in that, include: A memory and a processor, wherein the memory stores a processing program, and the processing program, when executed by the processor, implements the steps of the processing method as described in any one of claims 1 to 12.

14. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the processing method as described in any one of claims 1 to 12.

Images