Processing method, processing equipment and storage medium

Abstract

The invention provides a processing method, processing equipment and a storage medium, the processing method can be applied to the processing equipment, and the processing method comprises the following steps: determining or obtaining at least one prediction mode of a current block according to matching information related to at least one candidate mode and the current block. According to the technical scheme, a proper prediction mode can be matched for the current block, so that the prediction accuracy of the current block can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a processing method, a processing device, and a storage medium. Background Art

[0002] The existing video coding standard (H.266 / VVC) proposes a video frame encoding technology. For example, when encoding and decoding a video frame, the protocol divides each frame into different blocks and performs prediction processing and encoding and decoding processing.

[0003] During the process of conceiving and implementing this application, the inventors discovered that there are at least the following problems: in the intra-frame prediction and / or inter-frame prediction process, it is difficult to match a suitable prediction mode for the image block, resulting in unsatisfactory prediction effect for the image block.

[0004] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the Invention

[0005] In response to the above technical problems, the present application provides a processing method, a processing device and a storage medium, which can match a suitable prediction mode for the current block from at least one candidate mode, thereby supporting the improvement of the prediction accuracy of the current block.

[0006] The present application provides a processing method that can be applied to a processing device, comprising the steps of:

[0007] S10 , determining or obtaining at least one prediction mode of the current block according to matching information related to at least one candidate mode and the current block.

[0008] Optionally, the candidate mode is determined or obtained according to at least one of the following:

[0009] At least one gradient histogram corresponding to the current block;

[0010] at least one area histogram corresponding to the current block;

[0011] at least one histogram of usage counts of a prediction mode corresponding to an encoded image block corresponding to the current block;

[0012] at least one gradient operator;

[0013] First mode list.

[0014] Optionally, the first pattern list is determined or obtained according to at least one of the following:

[0015] at least one reference region and a neural network model of the current block;

[0016] matching information associated with at least one reference region of the current block;

[0017] matching information associated with at least one reference region of a sub-block of the current block;

[0018] a prediction mode corresponding to at least one of a subblock of the current block, an adjacent block, a non-adjacent block, a co-located block, and an encoded image block at a preset position;

[0019] a prediction mode corresponding to at least one of an adjacent block, a non-adjacent block, a co-located block, and an encoded image block at a preset position of a subblock of the current block;

[0020] The prediction mode corresponding to the sub-block of the encoded image block of at least one of an adjacent block, a non-adjacent block, a co-located block and a preset position of the current block.

[0021] Optionally, the manner of determining or obtaining the first pattern list according to at least one reference area of ​​the current block and the neural network model includes at least one of the following:

[0022] Determine or obtain a first pattern list based on pixels of at least one reference area of ​​the current block and the neural network model;

[0023] A first pattern list is determined or obtained based on gradient information of at least one reference region of the current block and a neural network model.

[0024] Optionally, the gradient histogram and / or the area histogram is determined or obtained according to at least one of the following:

[0025] Gradient information of at least one reference region of the current block;

[0026] area information of at least one reference region of the current block;

[0027] at least one gradient operator corresponding to at least one lookup table;

[0028] Weight information corresponding to at least one pair of gradient operators;

[0029] Weight information corresponding to gradient information of at least one reference region of the current block;

[0030] At least one reference region and at least one lookup table of the current block.

[0031] Optionally, the at least one gradient operator includes at least one pair of mutually perpendicular gradient operators.

[0032] Optionally, the gradient information includes at least one of the following:

[0033] Gradient information of pixels in the reference area;

[0034] The overall gradient information of the reference area;

[0035] The gradient information, the gradient information of the pixels in the reference region and / or the overall gradient information of the reference region include a gradient magnitude and / or a gradient direction.

[0036] Optionally, the processing method further includes at least one of the following:

[0037] The first gradient information is determined or obtained according to a first pair of mutually perpendicular gradient operators;

[0038] The second gradient information is determined or obtained according to a second pair of mutually perpendicular gradient operators;

[0039] The third gradient information is determined or obtained according to the first gradient information and the second gradient information.

[0040] Optionally, at least one reference area of ​​the current block is determined or obtained according to at least one of the following:

[0041] At least one of an upper adjacent pixel, an upper non-adjacent pixel, a left adjacent pixel, a left non-adjacent pixel, an upper left adjacent pixel, and an upper left non-adjacent pixel of the current block;

[0042] At least one of the neighboring block, non-neighboring block, co-located block, time domain block, and default block corresponding to the current block;

[0043] At least one of the width, height, block size, and block area of ​​the current block;

[0044] The candidate block is determined or obtained by the candidate motion vector or candidate block vector of the current block.

[0045] The present application also provides a processing device, including: a memory and a processor, wherein a processing program is stored in the memory, and when the processing program is executed by the processor, the steps of any of the above processing methods are implemented.

[0046] The present application also provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the processing methods described above are implemented.

[0047] As described above, the processing method of the present application can be applied to a processing device, including: determining or obtaining at least one prediction mode for the current block based on matching information related to at least one candidate mode and the current block. Through the technical solution of the present application, it is possible to match a suitable prediction mode for the current block from the at least one candidate mode, thereby supporting improved prediction accuracy for the current block. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for describing the embodiments. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without inventive work.

[0049] Figure 1 A schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present application;

[0050] Figure 2 A communication network system architecture diagram provided in an embodiment of the present application;

[0051] Figure 3 A schematic diagram of the hardware structure of a controller 140 provided in this application;

[0052] Figure 4 A schematic diagram of the hardware structure of a network node 150 provided in this application;

[0053] Figure 5 is a flowchart of a processing method according to the first embodiment;

[0054] Figure 6 is a schematic diagram of an encoding process of an encoder in an image processing method according to the first embodiment;

[0055] Figure 7 is a schematic diagram of a decoding process of a decoder in an image processing method according to the first embodiment;

[0056] Figure 8 is a schematic diagram of a reference area of ​​a DIMD mode in a processing method according to a second embodiment;

[0057] Figure 9 is a schematic diagram of the gradient amplitude in the processing method according to the second embodiment;

[0058] Figure 10 is a schematic diagram of an encoded area corresponding to a block to be predicted in a processing method according to the second embodiment;

[0059] Figure 11 is a schematic diagram of a decoded area corresponding to a block to be predicted in a processing method according to the second embodiment;

[0060] Figure 12 is a schematic diagram of a reference area of ​​a neural network model in a processing method according to a fourth embodiment;

[0061] Figure 13is a schematic structural diagram of a neural network model based on a fully connected layer according to the fourth embodiment;

[0062] Figure 14 is a schematic structural diagram of a convolutional layer-based neural network model according to the fourth embodiment;

[0063] Figure 15 2 is a schematic diagram of the structure of a neural network model based on a hybrid convolutional and fully connected layer according to the fourth embodiment;

[0064] Figure 16 is a schematic diagram of intra-frame prediction direction according to the sixth embodiment;

[0065] Figure 17 is a schematic diagram of a reference area corresponding to a block to be predicted in an image processing method according to a sixth embodiment;

[0066] Figure 18 It is a schematic diagram of a processing module of a processing device.

[0067] The purpose of this application, its features, and advantages will be further described in conjunction with the embodiments and with reference to the accompanying drawings. The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and the accompanying text are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of this application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0068] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0069] It should be noted that, in this document, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0070] It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this document, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determination." Furthermore, as used herein, the singular forms "one," "an," and "the" are intended to also include plural forms, unless otherwise indicated in the context. It should be further understood that the terms "comprising" and "including" indicate the presence of the described features, steps, operations, elements, components, items, types, and / or groups, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, types, and / or groups. The terms “or”, “and / or”, “including at least one of the following”, etc. used in this application may be interpreted as inclusive, or mean any one or any combination, for example, “including at least one of the following: A, B, C” means “any one of the following: A; B; C; A and B; A and C; B and C; A and B and C”, for example, “A, B or C” or “A, B and / or C” means “any one of the following: A; B; C; A and B; A and C; B and C; A and B and C”. An exception to this definition will only occur when a combination of elements, functions, steps or operations are inherently mutually exclusive in some way.

[0071] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence in the order indicated by the arrows. Unless clearly stated herein, the execution of these steps is not strictly limited in order, and they can be performed in other orders. Moreover, at least a portion of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0072] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0073] It should be noted that, in this article, step codes such as S10 are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the sequence.

[0074] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0075] In the subsequent description, the use of suffixes such as "module", "component" or "unit" to represent elements is only for the purpose of facilitating the description of the present application and has no specific meaning. Therefore, "module", "component" or "unit" can be used interchangeably.

[0076] The processing device can be implemented in various forms. For example, the processing device described in this application may include processing devices such as mobile phones, servers, tablet computers, laptops, PDAs, portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, and fixed terminals such as digital TVs and desktop computers.

[0077] The subsequent description will be made by taking a mobile terminal as an example. It will be understood by those skilled in the art that, in addition to components specifically used for mobile purposes, the configuration according to the embodiments of the present application can also be applied to fixed-type terminals.

[0078] See also Figure 1 , which is a schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present application. The mobile terminal 100 may include: an RF (Radio Frequency) unit 101, a WiFi module 102, an audio output unit 103, an A / V (audio / video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111. Those skilled in the art will understand that Figure 1 The structure of the mobile terminal shown in the figure does not constitute a limitation to the mobile terminal. The mobile terminal may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0079] The following combination Figure 1 A detailed introduction to the various components of the mobile terminal:

[0080] The RF unit 101 can be used to send and receive information or receive signals during calls. Specifically, it receives downlink information from the base station and transmits it to the processor 110 for processing; and / or sends uplink data to the base station. Typically, the RF unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, and more. Furthermore, the RF unit 101 can communicate with the network and other devices via wireless communication. The above-mentioned wireless communications may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution), 5G and 6G, etc.

[0081] WiFi is a short-range wireless transmission technology. Mobile terminals can help users send and receive emails, browse web pages, and access streaming media through the WiFi module 102. It provides users with wireless broadband Internet access. Figure 1 The WiFi module 102 is shown, but it is understandable that it is not an essential component of the mobile terminal and can be omitted as needed without changing the essence of the invention.

[0082] The audio output unit 103 can convert audio data received by the RF unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output it as sound when the mobile terminal 100 is in a call signal reception mode, a talk mode, a recording mode, a voice recognition mode, a broadcast reception mode, or the like. Furthermore, the audio output unit 103 can also provide audio output related to a specific function performed by the mobile terminal 100 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 103 may include a speaker, a buzzer, or the like.

[0083] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos captured by an image capture device (e.g., a camera) in video capture mode or image capture mode. The processed image frames may be displayed on the display unit 106. The image frames processed by the GPU 1041 may be stored in the memory 109 (or other storage medium) or transmitted via the RF unit 101 or the WiFi module 102. The microphone 1042 may receive sound (audio data) in operating modes such as phone call mode, recording mode, and voice recognition mode, and may process such sound into audio data. In phone call mode, the processed audio (voice) data may be converted into a format that can be transmitted to a mobile communication base station via the RF unit 101. The microphone 1042 may implement various types of noise cancellation (or suppression) algorithms to eliminate (or suppress) noise or interference generated during the reception and transmission of audio signals.

[0084] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Optionally, the light sensor includes an ambient light sensor and a proximity sensor. Optionally, the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 1061 and / or the backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that recognize the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be described here.

[0085] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0086] The user input unit 107 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile terminal. Optionally, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also known as a touch screen, can collect user touch operations on or near it (such as operations performed by the user using a finger, stylus, or any other suitable object or accessory on or near the touch panel 1071) and drive the corresponding connection device according to a pre-set program. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Optionally, the touch detection device detects the user's touch direction and detects the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device and converts it into touch point coordinates, which are then sent to the processor 110. It can also receive commands sent by the processor 110 and execute them. In addition, the touch panel 1071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may further include other input devices 1072. Optionally, the other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, power keys, etc.), a trackball, a mouse, a joystick, etc., and the specifics are not limited here.

[0087] Optionally, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. The processor 110 then provides a corresponding visual output on the display panel 1061 according to the type of touch event. Figure 1 In the embodiment, the touch panel 1071 and the display panel 1061 are two independent components to realize the input and output functions of the mobile terminal. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to realize the input and output functions of the mobile terminal, which is not limited here.

[0088] The interface unit 108 serves as an interface through which at least one external device can be connected to the mobile terminal 100; for example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 108 can be used to receive input (e.g., data information, power, etc.) from the external device and transmit the received input to one or more elements within the mobile terminal 100 or can be used to transmit data between the mobile terminal 100 and the external device.

[0089] Memory 109 can be used to store software programs and various data. Memory 109 may primarily include a program storage area and a data storage area. Optionally, the program storage area may store an operating system and at least one application required for a function (such as a sound playback function or an image playback function); the data storage area may store data generated based on the use of the mobile phone (such as audio data, a phone book, etc.). Furthermore, memory 109 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0090] Processor 110 is the control center of the mobile terminal, connecting all components of the mobile terminal using various interfaces and circuits. By running or executing software programs and / or modules stored in memory 109 and accessing data stored in memory 109, it executes various functions of the mobile terminal and processes data, thereby providing overall monitoring of the mobile terminal. Processor 110 may include one or more processing units; preferably, processor 110 may integrate an application processor and a modem processor. Optionally, the application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 110.

[0091] The mobile terminal 100 may also include a power supply 111 (such as a battery) for supplying power to various components. Preferably, the power supply 111 may be logically connected to the processor 110 through a power management system, thereby managing functions such as charging, discharging, and power consumption through the power management system.

[0092] although Figure 1 Not shown, the mobile terminal 100 may further include a Bluetooth module, etc., which will not be described in detail here.

[0093] To facilitate understanding of the embodiments of the present application, the communication network system on which the mobile terminal of the present application is based is described below.

[0094] See also Figure 2 , Figure 2 A communication network system architecture diagram is provided for an embodiment of the present application. The communication network system is an LTE system of universal mobile communication technology. The LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203 and an operator's IP service 204, which are connected in sequence.

[0095] Optionally, UE201 may be the above-mentioned terminal 100, which will not be described in detail here.

[0096] E-UTRAN 202 includes eNodeB 2021 and other eNodeBs 2022 . Optionally, eNodeB 2021 may be connected to other eNodeBs 2022 via a backhaul (eg, an X2 interface). eNodeB 2021 is connected to EPC 203 , and eNodeB 2021 may provide access from UE 201 to EPC 203 .

[0097] EPC 203 may include an MME (Mobility Management Entity) 2031, an HSS (Home Subscriber Server) 2032, other MMEs 2033, an SGW (Serving Gate Way) 2034, a PGW (PDN Gate Way) 2035, and a PCRF (Policy and Charging Rules Function) 2036. Optionally, MME 2031 is a control node that processes signaling between UE 201 and EPC 203, providing bearer and connection management. HSS 2032 provides registers for managing functions such as the Home Location Register (not shown) and stores user-specific information such as service features and data rates. All user data can be sent through SGW2034, PGW2035 can provide IP address allocation and other functions for UE 201, PCRF2036 is the policy and charging control policy decision point for service data flow and IP bearer resources, and it selects and provides available policy and charging control decisions for the policy and charging execution function unit (not shown in the figure).

[0098] The IP service 204 may include the Internet, an intranet, an IMS (IP Multimedia Subsystem), or other IP services.

[0099] Although the above introduction takes the LTE system as an example, those skilled in the art should know that this application is not only applicable to the LTE system, but can also be applied to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, 5G and future new network systems (such as 6G), etc., which are not limited here.

[0100] Figure 3This is a schematic diagram of the hardware structure of a controller 140 provided in this application. The controller 140 includes a memory 1401 and a processor 1402. The memory 1401 is used to store program instructions, and the processor 1402 is used to call the program instructions in the memory 1401 to execute the steps performed by the controller in the first embodiment of the above method. The implementation principles and beneficial effects are similar and will not be repeated here.

[0101] Optionally, the controller further includes a communication interface 1403, which can be connected to the processor 1402 via a bus 1404. The processor 1402 can control the communication interface 1403 to implement the receiving and sending functions of the controller 140.

[0102] Figure 4 This is a schematic diagram of the hardware structure of a network node 150 provided in this application. The network node 150 includes a memory 1501 and a processor 1502. The memory 1501 is used to store program instructions, and the processor 1502 is used to call the program instructions in the memory 1501 to execute the steps performed by the first node in the first embodiment of the above method. The implementation principles and beneficial effects are similar and will not be repeated here.

[0103] Optionally, the controller further includes a communication interface 1503, which can be connected to the processor 1502 via a bus 1504. The processor 1502 can control the communication interface 1503 to implement the receiving and sending functions of the network node 150.

[0104] The integrated modules implemented in the form of software function modules can be stored in a computer-readable storage medium. The software function modules stored in a storage medium include a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute some of the steps of the methods of various embodiments of the present application.

[0105] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive solid state disk, SSD), etc.

[0106] First embodiment

[0107] Reference Figure 5 , Figure 5 : is a flow chart of a processing method according to the first embodiment. The processing method of the embodiment of the present application can be applied to a processing device, including:

[0108] Step S10 : determining or obtaining at least one prediction mode of the current block based on matching information between at least one candidate mode and the current block.

[0109] In this embodiment, the processing device can be a smart terminal, such as a mobile phone, a computer, etc., or a server, such as a local server or a cloud server. In this embodiment and this application, the processing device is mainly described as a smart terminal.

[0110] Optionally, the technical solution of this embodiment can be applied to the fields of image coding and decoding, video coding and decoding, hardware video coding and decoding, dedicated circuit video coding and decoding, real-time video coding and decoding, and so on.

[0111] Optionally, the processing device may obtain video image data from a video source, segment each frame of the video image data to obtain at least one image block, and determine the image block to be predicted in the at least one image block as the current block.

[0112] Optionally, the candidate mode is a prediction mode that may be adopted by the current block in the prediction process. Optionally, in intra-frame prediction, the candidate mode may include: at least one angular prediction mode, at least one non-angular prediction mode and / or a prediction mode determined or obtained using at least one derivation mode.

[0113] Optionally, the derivation mode is a derivation mode for deriving an intra-frame prediction mode.

[0114] Optionally, the angular prediction mode is a technology for predicting the current pixel block by propagating the values ​​of adjacent pixels along a specific direction to generate a prediction block. The angular prediction mode is mainly used to process directional textures in images, which can effectively reduce spatial redundancy and improve compression efficiency. In the H.265 / HEVC (High Efficiency Video Coding) standard, the intra-frame prediction mode includes 33 angular prediction modes, which cover different angles from horizontal to vertical directions, ensuring accurate prediction of various texture directions. In the H.266 / VVC (Versatile Video Coding) standard, the angular mode is expanded to 65, and the newly added directions are more dense to more accurately capture the edges in natural videos.

[0115] Optionally, the non-angle prediction mode may be a prediction mode other than the angle prediction mode. For example, the non-angle prediction mode includes at least one of the following: a DC mode (direct current mode), a Planar mode (planar mode), and a neural network-based prediction mode.

[0116] Optionally, the derivation mode is a mode for determining or obtaining a prediction mode and / or a candidate mode, and the derivation mode is a method of deriving a prediction mode matching the current block by analyzing relevant information of the current block. For example, the derivation mode includes at least one of the following: a decoder side intra mode derivation (DIMD) mode, an occurrence-based intra coding mode derivation (OBIC) mode, and a template-based intra mode derivation (TIMD) mode.

[0117] Optionally, prediction processing is performed on the current block according to at least one prediction mode of the current block determined or obtained through matching information related to at least one candidate mode and the current block.

[0118] Optionally, matching information refers to information used to evaluate the degree of matching between different prediction modes and the current block and at least one reference area corresponding thereto. The matching information can be based on multiple indicators, such as the rate-distortion cost of the prediction results, prediction error, texture similarity, etc., to indicate the degree of matching between different prediction modes and the reference area.

[0119] Optionally, the matching information includes: SAD (Sum of Absolute Differences), SATD (Sum of Absolute Transformed Differences) and / or MRSAD (Mean-Removed Sum of Absolute Differences). Optionally, the smaller the SAD value, SATD value and / or MRSAD value, the higher the matching degree; and / or, the larger the SAD value, SATD value and / or MRSAD value, the lower the matching degree.

[0120] Optionally, the matching information includes: a gradient amplitude value corresponding to a prediction mode of a gradient histogram or a sum of corresponding gradient amplitude values. Optionally, a larger gradient amplitude value indicates a higher degree of matching; and / or a smaller gradient amplitude indicates a lower degree of matching.

[0121] Optionally, the matching information includes: confidence information, and optionally, the greater the confidence, the higher the matching degree; and / or, the smaller the confidence, the smaller the matching degree.

[0122] Optionally, the matching information related to the prediction mode and / or candidate mode and the current block may be matching information related to the prediction mode and / or candidate mode and at least one reference region of the current block.

[0123] Optionally, based on matching information between at least one candidate mode and the current block, a candidate mode with the highest matching degree with the current block is determined or obtained, and used as the prediction mode of the current block.

[0124] Optionally, step S10 includes: determining or obtaining at least one prediction mode of the current block from the candidate modes based on at least one candidate mode of the current block determined or obtained using at least one derivation mode and matching information related to the at least one candidate mode.

[0125] Optionally, at least one candidate mode of the current block is determined or obtained based on at least one derived mode and pixels in a first reference area of ​​the current block, and at least one prediction mode of the current block is determined or obtained from the candidate modes based on matching information related to the candidate mode and a second reference area of ​​the current block.

[0126] Optionally, the first reference region and the second reference region are the same or different.

[0127] Optionally, the matching information is matching information between predicted pixel values ​​of the second reference region with respect to at least one candidate mode of the current block and reconstructed pixel values ​​corresponding to the prediction mode adopted by the second reference region.

[0128] Optionally, the reconstructed pixel value corresponding to the prediction mode adopted by the second reference region is a reconstructed pixel value of a reconstructed pixel obtained during the encoding or decoding process of the second reference region itself.

[0129] Optionally, the method of determining or obtaining at least one prediction mode of the current block based on matching information related to at least one candidate mode and the current block includes: determining or obtaining first matching information related to at least one prediction mode and the current block based on at least one derivation mode, determining or obtaining at least one candidate mode of the current block based on the first matching information, determining second matching information related to at least one candidate mode and a second reference area of ​​the current block based on at least one candidate mode, and determining or obtaining at least one prediction mode of the current block from at least one candidate mode based on the first matching information and / or the second matching information.

[0130] Optionally, the method of determining or obtaining at least one prediction mode of the current block based on matching information related to at least one candidate mode and the current block includes: determining or obtaining a first mode list of the current block based on first matching information related to the prediction mode and the current block, and determining or obtaining at least one prediction mode of the current block based on the corresponding first matching information and / or corresponding second matching information of the candidate mode in the first mode list.

[0131] Optionally, the first matching information and / or the second matching information includes the gradient amplitude value corresponding to the prediction mode of the gradient histogram or the sum of the corresponding gradient amplitude values, confidence information, at least one of SAD, SATD and / or MRSAD, and the first matching information and the second matching information are different.

[0132] Optionally, the first matching information is a gradient magnitude value corresponding to a prediction mode of a gradient histogram or a sum of corresponding gradient magnitude values, and the second matching information is at least one of SAD, SATD and / or MRSAD.

[0133] Optionally, the first matching information is confidence information, and the second matching information is at least one of SAD, SATD and / or MRSAD.

[0134] Optionally, the method of determining or obtaining at least one prediction mode of the current block based on the first matching information and / or the corresponding second matching information corresponding to the candidate mode in the first pattern list includes: determining or obtaining at least one prediction mode of the current block based on the sorting result and / or comparison result of the first matching information and / or the second matching information.

[0135] Optionally, the method of determining or obtaining the first pattern list of the current block based on the prediction pattern and the currently related first matching information includes: determining whether to use the prediction pattern corresponding to the first matching information as a candidate pattern in the first pattern list of the current block based on the sorting result and / or comparison result of the first matching information.

[0136] Optionally, the method of determining or obtaining at least one candidate mode of the current block based on the first matching information includes: determining whether to use the prediction mode corresponding to the first matching information as at least one candidate mode of the current block based on the sorting result and / or comparison result of the first matching information.

[0137] Optionally, the sorting result includes: sorting the at least one prediction mode according to the size of the first matching information or the second matching information of the at least one prediction mode, determining or obtaining the sorting information about the at least one prediction mode, for example, if the matching degree corresponding to the first matching information or the second matching information of the first prediction mode is higher than the matching degree corresponding to the first matching information or the second matching information of the second prediction mode, the first matching information or the second matching information of the prediction mode is sorted in descending order, then in the sorted mode list, the first prediction mode is located before the second prediction mode; and / or, if the matching degree corresponding to the first matching information or the second matching information of the first prediction mode is higher than the matching degree corresponding to the first matching information or the second matching information of the second prediction mode, the first matching information or the second matching information of the prediction mode is sorted in ascending order, then in the sorted mode list, the first prediction mode is located after the second prediction mode.

[0138] Optionally, the comparison result includes: determining N prediction modes having the largest or smallest first matching information or second matching information among the prediction modes according to the values ​​of the first matching information or the second matching information of the prediction modes of the current block.

[0139] Optionally, at least one prediction mode of the current block is determined or obtained based on matching information between at least one candidate mode and the current block, and a target prediction result of the current block is determined or obtained based on the at least one prediction mode and at least one weight information.

[0140] Optionally, the weight information includes weight information corresponding to the prediction mode and / or weight information corresponding to the prediction result.

[0141] Optionally, the weight information may be information about the proportion of at least one prediction result and / or at least one prediction model that contributes to the target prediction result. For example, the weight information may be a weight coefficient. Optionally, the weight information may also be a weight vector and a weight matrix containing weight coefficients, etc., which represents indicative information for measuring the proportion of different elements (for example, prediction values) in the result or decision.

[0142] Optionally, the confidence of at least one prediction result and / or the confidence of at least one prediction model are in positive proportion and / or positive correlation with the corresponding weight information.

[0143] Optionally, the weight information may include at least one of a weight coefficient, a weight vector, and a weight matrix.

[0144] Optionally, the weight information corresponding to the at least one prediction mode is determined or obtained according to matching information related to the at least one prediction mode and the current block.

[0145] Optionally, the weight information corresponding to the at least one prediction result is determined or obtained based on matching information related to the current block and at least one prediction mode of the at least one prediction result determined or obtained.

[0146] Optionally, the weight information corresponding to at least one prediction mode is in direct proportion and / or positive correlation with the matching information related to the current block, that is, the higher the matching degree, the greater the weight value corresponding to the weight information.

[0147] Optionally, prediction processing is performed on the current block according to the determined or obtained at least one prediction mode of the current block and at least one reference area of ​​the current block.

[0148] Optionally, at least one prediction mode is determined or obtained based on at least one reference area of ​​the current block and matching information between at least one candidate mode and the current block.

[0149] Optionally, the reference area includes at least one of a reference pixel, a reference block, and a reference template.

[0150] Optionally, the reference area used to determine or obtain matching information related to at least one prediction mode is the same as or different from the reference area used to determine or obtain at least one prediction mode of the current block.

[0151] Optionally, at least one reference area of ​​the current block is determined or obtained according to the size of the current block.

[0152] Optionally, based on the first target derivation mode, at least one candidate mode is determined or obtained, the first target derivation mode including at least one of the following: DIMD mode, OBIC mode and TIMD mode, and based on the matching information related to the at least one candidate mode and the current block, at least one prediction mode of the current block is determined or obtained from the at least one candidate mode.

[0153] Optionally, at least one candidate mode of a first preset number is determined or obtained based on a first target derivation mode (for example, a DIMD mode), and based on matching information related to at least one candidate mode and a reference area of ​​the current block, a mode with the highest matching degree is selected from at least one candidate mode and determined as the prediction mode of the current block for performing prediction processing on the current block.

[0154] Optionally, at least one candidate mode of a first preset number is determined or obtained based on a first target derivation mode (for example, a DIMD mode), and based on matching information related to the reference area of ​​the current block between the at least one candidate mode of the first preset number and the at least one prediction mode of the second preset number and weight information corresponding to each of the at least one prediction mode of the second preset number are determined or obtained from the at least one candidate mode, prediction processing is performed on the current block through the at least one prediction mode, and prediction results corresponding to each of the at least one prediction mode are determined or obtained, and based on the prediction results corresponding to each of the at least one prediction mode and the weight information corresponding to each of the at least one prediction mode, the target prediction result of the current block is determined or obtained.

[0155] Optionally, the first preset number is greater than or equal to the second preset number.

[0156] Optionally, the first preset number is 3 to 8, and the second preset number is 3.

[0157] Optionally, compared to determining the prediction mode only by the amplitude of the DIMD mode, OBIC mode, etc., the embodiment of the present application first selects at least one potentially valid candidate mode through the derivation mode such as the DIMD mode, OBIC mode, etc., and then directly verifies the actual performance of the mode through the matching information related to the candidate mode and the current block, so as to perform secondary screening, so that the determined or obtained prediction mode has higher accuracy.

[0158] Optionally, based on the first target derivation mode of the current block (for example, the DIMD mode), at least one candidate mode is determined or obtained, based on the second target derivation mode (for example, the TIMD mode), matching information related to at least one candidate mode is determined or obtained, and based on the matching information related to at least one candidate mode, at least one prediction mode of the current block is determined or obtained.

[0159] Optionally, the candidate modes include modes belonging to a first number of prediction modes and / or modes belonging to a second number of prediction modes, and the first number and the second number are the same or different. For example, the candidate modes include prediction modes among 67 prediction modes and / or prediction modes among 131 prediction modes.

[0160] Optionally, the first number is greater than the second number, and by increasing the number of prediction modes (eg, angle prediction modes), the candidate modes are made to better fit the actual edge direction of the current block, thereby improving the prediction accuracy of the current block.

[0161] Optionally, at least one prediction result of the current block determined or obtained by performing prediction processing on the current block through at least one prediction mode is fused to determine or obtain a target prediction result of the current block.

[0162] Optionally, at least one prediction result of the current block is weightedly fused to determine or obtain a target prediction result of the current block. Optionally, weighted fusion refers to assigning different weight information to at least one prediction result based on the confidence of at least one prediction result and / or the confidence of at least one prediction mode corresponding to at least one prediction result, and performing fusion based on the weight information. For example, if a prediction result and / or its corresponding prediction mode has a higher confidence, the prediction result may be given a higher weight information; and / or, if a prediction result and / or its corresponding prediction mode has a lower confidence, the prediction result may be given a lower weight information.

[0163] Optionally, at least one prediction result is fused using weight information, thereby combining the advantages of different prediction modes and improving prediction accuracy.

[0164] Optionally, at least one candidate mode is determined or obtained according to at least one target derivation mode of the current block determined or obtained by the size of the current block.

[0165] Optionally, at least one candidate mode is determined or obtained according to a prediction mode corresponding to at least one target derivation mode.

[0166] Optionally, during the intra-frame prediction and / or inter-frame prediction process, the same derivation mode is used for image blocks of different sizes, resulting in unsatisfactory prediction effects of candidate modes / prediction modes obtained based on the derivation mode for image blocks of certain sizes, thereby causing poor encoding and decoding quality during video encoding and / or decoding. However, through the technical solution of the present application, suitable target derivation modes can be matched for current blocks of different sizes, and then suitable candidate modes and / or prediction modes can be further matched for the current block based on the target derivation mode, thereby supporting improved prediction accuracy of the current block.

[0167] Optionally, at least one target derivation mode of the current block is determined or obtained according to the size of the current block, and at least one candidate mode is determined or obtained according to the at least one target derivation mode.

[0168] Optionally, if the size of the current block satisfies the first condition, the target derivation mode is the first derivation mode, and at least one candidate mode is determined or obtained according to the first derivation mode.

[0169] Optionally, the size of the current block includes at least one of width, height, scale, depth, area, resolution and number of pixels of the current block.

[0170] Optionally, the first condition may be a condition set in advance. Optionally, the first condition is not fixed and may be adaptively adjusted according to different scenarios.

[0171] Optionally, the first condition may be set according to the value of the quantization parameter or the quantization information.

[0172] Optionally, when the quantization parameter is the first quantization parameter (for example, the value of the quantization parameter is 37), the first condition is that the width × height of the current block is greater than a first threshold, and the first threshold can be 1024, 512, etc. For example, the first condition is that the width × height of the current block is greater than 1024, or the first condition is that the width × height of the current block is greater than 512.

[0173] Optionally, when the quantization parameter is the second quantization parameter (for example, the value of the quantization parameter is 22), the first condition is that the width × height of the current block is greater than the second threshold, and the second threshold can be 1024, 512, etc. For example, the first condition is that the width × height of the current block is greater than 1024, or the first condition is that the width × height of the current block is greater than 512.

[0174] Optionally, the first quantization parameter and the second quantization parameter may be the same or different, and the first threshold and the second threshold may be the same or different.

[0175] Optionally, the size of the current block satisfies a first condition, including at least one of the following:

[0176] The size of the current block is a first value;

[0177] The value of the size of the current block is within a first numerical interval;

[0178] The size of the current block is greater than or equal to a first threshold.

[0179] Optionally, the first value may be a size of the image block set in advance, for example, a value of at least one of width, height, block size and block area.

[0180] Optionally, the first value may be a value of the width and / or height of the image block, for example, the first value may be 32, 64, 128, 256, etc.

[0181] Optionally, the first value may be a value of a block area of ​​an image block, for example, the first value may be 256, 512, 1024, etc.

[0182] Optionally, the first numerical interval may be a size of the image block set in advance, for example, a numerical interval of at least one of width, height, block size and block area.

[0183] Optionally, the first numerical interval may be a numerical interval of width and / or height of the image block, for example, the first numerical interval is [32, 256].

[0184] Optionally, the first numerical interval may be a numerical interval of block areas of image blocks, for example, the first numerical interval is [256, 1024].

[0185] Optionally, the first threshold may be a size of the image block set in advance, for example, a threshold of at least one of width, height, block size, and block area.

[0186] Optionally, the first threshold may be a threshold of width and / or height of the image block, for example, the first threshold may be 32, 64, 128, 256, etc.

[0187] Optionally, the first threshold may be a threshold of a block area of ​​an image block, for example, the first threshold may be 256, 512, 1024, etc.

[0188] Optionally, if the size of the current block is greater than or equal to a first threshold, the target derivation mode is the first derivation mode.

[0189] Optionally, the number of prediction modes corresponding to the first derivation mode is a first number. If the size of the current block is greater than or equal to a first threshold, indicating that the current block belongs to a large-size image block, the target derivation mode is the first derivation mode, and at least one candidate mode is determined or obtained based on the first derivation mode.

[0190] Optionally, the first number is 35, 67 or 131.

[0191] Optionally, the first number is greater than or equal to 35, 67 or 131.

[0192] Optionally, the first derivation mode includes at least one of the following: DIMD mode, OBIC mode, TIMD mode, improved DIMD mode, improved OBIC mode and improved TIMD mode.

[0193] Optionally, the number of prediction modes corresponding to the improved DIMD mode is greater than the number of prediction modes corresponding to the DIMD mode, and / or the number of prediction modes corresponding to the improved OBIC mode is greater than the number of prediction modes corresponding to the OBIC mode, and / or the number of prediction modes corresponding to the improved TIMD mode is greater than the number of prediction modes corresponding to the TIMD mode.

[0194] Optionally, the number of prediction modes corresponding to the DIMD mode, the OBIC mode, and the TIMD mode is 67, and the number of prediction modes corresponding to the improved DIMD mode, the improved OBIC mode, and the improved TIMD mode is 131.

[0195] Optionally, the processing method of the improved DIMD mode is different from or at least partially the same as the processing method of the DIMD mode, the processing method of the improved OBIC mode is different from or at least partially the same as the processing method of the OBIC mode, and the processing method of the improved TIMD mode is different from or at least partially the same as the processing method of the TIMD mode.

[0196] Optionally, the prediction accuracy can be improved by increasing the number of prediction modes, but more mode signaling needs to be transmitted, resulting in an increase in bitstream overhead. However, in the derivation mode, there is no need to explicitly transmit the mode index. Therefore, increasing the number of prediction modes corresponding to the derivation mode will not increase the signaling amount, but may increase the computational complexity. Image blocks of different sizes have different sensitivities to the number of prediction modes. For example, for large-size image blocks with a size greater than or equal to the first threshold, the reference area covers a wider range of pixels. Therefore, the accuracy of the angle prediction in the prediction mode is more sensitive to the texture direction. Increasing the number of angle prediction modes in the prediction mode can more accurately match the actual texture direction. For small-size image blocks with a size less than the first threshold, the reference area covers a limited number of pixels, and the local correlation between adjacent pixels is stronger. Therefore, the sensitivity to the number of angle prediction modes in the prediction mode is lower. Therefore, for large-size image blocks with a size greater than or equal to the first threshold, using the first derivation mode with a larger number of corresponding prediction modes can achieve higher benefits.

[0197] Optionally, in the H.266 / VVC standard, the number of prediction modes is 67, and its prediction accuracy for large-size image blocks is low. Therefore, in an embodiment of the present application, when the size of the current block is greater than the first threshold, the target derivation mode of the current block is determined to be the first derivation mode, and the number of prediction modes of the first derivation mode is the first number, which is greater than 67. For example, the first number is 131. Since the number of prediction modes corresponding to the first derivation mode is greater than 67, the angle prediction mode corresponding to the first derivation mode can be more in line with the actual edge direction of the large-size image block, thereby improving the prediction accuracy for the current block.

[0198] Optionally, at least one candidate mode is determined or obtained according to the prediction mode corresponding to the first derivation mode.

[0199] Optionally, the prediction direction corresponding to the prediction mode corresponding to the first derivation mode at least partially covers the prediction directions corresponding to 67 prediction modes in the H.266 / VVC standard, and the number of prediction modes is increased by filling the angular intervals between the angular prediction modes in the original prediction mode.

[0200] Optionally, the first derivation mode and its corresponding prediction mode are associated via a first lookup table.

[0201] Optionally, the first lookup table is a predefined static table that stores a mapping relationship between the first derivation mode and its corresponding prediction mode to avoid real-time calculation. Optionally, the first lookup table can be implemented by a storage unit.

[0202] Optionally, in the first derivation mode, the prediction mode index is mapped to the prediction mode index through the first lookup table, and the prediction mode is determined according to the prediction mode index and used as a candidate mode. Optionally, the number of the first lookup table can be one or more.

[0203] Optionally, the first lookup table includes a prediction mode lookup table and / or a prediction angle lookup table.

[0204] Optionally, the prediction mode lookup table stores a mapping relationship between the index of the prediction mode and the corresponding prediction direction.

[0205] Optionally, the prediction angle lookup table stores specific geometric directions corresponding to the angle prediction modes, for example, 0° to +45°.

[0206] In this embodiment, when the size of the current block satisfies the first condition, the target derivation mode is determined to be the first derivation mode, and then at least one candidate mode is determined or obtained based on the first derivation mode, so as to match the current blocks of different sizes with suitable target derivation modes, and then the current block can be further matched with suitable candidate modes and / or prediction modes based on the target derivation mode, thereby supporting the improvement of the prediction accuracy of the current block, and further supporting the improvement of the encoding and decoding quality in the video encoding and / or decoding process.

[0207] Optionally, if the size of the current block does not meet the first condition, the target derivation mode is the second derivation mode, and at least one candidate mode of the current block is determined or obtained according to the second derivation mode.

[0208] Optionally, the second derivation mode is the same as or different from the first derivation mode.

[0209] Optionally, the second derivation mode and the first derivation mode respectively correspond to different numbers of prediction modes, and / or the second derivation mode and the first derivation mode belong to different derivation mode types.

[0210] Optionally, the size of the current block includes at least one of: width, height, area and aspect ratio of the current block.

[0211] Optionally, the first condition may be a condition set in advance. Optionally, the first condition is not fixed and may be adaptively adjusted according to different scenarios.

[0212] Optionally, the first condition may be set according to the value of the quantization parameter or the quantization information.

[0213] Optionally, when the quantization parameter is the first quantization parameter (for example, the value of the quantization parameter is 37), the first condition is that the width×height of the current block is greater than a first threshold.

[0214] Optionally, the first threshold may be 1024, 512, etc. For example, the first condition is that the width×height of the current block is greater than 1024, or the first condition is that the width×height of the current block is greater than 512.

[0215] Optionally, when the quantization parameter is the second quantization parameter (for example, the value of the quantization parameter is 22), the first condition is that the width×height of the current block is greater than the second threshold.

[0216] Optionally, the second threshold may be 1024, 512, etc. For example, the first condition is that the width×height of the current block is greater than 1024, or the first condition is that the width×height of the current block is greater than 512.

[0217] Optionally, the first quantization parameter and the second quantization parameter may be the same or different, and the first threshold and the second threshold may be the same or different.

[0218] Optionally, the size of the current block does not satisfy the first condition, including at least one of the following:

[0219] The value of the size of the current block is not the first value;

[0220] The value of the size of the current block is not within the first numerical interval;

[0221] The value of the size of the current block is smaller than the first threshold.

[0222] Optionally, the first value may be a size of the image block set in advance, for example, a value of at least one of width, height, block size and block area.

[0223] Optionally, the first value may be a value of the width and / or height of the image block, for example, the first value may be 32, 64, 128, 256, etc.

[0224] Optionally, the first value may be a value of a block area of ​​an image block, for example, the first value may be 256, 512, 1024, etc.

[0225] Optionally, the first numerical interval may be a size of the image block set in advance, for example, a numerical interval of at least one of width, height, block size and block area.

[0226] Optionally, the first numerical interval may be a numerical interval of width and / or height of the image block, for example, the first numerical interval is [32, 256].

[0227] Optionally, the first numerical interval may be a numerical interval of block areas of image blocks, for example, the first numerical interval is [256, 1024].

[0228] Optionally, the first threshold may be a size of the image block set in advance, for example, a threshold of at least one of width, height, block size, and block area.

[0229] Optionally, the first threshold may be a threshold of width and / or height of the image block, for example, the first threshold may be 32, 64, 128, 256, etc.

[0230] Optionally, the first threshold may be a threshold of a block area of ​​an image block, for example, the first threshold may be 256, 512, 1024, etc.

[0231] Optionally, if the size of the current block is smaller than the first threshold, the target derivation mode is the second derivation mode.

[0232] Optionally, the number of prediction modes corresponding to the second derivation mode is a second number. If the size of the current block is smaller than the first threshold, indicating that the current block belongs to a small-size image block, the target derivation mode is the second derivation mode. According to the second derivation mode, at least one prediction mode of the current block is determined or obtained.

[0233] Optionally, the number of prediction modes corresponding to the first derivation mode is a first number, the number of prediction modes corresponding to the second derivation mode is a second number, and the first number and the second number are the same or different.

[0234] Optionally, the first number and the second number are the same, for example, the first number and the second number are both 35, 67 or 131.

[0235] Optionally, the second number is smaller than the first number.

[0236] Optionally, the second number is 35, 67 or 131.

[0237] Optionally, the second number is greater than or equal to 35, 67 or 131.

[0238] Optionally, the second derivation mode includes at least one of the following: DIMD mode, OBIC mode, TIMD mode, improved DIMD mode, improved OBIC mode and improved TIMD mode.

[0239] Optionally, the number of prediction modes corresponding to the improved DIMD mode is greater than the number of prediction modes corresponding to the DIMD mode, and / or the number of prediction modes corresponding to the improved OBIC mode is greater than the number of prediction modes corresponding to the OBIC mode, and / or the number of prediction modes corresponding to the improved TIMD mode is greater than the number of prediction modes corresponding to the TIMD mode.

[0240] Optionally, the number of prediction modes corresponding to the DIMD mode, the OBIC mode, and the TIMD mode is 67, and the number of prediction modes corresponding to the improved DIMD mode, the improved OBIC mode, and the improved TIMD mode is greater than 67.

[0241] Optionally, the processing method of the improved DIMD mode is different from or at least partially the same as the processing method of the DIMD mode, the processing method of the improved OBIC mode is different from or at least partially the same as the processing method of the OBIC mode, and the processing method of the improved TIMD mode is different from or at least partially the same as the processing method of the TIMD mode.

[0242] Optionally, the number of prediction modes corresponding to the first derivation mode is a first number, and the number of prediction modes corresponding to the second derivation mode is a second number. The first number is greater than the second number, indicating that the number of prediction modes corresponding to the first derivation mode is greater than the number of prediction modes corresponding to the second derivation mode. If the size of the current block meets the first condition, for example, the value of the size of the current block is greater than or equal to the first threshold, indicating that the current block belongs to a large-size image block, then the target derivation mode of the current block is the first derivation mode with a larger number of corresponding prediction modes, and at least one candidate mode of the current block is determined or obtained based on the first derivation mode, and the candidate mode belongs to the first derivation mode. One of the prediction modes under a number, to meet the prediction requirements of large-size image blocks for which the accuracy of angle prediction in the prediction mode is more sensitive to the texture direction, more finely match the actual texture direction of the current block, and improve the prediction accuracy. If the size of the current block does not meet the first condition, for example, the value of the size of the current block is less than the first threshold, indicating that the current block belongs to a small-size image block, then the target derivation mode of the current block is a second derivation mode with a smaller number of corresponding prediction modes, and at least one candidate mode of the current block is determined or obtained based on the second derivation mode, and the candidate mode belongs to one of the prediction modes under the second number, so as to avoid increasing the computational complexity.

[0243] Optionally, at least one candidate mode is determined or obtained according to the prediction mode corresponding to the second derivation mode.

[0244] Optionally, the first number is 35, 67 or 131, for example, the first number is 131.

[0245] Optionally, the first number is greater than 35, 67 or 131.

[0246] Optionally, the second number is 35, 67 or 131, for example, the second number is 67.

[0247] Optionally, the second number is less than or equal to 35, 67 or 131.

[0248] Optionally, the first number and the second number are the same, that is, the number of prediction modes corresponding to the first derivation mode is the same as the number of prediction modes corresponding to the second derivation mode. For example, the first derivation mode and the second derivation mode each correspond to the same number of prediction modes, but the types of the corresponding prediction modes are at least partially different.

[0249] Optionally, the first number is greater than the second number, and the second number is greater than or equal to 67.

[0250] Optionally, the number of prediction modes corresponding to the second derivation mode is a second number, which is 67; the number of prediction modes corresponding to the first derivation mode is a first number, which is 131.

[0251] Optionally, the prediction direction corresponding to the prediction mode corresponding to the second derivation mode at least partially covers the prediction directions corresponding to 67 prediction modes in the H.266 / VVC standard, and the number of prediction modes is increased by filling the angular intervals between the angular prediction modes in the original prediction mode.

[0252] Optionally, the second derivation mode and its corresponding prediction mode are associated via a second lookup table.

[0253] Optionally, the second lookup table is a predefined static table that stores a mapping relationship between the second derivation mode and its corresponding prediction mode to avoid real-time calculation. Optionally, the first lookup table can be implemented by a storage unit.

[0254] Optionally, in the second derivation mode, the prediction mode index is mapped to the prediction mode index through the second lookup table, and the prediction mode is determined according to the prediction mode index and used as the candidate mode.

[0255] Optionally, the second lookup table includes a prediction mode lookup table and / or a prediction angle lookup table. Optionally, the number of the second lookup table may be one or more.

[0256] Optionally, the prediction mode lookup table stores a mapping relationship between the index of the prediction mode and the corresponding prediction direction.

[0257] Optionally, the prediction angle lookup table stores specific geometric directions corresponding to the angle prediction modes, for example, 0° to +45°.

[0258] In this embodiment, when the size of the current block does not meet the first condition, the target derivation mode is determined to be the second derivation mode, and then at least one candidate mode is determined or obtained based on the second derivation mode, so as to match the current blocks of different sizes with suitable target derivation modes, and then the current block can be further matched with suitable candidate modes and / or prediction modes based on the target derivation mode, thereby supporting the improvement of the prediction accuracy of the current block, and further supporting the improvement of the encoding and decoding quality in the video encoding and / or decoding process.

[0259] Reference Figure 6 When the processing device is an encoder on the encoding side, the encoder can receive video data input from a video source, for example, the encoder receives a video image from the video source, determines an image to be predicted in the video image, divides the image to be predicted into at least one image block, and uses the temporal and / or spatial correlation between the video images to perform prediction processing on each of the at least one image block, including intra-frame prediction processing and / or inter-frame prediction processing, and the intra-frame prediction processing and / or inter-frame prediction processing include at least one prediction mode derivation mode and / or at least one prediction mode. For the prediction mode, the encoder uses, for example, rate-distortion cost to determine the prediction mode ultimately adopted by each of the at least one image block, for example, calculating the rate-distortion cost corresponding to each prediction mode or the rate-distortion cost of a combination of several prediction modes to determine the minimum rate-distortion cost from the at least one rate-distortion cost. The prediction mode or combination of prediction modes corresponding to the minimum rate-distortion cost is the prediction mode ultimately adopted by the image block. At least one prediction mode of the current block can be determined or obtained based on matching information related to at least one candidate mode and the current block.

[0260] Optionally, after determining or obtaining at least one prediction mode of the image block to be predicted (ie, the current block), prediction processing is performed on the image block to be predicted using at least one prediction mode to determine or obtain a prediction block (ie, a prediction result) of the image block to be predicted.

[0261] Optionally, a residual block between the predicted block and the current block may be calculated, and the residual block may be transformed and quantized before being encoded by an entropy encoder to form an encoded bitstream.

[0262] Optionally, the encoded bitstream may include prediction parameters corresponding to the determined prediction mode and related side information.

[0263] Optionally, the prediction parameters are entropy coded and then packed into the coded bitstream.

[0264] Optionally, the prediction parameters include indicative information of the prediction mode.

[0265] Optionally, the transformed and quantized residual block can be added to the corresponding prediction data (such as the prediction block) obtained using the prediction mode after inverse quantization and inverse transformation to obtain a reconstructed block. After obtaining the reconstructed block, the loop filtering module performs loop filtering on the reconstructed block according to the filter control parameters to reduce distortion.

[0266] Optionally, after the loop filtering process is performed, the reconstructed block after the loop filtering process is stored according to the coded image buffer.

[0267] Reference Figure 7 When the processing device is a decoder on the decoding side, after receiving the encoded bitstream, the decoder's entropy decoding unit parses and decodes the encoded bitstream to obtain transform coefficients. The decoder's inverse transform unit and inverse quantization unit inverse transform and inverse quantize the transform coefficients to obtain a residual block.

[0268] Optionally, the entropy decoding unit of the decoder parses and decodes the encoded bit stream to obtain prediction data, such as prediction parameters and related auxiliary information.

[0269] Optionally, the prediction processing unit of the decoder performs prediction processing using the prediction parameters, thereby determining a prediction block corresponding to the residual block.

[0270] Optionally, the prediction process includes intra-frame prediction process and / or inter-frame prediction process, and the intra-frame prediction process and / or inter-frame prediction process includes a combination of at least one derivation mode and / or at least one prediction mode.

[0271] Optionally, the processing method includes: determining or obtaining at least one prediction mode of the current block according to matching information related to at least one candidate mode and the current block.

[0272] Optionally, prediction processing is performed on the image block to be predicted according to at least one prediction mode, and a prediction block of the image block to be predicted is determined or obtained. The obtained residual block and the corresponding prediction block (including the predicted luminance block and the predicted chrominance block) are added to obtain a reconstructed block. The loop filtering unit of the decoder performs loop filtering processing on the reconstructed block to reduce distortion and improve video quality.

[0273] Optionally, the processing method further includes: the reconstructed blocks that have undergone loop filtering are further combined into a decoded image and stored in a decoded image buffer or output as a decoded video signal.

[0274] Optionally, when the processing device is an encoder, the initially obtained prediction value may be a prediction value obtained in a corresponding prediction mode, and the prediction value may be directly used in the rate-distortion cost process.

[0275] Optionally, when the processing device is a decoder, the initially obtained prediction value may be a prediction value obtained in a prediction mode corresponding to a block to be predicted (ie, an image block located in a decoding end) indicated by a syntax element obtained by parsing the bitstream.

[0276] Alternatively, the predicted block may be used as the target image block, and a residual block between the target image block and the current block may be calculated. The residual block may then be transformed and quantized, and then encoded by an entropy encoder to form an encoded bitstream. Alternatively, the predicted block may be processed in some corresponding manner, such as by using another model, and the resulting image block may be used as the target image block, and the step of calculating the residual block between the target image block and the current block and subsequent steps may be performed.

[0277] In this embodiment, a primary screening is implemented through candidate modes to narrow the selection range of the prediction mode of the current block, select at least one potentially effective candidate mode, and then directly verify the actual performance of the mode through the matching information related to the candidate mode and the current block, so as to perform a secondary screening, so that the determined or obtained prediction mode has higher accuracy.

[0278] Second embodiment

[0279] Based on the above-mentioned first embodiment, a second embodiment is proposed.

[0280] In this embodiment, the candidate mode is determined or obtained according to at least one of the following methods 1 to 5:

[0281] Method 1: at least one gradient histogram corresponding to the current block.

[0282] Optionally, step S10 includes: determining or obtaining at least one candidate mode according to at least one gradient histogram corresponding to the current block, and determining or obtaining at least one prediction mode of the current block according to matching information related to the at least one candidate mode and the current block.

[0283] Optionally, a gradient histogram is a statistical tool used to describe the distribution of gradient amplitudes of the current block in different directions. It calculates the gradient amplitude value of each pixel in at least one reference area of ​​the current block in the direction corresponding to each intra-frame prediction mode, and statistically analyzes the distribution of the gradient amplitude values, thereby obtaining a histogram containing the gradient amplitude values ​​corresponding to each prediction direction.

[0284] Optionally, in the prediction mode of intra-frame prediction, different prediction modes correspond to different prediction directions, for example, the vertical mode corresponds to the vertical direction, the horizontal mode corresponds to the horizontal direction, and the diagonal mode corresponds to the diagonal direction, and each prediction mode has a specific direction for describing its prediction direction.

[0285] Optionally, for each prediction direction, the sum of the gradient magnitudes of each pixel in at least one reference region of the current block in that direction is calculated. This sum reflects the overall gradient strength of the current block in that prediction direction. The gradient histogram records the sum of the gradient magnitudes corresponding to each prediction direction.

[0286] Optionally, based on at least one gradient histogram corresponding to the current block, it is determined whether the sum of the gradient amplitude values ​​related to at least one prediction direction corresponding to the prediction mode contained in the gradient histogram meets the second condition. If the preset condition is met, at least one candidate mode is determined or obtained.

[0287] Optionally, the second condition may be whether the sum of the gradient amplitudes is one of the sums of the N (for example, N is 3) gradient amplitude values ​​with the largest gradient amplitudes in the gradient histogram. For example, at least one candidate mode is determined or obtained based on whether the sum of the gradient amplitude values ​​related to at least one prediction direction corresponding to the prediction mode contained in at least one gradient histogram is one of the sums of the N gradient amplitude values ​​with the largest gradient amplitudes in the gradient histogram.

[0288] Optionally, the sums of the gradient amplitude values ​​in the gradient histogram are sorted in ascending or descending order, and the sum of the first N gradient amplitude values ​​in the ascending order or the sum of the last N gradient amplitude values ​​in the descending order is determined or obtained. For example, after ascending sorting, the prediction mode corresponding to the prediction direction with the sum of the gradient amplitude values ​​in the top 3 is determined as the candidate mode.

[0289] Optionally, the gradient amplitude values ​​in the gradient histogram are compared to determine or obtain the sum of the N gradient amplitude values ​​with the largest gradient amplitudes in the gradient histogram, and at least one candidate mode of the current block is determined or obtained based on the prediction mode corresponding to the sum of the N gradient amplitude values ​​with the largest gradient amplitudes.

[0290] Optionally, a gradient histogram of at least one reference area with respect to the intra-frame prediction direction can be determined or obtained based on the gradient amplitude value and gradient direction of pixels in at least one reference area of ​​the current block, the corresponding intra-frame prediction direction can be determined based on the gradient histogram, and at least one candidate mode can be determined or obtained.

[0291] Optionally, at least one reference area of ​​the current block includes: at least one of: a reference pixel, a reference block and a reference template. The reference area refers to a group of neighboring pixels surrounding the current block. These neighboring pixels can provide important clues about the edge and texture direction of the block to be predicted. Therefore, the target derivation mode can analyze and derive the intra-frame prediction mode of the current block based on these pixels.

[0292] Optionally, the processing method in the embodiment of the present application may be a DIMD mode and / or an improved DIMD mode.

[0293] Reference Figure 8 The processing steps of the DIMD mode, the processing steps of the improved DIMD mode, and / or the method for determining or obtaining the gradient histogram include: determining a reference template adjacent to the to-be-predicted block (current block) above and to the left of the to-be-predicted block, i.e., three pixel lines / pixel rows / pixel columns on the left and above the to-be-predicted block, and then taking pixels in the middle line (e.g., pixel A) as pixels for calculating gradients. By calculating the direction of the gradient of at least one pixel in the middle line, as well as the magnitude of the horizontal gradient and the vertical gradient, the gradient direction of the at least one pixel and the gradient magnitude value corresponding to the gradient can be obtained.

[0294] Optionally, the gradient amplitude value is the sum of the absolute values ​​of the horizontal gradient and the vertical gradient. If the gradient amplitude values ​​of at least one pixel with the same gradient direction are added together, the sum of the gradient amplitude values ​​corresponding to the gradient direction can be obtained.

[0295] Optionally, a histogram of gradient magnitude values ​​of different gradient directions of at least one pixel may be established, and a prediction direction perpendicular to the gradient direction of the maximum gradient magnitude value may be used as the prediction direction of the intra-frame prediction mode of the current block and / or the prediction direction of the candidate mode.

[0296] Optionally, a 3X3 horizontal Sobel operator and a vertical Sobel operator can be used to calculate the horizontal gradient Gx and the vertical gradient Gy. For example, the horizontal gradient Gx and the vertical gradient Gy of a pixel x4 in a pixel line can be calculated according to the following formula (1) and formula (2).

[0297]

[0298] Optionally, A can be a matrix consisting of 9 pixels centered on the pixel x4, and including the pixel x1 above, the pixel x3 on the left, the pixel x7 below, the pixel x5 on the right, the upper left pixel x0, the lower left pixel x6, the upper right pixel x2, and the lower right pixel x8, as shown in the following formula (3).

[0299]

[0300] Optionally, the G gradient amplitude value is the sum of the absolute values ​​of the horizontal gradient and the vertical gradient, and its calculation formula is shown in formula (4).

[0301] G=|Gx|+|Gy| Formula (4);

[0302] Optionally, the gradient direction corresponding to the pixel can be calculated by arctan(Gx / Gy) or arctan(Gy / Gx).

[0303] Optionally, since each gradient direction corresponds to a specific gradient direction range, and each gradient direction range corresponds to a prediction direction of an intra-frame prediction mode, for at least one pixel in a pixel line, the gradient amplitude values ​​of the at least one pixel with the same gradient direction range can be added together to obtain the sum of the gradient amplitude values ​​corresponding to the gradient direction range.

[0304] Optionally, the sum of the gradient amplitude values ​​in the prediction direction of the corresponding intra prediction mode may also be obtained.

[0305] Reference Figure 9 , including the gradient magnitude values ​​of the gradient magnitude corresponding to the prediction direction of each intra prediction mode, optionally, according to Figure 9 , the final selected intra-frame prediction mode is mode 30.

[0306] Optionally, the number of prediction modes corresponding to the DIMD mode is 67. In an embodiment of the present application, the number of prediction modes corresponding to the improved DIMD mode is greater than 67, for example, 131.

[0307] Optionally, the prediction directions corresponding to the prediction modes corresponding to the improved DIMD mode at least partially cover the prediction directions corresponding to the 67 prediction modes corresponding to the DIMD mode, and the number of prediction modes is increased by filling the angular intervals between the angular prediction modes in the original prediction mode.

[0308] In this embodiment, irrelevant prediction modes are quickly eliminated through at least one gradient histogram, thereby achieving a primary screening of the prediction modes, determining or obtaining at least one candidate mode, and then directly verifying the actual performance of the mode through matching information related to the candidate mode and the current block, so as to perform a secondary screening, so that the determined or obtained prediction mode has higher accuracy. The gradient histogram can accurately capture the dominant prediction direction (such as vertical, horizontal, or diagonal) related to at least one reference area of ​​the current block, thereby matching the current block with a suitable candidate mode and / or prediction mode.

[0309] Method 2: at least one area histogram corresponding to the current block.

[0310] Optionally, step S10 includes: determining or obtaining at least one candidate mode according to at least one area histogram corresponding to the current block, and determining or obtaining at least one prediction mode of the current block according to matching information related to the at least one candidate mode and the current block.

[0311] Optionally, the area histogram is used to describe at least one reference area of ​​the current block, for example, the area amplitude distribution of adjacent coding blocks and / or non-adjacent coding blocks in different intra-frame prediction directions. Optionally, the reference area can be an encoded area or a decoded area.

[0312] Optionally, an area histogram and / or statistical results may be generated by analyzing the relationship between the area magnitude value (eg, block size or texture coverage) of each coding block and the intra prediction direction.

[0313] Optionally, the area amplitude value can represent the size of the coding block, texture coverage or other area-related features. It reflects the texture distribution intensity of the coding block in a specific direction, and each coding block has a corresponding intra-frame prediction direction (for example, vertical, horizontal, diagonal, etc.). The area histogram counts the area amplitude values ​​in these directions. By analyzing the distribution of area amplitude values ​​in different intra-frame prediction directions recorded on the area histogram, it can be determined which prediction directions are statistically more consistent with the texture characteristics of the current block.

[0314] Optionally, the processing method in the embodiment of the present application may be an OBIC mode and / or an improved OBIC mode.

[0315] Optionally, at least one candidate mode is determined or obtained according to the prediction mode corresponding to the prediction direction in which the area magnitude value in the area histogram meets the third condition.

[0316] Optionally, the third condition may be an area-amplitude value that is at a higher position or order after the area-amplitude value is sorted according to a preset sorting rule, and / or the third condition may be an area-amplitude value that is at a preset position (for example, the preset position is the first) or a preset order range (for example, the preset order range is 1 to 3) after the area-amplitude value is sorted according to a preset sorting rule. For example, the preset sorting may be a sorting rule that arranges the area-amplitude values ​​from large to small, and the prediction mode corresponding to the prediction direction with the area-amplitude value in the top 3 in the area histogram is determined as a candidate mode.

[0317] Optionally, the basic principle of the improved OBIC mode proposed in the present application may be to determine at least one candidate mode by analyzing usage of intra-frame prediction modes used by adjacent coding blocks and / or non-adjacent coding blocks.

[0318] Optionally, the implementation process of the improved OBIC mode may include: determining the usage of the intra-frame prediction mode of at least one reference area of ​​the current block (including at least one adjacent coding block and / or non-adjacent coding block of the current block), calculating the area amplitude value and intra-frame prediction direction of at least one reference area, determining at least one area amplitude histogram or statistical result, and determining at least one prediction direction of the current block based on the area amplitude histogram or statistical result.

[0319] Optionally, determining the intra-frame prediction mode usage of the coding block includes: determining a coded area and / or a decoded area, determining the intra-frame prediction mode usage of the coded block in the coded area, and / or determining the intra-frame prediction mode usage of the decoded block in the decoded area.

[0320] Optionally, the coding unit includes coding blocks of three color components, the three color components including a luminance component and two chrominance components.

[0321] Optionally, determining the intra-frame prediction mode usage of at least one reference area of ​​the current block (including at least one adjacent coding block and / or non-adjacent coding block of the current block) may include: determining at least one coded area or at least one decoded area, determining the intra-frame prediction mode usage of the coded blocks in the coded area, and / or determining the intra-frame prediction mode usage of the decoded blocks in the decoded area, for example, Figure 10 As shown, in the coded area, there are coding blocks 4 and 6 adjacent to the block to be predicted (i.e., the current block), and coding blocks 1, 2, 3, 5, 7, and 8 not adjacent to the block to be predicted, as shown in FIG. Figure 11 As shown, in the decoded area there are decoding block 4 and decoding block 6 adjacent to the block to be predicted, and decoding block 1, decoding block 2, decoding block 3, decoding block 5, decoding block 7 and decoding block 8 not adjacent to the block to be predicted.

[0322] Optionally, the number of prediction modes corresponding to the OBIC mode is 67. In the embodiment of the present application, the number of prediction modes corresponding to the improved OBIC mode is greater than 67, for example, 131.

[0323] Optionally, the prediction direction corresponding to the prediction mode corresponding to the improved OBIC mode at least partially covers the prediction directions corresponding to the 67 prediction modes corresponding to the OBIC mode, and the number of prediction modes is increased by filling the angular intervals between the angular prediction modes in the original prediction mode.

[0324] In this embodiment, irrelevant prediction modes are quickly eliminated through at least one area histogram, thereby achieving a primary screening of the prediction modes, determining or obtaining at least one candidate mode, and then directly verifying the actual performance of the mode through matching information related to the candidate mode and the current block, so as to perform a secondary screening, so that the determined or obtained prediction mode has higher accuracy. The area histogram can accurately capture the dominant prediction direction related to at least one reference area of ​​the current block, thereby matching the current block with a suitable candidate mode and / or prediction mode.

[0325] Method three: at least one usage count histogram of the prediction mode corresponding to the encoded image block corresponding to the current block.

[0326] Optionally, step S10 includes: determining or obtaining at least one candidate mode based on at least one usage count histogram of the prediction mode corresponding to the encoded image block corresponding to the current block, and determining or obtaining at least one prediction mode of the current block based on matching information related to the at least one candidate mode and the current block.

[0327] Optionally, the coded image block corresponding to the current block is an adjacent or reference image block that has been coded (or decoded) before the current block, and these blocks can provide spatial and / or temporal prediction information for the prediction of the current block.

[0328] Optionally, the prediction mode usage count histogram corresponding to the encoded image block records the usage frequencies of different intra-frame prediction modes in the encoded image block. By recording the usage frequencies of different intra-frame prediction modes in the usage count histogram, it is possible to determine which prediction modes are frequently used in the encoded area.

[0329] Optionally, step S10 includes: determining or obtaining at least one candidate mode according to the prediction mode corresponding to the prediction direction whose usage count in the usage count histogram satisfies the fourth condition.

[0330] Optionally, the fourth condition may be the number of times used that is at a higher position or order after the number of times used is sorted according to a preset sorting rule, and / or the fourth condition may be the number of times used that is at a preset position (for example, the preset position is the first) or a preset order range (for example, the preset order range is 1 to 3) after the number of times used is sorted according to a preset sorting rule. For example, the preset sorting may be a sorting rule that arranges the number of times used from large to small, and the prediction mode corresponding to the prediction direction whose number of times used is in the top 3 in the number of times used histogram is determined as a candidate mode.

[0331] In this embodiment, irrelevant prediction modes are quickly excluded through at least one usage count histogram, thereby achieving a primary screening of the prediction modes, determining or obtaining at least one candidate mode, and then directly verifying the actual performance of the mode through matching information related to the candidate mode and the current block, so as to perform a secondary screening, so that the determined or obtained prediction mode has higher accuracy. The usage count histogram can accurately capture the prediction modes that are used more frequently in the encoded image blocks related to the current block, thereby matching the current block with a suitable candidate mode and / or prediction mode.

[0332] Method 4: at least one gradient operator.

[0333] Optionally, step S10 includes: determining or obtaining at least one candidate mode according to at least one gradient operator, and determining or obtaining at least one prediction mode of the current block according to matching information related to the at least one candidate mode and the current block.

[0334] Optionally, the gradient operator is a tool and / or method for determining or obtaining at least one gradient information, and can extract pixel gradient information from an image through a convolution operation or other mathematical operations.

[0335] Optionally, the gradient information includes: gradient direction and / or gradient magnitude.

[0336] Optionally, the at least one gradient operator includes at least one pair of mutually perpendicular gradient operators.

[0337] In this embodiment, irrelevant prediction modes are quickly eliminated through at least one gradient operator, thereby achieving a primary screening of the prediction modes, determining or obtaining at least one candidate mode, and then directly verifying the actual performance of the mode through matching information related to the candidate mode and the current block, so as to perform a secondary screening, so that the determined or obtained prediction mode has higher accuracy.

[0338] Method five, first mode list.

[0339] Optionally, step S10 includes: determining or obtaining at least one candidate mode according to the first mode list, and determining or obtaining at least one prediction mode of the current block according to matching information related to the at least one candidate mode and the current block.

[0340] Optionally, step S10 includes: determining or obtaining at least one candidate mode based on at least one gradient histogram corresponding to the current block, at least one area histogram corresponding to the current block, at least one usage count histogram of the prediction mode corresponding to the encoded image block corresponding to the current block, at least one gradient operator corresponding to at least one lookup table, and at least one item in the first mode list; and determining or obtaining at least one prediction mode of the current block based on matching information related to the at least one candidate mode and the current block.

[0341] Optionally, the first mode list is a set for storing at least one prediction mode, providing a variety of possible mode options for the prediction of the current block, and by comparing the prediction effects under different modes in the first mode list, a suitable prediction mode can be matched for the current block to improve the prediction accuracy for the current block.

[0342] Optionally, the first mode list is at least one mode list, and the prediction modes in the at least one mode list are different or at least partially the same.

[0343] Optionally, the first mode list includes an MPM (Most Probable Modes) list. Optionally, the first mode list in the embodiment of the present application may be an improved MPM list.

[0344] Optionally, the MPM list is a set of several prediction modes, which are considered to be the prediction modes most likely to be applicable to the current block, and the MPM list is determined or obtained based on the neighborhood information of the current block (such as the prediction mode of the upper block and the left block) and statistical information (such as the probability of the mode occurring). In this way, the MPM list can reduce the amount of mode information that the encoder needs to transmit, while improving the decoder's prediction accuracy of the prediction mode.

[0345] Optionally, the first pattern list includes at least one candidate pattern.

[0346] Optionally, in intra prediction, the candidate modes in the first mode list may include: at least one angular prediction mode, at least one non-angular prediction mode and / or a derived mode of at least one prediction mode.

[0347] Optionally, the candidate modes in the first mode list include prediction modes belonging to the first number of prediction modes and / or prediction modes belonging to the second number of prediction modes.

[0348] Optionally, the first mode list is determined or obtained according to gradient information of at least one reference area of ​​the current block.

[0349] Optionally, a first pattern list is determined or obtained based on at least one reference area of ​​the current block and a neural network model.

[0350] Optionally, a first pattern list is determined or obtained based on pixels of at least a reference area of ​​the current block and a neural network model.

[0351] Optionally, a first pattern list is determined or obtained based on pixels, gradient information and a neural network model of at least one reference area of ​​the current block.

[0352] Optionally, by using at least one reference area of ​​the current block, pixels and / or gradient information of at least one reference area as input to the neural network model, at least one prediction mode and / or the confidence of at least one prediction mode under the first quantity and / or the second quantity is determined or obtained, and based on the at least one prediction mode and / or the confidence of at least one prediction mode output by the neural network model, a first mode list is determined or obtained.

[0353] Optionally, based on the confidence of at least one prediction model determined or obtained by the neural network model, at least one candidate model is determined or obtained from at least one prediction model under the first quantity and / or second quantity output by the neural network model, and based on the at least one candidate model, a first model list is determined or obtained.

[0354] Optionally, confidence is a quantitative indicator of the degree of certainty of the model about the current prediction result.

[0355] In this embodiment, the first pattern list can provide candidate patterns with better adaptability for the current block, effectively narrowing the search range of candidate patterns, improving processing efficiency, and supporting improved prediction effects for the current block.

[0356] Third embodiment

[0357] Based on any of the above embodiments, a third embodiment is proposed.

[0358] In this embodiment, the gradient histogram and / or the area histogram is determined or obtained according to at least one of the following methods A1 to A6:

[0359] Method A1: gradient information of at least one reference area of ​​the current block.

[0360] Optionally, step S10 includes: determining or obtaining at least one gradient histogram corresponding to the current block based on gradient information of at least one reference area of ​​the current block, determining or obtaining at least one candidate mode based on the at least one gradient histogram, and determining or obtaining at least one prediction mode of the current block based on matching information related to the at least one candidate mode and the current block.

[0361] Optionally, the gradient information includes a gradient direction and / or a gradient magnitude. The gradient direction indicates the direction in which the grayscale value at a certain pixel point in the image (e.g., at least one reference area) changes fastest, that is, the direction perpendicular to the edge. The gradient magnitude indicates the intensity of the grayscale change at the pixel point, that is, the significance of the edge.

[0362] Optionally, the gradient information of the at least one reference region includes gradient information of pixels in the at least one reference region, and the gradient information of the pixels in the at least one reference region includes a quantized gradient direction.

[0363] Optionally, an initial gradient direction of pixels in at least one reference region is determined or obtained, and a quantized gradient direction of the pixels in the at least one reference region is determined or obtained based on an angle difference between the determined or obtained initial gradient direction and a gradient direction adjacent to the initial gradient direction in at least one lookup table.

[0364] Optionally, the angle difference may be information indicating an angle difference associated with a gradient direction.

[0365] Optionally, the initial gradient direction is compared with gradient directions adjacent to the initial gradient direction on the left and right sides in at least one lookup table. If an angular difference between the initial gradient direction and the left-adjacent gradient direction is smaller than an angular difference between the initial gradient direction and the right-adjacent gradient direction, the quantized gradient direction of the pixel is the left-adjacent gradient direction; and / or if the angular difference between the initial gradient direction and the left-adjacent gradient direction is larger than an angular difference between the initial gradient direction and the right-adjacent gradient direction, the quantized gradient direction of the pixel is the right-adjacent gradient direction.

[0366] Optionally, an initial gradient direction of the pixel is determined or obtained, initial gradient direction information related to the initial gradient direction of the pixel is determined or obtained, and quantized gradient direction information of the pixel is determined or obtained based on a difference between the determined or obtained initial gradient direction information and gradient direction information adjacent to the initial gradient direction information in at least one lookup table.

[0367] Optionally, the initial gradient direction information is compared with gradient direction information adjacent to the initial gradient direction information on the left and right sides in at least one lookup table. If a difference between the initial gradient direction information and the left-adjacent gradient direction information is smaller than a difference between the initial gradient direction information and the right-adjacent gradient direction information, the quantized gradient direction information of the pixel is the left-adjacent gradient direction information; and / or if a difference between the initial gradient direction information and the left-adjacent gradient direction information is larger than a difference between the initial gradient direction information and the right-adjacent gradient direction information, the quantized gradient direction information of the pixel is the right-adjacent gradient direction information.

[0368] Optionally, the gradient direction information is the tangent value of the gradient direction or the scaled tangent value.

[0369] Optionally, if the gradient direction x in the gradient information is between a first gradient direction and a second gradient direction in at least one lookup table, at least one gradient histogram is determined or obtained according to an angle difference or a difference between the gradient direction information x and the first gradient direction information and the second gradient direction information.

[0370] Optionally, if the gradient direction information x in the gradient information is between the first gradient direction information and the second gradient direction information in the at least one lookup table, the gradient direction X is decomposed into a gradient direction A and a gradient direction B based on an angle difference or a difference between the gradient direction information and the first gradient direction information and the second gradient direction information. At least one gradient histogram corresponding to the current block is determined or obtained based on the gradient direction A and the gradient direction B and the gradient magnitudes MA and MB.

[0371] Optionally, a first weight for gradient direction A is determined or obtained based on an angle difference or difference between gradient direction information x corresponding to gradient direction X and the first gradient direction information. A second weight for gradient direction B is determined or obtained based on an angle difference or difference between gradient direction information x corresponding to gradient direction X and the second gradient direction information. A gradient amplitude MA for gradient direction A is determined or obtained based on the first weight and a gradient amplitude M corresponding to gradient direction X. A gradient amplitude MB for gradient direction B is determined or obtained based on the second weight and the gradient amplitude M corresponding to gradient direction X. For example, the gradient amplitude MA is equal to the first weight*the gradient amplitude M, and the gradient amplitude MB is equal to the second weight*the gradient amplitude M.

[0372] Optionally, the gradient direction A is the first gradient direction described above, and the gradient direction B is the second gradient direction described above. Optionally, the first weight is equal to the difference between the gradient direction information x and the second gradient direction information, or the difference between the first gradient direction information and the second gradient direction information, and the second weight is equal to the difference between the gradient direction information x and the first gradient direction information, or the difference between the first gradient direction information and the second gradient direction information.

[0373] Optionally, if the gradient direction in the gradient information is between the first direction and the second direction in at least one lookup table, weight information corresponding to each of the first direction and the second direction is determined or obtained based on an angle difference between the gradient direction and the first direction and the second direction, and at least one gradient histogram is determined or obtained based on the gradient magnitude in the gradient information and the weight information corresponding to each of the first direction and the second direction.

[0374] Optionally, if the gradient direction in the gradient information is a, which is located between a first direction (for example, a1) and a second direction (for example, a2) in at least one lookup table, the weight information corresponding to a1 and a2 is determined or obtained based on the angle difference between a and a1 and a2, for example, a1 corresponds to weight w1, a2 corresponds to weight w2, and the gradient amplitude in the gradient information is b. Based on the gradient amplitude b and the weight information w1 and w2 corresponding to the first direction and the second direction, at least one gradient histogram is determined or obtained, and the gradient amplitude corresponding to the first direction a1 in the gradient histogram is b*w1, and the gradient amplitude corresponding to the second direction a2 is b*w2. Optionally, the weight w1 is equal to the angle difference between a and a2 or the angle difference between a1 and a2, and the weight w2 is equal to the angle difference between a and a1 or the angle difference between a1 and a2.

[0375] In this embodiment, when the gradient direction is between two prediction directions, the gradient amplitude is allocated according to the angular difference between the gradient direction and the prediction direction, thereby avoiding errors caused by rigid classification and improving the accuracy of angle prediction.

[0376] Optionally, the gradient information of the reference region includes gradient information of at least one pixel in the reference region and / or overall gradient information of the reference region.

[0377] Optionally, at least one gradient histogram is determined or obtained based on the overall gradient information and / or the gradient information of the pixels of at least one reference area of ​​the current block.

[0378] Optionally, the gradient information of the pixel is the gradient direction and / or gradient amplitude independently calculated for each pixel point in the reference area, reflecting the local change characteristics of the point.

[0379] Optionally, the overall gradient information of the reference region is obtained by statistically analyzing and / or aggregating the gradient information of all pixels in the reference region, and reflects the global change trend of the region.

[0380] Optionally, the pixel gradient information includes: a first pixel gradient component, a second pixel gradient component, a pixel gradient direction and / or a pixel gradient amplitude.

[0381] Optionally, gradient information of at least one pixel in the reference area is determined or obtained based on at least one pair of mutually perpendicular gradient operators, where the pair of mutually perpendicular gradient operators includes: a first gradient operator (e.g., a horizontal gradient operator Gx) and a second gradient operator (e.g., a vertical gradient operator Gy) corresponding to different directions, a value determined or obtained based on the first gradient operator is a first pixel gradient component (e.g., a value of Gx), and a value determined or obtained based on the second gradient operator is a second pixel gradient component (e.g., a value of Gy).

[0382] Optionally, the pixel gradient direction and / or the pixel gradient amplitude are determined or obtained according to the first pixel gradient component and the second pixel gradient component.

[0383] Optionally, the gradient information includes: a first gradient component (e.g., a first pixel gradient component and / or a first overall gradient component), a second gradient component (e.g., a second pixel gradient component and / or a second overall gradient component), a gradient direction (e.g., a pixel gradient direction and / or an overall gradient direction), and / or a gradient magnitude (e.g., a pixel gradient magnitude and / or an overall gradient magnitude).

[0384] Optionally, the overall gradient information includes: a first overall gradient component, a second overall gradient component, an overall gradient direction and / or an overall gradient magnitude.

[0385] Optionally, the method for determining or obtaining the overall gradient information of the reference area includes: determining or obtaining gradient information of at least one pixel in the reference area, the gradient information of at least one pixel includes: a first pixel gradient component and a second pixel gradient component, determining or obtaining the first overall gradient component based on the sum of the first pixel gradient components corresponding to the reference area, determining or obtaining the second overall gradient component based on the sum of the second pixel gradient components corresponding to the reference area, determining or obtaining the overall gradient amplitude based on the sum of the absolute values ​​of the first overall gradient component and the second overall gradient component, and determining or obtaining the overall gradient direction based on the first overall gradient component and the second overall gradient component.

[0386] Optionally, direct summation in the above manner can preserve the directional statistical characteristics, and the synthesized result can represent the overall gradient vector.

[0387] Optionally, determining or obtaining gradient information of at least one pixel in the reference area includes: a first pixel gradient component (Gx1, Gx2, ... Gx n ) and the second pixel gradient component (Gy1, Gy2, ... Gy n ), determine or obtain the first overall gradient component according to the sum of the first pixel gradient components corresponding to the reference area, that is, the first overall gradient component Gx sum =Gx1+Gx2+...+Gx n, determine or obtain the second overall gradient component, i.e., the second overall gradient component Gy, according to the sum of the second pixel gradient components corresponding to the reference region sum = Gy1 + Gy2 +... + Gy n , determine or obtain the overall gradient magnitude AMP according to the sum of the absolute values of the first overall gradient component and the second overall gradient component sum , that is, the overall gradient magnitude AMP sum = |Gx sum | + |Gy sum |, the overall gradient direction is calculated by arctan(Gx sum / Gy sum ) or arctan(Gy sum / Gx sum ).

[0388] Optionally, the method for determining or obtaining the overall gradient information of the reference region includes: determining or obtaining the gradient information of at least one pixel in the reference region, including: the first pixel gradient component, the second pixel gradient component, the pixel gradient direction, and the pixel gradient magnitude. According to the sum of the absolute values of the first pixel gradient component and the second pixel gradient component corresponding to the reference region, and / or, the sum of the absolute values of the first pixel gradient component and the second pixel gradient component corresponding to the reference region and the number of pixels with non-zero corresponding pixel gradient magnitudes in the pixel gradient direction of the reference region, determine or obtain the overall gradient magnitude. According to the sum of the first pixel gradient components corresponding to the reference region, determine or obtain the first overall gradient component. According to the sum of the second pixel gradient components corresponding to the reference region, determine or obtain the second overall gradient component. According to the first overall gradient component and the second overall gradient component, determine or obtain the overall gradient direction.

[0389] Optionally, determining or obtaining the gradient information of at least one pixel in the reference region includes: the first pixel gradient components (Gx1, Gx2,... Gxn) and the second pixel gradient components (Gy1, Gy2,... Gyn), the pixel gradient direction, and the pixel gradient magnitude. According to the sum of the absolute values of the first pixel gradient component and the second pixel gradient component corresponding to the reference region, and the number of pixels with non-zero corresponding pixel gradient magnitudes in the pixel gradient direction of the reference region, determine or obtain the overall gradient magnitude, that is, the overall gradient magnitude AMP sum = |Gx1| + |Gx2| +... + |Gx n | + |Gy1| + |Gy2| +... + |Gy n |, or, the overall gradient magnitude AMP sum = (|Gx1| + |Gx2| +... + |Gx n | + |Gy1| + |Gy2| +... + |Gy n丨) / n, where n is the number of non-zero pixel gradient magnitudes corresponding to the pixel gradient directions in the reference region. For example, if there are 10 directions in the pixel gradient directions of the reference region where the pixel gradient magnitudes are non-zero, then n is 10. Based on the sum of the first pixel gradient components corresponding to the reference region, the first overall gradient component is determined or obtained, that is, the first overall gradient component Gx sum = Gx1 + Gx2 +... + Gx n Based on the sum of the second pixel gradient components corresponding to the reference region, the second overall gradient component is determined or obtained, that is, the second overall gradient component Gy sum = Gy1 + Gy2 +... + Gy n The overall gradient direction is calculated by arctan(Gx sum / Gy sum ) or arctan(Gy sum / Gx sum ).

[0390] Optionally, by the above method, the average value of the absolute values can be obtained, which can resist direction interference and stably measure the gradient intensity.

[0391] Optionally, when determining or obtaining the overall gradient information of the reference region based on the sum of the absolute values of the first pixel gradient component and the second pixel gradient component corresponding to the reference region, if the overall gradient information is directly fused with the gradient information of the pixels corresponding to the reference region, then the overall gradient magnitude corresponding to the overall gradient information in the determined or obtained gradient histogram will be the maximum magnitude in the histogram, thus interfering with the selection of candidate patterns. To avoid the influence of the overall gradient magnitude information, the gradient histogram can be determined or obtained according to the overall gradient information, the weight information corresponding to the overall gradient magnitude, and the gradient information of at least one pixel of the reference region, so as to ensure fair competition of gradient information, balance the consistency requirements of intensity and direction, and / or determine the prediction pattern corresponding to the overall gradient direction in the overall gradient information as pattern 1, determine or obtain the gradient histogram according to the gradient information of at least one pixel of the reference region, determine or obtain at least one pattern 2 according to the gradient histogram, determine pattern 1 and at least one pattern 2 as candidate patterns. When at least one prediction pattern of the current block includes pattern 1 and at least one pattern 2, at least one prediction result 1 of the current block is determined or obtained according to pattern 1, at least one prediction result 2 of the current block is determined or obtained according to at least one pattern 2, and the target prediction result of the current block is determined or obtained according to at least one prediction result 1, at least one prediction result 2, and the weight information corresponding to at least one prediction result 1.

[0392] Optionally, the overall gradient information includes the overall gradient direction and the overall gradient magnitude. According to the gradient information of at least one pixel corresponding to the reference region, the gradient histogram is determined or obtained Figure 1 and the gradient histogram Figure 1 The gradient magnitude corresponding to the direction that is the same as the overall gradient direction is replaced by the overall gradient magnitude to determine or obtain the gradient histogram. Figure 2 , according to the gradient histogram Figure 2 , determine or obtain at least one candidate mode.

[0393] Optionally, the candidate modes include: a prediction mode corresponding to overall gradient information, a prediction mode determined or obtained according to a gradient histogram, and / or a non-angle prediction mode.

[0394] Optionally, in the case where the prediction mode determined or obtained according to the gradient histogram includes a prediction mode corresponding to the overall gradient information, weight information corresponding to at least one candidate mode is determined or obtained according to the weight information corresponding to the prediction mode corresponding to the overall gradient information, and the candidate modes include: the prediction mode corresponding to the overall gradient information, the prediction mode determined or obtained according to the gradient histogram and / or the non-angle prediction mode.

[0395] Optionally, when a candidate mode is determined to be the prediction mode of the current block, the corresponding weight information when the candidate mode is used as the prediction mode can be determined or obtained based on the weight information corresponding to the candidate mode.

[0396] Optionally, when a certain candidate mode is determined to be the prediction mode of the current block, the weight information corresponding to the candidate mode is determined as the weight information corresponding to the candidate mode as the prediction mode.

[0397] Optionally, the weight information corresponding to the candidate mode and the weight information corresponding to when the candidate mode is determined as the prediction mode may be the same as or different from.

[0398] Optionally, the prediction mode determined or obtained according to the gradient histogram determined or obtained based on the gradient information of at least one pixel in the reference area includes: mode a1, mode a2 and mode a3, and the weights of each of mode a1, mode a2 and mode a3 can be determined according to the sum of the corresponding gradient amplitudes in the gradient histogram. For example, according to the sum of the corresponding gradient amplitudes in the gradient histogram, the weight value of mode a1 is 0.4, the weight value of mode a2 is 0.3, and the weight value of mode a3 is 0.3, and mode a1 is the prediction mode corresponding to the overall gradient information. According to the weight value of mode a1 being 0.4, the weights of mode a2 and mode a3 are adjusted. For example, since the prediction mode corresponding to the overall gradient information is taken as the candidate mode by default, the weight value of mode a1 can be modified from the weight obtained according to the sum of the corresponding gradient amplitudes in the gradient histogram to a fixed weight corresponding to the overall gradient information. The value of the fixed weight is a preset value. The weights of mode a2 and mode a3 are determined according to the fixed weight and the sum of the gradient amplitudes of mode a2 and mode a3 in the gradient histogram. For example, if the fixed weight is 0.6, the sum of the gradient amplitudes of mode a2 is 50, and the sum of the gradient amplitudes of mode a3 is 50, then the weight of mode a2 is 0.2, and the weight of mode a3 is 0.2.

[0399] Optionally, the candidate modes include: a prediction mode determined or obtained according to a gradient histogram and a planar mode (planar mode), the prediction mode determined or obtained according to the gradient histogram includes a prediction mode corresponding to the overall gradient information. In this embodiment, the prediction mode determined or obtained according to the gradient histogram determined or obtained based on the gradient information of at least one pixel in the reference area includes: mode a1, mode a2, and mode a3. Mode a1 is the prediction mode corresponding to the overall gradient information. A fixed weight is used for the prediction mode corresponding to the overall gradient information, for example, the fixed weight is equal to 0.2, and the planar mode uses a fixed weight of 0.3. The weights of modes a2 and a3 are determined by the sum of the gradient amplitudes corresponding to modes a2 and a3 in the gradient histogram. If the sum of the gradient amplitudes of mode a2 is 20 and the sum of the gradient amplitudes of mode a3 is 30, the weight corresponding to mode a2 is 0.2 and the weight corresponding to mode a3 is 0.3. The weight of mode a2 can be determined according to formula (5);

[0400] W a2 =(1-W a1 -W planar )*(AMP a2 / (AMP a2 +AMP a3 )); Formula (5);

[0401] Wa2 is the weight of mode a2, W a1 is the weight of mode a1, AMP a2is the sum of the gradient amplitude values ​​of mode a2, AMP a3 is the sum of the gradient amplitude values ​​of mode a3.

[0402] The weight of mode a3 can be determined according to formula (6);

[0403] W a3 =(1-W a1 -W planar )*(AMP a3 / (AMP a2 +AMP a3 )); Formula (VI);

[0404] Wa2 is the weight of mode a2, W a1 is the weight of mode a1, AMP a2 is the sum of the gradient amplitude values ​​of mode a2, AMP a3 is the sum of the gradient amplitude values ​​of mode a3.

[0405] Optionally, the candidate modes include: a prediction mode determined or obtained according to a gradient histogram and a planar mode (planar mode). The prediction mode determined or obtained according to the gradient histogram does not include a prediction mode corresponding to the overall gradient information. In this embodiment, a fixed weight is used for the prediction mode corresponding to the overall gradient information, for example, the fixed weight is equal to 0.2, and the planar mode uses a fixed weight of 0.3. On this basis, the weights of mode a2 and mode a3 are determined by the sum of the gradient amplitudes corresponding to mode a2 and mode a3 in the gradient histogram. If the sum of the gradient amplitude values ​​of mode a2 is 20 and the sum of the gradient amplitude values ​​of mode a3 is 30, the weight corresponding to mode a2 is 0.2, and the weight corresponding to mode a3 is 0.3. Optionally, the weights corresponding to mode a2 and mode a3 can be determined or obtained by the above formula (5) and formula (6).

[0406] In this embodiment, by combining the gradient information of the pixel and / or the overall gradient information, the determined or obtained gradient histogram can take into account both local details and global structure, and then at least one candidate mode determined or obtained based on the gradient histogram can be consistent with the content of the current block, thereby improving the prediction accuracy of the prediction mode determined or obtained based on the candidate mode.

[0407] Method A2: at least one reference area and at least one lookup table of the current block.

[0408] Optionally, at least one lookup table includes: lookup table A and / or lookup table B, the number of prediction modes corresponding to lookup table A is a first number, the number of prediction modes corresponding to lookup table B is a second number, and the first number and the second number are the same or different.

[0409] Optionally, the first number is 35, 67 or 131, for example, the first number is 131.

[0410] Optionally, the first number is greater than 35, 67 or 131.

[0411] Optionally, the second number is 35, 67 or 131, for example, the second number is 67.

[0412] Optionally, the second number is less than or equal to 35, 67 or 131.

[0413] Optionally, the first number and the second number are the same, that is, the number of prediction modes corresponding to lookup table A is the same as the number of prediction modes corresponding to lookup table B. For example, lookup table A and lookup table B each correspond to the same number of prediction modes, but the types of the corresponding prediction modes are at least partially different.

[0414] Optionally, the prediction mode corresponding to the lookup table A and the prediction mode corresponding to the lookup table B are different, or the prediction mode corresponding to the lookup table A and the prediction mode corresponding to the lookup table B are at least partially the same.

[0415] Optionally, lookup table A may be a lookup table corresponding to 37 prediction modes or a lookup table corresponding to 131 prediction modes.

[0416] Optionally, the lookup table B may be a lookup table corresponding to 37 prediction modes or a lookup table corresponding to 131 prediction modes.

[0417] Optionally, step S10 includes: determining or obtaining at least one gradient histogram and / or at least one area histogram based on at least one reference area of ​​the current block and at least one lookup table, and determining or obtaining at least one candidate mode based on the at least one gradient histogram and / or at least one area histogram.

[0418] Optionally, the number of prediction modes corresponding to lookup table A is a first number, and at least one statistical histogram A is determined or obtained based on at least one reference area of ​​the current block and lookup table A, and the prediction mode corresponding to the statistical histogram A is the number of prediction modes corresponding to the lookup table A, that is, the first number.

[0419] Optionally, the statistical histogram includes: a gradient histogram and / or an area histogram.

[0420] Optionally, the number of prediction modes corresponding to lookup table B is a second number. Based on at least one reference area of ​​the current block and lookup table B, at least one statistical histogram B is determined or obtained, and the number of prediction modes corresponding to the statistical histogram B is the number of corresponding prediction modes in lookup table B, that is, the second number.

[0421] Optionally, at least one statistical histogram A is determined or obtained based on at least one reference area of ​​the current block and lookup table A, at least one statistical histogram B is determined or obtained based on at least one reference area of ​​the current block and lookup table B, and at least one candidate mode is determined or obtained based on at least one statistical histogram A and at least one statistical histogram B.

[0422] Optionally, at least one gradient histogram is determined or obtained based on gradient information of at least one reference region of the current block and at least one lookup table, where the gradient information includes pixel gradient information and / or overall gradient information, and the gradient information, pixel gradient information, and / or overall gradient information include at least one of a gradient magnitude and a gradient direction.

[0423] Optionally, at least one lookup table includes: lookup table A, the number of prediction modes corresponding to lookup table A is a first number, for example, 131, and at least one gradient histogram A is determined or obtained based on the gradient information of at least one reference area of ​​the current block and lookup table A, and the number of prediction modes corresponding to gradient histogram A is a first number.

[0424] Optionally, at least one lookup table includes: lookup table B, the number of prediction modes corresponding to lookup table B is the second number, for example, 67, based on the gradient information of at least one reference area of ​​the current block and lookup table B, at least one gradient histogram B is determined or obtained, and the number of prediction modes corresponding to gradient histogram B is the second number.

[0425] Optionally, at least one lookup table includes: lookup table A and lookup table B, the number of prediction modes corresponding to lookup table A is a first number, for example, 131, and the number of prediction modes corresponding to lookup table B is a second number, for example, 67. Based on the gradient information of at least one reference area of ​​the current block and lookup table A, at least one gradient histogram A is determined or obtained, and the number of prediction modes corresponding to gradient histogram A is the first number. Based on the gradient information of at least one reference area of ​​the current block and lookup table B, at least one gradient histogram B is determined or obtained, and the number of prediction modes corresponding to gradient histogram B is the second number.

[0426] Optionally, the processing method of the embodiment of the present application can be applied to the intra-frame prediction module of the codec, and the upstream / downstream modules of the intra-frame prediction module, for example, upstream modules such as reference pixel management and mode candidate list generation, and downstream modules such as residual transformation, entropy coding, and loop filtering, which need to adjust parameters according to the prediction mode, may have the same or different corresponding number of prediction modes as the intra-frame prediction module. When the number of prediction modes corresponding to the intra-frame prediction module is different from the number of prediction modes corresponding to its upstream and downstream modules, mode index remapping can be performed. For example, the indices of the 131 prediction modes in the intra-frame prediction module are mapped to 67 prediction modes. The remapping of the mode index can be achieved by table lookup or rule deduction to achieve compatibility. Optionally, some modes may trigger specific residual corrections, and the 131 modes can be converted into compatible correction parameters before mapping.

[0427] In this embodiment, based on at least one reference area of ​​the current block and at least one lookup table, a gradient histogram and / or area histogram corresponding to different prediction modes can be determined or obtained. The gradient histogram and / or area histogram can be used to quickly eliminate irrelevant prediction modes, improve prediction efficiency, and accurately capture the dominant prediction direction related to at least one reference area of ​​the current block, thereby matching the current block with a suitable candidate mode and / or prediction mode.

[0428] Mode A3: at least one lookup table corresponds to at least one gradient operator.

[0429] Optionally, step S10 includes: determining or obtaining at least one gradient histogram based on at least one reference area of ​​the current block and at least one gradient operator corresponding to at least one lookup table, and determining or obtaining at least one candidate mode based on the at least one gradient histogram.

[0430] Optionally, at least one lookup table includes: lookup table A and / or lookup table B, the number of prediction modes corresponding to lookup table A is a first number, the number of prediction modes corresponding to lookup table B is a second number, and the first number and the second number are the same or different.

[0431] Optionally, lookup table A corresponds to gradient operator A1 and gradient operator A2, lookup table B corresponds to gradient operator B1 and gradient operator B2, and gradient operators A1-A2 and gradient operators B1-B2 are different.

[0432] Optionally, the prediction mode corresponding to the lookup table A and the prediction mode corresponding to the lookup table B are different, or the prediction mode corresponding to the lookup table A and the prediction mode corresponding to the lookup table B are at least partially the same.

[0433] Optionally, lookup table A may be a lookup table corresponding to 37 prediction modes or a lookup table corresponding to 131 prediction modes.

[0434] Optionally, the lookup table B may be a lookup table corresponding to 37 prediction modes or a lookup table corresponding to 131 prediction modes.

[0435] Optionally, based on at least one reference area of ​​the current block and at least one gradient operator corresponding to at least one lookup table, gradient information of at least one reference area of ​​the current block is determined or obtained, and based on the gradient information of at least one reference area of ​​the current block, at least one gradient histogram is determined or obtained.

[0436] Optionally, the first number is greater than the second number, and the prediction direction corresponding to the prediction mode corresponding to lookup table A at least partially covers the prediction direction corresponding to the prediction mode corresponding to lookup table B. However, gradient operators need to match different directions to accurately calculate gradient information. Therefore, when the first number and the second number are different, gradient information of at least one reference area of ​​the current block can be determined or obtained using at least one gradient operator corresponding to lookup table A and / or at least one gradient operator corresponding to lookup table B, and at least one gradient histogram can be determined or obtained based on the gradient information of at least one reference area of ​​the current block.

[0437] Optionally, the first number is 131, and gradient information A of at least one reference area of ​​the current block is determined or obtained based on at least one gradient operator corresponding to the lookup table A. At least one gradient histogram A is determined or obtained based on the gradient information A of the at least one reference area, and the gradient histogram A includes gradient amplitude changes of the at least one reference area in prediction directions corresponding to the 131 prediction modes.

[0438] Optionally, the second number is 67, and gradient information B of at least one reference area of ​​the current block is determined or obtained based on at least one gradient operator corresponding to the lookup table B. At least one gradient histogram B is determined or obtained based on the gradient information B of the at least one reference area, and the gradient histogram B includes gradient amplitude changes of the at least one reference area in prediction directions corresponding to the 67 prediction modes.

[0439] Optionally, at least one candidate mode is determined or obtained based on at least one gradient histogram A and / or at least one gradient histogram B.

[0440] Optionally, the gradient operator is a tool and / or method for determining or obtaining at least one gradient information, and can extract pixel gradient information from an image through convolution operation or other mathematical operations, including gradient direction and / or gradient magnitude.

[0441] Optionally, the at least one gradient operator includes at least one pair of mutually perpendicular gradient operators.

[0442] Optionally, the mutually perpendicular gradient operators include: a first pair of mutually perpendicular gradient operators and / or a second pair of mutually perpendicular gradient operators, and the first pair of gradient operators and the second pair of gradient operators are different.

[0443] Optionally, first gradient information is determined or obtained based on a first pair of mutually perpendicular gradient operators, and at least one gradient histogram is determined or obtained based on the first gradient information.

[0444] Optionally, second gradient information is determined or obtained based on a second pair of mutually perpendicular gradient operators, and at least one gradient histogram is determined or obtained based on the second gradient information.

[0445] Optionally, the first pair of mutually perpendicular gradient operators include: a 0° gradient operator and a 90° gradient operator. The horizontal gradient Gx and the vertical gradient Gy can be calculated using a 3×3 horizontal (0°) Sobel operator and a vertical (90°) Sobel operator. For example, the horizontal (0°) gradient Gx and the vertical (90°) gradient Gy of a pixel x4 in a pixel line can be calculated according to the following formulas (1) and (2).

[0446]

[0447] Optionally, A can be a matrix consisting of 9 pixels centered on the pixel x4, and including the pixel x1 above, the pixel x3 on the left, the pixel x7 below, the pixel x5 on the right, the upper left pixel x0, the lower left pixel x6, the upper right pixel x2, and the lower right pixel x8, as shown in the following formula (3).

[0448]

[0449] Optionally, the G gradient amplitude value is the sum of the absolute values ​​of the horizontal gradient and the vertical gradient, and its calculation formula is shown in formula (4).

[0450] G=|Gx|+|Gy| Formula (4);

[0451] Optionally, the gradient direction corresponding to the pixel can be calculated by arctan(Gx / Gy) or arctan(Gy / Gx).

[0452] Optionally, the second pair of mutually perpendicular gradient operators include: a +45° gradient operator and a -45° gradient operator, and the +45° gradient G can be calculated using a Prewitt operator, a Scharr operator and / or a Sobel operator. +45° and -45° gradient G -45° , for example, for a pixel x4 in the pixel line G +45° Gradient and G -45° The gradient can be calculated using any pair of operators from the following formula (7) to formula (12).

[0453] Formula (7) and Formula (8) are Prewitt operators, which can directly respond to diagonal edges through the reference template.

[0454]

[0455] Formula (9) and Formula (10) are Sobel operators, in which the weight of the central pixel is 0 and increases and decreases symmetrically on both sides, which can enhance the diagonal edge response.

[0456]

[0457] Formula (11) and Formula (12) are Schall operators, and their diagonal gradient calculations are more accurate.

[0458]

[0459]

[0460] Optionally, applying only the gradient operators in the horizontal and vertical directions to determine the gradient histogram has a low prediction accuracy for the diagonal direction. Therefore, the embodiment of the present application introduces a ±45° gradient operator to effectively improve the prediction accuracy in the diagonal direction.

[0461] Optionally, at least one gradient histogram of the current block is determined or obtained based on at least one reference region of the current block and a first pair of mutually perpendicular gradient operators corresponding to at least one lookup table, where the at least one lookup table includes lookup table A and / or lookup table B, and the first pair of gradient operators includes a 0° (horizontal direction) gradient operator and a 90° (vertical direction) gradient operator.

[0462] Optionally, at least one piece of gradient information of the current block is determined or obtained based on at least one reference region of the current block and a first pair of mutually perpendicular gradient operators corresponding to at least one lookup table, and at least one gradient histogram is determined or obtained based on the at least one piece of gradient information, the at least one lookup table includes lookup table A and / or lookup table B, and the first pair of gradient operators includes a 0° (horizontal direction) gradient operator and a 90° (vertical direction) gradient operator.

[0463] Optionally, at least one piece of gradient information of the current block is determined or obtained based on at least one reference region of the current block and a second pair of mutually perpendicular gradient operators corresponding to at least one lookup table, and at least one gradient histogram of the current block is determined or obtained based on the at least one piece of gradient information, the at least one lookup table includes lookup table A and / or lookup table B, and the second pair of gradient operators includes a +45° gradient operator and a -45° gradient operator.

[0464] Optionally, the prediction direction a corresponding to arctan(Gx / Gy)) or arctan(Gy / Gx) is determined or obtained, and the prediction direction corresponding to arctan(Gx / Gy)) or arctan(Gy / Gx) is determined or obtained based on the angle difference between each prediction direction in at least one lookup table and the prediction direction a. For example, the direction with the smallest absolute angle difference is determined as the prediction direction corresponding to arctan(Gx / Gy)) or arctan(Gy / Gx).

[0465] Optionally, the gradient direction can be determined or obtained by a lookup table. For example, using a lookup table for arctan(a), the two indexes index1 and index2 closest to arctan(Gx / Gy)) or arctan(Gy / Gx) are determined, and the corresponding predicted direction of the pixel is determined based on the angular difference between arctan(Gx / Gy)) or arctan(Gy / Gx) and index1 and index2.

[0466] Optionally, if the gradient direction corresponding to arctan(Gx / Gy) or arctan(Gy / Gx) determined or obtained based on the gradient information is between a first direction and a second direction in at least one lookup table, at least one gradient histogram is determined or obtained based on an angle difference between the gradient direction corresponding to arctan(Gx / Gy) or arctan(Gy / Gx) and the first direction and the second direction.

[0467] Optionally, if the gradient direction corresponding to arctan(Gx / Gy) or arctan(Gy / Gx) determined or obtained based on the gradient information is between the first direction and the second direction in at least one lookup table, then based on the angle difference between the gradient direction corresponding to arctan(Gx / Gy)) or arctan(Gy / Gx) and the first direction and the second direction, weight information corresponding to each of the first direction and the second direction is determined or obtained, and based on the weight information corresponding to each of the first direction and the second direction, at least one gradient histogram is determined or obtained.

[0468] Optionally, if the gradient direction corresponding to arctan(Gx / Gy) or arctan(Gy / Gx) determined or obtained based on the gradient information is between the first direction and the second direction in at least one lookup table, then weight information corresponding to each of the first and second directions is determined or obtained based on the angle difference between the gradient direction corresponding to arctan(Gx / Gy)) or arctan(Gy / Gx) and the first and second directions, the gradient magnitude is determined or obtained based on the gradient information, and at least one gradient histogram is determined or obtained based on the gradient magnitude and the weight information corresponding to each of the first and second directions.

[0469] Optionally, if the gradient direction corresponding to arctan(Gx / Gy) or arctan(Gy / Gx) is determined or obtained based on the gradient information, and is located between the first direction (for example, a1) and the second direction (for example, a2) in at least one lookup table, the weight information corresponding to a1 and a2 is determined or obtained based on the angle difference between a and a1 and a2, for example, a1 corresponds to weight w1, a2 corresponds to weight w2, the gradient amplitude b is determined or obtained based on the gradient information, and at least one gradient histogram is determined or obtained based on the gradient amplitude b and the weight information w1 and w2 corresponding to the first direction and the second direction, the gradient amplitude corresponding to the first direction a1 in the gradient histogram is b*w1, and the gradient amplitude corresponding to the second direction a2 is b*w2.

[0470] In this embodiment, only the gradient operators in the horizontal and vertical directions are applied to determine the prediction accuracy of the gradient histogram for the diagonal direction is low. Therefore, different directions, for example, gradient operators of ±45°, are introduced through the embodiments of the present application to effectively improve the prediction accuracy in the diagonal direction.

[0471] Method A4: weight information corresponding to at least one pair of gradient operators.

[0472] Optionally, step S10 includes: determining or obtaining at least one gradient histogram based on at least one reference area of ​​the current block and weight information of at least one pair of gradient operators, determining or obtaining at least one candidate mode based on the at least one gradient histogram, and determining or obtaining at least one prediction mode of the current block based on matching information related to the at least one candidate mode and the current block.

[0473] Optionally, at least one gradient histogram is determined or obtained based on at least one reference area of ​​the current block, at least one pair of gradient operators corresponding to at least one lookup table, and weight information corresponding to at least one pair of gradient operators, and at least one candidate mode is determined or obtained based on the at least one gradient histogram.

[0474] Optionally, based on at least one reference area of ​​the current block and weight information corresponding to at least one pair of gradient operators, gradient information of at least one reference area of ​​the current block is determined or obtained, based on the at least one gradient information, at least one gradient histogram is determined or obtained, and based on the at least one gradient histogram, at least one candidate mode is determined or obtained.

[0475] Optionally, based on at least one reference area of ​​the current block, at least one pair of gradient operators corresponding to at least one lookup table, and weight information corresponding to the at least one pair of gradient operators, gradient information of at least one reference area of ​​the current block is determined or obtained; based on the at least one gradient information, at least one gradient histogram is determined or obtained; and based on the at least one gradient histogram, at least one candidate mode is determined or obtained.

[0476] Optionally, at least one lookup table includes: lookup table A and / or lookup table B, the number of prediction modes corresponding to lookup table A is a first number, the number of prediction modes corresponding to lookup table B is a second number, and the first number and the second number are the same or different.

[0477] Optionally, the first number is 35, 67 or 131, for example, the first number is 131.

[0478] Optionally, the first number is greater than 35, 67 or 131.

[0479] Optionally, the second number is 35, 67 or 131, for example, the second number is 67.

[0480] Optionally, the second number is less than or equal to 35, 67 or 131.

[0481] Optionally, the first number and the second number are the same, that is, the number of prediction modes corresponding to lookup table A is the same as the number of prediction modes corresponding to lookup table B. For example, lookup table A and lookup table B each correspond to the same number of prediction modes, but the types of the corresponding prediction modes are at least partially different.

[0482] Optionally, the prediction mode corresponding to the lookup table A and the prediction mode corresponding to the lookup table B are different, or the prediction mode corresponding to the lookup table A and the prediction mode corresponding to the lookup table B are at least partially the same.

[0483] Optionally, lookup table A may be a lookup table corresponding to 37 prediction modes or a lookup table corresponding to 131 prediction modes.

[0484] Optionally, the lookup table B may be a lookup table corresponding to 37 prediction modes or a lookup table corresponding to 131 prediction modes.

[0485] Optionally, the mutually perpendicular gradient operators include: a first pair of mutually perpendicular gradient operators and / or a second pair of mutually perpendicular gradient operators, where the first pair of gradient operators and the second pair of gradient operators are different. For example, the first pair of gradient operators includes a 0° (horizontal direction) gradient operator and a 90° (vertical direction) gradient operator, and the second pair of gradient operators includes a +45° gradient operator and a -45° gradient operator.

[0486] Optionally, first gradient information is determined or obtained based on a first pair of mutually perpendicular gradient operators, and at least one gradient histogram is determined or obtained based on the first gradient information.

[0487] Optionally, second gradient information is determined or obtained based on a second pair of mutually perpendicular gradient operators, and at least one gradient histogram is determined or obtained based on the second gradient information.

[0488] Optionally, the first pair of gradient operators and the second pair of gradient operators are different, third gradient information is determined or obtained based on the first gradient information and the second gradient information, and at least one gradient histogram is determined or obtained based on the third gradient information.

[0489] Optionally, third gradient information is determined or obtained based on the first gradient information, weight information corresponding to the first pair of gradient operators, the second gradient information, and weight information corresponding to the second pair of gradient operators, and at least one gradient histogram is determined or obtained based on the third gradient information.

[0490] Optionally, the third gradient information may be gradient information determined or obtained by fusing the first gradient information and the second gradient information through weight information corresponding to at least one pair of gradient operators, for example, including gradient direction and / or gradient magnitude.

[0491] Optionally, gradient information of at least one reference region of the current block is determined or obtained based on mutually perpendicular gradient operators corresponding to at least one lookup table; at least one gradient histogram corresponding to the current block is determined or obtained based on the gradient information of the at least one reference region of the current block; and at least one candidate mode is determined or obtained based on the at least one gradient histogram. The at least one lookup table includes lookup table A and / or lookup table B, and the mutually perpendicular gradient operators include a first pair of mutually perpendicular gradient operators and / or a second pair of mutually perpendicular gradient operators, where the first pair of gradient operators and the second pair of gradient operators are different.

[0492] Optionally, first gradient information of the at least one reference area of ​​the current block is determined or obtained based on the at least one reference area of ​​the current block, the at least one lookup table, and a first pair of mutually perpendicular gradient operators. Second gradient information of the at least one reference area of ​​the current block is determined or obtained based on the at least one reference area of ​​the current block, the at least one lookup table, and a second pair of mutually perpendicular gradient operators. At least one gradient histogram is determined or obtained based on the first gradient information of the at least one reference area, the second gradient information of the at least one reference area, weight information corresponding to the first pair of mutually perpendicular gradient operators, and weight information corresponding to the second pair of mutually perpendicular gradient operators.

[0493] Optionally, first gradient information of at least one reference region of the current block is determined or obtained based on a first pair of mutually perpendicular gradient operators corresponding to at least one lookup table; at least one gradient histogram C corresponding to the current block is determined or obtained based on the first gradient information of the at least one reference region of the current block; and at least one candidate mode is determined or obtained based on the at least one gradient histogram C.

[0494] Optionally, second gradient information of at least one reference area of ​​the current block is determined or obtained based on a second pair of mutually perpendicular gradient operators corresponding to at least one lookup table, at least one gradient histogram D corresponding to the current block is determined or obtained based on the second gradient information of the at least one reference area of ​​the current block, and at least one candidate mode is determined or obtained based on the at least one gradient histogram D.

[0495] Optionally, based on at least one gradient histogram C and at least one gradient histogram D, at least one third gradient information of at least one reference area of ​​the current block is determined or obtained, based on the at least one third gradient information, at least one gradient histogram E is determined or obtained, and based on the at least one gradient histogram E, at least one candidate mode is determined or obtained.

[0496] Optionally, at least one candidate mode is determined or obtained based on at least one gradient histogram C and at least one gradient histogram D.

[0497] Optionally, the step of determining or obtaining at least one candidate mode based on at least one gradient histogram C and at least one gradient histogram D includes: fusing at least one gradient histogram C and at least one gradient histogram D based on weight information corresponding to at least one pair of gradient operators, determining or obtaining at least one third gradient information of at least one reference area of ​​the current block, determining or obtaining at least one gradient histogram E based on the at least one third gradient information, and determining or obtaining at least one candidate mode based on the at least one gradient histogram E. Optionally, the number of prediction modes corresponding to the fused gradient histogram C and gradient histogram D is the same.

[0498] Optionally, based on the weight information corresponding to at least one pair of gradient operators, at least one gradient histogram C and at least one gradient histogram D are fused to determine or obtain at least one gradient histogram E, and at least one candidate mode is determined or obtained based on at least one gradient histogram E. The weight information corresponding to the gradient operator includes: first weight information, the first weight information is the weight information corresponding to the gradient amplitude of each gradient direction in the gradient histogram, including: weight information corresponding to the gradient amplitude of at least one gradient histogram C and / or weight information corresponding to the gradient amplitude of at least one gradient histogram D. Optionally, the gradient histogram C and the gradient histogram D are different.

[0499] Optionally, the first weight information includes a weight value corresponding to the gradient amplitude of each prediction direction (i.e., a direction perpendicular to the gradient direction) in the gradient histogram, and the size of the weight value corresponding to the gradient amplitude of each prediction direction of the gradient histogram is inversely proportional and / or negatively correlated with the angle difference between the prediction direction and the direction of the gradient operator corresponding to the gradient histogram. For example, if the directions of the gradient operator corresponding to the gradient histogram are 0° and 90°, then, among the prediction directions of the gradient histogram, the smaller the angle difference between the prediction direction and the horizontal or vertical direction, the larger the corresponding weight value; and / or, the larger the angle difference between the prediction direction and the horizontal or vertical direction, the smaller the weight value.

[0500] Optionally, the first pair of mutually perpendicular gradient operators includes a 0° gradient operator and a 90° gradient operator, the weight value corresponding to the gradient amplitude of mode 3 in the gradient histogram C is 0.9, and the gradient amplitude is C1, and the weight value corresponding to the gradient amplitude of mode 3 in the gradient histogram D is 0.1, and the gradient amplitude is D1. Then, the gradient histogram C and the gradient histogram D are fused to determine or obtain third gradient information and / or gradient histogram E, the gradient direction of the third gradient information is the gradient direction corresponding to mode 3, the gradient amplitude is C1*0.9+D1*0.1, and / or the gradient amplitude value corresponding to mode 3 in the gradient histogram E is C1*0.9+D1*0.1.

[0501] Optionally, since the convolution response is maximized and the detection is most accurate when the operator design direction (i.e., the first direction and / or second direction mentioned above) is consistent with the predicted edge direction, the setting of the first weight information mentioned above can effectively improve the accuracy of the obtained prediction model.

[0502] Optionally, the first pair of mutually perpendicular gradient operators include a 0° gradient operator and a 90° gradient operator, and the second pair of mutually perpendicular gradient operators include a +45° gradient operator and a -45° gradient operator. A gradient histogram C is determined or obtained based on the first pair of mutually perpendicular gradient operators, and a gradient histogram D is determined or obtained based on the second pair of mutually perpendicular gradient operators. Since the gradient histogram C has higher accuracy for horizontal and vertical edges, the values ​​of the weight information corresponding to the gradient amplitudes in the horizontal and vertical directions in the gradient histogram C are greater than the values ​​of the weight information corresponding to the gradient amplitudes in the horizontal and vertical directions in the gradient histogram D. The gradient histogram D has higher accuracy for diagonal edges, and therefore, the values ​​of the weight information corresponding to the gradient amplitudes in the diagonal direction in the gradient histogram D are greater than the values ​​of the weight information corresponding to the gradient amplitudes in the diagonal direction in the gradient histogram C.

[0503] Optionally, the first weight information corresponding to the gradient amplitude in the prediction direction of the gradient histogram C is as shown in Table 1 below:

[0504] Table 1

[0505] Prediction direction w12 w13 w14 ... w111 w112 ... w117 w118 Value 1 0.9 0.8 ... 0.1 0 ... 0 0 Prediction direction w119 ... w124 ... w130 w131 w132 w133 w134 Value 0 ... 0 ... 0.6 0.7 0.8 0.9 1 Prediction direction w135 w136 w137 .... w142 w143 ... w149 w150 Value 0.9 0.8 0.7 0.2 0.1 ... 0 0 Prediction direction w151 w152 ... w157 ... w163 w164 w165 w166 Value 0 0 0.1 ... 0.7 0.8 0.9 1

[0506] Optionally, the first weight information corresponding to the gradient amplitude in the prediction direction of the gradient histogram D is as shown in Table 2 below:

[0507] Table 2

[0508] Prediction direction w22 w23 w24 ... w211 w212 ... w217 w218 Value 0 0.1 0.2 ... 0.9 1 ... 1 1 Prediction direction w219 ... w224 ... w230 w231 w232 w233 w234 Value 1 ... 1 ... 0.4 0.3 0.2 0.1 0 Prediction direction w235 w236 w237 .... w242 w243 ... w249 w250 Value 0.1 0.2 0.3 0.8 0.9 ... 1 1 Prediction direction w251 w252 ... w257 ... w263 w264 w265 w266 Value 1 1 0.9 ... 0.3 0.2 0.1 0

[0509] Optionally, a first pair of mutually perpendicular gradient operators includes a 0° gradient operator and a 90° gradient operator, and a second pair of mutually perpendicular gradient operators includes a +45° gradient operator and a -45° gradient operator. A gradient histogram C is determined or obtained based on the first pair of mutually perpendicular gradient operators, and a gradient histogram D is determined or obtained based on the second pair of mutually perpendicular gradient operators. The weight value corresponding to the gradient magnitude in the predicted direction of the gradient histogram D, where the angular difference between the predicted direction and the horizontal or vertical direction is less than a first angular difference threshold, is 0, and the weight value corresponding to the gradient magnitude in the predicted direction of the gradient histogram D, where the angular difference between the predicted direction and the angular difference between 45°, 135°, and -45° is less than a second angular difference threshold, is 1. Optionally, the first angular difference threshold and the second angular difference threshold are different.

[0510] Optionally, only applying the gradient operators in the horizontal and vertical directions to determine the gradient histogram has a low prediction accuracy for the diagonal direction. Therefore, the embodiment of the present application introduces a ±45° gradient operator to effectively improve the prediction accuracy in the diagonal direction. Since horizontal (such as the horizon, buildings) and vertical (such as trees, people) structures dominate in natural scenes, and oblique textures (such as 45° diagonals) are relatively rare, for prediction directions close to 0° and 90°, the gradient operators corresponding to the 0° gradient operator and the 90° gradient operator have higher accuracy. Therefore, by setting the above weight values, the prediction accuracy of the obtained prediction model can be effectively improved.

[0511] Optionally, the mutually perpendicular gradient operators include: a first pair of mutually perpendicular gradient operators and / or a second pair of mutually perpendicular gradient operators, and the first pair of gradient operators and the second pair of gradient operators are different.

[0512] Optionally, the first pair of mutually perpendicular gradient operators include a 0° gradient operator and a 90° gradient operator, and the second pair of mutually perpendicular gradient operators include a +45° gradient operator and a -45° gradient operator. At least one first gradient information of the at least one reference area of ​​the current block is determined or obtained based on the at least one reference area of ​​the current block and the first pair of mutually perpendicular gradient operators. At least one second gradient information of the at least one reference area of ​​the current block is determined or obtained based on the at least one reference area of ​​the current block and the second pair of mutually perpendicular gradient operators. The second gradient information includes: a gradient component G corresponding to +45°; +45°, the gradient component G corresponding to -45° -45° , for the gradient component G +45° and the gradient component G -45° Perform projection to determine or obtain the gradient component D in the direction of the 0° gradient operator and the 90° gradient operator x1 and D y1 , D x1 and D y1 Calculated according to the following formula (13) and formula (14):

[0513]

[0514] Optionally, the first gradient information includes the gradient component D corresponding to 0° x2 , the gradient component D corresponding to 90° y2 , according to the weight information corresponding to the first pair of mutually perpendicular gradient operators, the weight information corresponding to the second pair of mutually perpendicular gradient operators, and the gradient component D x1 、D y1 , gradient component D x2 and D y2 , determine or obtain at least one fused gradient information, the fused gradient information including: a fused gradient component corresponding to 0° and a fused gradient component corresponding to 90°, and determine or obtain at least one gradient histogram based on the at least one fused gradient information.

[0515] Optionally, the target gradient component Gx corresponding to 0° and the target gradient component Gy corresponding to 90° are calculated according to the following formula (15) and formula (16):

[0516] Gx=w a *D x1 +w b *D x2 Formula (XV);

[0517] Gy=w a *D y1 +w b *D y2 Formula (XVI);

[0518] Optionally, w a is the weight information corresponding to the second pair of mutually perpendicular gradient operators, w b It is weight information corresponding to a first pair of mutually perpendicular gradient operators. Optionally, the weight corresponding to the above weight information is a preset fixed value, and / or the above weight value can be adaptively adjusted according to different scenarios.

[0519] Optionally, the weight information corresponding to the gradient operators includes: weight information corresponding to a first pair of mutually perpendicular gradient operators and / or weight information corresponding to a second pair of mutually perpendicular gradient operators.

[0520] Optionally, the fused gradient information of each pixel can be calculated in the above manner, so that the distribution of the fused gradient component of the local area of ​​the reference area can be determined, and at least part of the pixels in the reference area can be enhanced according to the distribution of the fused gradient component of the local area of ​​the reference area.

[0521] Optionally, weight information corresponding to the gradient information of at least one reference region is determined or obtained according to the fused gradient information of at least one reference region.

[0522] In this embodiment, by fusing the gradient operators in the horizontal, vertical and diagonal directions, all dominant edge directions in the current block can be accurately detected, avoiding the blind spot of a single direction operator, thereby effectively improving the prediction accuracy of the current block.

[0523] Mode A5: weight information corresponding to gradient information of at least one reference region of the current block.

[0524] Optionally, step S10 includes: determining or obtaining at least one gradient histogram based on weight information corresponding to gradient information of at least one reference area of ​​the current block, determining or obtaining at least one candidate mode based on the at least one gradient histogram, and determining or obtaining at least one prediction mode of the current block based on matching information related to the at least one candidate mode and the current block.

[0525] Optionally, at least one gradient histogram is determined or obtained according to gradient information of at least one reference region of the current block and weight information corresponding to the gradient information.

[0526] Optionally, at least one gradient histogram is determined or obtained according to weight information corresponding to gradient information of at least one reference region of the current block and at least one lookup table.

[0527] Optionally, at least one gradient histogram is determined or obtained according to weight information corresponding to gradient information of at least one reference region of the current block and at least one gradient operator corresponding to at least one lookup table.

[0528] Optionally, at least one gradient histogram is determined or obtained according to weight information corresponding to gradient information of at least one reference region of the current block and weight information corresponding to at least one pair of gradient operators.

[0529] Optionally, gradient information of at least one reference area of ​​the current block is determined or obtained based on at least one reference area of ​​the current block and at least one gradient operator corresponding to at least one lookup table, and at least one gradient histogram is determined or obtained based on the gradient information of at least one reference area of ​​the current block and weight information corresponding to the gradient information.

[0530] Optionally, gradient information of at least one reference region of the current block is determined or obtained based on at least one reference region of the current block, at least one gradient operator corresponding to at least one lookup table, and weight information corresponding to at least one pair of gradient operators; and at least one gradient histogram is determined or obtained based on the gradient information of at least one reference region of the current block and weight information corresponding to the gradient information.

[0531] Optionally, the weight information corresponding to the gradient information includes: second weight information and / or third weight information, the second weight information is weight information based on spatial distance, and the third weight information is weight based on directional consistency.

[0532] Optionally, the second weight information (i.e., the weight based on spatial distance) of a pixel in at least one reference area of ​​the current block is inversely proportional and / or negatively correlated with the distance between the pixel and the center of the current block, i.e., the farther the distance between the pixel and the center of the current block, the smaller the weight value of the second weight information of the pixel, and the closer the distance between the pixel and the center of the current block, the larger the weight value of the second weight information of the pixel. The above-mentioned weight based on spatial distance can suppress the interference of edge pixels and enhance the dominance of the central area.

[0533] Optionally, the weight information corresponding to the gradient information includes: second weight information, the gradient information of at least one reference area of ​​the current block includes: gradient direction 1 of pixel a, gradient amplitude x of pixel a, gradient direction 2 of pixel b, and gradient amplitude y of pixel a. Since the distance between pixel a and the center of the current block is less than the distance between pixel b and the center of the current block, the second weight information w1 of pixel a is greater than the second weight information w2 of pixel b. Based on the gradient information of at least one reference area of ​​the current block and the weight information corresponding to the gradient information of at least one reference area of ​​the current block, the gradient amplitude of pixel a in gradient direction 1 is x*w1, and the gradient amplitude of pixel b in gradient direction 2 is y*w2. Based on the gradient amplitude x*w1 of pixel a in gradient direction 1 and the gradient amplitude y*w2 of pixel b in gradient direction 2, at least one gradient histogram is determined or obtained.

[0534] Optionally, the third weight information (i.e., the weight based on directional consistency) of a pixel in at least one reference area of ​​the current block is positively proportional and / or positively correlated with the gradient directional consistency between the pixel and at least one pixel in its adjacent area. For example, the stronger the consistency between the gradient direction of pixel A in the reference area and the gradient direction of at least one pixel in the adjacent area of ​​pixel A, the greater the weight value of the third weight information; and / or, the weaker the consistency between the gradient direction of pixel A in the reference area and the gradient direction of at least one pixel in the adjacent area of ​​pixel A, the smaller the weight value of the third weight information. The above-mentioned weight based on directional consistency can enhance the statistical weight of the locally consistent texture direction and improve the significance of the dominant edge pattern.

[0535] Optionally, the smaller the angular difference between the gradient direction of a pixel and the gradient direction of at least one pixel in its adjacent area, the stronger the gradient direction consistency; and / or, the larger the angular difference between the gradient direction of a pixel and the gradient direction of at least one pixel in its adjacent area, the weaker the gradient direction consistency.

[0536] Optionally, third weight information of pixels in at least one reference area of ​​the current block is determined or obtained according to fused gradient information of at least one pixel in at least one reference area of ​​the current block.

[0537] Optionally, the weight information corresponding to the gradient information includes: third weight information, the gradient information of at least one reference area of ​​the current block includes: gradient direction 1 of pixel a, gradient amplitude x of pixel a, gradient direction 2 of pixel b, and gradient amplitude y of pixel a. Since the gradient direction consistency between pixel a and at least one pixel in its adjacent area is stronger than the gradient direction consistency between pixel b and at least one pixel in its adjacent area, the third weight information w3 of pixel a is greater than the third weight information w4 of pixel b. Based on the gradient information of at least one reference area of ​​the current block and the weight information corresponding to the gradient information of at least one reference area of ​​the current block, the gradient amplitude of pixel a in gradient direction 1 is x*w3, and the gradient amplitude of pixel b in gradient direction 2 is y*w4. Based on the gradient amplitude x*w3 of pixel a in gradient direction 1 and the gradient amplitude y*w4 of pixel b in gradient direction 2, at least one gradient histogram is determined or obtained.

[0538] Optionally, the weight information corresponding to the gradient information includes: second weight information and third weight information, and the gradient information of at least one reference area of ​​the current block includes: gradient direction 1 of pixel a, gradient amplitude x of pixel a, gradient direction 2 of pixel b, and gradient amplitude y of pixel a. Since the angle difference between pixel a and the center of the current block is smaller than the angle difference between pixel b and the center of the current block, the second weight information w1 of pixel a is greater than the second weight information w2 of pixel b. Since the gradient direction consistency between pixel a and at least one pixel in the adjacent area is stronger than that between pixel b and at least one pixel in the adjacent area, the gradient direction consistency between pixel a and at least one pixel in the adjacent area is stronger than that between pixel b and at least one pixel in the adjacent area. The gradient direction consistency between pixels is obtained. Therefore, the third weight information w3 of pixel a is greater than the third weight information w4 of pixel b. Based on the gradient information of at least one reference area of ​​the current block and the weight information corresponding to the gradient information of at least one reference area of ​​the current block, the gradient amplitude of pixel a in gradient direction 1 is x*w1*w3, and the gradient amplitude of pixel b in gradient direction 2 is y*w2*w4. Based on the gradient amplitude x*w1*w3 of pixel a in gradient direction 1 and the gradient amplitude y*w2*w4 of pixel b in gradient direction 2, at least one gradient histogram is determined or obtained.

[0539] In this embodiment, at least one gradient histogram can be determined or obtained based on the second weight information (i.e., the weight based on spatial distance) and / or the third weight information (i.e., the weight based on directional consistency). The weight based on spatial distance can suppress the interference of edge pixels and enhance the dominance of the central area. The weight based on directional consistency can enhance the statistical weight of the local consistent texture direction and improve the significance of the dominant edge pattern, thereby improving the accuracy of the candidate pattern and / or prediction pattern determined or obtained by at least one gradient histogram and improving the prediction accuracy of the prediction pattern for the current block.

[0540] Method A6: area information of at least one reference region of the current block.

[0541] Optionally, step S10 includes: determining or obtaining at least one area histogram based on area information of at least one reference area of ​​the current block, determining or obtaining at least one candidate mode based on at least one area histogram, and determining or obtaining at least one prediction mode of the current block based on matching information related to at least one candidate mode and the current block.

[0542] Optionally, at least one area histogram is determined or obtained based on area information of at least one reference area of ​​the current block, and at least one candidate mode is determined or obtained based on the at least one area histogram and at least one gradient histogram.

[0543] Optionally, based on the usage of the intra-frame prediction mode of at least one reference area of ​​the current block (including at least one adjacent coding block and / or non-adjacent coding block of the current block), the area information of at least one reference area is determined or obtained, and the area information includes: an area amplitude value and an intra-frame prediction direction. Based on the area information of at least one reference area, at least one area histogram is determined or obtained.

[0544] Optionally, at least one reference area includes at least one adjacent coding block and / or non-adjacent coding block of the current block, and the area information of at least one reference area includes: the area amplitude value and intra-frame prediction direction of at least one adjacent coding block, and / or the area amplitude value and intra-frame prediction direction of at least one non-adjacent coding block.

[0545] Reference Figure 10 , the reference area of ​​the block to be predicted (i.e., the current block) in the coded area includes: coding block 4 and coding block 6 adjacent to the block to be predicted, and coding block 1, coding block 2, coding block 3, coding block 5, coding block 7, and coding block 8 not adjacent to the block to be predicted, as shown in Figure 11 As shown, in the decoded area, there are decoding blocks 4 and 6 adjacent to the block to be predicted, and decoding blocks 1, 2, 3, 5, 7, and 8 that are not adjacent to the block to be predicted. At least one area histogram is determined or obtained based on the area amplitude value and intra-frame prediction direction of the at least one encoded block and / or at least one decoded block.

[0546] Optionally, the number of prediction modes corresponding to the area histogram is the first number or the second number.

[0547] Optionally, at least one area histogram is determined or obtained according to at least one lookup table and area information of at least one reference region of the current block.

[0548] Optionally, at least one lookup table includes: lookup table A and / or lookup table B, the number of prediction modes corresponding to lookup table A is a first number, the number of prediction modes corresponding to lookup table B is a second number, and the first number and the second number are the same or different.

[0549] Optionally, the first number is 35, 67 or 131, for example, the first number is 131.

[0550] Optionally, the first number is greater than 35, 67 or 131.

[0551] Optionally, the second number is 35, 67 or 131, for example, the second number is 67.

[0552] Optionally, the second number is less than or equal to 35, 67 or 131.

[0553] Optionally, the first number and the second number are the same, that is, the number of prediction modes corresponding to lookup table A is the same as the number of prediction modes corresponding to lookup table B. For example, lookup table A and lookup table B each correspond to the same number of prediction modes, but the types of the corresponding prediction modes are at least partially different.

[0554] Optionally, if the prediction mode corresponding to at least one adjacent coding block and / or non-adjacent coding block of the current block is determined or obtained according to lookup table A, and the number of prediction modes corresponding to lookup table A is a first number, then the number of prediction modes corresponding to at least one area histogram determined or obtained based on the area information of at least one reference area of ​​the current block is the first number, for example, the first number is 131.

[0555] Optionally, if the prediction mode corresponding to at least one adjacent coding block and / or non-adjacent coding block of the current block is determined or obtained according to lookup table B, and the number of prediction modes corresponding to lookup table B is the second number, then the number of prediction modes corresponding to at least one area histogram determined or obtained based on the area information of at least one reference area of ​​the current block is the second number, for example, the second number is 67.

[0556] In this embodiment, the area and direction information of the adjacent and / or non-adjacent blocks of the current block are used to quickly infer the possible edge structure of the current block, and an area histogram is established. The candidate modes are then directly screened through the area histogram, for example, high-frequency prediction directions are retained, thereby improving the matching accuracy of the prediction mode and improving the prediction accuracy for the current block.

[0557] Optionally, the method of determining or obtaining at least one candidate pattern according to at least one statistical histogram includes the following method a1 and / or method a2:

[0558] Mode a1: determining or obtaining at least one candidate mode according to a first statistical histogram of the current block under a first number of prediction modes and / or a second statistical histogram of the current block under a second number of prediction modes.

[0559] Optionally, the statistical histogram includes: at least one of a gradient histogram, an area histogram, and a histogram of the number of times a prediction mode is used corresponding to an encoded image block, and a histogram obtained by adjusting or merging the above histograms.

[0560] Optionally, step S10 includes: determining or obtaining at least one candidate mode based on a first statistical histogram of the current block under a first number of prediction modes and / or a second statistical histogram of the current block under a second number of prediction modes, and determining or obtaining at least one prediction mode of the current block based on matching information related to the at least one candidate mode and the current block.

[0561] Optionally, the first statistical histogram and the second statistical histogram belong to the same or different histogram types. For example, the first statistical histogram is a gradient histogram, and the second statistical histogram is an area histogram.

[0562] Optionally, at least one candidate mode is determined or obtained according to a first statistical histogram of the current block under a first number of prediction modes, and the obtained candidate mode is one of the first number of prediction modes.

[0563] Optionally, the first number is 131, and a first gradient histogram of at least one reference area of ​​the current block in 131 prediction modes is determined or obtained, and the first gradient histogram records the gradient magnitudes of at least one reference area of ​​the current block in each gradient direction corresponding to the 131 prediction modes. Based on the gradient magnitudes in each gradient direction in the first gradient histogram, at least one candidate mode is determined or obtained, for example, modes corresponding to the top three gradient directions with the highest gradient magnitudes are determined as candidate modes.

[0564] Optionally, the first number is 67, and a first gradient histogram of at least one reference area of ​​the current block in 67 prediction modes is determined or obtained, and the first gradient histogram records the gradient magnitudes of the at least one reference area of ​​the current block in the gradient directions corresponding to the 67 prediction modes. Based on the gradient magnitudes in each gradient direction in the first gradient histogram, at least one candidate mode is determined or obtained, for example, modes corresponding to the top three gradient directions with the highest gradient magnitudes are determined as candidate modes.

[0565] Optionally, at least one candidate mode is determined or obtained according to a second statistical histogram of the current block under a second number of prediction modes, and the obtained candidate mode is one of the second number of prediction modes.

[0566] Optionally, the second number is 67, and a second gradient histogram of at least one reference area of ​​the current block in 67 prediction modes is determined or obtained, and the second gradient histogram records the gradient magnitudes of the at least one reference area of ​​the current block in the gradient directions corresponding to the 67 prediction modes. Based on the gradient magnitudes in each gradient direction in the second gradient histogram, at least one candidate mode is determined or obtained, for example, modes corresponding to the top three gradient directions with the highest gradient magnitudes are determined as candidate modes.

[0567] Optionally, the second number is 131, and a second gradient histogram of at least one reference area of ​​the current block in 131 prediction modes is determined or obtained, and the second gradient histogram records the gradient magnitudes of at least one reference area of ​​the current block in the gradient directions corresponding to the 131 prediction modes. Based on the gradient magnitudes in each gradient direction in the second gradient histogram, at least one prediction mode is determined or obtained, for example, the modes corresponding to the gradient directions with the top three gradient magnitudes are determined as the prediction mode of the current block.

[0568] Optionally, the first number is different from the second number, for example, the first number is greater than the second number. If the size of the current block meets the first condition, at least one candidate mode is determined or obtained based on the first statistical histogram of the current block under the prediction mode of the first number. If the size of the current block does not meet the first condition, at least one candidate mode is determined or obtained based on the second statistical histogram of the current block under the prediction mode of the second number.

[0569] Optionally, the first number is different from the second number, for example, the first number is greater than the second number. If the size of the current block meets the first condition, at least one candidate mode is determined or obtained based on the first statistical histogram of the current block under the prediction mode of the first number and / or the second statistical histogram of the current block under the prediction mode of the second number. If the size of the current block does not meet the first condition, at least one candidate mode is determined or obtained based on the second statistical histogram of the current block under the prediction mode of the second number.

[0570] Optionally, the first number is different from the second number. If the size of the current block meets the first condition, at least one candidate mode is determined or obtained based on the first statistical histogram of the current block under the prediction mode of the first number. If the size of the current block does not meet the first condition, at least one candidate mode is determined or obtained based on the first statistical histogram of the current block under the prediction mode of the first number and / or the second statistical histogram of the current block under the prediction mode of the second number.

[0571] Optionally, the first number and the second number are different, and at least one candidate mode is determined or obtained based on a first statistical histogram of the current block under the first number of prediction modes and a second statistical histogram of the current block under the second number of prediction modes.

[0572] Optionally, the first number and the second number are different, and at least one prediction mode A is determined or obtained based on the first statistical histogram of the current block under the prediction mode of the first number, and at least one prediction mode B is determined or obtained based on the second statistical histogram of the current block under the prediction mode of the second number, and at least one candidate mode is determined or obtained from at least one prediction mode A and at least one prediction mode B based on matching information of at least one prediction mode A and at least one prediction mode B respectively related to at least one reference area of ​​the current block.

[0573] Optionally, matching information refers to information used to evaluate the degree of matching between different prediction modes and the reference area and its corresponding current block. The matching information can be based on multiple indicators, such as the rate-distortion cost of the prediction results, prediction error, texture similarity, etc., to indicate the degree of matching between different prediction modes and the reference area.

[0574] Optionally, the matching information includes: SAD, SATD and / or MRSAD. Optionally, the smaller the SAD value, SATD value and / or MRSAD value, the higher the matching degree, and the larger the SAD value, SATD value and / or MRSAD value, the lower the matching degree.

[0575] In this embodiment, the prediction mode matching requirements of current blocks of different sizes are met by the first statistical histogram under the first number of prediction modes and / or the second statistical histogram of the current block under the second number of prediction modes. For example, the accuracy of the angle prediction in the prediction mode of a large-sized image block is more sensitive to the texture direction. Therefore, the number of angle prediction modes in the prediction mode can be increased to more finely match the actual texture direction and improve the prediction accuracy. The small-sized image block is less sensitive to the number of angle prediction modes in the prediction mode. Therefore, a statistical histogram with a small number of prediction modes can be used to determine the prediction mode, thereby improving the prediction efficiency.

[0576] Method a2, determining or obtaining at least one first mode based on the second statistical histogram of the current block under the second number of prediction modes, and determining or obtaining at least one candidate mode based on at least one second mode of the current block under the first number of prediction modes determined or obtained by at least one first mode.

[0577] Optionally, step S10 includes: determining or obtaining at least one first mode based on a second statistical histogram of the current block under a second number of prediction modes, determining or obtaining at least one candidate mode based on at least one second mode of the current block under the first number of prediction modes determined or obtained by the at least one first mode, and determining or obtaining at least one prediction mode of the current block based on matching information related to the at least one candidate mode and the current block.

[0578] Optionally, the statistical histogram includes: at least one of a gradient histogram, an area histogram, and a histogram of the number of times a prediction mode is used corresponding to an encoded image block, and a histogram obtained by adjusting or merging the above histograms.

[0579] Optionally, the first number is greater than the second number, and at least one first mode is determined or obtained based on a second statistical histogram of the current block under the prediction mode of the second number, and the first mode is one of the prediction modes of the second number. At least one candidate mode is determined or obtained based on at least one second mode of the current block under the prediction mode of the first number determined or obtained by at least one first mode.

[0580] Optionally, the first number is 131 and the second number is 67. The prediction modes under the first number completely cover the prediction modes under the second number, and the number of prediction modes is increased by filling the angular intervals between the angular prediction modes in the original prediction modes under the second number. Step S10 includes: determining or obtaining a second gradient histogram of at least one reference area of ​​the current block under the 67 prediction modes, and the second gradient histogram records the gradient magnitudes of the gradient directions corresponding to the at least one reference area of ​​the current block in the 67 prediction modes. According to the gradient magnitudes in each gradient direction in the second gradient histogram, at least one first mode is determined or obtained. For example, the modes corresponding to the first three gradient directions of the gradient magnitudes are determined as the first mode, and then the first mode is converted into the modes corresponding to the 131 prediction modes to obtain at least one second mode. For example, mode 23 (i.e., the first mode) among the 67 prediction modes corresponds to mode 44 (i.e., the second mode) among the 131 prediction modes, and then at least one candidate mode is determined or obtained according to the at least one second mode.

[0581] Optionally, at least one second mode of the current block in the first number of prediction modes is determined or obtained according to at least one mapping table and at least one first mode, and the mapping table includes: a first mapping table and / or a second mapping table.

[0582] Optionally, the first number is 131, the second number is 67, and the conversion from 67 prediction modes to 131 prediction modes can be determined or obtained according to a first mapping table. The first mapping table refers to the following Table 3:

[0583] Table 3

[0584]

[0585]

[0586] Optionally, the first number is 131, the second number is 67, and the conversion from 131 prediction modes to 67 prediction modes can be determined or obtained according to a second mapping table. The second mapping table refers to the following Table 4:

[0587] Table 4

[0588]

[0589]

[0590] Optionally, the step of determining or obtaining at least one candidate mode according to at least one second mode includes: determining or obtaining at least one candidate mode according to at least one second mode and / or adjacent modes of at least one second mode in the first number of prediction modes.

[0591] Optionally, in the first number and / or second number of prediction modes, each mode corresponds to a discretized direction. For example, among the 67 prediction modes, mode 18 is 0° (horizontal direction) and mode 34 is a 45° diagonal. Therefore, the adjacent mode can be at least one prediction mode that is closest to the angle value and / or index of the prediction mode. For example, among the 67 prediction modes, if the prediction mode is mode 34 (45°), then its adjacent modes can be modes 33 and 35.

[0592] Optionally, at least one first mode is determined or obtained based on the second statistical histogram of the current block in 67 prediction modes, for example, a1 and a2; at least one second mode of the current block in the first number of prediction modes is determined or obtained based on the at least one first mode, for example, b1 and b2; adjacent modes of at least one second mode in the first number of prediction modes are determined or obtained, for example, b1 is adjacent to b1' and b1", and b2 is adjacent to b2' and b2"; at least one candidate mode is determined or obtained based on the at least one second mode and the adjacent modes of at least one second mode in the first number of prediction modes, for example, at least one candidate mode is determined or obtained from b1, b1', b1", b2, b2' and b2" based on the matching information of at least one second mode and the adjacent modes of at least one second mode in the first number of prediction modes related to at least one reference area of ​​the current block.

[0593] Optionally, matching information refers to information used to evaluate the degree of matching between different prediction modes and the reference area and its corresponding current block. The matching information can be based on multiple indicators, such as the rate-distortion cost of the prediction results, prediction error, texture similarity, etc., to indicate the degree of matching between different prediction modes and the reference area.

[0594] Optionally, the matching information includes: SAD, SATD and / or MRSAD.

[0595] In this embodiment, at least one first mode is determined or obtained based on the second statistical histogram of the current block under the second number of prediction modes, and at least one candidate mode is determined or obtained based on at least one second mode of the current block under the first number of prediction modes determined or obtained through the at least one first mode. This can reduce the number of statistical histograms established and the computational complexity, while meeting the prediction mode matching requirements of current blocks of different sizes, matching appropriate prediction modes for current blocks of different sizes, and thereby supporting improved prediction accuracy of the current block, thereby supporting improved encoding and decoding quality during video encoding and / or decoding.

[0596] Fourth embodiment

[0597] Based on any of the above embodiments, a fourth embodiment is proposed.

[0598] In this embodiment, the first mode list is determined or obtained according to at least one of the following methods B1 to B6:

[0599] Method B1, at least one reference area of ​​the current block and a neural network model.

[0600] Optionally, step S10 includes: determining or obtaining a first pattern list based on at least one reference area and a neural network model of the current block, determining or obtaining at least one candidate pattern based on the first pattern list, and determining or obtaining at least one prediction pattern of the current block based on matching information related to the at least one candidate pattern and the current block.

[0601] Optionally, the first mode list is a set for storing at least one prediction mode, providing a variety of possible mode options for the prediction of the current block, and by comparing the prediction effects under different modes in the first mode list, a suitable prediction mode can be matched for the current block to improve the prediction accuracy for the current block.

[0602] Optionally, the first mode list is at least one mode list, and the prediction modes in the at least one mode list are different or at least partially the same.

[0603] Optionally, the first mode list includes an MPM (Most Probable Modes) list. Optionally, the first mode list in the embodiment of the present application may be an improved MPM list.

[0604] Optionally, the MPM list is a set of several prediction modes, which are considered to be the prediction modes most likely to be applicable to the current block, and the MPM list is determined or obtained based on the neighborhood information of the current block (such as the prediction mode of the upper block and the left block) and statistical information (such as the probability of the mode occurring). In this way, the MPM list can reduce the amount of mode information that the encoder needs to transmit, while improving the decoder's prediction accuracy of the prediction mode.

[0605] Optionally, in intra prediction, the candidate modes in the first mode list may include: at least one angular prediction mode, at least one non-angular prediction mode and / or a derived mode of at least one prediction mode.

[0606] Optionally, a first pattern list is determined or obtained based on pixels of at least a reference area of ​​the current block and a neural network model.

[0607] Optionally, a first pattern list is determined or obtained based on gradient information of at least one reference area of ​​the current block and a neural network model.

[0608] Optionally, a first pattern list is determined or obtained based on pixels, gradient information and a neural network model of at least one reference area of ​​the current block.

[0609] Optionally, by using the pixels and / or gradient information of at least one reference area of ​​the current block as input to the neural network model, at least one prediction mode and / or the confidence of at least one prediction mode under the first quantity and / or the second quantity is determined or obtained, and based on the at least one prediction mode and / or the confidence of at least one prediction mode output by the neural network model, a first mode list is determined or obtained.

[0610] Optionally, the first number is 35, 67 or 131, for example, the first number is 131.

[0611] Optionally, the first number is greater than 35, 67 or 131.

[0612] Optionally, the second number is 35, 67 or 131, for example, the second number is 67.

[0613] Optionally, the second number is less than or equal to 35, 67 or 131.

[0614] Optionally, according to the confidence of at least one prediction model determined or obtained by the neural network model, a first pattern list is determined or obtained based on at least one prediction model under the first quantity and / or second quantity output by the neural network model.

[0615] Optionally, confidence is a quantitative indicator of the degree of certainty of the model about the current prediction result.

[0616] Optionally, a neural network model is a computational model that mimics the way neurons in the human brain process information. It performs various tasks by learning complex patterns and features in data. The neural network model consists of at least one layer of neurons, including an input layer, at least one hidden layer, and an output layer. Each neuron receives an input signal, performs weighted summation, and then generates an output through a nonlinear activation function. The weights of these connected neurons determine the degree of influence of the input signal on the output, and the bias is used to adjust the activation threshold of the neuron. Activation functions, such as ReLU, Sigmoid, or Tanh, can give the neural network the ability of nonlinear modeling, enabling it to solve problems that linear models cannot solve.

[0617] Reference Figure 12 The specific implementation process of determining or obtaining the first pattern list through the neural network model may include: determining or obtaining the neural network model corresponding to the current block, determining or obtaining a reference template of the neural network model based on at least one reference area of ​​the current block (for example, Figure 12The reference template 1 on the left, the reference template 2 above, and the reference template 3 on the upper left) are used as the reconstructed reference pixels of the reference templates as the input of the neural network model, and the association relationship between the current block and its reference template learned by the neural network model is used to obtain the output of the neural network model, for example, at least one candidate pattern and / or the confidence of at least one candidate pattern under the first quantity and / or the second quantity.

[0618] Optionally, the reference area includes: a reference block, a reference pixel, a reference frame and / or a reference template.

[0619] Optionally, the reference area may be an image area adjacent to the current block, or an image area non-adjacent to the current block. Optionally, the reference area may be an area consisting of encoded and reconstructed pixels or blocks from the same frame and / or different frames.

[0620] Optionally, since there is a high degree of correlation between the reference area and the current block (for example, temporal correlation, spatial correlation, etc.), the embodiments of the present application can significantly improve the degree of matching between the obtained candidate mode and the current block by utilizing these associations, and improve the accuracy of the prediction result of the current block determined or obtained using the candidate mode as the prediction mode.

[0621] Optionally, the reference block may be a pixel block corresponding to the current block extracted from a reference frame (also referred to as a reference image) or a current frame (also referred to as a current image). The reference block may have the same size as the current block, and the reference block may serve as a prediction block for generating the current block. By using the reference block, the encoder may utilize temporal or spatial correlation to reduce redundant information in the current frame, thereby achieving efficient compression.

[0622] Optionally, at least one reference block of the current block is determined or obtained according to an index obtained from the bitstream.

[0623] Optionally, a reference frame refers to a frame (also called an image) that has been encoded and reconstructed, which can be a forward reference frame (i.e., a frame / image that is located before the current frame in the playback order) or a backward reference frame (i.e., a frame / image that is located after the current frame in the playback order).

[0624] Optionally, the neural network model may include: an input layer, a hidden layer, a dropout layer, and an output layer.

[0625] Optionally, the input layer is the first layer that receives input data. In this layer, each node (or neuron) generally represents a feature of the input data.

[0626] Optionally, hidden layers are located between the input layer and the output layer. There can be multiple hidden layers, which process the input data in a weighted manner.

[0627] Optionally, the hidden layer includes at least one of a fully connected layer, a convolutional layer, a pooling layer, a recursive layer, and a dropout layer.

[0628] Optionally, each neuron in a dense or fully connected layer is connected to every neuron in the previous layer.

[0629] Optionally, a convolutional layer is used to extract local features from the input data and is commonly used in image processing.

[0630] Optionally, a pooling layer is used for downsampling to reduce the amount of data and computation.

[0631] Optionally, a recurrent layer such as LSTM or GRU is used to process sequence data.

[0632] Optionally, a dropout layer is used to randomly "drop out" (i.e., temporarily set the output to zero) some neurons during model training. This can reduce the complex co-adaptation relationship between neurons, improve the generalization ability of the model, and reduce the risk of overfitting.

[0633] Optionally, a dropout layer can be added between at least one fully connected layer to prevent the neurons in these layers from being overly dependent on the specific output of the previous layer, thereby enhancing the generalization ability of the model. A dropout layer can be added between the convolutional layer and the fully connected layer to help reduce overfitting introduced by the fully connected layer. The output layer is the layer that generates the final prediction result.

[0634] Optionally, the first layer of the neural network model is used to construct a learnable gradient convolution, and the gradient information of the current block in at least one reference area is determined or obtained through a convolution operation.

[0635] Optionally, when the first layer of the neural network model is used to construct a learnable gradient convolution, the neural network model can determine or obtain the gradient information of the current block in at least one reference area through a convolution operation. Therefore, the pixels of at least one reference area of ​​the current block can be used as input of the neural network model.

[0636] Optionally, when the first layer of the neural network model does not construct a learnable gradient convolution, the gradient information of at least one reference area of ​​the current block is used as the input of the neural network model.

[0637] Optionally, a first neural network model can be determined or obtained from a group of neural network models; a first pattern list can be determined or obtained based on the first neural network model and at least one reference area of ​​the current block, and optionally, the first neural network model is at least one model.

[0638] Optionally, a first neural network model is determined or obtained from a group of neural network models based on at least one reference area of ​​the current block and a preset model mapping / correspondence rule.

[0639] Optionally, a group of neural network models are pre-selected and trained, including at least one of a neural network model based on a fully connected layer, a neural network based on a convolutional layer, and a neural network based on a hybrid convolutional and fully connected layer.

[0640] Optionally, a neural network model based on fully connected layers is also called a multilayer perceptron (MLP); the model includes: an input layer, one or more hidden layers (i.e., fully connected layers) and an output layer, each layer contains a certain number of neurons, and all neurons between two adjacent layers are interconnected, i.e., "fully connected" between layers. Optionally, a dropout layer can be set between at least one group of fully connected layers of the neural network model based on fully connected layers to help reduce overfitting introduced by the fully connected layers.

[0641] Reference Figure 13 The neural network model based on the fully connected layer includes: an input layer, at least one hidden layer (i.e., a fully connected layer) and an output layer. The fully connected layer includes: a linear layer (Linear layer) and an activation function layer (e.g., a LeakyReLU layer); the pixels of at least one reference area of ​​the current block and the gradient information of at least one reference area are input into the neural network model based on the fully connected layer; since the pixels of the reference area may exist in the form of a two-dimensional or three-dimensional matrix, in order to be able to feed the above data into the fully connected layer for processing, the input layer may be flattened to flatten the pixels of at least one reference area into a one-dimensional vector f, and the ReLU activation function is used to perform a nonlinear transformation to determine or obtain f i ; This operation introduces nonlinearity, allowing the model to learn more complex patterns, f i It will enter the network composed of fully connected layers, each layer performs linear transformation and activation function processing, and outputs a new feature vector f r , to extract higher-level abstract features from the input, and then convert the high-level features f r With low-level features f i Splice in the channel dimension to generate the feature vector f c , the eigenvector f cThe data is fed into a linear layer, and at least one prediction mode and / or the confidence of at least one prediction mode under the first quantity and / or the second quantity is output through an output layer.

[0642] Optionally, a convolutional neural network, i.e., a convolutional neural network (CNNs), includes: an input layer, at least one hidden layer, and an output layer, wherein the hidden layer includes: a convolutional layer for extracting features; each convolutional layer may be followed by a pooling layer for reducing the spatial size and reducing the computational complexity, and each convolutional layer may also be followed by a nonlinear activation function (e.g., ReLU) to introduce nonlinear factors so that the network can learn more complex patterns; as the depth of the convolutional neural network increases, the convolutional neural network can automatically learn feature representations from low-level to high-level, thereby improving prediction accuracy.

[0643] Reference Figure 14 The convolutional neural network includes: an input layer, at least one hidden layer and an output layer. The hidden layer includes: a convolutional layer, an activation function layer (for example, a LeakyReLU layer) and a linear layer. The pixels of the reference area of ​​the current block are input into the convolutional neural network, and a convolution operation is performed on the pixels of at least one reference area through at least one convolutional layer. The nonlinear transformation is performed through the ReLU activation function after each convolution layer to obtain the feature f. 1 、f 2 and f 3 . The features f1, f 2 and f 3 The fusion splicing is performed on the channel dimension to determine or obtain a new merged feature f containing three features to retain information from different convolution paths, allowing the model to combine low-level and high-level features and enhance representation capabilities. The fused feature f is processed again through the ReLU activation function to obtain f i ; This additional activation step helps to further improve the expressiveness and flexibility of the model. i The data are sent to a linear layer to obtain at least one prediction mode and / or the confidence of at least one prediction mode under the first quantity and / or the second quantity.

[0644] Optionally, a neural network model based on hybrid convolution and fully connected layers is an architecture that combines the advantages of CNN and MLP, including a convolution layer part and a fully connected layer part. The convolution layer can automatically learn the spatial hierarchical structure of the input data, such as edges, textures, shapes and other features in the image; the convolution operation can effectively capture local patterns and reduce the complexity of the model through a parameter sharing mechanism; the fully connected layer can flatten the features extracted by the convolution layer into a one-dimensional vector, and then send it to a series of fully connected layers, which are responsible for combining low-level features to determine or obtain higher-level abstract representations. The neural network model based on hybrid convolution and fully connected layers can combine the advantages of two different types of layers, which can not only efficiently extract features from the input data, but also make accurate predictions based on the features.

[0645] Optionally, the neural network model based on hybrid convolutional and fully connected layers includes: an input layer, at least one convolutional layer, at least one fully connected layer and an output layer.

[0646] Reference Figure 15 The neural network model based on hybrid convolution and fully connected layers includes: an input layer, at least one hidden layer and an output layer. The hidden layer includes: a convolutional layer, a fully connected layer (i.e., a linear layer) and an activation function layer (e.g., a LeakyReLU layer). The pixels of the reference area of ​​the current block are input into the neural network model based on hybrid convolution and fully connected layers. A convolution operation is performed on the pixels of at least one reference area through at least one convolution layer. A nonlinear transformation is performed through the ReLU activation function after each convolution layer to obtain the feature f. 1 、f 2 and f 3 , the feature f 1 、f 2 and f 3 Perform fusion and splicing in the channel dimension to determine or obtain a new merged feature f containing three features 4 , to retain information from different convolution paths, allowing the model to combine low-level and high-level features, enhance representation capabilities, and obtain feature f through at least one stacked linear layer 5 , the feature f 4 and f 5 Splicing is performed on the channel dimension to form a fusion feature f i , through at least one convolutional layer to fusion feature f i Convolution processing is performed to obtain at least one prediction mode and / or the confidence of at least one prediction mode under the first quantity and / or the second quantity.

[0647] Optionally, the construction of the neural network model mainly includes: data set construction and neural network construction and training.

[0648] Optionally, a first pattern list is determined or obtained based on an output result of the neural network model.

[0649] In this embodiment, the neural network model has powerful nonlinear modeling capabilities and can automatically learn the complex mapping relationship between input and output from a large amount of data. Therefore, the neural network can identify and utilize the subtle connection between the current block and its reference area: for example, effectively capture the temporal correlation between frames and the spatial correlation within frames, and understand the continuity and change patterns between different frames or adjacent regions within the same frame; and / or perceive the consistency of local features such as edges and textures, even if these features are displaced or deformed between different frames. Through multi-level feature extraction, it is ensured that the prediction results are not only accurate in local details, but also consistent with the actual content in terms of overall layout, so as to improve the accuracy of the candidate mode in the determined or obtained first mode list for the prediction of the current block from at least one dimension, and improve the encoding and decoding quality during the video encoding and / or decoding process, and / or the neural network model can output the confidence of at least one prediction mode at one time through parallel reasoning, thereby improving prediction efficiency.

[0650] Optionally, step S10 includes: determining or obtaining a first pattern list based on the first neural network model and at least one reference area of ​​the current block, determining or obtaining at least one candidate pattern based on the first pattern list, and determining or obtaining at least one prediction pattern of the current block based on matching information related to the at least one candidate pattern and the current block.

[0651] Optionally, the first neural network model is determined or obtained according to the following method b1 and / or b2:

[0652] Mode b1: at least one of the width, height, block size, and block area of ​​the current block;

[0653] Optionally, based on at least one of the width, height, size and area of ​​the current block, a first neural network model corresponding to the target model structure is determined from at least one model structure.

[0654] Optionally, based on at least one of the geometric properties of the current block (i.e., width, height, block size, and block area), a model structure that is most suitable for processing these features is selected, so that the first neural network model can more accurately reflect the characteristics of the input data and improve the model prediction effect.

[0655] Optionally, the structure of the first neural network model is determined based on at least one of the width, height, size and area of ​​the current block.

[0656] Optionally, since there is an association between at least one of the width, height, block size and block area of ​​the current block and at least one reference area of ​​the current block, a first neural network model suitable for processing the reference area and / or reference block data of the current block can be determined or obtained through at least one of the width, height, size and area of ​​the current block.

[0657] Optionally, the specific structure of the selected model is adjusted according to at least one of the geometric properties of the current block (i.e., width, height, block size and block area). For example, the number of hidden layers (e.g., convolutional layers, fully connected layers), filter size, pooling strategy, etc. of the neural network model are adjusted according to at least one of the width, height, size and area of ​​the current block.

[0658] Optionally, the structure of the first neural network model can be determined according to the width, height and the third mapping table of the current block.

[0659] Optionally, the third mapping table may be as shown in Table 5 below:

[0660] Table 5

[0661] Width, height The first neural network model <![CDATA[Either the width or the height of the current block is less than X1]]> First structure <![CDATA[The width and height of the current block are both greater than or equal to X1, and either the width or the height is less than X2]]> Second structure <![CDATA[The width and height of the current block are both greater than or equal to X2]]> The third structure <![CDATA[The width and height of the current block are both greater than or equal to X2, and either the width or the height is less than X3]]> Fourth Structure

[0662] Optionally, the structure of the first neural network model can be determined according to the block size of the current block and the fourth mapping table.

[0663] Optionally, the fourth mapping table may be as shown in Table 6 below:

[0664] Table 6

[0665] Block size The first neural network model <![CDATA[The block size of the current block is smaller than X4]]> First structure <![CDATA[The block size of the current block is greater than X4 and less than X5]]> Second structure <![CDATA[The block size of the current block is greater than X5 and less than X6]]> The third structure <![CDATA[The block size of the current block is greater than X6 and less than X7]]> Fourth Structure

[0666] Optionally, the structure of the first neural network model can be determined according to the block size of the current block and the fifth mapping table.

[0667] Optionally, the fifth mapping table may be as shown in Table 7 below:

[0668] Table 7

[0669]

[0670]

[0671] Optionally, if the current block or at least one reference area of ​​the current block does not match all neural network models, the reconstructed reference pixels of the current block or at least one reference area of ​​the current block can be transposed and / or downsampled so that the geometric properties of the current block and / or at least one reference area of ​​the current block match the neural network model (for example, the same size), and then the reconstructed reference pixels obtained by transposition and / or downsampling are input into the neural network model to obtain the prediction result of the model. The number of target models and / or target model structures is reduced as a whole by transposition or downsampling.

[0672] Optionally, the number of neural network models is N, for example, including neural network model NN0, neural network model NN1, neural network model NN2... neural network model NN N-1 Neural network model NN0, neural network model NN1, neural network model NN2, ..., neural network model NN N-1 Corresponding to block sizes W0xH0, W1xH1, W2xH2, ..., W N-1 xH N-1 .

[0673] In this embodiment, based on at least one of the width, height, block size and block area of ​​the current block, a model that is most suitable for processing these features is determined or obtained, so that the first neural network model can more accurately reflect the characteristics of the input data, improve the accuracy of at least one candidate mode in the first mode list determined or obtained by the model for predicting the current block, and improve the encoding and decoding quality in the video encoding and / or decoding process.

[0674] Mode b2: at least one of the width, height, size, and area of ​​at least one reference region of the current block;

[0675] Optionally, a first neural network model corresponding to the target model structure is determined from at least one model structure based on at least one item of the width, height, size and area of ​​at least one reference region of the current block.

[0676] Optionally, based on at least one of the geometric properties of at least one reference area of ​​the current block (i.e., width, height, block size and block area), a model structure that is most suitable for processing these features is selected, so that the first neural network model can more accurately reflect the characteristics of the input data and improve the model prediction effect.

[0677] Optionally, the structure of the first neural network model is determined based on at least one of the width, height, size and area of ​​at least one reference region of the current block.

[0678] Optionally, the specific structure of the selected model is adjusted based on at least one of the geometric properties of at least one reference region of the current block (i.e., width, height, block size, and block area). For example, the number of hidden layers (e.g., convolutional layers, fully connected layers), filter size, pooling strategy, etc. of the neural network model are adjusted based on at least one of the width, height, size, and area of ​​at least one reference region.

[0679] Optionally, the structure of the first neural network model may be determined based on the width, height, and sixth mapping table of at least one reference area of ​​the current block.

[0680] Optionally, the sixth mapping table may be as shown in Table 8 below:

[0681] Table 8

[0682] Width, height The first neural network model <![CDATA[Either the width or the height of the reference area is less than X1]]> First structure <![CDATA[The width and height of the reference area are both greater than or equal to X1, and either the width or the height is less than X2]]> Second structure <![CDATA[The width and height of the reference area are both greater than or equal to X2]]> The third structure <![CDATA[The width and height of the reference area are both greater than or equal to X2, and either the width or the height is less than X3]]> Fourth Structure

[0683] Optionally, the structure of the first neural network model may be determined based on the block size of at least one reference area of ​​the current block and the seventh mapping table.

[0684] Optionally, the seventh mapping table may be as shown in Table 9 below:

[0685] Table 9

[0686] Block size The first neural network model <![CDATA[The block size of the reference area is smaller than X4]]> First structure <![CDATA[The block size of the reference area is greater than X4 and less than X5]]> Second structure <![CDATA[The block size of the reference area is greater than X5 and less than X6]]> The third structure <![CDATA[The block size of the reference area is greater than X6 and less than X7]]> Fourth Structure

[0687] Optionally, the block size includes at least one of the width, height, ratio, depth, area, resolution, and number of pixels of the block. Optionally, X4 to X7 can be preset thresholds corresponding to at least one of the width, height, ratio, depth, area, resolution, and number of pixels in the block size.

[0688] Optionally, the structure of the first neural network model may be determined based on the block size of at least one reference area of ​​the current block and an eighth mapping table.

[0689] Optionally, the eighth mapping table may be as shown in Table 10 below:

[0690] Table 10

[0691] Block area The first neural network model <![CDATA[The block area of the reference region is less than X8]]> First structure <![CDATA[The block area of the reference region is greater than X8 and less than X9]]> Second structure <![CDATA[The block area of the reference region is greater than X9 and less than X 10 > The third structure <![CDATA[The block area of the reference region is greater than X 10 and less than X 11 > Fourth Structure

[0692] In this embodiment, based on at least one of the width, height, size and area of ​​at least one reference region of the current block, a model that is most suitable for processing these features is determined or obtained, so that the first neural network model can more accurately reflect the characteristics of the input data, improve the accuracy of the prediction of the current block by at least one candidate mode in the first mode list determined or obtained by the model, and improve the encoding and decoding quality in the video encoding and / or decoding process.

[0693] Mode B2: matching information related to at least one reference area of ​​the current block.

[0694] Optionally, step S10 includes: determining or obtaining a first pattern list based on matching information related to at least one reference area of ​​the current block, determining or obtaining at least one candidate pattern based on the first pattern list, and determining or obtaining at least one prediction pattern of the current block based on matching information related to at least one candidate pattern and the current block.

[0695] Optionally, the at least one reference region of the block is a pixel region or a pixel sampling region of a reconstructed or encoded region located adjacent to or not adjacent to the left, above, or above the left of the current block. Pixel values ​​or features of these regions can be used for prediction or other processing of the current block. Optionally, the reference region can include: a reference template, a reference block, and / or a reference pixel.

[0696] Optionally, matching information refers to information used to evaluate the degree of matching between different prediction modes and reference areas and their corresponding current blocks. The higher the degree of matching between the prediction mode and the current block and / or at least one reference area of ​​the current block, the higher the accuracy of the prediction result of the current block obtained using the prediction mode.

[0697] Optionally, the matching information includes: SAD, SATD and / or MRSAD, and the smaller the SAD value, SATD value and / or MRSAD value, the higher the matching degree, and the larger the SAD value, SATD value and / or MRSAD value, the lower the matching degree.

[0698] Optionally, a reference area of ​​the current block may be matched with at least one prediction mode to determine or obtain at least one matching information related to the reference area.

[0699] In this embodiment, the matching degree between the prediction mode and at least one reference area of ​​the current block is quantified by matching information, so that the mode that is more closely matched with the current block can be quickly screened out as the candidate mode in the first mode list, which can improve the adaptability between the candidate mode in the mode list and the current block, and improve the adaptability between the prediction mode determined or obtained based on the first mode list and the current block, thereby improving the prediction effect of the current block, and / or the matching information screening method can also save computing resources, reduce computing complexity, and improve coding efficiency.

[0700] Mode B3: matching information related to at least one reference region of a sub-block of the current block.

[0701] Optionally, step S10 includes: determining or obtaining a first pattern list based on matching information related to at least one reference area of ​​the current block, determining or obtaining at least one candidate pattern based on the first pattern list, and determining or obtaining at least one prediction pattern of the current block based on matching information related to at least one candidate pattern and the current block.

[0702] Optionally, the sub-block is part of the current block and inherits the position and some characteristics of the current block in the image. Therefore, the reference area of ​​the sub-block can also provide a reference for the prediction mode of the current block, and then the candidate mode in the first mode list can be determined or obtained based on the matching information related to at least one reference area of ​​the sub-block of the current block.

[0703] Optionally, at least one reference area of ​​a sub-block refers to a pixel area or a pixel sampling area of ​​a reconstructed or encoded area located adjacent to or not adjacent to the left, above, or upper left of the sub-block of the current block. The pixel values ​​or features of these areas can be used for prediction or other processing of the current block and / or its sub-blocks.

[0704] Optionally, the pattern matching information includes: SAD, SATD and / or MRSAD.

[0705] Optionally, a candidate mode in the first mode list is determined or obtained from each prediction mode according to mode matching information associated with each prediction mode and at least one reference region of a sub-block of the current block.

[0706] Optionally, a reference region of a sub-block of the current block may be matched with at least one prediction mode to determine or obtain at least one mode matching information related to a reference region of the sub-block.

[0707] In this embodiment, the matching degree between the prediction mode and at least one reference area of ​​the current block sub-block is quantified by pattern matching information, so that the mode that is more matched with the current block and its sub-blocks can be quickly screened out as the candidate mode in the first pattern list, which can improve the adaptability between the candidate mode in the pattern list and the current block, and improve the adaptability between the prediction mode determined or obtained based on the first pattern list and the current block, thereby improving the prediction effect of the current block, and / or the screening method of matching information can also save computing resources, reduce computing complexity, and improve coding efficiency.

[0708] Mode B4: the prediction mode corresponding to at least one of the coded image blocks of the sub-block, adjacent block, non-adjacent block, co-located block and preset position of the current block.

[0709] Optionally, step S10 includes: determining or obtaining a first mode list based on a prediction mode corresponding to at least one of a sub-block, an adjacent block, a non-adjacent block, a co-located block, and an encoded image block at a preset position of the current block; determining or obtaining at least one candidate mode based on the first mode list; and determining or obtaining at least one prediction mode of the current block based on matching information related to the at least one candidate mode and the current block.

[0710] Optionally, the current block, adjacent blocks, non-adjacent blocks, co-located blocks, and the encoded image block at the predicted position are image areas of the same division level, the sub-blocks of the current block, the sub-blocks of the adjacent blocks, the sub-blocks of the co-located blocks, and the sub-blocks of the encoded image block at the predicted position are image areas of the same division level, and the sub-blocks of the current block, the sub-blocks of the adjacent blocks, the sub-blocks of the co-located blocks, and the sub-blocks of the encoded image block at the predicted position are image areas of the next division level of the current block, adjacent blocks, non-adjacent blocks, co-located blocks, and the encoded image block at the predicted position.

[0711] Optionally, in video coding and decoding technology, it is necessary to divide the image into blocks. For example, the image can be divided into at least one strip, the strip can be divided into at least one coding unit, the coding unit can be divided into at least one sub-block, different strips are at the same division level, different coding units are at the same division level, different sub-blocks are at the same division level, while strips and coding units belong to different division levels, and coding units and sub-blocks belong to different division levels.

[0712] Optionally, the current block includes at least one sub-block, which is a part of the current block and inherits the position and some characteristics of the current block in the image. Therefore, it can provide a reference for the prediction mode of the current block, and then determine or obtain the candidate mode in the first mode list based on the prediction mode corresponding to the sub-block of the current block.

[0713] Optionally, the prediction mode corresponding to the sub-block of the current block can also provide a reference for the prediction modes of other sub-blocks of the current block. Therefore, the candidate mode in the first mode list of another sub-block of the current block can be determined based on the prediction mode corresponding to a sub-block of the current block.

[0714] Optionally, the adjacent blocks of the current block refer to image blocks that are directly adjacent to the current block in space, and the adjacent blocks may be blocks that have been predicted or reconstructed. The adjacent blocks are highly spatially correlated with the current block. Therefore, the prediction mode and texture features of the adjacent blocks are often similar to those of the current block. By analyzing the prediction mode of the adjacent blocks, a reference can be provided for the prediction mode of the current block, and the prediction accuracy of the current block can be further improved.

[0715] Optionally, the adjacent blocks include image blocks located above, to the left and / or above and to the left of the current block.

[0716] Optionally, the co-located block may be an image block in the co-located image with the same position and size as the current block, and the co-located image may be an image in the reference image that is closest to the current image in time. Since there is a strong temporal correlation between the co-located block and the current block, using the prediction mode of the co-located block as a candidate mode can better capture the motion characteristics in the video sequence and improve the prediction accuracy.

[0717] Optionally, the non-adjacent blocks of the current block refer to image blocks that are not directly adjacent to the current block in space, but may be within a certain range or have a certain correlation, and the non-adjacent blocks may also be blocks that have been predicted or reconstructed. Although the non-adjacent blocks are not directly adjacent to the current block, they may still have a certain correlation with the current block. Therefore, they have a certain reference value for the selection of the target prediction mode for the current block and the prediction of the current block.

[0718] Optionally, the encoded image blocks at preset positions refer to encoded blocks at certain positions pre-set according to specific rules or algorithms. These positions can be fixed (such as the upper left corner block) or dynamically determined according to the image content. Although non-adjacent blocks and encoded image blocks at preset positions may be far away from the current block in space, they may contain some unique texture or pattern information, and this information can serve as a supplement to the information of adjacent blocks and co-located blocks to further optimize the selection of prediction mode.

[0719] Optionally, the candidate modes in the first mode list are determined or obtained based on the adjacent prediction modes of the prediction modes corresponding to the sub-blocks, adjacent blocks, non-adjacent blocks, co-located blocks and / or encoded image blocks at preset positions of the current block. Optionally, the adjacent prediction modes of the prediction modes may be prediction modes that are adjacent in mode and / or adjacent in value.

[0720] Optionally, mode-adjacent refers to other modes that are adjacent in direction or type to a particular mode (e.g., the first prediction mode) in the definition or arrangement of prediction modes. These modes are typically logically or functionally associated with the first prediction mode and may have similar prediction directions or texture features.

[0721] Alternatively, numerically adjacent refers to other modes that are numerically adjacent to a particular mode in the numerical representation or index of the mode. These modes typically have consecutive index values ​​in the coding standard, indicating that they are similar in function or direction. Alternatively, in the HEVC standard, angular modes are represented by index values. If the index of the first possible mode is n, the numerically adjacent modes may be n-1 and n+1.

[0722] In this embodiment, determining or obtaining the candidate mode in the first mode list based on the prediction mode corresponding to the sub-block, adjacent block, non-adjacent block, co-located block and / or the encoded image block at a preset position of the current block can improve the adaptability between the mode and the current block, thereby improving the prediction effect of the current block, and / or compared with the full range of candidate mode screening, the embodiment of the present application utilizes existing information, i.e., the prediction mode corresponding to the adjacent block, non-adjacent block, co-located block and / or the encoded image block at a preset position, effectively narrows the search range of the candidate mode, and reduces the computational complexity to a certain extent.

[0723] Mode B5: the prediction mode corresponding to at least one of the coded image blocks selected from the group consisting of the adjacent block, the non-adjacent block, the co-located block and the preset position of the sub-block of the current block.

[0724] Optionally, step S10 includes: determining or obtaining a first mode list based on the prediction mode corresponding to at least one of the encoded image blocks of the adjacent blocks, non-adjacent blocks, co-located blocks and preset positions of the sub-blocks of the current block, determining or obtaining at least one candidate mode based on the first mode list, and determining or obtaining at least one prediction mode of the current block based on matching information related to the at least one candidate mode and the current block.

[0725] Optionally, the adjacent blocks, non-adjacent blocks, co-located blocks and / or encoded image blocks at preset positions of the sub-blocks of the current block include: adjacent blocks of the sub-blocks of the current block, non-adjacent blocks of the sub-blocks of the current block, co-located blocks of the sub-blocks of the current block and / or encoded image blocks at preset positions of the sub-blocks of the current block.

[0726] Optionally, prediction modes corresponding to adjacent blocks, non-adjacent blocks, co-located blocks and / or encoded image blocks at preset positions of sub-blocks of the current block are determined as candidate modes in the first mode list.

[0727] Optionally, the neighboring block of the subblock of the current block refers to an image block that is directly adjacent to the subblock of the current block in space, and the neighboring block may be a block that has been predicted or reconstructed.

[0728] Optionally, the co-located block of the sub-block of the current block may be an image block in the co-located image, having the same position and size as the sub-block of the current block.

[0729] Optionally, the non-adjacent blocks of the sub-block of the current block refer to image blocks that are not directly adjacent to the sub-block in space, but may be within a certain range or have some correlation, and the non-adjacent blocks may also be blocks that have been predicted or reconstructed.

[0730] Optionally, the encoded image blocks at preset positions refer to encoded blocks at certain positions preset according to specific rules or algorithms. These positions may be fixed (such as the upper left corner block) or dynamically determined according to image content.

[0731] Optionally, the candidate modes in the first mode list are determined or obtained based on the adjacent prediction modes of the prediction modes corresponding to the adjacent blocks, non-adjacent blocks, co-located blocks and / or encoded image blocks at preset positions of the sub-blocks of the current block. Optionally, the adjacent prediction modes of the prediction modes can satisfy the requirements of being adjacent in mode and / or adjacent in value.

[0732] In this embodiment, determining or obtaining the candidate mode in the first mode list based on the prediction mode corresponding to the encoded image block of at least one of the adjacent blocks, non-adjacent blocks, co-located blocks and preset positions of the sub-blocks of the current block can improve the adaptability between the mode and the current block, thereby improving the prediction effect of the current block, and / or compared to the full range of candidate mode screening, the embodiment of the present application utilizes existing information, namely, the prediction mode corresponding to the sub-blocks of the encoded image block of the adjacent blocks, non-adjacent blocks, co-located blocks and / or preset positions, effectively narrows the search range of the candidate mode, and reduces the computational complexity to a certain extent.

[0733] Mode B6: prediction mode corresponding to a sub-block of an encoded image block in at least one of an adjacent block, a non-adjacent block, a co-located block, and a preset position of the current block.

[0734] Optionally, step S10 includes: determining or obtaining a first mode list based on a prediction mode corresponding to a sub-block of an encoded image block of at least one of adjacent blocks, non-adjacent blocks, co-located blocks, and a preset position of the current block; determining or obtaining at least one candidate mode based on the first mode list; and determining or obtaining at least one prediction mode of the current block based on matching information related to the at least one candidate mode and the current block.

[0735] Optionally, the adjacent blocks, non-adjacent blocks, co-located blocks and / or sub-blocks of the encoded image blocks at a preset position of the current block include: sub-blocks of the adjacent blocks, non-adjacent blocks, co-located blocks and / or sub-blocks of the encoded image blocks at a preset position of the current block.

[0736] Optionally, the adjacent blocks, non-adjacent blocks, co-located blocks and / or the encoded image blocks at the preset position of the current block may also include at least one sub-block. The above sub-block is part of the adjacent blocks, non-adjacent blocks, co-located blocks and / or the encoded image blocks at the preset position, and inherits the position and some characteristics of the adjacent blocks, non-adjacent blocks, co-located blocks and / or the encoded image blocks at the preset position in the image. Therefore, it can also provide a reference for the prediction mode of the current block.

[0737] Optionally, at least some prediction modes corresponding to adjacent blocks, non-adjacent blocks, co-located blocks of the current block and / or sub-blocks of an encoded image block at a preset position are determined as candidate modes in the first mode list.

[0738] Optionally, the prediction mode corresponding to the adjacent blocks, non-adjacent blocks, co-located blocks and / or sub-blocks of the encoded image block at a preset position of the current block includes: at least one of: an angular prediction mode, a non-angular prediction mode and a derivation mode.

[0739] In this embodiment, determining or obtaining the candidate mode in the first mode list based on the prediction mode corresponding to the adjacent blocks, non-adjacent blocks, co-located blocks and / or sub-blocks of the encoded image block at a preset position of the current block can improve the adaptability between the mode and the current block, thereby improving the prediction effect of the current block, and / or, compared with screening the candidate modes in the full range, the embodiment of the present application uses existing information, i.e., the prediction mode corresponding to the adjacent blocks, non-adjacent blocks, co-located blocks and / or sub-blocks of the encoded image block at a preset position, to effectively narrow the search range of the candidate modes and reduce the computational complexity to a certain extent.

[0740] Fifth embodiment

[0741] Based on any of the above embodiments, a fifth embodiment is proposed.

[0742] In this embodiment, the method of determining or obtaining at least one candidate mode according to at least one gradient operator includes at least one of the following methods C1 to C4:

[0743] Mode C1: Determine or obtain at least one candidate mode based on at least one piece of gradient information determined or obtained by at least one gradient operator.

[0744] Optionally, the gradient information includes a gradient direction and / or a gradient magnitude. The gradient direction indicates the direction in which the grayscale value at a certain pixel point in the image (e.g., at least one reference area) changes fastest, that is, the direction perpendicular to the edge. The gradient magnitude indicates the intensity of the grayscale change at the pixel point, that is, the significance of the edge.

[0745] Optionally, step S10 includes: determining or obtaining at least one gradient information determined or obtained based on at least one gradient operator, determining or obtaining at least one gradient histogram and / or statistical result, determining or obtaining at least one candidate mode based on the at least one gradient histogram and / or statistical result, and determining or obtaining at least one prediction mode of the current block based on matching information related to the at least one candidate mode and the current block.

[0746] Optionally, at least one piece of gradient information is determined or obtained based on at least one reference area and at least one gradient operator of the current block, and at least one candidate mode is determined or obtained based on the at least one piece of gradient information.

[0747] Optionally, a prediction direction corresponding to the gradient direction is determined or obtained based on the gradient direction in the gradient information of at least one reference region of the current block and an angular difference between each prediction direction and the prediction direction in at least one lookup table (for example, the first lookup table and / or the second lookup table). For example, a direction with the smallest absolute angular difference is determined as the prediction direction corresponding to the gradient direction, and the magnitude value of the gradient magnitude in the gradient information of the at least one reference region is assigned to the prediction direction.

[0748] Optionally, the gradient information of the at least one reference region includes gradient information of pixels in the at least one reference region, and the gradient information of the pixels in the at least one reference region includes a quantized gradient direction.

[0749] Optionally, an initial gradient direction of pixels in at least one reference region is determined or obtained, and a quantized gradient direction of the pixels in the at least one reference region is determined or obtained based on an angle difference between the determined or obtained initial gradient direction and a gradient direction adjacent to the initial gradient direction in at least one lookup table.

[0750] Optionally, the initial gradient direction is compared with gradient directions adjacent to the initial gradient direction on the left and right sides in at least one lookup table. If an angular difference between the initial gradient direction and the left-adjacent gradient direction is smaller than an angular difference between the initial gradient direction and the right-adjacent gradient direction, the quantized gradient direction of the pixel is the left-adjacent gradient direction; and / or if the angular difference between the initial gradient direction and the left-adjacent gradient direction is larger than an angular difference between the initial gradient direction and the right-adjacent gradient direction, the quantized gradient direction of the pixel is the right-adjacent gradient direction.

[0751] Optionally, an initial gradient direction of the pixel is determined or obtained, initial gradient direction information related to the initial gradient direction of the pixel is determined or obtained, and quantized gradient direction information of the pixel is determined or obtained based on a difference between the determined or obtained initial gradient direction information and gradient direction information adjacent to the initial gradient direction information in at least one lookup table.

[0752] Optionally, the initial gradient direction information is compared with gradient direction information adjacent to the initial gradient direction information on the left and right sides in at least one lookup table. If a difference between the initial gradient direction information and the left-adjacent gradient direction information is smaller than a difference between the initial gradient direction information and the right-adjacent gradient direction information, the quantized gradient direction information of the pixel is the left-adjacent gradient direction information; and / or if a difference between the initial gradient direction information and the left-adjacent gradient direction information is larger than a difference between the initial gradient direction information and the right-adjacent gradient direction information, the quantized gradient direction information of the pixel is the right-adjacent gradient direction information.

[0753] Optionally, the gradient direction information is the tangent value of the gradient direction or the scaled tangent value.

[0754] Optionally, if a gradient direction X in the gradient information is between a first direction (i.e., the first gradient direction) and a second direction (i.e., the second gradient direction) in the first lookup table or the second lookup table, the gradient direction X is decomposed into a gradient direction A and a gradient direction B based on an angle difference or a difference between the gradient direction information x and the first gradient direction information and the second gradient direction information. At least one candidate mode is determined or obtained based on the gradient direction A and the gradient direction B.

[0755] Optionally, a first weight for gradient direction A is determined or obtained based on an angle difference or difference between gradient direction information x corresponding to gradient direction X and the first gradient direction information. A second weight for gradient direction B is determined or obtained based on an angle difference or difference between gradient direction information x corresponding to gradient direction X and the second gradient direction information. A gradient amplitude MA for gradient direction A is determined or obtained based on the first weight and a gradient amplitude M corresponding to gradient direction X. A gradient amplitude MB for gradient direction B is determined or obtained based on the second weight and the gradient amplitude M corresponding to gradient direction X. For example, the gradient amplitude MA is equal to the first weight*the gradient amplitude M, and the gradient amplitude MB is equal to the second weight*the gradient amplitude M.

[0756] Optionally, the gradient direction A is the first gradient direction described above, and the gradient direction B is the second gradient direction described above. Optionally, the first weight is equal to the difference between the gradient direction information x and the second gradient direction information, or the difference between the first gradient direction information and the second gradient direction information, and the second weight is equal to the difference between the gradient direction information x and the first gradient direction information, or the difference between the first gradient direction information and the second gradient direction information.

[0757] Optionally, if the gradient direction in the gradient information is between the first direction and the second direction in at least one lookup table, then based on the angular difference between the gradient direction and the first direction and the second direction, the weight information corresponding to each of the first direction and the second direction is determined or obtained, and based on the weight information corresponding to each of the first direction and the second direction, at least one candidate mode is determined or obtained. Optionally, based on the weight information corresponding to each of the first direction and the second direction, at least one gradient histogram and / or statistical result is determined or obtained, and based on the at least one gradient histogram and / or statistical result, at least one candidate mode is determined or obtained.

[0758] Optionally, if the gradient direction in the gradient information is between the first direction and the second direction in the first lookup table or the second lookup table, weight information corresponding to the first direction and the second direction is determined or obtained based on the angle difference between the gradient direction and the first direction and the second direction, and at least one candidate mode is determined or obtained based on the gradient magnitude in the gradient information and the weight information corresponding to the first direction and the second direction. Optionally, at least one gradient histogram and / or statistical result is determined or obtained based on the gradient magnitude in the gradient information and the weight information corresponding to the first direction and the second direction, and at least one candidate mode is determined or obtained based on the at least one gradient histogram and / or statistical result.

[0759] In this embodiment, when the gradient direction is between two prediction directions, the gradient amplitude is allocated according to the angular difference between the gradient direction and the prediction direction, thereby avoiding errors caused by rigid classification and improving the accuracy of angle prediction.

[0760] Mode C2: Determine or obtain at least one candidate mode based on overall gradient information corresponding to at least one reference region determined or obtained by at least one gradient operator.

[0761] Optionally, step S10 includes: determining or obtaining at least one gradient histogram and / or statistical result based on overall gradient information corresponding to at least one reference area determined or obtained by at least one gradient operator, determining or obtaining at least one candidate mode based on the at least one gradient histogram and / or statistical result, and determining or obtaining at least one prediction mode of the current block based on matching information related to the at least one candidate mode and the current block.

[0762] Optionally, the gradient information includes gradient information of at least one pixel in the reference region and / or overall gradient information of the reference region.

[0763] Optionally, at least one candidate mode is determined or obtained based on the overall gradient information of at least one reference area of ​​the current block and / or the gradient information of the pixels. For example, at least one gradient histogram and / or statistical result is determined or obtained based on the overall gradient information of at least one reference area of ​​the current block and / or the gradient information of the pixels. At least one candidate mode is determined or obtained based on the at least one gradient histogram and / or statistical result.

[0764] Optionally, the gradient information of the pixel is the gradient direction and / or gradient amplitude independently calculated for each pixel point in the reference area, reflecting the local change characteristics of the point.

[0765] Optionally, the overall gradient information of the reference region is obtained by statistically analyzing and / or aggregating the gradient information of all pixels in the reference region, and reflects the global change trend of the region.

[0766] Optionally, the pixel gradient information includes: a first pixel gradient component, a second pixel gradient component, a pixel gradient direction and / or a pixel gradient amplitude.

[0767] Optionally, gradient information of at least one pixel in the reference area is determined or obtained based on mutually perpendicular gradient operators, where a pair of mutually perpendicular gradient operators includes: a first gradient operator (e.g., a horizontal gradient operator Gx) and a second gradient operator (e.g., a vertical gradient operator Gy) corresponding to different directions, the value determined or obtained based on the first gradient operator is a first pixel gradient component (e.g., the value of Gx), and the value determined or obtained based on the second gradient operator is a second pixel gradient component (e.g., the value of Gy).

[0768] Optionally, the pixel gradient direction and / or the pixel gradient amplitude are determined or obtained according to the first pixel gradient component and the second pixel gradient component.

[0769] Optionally, the gradient information includes: a first gradient component (e.g., a first pixel gradient component and / or a first overall gradient component), a second gradient component (e.g., a second pixel gradient component and / or a second overall gradient component), a gradient direction (e.g., a pixel gradient direction and / or an overall gradient direction), and / or a gradient magnitude (e.g., a pixel gradient magnitude and / or an overall gradient magnitude).

[0770] Optionally, the overall gradient information includes: a first overall gradient component, a second overall gradient component, an overall gradient direction, and / or an overall gradient magnitude.

[0771] Optionally, the method for determining or obtaining the overall gradient information of the reference region includes: determining or obtaining the gradient information of at least one pixel in the reference region, where the gradient information of the at least one pixel includes: a first pixel gradient component and a second pixel gradient component, determining or obtaining the first overall gradient component according to the sum of the first pixel gradient components corresponding to the reference region, determining or obtaining the second overall gradient component according to the sum of the second pixel gradient components corresponding to the reference region, determining or obtaining the overall gradient magnitude according to the sum of the absolute values of the first overall gradient component and the second overall gradient component, and determining or obtaining the overall gradient direction according to the first overall gradient component and the second overall gradient component.

[0772] Optionally, within a local region, the edge directions generally tend to be consistent. Therefore, directly summing in the above manner can preserve the direction statistical characteristics, and the synthesized result can represent the overall gradient vector.

[0773] Optionally, determining or obtaining the gradient information of at least one pixel in the reference region includes: a first pixel gradient component (Gx1, Gx2,... Gx n ), and a second pixel gradient component (Gy1, Gy2,... Gy n ), determining or obtaining the first overall gradient component according to the sum of the first pixel gradient components corresponding to the reference region, that is, the first overall gradient component Gx sum = Gx1 + Gx2 +... + Gx n ), determining or obtaining the second overall gradient component according to the sum of the second pixel gradient components corresponding to the reference region, that is, the second overall gradient component Gy sum = Gy1 + Gy2 +... + Gy n ), determining or obtaining the overall gradient magnitude AMP sum , that is, the overall gradient magnitude AMP sum = |Gx sum | + |Gy sum |, and the overall gradient direction is obtained through arctan(Gx sum / Gy sum ) or arctan(Gysum / Gx sum ) is calculated.

[0774] Optionally, the method for determining or obtaining the overall gradient information of the reference region includes: determining or obtaining the gradient information of at least one pixel in the reference region, including: the first pixel gradient component, the second pixel gradient component, the pixel gradient direction, and the pixel gradient magnitude. According to the sum of the absolute values of the first pixel gradient component and the second pixel gradient component corresponding to the reference region, and / or, the sum of the absolute values of the first pixel gradient component and the second pixel gradient component corresponding to the reference region and the number of non-zero pixel gradient magnitudes corresponding to the pixel gradient direction of the reference region, determine or obtain the overall gradient magnitude. According to the sum of the first pixel gradient components corresponding to the reference region, determine or obtain the first overall gradient component. According to the sum of the second pixel gradient components corresponding to the reference region, determine or obtain the second overall gradient component. According to the first overall gradient component and the second overall gradient component, determine or obtain the overall gradient direction.

[0775] Optionally, determining or obtaining the gradient information of at least one pixel in the reference region includes: the first pixel gradient components (Gx1, Gx2,... Gxn) and the second pixel gradient components (Gy1, Gy2,... Gyn), the pixel gradient direction, and the pixel gradient magnitude. According to the sum of the absolute values of the first pixel gradient component and the second pixel gradient component corresponding to the reference region, and the number of non-zero pixel gradient magnitudes corresponding to the pixel gradient direction of the reference region, determine or obtain the overall gradient magnitude, that is, the overall gradient magnitude AMP sum = |Gx1| + |Gx2| +... + |Gx n | + |Gy1| + |Gy2| +... + |Gy n |, or the overall gradient magnitude AMP sum = (|Gx1| + |Gx2| +... + |Gx n | + |Gy1| + |Gy2| +... + |Gy n |) / n, where n is the number of non-zero pixel gradient magnitudes corresponding to the pixel gradient direction of the reference region. For example, if there are 10 directions in the pixel gradient direction of the reference region corresponding to non-zero pixel gradient magnitudes, then n is 10. According to the sum of the first pixel gradient components corresponding to the reference region, determine or obtain the first overall gradient component, that is, the first overall gradient component Gx sum = Gx1 + Gx2 +... + Gx n According to the sum of the second pixel gradient components corresponding to the reference region, determine or obtain the second overall gradient component, that is, the second overall gradient component Gy sum = Gy1 + Gy2 +... + Gy n The overall gradient direction is obtained by arctan(Gx sum / Gy sum ) or arctan(Gy sum / Gx sum ) calculated.

[0776] Optionally, the above method can be used to obtain an average value of the absolute values, which can resist directional interference and stably measure the gradient strength.

[0777] Optionally, when determining or obtaining the overall gradient information of the reference area based on the sum of the absolute values ​​of the first pixel gradient component and the second pixel gradient component corresponding to the reference area, if the overall gradient information is directly fused with the gradient information of the pixels corresponding to the reference area, the overall gradient amplitude corresponding to the overall gradient information in the determined or obtained gradient histogram and / or statistical result will be the maximum amplitude in the histogram and / or statistical result, thereby interfering with the selection of the prediction mode. To avoid the influence of the overall gradient amplitude information, the gradient histogram and / or statistical result can be determined or obtained based on the overall gradient information, the weight information corresponding to the overall gradient amplitude, and the gradient information of at least one pixel in the reference area, thereby ensuring fair competition of the gradient information and balancing the consistency requirements of intensity and direction, and / or The prediction mode corresponding to the overall gradient direction in the overall gradient information is determined as mode 1, a gradient histogram and / or statistical results are determined or obtained based on the gradient information of at least one pixel in the reference area, at least one mode 2 is determined or obtained based on the gradient histogram and / or statistical results, mode 1 and at least one mode 2 are determined as candidate modes, and when the at least one prediction mode of the current block includes mode 1 and at least one mode 2, mode 1 and mode 2 are determined as the at least one prediction mode of the current block, at least one prediction result 1 of the current block is determined or obtained based on mode 1, at least one prediction result 2 of the current block is determined or obtained based on the at least one mode 2, and a target prediction result of the current block is determined or obtained based on the at least one prediction result 1, the at least one prediction result 2, and weight information corresponding to the at least one prediction result 1, the at least one prediction result 2, and the at least one prediction result 1.

[0778] Optionally, the overall gradient information includes the overall gradient direction and the overall gradient magnitude, and the gradient histogram is determined or obtained based on the gradient information of at least one pixel corresponding to the reference area. Figure 1 And / or statistical result 1, the gradient histogram Figure 1 And / or the gradient amplitude corresponding to the direction that is the same as the overall gradient direction in the statistical result 1 is replaced by the overall gradient amplitude to determine or obtain the gradient histogram Figure 2 And / or statistical result 2, according to the gradient histogram Figure 2 and / or statistical result 2, determine or obtain at least one candidate mode.

[0779] Optionally, the target prediction result of the current block is determined or obtained based on at least one candidate mode and the weight information corresponding to the candidate mode, and the candidate modes include: a prediction mode corresponding to the overall gradient information, a prediction mode determined or obtained based on a gradient histogram, and / or a non-angle prediction mode.

[0780] Optionally, when the prediction mode determined or obtained based on the gradient histogram and / or statistical results includes a prediction mode corresponding to the overall gradient information, the weight information corresponding to at least one candidate mode is determined or obtained based on the weight information corresponding to the prediction mode corresponding to the overall gradient information, and the candidate modes include: the prediction mode corresponding to the overall gradient information, the prediction mode determined or obtained based on the gradient histogram and / or the non-angle prediction mode.

[0781] Optionally, the prediction mode determined or obtained according to the gradient histogram determined or obtained based on the gradient information of at least one pixel in the reference area includes: mode a1, mode a2 and mode a3, and the weights of mode a1, mode a2 and mode a3 can be determined according to the sum of the corresponding gradient amplitudes in the gradient histogram. For example, according to the sum of the corresponding gradient amplitudes in the gradient histogram, the weight value of mode a1 is 0.4, the weight value of mode a2 is 0.3, and the weight value of mode a3 is 0.3, and mode a1 is the prediction mode corresponding to the overall gradient information. According to the weight value of mode a1 being 0.4, the weights of modes a2 and a3 are adjusted. Weight value, for example, since the prediction mode corresponding to the overall gradient information is taken as the candidate mode by default, the weight value of mode a1 can be modified from the weight obtained according to the sum of the corresponding gradient amplitudes in the gradient histogram to a fixed weight corresponding to the overall gradient information. The value of the fixed weight is a preset value. The weights of mode a2 and mode a3 are determined according to the fixed weight and the sum of the gradient amplitudes of mode a2 and mode a3 in the gradient histogram. For example, if the fixed weight is 0.6, the sum of the gradient amplitudes of mode a2 and mode a3 is 50, then the weight of mode a2 is 0.2, and the weight of mode a3 is 0.2.

[0782] Optionally, when a candidate mode is determined to be the prediction mode of the current block, the corresponding weight information when the candidate mode is used as the prediction mode can be determined or obtained based on the weight information corresponding to the candidate mode.

[0783] Optionally, when a certain candidate mode is determined to be the prediction mode of the current block, the weight information corresponding to the candidate mode is determined as the weight information corresponding to the candidate mode as the prediction mode.

[0784] Optionally, the weight information corresponding to the candidate mode and the weight information corresponding to when the candidate mode is determined as the prediction mode may be the same as or different from.

[0785] Optionally, the candidate modes include: a prediction mode determined or obtained according to a gradient histogram and a planar mode (planar mode), the prediction mode determined or obtained according to the gradient histogram includes a prediction mode corresponding to the overall gradient information. In this embodiment, the prediction mode determined or obtained according to the gradient histogram determined or obtained based on the gradient information of at least one pixel in the reference area includes: mode a1, mode a2, and mode a3. Mode a1 is the prediction mode corresponding to the overall gradient information. A fixed weight is used for the prediction mode corresponding to the overall gradient information, for example, the fixed weight is equal to 0.2, and the planar mode uses a fixed weight of 0.3. The weights of modes a2 and a3 are determined by the sum of the gradient amplitudes corresponding to modes a2 and a3 in the gradient histogram. If the sum of the gradient amplitudes of mode a2 is 20 and the sum of the gradient amplitudes of mode a3 is 30, the weight corresponding to mode a2 is 0.2 and the weight corresponding to mode a3 is 0.3. The weight of mode a2 can be determined according to formula (5);

[0786] W a2 =(1-W a1 -W planar )*(AMP a2 / (AMP a2 +AMP a3 )); Formula (5);

[0787] Wa2 is the weight of mode a2, W a1 is the weight of mode a1, AMP a2 is the sum of the gradient amplitude values ​​of mode a2, AMP a3 is the sum of the gradient amplitude values ​​of mode a3.

[0788] The weight of mode a3 can be determined according to formula (6);

[0789] W a3 =(1-W a1 -W planar )*(AMP a3 / (AMP a2 +AMP a3 )); Formula (VI);

[0790] Wa2 is the weight of mode a2, W a1 is the weight of mode a1, AMP a2 is the sum of the gradient amplitude values ​​of mode a2, AMP a3 is the sum of the gradient amplitude values ​​of mode a3.

[0791] Optionally, the candidate modes include: a prediction mode determined or obtained according to a gradient histogram and a planar mode (planar mode). The prediction mode determined or obtained according to the gradient histogram does not include a prediction mode corresponding to the overall gradient information. In this embodiment, a fixed weight is used for the prediction mode corresponding to the overall gradient information, for example, the fixed weight is equal to 0.2, and the planar mode uses a fixed weight of 0.3. On this basis, the weights of mode a2 and mode a3 are determined by the sum of the gradient amplitudes corresponding to mode a2 and mode a3 in the gradient histogram. If the sum of the gradient amplitude values ​​of mode a2 is 20 and the sum of the gradient amplitude values ​​of mode a3 is 30, the weight corresponding to mode a2 is 0.2, and the weight corresponding to mode a3 is 0.3. Optionally, the weights corresponding to mode a2 and mode a3 can be determined or obtained by the above formula (5) and formula (6).

[0792] In this embodiment, by combining pixel gradient information and / or overall gradient information, the determined or obtained candidate mode can take into account both local details and global structure, and then at least one prediction mode determined or obtained based on the candidate mode can be consistent with the content of the current block, thereby improving the prediction accuracy.

[0793] Method C3: Determine or obtain at least one candidate mode according to a gradient operator corresponding to at least one lookup table.

[0794] Optionally, step S10 includes: determining or obtaining at least one candidate mode based on a gradient operator corresponding to at least one lookup table, determining or obtaining at least one prediction mode of the current block based on matching information related to the at least one candidate mode and the current block, and determining or obtaining at least one prediction mode of the current block based on matching information related to the at least one candidate mode and the current block.

[0795] Optionally, at least one lookup table includes: lookup table A and / or lookup table B, the number of prediction modes corresponding to lookup table A is a first number, the number of prediction modes corresponding to lookup table B is a second number, and the first number and the second number are the same or different.

[0796] Optionally, the first number is 35, 67 or 131, for example, the first number is 131.

[0797] Optionally, the first number is greater than 35, 67 or 131.

[0798] Optionally, the second number is 35, 67 or 131, for example, the second number is 67.

[0799] Optionally, the second number is less than or equal to 35, 67 or 131.

[0800] Optionally, the first number and the second number are the same, that is, the number of prediction modes corresponding to lookup table A is the same as the number of prediction modes corresponding to lookup table B. For example, lookup table A and lookup table B each correspond to the same number of prediction modes, but the types of the corresponding prediction modes are at least partially different.

[0801] Optionally, the first number and the second number are different, for example, the first number is 131 and the second number is 67.

[0802] Optionally, lookup table A corresponds to gradient operator A1 and gradient operator A2, lookup table B corresponds to gradient operator B1 and gradient operator B2, and gradient operators A1-A2 and gradient operators B1-B2 are different.

[0803] Optionally, the prediction mode corresponding to the lookup table A and the prediction mode corresponding to the lookup table B are different, or the prediction mode corresponding to the lookup table A and the prediction mode corresponding to the lookup table B are at least partially the same.

[0804] Optionally, lookup table A may be a lookup table corresponding to 37 prediction modes or a lookup table corresponding to 131 prediction modes.

[0805] Optionally, the lookup table B may be a lookup table corresponding to 37 prediction modes or a lookup table corresponding to 131 prediction modes.

[0806] Optionally, based on at least one reference area of ​​the current block and at least one gradient operator corresponding to at least one lookup table, gradient information of at least one reference area of ​​the current block is determined or obtained, and based on the gradient information of at least one reference area of ​​the current block, at least one gradient histogram is determined or obtained.

[0807] Optionally, the first number is greater than the second number, and the prediction direction corresponding to the prediction mode corresponding to lookup table A at least partially covers the prediction direction corresponding to the prediction mode corresponding to lookup table B, and the gradient operator ...

Claims

1. A processing method, characterized in that: Including steps: S10 , determining or obtaining at least one prediction mode of the current block according to matching information related to at least one candidate mode and the current block.

2. The processing method according to claim 1, characterized in that The candidate mode is determined or obtained based on at least one of the following: At least one gradient histogram corresponding to the current block; at least one area histogram corresponding to the current block; at least one histogram of usage counts of a prediction mode corresponding to an encoded image block corresponding to the current block; at least one gradient operator; First pattern list.

3. The processing method according to claim 2, characterized in that The first pattern list is determined or obtained according to at least one of the following: at least one reference region and a neural network model of the current block; matching information associated with at least one reference region of the current block; matching information associated with at least one reference region of a sub-block of the current block; a prediction mode corresponding to at least one of a subblock of the current block, an adjacent block, a non-adjacent block, a co-located block, and an encoded image block at a preset position; a prediction mode corresponding to at least one of an adjacent block, a non-adjacent block, a co-located block, and an encoded image block at a preset position of a subblock of the current block; The prediction mode corresponding to the sub-block of the encoded image block of at least one of an adjacent block, a non-adjacent block, a co-located block and a preset position of the current block.

4. The processing method according to claim 3, characterized in that The method of determining or obtaining the first pattern list based on at least one reference area of ​​the current block and the neural network model includes at least one of the following: Determine or obtain a first pattern list based on pixels of at least one reference area of ​​the current block and the neural network model; A first pattern list is determined or obtained based on gradient information of at least one reference region of the current block and a neural network model.

5. The processing method according to claim 2, characterized in that The gradient histogram and / or the area histogram is determined or obtained according to at least one of the following: Gradient information of at least one reference region of the current block; area information of at least one reference region of the current block; at least one gradient operator corresponding to at least one lookup table; Weight information corresponding to at least one pair of gradient operators; Weight information corresponding to gradient information of at least one reference region of the current block; At least one reference region and at least one lookup table of the current block.

6. The processing method according to claim 5, characterized in that The at least one gradient operator includes at least one pair of mutually perpendicular gradient operators, and / or the gradient information includes at least one of the following: Gradient information of pixels in the reference area; The overall gradient information of the reference area; The gradient information, the gradient information of the pixels in the reference region and / or the overall gradient information of the reference region include a gradient magnitude and / or a gradient direction.

7. The processing method according to claim 6, characterized in that Also include at least one of the following: The first gradient information is determined or obtained according to a first pair of mutually perpendicular gradient operators; The second gradient information is determined or obtained according to a second pair of mutually perpendicular gradient operators; The third gradient information is determined or obtained according to the first gradient direction and the second gradient direction.

8. The processing method according to any one of claims 3 to 5, characterized in that At least one reference area of ​​the current block is determined or obtained according to at least one of the following: At least one of an upper adjacent pixel, an upper non-adjacent pixel, a left adjacent pixel, a left non-adjacent pixel, an upper left adjacent pixel, and an upper left non-adjacent pixel of the current block; At least one of the neighboring block, non-neighboring block, co-located block, time domain block, and default block corresponding to the current block; At least one of the width, height, block size, and block area of ​​the current block; The candidate block is determined or obtained by the candidate motion vector or candidate block vector of the current block.

9. A processing device, characterized in that include: A memory and a processor, wherein a processing program is stored in the memory, and when the processing program is executed by the processor, the steps of the processing method according to any one of claims 1 to 8 are implemented.

10. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the steps of the processing method according to any one of claims 1 to 8.

Images