Processing method, processing device, and storage medium

Abstract

Provided in the present application are a processing method, a processing device, and a storage medium. The processing method can be applied to a processing device, and comprises: on the basis of a non-angular predictor, determining or generating a prediction result for the current block. The technical solution of the present application can improve the prediction accuracy of image blocks, pixels of which are in a non-angular arrangement, and can thus improve the coding and decoding quality during video coding and / or decoding.

Description

Processing method, processing device and storage medium Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a processing method, a processing device, and a storage medium. Background Art

[0002] The existing video coding standard (H.266 / VVC) proposes a video frame encoding technology. For example, when encoding and decoding video frames, the protocol divides each frame into different blocks and performs prediction processing and encoding and decoding processing.

[0003] During the process of conceiving and implementing this application, the inventors discovered that there are at least the following problems: Under natural circumstances, the distribution of object pixels within a graphic block is not completely arranged at an angle. Therefore, during the intra-frame prediction and / or inter-frame prediction process, the simple angle prediction mode has an unsatisfactory prediction effect on the image block, which in turn leads to poor encoding and decoding quality during the video encoding and / or decoding process.

[0004] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the Invention

[0005] In response to the above technical problems, the present application provides a processing method, a processing device and a storage medium, which can determine or generate the prediction result of the current block based on the non-angular predictor, and improve the prediction accuracy of the current block with non-angular pixels.

[0006] The present application provides a processing method that can be applied to a processing device, comprising the steps of:

[0007] S10 , determining or generating a prediction result of the current block according to the non-angle predictor.

[0008] Optionally, the non-angular predictor is associated with at least one non-angular prediction mode; and / or the current block includes at least one sub-block.

[0009] Optionally, the processing method further includes at least one of the following:

[0010] The prediction result of the current block is determined or generated by a non-angular predictor associated with at least one non-angular prediction mode of at least one sub-block;

[0011] The prediction result of the current block is determined or obtained based on the non-angular predictor associated with the first mode;

[0012] Non-angular prediction modes include at least one of the following:

[0013] Intra-frame prediction mode based on neural network;

[0014] Intra-frame prediction mode based on extrapolation filter;

[0015] Block vector prediction mode;

[0016] Plane mode;

[0017] DC mode.

[0018] Optionally, the first model is applied to a neural network-based intra-frame prediction mode.

[0019] Optionally, in the case where the first mode includes the planar mode, if the current block meets the enabling conditions of the neural network-based intra-frame prediction mode, the planar mode will be replaced by the neural network-based intra-frame prediction mode.

[0020] Optionally, the processing method further includes at least one of the following:

[0021] The first model is determined or obtained according to at least one of the width, height, block size and block area of ​​the current block;

[0022] The first model is determined or obtained according to at least one of the width, height, size and area of ​​at least one reference region of the current block;

[0023] The first model is determined or obtained based on at least one of the width, height, size and area of ​​at least one reference block of the current block;

[0024] The method of determining or obtaining the reference area and / or reference block includes at least one of the following:

[0025] According to at least one of an upper adjacent pixel, an upper non-adjacent pixel, a left adjacent pixel, a left non-adjacent pixel, an upper left adjacent pixel, and an upper left non-adjacent pixel of the current block;

[0026] According to at least one of a neighbor block, a non-neighbor block, a co-located block, a time domain block and a default block corresponding to the current block;

[0027] According to at least one of the width, height, block size and block area of ​​the current block;

[0028] A candidate block determined or obtained based on a candidate motion vector or a candidate block vector of a current block;

[0029] If the first information of the current block satisfies the first condition, the reference region is the first reference region and / or the reference block is the first reference block;

[0030] If the first information of the current block does not satisfy the first condition, the reference area is the second reference area and / or the reference block is the second reference block.

[0031] Optionally, the first information of the current block satisfies a first condition, including at least one of the following:

[0032] The value of the first information is a first value;

[0033] The value of the first information is within a first numerical range;

[0034] The value of the first information is greater than or equal to the first threshold;

[0035] The value of the first information is less than or equal to the second threshold.

[0036] Optionally, step S10 includes at least one of the following:

[0037] Performing prediction processing on the current block according to at least one non-angle prediction mode, determining or generating at least one non-angle predictor, and determining or generating a prediction result of the current block according to the at least one non-angle predictor;

[0038] performing prediction processing on the current block according to at least one non-angle prediction mode and at least one angular prediction mode, respectively, to determine or generate at least one non-angle predictor and at least one angular predictor, and determining or generating a prediction result of the current block according to the at least one non-angle predictor and the at least one angular predictor;

[0039] Determine or obtain a first non-angle predictor from the non-angle predictors, and determine or generate a prediction result of the current block based on the first non-angle predictor;

[0040] Determining or obtaining a second mode according to rate-distortion costs corresponding to each non-angular prediction mode, and determining or generating a prediction result of the current block according to a non-angular predictor associated with the second mode;

[0041] A prediction result of the current block is determined or generated based on the non-angle predictor and the angle predictor.

[0042] Optionally, the non-angle predictor and / or the angle predictor is determined or obtained according to at least one of the following:

[0043] at least one reference region of the current block;

[0044] at least one reference block of the current block;

[0045] At least one reference pixel of the current block.

[0046] The present application also provides a processing device, including: a memory and a processor, wherein a processing program is stored in the memory, and when the processing program is executed by the processor, the steps of any of the above processing methods are implemented.

[0047] The present application also provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the processing methods described above are implemented.

[0048] As described above, the processing method of the present application can be applied to a processing device, including: determining or generating a prediction result for the current block based on a non-angle predictor. Through the technical solution of the present application, it is possible to determine or generate a prediction result for the current block based on a non-angle predictor, thereby improving the prediction accuracy of the current block whose pixels are arranged in a non-angle manner, and further supporting improved codec quality during video encoding and / or decoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for describing the embodiments. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without inventive work.

[0050] FIG1 is a schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present application;

[0051] FIG2 is a diagram of a communication network system architecture provided by an embodiment of the present application;

[0052] FIG3 is a schematic diagram of the hardware structure of a controller 140 provided in this application;

[0053] FIG4 is a schematic diagram of the hardware structure of a network node 150 provided in this application;

[0054] FIG5 is a schematic flow chart of a processing method according to the first embodiment;

[0055] FIG6 is a schematic diagram of an encoding process of an encoder in an image processing method according to the first embodiment;

[0056] 7 is a schematic diagram of a decoding process of a decoder in an image processing method according to the first embodiment;

[0057] FIG8 is a schematic diagram of a template of the NNIP mode in the processing method according to the second embodiment;

[0058] FIG9 is a schematic structural diagram of a neural network model based on a fully connected layer according to the second embodiment;

[0059] FIG10 is a schematic structural diagram of a convolutional layer-based neural network model according to the second embodiment;

[0060] FIG11 is a schematic diagram of the structure of a neural network model based on a hybrid convolutional and fully connected layer according to the second embodiment;

[0061] FIG12 is a schematic diagram of a reference template of the DIMD mode in the processing method according to the third embodiment;

[0062] FIG13 is a first schematic diagram of a reference template in the TIMD mode in a processing method according to the third embodiment;

[0063] 14 is a second schematic diagram of a reference template of a DIMD mode in a processing method according to a third embodiment;

[0064] FIG15 is a second schematic diagram of a reference template in the TIMD mode in the processing method according to the third embodiment;

[0065] FIG16 is a first schematic diagram of candidate blocks in the TIMD mode in a processing method according to the fourth embodiment;

[0066] FIG17 is a second schematic diagram of candidate blocks in the TIMD mode in the processing method according to the fourth embodiment;

[0067] FIG18 is a third schematic diagram of candidate blocks in the TIMD mode in the processing method according to the fourth embodiment;

[0068] FIG19 is a fourth schematic diagram of candidate blocks in the TIMD mode in the processing method according to the fourth embodiment;

[0069] FIG20 is a first schematic diagram of candidate blocks in the OBIC mode in the processing method according to the fourth embodiment;

[0070] FIG21 is a second schematic diagram of candidate blocks in the OBIC mode in the processing method according to the fourth embodiment;

[0071] FIG22 is a third schematic diagram of candidate blocks in the OBIC mode in the processing method according to the fourth embodiment;

[0072] FIG23 is a fourth schematic diagram of candidate blocks in the OBIC mode in the processing method according to the fourth embodiment;

[0073] FIG24 is a fifth schematic diagram of candidate blocks in the OBIC mode in the processing method according to the fourth embodiment;

[0074] FIG25 is a sixth schematic diagram of candidate blocks in the OBIC mode in the processing method according to the fourth embodiment;

[0075] FIG26 is a seventh schematic diagram of candidate blocks in the OBIC mode in the processing method according to the fourth embodiment;

[0076] FIG27 is a schematic diagram eight of candidate blocks in the OBIC mode in the processing method according to the fourth embodiment;

[0077] FIG28 is a ninth schematic diagram of candidate blocks in the OBIC mode in the processing method according to the fourth embodiment;

[0078] FIG29 is a schematic diagram of a processing module of a processing device.

[0079] The purpose of this application, its features, and advantages will be further described in conjunction with the embodiments and with reference to the accompanying drawings. The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and the accompanying text are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of this application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0080] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0081] It should be noted that, in this document, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0082] It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this document, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determination." Furthermore, as used herein, the singular forms "one," "an," and "the" are intended to also include plural forms, unless otherwise indicated in the context. It should be further understood that the terms "comprising" and "including" indicate the presence of the described features, steps, operations, elements, components, items, types, and / or groups, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, types, and / or groups. The terms “or”, “and / or”, “including at least one of the following”, etc. used in this application may be interpreted as inclusive, or mean any one or any combination, for example, “including at least one of the following: A, B, C” means “any one of the following: A; B; C; A and B; A and C; B and C; A and B and C”, for example, “A, B or C” or “A, B and / or C” means “any one of the following: A; B; C; A and B; A and C; B and C; A and B and C”. An exception to this definition will only occur when a combination of elements, functions, steps or operations are inherently mutually exclusive in some way.

[0083] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence in the order indicated by the arrows. Unless clearly stated herein, the execution of these steps is not strictly limited in order, and they can be performed in other orders. Moreover, at least a portion of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0084] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0085] It should be noted that, in this article, step codes such as S10 are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the sequence.

[0086] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0087] In the subsequent description, the use of suffixes such as "module", "component" or "unit" to represent elements is only for the purpose of facilitating the description of the present application and has no specific meaning. Therefore, "module", "component" or "unit" can be used interchangeably.

[0088] The processing device can be implemented in various forms. For example, the processing device described in this application may include processing devices such as mobile phones, servers, tablet computers, laptops, PDAs, portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, and fixed terminals such as digital TVs and desktop computers.

[0089] The subsequent description will be made by taking a mobile terminal as an example. It will be understood by those skilled in the art that, in addition to components specifically used for mobile purposes, the configuration according to the embodiments of the present application can also be applied to fixed-type terminals.

[0090] Please refer to Figure 1, which is a schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present application. The mobile terminal 100 may include components such as an RF (Radio Frequency) unit 101, a WiFi module 102, an audio output unit 103, an A / V (Audio / Video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111. Those skilled in the art will understand that the mobile terminal structure shown in Figure 1 does not limit the mobile terminal. The mobile terminal may include more or fewer components than shown, or may combine certain components, or arrange the components differently.

[0091] The following is a detailed introduction to the various components of the mobile terminal in conjunction with Figure 1:

[0092] The RF unit 101 can be used to send and receive information or receive signals during calls. Specifically, it receives downlink information from the base station and transmits it to the processor 110 for processing; and / or sends uplink data to the base station. Typically, the RF unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, and more. Furthermore, the RF unit 101 can communicate with the network and other devices via wireless communication. The above-mentioned wireless communications can use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution), 5G and 6G, etc.

[0093] WiFi is a short-range wireless transmission technology. A mobile terminal, through WiFi module 102, enables users to send and receive emails, browse web pages, and access streaming media, providing wireless broadband Internet access. Although FIG1 illustrates WiFi module 102, it is understood that it is not a required component of the mobile terminal and can be omitted as needed without altering the essence of the invention.

[0094] The audio output unit 103 can convert audio data received by the RF unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output it as sound when the mobile terminal 100 is in a call signal reception mode, a talk mode, a recording mode, a voice recognition mode, a broadcast reception mode, or the like. Furthermore, the audio output unit 103 can also provide audio output related to a specific function performed by the mobile terminal 100 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 103 may include a speaker, a buzzer, or the like.

[0095] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos captured by an image capture device (e.g., a camera) in video capture mode or image capture mode. The processed image frames may be displayed on the display unit 106. The image frames processed by the GPU 1041 may be stored in the memory 109 (or other storage medium) or transmitted via the RF unit 101 or the WiFi module 102. The microphone 1042 may receive sound (audio data) in operating modes such as a phone call mode, a recording mode, and a voice recognition mode, and may process such sound into audio data. In the phone call mode, the processed audio (voice) data may be converted into a format that can be transmitted to a mobile communication base station via the RF unit 101. The microphone 1042 may implement various types of noise cancellation (or suppression) algorithms to eliminate (or suppress) noise or interference generated during the reception and transmission of audio signals.

[0096] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Optionally, the light sensor includes an ambient light sensor and a proximity sensor. Optionally, the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 1061 and / or the backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that recognize the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be described here.

[0097] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0098] The user input unit 107 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile terminal. Optionally, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also known as a touch screen, can collect user touch operations on or near it (such as operations performed by the user using a finger, stylus, or any other suitable object or accessory on or near the touch panel 1071) and drive the corresponding connection device according to a pre-set program. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Optionally, the touch detection device detects the user's touch direction and detects the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device and converts it into touch point coordinates, which are then sent to the processor 110. It can also receive commands sent by the processor 110 and execute them. In addition, the touch panel 1071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may further include other input devices 1072. Optionally, the other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, power keys, etc.), a trackball, a mouse, a joystick, etc., and the specifics are not limited here.

[0099] Optionally, the touch panel 1071 may overlay the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. The processor 110 then provides a corresponding visual output on the display panel 1061 based on the type of touch event. Although in FIG1 , the touch panel 1071 and the display panel 1061 are shown as two separate components to implement the input and output functions of the mobile terminal, in some embodiments, the touch panel 1071 and the display panel 1061 may be integrated to implement the input and output functions of the mobile terminal, which is not limited to this specific embodiment.

[0100] The interface unit 108 serves as an interface through which at least one external device can be connected to the mobile terminal 100; for example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 108 can be used to receive input (e.g., data information, power, etc.) from the external device and transmit the received input to one or more elements within the mobile terminal 100 or can be used to transmit data between the mobile terminal 100 and the external device.

[0101] Memory 109 can be used to store software programs and various data. Memory 109 may primarily include a program storage area and a data storage area. Optionally, the program storage area may store an operating system and at least one application required for a function (such as a sound playback function or an image playback function); the data storage area may store data generated based on the use of the mobile phone (such as audio data, a phone book, etc.). Furthermore, memory 109 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0102] Processor 110 is the control center of the mobile terminal, connecting all components of the mobile terminal using various interfaces and circuits. By running or executing software programs and / or modules stored in memory 109 and accessing data stored in memory 109, it executes various functions of the mobile terminal and processes data, thereby providing overall monitoring of the mobile terminal. Processor 110 may include one or more processing units; preferably, processor 110 may integrate an application processor and a modem processor. Optionally, the application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 110.

[0103] The mobile terminal 100 may also include a power supply 111 (such as a battery) for supplying power to various components. Preferably, the power supply 111 may be logically connected to the processor 110 through a power management system, thereby managing functions such as charging, discharging, and power consumption through the power management system.

[0104] Although not shown in FIG. 1 , the mobile terminal 100 may further include a Bluetooth module, etc., which will not be described in detail here.

[0105] To facilitate understanding of the embodiments of the present application, the communication network system on which the mobile terminal of the present application is based is described below.

[0106] Please refer to Figure 2, which is a communication network system architecture diagram provided in an embodiment of the present application. The communication network system is an LTE system of universal mobile communication technology. The LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203 and an operator's IP service 204, which are connected in sequence.

[0107] Optionally, UE201 may be the above-mentioned terminal 100, which will not be described in detail here.

[0108] E-UTRAN 202 includes eNodeB 2021 and other eNodeBs 2022 . Optionally, eNodeB 2021 may be connected to other eNodeBs 2022 via a backhaul (eg, an X2 interface). eNodeB 2021 is connected to EPC 203 , and eNodeB 2021 may provide access from UE 201 to EPC 203 .

[0109] EPC 203 may include an MME (Mobility Management Entity) 2031, an HSS (Home Subscriber Server) 2032, other MMEs 2033, an SGW (Serving Gate Way) 2034, a PGW (PDN Gate Way) 2035, and a PCRF (Policy and Charging Rules Function) 2036. Optionally, MME 2031 is a control node that processes signaling between UE 201 and EPC 203, providing bearer and connection management. HSS 2032 provides registers for managing functions such as the Home Location Register (not shown) and stores user-specific information such as service features and data rates. All user data can be sent through SGW2034, PGW2035 can provide IP address allocation and other functions for UE 201, PCRF2036 is the policy and charging control policy decision point for service data flow and IP bearer resources, and it selects and provides available policy and charging control decisions for the policy and charging execution function unit (not shown in the figure).

[0110] The IP service 204 may include the Internet, an intranet, an IMS (IP Multimedia Subsystem), or other IP services.

[0111] Although the above introduction takes the LTE system as an example, those skilled in the art should know that this application is not only applicable to the LTE system, but can also be applied to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, 5G and future new network systems (such as 6G), etc., which are not limited here.

[0112] FIG3 is a schematic diagram of the hardware structure of a controller 140 provided in this application. The controller 140 includes a memory 1401 and a processor 1402. The memory 1401 is used to store program instructions, and the processor 1402 is used to call the program instructions in the memory 1401 to execute the steps performed by the controller in the first embodiment of the above method. The implementation principles and beneficial effects are similar and will not be repeated here.

[0113] Optionally, the controller further includes a communication interface 1403, which can be connected to the processor 1402 via a bus 1404. The processor 1402 can control the communication interface 1403 to implement the receiving and sending functions of the controller 140.

[0114] Figure 4 is a schematic diagram of the hardware structure of a network node 150 provided in this application. Network node 150 includes: a memory 1501 and a processor 1502. Memory 1501 is used to store program instructions, and processor 1502 is used to call the program instructions in memory 1501 to execute the steps performed by the first node in the first embodiment of the above method. The implementation principles and beneficial effects are similar and will not be repeated here.

[0115] Optionally, the controller further includes a communication interface 1503, which can be connected to the processor 1502 via a bus 1504. The processor 1502 can control the communication interface 1503 to implement the receiving and sending functions of the network node 150.

[0116] The integrated modules implemented in the form of software function modules can be stored in a computer-readable storage medium. The software function modules stored in a storage medium include a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute some of the steps of the methods of various embodiments of the present application.

[0117] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive solid state disk, SSD), etc.

[0118] First embodiment

[0119] 5 , which is a flow chart of a processing method according to a first embodiment, the processing method of the embodiment of the present application can be applied to a processing device, including:

[0120] Step S10: determining or generating a prediction result of the current block according to the non-angle predictor.

[0121] In this embodiment, the processing device can be a smart terminal, such as a mobile phone, a computer, etc., or a server, such as a local server or a cloud server. In this embodiment and this application, the processing device is mainly described as a smart terminal.

[0122] Optionally, the technical solution of this embodiment can be applied to the fields of image coding and decoding, video coding and decoding, hardware video coding and decoding, dedicated circuit video coding and decoding, real-time video coding and decoding, and so on.

[0123] Optionally, the processing device may obtain video image data from a video source, segment each frame of the video image data to obtain multiple image blocks, and determine an image block to be predicted among the multiple image blocks as the current block.

[0124] Optionally, the non-angle predictor is determined or obtained by a prediction mode based on a neural network, for example, the non-angle predictor is determined or obtained by a neural network intra prediction (NNIP) mode based on a neural network.

[0125] Optionally, the current block includes at least one sub-block; in order to improve the prediction effect for a larger image block (i.e., the current block), the current block can be divided into at least one sub-block, and the prediction result of the current block can be determined or generated based on the non-angular predictor of at least one sub-block; by predicting the sub-blocks of the current block, it is helpful to capture and analyze the local features of the current block more carefully, thereby improving the prediction effect, and / or significantly reducing the amount of data processed at a single time, making the calculation more efficient.

[0126] Optionally, the number of sub-blocks included in the current block is set according to actual conditions: for example, the current block includes 4, 8, 16 sub-blocks, etc. Optionally, the sizes of the sub-blocks may be at least partially the same or different.

[0127] Optionally, the current block may be divided into four sub-blocks of the same size, and prediction processing may be performed on each sub-block according to a preset processing order.

[0128] Optionally, the prediction result of the current block is determined or generated by a non-angle predictor associated with at least one non-angle prediction mode of at least one sub-block. Optionally, the prediction result may be a predicted pixel, a predicted block, or the like.

[0129] Optionally, the prediction result of the current block is determined or generated by at least one non-angular prediction mode-related non-angular predictor and / or at least one angular prediction mode-related angular predictor of at least one sub-block.

[0130] Optionally, the prediction result of the sub-block can be determined or generated based on at least one non-angle predictor and / or at least one angle predictor of the sub-block; and the prediction result of the current block can be determined or obtained based on the prediction result of the sub-block.

[0131] Optionally, in the prediction derivation mode, at least one non-angle predictor is determined or obtained through a prediction mode based on a neural network, and a prediction result of the current block is determined or generated based on the at least one non-angle predictor and / or at least one angle predictor. Optionally, the prediction derivation mode includes at least one of a DIMD (Decoder side intra mode derivation) mode, an OBIC (Occurrence-based Intra Coding) mode and a TIMD (template-based intra mode derivation) mode.

[0132] Optionally, in the prediction derivation mode, a prediction result of the current block or and / or at least one sub-block of the current block is determined or generated based on the first prediction result and the second prediction result. Optionally, the prediction derivation mode includes at least one of a DIMD mode, an OBIC mode and a TIMD mode. Optionally, the first prediction result is determined or obtained by a prediction mode based on a neural network.

[0133] Optionally, the first prediction result is determined or obtained by at least one first non-angle predictor and / or at least one first angle predictor.

[0134] Optionally, the second prediction result is determined or obtained by at least one second non-angle predictor and / or at least one second angle predictor.

[0135] Optionally, the first prediction result and the second prediction result are different.

[0136] Optionally, the at least one first non-angle predictor and the at least one second non-angle predictor are different, and the at least one first angle predictor and the at least one second angle predictor are different.

[0137] Optionally, the first prediction result and the second prediction result are respectively determined or obtained by a neural network-based prediction mode and a non-neural network-based prediction mode. Optionally, the non-neural network-based prediction mode includes: an angle prediction mode, a DC mode (direct current mode) and / or a Planar mode (planar mode).

[0138] Optionally, the prediction result of the current block or the sub-block of the current block may be determined or generated by fusing the first prediction result and the second prediction result.

[0139] Optionally, in the intra-frame prediction mode, the prediction result of the current block or the sub-block of the current block is determined or generated through the first prediction result and the second prediction result. Optionally, the first prediction result is determined or obtained through a prediction mode based on a neural network, and the second prediction result is determined or obtained through a prediction derivation mode.

[0140] Optionally, the above-mentioned prediction derivation mode includes at least one of: DIMD mode, OBIC mode and TIMD mode.

[0141] Optionally, the prediction results of the sub-blocks of the current block may be directly spliced, weighted fusion, overlapping area smoothing, etc., to determine or obtain the prediction result of the current block.

[0142] Optionally, direct splicing refers to directly splicing the prediction results of each sub-block according to the position of each sub-block in the current block; it is applicable to the case where there is no overlap between sub-blocks or the overlap can be ignored.

[0143] Optionally, if there is overlap between sub-blocks, smoothing techniques can be used in the overlapping area to reduce boundary effects: for example, using an averaging method to average the pixel values ​​in the overlapping area as the final predicted pixel; and / or using Gaussian weights, etc. for weighted averaging to achieve a smoother transition effect.

[0144] Optionally, weighted fusion refers to assigning different weight information to each sub-block according to the confidence or other indicators of the prediction results of each sub-block, and then fusing based on this weight information. For example, if the prediction result of a sub-block is considered to be more accurate, it can be given a higher weight information; otherwise, it can be given a lower weight information.

[0145] Optionally, the weight may include at least one of a weight coefficient, a weight vector, and a weight matrix.

[0146] Optionally, the current block may be an image block to be predicted in the encoder and / or decoder: for example, the current block is a coding unit, and the sub-block is a sub-block within the coding unit; for the sub-block, the corresponding syntax elements and coding information are derived through the syntax elements and coding information of the current block: for example, the aspect ratio of the sub-block is the same as the aspect ratio of the current block.

[0147] Optionally, the coding mode of the sub-block is the same as the coding mode of the current block.

[0148] Optionally, the sub-block does not need to send signaling to indicate the syntax elements or coding information of the sub-block.

[0149] Optionally, the current block includes 4 sub-blocks of the same area; when the current block is divided into 4 sub-blocks of the same size, it means that each sub-block can apply a unified algorithm and parameter setting during encoding and decoding operations, without the need for separate adjustments for different sizes, thereby simplifying the calculation and determination process of the coding information, prediction information and syntax elements of the sub-blocks.

[0150] Optionally, the non-angular predictor is associated with at least one non-angular prediction mode.

[0151] Optionally, for intra-frame prediction and / or inter-frame prediction of graphic blocks, the image blocks can be predicted using an angle prediction mode. However, under natural conditions, the distribution of object pixels within the graphic blocks is not completely arranged at an angle. Therefore, during the intra-frame prediction and / or inter-frame prediction process, the simple angle prediction mode does not have an ideal prediction effect on the image blocks, which in turn leads to poor encoding and decoding quality during the video encoding and / or decoding process.

[0152] Based on this, the technical solution of this embodiment introduces the non-angle prediction mode into the prediction processing of the image block, thereby determining or generating the prediction result of the current block based on the non-angle predictor, which can improve the prediction accuracy of the current block with non-angle arrangement of pixels, and further support the improvement of the encoding and decoding quality in the video encoding and / or decoding process.

[0153] Optionally, the non-angular prediction mode may be a prediction mode other than the angular prediction mode.

[0154] Optionally, in this embodiment, the non-angle prediction mode includes at least one of the first to fifth modes in the second embodiment.

[0155] Optionally, angular prediction mode is a technique used to predict the current pixel block by propagating the values ​​of adjacent pixels along a specific direction to generate a predicted block. This angular prediction mode is primarily used to process directional textures in images, effectively reducing spatial redundancy and improving compression efficiency. The HEVC (High Efficiency Video Coding) standard includes 33 angular prediction modes for intra-frame prediction, covering different angles from horizontal to vertical, ensuring accurate prediction of various texture directions.

[0156] Optionally, the angular prediction mode predicts the pixel values ​​of the current block by selecting a specific angle and extracting values ​​from neighboring pixels along that direction: for example, if a 45-degree angle is selected, the predicted value will be extracted from the neighboring pixels in the upper left or lower right corner.

[0157] Optionally, the non-angle predictor is determined as the prediction result of the current block.

[0158] Optionally, the non-angle predictor and the angle predictor are fused, and the prediction result of the current block is determined or obtained based on the fusion processing result. Since the angle prediction mode is good at processing textures or edges with obvious directionality, and the non-angle prediction mode is suitable for the prediction of smooth areas and / or gradient areas, by fusing these two prediction results, various complex features of the image block can be captured more accurately, thereby improving the overall prediction quality of the current block.

[0159] Optionally, the angle predictor and the non-angle predictor may be fused using weight information, thereby flexibly combining the advantages of different prediction modes and improving prediction accuracy.

[0160] Optionally, the weight information can be the proportion information of the contribution of each angle predictor and non-angle predictor to the final prediction result: for example, the weight information can be a weight coefficient. Optionally, the weight information can also be a weight vector and weight matrix containing weight coefficients, etc., which represent indicative information used to measure the proportion of different elements (for example, predicted values) in the result or decision.

[0161] Optionally, a non-angle predictor of the current block is determined or obtained based on at least one reference area, at least one reference block and / or at least one reference pixel of the current block, and a prediction result of the current block is determined or generated based on the non-angle predictor.

[0162] 6 , when the processing device is an encoder on the encoding side, the encoder can receive video data input from a video source, for example, the encoder receives a video image from a video source, determines an image to be predicted in the video image, divides the image to be predicted into multiple image blocks, and utilizes the temporal and / or spatial correlation between the video images to perform prediction processing on each of the multiple image blocks, including intra-frame prediction processing and / or inter-frame prediction processing, and the intra-frame prediction processing and / or inter-frame prediction processing respectively include multiple prediction modes. For these prediction modes, the encoder uses, for example, rate-distortion cost to determine the prediction mode ultimately adopted by each of the multiple image blocks: for example, the rate-distortion cost corresponding to each prediction mode or the rate-distortion cost of a combination of several prediction modes is calculated to determine the minimum rate-distortion cost from the multiple rate-distortion costs. The prediction mode or combination of prediction modes corresponding to the minimum rate-distortion cost is the prediction mode ultimately adopted by the image block.

[0163] Optionally, the prediction modes include an intra prediction mode and an inter prediction mode, and the intra prediction mode may include an angular prediction mode and a non-angular prediction mode.

[0164] Optionally, the prediction mode of the current block may be determined according to the first information of the current block.

[0165] Optionally, the first information may include reference region information, reference block information, reference template information, reference pixel information, etc. of the current block. After determining the prediction mode of the image block to be predicted (i.e., the current block) using the above method, prediction processing is performed on the image block to be predicted using the prediction mode to determine or obtain a prediction block for the image block to be predicted.

[0166] Optionally, a residual block between the predicted block and the current block may be further calculated, and the residual block may be transformed and quantized before being encoded by an entropy encoder to form an encoded bit stream.

[0167] Optionally, the encoded bitstream may include prediction parameters corresponding to the determined prediction mode and related side information.

[0168] Optionally, the prediction parameters are entropy coded and packed into the coded bitstream.

[0169] Optionally, the prediction parameters include indicative information of the prediction mode.

[0170] Optionally, the transformed and quantized residual block can be added to the corresponding prediction data (such as the prediction block) obtained using the prediction mode after inverse quantization and inverse transformation to obtain a reconstructed block. After obtaining the reconstructed block, the loop filtering module performs loop filtering on the reconstructed block according to the filter control parameters to reduce distortion.

[0171] Optionally, after the loop filtering process is performed, the reconstructed block after the loop filtering process is stored according to the coded image buffer.

[0172] Referring to Figure 7, when the processing device is a decoder on the decoding side, after receiving the encoded bitstream, the decoder's entropy decoding unit parses and decodes the encoded bitstream to obtain transform coefficients. The decoder's inverse transform unit and inverse quantization unit inverse transform and inverse quantize the transform coefficients to obtain a residual block.

[0173] Optionally, the entropy decoding unit of the decoder parses and decodes the encoded bit stream to obtain prediction data, such as prediction parameters and related auxiliary information.

[0174] Optionally, the prediction processing unit of the decoder performs prediction processing using the prediction parameters, thereby determining a prediction block corresponding to the residual block.

[0175] Optionally, the prediction process includes intra-frame prediction process and / or inter-frame prediction process, and the intra-frame prediction process and / or inter-frame prediction process respectively include a manner of combining one or more prediction modes among multiple prediction modes.

[0176] Optionally, the processing method includes: determining a prediction mode of the current block according to first information of the current block.

[0177] Optionally, the processing method further includes: after determining the prediction mode of the image block to be predicted (ie, the current block), performing prediction processing on the image block to be predicted using the prediction mode to determine or obtain a prediction block of the image block to be predicted.

[0178] Optionally, the processing method further includes: adding the obtained residual block and the corresponding prediction block (including the predicted luminance block and the predicted chrominance block) to obtain a reconstructed block.

[0179] Optionally, the processing method further includes: a loop filtering unit of the decoder performs loop filtering on the reconstructed block to reduce distortion and improve video quality.

[0180] Optionally, the processing method further includes: the reconstructed blocks that have undergone loop filtering are further combined into a decoded image and stored in a decoded image buffer or output as a decoded video signal.

[0181] Optionally, when the processing device is an encoder, the initially obtained prediction value may be a prediction value obtained in a corresponding prediction mode, and the prediction value may be directly used in the rate-distortion cost process.

[0182] Optionally, when the processing device is a decoder, the initially obtained prediction value may be a prediction value obtained in a prediction mode corresponding to a block to be predicted (ie, an image block located in a decoding end) indicated by a syntax element obtained by parsing the bitstream.

[0183] Alternatively, the predicted block may be used as the target image block, and a residual block between the target image block and the current block may be calculated. The residual block may then be transformed and quantized, and then encoded by an entropy encoder to form an encoded bitstream. Alternatively, the predicted block may be processed in some corresponding manner, such as by using another model, and the resulting image block may be used as the target image block, and the step of calculating the residual block between the target image block and the current block and subsequent steps may be performed.

[0184] In this embodiment, since the angular prediction mode is good at processing textures or edges with obvious directionality, and the non-angular prediction mode is suitable for the prediction of smooth areas and / or gradient areas, by fusing these two prediction results, various complex features of the image block can be captured more accurately, thereby improving the prediction accuracy of the image block, and further supporting the improvement of the encoding and decoding quality in the video encoding and / or decoding process.

[0185] Second embodiment

[0186] Based on the first embodiment, a second embodiment is proposed.

[0187] In this embodiment, the non-angle prediction mode includes at least one of the following methods 1 to 5:

[0188] Method 1: Intra-frame prediction mode based on neural network;

[0189] Optionally, the NNIP mode is a method of making predictions through a neural network model. A neural network model is a computational model that mimics the way neurons in the human brain process information. It performs various tasks by learning complex patterns and features in the data. The neural network model consists of multiple layers of neurons, including an input layer, at least one hidden layer, and an output layer. Each neuron receives an input signal, performs weighted summation, and then generates an output through a nonlinear activation function. The weights of these connected neurons determine the degree of influence of the input signal on the output, and the bias is used to adjust the activation threshold of the neuron. Activation functions, such as ReLU, Sigmoid, or Tanh, can give the neural network the ability of nonlinear modeling, enabling it to solve problems that linear models cannot solve.

[0190] 8 , the specific implementation process of the NNIP mode may include: determining or obtaining the neural network model corresponding to the current block, using the reconstructed reference pixels of the reference template of the neural network of the current block (for example, reference template 1 on the left, reference template 2 above, and reference template 3 on the upper left in FIG8 ) as the input of the neural network model, and using the association relationship between the current block and its reference template learned by the neural network model to obtain the output of the neural network model, for example, a non-angular predictor, that is, a predicted value.

[0191] Optionally, the reference template includes at least one of a reference area, a reference block and / or a reference pixel.

[0192] Optionally, the reference area may be an image area used to determine or obtain at least one non-angle predictor of the current block, and may be an image area adjacent to the current block, or an image area non-adjacent to the current block.

[0193] Alternatively, the reference region may be a region consisting of coded and reconstructed pixels or blocks from the same frame and / or different frames.

[0194] Optionally, the reference area may include at least one of a pixel, an image block, and an image area used to determine or generate at least one predicted pixel of the current block; since there is a high degree of correlation (for example, temporal correlation, spatial correlation, etc.) between the reference area and the current block, the embodiments of the present application can significantly improve the prediction accuracy of the current block by utilizing these associations.

[0195] Optionally, the reference block may be a pixel block corresponding to the current block extracted from a reference frame (also referred to as a reference image) or a current frame (also referred to as a current image). The reference block may have the same size as the current block, and the reference block may serve as a prediction block for generating the current block. By using the reference block, the encoder may utilize temporal or spatial correlation to reduce redundant information in the current frame, thereby achieving efficient compression.

[0196] Optionally, at least one reference block of the current block is determined or obtained according to an index obtained from the bitstream.

[0197] Optionally, a reference frame refers to a frame (also called an image) that has been encoded and reconstructed, which can be a forward reference frame (i.e., a frame / image that is located before the current frame in the playback order) or a backward reference frame (i.e., a frame / image that is located after the current frame in the playback order).

[0198] Optionally, based on the NNIP mode and at least one reference area, at least one reference block and / or at least one reference pixel of the current block, at least one non-angle predictor of the current block is determined or obtained, and based on the at least one non-angle predictor, a prediction result of the current block is determined or generated.

[0199] Optionally, the neural network model may include: an input layer, a hidden layer, a dropout layer, and an output layer.

[0200] Optionally, the input layer is the first layer that receives input data. In this layer, each node (or neuron) generally represents a feature of the input data.

[0201] Optionally, hidden layers are located between the input layer and the output layer. There can be multiple hidden layers, which process the input data in a weighted manner.

[0202] Optionally, the hidden layer includes at least one of a fully connected layer, a convolutional layer, a pooling layer, a recursive layer, and a dropout layer.

[0203] Optionally, each neuron in a dense or fully connected layer is connected to every neuron in the previous layer.

[0204] Optionally, a convolutional layer is used to extract local features from the input data and is commonly used in image processing.

[0205] Optionally, a pooling layer is used for downsampling to reduce the amount of data and computation.

[0206] Optionally, a recurrent layer such as LSTM or GRU is used to process sequence data.

[0207] Optionally, a dropout layer is used to randomly "drop out" (i.e., temporarily set the output to zero) some neurons during model training. This can reduce the complex co-adaptation relationship between neurons, improve the generalization ability of the model, and reduce the risk of overfitting.

[0208] Optionally, dropout layers can be added between multiple fully connected layers to prevent the neurons in these layers from becoming overly dependent on the specific outputs of the previous layer, thereby enhancing the generalization ability of the model. Dropout layers can be added between convolutional layers and fully connected layers to help reduce overfitting introduced by the fully connected layers. The output layer is the layer that generates the final prediction results.

[0209] Optionally, in this embodiment, the reference area, reference block and / or reference pixel is determined or obtained according to at least one of Method D to Method K in the third embodiment.

[0210] Optionally, a first neural network model applied to the NNIP mode can be determined or obtained from a group of neural network models; the current block and / or at least one sub-block of the current block is predicted and processed according to the first neural network model. Optionally, the first neural network model can be one or at least one model.

[0211] Optionally, a first neural network model is determined or obtained from a group of neural network models based on at least one reference area of ​​the current block and / or at least one sub-block of the current block, at least one reference block and / or at least one reference pixel, and a preset model mapping / correspondence rule.

[0212] Optionally, a set of neural network models are pre-selected and trained, which may include at least one of: a neural network model based on a fully connected layer, a neural network based on a convolutional layer, and a neural network based on a hybrid convolutional and fully connected layer.

[0213] Optionally, a neural network model based on fully connected layers is also called a multilayer perceptron (MLP); the model includes: an input layer, one or more hidden layers (i.e., fully connected layers) and an output layer, each layer contains a certain number of neurons, and all neurons between two adjacent layers are interconnected, i.e., "fully connected" between layers. Optionally, a dropout layer can be set between at least one group of fully connected layers of the neural network model based on fully connected layers to help reduce overfitting introduced by the fully connected layers.

[0214] 9 , the neural network model based on the fully connected layer includes: an input layer, multiple hidden layers (i.e., fully connected layers) and an output layer, and the fully connected layer includes: a linear layer (Linear layer) and an activation function layer (e.g., a LeakyReLU layer); the pixels of the reference template of the current block and / or at least one sub-block of the current block are input into the neural network model based on the fully connected layer; since the pixels of the reference template may exist in the form of a two-dimensional or three-dimensional matrix, in order to be able to feed the above data into the fully connected layer for processing, the pixels of at least one reference template may be flattened through the input layer to flatten into a one-dimensional vector f, and a nonlinear transformation is performed through the ReLU activation function to determine or obtain f i ; This operation introduces nonlinearity, allowing the model to learn more complex patterns, f i It will enter the network composed of fully connected layers, each layer performs linear transformation and activation function processing, and outputs a new feature vector f r , to extract higher-level abstract features from the input, and then transform the high-level features f r With low-level features f i Splice in the channel dimension to generate the feature vector f c , the eigenvector f c It is fed into the linear layer and outputs the non-angle predictor of the current block through the output layer.

[0215] Optionally, a convolutional neural network, i.e., a convolutional neural network (CNNs), includes: an input layer, at least one hidden layer, and an output layer, wherein the hidden layer includes: a convolutional layer for extracting features; each convolutional layer may be followed by a pooling layer for reducing the spatial size and reducing the computational complexity, and each convolutional layer may also be followed by a nonlinear activation function (e.g., ReLU) to introduce nonlinear factors so that the network can learn more complex patterns; as the depth of the convolutional neural network increases, the convolutional neural network can automatically learn feature representations from low-level to high-level, thereby improving prediction accuracy.

[0216] 10 , the convolutional neural network includes: an input layer, multiple hidden layers and an output layer, wherein the hidden layer includes: a convolutional layer, an activation function layer (e.g., a LeakyReLU layer) and a linear layer (Linear layer); the pixels of the reference template of the current block are input into the convolutional neural network, and a convolution operation is performed on the pixels of at least one reference template through at least one convolutional layer, and a nonlinear transformation is performed through the ReLU activation function after each convolutional layer to obtain the feature f 1 、f 2 and f 3 . The features f1, f 2 and f 3The fusion splicing is performed on the channel dimension to determine or generate a new merged feature f containing three features to retain information from different convolution paths, allowing the model to combine low-level and high-level features and enhance representation capabilities. The fused feature f is processed again through the ReLU activation function to obtain f i ; This additional activation step helps to further improve the expressiveness and flexibility of the model. i Send it to the linear layer to get the non-angle predictor of the current block.

[0217] Optionally, a neural network model based on hybrid convolution and fully connected layers is an architecture that combines the advantages of CNN and MLP, including a convolution layer part and a fully connected layer part. The convolution layer can automatically learn the spatial hierarchical structure of the input data, such as edges, textures, shapes and other features in the image; the convolution operation can effectively capture local patterns and reduce the complexity of the model through a parameter sharing mechanism; the fully connected layer can flatten the features extracted by the convolution layer into a one-dimensional vector, and then send it to a series of fully connected layers, which are responsible for combining low-level features and determining or generating higher-level abstract representations. The neural network model based on hybrid convolution and fully connected layers can combine the advantages of two different types of layers, which can not only efficiently extract features from the input data, but also make accurate predictions based on the features.

[0218] Optionally, the neural network model based on hybrid convolutional and fully connected layers includes: an input layer, at least one convolutional layer, at least one fully connected layer and an output layer.

[0219] Referring to Figure 11, the neural network model based on the hybrid convolution and fully connected layer includes: an input layer, multiple hidden layers and an output layer. The hidden layer includes: a convolutional layer, a fully connected layer (i.e., a linear layer) and an activation function layer (e.g., a LeakyReLU layer). The pixels of the reference template of the current block are input into the neural network model based on the hybrid convolution and fully connected layer. The pixels of at least one reference template are convolved through at least one convolution layer, and a nonlinear transformation is performed through the ReLU activation function after each convolution layer to obtain the feature f 1 、f 2 and f 3 , the feature f 1 、f 2 and f 3 Perform fusion splicing on the channel dimension to determine or generate a new merged feature f containing three features 4 , to retain information from different convolution paths, allowing the model to combine low-level and high-level features, enhance representation capabilities, and obtain feature f through multiple stacked linear layers 5 , the feature f 4 and f 5Splicing is performed on the channel dimension to form a fusion feature f i , through at least one convolutional layer to fusion feature f i Perform convolution processing to obtain the non-angle predictor of the current block.

[0220] Optionally, the construction of the neural network model mainly includes: data set construction and neural network construction and training.

[0221] Optionally, during the dataset construction process, the ECM code library can be used to extract data used for intra-frame coding in the video encoding and decoding process, and the dataset can be constructed based on this data. The dataset contains the following fields: frame index (poc), image block (e.g., coding unit) coding unit position information (x, y), image block size (width, height), channel type (compidx), split angle (splitdir), pixel value offset (shiftWeighted) and compensation (offsetWeighted), the original image pixel value of the image block (org), and the pixel value of the reference block of the image block. After the dataset is constructed, it needs to be further processed and divided to meet the input requirements of the neural network.

[0222] Optionally, define the predicted pixel of the neural network as p ij , let the target label of the predicted pixel be the original pixel p ij =org ij . Define the fusion matrix k i,j Indicates that the weights of each pixel of src0 (reference block 0) and src1 (reference block 1) are fused using the GPM tool. The fused predicted pixels satisfy: p i,j =k i,j src0 i,j +(1-k i,j )·src1 i,j ; Let the target label of the predicted pixel be the original pixel value p i,j =org i,j , the target label of the fusion matrix is ​​solved to satisfy:

[0223] Optionally, based on the position and size of the current block, a reference area adjacent to the current block is extracted from the decoded reconstructed frame, which may also be called a reference template, including three areas: top, left, and topleft, as shown in FIG8 .

[0224] Optionally, during the construction and training process of the neural network model, the neural network function can be designed first. For example, the predicted value of the current block is predicted based on the reference pixels in the frame, the reconstructed reference pixels in the reference templates on the left side left, top above, and topleft of the upper left of the current block are used as input, and the predicted value p of the current block is used as output. The target label of the neural network model is that the predicted value p of the current block is equal to the original pixel value corresponding to the current block.

[0225] In this embodiment, the neural network model has powerful nonlinear modeling capabilities and can automatically learn the complex mapping relationship between input and output from a large amount of data. Therefore, the neural network can identify and utilize the subtle connection between the current block and its reference template: for example, effectively capture the temporal correlation between frames and the spatial correlation within frames, understand the continuity and change patterns between different frames or adjacent regions within the same frame; and / or perceive the consistency of local features such as edges and textures, even if these features are displaced or deformed between different frames. Through multi-level feature extraction, it is ensured that the prediction results are not only accurate in local details, but also consistent with the actual content in the overall layout, so as to improve the accuracy of the current block prediction from multiple dimensions and improve the encoding and decoding quality during video encoding and / or decoding.

[0226] Optionally, the first model is applied to a neural network-based intra-frame prediction mode. Optionally, the first model is a neural network model.

[0227] Optionally, the first model is determined or obtained according to at least one of the following methods A to C:

[0228] Mode A: at least one of the width, height, block size, and block area of ​​the current block;

[0229] Optionally, according to at least one of the width, height, size and area of ​​the current block, a first model corresponding to the target model type is determined from at least one model type.

[0230] Optionally, based on at least one of the geometric properties of the current block (i.e., width, height, block size, and block area), a model type that is most suitable for processing these features is selected, so that the first model can more accurately reflect the characteristics of the input data and improve the prediction effect.

[0231] Optionally, the structure of the first model is determined according to at least one of the width, height, size and area of ​​the current block.

[0232] Optionally, since there is an association between at least one of the width, height, block size and block area of ​​the current block and at least one reference region and / or at least one reference block of the current block, a first model suitable for processing the reference region and / or reference block data of the current block can be determined or obtained through at least one of the width, height, size and area of ​​the current block.

[0233] Optionally, the specific structure of the selected model is adjusted according to at least one of the geometric properties of the current block (i.e., width, height, block size and block area). For example, when the first model belongs to a neural network model, the number of hidden layers (e.g., convolutional layers, fully connected layers), filter size, pooling strategy, etc. can be adjusted according to at least one of the width, height, size and area of ​​the current block.

[0234] Optionally, the structure of the first model may be determined according to the width and height of the current block and the first mapping table.

[0235] Optionally, the first mapping table may be as shown in Table 1 below:

[0236] Table 1

[0237] Optionally, the structure of the first model may be determined according to the block size of the current block and the second mapping table.

[0238] Optionally, the second mapping table may be as shown in Table 2 below:

[0239] Table 2

[0240] Optionally, the structure of the first model may be determined according to the block size of the current block and the third mapping table.

[0241] Optionally, the third mapping table may be as shown in Table 3 below:

[0242] Table 3

[0243] Optionally, if the current block, at least one reference area of ​​the current block and / or at least one reference block do not match all neural network models, the reconstructed reference pixels of the current block, at least one reference area of ​​the current block and / or at least one reference block can be transposed and / or downsampled so that the geometric properties of the current block, at least one reference area of ​​the current block and / or at least one reference block match the neural network model (for example, the same size), and then the reconstructed reference pixels obtained by transposition and / or downsampling are input into the neural network model to obtain the prediction result of the current block. In this way, more current blocks, reference areas and / or reference blocks can be matched to the target model or target model type by transposition or downsampling, thereby reducing the number of target models or target model types as a whole.

[0244] Optionally, if the current block includes at least one sub-block, and the prediction result of the current block is determined or generated by a non-angular predictor of each sub-block, then the at least one reference area and / or at least one reference block of the above-mentioned current block can be replaced by at least one reference area and / or at least one reference block of at least one sub-block of the current block.

[0245] Optionally, the number of neural network models is N, for example, including neural network model NN0, neural network model NN1, neural network model NN2... neural network model NN N-1 Neural network model NN0, neural network model NN1, neural network model NN2, ..., neural network model NN N-1 Corresponding to block sizes of W0xH0, W1xH1, W2xH2, ..., W N-1 xH N-1 .

[0246] In this embodiment, based on at least one of the width, height, block size and block area of ​​the current block, a model that is most suitable for processing these features is determined or obtained, so that the first model can more accurately reflect the characteristics of the input data, improve the prediction accuracy of the current block, and further support improving the prediction effect in the video encoding and / or decoding process.

[0247] Mode B: at least one of the width, height, size, and area of ​​at least one reference region of the current block;

[0248] Optionally, a first model corresponding to the target model type is determined from at least one model type according to at least one of width, height, size and area of ​​at least one reference region of the current block.

[0249] Optionally, the at least one reference region of the current block may be at least one reference region of at least one sub-block of the current block.

[0250] Optionally, based on at least one of the geometric properties of at least one reference area of ​​the current block (i.e., width, height, block size and block area), a model type that is most suitable for processing these features is selected, so that the first model can more accurately reflect the characteristics of the input data and improve the prediction effect.

[0251] Optionally, the structure of the first model is determined according to at least one of width, height, size and area of ​​at least one reference region of the current block.

[0252] Optionally, the specific structure of the selected model is adjusted according to at least one of the geometric properties of at least one reference area of ​​the current block (i.e., width, height, block size and block area). For example, when the first model belongs to a neural network model, the number of hidden layers (e.g., convolutional layers, fully connected layers), filter size, pooling strategy, etc. can be adjusted according to at least one of the width, height, size and area of ​​at least one reference area.

[0253] Optionally, the structure of the first model may be determined according to the width and height of at least one reference area of ​​the current block and the fourth mapping table.

[0254] Optionally, the fourth mapping table may be as shown in Table 4 below:

[0255] Table 4

[0256] Optionally, the structure of the first model may be determined according to the block size of at least one reference area of ​​the current block and the fifth mapping table.

[0257] Optionally, the fifth mapping table may be as shown in Table 5 below:

[0258] Table 5

[0259] Optionally, the block size includes at least one of the width, height, ratio, depth, area, resolution, and number of pixels of the block. Optionally, X4 to X7 can be preset thresholds corresponding to at least one of the width, height, ratio, depth, area, resolution, and number of pixels in the block size.

[0260] Optionally, the structure of the first model may be determined according to the block size of at least one reference area of ​​the current block and the sixth mapping table.

[0261] Optionally, the sixth mapping table may be as shown in Table 6 below:

[0262] Table 6

[0263] In this embodiment, based on at least one of the width, height, size and area of ​​at least one reference region of the current block, a model that is most suitable for processing these features is determined or obtained, so that the first model can more accurately reflect the characteristics of the input data, improve the prediction accuracy of the current block, and further support improving the prediction effect in the video encoding and / or decoding process.

[0264] Mode C: at least one of the width, height, size, and area of ​​at least one reference block of the current block.

[0265] Optionally, a first model corresponding to the target model type is determined from at least one model type according to at least one item of width, height, size and area of ​​at least one reference block of the current block.

[0266] Optionally, the at least one reference block of the current block may be at least one reference block of at least one sub-block of the current block.

[0267] Optionally, based on at least one of the geometric properties of at least one reference block of the current block (i.e., width, height, block size and block area), a model type that is most suitable for processing these features is selected, so that the first model can more accurately reflect the characteristics of the input data and improve the prediction effect.

[0268] Optionally, the structure of the first model is determined according to at least one of width, height, size and area of ​​at least one reference block of the current block.

[0269] Optionally, the specific structure of the selected model is adjusted according to at least one of the geometric properties of at least one reference block of the current block (i.e., width, height, block size and block area). For example, when the first model belongs to a neural network model, the number of hidden layers (e.g., convolutional layers, fully connected layers), filter size, pooling strategy, etc. can be adjusted according to at least one of the width, height, size and area of ​​at least one reference block.

[0270] Optionally, the structure of the first model may be determined according to the width and height of at least one reference block of the current block and the seventh mapping table.

[0271] Optionally, the seventh mapping table may be as shown in Table 7 below:

[0272] Table 7

[0273] Optionally, the structure of the first model may be determined according to the block size of at least one reference block of the current block and an eighth mapping table.

[0274] Optionally, the eighth mapping table may be as shown in Table 8 below:

[0275] Table 8

[0276] Optionally, the block size includes at least one of the width, height, ratio, depth, area, resolution, and number of pixels of the block. Optionally, X4 to X7 can be preset thresholds corresponding to at least one of the width, height, ratio, depth, area, resolution, and number of pixels in the block size.

[0277] Optionally, the structure of the first model may be determined according to the block size of at least one reference block of the current block and a ninth mapping table.

[0278] Optionally, the ninth mapping table may be as shown in Table 9 below:

[0279] Table 9

[0280] In this embodiment, based on at least one of the width, height, size and area of ​​at least one reference block of the current block, a model that is most suitable for processing these features is determined or obtained, so that the first model can more accurately reflect the characteristics of the input data, improve the prediction accuracy of the current block, and further support improving the prediction effect in the video encoding and / or decoding process.

[0281] Method 2: Intra-frame prediction mode based on extrapolation filter;

[0282] Optionally, an intra-frame prediction mode based on an extrapolation filter (i.e., EIP mode) has the basic principle of deriving filter coefficients based on adjacent reconstructed luminance pixels or using filter coefficients inherited from an encoded luminance block and predicting the luminance pixels of the current block.

[0283] Optionally, the prediction process of the EIP mode may include: determining or obtaining reference pixels of the current block, where the reference pixels are usually located above and to the left of the current block and have been encoded and reconstructed; determining or obtaining filter coefficients applicable to the current block from these pixels through a preset algorithm (such as minimum mean square error estimation, etc.) based on the spatial correlation of the reference pixels; and / or directly using the filter coefficients used by the encoded adjacent luminance blocks, and then applying the filter coefficients to the reference pixels, and generating a prediction value of the current block through a convolution operation.

[0284] Optionally, at least one non-angle predictor of the current block is determined or obtained based on the EIP mode and at least one reference area, at least one reference block and / or at least one reference pixel of the current block, and a prediction result of the current block is determined or generated based on the at least one non-angle predictor.

[0285] Optionally, in this embodiment, the reference area, reference block and / or reference pixel is determined or obtained according to at least one of Method D to Method K in the third embodiment.

[0286] In this embodiment, the EIP mode is used to generate a non-angular predictor of the current block using filter coefficients derived or inherited from adjacent reconstructed pixels, and further determine the prediction result of the current block, thereby more accurately capturing the complex texture and edge features in the image. In particular, for areas with directional texture and rich details, the EIP mode can better adapt to their characteristics, reduce prediction errors, and improve the prediction accuracy of the current block.

[0287] Mode three, block vector prediction mode;

[0288] Optionally, the block vector prediction mode (i.e., BV mode) is a method for achieving efficient intra-frame prediction by searching for an encoded block similar to the current block in the current frame and using a block vector (BV) to describe the displacement relationship between the current block and the reference block.

[0289] Optionally, the prediction process of the BV mode may include: using a block matching algorithm to search for a reference block that is most similar to the current block in the encoded area of ​​the current frame, and calculating the displacement between the current block and the reference block, that is, the block vector (BV); BV is usually expressed with integer pixel precision to describe the relative position of the current block to the reference block, and then according to the BV, extracting pixel values ​​from the reference block as the non-angular predictor of the current block.

[0290] Optionally, based on the BV mode and at least one reference area, at least one reference block and / or at least one reference pixel of the current block, at least one non-angle predictor of the current block is determined or obtained, and based on the at least one non-angle predictor, a prediction result of the current block is determined or generated.

[0291] Optionally, in this embodiment, the reference area, reference block and / or reference pixel is determined or obtained according to at least one of Method D to Method K in the third embodiment.

[0292] In this embodiment, the BV mode effectively utilizes spatial redundancy in screen content to obtain prediction results. This method is particularly suitable for screen content coding, as screen content often contains a large number of repeated patterns and characters. The BV mode can efficiently capture these repetitive structures, reducing the number of bits required for coding. Furthermore, by limiting the reference area to the already coded portion of the current frame, the BV mode avoids the complex motion estimation and pixel-by-pixel interpolation required in inter-frame prediction, thereby reducing coding complexity.

[0293] Mode 4: Plane mode;

[0294] Optionally, the planar mode is a method of predicting the pixel values ​​of the current block by linear interpolation in both horizontal and vertical directions, thereby generating a smooth predicted image.

[0295] Optionally, the prediction process of the planar mode may include: obtaining reference pixel values ​​from the left and above the current block, calculating linear interpolation prediction values ​​in the horizontal and vertical directions respectively, and averaging the horizontal and vertical prediction values ​​to obtain a final prediction value.

[0296] Optionally, at least one non-angle predictor of the current block is determined or obtained based on the planar mode and at least one reference area, at least one reference block and / or at least one reference pixel of the current block, and a prediction result of the current block is determined or generated based on the at least one non-angle predictor.

[0297] In this embodiment, the planar mode used is simple to calculate and has low complexity. It is particularly suitable for processing large flat areas or gradient textures, such as scenes such as the sky and water surface, and can effectively improve encoding and decoding efficiency.

[0298] Mode five, DC mode.

[0299] Optionally, the basic principle of the DC mode is to assume that all pixel values ​​in the current block are the same, and this value is equal to the average of the adjacent reconstructed pixels. Optionally, for a square block, the predicted value is the average of the left and top reference pixels; for a rectangular block, the predicted value is the average of the long side.

[0300] Optionally, the prediction process of the DC mode may include: determining the reference pixel based on the reconstructed pixels on the left and top of the current block; if the current block is a square, the prediction value is equal to the average of the reference pixels on the left and top; if the current block is a rectangle, the prediction value is equal to the average of the long side; the calculated prediction value is assigned to all pixels in the current block to generate a predicted block.

[0301] Optionally, at least one non-angle predictor of the current block is determined or obtained based on the DC mode and at least one reference area, at least one reference block and / or at least one reference pixel of the current block, and a prediction result of the current block is determined or generated based on the at least one non-angle predictor.

[0302] Optionally, in this embodiment, the reference area, reference block and / or reference pixel is determined or obtained according to at least one of Method D to Method K in the third embodiment.

[0303] In this embodiment, the DC mode can avoid complex prediction calculations when processing image regions with simple textures, simplify the encoding process, and reduce computational complexity.

[0304] Third embodiment

[0305] Based on any of the above embodiments, a third embodiment is proposed.

[0306] In this embodiment, the reference area and / or reference block is determined or obtained by at least one of the following methods D to K:

[0307] Mode D, based on at least one of the upper adjacent pixel, upper non-adjacent pixel, left adjacent pixel, left non-adjacent pixel, upper left adjacent pixel, and upper left non-adjacent pixel of the current block;

[0308] Optionally, the reference area of ​​the current block is determined or obtained based on at least one of the upper adjacent pixels, upper non-adjacent pixels, left adjacent pixels, left non-adjacent pixels, upper left adjacent pixels and upper left non-adjacent pixels of the current block.

[0309] Optionally, the reference block of the current block is determined or obtained based on at least one of the upper adjacent pixels, upper non-adjacent pixels, left adjacent pixels, left non-adjacent pixels, upper left adjacent pixels and upper left non-adjacent pixels of the current block.

[0310] Optionally, the upper adjacent pixel may be a pixel located above and adjacent to the current block in the same frame image.

[0311] Optionally, the upper non-adjacent pixel may be a pixel in the same frame image that is located above the current block but is not adjacent to the current block.

[0312] Optionally, the left adjacent pixel may be a pixel located on the left side of and adjacent to the current block in the same frame image.

[0313] Optionally, the left non-adjacent pixel may be a pixel located on the left side of the current block but not adjacent to the current block in the same frame image.

[0314] Optionally, the upper left adjacent pixel may be a pixel located in the upper left of the current block and adjacent in the same frame image.

[0315] Optionally, the upper left non-adjacent pixel may be a pixel in the same frame image that is located above the left of the current block but is not adjacent to the current block.

[0316] Optionally, at least one of the upper adjacent pixel, the upper non-adjacent pixel, the left adjacent pixel, the left non-adjacent pixel, the upper left adjacent pixel and the upper left non-adjacent pixel may be a reconstructed pixel or a predicted pixel.

[0317] Optionally, at least one of the upper adjacent pixels, upper non-adjacent pixels, left adjacent pixels, left non-adjacent pixels, upper left adjacent pixels and upper left non-adjacent pixels can be directly used as a reference area and / or reference block, or at least one acquired pixel can be deduced or calculated to obtain a reference area and / or reference block.

[0318] Optionally, the reference area and / or reference block of the current block may be acquired according to a preset mapping / correspondence rule.

[0319] Optionally, a reference area and / or reference block can be selected from the upper adjacent pixels, upper non-adjacent pixels, left adjacent pixels, left non-adjacent pixels, upper left adjacent pixels and upper left non-adjacent pixels of the current block according to the prediction model.

[0320] Optionally, if some locations lack valid pixel data, they can be filled with adjacent valid pixels.

[0321] Optionally, in order to further improve the prediction accuracy of the current block, the current block may be divided into at least one sub-block, and the prediction result of the current block may be determined or generated based on the non-angular predictor of each sub-block.

[0322] Optionally, the current block may be divided into at least one sub-block to obtain at least one sub-block of the current block.

[0323] Optionally, when the block size of the current block is greater than or equal to a first threshold (e.g., 128), the current block may be divided into at least one sub-block; if the block size of the current block is less than the first threshold, prediction processing may be performed directly without division.

[0324] Optionally, the determined or obtained block size of the at least one sub-block may be at least partially the same, or may be completely different.

[0325] Optionally, the block size may include width, height, scale, depth, area, resolution, number of pixels, etc. of the block.

[0326] Optionally, the current block may be divided into multiple sub-blocks of the same size by uniform division.

[0327] Optionally, the current block may be divided into a plurality of sub-blocks of unequal sizes by non-uniform division.

[0328] Optionally, the current block may be divided into four sub-blocks of the same size by quadtree partitioning.

[0329] Optionally, at the encoding end, the video image will be divided into multiple image blocks, and rate-distortion optimization will be performed on these image blocks to determine the optimal image block division mode and prediction mode. In the process of dividing the image blocks, a quadtree division method is usually adopted to divide an image block into four next-level image blocks, and then all prediction modes are traversed for each next-level image block. During this process, the next-level image block will calculate the prediction results under each mode, and the prediction results include the prediction results of DIMD mode, OBIC mode, TIMD mode, LIC (Local Illumination Compensation, local illumination compensation) mode, NNIP mode, etc. After an image block is divided into multiple image blocks, if this division method is adopted, this image block will be replaced by multiple image blocks. The processing method proposed in this application can at least partially reuse the prediction results of these next-level image blocks without increasing the amount of additional calculations.

[0330] Optionally, a reference area and / or reference block of at least one sub-block of the current block can be determined or generated based on at least one of the upper adjacent pixels, upper non-adjacent pixels, left adjacent pixels, left non-adjacent pixels, upper left adjacent pixels and upper left non-adjacent pixels of at least one sub-block of the current block.

[0331] Optionally, the reference template includes a reference block and / or a reference area. Optionally, the reference template is used to determine or obtain the angle predictor and / or non-angle predictor of the current block; the reference template is also used to determine or obtain the angle prediction mode and / or non-angle prediction mode of the current block.

[0332] Optionally, at least one non-angle prediction mode and at least one angular prediction mode of the current block can be determined or obtained based on the reference template of the current block, and at least one non-angle predictor of the current block can be determined or generated based on the at least one non-angle prediction mode and the reference template, and at least one angular predictor of the current block can be determined or obtained based on the at least one angular prediction mode and the reference template, and then the prediction result of the current block can be determined or obtained based on the at least one angular predictor and the at least one non-angle predictor.

[0333] Optionally, at least one non-angle prediction mode and at least one angular prediction mode of at least one sub-block of the current block can be determined or obtained based on a reference template of at least one sub-block of the current block, and at least one non-angle predictor of at least one sub-block can be determined or generated based on at least one non-angle prediction mode of at least one sub-block and a reference template of at least one sub-block, and at least one angular predictor of at least one sub-block can be determined or obtained based on at least one angular prediction mode of at least one sub-block and a reference template of at least one sub-block, and then at least one sub-block prediction result of the current block can be determined or obtained based on the at least one angular predictor and the at least one non-angle predictor, and the prediction result of the current block can be determined or obtained based on the prediction result of at least one sub-block.

[0334] Optionally, in an embodiment of the present application, when performing prediction deduction mode processing, the current block can be divided into at least one sub-block for prediction processing. Optionally, when performing prediction deduction mode processing, the angle prediction mode and / or non-angle prediction mode of the current block can be determined or obtained by referring to a template. Optionally, the prediction deduction mode includes at least one of a DIMD mode, an OBIC mode and a TIMD mode.

[0335] Optionally, prediction processing may be performed on each sub-block of the current block using the same prediction derivation mode.

[0336] Optionally, prediction processing may be performed on each sub-block of the current block using different prediction derivation modes.

[0337] Optionally, prediction processing may be performed on each sub-block of the current block using the same non-angular prediction mode.

[0338] Optionally, prediction processing may be performed on each sub-block of the current block using different non-angular prediction modes.

[0339] Optionally, when using the same prediction derivation mode (e.g., DIMD mode, OBIC mode, and TIMD mode), each sub-block of the current block can be further predicted using the same or different angular prediction modes and / or non-angular prediction modes.

[0340] Optionally, if the current block is divided into 4 sub-blocks, and it is determined through rate-distortion optimization that the DIMD mode is adopted for all 4 sub-blocks, then in the conventional method, a separate signaling needs to be sent for each sub-block to indicate the adoption of the mode. In this embodiment, by adopting the same prediction mode for each sub-block, only one signaling is required to indicate the prediction mode of the current block, which greatly saves the signaling overhead in the encoding and decoding process.

[0341] Optionally, in DIMD mode, TIMD mode and / or OBIC mode, at least one angle prediction mode and / or at least one non-angle prediction mode of the current block can be determined or obtained based on at least one reference template of the current block, at least one angle predictor can be determined or obtained based on at least one reference template and at least one angle prediction mode of at least one sub-block of the current block, at least one non-angle predictor can be determined or obtained based on at least one reference template and at least one non-angle prediction mode of at least one sub-block of the current block, and the prediction result of the current block can be determined or obtained based on at least one non-angle predictor and / or at least one angle predictor.

[0342] Optionally, the basic principle of the DIMD mode may be to derive the intra-frame prediction mode of the current block by analyzing the magnitude and direction of adjacent pixel gradients, or the usage of intra-frame prediction modes of adjacent image blocks / non-adjacent image blocks.

[0343] Optionally, the implementation process of the DIMD mode may include: determining the gradient amplitude value and gradient direction of the pixels in the DIMD template, determining the gradient histogram or statistical results of the intra-frame prediction direction of the DIMD template, and determining the intra-frame prediction direction of the block to be predicted based on the gradient histogram.

[0344] Optionally, the basic principle of the OBIC mode may be to determine the intra prediction mode of the current block by analyzing usage of intra prediction modes used by adjacent coding blocks and / or non-adjacent coding blocks.

[0345] Optionally, the implementation process of the OBIC mode may include: determining the usage of the intra-frame prediction mode of multiple image blocks, calculating the area amplitude values ​​and intra-frame prediction directions of multiple image blocks, determining the area amplitude histogram or statistical results of the intra-frame prediction directions of multiple image blocks, and determining the intra-frame prediction direction of the block to be predicted based on the area amplitude histogram or statistical results.

[0346] Optionally, determining the intra-frame prediction mode usage of multiple encoded blocks includes first determining a coded area or a decoded area; then determining the intra-frame prediction mode usage of the coded blocks in the coded area, or determining the intra-frame prediction mode usage of the decoded blocks in the decoded area, optionally, the coding unit includes coding blocks of three color components, the three color components include a luminance component, and two chrominance components.

[0347] Optionally, the basic principle of the TIMD mode is to generate a prediction value for the template by using each mode in the MPM (Most Probable Mode), and then calculate the SATD (Sum of Absolute Transformed Differences) of the template prediction value and the reconstructed value, and select the mode with the smallest SATD as the TIMD mode, and use it for the prediction of the current block.

[0348] 12 , the implementation process of the TIMD mode may include: determining a template for the block to be predicted; constructing a template using reconstructed pixels on the left or above the block to be predicted, generating a prediction value for the template by using each mode in the MPM combined with the reference pixels of the template, and selecting the mode with the smallest SATD value from the MPM mode for prediction based on the SATD of the predicted value and the reconstructed value.

[0349] Exemplarily, the current block includes at least one sub-block. When prediction processing is performed on at least one sub-block of the current block through the DIMD mode, the selection of the reference template of the current block can refer to Figure 12; the solid line is the current block, and it is divided into sub-block A, sub-block B, sub-block C and sub-block D, and the reference templates of sub-blocks A to D are determined based on the adjacent pixels above, to the left and / or to the upper left of the current block, and are all outside the current block.

[0350] Exemplarily, the current block includes at least one sub-block. When at least one sub-block of the current block is predicted through the TIMD mode, the selection of the reference template of the current block can refer to Figure 13; the solid line is the current block, and it is divided into sub-block A, sub-block B, sub-block C and sub-block D, and the reference templates of sub-blocks A to D are determined based on the adjacent pixels above, to the left and / or to the upper left of the current block, and are all outside the current block.

[0351] Optionally, the reference area and / or reference block of each sub-block of the current block can be determined or obtained based on at least one of the upper adjacent pixels, upper non-adjacent pixels, left adjacent pixels, left non-adjacent pixels, upper left adjacent pixels and upper left non-adjacent pixels of each sub-block. That is, the reference area and / or reference block of each sub-block may include: an area located inside the current block and / or an area located outside the current block.

[0352] Optionally, in this embodiment, the non-angle prediction mode includes at least one of the first to fifth modes in the second embodiment.

[0353] In this embodiment, since the current block usually has a high similarity with at least one of the upper adjacent pixels, upper non-adjacent pixels, left adjacent pixels, left non-adjacent pixels, upper left adjacent pixels and upper left non-adjacent pixels of the current block, the reference area and / or reference block determined based on these adjacent pixels and / or non-adjacent pixels can improve the prediction accuracy of the current block.

[0354] Mode E, based on at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, time domain blocks, and default blocks corresponding to the current block;

[0355] Optionally, a reference area of ​​at least one subblock of the current block is determined or obtained according to image block information of at least one of a neighbor block, a non-neighbor block, a co-located block, a time domain block and a default block corresponding to the current block.

[0356] Optionally, a reference block of at least one subblock of the current block is determined or obtained according to image block information of at least one of a neighbor block, a non-neighbor block, a co-located block, a time domain block and a default block corresponding to the current block.

[0357] Optionally, at least one of a neighbor block, a non-neighbor block, a co-located block, a time domain block and a default block corresponding to the current block is determined as a reference area of ​​at least one sub-block of the current block.

[0358] Optionally, at least one of a neighbor block, a non-neighbor block, a co-located block, a time domain block and a default block corresponding to the current block is determined as a reference block of at least one subblock of the current block.

[0359] Optionally, the image block information may include at least one of block size, block area, image block attribute and image block type.

[0360] Optionally, the block size includes the width, height, scale, depth, area, resolution, and number of pixels of the block. The image block attributes may include the position and / or image texture of the block. The image block type may include a natural image or a screen content image.

[0361] Optionally, the reference area of ​​at least one sub-block of the current block can be determined or obtained based on at least one of the upper adjacent pixels, upper non-adjacent pixels, left adjacent pixels, left non-adjacent pixels, upper left adjacent pixels and upper left non-adjacent pixels of at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, time domain blocks and default blocks corresponding to the current block.

[0362] Optionally, a reference block of at least one sub-block of the current block can be determined or obtained based on at least one of the upper adjacent pixels, upper non-adjacent pixels, left adjacent pixels, left non-adjacent pixels, upper left adjacent pixels and upper left non-adjacent pixels of at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, time domain blocks and default blocks corresponding to the current block.

[0363] Optionally, the reference area of ​​at least one sub-block of the current block can be determined or obtained based on at least one of the width, height, block size and block area of ​​at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, time domain blocks and default blocks corresponding to the current block.

[0364] Optionally, a reference block of at least one sub-block of the current block can be determined or obtained based on at least one of the width, height, block size and block area of ​​at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, time domain blocks and default blocks corresponding to the current block.

[0365] Optionally, the neighbor block may be a block adjacent to the current block, and may be a block that has been predicted or reconstructed.

[0366] Alternatively, the non-neighbor block may be a block that is not adjacent to the current block and may be a block that has been predicted or reconstructed.

[0367] Optionally, the co-located block may be an image block in a co-located image that has the same position and size as the current block, and the co-located image may be an image in the reference image that is closest to the current image in terms of time.

[0368] Optionally, the time domain block can be a block distinguished from the time domain, such as an image block in other frames before or after the current frame. For example, there is video data containing a first frame image, a second frame image, and a third frame image played at the first second, the second second, and the third second respectively, and the current block is a block divided from the second frame image, then it can be determined that the time domain block corresponding to the current block is the corresponding image block in other frames except the second frame.

[0369] Optionally, the default block may be a block set in advance, for example, a block with typical pixel features pre-set by an encoder and / or a decoder.

[0370] In this embodiment, at least one reference area is determined or obtained based on at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, time domain blocks and default blocks corresponding to the current block, thereby ensuring a close association between the determined or obtained reference area and / or reference block and the current block, thereby making the subsequent prediction results obtained based on the reference area and / or reference block more accurate.

[0371] Mode F, based on at least one of the width, height, block size, and block area of ​​the current block;

[0372] Optionally, at least one reference area and / or at least one reference block may be determined or obtained according to the width and height of the current block and a tenth mapping table. Optionally, the tenth mapping table may be as shown in Table 10 below:

[0373] Table 10

[0374] Optionally, the height of at least one reference area and / or reference block is equal to a first preset multiple of the height of the current block. Optionally, the width of at least one reference area and / or reference block is equal to a second preset multiple of the width of the current block.

[0375] Optionally, the reference area or the image area in the reference area may be an encoded area or a decoded area.

[0376] Optionally, the encoded area or the decoded area can be determined by the position of the upper left corner pixel, the height and width of the encoded area or the decoded area. For example, the position of the upper left corner pixel can be a position N times (N is greater than 1) times the height of the image block above and N times the width of the image block to the left of the upper left corner position of the current block. The width of the encoded area or the decoded area is an integer multiple of the width of the current block, and the height of the encoded area or the decoded area is an integer multiple of the height of the current block.

[0377] Optionally, the reference area and / or reference block may be determined according to the block size of the current block and the eleventh mapping table.

[0378] Optionally, the eleventh mapping table may be as shown in Table 11 below:

[0379] Table 11

[0380] Optionally, the block size includes at least one of the width, height, ratio, depth, area, resolution, and number of pixels of the block. Optionally, X4 to X7 can be preset thresholds corresponding to at least one of the width, height, ratio, depth, area, resolution, and number of pixels in the block size.

[0381] Optionally, the reference area and / or reference block may be determined according to the block area of ​​the current block and a twelfth mapping table. Optionally, the twelfth mapping table may be as shown in Table 12 below:

[0382] Table 12

[0383] Optionally, the position of the reference area, reference block and / or reference pixel can be determined or obtained based on the upper adjacent pixels, left adjacent pixels and upper left adjacent pixels of the current block, and then the size of the reference area and / or reference block can be determined based on the current block width and height and the first mapping table.

[0384] Exemplarily, the reference area of ​​the current block includes: a first area adjacent to the top of the current block, a second area adjacent to the left of the current block, and a third area adjacent to the upper left of the current block. The width of the first area is equal to twice the width of the current block, and the height of the second area is equal to twice the height of the current block. If the width and height of the current block are both greater than or equal to 8, the height of the first area is 8, the width of the second area is 8, and the width and height of the third area are both 8; or, if either of the width and height of the current block is less than 8, the height of the first area is 4, the width of the second area is 4, and the width and height of the third area are both 4.

[0385] In this embodiment, by determining or obtaining a reference area, a reference block and / or a reference pixel based on at least one of the width, height, block size and block area of ​​the current block, it is ensured that the reference area and / or reference block are closely associated with the current block, thereby making the subsequent prediction results obtained based on the reference area and / or reference block more accurate.

[0386] Mode G, determining or obtaining a candidate block based on a candidate motion vector or a candidate block vector of a current block;

[0387] Optionally, a candidate block may be determined or obtained based on multiple candidate motion vectors or multiple candidate block vectors in the candidate list of the current block, and used as at least one reference area and / or at least one reference block.

[0388] Optionally, a candidate block can be determined or obtained based on the candidate motion vector or candidate block vector of the current block, and at least one reference area and / or at least one reference block can be determined or obtained based on at least one of the upper adjacent pixels, upper non-adjacent pixels, left adjacent pixels, left non-adjacent pixels, upper left adjacent pixels and upper left non-adjacent pixels of the candidate block.

[0389] Optionally, a candidate block can be determined or obtained based on the candidate motion vector or candidate block vector of the current block, and at least one of the upper adjacent pixels, upper non-adjacent pixels, left adjacent pixels, left non-adjacent pixels, upper left adjacent pixels and upper left non-adjacent pixels of the candidate block is used as at least one reference area and / or at least one reference block.

[0390] Optionally, a candidate block can be determined or obtained based on the candidate motion vector or candidate block vector of the current block, and at least one reference area and / or at least one reference block can be determined or obtained based on at least one of the width, height, block size and block area of ​​the candidate block. The specific implementation process can refer to the scheme in the above-mentioned method F, that is, the current block in method F can be replaced with the candidate block, and it will not be repeated here.

[0391] Optionally, a candidate block can be determined or obtained based on the candidate motion vector or candidate block vector of the current block, and at least one reference area and / or at least one reference block can be determined or obtained based on at least one of a neighbor block, a non-neighbor block, a co-located block, a time domain block and a default block corresponding to the candidate block.

[0392] Optionally, a candidate block can be determined or obtained based on the candidate motion vector or candidate block vector of the current block, and at least one of the neighboring block, non-neighboring block, co-located block, time domain block and default block corresponding to the candidate block is used as at least one reference area and / or at least one reference block.

[0393] In this embodiment, the candidate block determined or obtained based on the candidate motion vector or candidate block vector of the current block ensures that the determined or obtained reference area is closely related to the current block, thereby making the subsequent prediction results obtained based on the reference area and / or reference block more accurate.

[0394] Mode H: if the first information of the current block satisfies the first condition, the reference region is the first reference region and / or the reference block is the first reference block;

[0395] Optionally, the first information may be at least one of flag information of the current block, syntax elements, indication information, an index of a list, an aspect ratio of the current block, a width and height value range, and an area value range.

[0396] Optionally, flag information may be used to indicate a specific state or option, for example, to control the behavior of an encoder or decoder.

[0397] Optionally, the syntax element refers to a series of parameters and data structures defined in a video coding standard, and is used to describe specific coding information of video frames and blocks.

[0398] Optionally, the indication information is information used to indicate a certain state or condition, and may include flag information or more complex syntax elements.

[0399] Optionally, an index about a list refers to an integer value used to identify a specific item in a list, and these lists may contain candidate motion vectors, reference frame indices, transform coefficients, etc. The index can be used to quickly find and access a specific item in the list.

[0400] Optionally, the first information may be at least one of methods D to G, that is, the first information may be determining or obtaining at least one reference area and / or reference block based on at least one of the upper adjacent pixels, upper non-adjacent pixels, left adjacent pixels, left non-adjacent pixels, upper left adjacent pixels and upper left non-adjacent pixels of the current block; or, it may be determining or obtaining at least one reference area and / or reference block domain based on at least one of the neighboring blocks, non-neighboring blocks, co-located blocks, time domain blocks and default blocks corresponding to the current block; or, it may be determining or obtaining at least one reference area and / or reference block based on at least one of the width, height, block size and block area of ​​the current block; or, it may be determining or obtaining at least one reference area and / or reference block based on a candidate block determined or obtained based on a candidate motion vector or candidate block vector of the current block.

[0401] Optionally, the first condition may be a condition set in advance. Optionally, the first condition is not fixed and may be adaptively adjusted according to different scenarios.

[0402] Optionally, the current block satisfies a first condition, including at least one of the following:

[0403] The value of the first information is a first value;

[0404] The value of the first information is within a first numerical range;

[0405] The value of the first information is greater than or equal to the first threshold;

[0406] The value of the first information is less than or equal to the second threshold.

[0407] Optionally, the value of the first information may be at least one of the width, height, block size and block area of ​​the current block.

[0408] Optionally, the first value may be a value of at least one of width, height, block size and block area of ​​the image block set in advance.

[0409] Optionally, the first numerical interval may be a numerical interval of at least one of width, height, block size and block area of ​​the image block set in advance.

[0410] Optionally, the first threshold may be a threshold of at least one of width, height, block size and block area of ​​the image block set in advance.

[0411] Optionally, the second threshold may be a threshold of at least one of width, height, block size and block area of ​​the image block set in advance.

[0412] Optionally, when detecting that the first information of the current block meets the first condition, the processing device may select the first information for calculation to determine or obtain at least one reference area and / or at least one reference block.

[0413] Optionally, if the processing device detects that the current block meets the first condition, it may select the first information for calculation to determine or obtain at least one reference area.

[0414] Optionally, if the first information of the current block satisfies the first condition, the reference area is the first reference area; and at least one predicted pixel is determined or generated based on pixels of at least one reference area, pixels of at least one reference block, and the first model.

[0415] In this embodiment, the selection of the reference area through the first information can improve the similarity between the reference area and / or reference block and the current block (for example, the similarity of texture features, the similarity of pixel features, etc.), thereby making the prediction result determined or generated based on the reference area and / or reference block with high similarity to the current block have higher accuracy, and further improving the encoding and decoding quality in the video encoding and / or decoding process.

[0416] Mode I: if the first information of the current block does not satisfy the first condition, the reference region is the second reference region and / or the reference block is the second reference block;

[0417] Optionally, the first reference region and the second reference region are different.

[0418] Optionally, the first reference block and the second reference block are different.

[0419] Optionally, the first reference area and / or the first reference block may be a portion of at least one of the left adjacent area, the upper adjacent area, and the upper left adjacent area of ​​the current block, and the second reference area may be an area of ​​the above areas that is different from the first reference area, and the second reference block may be an area of ​​the above areas that is different from the first reference block.

[0420] Optionally, if the first information of the current block does not meet the first condition, the reference area is the second reference area and / or the reference block is the second reference block.

[0421] In this embodiment, by selecting the reference area based on the first information, the similarity between the reference area and / or reference block and the current block (for example, the similarity of texture features, the similarity of pixel features, etc.) can be improved, so that the prediction result determined or generated based on the reference area and / or reference block with high similarity to the current block has higher accuracy, thereby improving the encoding and decoding quality during the video encoding and / or decoding process.

[0422] Mode J, based on at least one of the region, block, and pixel included in the current block;

[0423] Optionally, in order to further improve the prediction accuracy of the current block, the current block can be divided into at least one sub-block, and the prediction result of the current block can be determined or generated based on the non-angle prediction sub-block of each sub-block. By predicting the sub-blocks of the current block, it helps to capture and analyze the local features of the current block more carefully, thereby improving the prediction effect; and / or significantly reducing the amount of data processed at a single time, making the calculation more efficient.

[0424] Optionally, the number of sub-blocks included in the current block is set according to actual conditions: for example, the current block includes 4, 8, 16 sub-blocks, etc., and the sizes of the sub-blocks may be at least partially the same or different.

[0425] Optionally, the current block may be divided into four sub-blocks of the same size, and prediction processing may be performed on each sub-block according to a preset processing order.

[0426] Optionally, the reference template includes a reference block and / or a reference area. In DIMD mode, OBIC mode, and / or TIMD mode, at least one angle prediction mode of at least one sub-block of the current block is determined or obtained based on at least one reference template of at least one sub-block; or, an angle predictor corresponding to at least one sub-block is determined or obtained based on at least one reference template and at least one angle prediction mode of at least one sub-block; or, a non-angle predictor corresponding to at least one sub-block is determined or obtained based on at least one reference template and at least one non-angle prediction mode of at least one sub-block; or, a prediction result of the current block is determined or generated based on the angle predictor and non-angle predictor corresponding to at least one sub-block.

[0427] Optionally, the reference area of ​​at least one subblock of the current block includes at least one of an area, a block and a pixel located inside the current block, and / or at least one of an area, a block and a pixel located outside the current block.

[0428] Optionally, the reference block of at least one subblock of the current block includes at least one of a region, a block and a pixel located inside the current block, and / or at least one of a region, a block and a pixel located outside the current block.

[0429] Optionally, the current block includes a first sub-block and a second sub-block, and the prediction processing order of the first sub-block precedes that of the second sub-block. Optionally, at least part of the reference template of the second sub-block is determined or obtained based on the prediction result of the first sub-block. Optionally, at least part of the reference template of the second sub-block is determined or obtained based on the reconstructed block of the first sub-block.

[0430] Exemplarily, when predicting at least one sub-block of the current block through the DIMD mode, the selection of the reference template of at least one sub-block of the current block can refer to FIG14 , where the solid line is the current block, and is divided into sub-block A, sub-block B, sub-block C, and sub-block D. The reference template of sub-block A is the external area adjacent to sub-block A. Since the present application can encode or decode the sub-blocks in a preset order, when encoding the subsequent sub-block, the sampling of the previous sub-block in the preset encoding order can be referred to. Therefore, for the sub-block B in the subsequent order, , the prediction result and / or reconstructed block of sub-block A can be used to determine or obtain the left reference template corresponding to sub-block B, and then the upper reference template can be determined or obtained based on the external area adjacent to sub-block B. For sub-block C, the prediction result and / or reconstructed block of sub-block A can also be used to determine or obtain the upper reference template corresponding to sub-block C, and then the left reference template can be determined or obtained based on the external area adjacent to sub-block C. For sub-block D, the reference template corresponding to sub-block D can be determined or obtained using the prediction results and / or reconstructed blocks of sub-blocks A, B, and C respectively.

[0431] Optionally, the boundary of the reference template may be the same as the boundary of the sub-block, or may be smaller than the boundary of the sub-block.

[0432] Optionally, the size of the reference template of at least one sub-block of the current block may be smaller than the size of the reference template of the current block, thereby reducing the complexity of the algorithm during the prediction process.

[0433] Exemplarily, the width of the reference template of the current block is 4, while the width of the reference target of each sub-block of the current block is 2, and the sum of the sizes of all reference templates of at least one sub-block of the current block is smaller than the sum of the sizes of all reference templates of the current block. Therefore, during the prediction process, the complexity of the algorithm can be reduced by reducing the size of the reference template.

[0434] Exemplarily, when prediction processing is performed on at least one sub-block of the current block using the TIMD mode, the selection of the reference template of at least one sub-block of the current block can refer to FIG15 , where the solid line represents the current block and is divided into sub-block A, sub-block B, sub-block C, and sub-block D. The reference template of sub-block A is the external area adjacent to sub-block A. Since the present application can encode or decode the sub-blocks in a preset order, when encoding the subsequent sub-block, the sampling of the previous sub-block in the preset encoding order can be referred to. Therefore, for the sub-block B in the subsequent order, The prediction result and / or reconstructed block of sub-block A can be used to determine or obtain a portion of the reference template corresponding to sub-block B, and then the remaining reference template can be determined or obtained based on the external area adjacent to sub-block B. For sub-block C, the prediction result and / or reconstructed block of sub-block A can also be used to determine or obtain a portion of the reference template corresponding to sub-block C, and then the remaining reference template can be determined or obtained based on the external area adjacent to sub-block C. For sub-block D, the reference template corresponding to sub-block D can be determined or obtained using the prediction results and / or reconstructed blocks of sub-blocks A, B and C respectively.

[0435] Optionally, in this embodiment, the non-angle prediction mode includes at least one of the first to fifth modes in the second embodiment.

[0436] In this embodiment, since the distance between the previous sub-block and the current sub-block may be closer than that between the external adjacent areas, there may be a higher correlation between the previous sub-block and the current sub-block. Then, by determining or obtaining the reference template of the current sub-block based on the prediction result and / or the reconstructed area of ​​the previous sub-block, the prediction accuracy can be effectively improved based on the correlation between the sub-blocks, thereby improving the encoding and decoding quality.

[0437] Mode K, based on the predicted block and / or reconstructed block of the first sub-block of the current block;

[0438] Optionally, the current block is divided into at least one sub-block, and the sub-block includes a first sub-block and a second sub-block.

[0439] Optionally, based on the prediction block and / or reconstructed block of the first subblock of the current block, the reference area, reference block and / or reference pixel corresponding to the second subblock of the current block is determined or obtained according to at least one of the following items.

[0440] Optionally, the first sub-block is processed before the second sub-block.

[0441] Optionally, in this embodiment, the non-angle prediction mode includes at least one of the first to fifth modes in the second embodiment.

[0442] In this embodiment, since the distance between the previous sub-block and the current sub-block may be closer than that between the external adjacent areas, there may be a higher correlation between the previous sub-block and the current sub-block. Then, by determining or obtaining the reference template of the current sub-block based on the prediction result and / or the reconstructed area of ​​the previous sub-block, the prediction accuracy can be effectively improved based on the correlation between the sub-blocks, thereby improving the encoding and decoding quality.

[0443] Optionally, the prediction result of the current block is determined or obtained according to a non-angular predictor related to the first mode.

[0444] Optionally, the first mode may be the non-angle prediction mode with the best performance, and the performance comparison result may be determined by comparing the differences and block vectors between the non-angle predictors associated with each non-angle prediction mode and the corresponding original image blocks, for example, by determining the SAD or SATD value between the non-angle predictors and the corresponding original image blocks. Optionally, the larger the SAD or SATD value, the greater the difference.

[0445] Exemplarily, the non-angle predictors related to the NNIP mode, EIP mode, BV mode, planar mode and DC mode are determined, and the SATD or SAD between each non-angle predictor and the current block is determined, and the non-angle prediction mode corresponding to the non-angle predictor with the smallest SATD or SAD is determined as the first mode.

[0446] Optionally, in the case where the first mode includes the planar mode, if the current block meets the enabling conditions of the neural network-based intra-frame prediction mode, the planar mode will be replaced by the neural network-based intra-frame prediction mode.

[0447] Optionally, when the first mode used to determine the prediction result of the current block includes the planar mode, determine whether the current block meets the preset NNIP activation conditions. Since the NNIP mode is equivalent to the planar mode, which is more complex and has better prediction effect, when the current block meets the NNIP activation conditions, the NNIP mode can replace the planar mode and be used to determine the prediction result of the current block.

[0448] Exemplarily, the non-angular predictors associated with the EIP mode, the BV mode, the planar mode, and the DC mode are determined, and the SATD or SAD between each non-angular predictor and the current block is determined; if the planar mode is the non-angular prediction mode with the smallest SATD or SAD, the NNIP mode is determined as the first mode and is used to determine the prediction result of the current block; and / or, if the BV mode is the non-angular prediction mode with the smallest SATD or SAD, the BV mode is determined as the first mode and is used to determine the prediction result of the current block.

[0449] Optionally, the enabling condition of the NNIP mode includes: the first information of the current block satisfies the first condition. Optionally, the first information of the current block satisfies the first condition, including at least one of the following:

[0450] The value of the first information is a first value;

[0451] The value of the first information is within a first numerical range;

[0452] The value of the first information is greater than or equal to the first threshold;

[0453] The value of the first information is less than or equal to the second threshold.

[0454] Exemplarily, if the block area of ​​the current block is smaller than the first threshold, it is determined that the current block meets the enabling condition of the NNIP mode.

[0455] Optionally, when the current block does not meet the enabling conditions of the NNIP mode, the current block may be transposed and / or downsampled so that the current block meets the enabling conditions of the NNIP mode.

[0456] In this embodiment, since the NNIP mode is equivalent to a more complex planar mode with better prediction effect, in the process of determining the first mode, the non-angle predictor related to the NNIP mode can be omitted from the calculation, and the non-angle predictor related to the planar mode can be compared with other non-angle prediction modes. Then, when it is determined that the first mode includes the planar mode, the NNIP mode is replaced by the planar mode to perform prediction processing on the current block, thereby effectively reducing the amount of calculation and improving the prediction effect.

[0457] Fourth embodiment

[0458] Based on any of the above embodiments, a fourth embodiment is proposed.

[0459] In this embodiment, step S10 includes at least one of the following methods:

[0460] Method 6: performing prediction processing on the current block according to at least one non-angle prediction mode, determining or generating at least one non-angle predictor, and determining or generating a prediction result of the current block according to the at least one non-angle predictor;

[0461] Optionally, the reference template includes a reference area, a reference block and / or a reference pixel.

[0462] Optionally, prediction processing is performed on the current block according to at least one non-angle prediction mode to determine or generate at least one non-angle predictor, and fusion processing is performed on the at least one non-angle predictor to determine or generate a prediction result of the current block.

[0463] Optionally, the current block is predicted based on at least one reference template and at least one non-angle prediction mode of the current block to determine or generate at least one non-angle predictor, and the at least one non-angle predictor is fused to determine or generate a prediction result of the current block.

[0464] Optionally, the current block includes at least one sub-block. Optionally, prediction processing is performed on at least one sub-block of the current block according to at least one non-angle prediction mode to determine or generate a non-angle prediction sub-sub-block corresponding to the at least one sub-block. Fusion processing is performed on the non-angle prediction sub-sub-block corresponding to the at least one sub-block to determine or generate a prediction result of the current block.

[0465] Optionally, based on a reference template corresponding to at least one sub-block of the current block and at least one non-angle prediction mode, prediction processing is performed on at least one sub-block of the current block, a non-angle prediction sub-sub-block corresponding to the at least one sub-block is determined or generated, and fusion processing is performed on the non-angle prediction sub-sub-block corresponding to the at least one sub-block to determine or generate a prediction result of the current block.

[0466] Optionally, in this embodiment, the non-angle prediction mode includes at least one of the first to fifth modes in the second embodiment.

[0467] Optionally, in this embodiment, the reference area, reference block and / or reference pixel is determined or obtained according to at least one of Method D to Method K in the third embodiment.

[0468] Optionally, in this embodiment, the angle prediction mode and / or the non-angle prediction mode is determined or obtained according to any one of the eleventh to thirteenth methods in the fourth embodiment.

[0469] In this embodiment, since under natural conditions, the distribution of object pixels within a graphic block is not completely angularly arranged, the technical solution of the present application can improve the prediction accuracy of the current block with non-angular pixels, thereby supporting the improvement of the encoding and decoding quality during video encoding and / or decoding. If the current block is divided into at least one sub-block and then subjected to prediction processing, it will help to capture and analyze the local features of the current block more carefully, thereby further improving the prediction effect, and / or significantly reducing the amount of data processed in a single time, making the calculation more efficient.

[0470] Method seven: performing prediction processing on the current block according to at least one non-angle prediction mode and at least one angular prediction mode, determining or generating at least one non-angle predictor and at least one angular predictor, and determining or generating a prediction result for the current block according to the at least one non-angle predictor and the at least one angular predictor;

[0471] Optionally, the reference template includes a reference area, a reference block and / or a reference pixel.

[0472] Optionally, prediction processing is performed on the current block according to at least one non-angle prediction mode to determine or generate at least one non-angle predictor, prediction processing is performed on the current block according to at least one angle prediction mode to determine or generate at least one angle predictor, and the at least one non-angle predictor and the at least one angle predictor are fused to determine or generate a prediction result of the current block.

[0473] Optionally, the at least one non-angle prediction mode includes a first non-angle prediction mode, a second non-angle prediction mode, and a third non-angle prediction mode. If a comparison result of the first non-angle prediction mode is better than a comparison result of the second non-angle prediction mode, the third non-angle prediction mode is used as the non-angle prediction mode for performing prediction processing on the current block, and is used to determine or generate the non-angle predictor.

[0474] Optionally, the first non-angular prediction mode includes a planar mode, and the third non-angular prediction mode includes a NNIP mode.

[0475] Optionally, prediction processing is performed on the current block according to at least one non-angle prediction mode to determine or generate at least one non-angle predictor, prediction processing is performed on the current block according to at least one angle prediction mode to determine or generate at least one angle predictor, and a target predictor is determined from the at least one angle predictor and the at least one non-angle predictor based on the SATD and / or SAD between the at least one non-angle predictor, the at least one angle predictor and the current block; and a prediction result of the current block is determined or generated based on the target predictor.

[0476] Optionally, the target predictor is determined to be the prediction result of the current block.

[0477] Optionally, a non-angle predictor or an angle predictor having the smallest SATD and / or SAD with the current block is determined as the target predictor.

[0478] Optionally, prediction processing is performed on the current block based on at least one reference template and at least one non-angle prediction mode of the current block to determine or generate at least one non-angle predictor, prediction processing is performed on the current block based on at least one reference template and at least one angle prediction mode of the current block to determine or generate at least one angle predictor, and the at least one non-angle predictor and the at least one angle predictor are fused to determine or generate a prediction result of the current block.

[0479] Optionally, prediction processing is performed on the current block based on at least one reference template and at least one non-angle prediction mode of the current block to determine or generate at least one non-angle predictor, prediction processing is performed on the current block based on at least one reference template and at least one angle prediction mode of the current block to determine or generate at least one angle predictor, and a target predictor is determined from the at least one angle predictor and the at least one non-angle predictor based on the SATD and / or SAD between the at least one non-angle predictor and the current block, and a prediction result of the current block is determined or generated based on the target predictor.

[0480] Optionally, the current block includes at least one sub-block.

[0481] Optionally, prediction processing is performed on at least one sub-block of the current block according to at least one non-angle prediction mode to determine or generate at least one non-angle predictor corresponding to the at least one sub-block, prediction processing is performed on at least one sub-block of the current block according to at least one angle prediction mode to determine or generate at least one angle predictor corresponding to the at least one sub-block, and fusion processing is performed on the at least one non-angle predictor and the at least one angle predictor to determine or generate a prediction result of the current block.

[0482] Optionally, prediction processing is performed on at least one sub-block of the current block according to at least one non-angle prediction mode to determine or generate at least one non-angle predictor corresponding to the at least one sub-block, prediction processing is performed on at least one sub-block of the current block according to at least one angular prediction mode to determine or generate at least one angular predictor corresponding to the at least one sub-block, and a target predictor is determined from the at least one angular predictor and the at least one non-angle predictor based on the SATD and / or SAD between the at least one non-angle predictor and the current block, and a prediction result of the current block is determined or generated based on the target predictor.

[0483] Optionally, in this embodiment, the non-angle prediction mode includes at least one of the first to fifth modes in the second embodiment.

[0484] Optionally, in this embodiment, the reference area, reference block and / or reference pixel is determined or obtained according to at least one of Method D to Method K in the third embodiment.

[0485] Optionally, in this embodiment, the angle prediction mode and / or the non-angle prediction mode is determined or obtained according to any one of the eleventh to thirteenth methods in the fourth embodiment.

[0486] In this embodiment, by fusing the predictor of the angular prediction mode that is good at processing textures or edges with obvious directionality and the predictor of the non-angular prediction mode that is suitable for predicting smooth areas and / or gradient areas, various complex features of the image blocks can be captured more accurately, thereby improving the prediction accuracy of the image blocks, and further supporting the improvement of the encoding and decoding quality in the video encoding and / or decoding process.

[0487] Method eight, determining or obtaining a first non-angle predictor from the non-angle predictors, and determining or generating a prediction result of the current block based on the first non-angle predictor;

[0488] Optionally, the reference template includes a reference area, a reference block and / or a reference pixel.

[0489] Optionally, prediction processing is performed on the current block according to at least one non-angle prediction mode to determine or generate at least one non-angle predictor, and based on the SATD and / or SAD between each non-angle predictor and the current block, a first non-angle predictor is determined or obtained from the non-angle predictors, and based on the first non-angle predictor, a prediction result of the current block is determined or generated.

[0490] Optionally, a non-angle predictor having the smallest SATD and / or SAD with the current block is determined as a first non-angle predictor. Optionally, the first non-angle predictor is used as a prediction result of the current block.

[0491] Optionally, prediction processing is performed on the current block based on at least one reference template and at least one non-angle prediction mode of the current block to determine or generate at least one non-angle predictor, and a first non-angle predictor is determined based on the SATD and / or SAD between each non-angle predictor and the current block, and a prediction result of the current block is determined or generated based on the first non-angle predictor.

[0492] Optionally, the current block includes at least one sub-block.

[0493] Optionally, prediction processing is performed on at least one sub-block of the current block according to at least one non-angle prediction mode, a non-angle predictor corresponding to the at least one sub-block is determined or generated, a first non-angle predictor is determined based on the SATD and / or SAD between the non-angle predictor corresponding to the at least one sub-block and the current block, and a prediction result of the current block is determined or generated based on the first non-angle predictor.

[0494] Optionally, the first non-angle predictor is used as the prediction result of the current block.

[0495] Optionally, prediction processing is performed on at least one sub-block of the current block based on at least one reference template and at least one non-angle prediction mode of at least one sub-block of the current block, a non-angle predictor corresponding to the at least one sub-block is determined or generated, a first non-angle predictor is determined based on the SATD and / or SAD between the non-angle predictor corresponding to the at least one sub-block and the current block, and a prediction result of the current block is determined or generated based on the first non-angle predictor.

[0496] Optionally, in this embodiment, the non-angle prediction mode includes at least one of the first to fifth modes in the second embodiment.

[0497] Optionally, in this embodiment, the reference area, reference block and / or reference pixel is determined or obtained according to at least one of Method D to Method K in the third embodiment.

[0498] Optionally, in this embodiment, the angle prediction mode and / or the non-angle prediction mode is determined or obtained according to any one of the eleventh to thirteenth methods in the fourth embodiment.

[0499] In this embodiment, since under natural conditions, the distribution of object pixels within a graphic block is not completely angularly arranged, the technical solution of the present application can improve the prediction accuracy of the current block with non-angular pixels, thereby supporting the improvement of the encoding and decoding quality during video encoding and / or decoding. Optionally, if the current block is divided into at least one sub-block for prediction processing, it will help to capture and analyze the local features of the current block more carefully, thereby further improving the prediction effect, and / or significantly reducing the amount of data processed at a single time, making the calculation more efficient.

[0500] Mode nine: determining or obtaining a second mode according to the rate-distortion cost corresponding to each non-angular prediction mode, and determining or generating a prediction result of the current block according to the non-angular predictor associated with the second mode;

[0501] Optionally, the reference template includes a reference area, a reference block and / or a reference pixel.

[0502] Optionally, prediction processing is performed on the current block according to at least one non-angle prediction mode to determine or generate at least one non-angle predictor, and based on each non-angle predictor, the rate-distortion cost corresponding to each non-angle prediction mode is determined, and based on the rate-distortion cost corresponding to each non-angle prediction mode, a second mode is determined or obtained, and based on the non-angle predictor related to the second mode, a prediction result of the current block is determined or generated.

[0503] Optionally, the non-angular prediction mode with the minimum rate-distortion cost is determined as the second mode.

[0504] Optionally, prediction processing is performed on the current block based on at least one reference template and at least one non-angle prediction mode of the current block to determine or generate at least one non-angle predictor; based on each non-angle predictor, the rate-distortion cost corresponding to each non-angle prediction mode is determined; based on the rate-distortion cost corresponding to each non-angle prediction mode, a second mode is determined or obtained; and based on the non-angle predictor related to the second mode, a prediction result of the current block is determined or generated.

[0505] Optionally, the current block includes at least one sub-block.

[0506] Optionally, prediction processing is performed on at least one sub-block of the current block according to at least one non-angle prediction mode, and a non-angle predictor corresponding to the at least one sub-block is determined or generated. Based on each non-angle predictor, a rate-distortion cost corresponding to each non-angle prediction mode is determined. Based on the rate-distortion cost corresponding to each non-angle prediction mode, a second mode is determined or obtained. Based on the non-angle predictor related to the second mode, a prediction result of the current block is determined or generated.

[0507] Optionally, prediction processing is performed on at least one sub-block of the current block based on at least one reference template and at least one non-angle prediction mode of at least one sub-block of the current block, and a non-angle predictor corresponding to the at least one sub-block is determined or generated. Based on each non-angle predictor, a rate-distortion cost corresponding to each non-angle prediction mode is determined. Based on the rate-distortion cost corresponding to each non-angle prediction mode, a second mode is determined or obtained. Based on the non-angle predictor related to the second mode, a prediction result of the current block is determined or generated.

[0508] Optionally, in this embodiment, the non-angle prediction mode includes at least one of the first to fifth modes in the second embodiment.

[0509] Optionally, in this embodiment, the reference area, reference block and / or reference pixel is determined or obtained according to at least one of Method D to Method K in the third embodiment.

[0510] Optionally, in this embodiment, the angle prediction mode and / or the non-angle prediction mode is determined or obtained according to any one of the eleventh to thirteenth methods in the fourth embodiment.

[0511] In this embodiment, since under natural conditions, the distribution of object pixels within a graphic block is not completely angularly arranged, the technical solution of the present application can improve the prediction accuracy of the current block with non-angular pixels, thereby supporting the improvement of the encoding and decoding quality during video encoding and / or decoding. If the current block is divided into at least one sub-block and then subjected to prediction processing, it will help to capture and analyze the local features of the current block more carefully, thereby further improving the prediction effect, and / or significantly reducing the amount of data processed in a single time, making the calculation more efficient.

[0512] Method 10: Determine or generate a prediction result of the current block based on the non-angle predictor and the angle predictor.

[0513] Optionally, the reference template includes a reference area, a reference block and / or a reference pixel.

[0514] Optionally, the prediction result of the current block is determined or generated according to at least one non-angle predictor related to the non-angle prediction mode and at least one angle predictor related to the angle prediction mode.

[0515] Optionally, prediction processing is performed on the reference block of the current block according to at least one non-angle prediction mode to determine or generate at least one non-angle predictor, prediction processing is performed on the reference block of the current block according to at least one angle prediction mode to determine or generate at least one angle predictor, and fusion processing is performed on the at least one non-angle predictor and at least one angle predictor to determine or generate a prediction result of the current block.

[0516] Optionally, prediction processing is performed on the reference block of the current block according to at least one non-angle prediction mode to determine or generate at least one non-angle predictor, prediction processing is performed on the reference block of the current block according to at least one angle prediction mode to determine or generate at least one angle predictor, and based on the at least one non-angle predictor, the SATD and / or SAD between the at least one angle predictor and the current block, a target predictor is determined from the at least one angle predictor and the at least one non-angle predictor, and based on the target predictor, a prediction result of the current block is determined or generated.

[0517] Optionally, the target predictor is determined to be the prediction result of the current block.

[0518] Optionally, a non-angle predictor or an angle predictor having the smallest SATD and / or SAD with the current block is determined as the target predictor.

[0519] Optionally, prediction processing is performed on the current block based on at least one reference template of the reference block of the current block and at least one non-angle prediction mode to determine or generate at least one non-angle predictor, prediction processing is performed on the current block based on at least one reference template of the reference block of the current block and at least one angle prediction mode to determine or generate at least one angle predictor, and fusion processing is performed on the at least one non-angle predictor and the at least one angle predictor to determine or generate a prediction result for the current block.

[0520] Optionally, prediction processing is performed on the current block based on at least one reference template of the reference block of the current block and at least one non-angle prediction mode to determine or generate at least one non-angle predictor, prediction processing is performed on the current block based on at least one reference template of the reference block of the current block and at least one angle prediction mode to determine or generate at least one angle predictor, and a target predictor is determined from the at least one angle predictor and the at least one non-angle predictor based on the SATD and / or SAD between the at least one non-angle predictor and the current block, and a prediction result of the current block is determined or generated based on the target predictor.

[0521] Optionally, the current block includes at least one sub-block. Optionally, prediction processing is performed on the reference block of at least one sub-block of the current block according to at least one non-angle prediction mode to determine or generate at least one non-angle predictor corresponding to the reference block of at least one sub-block. Prediction processing is performed on the reference block of at least one sub-block of the current block according to at least one angle prediction mode to determine or generate at least one angle predictor corresponding to the reference block of at least one sub-block. The at least one non-angle predictor and the at least one angle predictor are fused to determine or generate a prediction result of the current block.

[0522] Optionally, prediction processing is performed on the reference block of at least one sub-block of the current block according to at least one non-angle prediction mode to determine or generate at least one non-angle predictor corresponding to the reference block of the at least one sub-block, prediction processing is performed on the reference block of at least one sub-block of the current block according to at least one angular prediction mode to determine or generate at least one angular predictor corresponding to the reference block of the at least one sub-block, and a target predictor is determined from the at least one angular predictor and the at least one non-angle predictor based on the SATD and / or SAD between the at least one non-angle predictor and the current block, and a prediction result of the current block is determined or generated based on the target predictor.

[0523] Optionally, in this embodiment, the non-angle prediction mode includes at least one of the first to fifth modes in the second embodiment.

[0524] Optionally, in this embodiment, the reference area, reference block and / or reference pixel is determined or obtained according to at least one of Method D to Method K in the third embodiment.

[0525] Optionally, in this embodiment, the angle prediction mode and / or the non-angle prediction mode is determined or obtained according to any one of the eleventh to thirteenth methods in the fourth embodiment.

[0526] In this embodiment, by fusing the predictor of the angular prediction mode that is good at processing textures or edges with obvious directionality and the predictor of the non-angular prediction mode that is suitable for predicting smooth areas and / or gradient areas, various complex features of the image blocks can be captured more accurately, thereby improving the prediction accuracy of the image blocks, and further supporting the improvement of the encoding and decoding quality in the video encoding and / or decoding process.

[0527] Optionally, the angle prediction mode and / or the non-angle prediction mode is determined or obtained according to any one of the following methods 11 to 13:

[0528] Mode 11, based on at least one reference template of the current block and / or at least one sub-block of the current block;

[0529] Optionally, the reference template includes a reference area, a reference block and / or a reference pixel.

[0530] Optionally, the current block and / or the angle prediction mode of the current block is determined or obtained according to the prediction derivation mode and a reference template of the current block and / or at least one sub-block of the current block.

[0531] Optionally, the current block and / or the non-angular prediction mode of the current block is determined or obtained according to the prediction derivation mode and a reference template of the current block and / or at least one sub-block of the current block.

[0532] Optionally, the prediction derivation mode includes: DIMD mode, OBIC mode and / or TIMD mode.

[0533] Optionally, when the current block is predicted using the DIMD mode, a reference template of the current block is determined or obtained, the gradient amplitude value and gradient direction of the pixels in the reference template are determined, the statistical results of the reference template regarding the intra-frame prediction direction are determined, the intra-frame prediction direction of at least one sub-block of the current block is determined based on the statistical results, and the mode corresponding to the intra-frame prediction direction is determined as an angular prediction mode (for example, angular mode 34) and / or a non-angular prediction mode (for example, planar mode), and is used to determine the angular predictor and / or non-angular predictor of the current block.

[0534] Optionally, when prediction processing is performed on at least one sub-block of the current block through the DIMD mode, a reference template of at least one sub-block of the current block is determined or obtained, the gradient amplitude value and gradient direction of the pixels in the reference template are determined, the statistical results of the reference template regarding the intra-frame prediction direction are determined, the intra-frame prediction direction of at least one sub-block of the current block is determined based on the statistical results, and the mode corresponding to the intra-frame prediction direction is determined as an angular prediction mode (for example, angular mode 34) and / or a non-angular prediction mode (for example, planar mode), and is used to determine the angular predictor and / or non-angular predictor of at least one sub-block of the current block.

[0535] Optionally, the current block includes a first subblock. Optionally, the first subblock may be any subblock in the current block: for example, a subblock that has been subjected to prediction processing, or a subblock that has not been subjected to prediction processing.

[0536] Optionally, when prediction processing is performed on at least one sub-block of the current block through the DIMD mode, a reference template of the first sub-block is determined or obtained, the gradient amplitude value and gradient direction of the pixels in the reference template are determined, the statistical results of the reference template regarding the intra-frame prediction direction are determined, the intra-frame prediction direction of the first sub-block is determined based on the statistical results, and the mode corresponding to the intra-frame prediction direction is determined as an angle prediction mode and / or a non-angle prediction mode, and is used to determine the angle predictor and / or non-angle predictor of the first sub-block.

[0537] Optionally, when the current block is predicted using the OBIC mode, the intra-frame prediction mode usage of at least one reference block (i.e., adjacent coding blocks and / or non-adjacent coding blocks) of the current block is determined, the area amplitude value and intra-frame prediction direction of at least one reference block are calculated, the area amplitude histogram or statistical result of the intra-frame prediction direction of at least one reference block is determined, and the intra-frame prediction direction of the block to be predicted is determined based on the area amplitude histogram or statistical result, and the mode corresponding to the intra-frame prediction direction is determined as the angle prediction mode (for example, angle mode 34) and / or non-angle prediction mode of the current block, and is used to determine the angle predictor and / or non-angle predictor of the current block.

[0538] Optionally, when at least one sub-block of the current block is predicted using the OBIC mode, the intra-frame prediction mode usage of at least one reference block (i.e., adjacent coding blocks and / or non-adjacent coding blocks) of the current block is determined, the area amplitude value and intra-frame prediction direction of at least one reference block are calculated, the area amplitude histogram or statistical result of the intra-frame prediction direction of at least one reference block is determined, and the intra-frame prediction direction of the block to be predicted is determined based on the area amplitude histogram or statistical result, and the mode corresponding to the intra-frame prediction direction is determined as an angular prediction mode (for example, angular mode 34) and / or a non-angular prediction mode, and is used to determine the angular predictor and / or non-angular predictor of each sub-block.

[0539] Optionally, the current block includes a first sub-block. When prediction processing is performed on at least one sub-block of the current block through the OBIC mode, the intra-frame prediction mode usage of at least one reference block (i.e., adjacent coding blocks and / or non-adjacent coding blocks) of the first sub-block of the current block is determined, the area amplitude value and intra-frame prediction direction of the at least one reference block are calculated, and the area amplitude histogram or statistical result of the intra-frame prediction direction of the at least one reference block is determined; based on the area amplitude histogram or statistical result, the intra-frame prediction direction of the block to be predicted is determined, and the mode corresponding to the intra-frame prediction direction is determined as an angular prediction mode and / or a non-angular prediction mode, and is used to determine the angular predictor and / or non-angular predictor of the first sub-block.

[0540] Optionally, when the current block is predicted using the TIMD mode, a reference template of the current block is determined, a prediction value is generated for the reference template using each mode in the MPM list, and an angle prediction mode and / or a non-angle prediction mode is determined from the MPM list based on the SATD of the prediction value and the reconstructed value, and is used to determine the angle predictor and / or non-angle predictor of the current block.

[0541] Optionally, when prediction processing is performed on at least one sub-block of the current block through the TIMD mode, a reference template of at least one sub-block of the current block is determined; a prediction value is generated for the reference template through each mode in the MPM list; and based on the SATD of the prediction value and the reconstructed value, an angle prediction mode and / or a non-angle prediction mode is determined from the MPM list, and used to determine the angle predictor and / or non-angle predictor of at least one sub-block of the current block.

[0542] Optionally, the MPM list is the MPM list of the current block, and the MPM list may also be the MPM list of sub-blocks of the current block.

[0543] Optionally, when prediction processing is performed on at least one sub-block of the current block through the TIMD mode, a reference template of the first sub-block of the current block is determined, a prediction value is generated for the reference template through each mode in the MPM list, and an angle prediction mode and / or a non-angle prediction mode is determined from the MPM list based on the SATD of the prediction value and the reconstructed value, and is used to determine the angle predictor and / or non-angle predictor of the first sub-block.

[0544] Optionally, in this embodiment, the reference area, reference block and / or reference pixel is determined or obtained according to at least one of Method D to Method K in the third embodiment.

[0545] In this embodiment, since the reference template of the current block is usually highly correlated with the current block in space or time, the angle prediction mode obtained using this information can more accurately capture local texture, edge or brightness changes, thereby improving the prediction accuracy of the current block and improving the prediction effect of video encoding and / or decoding.

[0546] Method 12: determining or obtaining based on the most probable pattern list corresponding to the current block;

[0547] Optionally, the reference template includes a reference area, a reference block and / or a reference pixel.

[0548] Optionally, when the current block is predicted using the TIMD mode, the candidate angular prediction mode and / or candidate non-angular prediction mode of the current block may be determined using the MPM list.

[0549] Optionally, since the sub-block is part of the current block and there is a strong correlation between the two, when predicting at least one sub-block of the current block through the TIMD mode, the prediction mode corresponding to the sub-block can also be determined based on this inherent correlation according to the MPM list of the current block. The present application can improve the prediction accuracy without increasing the amount of calculation by at least partially reusing the MPM list of the current block.

[0550] Referring to Figure 16, the solid line block is the current block, and is divided into sub-block A, sub-block B, sub-block C and sub-block D; and some adjacent blocks of the current block will be used as pixel samples, and the prediction modes corresponding to the pixel samples will be analyzed to generate the most likely candidate mode set (i.e., MPM list) for the current block, and then the angular prediction mode and / or non-angular prediction mode corresponding to the current block and / or at least one sub-block of the current block can be determined or obtained according to the most likely mode list corresponding to the current block.

[0551] Optionally, the most probable mode list includes a most probable mode angle prediction list and a most probable mode non-angle prediction list. Optionally, a non-angle prediction mode is determined from the most probable non-angle prediction list, and a prediction result of the current block is determined or generated based on the determined non-angle prediction mode.

[0552] Alternatively, referring to FIG. 16 , pixel samples used to generate an MPM list of a current block typically include neighboring pixels at the following positions: left (L), above (A), lower left (BL), upper right (AR), and upper left (AL).

[0553] Optionally, the angle prediction mode corresponding to the current block is determined or obtained according to a most probable mode list corresponding to the current block and at least one reference template of the current block.

[0554] Optionally, the angle prediction mode corresponding to at least one sub-block of the current block is determined or obtained according to the most probable mode list corresponding to the current block and at least one reference template of at least one sub-block of the current block.

[0555] Optionally, a candidate mode list is determined or obtained based on the most probable mode list corresponding to the current block and the prediction modes corresponding to the adjacent blocks, non-adjacent blocks and / or first sub-blocks of the current block, and an angle prediction mode corresponding to at least one sub-block of the current block is determined or obtained from the candidate mode list based on at least one reference template of at least one sub-block of the current block.

[0556] In this embodiment, by at least partially reusing the MPM list of the current block, the angle prediction mode of the current block and / or at least one sub-block of the current block is determined or obtained. Since the sub-block is part of the current block and there is a strong correlation between the two, based on the inherent correlation between the current block and its sub-blocks, a more appropriate prediction mode can be selected to perform sub-block prediction without increasing the amount of calculation, thereby improving the prediction accuracy.

[0557] Mode 13: determining or obtaining the angular prediction mode and / or non-angular prediction mode corresponding to the second sub-block of the current block according to the most probable mode list corresponding to the first sub-block of the current block;

[0558] Optionally, based on the MPM list corresponding to the first sub-block of the current block, the MPM list corresponding to the second sub-block of the current block is determined or obtained, and based on the MPM list corresponding to the second sub-block, the angle prediction mode and / or non-angle prediction mode corresponding to the second sub-block is determined or obtained.

[0559] Optionally, based on the MPM list corresponding to the first sub-block of the current block, the MPM list corresponding to the second sub-block of the current block is determined or obtained, and based on the MPM list corresponding to the second sub-block and at least one reference template, the angle prediction mode and / or non-angle prediction mode corresponding to the second sub-block is determined or obtained.

[0560] Optionally, the reference template includes at least one of a reference block, a reference area and / or a reference pixel.

[0561] Exemplarily, the selection of pixel samples in each sub-area of ​​the current block can refer to Figure 17, where the solid line block is the current block and is divided into sub-block A, sub-block B, sub-block C and sub-block D. For sub-block A, the MPM list corresponding to sub-block A is determined or obtained based on the adjacent pixel points (a1 to a5) of sub-block A.

[0562] Exemplarily, as shown in FIG16 , the MPM list of the current block uses the prediction modes corresponding to the adjacent pixel samples at the five positions of the left (L), top (A), bottom left (BL), top right (AR) and top left (AL) of the current block as prediction candidates in the MPM list of the current block. Since the pixel sample at the top left (AL) position of the current block in FIG16 is the same as the pixel sample at the a1 position of sub-block A, the prediction mode corresponding to the pixel sample at the a1 position of sub-block A can be directly determined when determining the MPM list of the current block, while the prediction modes corresponding to the pixel samples at the remaining positions a2 to a5 of sub-block A need to be separately determined or the prediction modes of these pixel samples need to be searched and added to the MPM list corresponding to sub-block A.

[0563] Optionally, the prediction mode of the pixel samples at positions a2 to a5 of sub-block A is the prediction mode of the coding unit or prediction unit in which it is located. According to the MPM list corresponding to sub-block A, the prediction mode corresponding to sub-block A is determined or obtained, and according to the prediction mode corresponding to sub-block A and the reference template corresponding to sub-block A, the predictor corresponding to sub-block A is determined or obtained.

[0564] Optionally, the prediction mode corresponding to the above pixel sampling includes an angular prediction mode and / or a non-angular prediction mode.

[0565] Optionally, the prediction mode corresponding to the above sub-block A includes an angular prediction mode and / or a non-angular prediction mode.

[0566] Optionally, the determined or obtained predictor corresponding to the sub-block A includes an angle predictor and / or a non-angle predictor.

[0567] Referring to Figure 17, the present application can encode or decode sub-blocks in a preset order. When encoding the latter sub-block, the pixel samples of the previous sub-block in the preset encoding order can be referred to. Therefore, for the sub-block B in the latter order, the prediction result of the sub-block A can be used to determine or obtain the prediction mode of the adjacent pixel sample b2. Since the prediction mode corresponding to the pixel samples at the b3 position and the b4 position of the sub-block B can be directly determined when determining the MPM list of the current block, the amount of calculation can be effectively reduced. For the pixel sample at the b1 position of the sub-block B, it is necessary to separately determine or find the prediction mode of these pixel samples and add them to the MPM list corresponding to the sub-block B.

[0568] Optionally, the prediction mode of the pixel sample at the b1 position of sub-block B is the prediction mode of the coding unit or prediction unit in which it is located. According to the prediction mode corresponding to sub-block B and the reference template corresponding to sub-block B, the prediction sub-block corresponding to sub-block B is determined or obtained. Optionally, the prediction mode corresponding to the above-mentioned pixel sampling includes an angle prediction mode and / or a non-angle prediction mode. Optionally, the prediction mode corresponding to the above-mentioned sub-block B includes an angle prediction mode and / or a non-angle prediction mode. Optionally, the prediction sub-block corresponding to sub-block B is determined or obtained to include an angle prediction sub-block and / or a non-angle prediction sub-block.

[0569] Optionally, referring to Figure 17, for sub-block C, the prediction results of sub-blocks A and B can be used to determine or obtain the pixel samples at positions c4 and c5, respectively, and then from the MPM list corresponding to sub-block B, the prediction mode corresponding to the pixel samples at positions c2 and c3 of sub-block C can be directly determined when determining the MPM list of the current block. For the pixel sample at position c1 of sub-block C, it is necessary to separately determine or find the prediction mode of these pixel samples and add them to the MPM list corresponding to sub-block C. Optionally, the prediction mode of the pixel sample at position c1 of sub-block C is the prediction mode of the coding unit or prediction unit in which it is located.

[0570] Optionally, referring to Figure 17, for sub-block D, the prediction modes corresponding to the pixel samples at positions d1, d9, d10, d4, and d6 of sub-block D can be directly determined when determining the MPM list of the current block. For the pixel samples at positions d2, d3, d7, and d8 of sub-block D, the prediction modes of these pixel samples need to be separately determined or searched and added to the MPM list corresponding to sub-block D. Optionally, the prediction modes of the pixel samples at positions d2, d3, d7, and d8 of sub-block B are the prediction modes of the coding units or prediction units where the respective pixel samples are located. Optionally, based on the prediction mode corresponding to sub-block D and the reference template corresponding to sub-block D, the predictor corresponding to sub-block D is determined or obtained. Optionally, the prediction mode corresponding to the above-mentioned pixel samples includes an angular prediction mode and / or a non-angular prediction mode. Optionally, the prediction mode corresponding to the above-mentioned sub-block D includes an angular prediction mode and / or a non-angular prediction mode. Optionally, the prediction mode corresponding to sub-block D includes an angular prediction mode and / or a non-angular prediction mode. Optionally, the prediction mode corresponding to sub-block D is determined or obtained to include an angular predictor and / or a non-angular predictor.

[0571] Optionally, in this embodiment, the reference area, reference block and / or reference pixel is determined or obtained according to at least one of Method D to Method K in the third embodiment.

[0572] Optionally, pixel samples of each sub-block of the current block generally include adjacent pixels at the left, top and upper left positions, and / or at least one adjacent pixel at the lower left, upper right, middle left and / or upper middle positions is further added according to actual needs.

[0573] Exemplarily, each sub-block of the current block is used to determine the selection of pixel samples of the sub-block MPM list. Referring to Figure 18, the solid line block is the current block, and is divided into sub-block A, sub-block B, sub-block C and sub-block D. For sub-blocks A to C, the selection of pixel samples is the same as in Figure 17, and for sub-block D, the prediction modes of sub-blocks A to C can be used to respectively determine or obtain the prediction modes of the adjacent pixel samples of sub-block D, thereby obtaining the MPM list corresponding to sub-block D, and according to the MPM list corresponding to sub-block D, the angle prediction mode corresponding to sub-block D is determined or obtained, and according to the angle prediction mode corresponding to sub-block D and the reference template corresponding to sub-block D, the angle predictor corresponding to sub-block D is determined or obtained. Optionally, the prediction mode corresponding to the above-mentioned pixel samples includes an angle prediction mode and / or a non-angle prediction mode. Optionally, the prediction mode corresponding to the above-mentioned sub-block D includes an angle prediction mode and / or a non-angle prediction mode. Optionally, the prediction sub-block D includes an angle prediction mode and / or a non-angle prediction mode.

[0574] Exemplarily, the selection of adjacent pixel samples of each sub-area of ​​the current block can also refer to Figure 19. The solid line block is the current block, and is divided into sub-block A, sub-block B, sub-block C and sub-block D. For sub-block A, the difference from the MPM list construction method of sub-block A in Figure 17 is that sub-block A does not use the prediction mode of the pixel sample at position a3 in Figure 17. The reason is that position a2 and position a3 of sub-block A are adjacent to the left edge of the current block and are both located at the midpoint. The probability that the image blocks at the two positions are the same image block is relatively high. In order to reduce redundant operations, only the prediction mode of one of the two positions is used to determine the MPM list of sub-block A. Similarly, for sub-block D, the difference from the MPM list construction method of sub-block D in Figure 17 is that sub-block D does not use the prediction mode of pixel sampling at position d3 and position d8 in Figure 17. The reason is that position d2 and position d3 of sub-block D are adjacent to the left edge of the current block and are both located at the midpoint of the edge. The probability that the image blocks at the two positions are the same image block is relatively high. Position d7 and position d8 of sub-block D are adjacent to the upper edge of the current block and are both located at the midpoint of the edge. In order to reduce redundant operations, only the prediction mode of one of the two positions is used to determine the MPM list of sub-block D.

[0575] Optionally, when prediction processing is performed on at least one sub-block of the current block through OBIC, the surrounding pixel samples of at least one sub-block of the current block include adjacent pixel samples and non-adjacent pixel samples. Taking the first sub-block and the second sub-block as an example, the adjacent pixel samples of the first sub-block and the second sub-block can both be adjacent pixel samples of the current block, and the adjacent pixel samples of the second sub-block can also be pixel samples located inside the current block. The non-adjacent pixel samples of the first sub-block and the second sub-block can be non-adjacent pixel samples selected according to a preset rule with the current block as the center, the non-adjacent pixel samples of the first sub-block and the second sub-block and / or non-adjacent pixel samples selected according to a preset rule with the first sub-block and the second sub-block as the center respectively.

[0576] Optionally, the area amplitude histogram of at least one sub-block is determined based on the prediction mode corresponding to the surrounding pixel samples of at least one sub-block of the current block. Optionally, the area amplitude histogram of at least one sub-block can also be determined based on the prediction mode corresponding to the coding block (i.e., the adjacent coding block and / or non-adjacent coding block of the sub-block) where the surrounding pixel samples of at least one sub-block of the current block are located. Optionally, the area amplitude histogram of at least one sub-block can also be determined based on the prediction mode corresponding to the sub-block of the coding block where the surrounding pixel samples of at least one sub-block of the current block are located. The prediction mode with the largest amplitude of each area amplitude histogram is used as the prediction mode of each sub-block of the current block. The prediction sub-block is determined or obtained based on the prediction mode of each sub-block. Optionally, the above-mentioned prediction mode includes an angle prediction mode and / or a non-angle prediction mode. Optionally, the above-mentioned prediction sub-block includes an angle predictor and / or a non-angle predictor.

[0577] Exemplarily, the selection of surrounding pixel samples of the current block can refer to Figure 20, where the solid line block is the current block, and the surrounding pixel samples of the current block include: adjacent pixel samples and non-adjacent pixel samples. The adjacent pixel samples of the current block include pixel samples located above, above the left, and to the left of the current block, and the non-adjacent pixel samples of the current block include multiple pixel samples selected from the top, above the left, above the right, left, and below the left according to a preset rule with the current block as the center.

[0578] Exemplarily, when the current block includes at least one sub-block, the selection of surrounding pixel samples of each sub-block can refer to FIG. 21. The solid line block is the current block, and is divided into 4 sub-blocks, including sub-block A. As shown in FIG. 20 and FIG. 21, for sub-block A, since the pixel samples adjacent to the current block are also adjacent to sub-block A, at least part of the adjacent pixel samples of the current block can be used as the adjacent pixel samples of sub-block A. Optionally, non-adjacent pixel samples of sub-block A can be selected with the current block as the center, and then at least part of the non-adjacent pixel samples of the current block can be used as the adjacent pixel samples of sub-block A. Non-adjacent pixel samples of block A are sampled, and then the area amplitude histogram of sub-block A is determined according to the prediction mode corresponding to the surrounding pixel samples of the current block, or the area amplitude histogram of sub-block A is determined according to the prediction mode corresponding to the surrounding pixel samples of sub-block A, and then the prediction mode with the largest amplitude of each area amplitude histogram is used as the prediction mode of sub-block A, and the predictor of sub-block A is determined or obtained according to the prediction mode of sub-block A. Optionally, the above-mentioned prediction mode includes an angle prediction mode and / or a non-angle prediction mode. Optionally, the above-mentioned predictor includes an angle predictor and / or a non-angle predictor.

[0579] Exemplarily, the method for selecting surrounding pixel samples of at least one sub-block of the current block can also refer to Figure 22. The solid line block is the current block and is divided into 4 sub-blocks. For sub-block A, the difference from the method for selecting pixel samples around sub-block A in Figure 21 is that the non-adjacent pixel samples of sub-block A are no longer selected with the current block as the center, but with sub-block A as the center, multiple pixel samples are selected from the top, upper left, upper right, left and lower left according to a preset rule, and then the area amplitude histogram of sub-block A is determined according to the prediction mode corresponding to the surrounding pixel samples of sub-block A, and the prediction mode with the largest amplitude of each area amplitude histogram is used as the prediction mode of sub-block A. The predictor of sub-block A is determined or obtained according to the prediction mode of sub-block A. Optionally, the above-mentioned prediction mode includes an angle prediction mode and / or a non-angle prediction mode. Optionally, the above-mentioned predictor includes an angle predictor and / or a non-angle predictor.

[0580] Exemplarily, the method for selecting the surrounding pixel samples of at least one sub-block of the current block can also refer to Figure 23. The solid line block is the current block, and is divided into 4 sub-blocks, including sub-block B. As shown in Figures 20 and 23, for sub-block B, the non-adjacent pixel samples of sub-block B can be selected with the current block as the center, and then at least part of the non-adjacent pixel samples of the current block can be used as the non-adjacent pixel samples of sub-block B, and the pixel samples located above and to the upper left of sub-block B can be selected as the adjacent pixel samples of sub-block B. Since the present application can encode or decode the sub-blocks in a preset order, when encoding the next sub-block, reference can be made to the preset encoding method. The pixel samples of the previous sub-block in the previous order, therefore, for the sub-block B in the latter order, the prediction result of the previous sub-block can be used to select the pixel samples located inside the current block as the adjacent pixel samples on the left side of the sub-block B, and then determine the area amplitude histogram of the sub-block B according to the prediction mode corresponding to the surrounding pixel samples of the sub-block B, and use the prediction mode with the largest amplitude of each area amplitude histogram as the prediction mode of the sub-block B, and determine or obtain the prediction sub-block B according to the prediction mode of the sub-block B. Optionally, the above-mentioned prediction mode includes an angle prediction mode and / or a non-angle prediction mode. Optionally, the above-mentioned prediction sub-block includes an angle prediction sub-block and / or a non-angle prediction sub-block.

[0581] Exemplarily, the method for selecting surrounding pixel samples of at least one sub-block of the current block can also refer to Figure 24. The solid line block is the current block, which is divided into four sub-blocks. For sub-block B, the difference from the method for selecting surrounding pixel samples of sub-block B in Figure 23 is that the selection of non-adjacent pixel samples is no longer centered on the current block, but is centered on sub-block B. Multiple pixel samples are selected from the top, top left, top right, left, and bottom left according to a preset rule. Optionally, since the sub-block to the left of sub-block B has been encoded or decoded, the non-adjacent pixel samples of sub-block B may include pixel samples located within the current block. Then, based on the prediction modes corresponding to the surrounding pixel samples of sub-block B, the area amplitude histogram of sub-block B is determined, and the prediction mode with the largest amplitude in each area amplitude histogram is used as the prediction mode of sub-block B. The predictor of sub-block B is determined or obtained based on the prediction mode of sub-block B. Optionally, the above-mentioned prediction mode includes an angular prediction mode and / or a non-angular prediction mode. Optionally, the above-mentioned predictor includes an angular predictor and / or a non-angular predictor.

[0582] Exemplarily, the method for selecting the surrounding pixel samples of at least one sub-block of the current block can also refer to Figure 25. The solid line block is the current block, and is divided into 4 sub-blocks, including sub-block C. As shown in Figures 20 and 25, for sub-block C, the non-adjacent pixel samples of sub-block C can be selected with the current block as the center, and then at least part of the non-adjacent pixel samples of the current block can be used as the non-adjacent pixel samples of sub-block C, and the pixel samples located on the left and upper left of sub-block C are selected as the adjacent pixel samples of sub-block C. Since the present application can encode or decode the sub-blocks in a preset order, when encoding the next sub-block, reference can be made to the preset encoding method. The pixel samples of the previous sub-block in the previous order, therefore, for the sub-block C in the latter order, the prediction result of the previous sub-block can be used to select the pixel samples located inside the current block as the adjacent pixel samples above the sub-block C, and then determine the area amplitude histogram of the sub-block C according to the prediction mode corresponding to the surrounding pixel samples of the sub-block C, and use the prediction mode with the largest amplitude of each area amplitude histogram as the prediction mode of the sub-block C, and determine or obtain the prediction sub-block C according to the prediction mode of the sub-block C. Optionally, the above-mentioned prediction mode includes an angle prediction mode and / or a non-angle prediction mode. Optionally, the above-mentioned prediction sub-block includes an angle prediction sub-block and / or a non-angle prediction sub-block.

[0583] Exemplarily, the method for selecting surrounding pixel samples of at least one sub-block of the current block can also refer to Figure 26. The solid line block is the current block and is divided into four sub-blocks. For sub-block C, the difference from the method for selecting surrounding pixel samples of sub-block C in Figure 25 is that the selection of non-adjacent pixel samples is no longer centered on the current block, but is centered on sub-block C. Multiple pixel samples are selected from the top, top left, top right, left, and bottom left according to a preset rule. Optionally, since the sub-blocks above and top right of sub-block C have been encoded or decoded, the non-adjacent pixel samples of sub-block C may include pixel samples located within the current block. Then, based on the prediction modes corresponding to the surrounding pixel samples of sub-block C, the area amplitude histogram of sub-block C is determined, and the prediction mode with the largest amplitude in each area amplitude histogram is used as the prediction mode of sub-block C. The predictor of sub-block C is determined or obtained based on the prediction mode of sub-block C. Optionally, the above-mentioned prediction mode includes an angular prediction mode and / or a non-angular prediction mode. Optionally, the above-mentioned predictor includes an angular predictor and / or a non-angular predictor.

[0584] Exemplarily, the method for selecting the surrounding pixel samples of at least one sub-block of the current block can also refer to Figure 27. The solid line block is the current block, and is divided into 4 sub-blocks, including sub-block D. As shown in Figures 20 and 27, for sub-block D, the non-adjacent pixel samples of sub-block D can be selected with the current block as the center, and then at least part of the non-adjacent pixel samples of the current block can be used as the non-adjacent pixel samples of sub-block D. Since the present application can encode or decode the sub-blocks according to a preset order, when encoding the subsequent sub-block, the pixel samples of the previous sub-block with a preset encoding order can be referred to. Therefore, for The sub-block D in the latter order can use the prediction result of the previous sub-block to select pixel samples located inside the current block as adjacent pixel samples above, above and to the left of the sub-block D, and then determine the area amplitude histogram of the sub-block D according to the prediction mode corresponding to the surrounding pixel samples of the sub-block D, and use the prediction mode with the largest amplitude of each area amplitude histogram as the prediction mode of the sub-block D. The predictor of the sub-block D is determined or obtained according to the prediction mode of the sub-block D. Optionally, the above-mentioned prediction mode includes an angle prediction mode and / or a non-angle prediction mode. Optionally, the above-mentioned predictor includes an angle predictor and / or a non-angle predictor.

[0585] Exemplarily, the method for selecting surrounding pixel samples of at least one sub-block of the current block can also refer to Figure 28. The solid line block is the current block and is divided into four sub-blocks. For sub-block D, the difference from the method for selecting surrounding pixel samples of sub-block D in Figure 27 is that the selection of non-adjacent pixel samples is no longer centered on the current block, but is centered on sub-block D. Multiple pixel samples are selected from the top, top left, top right, left, and bottom left according to a preset rule. Optionally, since the sub-blocks above, top left, and left of sub-block D have been encoded or decoded, the non-adjacent pixel samples of sub-block D may include pixel samples located within the current block. Then, based on the prediction modes corresponding to the surrounding pixel samples of sub-block D, the area amplitude histogram of sub-block D is determined, and the prediction mode with the largest amplitude in each area amplitude histogram is used as the prediction mode of sub-block D. The predictor of sub-block D is determined or obtained based on the prediction mode of sub-block D. Optionally, the above-mentioned prediction mode includes an angular prediction mode and / or a non-angular prediction mode. Optionally, the above-mentioned predictor includes an angular predictor and / or a non-angular predictor.

[0586] In this embodiment, since the first sub-block and the second sub-block are both part of the current block and there is a strong correlation between the two, a more appropriate prediction mode can be selected to perform sub-block prediction based on the inherent correlation between the first sub-block and the second sub-block, thereby improving the prediction accuracy.

[0587] Fifth embodiment

[0588] The present application also provides a processing device, as shown in FIG29 . FIG29 is a schematic diagram of the functional modules of the processing device of the present application, which can be provided in or on a processing device. The processing device includes:

[0589] The processing module A10 is configured to determine or generate a prediction result of the current block according to the non-angle predictor.

[0590] Optionally, the non-angular predictor is associated with at least one non-angular prediction mode; and / or the current block includes at least one sub-block.

[0591] Optionally, the prediction result of the current block is determined or generated by a non-angular predictor associated with at least one non-angular prediction mode of at least one sub-block;

[0592] The prediction result of the current block is determined or obtained based on the non-angular predictor associated with the first mode;

[0593] Non-angular prediction modes include at least one of the following:

[0594] Intra-frame prediction mode based on neural network;

[0595] Intra-frame prediction mode based on extrapolation filter;

[0596] Block vector prediction mode;

[0597] Plane mode;

[0598] DC mode.

[0599] Optionally, the first model is applied to the neural network based intra-frame prediction mode; and / or, when the first mode includes the planar mode, if the current block meets the enabling conditions of the neural network based intra-frame prediction mode, the neural network based intra-frame prediction mode will replace the planar mode.

[0600] Optionally, the first model is determined or obtained based on at least one of a width, a height, a block size, and a block area of ​​the current block;

[0601] The first model is determined or obtained according to at least one of the width, height, size and area of ​​at least one reference region of the current block;

[0602] The first model is determined or obtained based on at least one of the width, height, size and area of ​​at least one reference block of the current block;

[0603] The method of determining or obtaining the reference area and / or reference block includes at least one of the following:

[0604] According to at least one of an upper adjacent pixel, an upper non-adjacent pixel, a left adjacent pixel, a left non-adjacent pixel, an upper left adjacent pixel, and an upper left non-adjacent pixel of the current block;

[0605] According to at least one of a neighbor block, a non-neighbor block, a co-located block, a time domain block and a default block corresponding to the current block;

[0606] According to at least one of the width, height, block size and block area of ​​the current block;

[0607] A candidate block determined or obtained based on a candidate motion vector or a candidate block vector of a current block;

[0608] If the first information of the current block satisfies the first condition, the reference region is the first reference region and / or the reference block is the first reference block;

[0609] If the first information of the current block does not satisfy the first condition, the reference area is the second reference area and / or the reference block is the second reference block.

[0610] Optionally, the first information of the current block satisfies a first condition, including at least one of the following:

[0611] The value of the first information is a first value;

[0612] The value of the first information is within a first numerical range;

[0613] The value of the first information is greater than or equal to the first threshold;

[0614] The value of the first information is less than or equal to the second threshold.

[0615] Optionally, the processing module A10 is further configured to:

[0616] Performing prediction processing on the current block according to at least one non-angle prediction mode, determining or generating at least one non-angle predictor, and determining or generating a prediction result of the current block according to the at least one non-angle predictor;

[0617] performing prediction processing on the current block according to at least one non-angle prediction mode and at least one angular prediction mode, respectively, to determine or generate at least one non-angle predictor and at least one angular predictor, and determining or generating a prediction result of the current block according to the at least one non-angle predictor and the at least one angular predictor;

[0618] Determine or obtain a first non-angle predictor from the non-angle predictors, and determine or generate a prediction result of the current block based on the first non-angle predictor;

[0619] Determining or obtaining a second mode according to rate-distortion costs corresponding to each non-angular prediction mode, and determining or generating a prediction result of the current block according to a non-angular predictor associated with the second mode;

[0620] A prediction result of the current block is determined or generated based on the non-angle predictor and the angle predictor.

[0621] Optionally, the non-angle predictor and / or the angle predictor is determined or obtained according to at least one of the following:

[0622] at least one reference region of the current block;

[0623] at least one reference block of the current block;

[0624] At least one reference pixel of the current block.

[0625] The processing device provided in the embodiment of the present application has similar implementation principles and beneficial effects to the technical solutions shown in the above-mentioned corresponding method embodiments, and will not be described in detail here.

[0626] An embodiment of the present application further provides a processing device, including a memory and a processor, wherein a processing program is stored in the memory, and when the processing program is executed by the processor, the steps of the processing method in any of the above embodiments are implemented.

[0627] An embodiment of the present application further provides a storage medium having a processing program stored thereon. When the processing program is executed by a processor, the steps of the processing method in any of the above embodiments are implemented.

[0628] In the embodiments of the processing device and storage medium provided in this application, all technical features of any of the above-mentioned processing method embodiments may be included. The expanded and explained contents of the specification are basically the same as those of the embodiments of the above-mentioned methods and will not be repeated here.

[0629] An embodiment of the present application further provides a computer program product, which includes computer program code. When the computer program code runs on a computer, the computer executes the methods in the various possible implementation modes described above.

[0630] An embodiment of the present application also provides a chip, including a memory and a processor, wherein the memory is used to store computer programs, and the processor is used to call and run the computer programs from the memory, so that a device equipped with the chip executes the methods in the various possible implementation modes as described above.

[0631] It can be understood that the above scenarios are merely examples and do not constitute a limitation on the application scenarios of the technical solutions provided in the embodiments of this application. The technical solutions of this application can also be applied to other scenarios. For example, ordinary technicians in this field know that with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0632] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0633] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0634] The units in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0635] In this application, the same or similar terminology, technical solutions and / or application scenario descriptions are generally only described in detail the first time they appear. When they appear again later, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of this application, for the same or similar terminology, technical solutions and / or application scenario descriptions that are not described in detail later, you can refer to the previous relevant detailed descriptions.

[0636] In this application, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0637] The various technical features of the technical solution of this application can be combined arbitrarily. In order to make the description concise, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0638] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as mentioned above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, controlled terminal, or network device, etc.) to execute the method of each embodiment of the present application.

[0639] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a storage disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state storage disk Solid State Disk (SSD)).

[0640] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A processing method, wherein, Includes steps: S10, determining or generating a prediction result of the current block according to the non-angle predictor.

2. The processing method according to claim 1, wherein The non-angular predictor is associated with at least one non-angular prediction mode; and / or the current block includes at least one sub-block.

3. The processing method according to claim 2, wherein, Also includes at least one of the following: The prediction result of the current block is determined or generated by a non-angular prediction sub-module associated with at least one non-angular prediction mode of at least one sub-block; The prediction result of the current block is determined or obtained according to the non-angle predictor associated with the first mode; Non-angular prediction modes include at least one of the following: Intra-frame prediction mode based on neural network; Intra-frame prediction mode based on extrapolation filter; Block vector prediction mode; Plane mode; DC mode.

4. The processing method according to claim 3, wherein, The first model is applied to the neural network based intra-frame prediction mode; and / or, when the first mode includes a planar mode, if the current block meets the enabling conditions of the neural network based intra-frame prediction mode, the neural network based intra-frame prediction mode will replace the planar mode.

5. The processing method according to claim 4, wherein, Also includes at least one of the following: The first model is determined or obtained according to at least one of the width, height, block size and block area of the current block; The first model is determined or obtained according to at least one of the width, height, size and area of at least one reference area of the current block; The first model is determined or obtained according to at least one of the width, height, size and area of at least one reference block of the current block; The method of determining or obtaining the reference area and / or reference block includes at least one of the following: According to at least one of an upper adjacent pixel, an upper non-adjacent pixel, a left adjacent pixel, a left non-adjacent pixel, an upper left adjacent pixel, and an upper left non-adjacent pixel of the current block; According to at least one of a neighbor block, a non-neighbor block, a co-located block, a time domain block and a default block corresponding to the current block; According to at least one of the width, height, block size and block area of the current block; A candidate block determined or obtained based on a candidate motion vector or a candidate block vector of a current block; If the first information of the current block satisfies the first condition, the reference region is the first reference region and / or the reference block is the first reference block; If the first information of the current block does not satisfy the first condition, the reference area is the second reference area and / or the reference block is the second reference block.

6. The processing method according to claim 5, wherein, The first information of the current block satisfies the first condition, including at least one of the following: The value of the first information is a first value; The value of the first information is within a first numerical range; The value of the first information is greater than or equal to the first threshold; The value of the first information is less than or equal to the second threshold.

7. The processing method according to claim 1, wherein, Step S10 includes at least one of the following: Performing prediction processing on the current block according to at least one non-angle prediction mode, determining or generating at least one non-angle predictor, and determining or generating a prediction result of the current block according to the at least one non-angle predictor; Predicting the current block according to at least one non-angle prediction mode and at least one angle prediction mode, respectively, to determine or generate at least one non-angle prediction sub-module and at least one angle prediction sub-module, and determining or generating a prediction result of the current block according to the at least one non-angle prediction sub-module and the at least one angle prediction sub-module; Determine or obtain a first non-angle predictor from the non-angle predictors, and determine or generate a prediction result of the current block according to the first non-angle predictor; Determine or obtain a second mode according to the rate-distortion cost corresponding to each non-angle prediction mode, and determine or generate a prediction result of the current block according to the non-angle prediction sub related to the second mode; Determine or generate a prediction result of the current block according to the non-angle prediction sub and the angle prediction sub.

8. The processing method according to claim 7, wherein, The non-angle prediction sub and / or the angle prediction sub are determined or obtained according to at least one of the following: At least one reference region of the current block; At least one reference block of the current block; At least one reference pixel of the current block.

9. A processing device, wherein, Including: A memory and a processor, wherein a processing program is stored on the memory, and when the processing program is executed by the processor, the steps of the processing method described in claim 1 are implemented.

10. A storage medium, wherein, A computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the processing method described in claim 1 are implemented.

Images