Coding unit division decision method, decoding, transcoding method, device and medium

Abstract

The present invention provides a coding unit division decision method, decoding method, transcoding method, device, and medium, wherein the division decision method includes: dividing the coding unit to obtain multiple sub-blocks; determining coding information for encoding each sub-block; determining the characteristics of the coding unit based on the coding information of each sub-block, the characteristics of the coding unit being used to determine whether the coding unit skips a subsequent division mode; before performing rate-distortion optimization processing on the coding unit for the subsequent division mode, determining whether the characteristics of the coding unit match target characteristics, the target characteristics being characteristics corresponding to the coding unit skipping the subsequent division mode; if the characteristics of the coding unit match the target characteristics, determining whether the coding unit skips the subsequent division mode. The present invention can reduce the coding complexity of the coding unit and improve coding efficiency.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of video coding and decoding technology, and specifically to a coding unit division decision method, decoding, transcoding method, device and medium. Background Art

[0002] A coding unit (CU) is the basic unit for encoding a video frame. It can be further divided into coding tree units (CTUs) after the video frame is divided into coding tree units (CTUs). When encoding a CU, the CU can be divided layer by layer. However, when dividing the CU at each layer, multiple division modes are tested. The division mode to use at each layer is determined by calculating the coding cost (e.g., rate-distortion optimization) of each division mode.

[0003] It can be seen that when encoding a coding unit, the division mode used by the coding unit at each layer needs to be determined by traversing and trying multiple division modes, which results in a high encoding complexity of the coding unit. Summary of the Invention

[0004] In view of this, the embodiments of the present application provide a method for deciding the division of a coding unit, a decoding method, a transcoding method, a device and a medium. After the coding unit is divided into sub-blocks, a mechanism is provided for the coding unit to skip subsequent division modes. In this way, when the coding unit can skip subsequent division modes, the need to try multiple division modes for the coding unit is avoided, the coding complexity of the coding unit is reduced, and the coding efficiency is improved.

[0005] To achieve the above objectives, the embodiments of the present application provide the following mechanism.

[0006] In a first aspect, an embodiment of the present application provides a method for deciding the division of coding units, including:

[0007] Dividing the coding unit to obtain a plurality of sub-blocks;

[0008] Determining coding information for encoding each sub-block;

[0009] Determining, based on the coding information of each sub-block, a characteristic of the coding unit, wherein the characteristic of the coding unit is used to decide whether the coding unit skips a subsequent division mode;

[0010] Before performing rate-distortion optimization processing of a subsequent division mode on the coding unit, determining whether a feature of the coding unit matches a target feature, the target feature being a feature corresponding to a subsequent division mode skipped by the coding unit;

[0011] If the feature of the coding unit matches the target feature, it is decided that the coding unit skips the subsequent division mode.

[0012] In a second aspect, an embodiment of the present application provides a decoding method, including:

[0013] Get the code stream to be decoded;

[0014] Determining, from the bitstream, a subsequent partitioning decision result of a coding unit in a video frame after the coding unit is divided into sub-blocks; determining the subsequent partitioning decision result based on a matching result between a feature of the coding unit and a target feature, the target feature being a feature corresponding to a mode in which the coding unit skips a subsequent partitioning; and determining the feature of the coding unit based on coding information of each sub-block of the coding unit, wherein the coding unit is divided into a plurality of sub-blocks;

[0015] If the subsequent partitioning decision result indicates that the coding unit skips the subsequent partitioning mode after partitioning the sub-blocks, it is determined that the coding unit skips the subsequent partitioning mode.

[0016] In a third aspect, an embodiment of the present application provides a transcoding method, which is applied to a cloud service platform. The method includes:

[0017] Obtain a cloud transcoding request, where the cloud transcoding request is used to request transcoding of a video in a first format into a second format;

[0018] Decoding the video in the first format using the decoding method described in the second aspect to obtain a decoded video;

[0019] When encoding the decoded video into the second format, using the coding unit division decision method described in the first aspect above to determine a division mode for the coding unit, and performing video encoding based on the determined division mode to obtain an encoded video in the second format;

[0020] Output the video in the second format.

[0021] In a fourth aspect, an embodiment of the present application provides an electronic device comprising at least one memory and at least one processor, wherein the memory stores one or more computer-executable instructions, and the processor calls the one or more computer-executable instructions to implement the coding unit division decision method as described in the first aspect above, or the decoding method as described in the second aspect above, or the transcoding method as described in the third aspect above.

[0022] In a fifth aspect, an embodiment of the present application provides a storage medium, which stores one or more computer-executable instructions. When the one or more computer-executable instructions are executed, the coding unit division decision method as described in the first aspect above, or the decoding method as described in the second aspect above, or the transcoding method as described in the third aspect above are implemented.

[0023] In a sixth aspect, an embodiment of the present application provides a computer program, which, when executed, implements the coding unit division decision method as described in the first aspect above, or the decoding method as described in the second aspect above, or the transcoding method as described in the third aspect above.

[0024] The coding unit division decision method provided in the embodiment of the present application can divide the coding unit to obtain multiple sub-blocks when encoding the coding unit, and determine the coding information used to encode each sub-block; thereby, the embodiment of the present application can determine the characteristics of the coding unit based on the coding information of each sub-block; and then before performing rate-distortion optimization processing of the subsequent division mode on the coding unit, the embodiment of the present application can determine whether the characteristics of the coding unit match the target characteristics, and the target characteristics are the characteristics corresponding to the coding unit skipping the subsequent division mode; if the characteristics of the coding unit match the target characteristics, the embodiment of the present application can decide that the coding unit skips the subsequent division mode, thereby avoiding the need to try multiple division modes for the coding unit in the future, reducing the coding complexity of the coding unit, and thus improving the coding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0026] Figure 1A Schematic diagram of dividing a video frame into multiple coding tree units.

[0027] Figure 1B These are example diagrams for partitioning of horizontal binary trees, vertical binary trees, horizontal ternary trees, and vertical ternary trees.

[0028] Figure 2 Flowchart of the method for determining target features provided in an embodiment of the present application.

[0029] Figure 3A An example diagram for determining coding information of sub-block samples.

[0030] Figure 3BAn example diagram of the relationship between the features of the coding unit.

[0031] Figure 3C This is an example diagram of a feature file.

[0032] Figure 4 A flow chart of a method for determining first target segmentation features provided in an embodiment of the present application.

[0033] Figure 5 A flowchart of a coding unit division decision method provided in an embodiment of the present application.

[0034] Figure 6 Another flowchart of the coding unit division decision method provided in an embodiment of the present application.

[0035] Figure 7A A flowchart of a decoding method provided in an embodiment of the present application.

[0036] Figure 7B A flowchart of the transcoding method provided in an embodiment of the present application.

[0037] Figure 8 A block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0039] With the development of cloud services, individual users and businesses (such as video service providers) can leverage the cloud to implement various video transcoding services, including live streaming, on-demand streaming, and video calls. Cloud transcoding can be considered a cloud computing technology that converts videos via a server (such as a cloud server) into a format suitable for playback on end devices. Cloud service providers can provide external media transcoding services to enable individual and business users to utilize cloud-based video transcoding services.

[0040] The cloud transcoding process and the traditional video processing process involve video encoding. When encoding videos, video encoding standards such as HEVC (High Efficiency Video Coding) and VVC (Versatile Video Coding) can be used.

[0041] As a new-generation international standard for video coding, VVC (Versatile Video Coding) significantly expands the functionality and product forms of video coding. It can provide professional encoding tools for computer-generated video content such as original screen sharing, cloud-based collaboration, and cloud gaming. It can also provide independent sub-screens for applications such as tiled streaming of 360-degree panoramic videos, enabling higher resolution video images within the viewing angle. In addition, VVC's product forms cover all links in the video industry chain, such as professionally produced copyrighted content, UGC (User Generated Content) captured in daily life, conference videos, live videos, on-demand videos, sports events, HDR (High Dynamic Range) videos, panoramic videos, surveillance videos, and other video categories. Electronic devices that use VVC include a variety of terminal devices commonly used by users, such as mobile phones, computers, cameras, set-top boxes, TVs, and head-mounted devices.

[0042] As users' demands for video quality continue to increase, video coding standards such as HEVC and VVC are also constantly evolving. However, while video compression rates continue to improve, the complexity of video coding is also increasing exponentially. Therefore, based on video coding standards such as HEVC and VVC, how to reduce the complexity of video coding without affecting the compression performance of video coding has become a hot research issue in the video coding field.

[0043] It should be noted that video coding such as VVC is based on block design and implementation. Therefore, when encoding a video, each frame of the video needs to be divided into a series of coding tree units, and then the coding tree units are further divided into coding units for actual encoding. In an example, Figure 1A The schematic diagram of dividing a video frame into multiple coding tree units is shown as an example, which can be used for reference. Figure 1A As shown, Figure 1A A small box in the image can be considered as a coding tree unit divided from the video frame. A video frame is usually divided into three channels (one luminance channel and two chrominance channels), so a coding tree unit can also contain one luminance channel and two chrominance channels.

[0044] The coding unit is the basic unit for encoding video frames. When using the VVC coding standard, VVC, in addition to continuing to use the quadtree for quad partitioning, also adds multi-type trees (MTT), such as horizontal partitioning trees and vertical partitioning trees. Horizontal partitioning trees are further divided into horizontal binary trees and horizontal ternary trees, while vertical partitioning trees are further divided into vertical binary trees and vertical ternary trees.

[0045] Based on this, the coding unit can be divided into multiple levels through a quadtree or a multi-type tree (MTT). Among them, the multi-type tree can include a horizontal binary tree, a vertical binary tree, a horizontal ternary tree and a vertical ternary tree, etc. The horizontal binary tree is used to implement horizontal binary division, the vertical binary tree is used to implement vertical binary division, the horizontal ternary tree is used to implement horizontal ternary division, and the vertical ternary tree is used to implement vertical ternary division. As an example, Figure 1B The following diagrams exemplify the division of a horizontal binary tree, a vertical binary tree, a horizontal ternary tree, and a vertical ternary tree, which can be used for reference.

[0046] When dividing the coding unit into multiple levels, based on the fact that the coding unit has multiple division modes (such as quadrifugal division, horizontal binary division, vertical binary division, horizontal trifurcated division, and vertical trifurcated division, etc.), when dividing the coding unit layer by layer, the division of the coding unit at each level needs to try the above-mentioned multiple division modes, so as to determine the target division mode with the minimum rate-distortion optimization, which is used for the actual division of the coding unit at each level; at the same time, the sub-blocks obtained by the division of the coding unit need to try the above-mentioned multiple division modes in turn, so as to determine the target division mode with the minimum rate-distortion optimization, which is used for the actual division of the sub-blocks. In the above manner, the coding unit can be divided layer by layer to realize the multi-level division of the coding unit; however, the division of the coding unit at each level needs to try multiple division methods, so as to find the target division mode with the minimum rate-distortion optimization, which results in a very high complexity in the encoding of the coding unit and a long encoding time.

[0047] Based on this, an embodiment of the present application provides an improved coding unit division decision scheme. After the coding unit is divided to obtain sub-blocks, the coding information of the further encoded sub-blocks is used to determine the characteristics of the coding unit; thereby, based on the characteristics of the coding unit, a decision is made as to whether the coding unit can skip the subsequent division mode, so as to provide a mechanism for the coding unit to skip the subsequent division mode; and further, when the coding unit can skip the subsequent division mode, the subsequent need to try multiple division modes for the coding unit is avoided, thereby reducing the coding complexity of the coding unit and improving the coding efficiency.

[0048] Based on the above ideas, the embodiments of the present application can determine the characteristics corresponding to the coding unit being able to skip the subsequent division mode. For the convenience of explanation, the characteristics corresponding to the coding unit skipping the subsequent division mode can be called target characteristics; thus, when encoding the coding unit, the embodiments of the present application can divide the coding unit into sub-blocks, and use the coding information of the sub-blocks for further encoding to determine the characteristics of the coding unit; and then match the characteristics of the coding unit with the target characteristics, so that when the characteristics of the coding unit match the target characteristics, it is decided that the coding unit can skip the subsequent division mode (for example, the coding unit can skip the subsequent horizontal division mode, vertical division mode, etc.).

[0049] As an optional implementation, Figure 2 The flowchart of the optional method for determining target features provided by the embodiment of the present application is exemplified. The method flow can use an offline statistical method to determine the target features of the coding unit that can skip the subsequent division mode; as an optional implementation, the features of the coding unit can be in the form of a vector (for example, a feature vector of the coding unit). Optionally, the method flow can be implemented by an encoder, for example, an encoder set in a server; refer to Figure 2 The method flow may include the following steps.

[0050] In step S210 , a plurality of coding unit samples are determined.

[0051] The coding unit samples may be coding units used by embodiments of the present application to determine target features, and may be obtained by dividing video frame samples; the video frame samples may be selected from a video sample sequence. In some embodiments, embodiments of the present application may select one or more video sample sequences and encode multiple frames of video frame samples for the one or more video sample sequences; thus, when encoding each frame of video frame samples, embodiments of the present application may divide each frame of video frame samples into multiple coding tree unit samples, and further divide each coding tree unit sample into multiple coding unit samples, so as to determine multiple coding unit samples used by embodiments of the present application to determine target features when encoding multiple frames of video frame samples in one or more video sample sequences.

[0052] It should be noted that the video sample sequence selected in the embodiment of the present application may be one or more, and the video frame samples selected for encoding from one or more video sample sequences may be multiple frames; when encoding a frame of video frame samples, a video frame sample can be divided into multiple coding tree unit samples, and a coding tree unit sample can be further divided into multiple coding unit samples by quadrilateral division or the like.

[0053] As an optional implementation, an embodiment of the present application may select multiple different types of video sample sequences, and then, under preset configuration conditions, use different QPs (Quantitative parameters) to encode multiple frames of video frame samples for the multiple different types of video sample sequences (for example, for 6 different types of video sample sequences, encode 10 frames of video frame samples), so that in the process of encoding each frame of video frame samples, the coding tree unit samples of each frame of video frame samples are divided into coding unit samples through quadrilateral division.

[0054] In one implementation example, based on the VVC video coding standard, the selected multiple different types of video sample sequences (e.g., six different types of video sample sequences) can be from a test sequence officially released by VVC. The multiple different types of video sample sequences have different resolutions, motion scenes, and texture features. As an example, under the VVC video coding standard, the preset configuration conditions for encoding video frame samples in the video sample sequence can be as shown in Table 1 below.

[0055]

[0056] Table 1

[0057] It should be noted that CampfireParty has very complex textures and intense motion scenes; FourPeople has very simple textures, mostly flat background blocks, and basically no moving objects; RaceHorsesC and DaylightRoad2 have relatively smooth motion and moderately complex scenes; the BasketballPass sequence has many moving objects with rapidly changing content; CampfireParty and DaylightRoad2 are newly added 4K ultra-high-definition (UHD) test sequences for VVC.

[0058] In step S211 , for each coding unit sample, a plurality of sub-block samples obtained by dividing the coding unit sample is determined.

[0059] Based on the multiple coding unit samples used to determine the target features in the embodiment of the present application, for each coding unit sample, the embodiment of the present application can divide the coding unit sample into multiple sub-block samples.

[0060] In some embodiments, a coding unit sample may be quad-partitioned to obtain multiple sub-block samples. For example, a coding unit sample may be quad-partitioned to obtain four sub-block samples. In other embodiments, a coding unit sample may be partitioned based on a multi-type tree to obtain multiple sub-block samples.

[0061] As an optional implementation, when the maximum allowable division level of the multi-type tree structure is 1, the embodiment of the present application may divide the coding unit sample by a quadripartition method to obtain multiple sub-block samples. In other possible implementations, when the maximum allowable division level of the multi-type tree structure is greater than 1, the embodiment of the present application may divide the coding unit sample by a quadripartition method or a multi-type tree method to obtain multiple sub-block samples.

[0062] It should be noted that, in the embodiment of the present application, after dividing the coding unit sample into sub-block samples, the coding information of the sub-block samples is further encoded to determine the characteristics of the coding unit sample, thereby counting the target characteristics of the coding unit that can skip the subsequent division mode; therefore, when the multi-type tree structure allows a maximum of 1 layer of division, in order to support further encoding of the sub-block samples, the embodiment of the present application needs to divide the coding unit sample into sub-block samples through quadripartitioning, so that the sub-block samples can determine the coding information for further encoding when the multi-type tree structure allows a maximum of 1 layer of division.

[0063] It should be further explained that when the maximum allowed division level of the multi-type tree is 1 (i.e., Max MTT Depth = 1), it only limits the coding unit to be divided only once under the multi-type tree structure, and does not limit the number of divisions of the coding unit in the quadtree. Therefore, the sub-blocks of the coding unit are obtained by quad-division. At this time, even if the multi-type tree structure only allows a maximum of one layer of division, the embodiment of the present application can also obtain the coding information of the sub-block further encoded under the multi-type tree structure (further encoding of the sub-block may include the process of further dividing the sub-block), thereby obtaining the characteristics of the coding unit based on the coding information further encoded by the sub-block.

[0064] In other possible implementations, if the maximum allowable division level of the multi-type tree structure is greater than 1, the embodiment of the present application can determine the coding information for further encoding for the sub-block samples even if the coding unit samples are divided into sub-block samples through the multi-type tree. Therefore, when the maximum allowable division level of the multi-type tree structure is greater than 1, the embodiment of the present application can divide the coding unit samples through the multi-type tree method or the quadripartition method to obtain multiple sub-block samples.

[0065] In step S212, for each coding unit sample, coding information of each sub-block sample of the coding unit sample is determined.

[0066] After obtaining multiple sub-block samples of each coding unit sample, the embodiment of the present application can determine the coding information of the multiple sub-block samples of the coding unit sample for each coding unit sample; wherein the coding information of the sub-block sample is used for further encoding the sub-block sample.

[0067] It should be noted that, in the process of video encoding, coding units, sub-blocks and other image blocks may have multiple division modes and multiple intra-frame direction modes; among them, the multiple division modes of coding units, sub-blocks and other image blocks may include: no division class, quadrature division class, horizontal division class and vertical division class and other division modes; at the same time, the multiple intra-frame direction modes of coding units, sub-blocks and other image blocks may include: horizontal direction class, vertical direction class, diagonal mode class and other intra-frame direction modes.

[0068] In some embodiments, when further encoding a subblock, the encoding information for the subblock may include a target partition mode and a target intra-frame direction mode. The target partition mode for the subblock can be considered as a partition mode determined from among multiple partition modes for the subblock; and the target intra-frame direction mode for the subblock can be considered as an intra-frame direction mode determined from among multiple intra-frame direction modes for the subblock. As an optional implementation, for a subblock, embodiments of the present application may try multiple partition modes for the subblock and determine the rate-distortion optimization corresponding to each partition mode, thereby selecting the partition mode with the lowest rate-distortion optimization as the target partition mode for the subblock. As an optional implementation, for a subblock, embodiments of the present application may try multiple intra-frame direction modes for the subblock and determine the rate-distortion optimization corresponding to each intra-frame direction mode, thereby selecting the intra-frame direction mode with the lowest rate-distortion optimization as the target intra-frame direction mode for the subblock. Furthermore, the target partition mode and target intra-frame direction mode for the subblock may form the encoding information used to encode the subblock.

[0069] As an optional implementation, after determining multiple sub-block samples of each coding unit sample, the embodiment of the present application can traverse and try multiple division modes of the sub-block samples for any sub-block sample in any coding unit, so as to determine the division mode with the minimum rate-distortion optimization as the target division mode of the sub-block sample; at the same time, traverse and try multiple intra-frame direction modes of the sub-block samples, so as to determine the intra-frame direction mode with the minimum rate-distortion optimization as the target intra-frame direction mode of the sub-block sample.

[0070] In an implementation example, taking the sub-block samples of the coding unit samples obtained by quadrilateral division as an example, Figure 3A The following example diagrams are exemplarily shown for determining the coding information of sub-block samples, such as Figure 3AAs shown, assume that the embodiment of the present application has n coding unit samples 1 to n, wherein one coding unit sample can obtain four sub-block samples through quadrilateral partitioning. For example, coding unit sample 1 can obtain four sub-block samples 10 to 13 through quadrilateral partitioning. Taking the determination of the coding information of each sub-block sample in coding unit sample 1 as an example, the embodiment of the present application can traverse and try multiple partitioning modes for sub-block sample 10, such as no partitioning, quadrilateral partitioning, horizontal partitioning, and vertical partitioning, and determine the rate-distortion optimization of sub-block sample 10 under each partitioning mode, thereby selecting the partitioning mode with the minimum rate-distortion optimization as the target partitioning mode for sub-block sample 10. Simultaneously, multiple intra-frame direction modes, such as horizontal direction, vertical direction, and diagonal direction, are traversed and tried for sub-block sample 10, and the rate-distortion optimization of sub-block sample 10 under each intra-frame direction mode is determined, thereby selecting the intra-frame direction mode with the minimum rate-distortion optimization as the target intra-frame direction mode for sub-block sample 10. Consequently, the target partitioning mode and the target intra-frame direction mode for sub-block sample 10 form the coding information for sub-block sample 10. In this example, the embodiment of the present application can determine the coding information of sub-block sample 10, and similarly determine the coding information of sub-block samples 11, 12 and 13 in coding unit sample 1, as well as the coding information of each sub-block sample in other coding unit samples, which will not be further explained here.

[0071] In step S213 , for each coding unit sample, a feature of the coding unit sample is determined according to the coding information of each sub-block sample of the coding unit sample.

[0072] For any coding unit sample, the embodiment of the present application can determine the characteristics of the coding unit sample based on the coding information of each sub-block sample in the coding unit sample.

[0073] In some embodiments, the features of a coding unit may be formed by a partition feature and a directional feature of the coding unit; wherein the partition feature of the coding unit may be determined based on a target partition pattern for each subblock in the coding unit; and the directional feature of the coding unit may be determined based on a target intra-frame directional pattern for each subblock in the coding unit. As an optional implementation, the partition feature may be a partition feature vector in vector form, and the directional feature may be a directional feature vector in vector form.

[0074] In the stage of offline statistical target features, based on multiple coding unit samples used to determine the target features, for any one of the coding unit samples, the embodiment of the present application can determine the division features and directional features of the coding unit sample, thereby forming the features of the coding unit from the division features and directional features of the coding unit sample.

[0075] In some embodiments, for any coding unit sample, when determining the division characteristics of the coding unit sample, the embodiment of the present application may determine the division characteristics of the coding unit sample according to the target division mode of each sub-block sample in the coding unit sample. As an optional implementation, for any coding unit sample, the embodiment of the present application may map the target division mode of each sub-block sample in the coding unit sample to a string, thereby obtaining the division characteristics of the coding unit sample. For example, in combination with Figure 3A As shown, when determining the division characteristics of coding unit sample 1, the embodiment of the present application can map the target division pattern of the four sub-block samples 10, 11, 12 and 13 of coding unit sample 1 into a character string, thereby obtaining the division characteristics of coding unit sample 1. Figure 3A The other coding unit samples in can be divided into features in the same way.

[0076] In one example, the embodiment of the present application may define the division feature of the coding unit as F split (f subSplit0 ,f subSplit1 ,f subSplit2 ,f subSplit3 ), which can be applied to the coding unit samples used to determine target features in the offline statistical stage, as well as to the decision of whether to skip the coding unit of the subsequent division mode during actual video encoding;

[0077] Among them, f subSplit0 ,f subSplit1 ,f subSplit2 ,f subSplit3 The target partitioning mode of the four sub-blocks of a coding unit is represented by the target partitioning mode of the four sub-blocks. The target partitioning mode of the four sub-blocks is collected (the collection form can be in the form of a string) to obtain the partitioning characteristics of the coding unit. That is, for the i-th sub-block in a coding unit, the target partitioning mode of the i-th sub-block can be defined as

[0078] In a further optional implementation, in order to save time and space costs, the embodiment of the present application may specify the value of the target partition mode. After a coding unit is divided into multiple sub-blocks, the target partition mode of the i-th sub-block is The value range of can be: {0, 1, 2, 3, 4}; wherein, the target partition mode value of 0 represents the target partition mode as other / abnormal class, the target partition mode value of 1 represents the target partition mode as non-division class, the target partition mode value of 2 represents the target partition mode as quadripartition class, the target partition mode value of 3 represents the target partition mode as horizontal partition class, and the target partition mode value of 4 represents the target partition mode as vertical partition class. It should be noted that the above target partition mode values ​​can also be applied to coding units.

[0079] In some embodiments, for any coding unit sample, embodiments of the present application may determine the directional characteristics of the coding unit sample based on the target intra-frame directional modes of each sub-block sample of the coding unit sample and the target intra-frame directional mode of the coding unit sample. The target intra-frame directional mode of the coding unit sample may be determined by traversing and trying multiple intra-frame directional modes of the coding unit sample, thereby selecting the intra-frame directional mode with the minimum rate-distortion optimization as the target intra-frame directional mode of the coding unit sample.

[0080] As an optional implementation, for any coding unit sample, the embodiment of the present application can map the target intra-frame direction mode of each sub-block sample in the coding unit sample and the target intra-frame direction mode of the coding unit sample into a character string, thereby obtaining the directional feature of the coding unit sample. Figure 3A As shown, when determining the directional features of the coding unit sample 1, the embodiment of the present application can traverse and try multiple intra-frame directional modes such as the horizontal direction class, the vertical direction class, and the diagonal mode class for the coding unit sample 1, and determine the rate-distortion optimization corresponding to the various intra-frame directional modes of the coding unit sample 1, thereby selecting the intra-frame directional mode with the smallest rate-distortion optimization as the target intra-frame directional mode of the coding unit sample 1; and then after obtaining the target intra-frame directional mode of the coding unit sample 1 and the target intra-frame directional mode of the four sub-block samples 10, 11, 12 and 13 of the coding unit sample 1, the embodiment of the present application can map the target intra-frame directional mode of the four sub-block samples of the coding unit sample 1 and the target intra-frame directional mode of the coding unit sample 1 into a character string, thereby obtaining the directional features of the coding unit sample 1. Figure 3A The directional features can be obtained similarly for the other coding unit samples in .

[0081] In one example, the embodiment of the present application may define the directional feature of the coding unit as F Dir (f subDir0 ,f subDir1 ,f subDir2 ,f subDir3 ,f curDir ), which can be applied to the coding unit samples used to determine target features in the offline statistical stage, as well as to the decision of whether to skip the coding unit of the subsequent division mode during actual video encoding;

[0082] Among them, f subDir0 ,f subDir1 ,f subDir2 ,f subDir3 Indicates the target intra-frame direction mode of the four sub-blocks divided into a coding unit, f curDirRepresents a target intra-frame direction mode of a coding unit, thereby combining the target intra-frame direction mode of a coding unit and the target intra-frame direction modes of the four sub-blocks of the coding unit (the set form can be in a string form) to obtain the directional feature of the coding unit.

[0083] For a coding unit, the target intra-frame direction mode of the i-th sub-block can be defined as In a further optional implementation, the embodiment of the present application may specify the value of the target intra-frame direction mode. After a coding unit is divided into four sub-blocks by quadrilateral, the target intra-frame direction mode of the i-th sub-block is and the target intra direction mode f of the coding unit curDir The value range of can be: {0, 1, 2, -1}; among them, if the target intra-frame direction mode is the horizontal direction class of modes 9 to 27, the value is 0; if the target intra-frame direction mode is the vertical direction class of modes 39 to 57, the value is 1; if the target intra-frame direction mode is an abnormal mode class less than mode -14 or greater than mode 80, the value is -1; if the target intra-frame direction mode is other mode classes other than the above modes, the target mode model is the diagonal mode class, and the value is 2.

[0084] For any coding unit sample, the embodiment of the present application obtains the division feature and directional feature of the coding unit sample, and can form the feature of the coding unit sample from the division feature and directional feature of the coding unit sample.

[0085] In an example, assuming that after a coding unit sample is quad-partitioned, the target partition modes of the four sub-block samples are: horizontal partition class, horizontal partition class, no partition class, vertical partition class; and the target intra-frame direction modes of the four sub-block samples are: 10, 19, 40, 63; at the same time, the target intra-frame direction mode of the coding unit sample is 25; then the partition feature of the coding unit sample can be expressed as:

[0086] F split (f subSplit0 ,f subSplit1 ,f subSplit2 ,f subSplit3 ) = "3,3,1,4";

[0087] The directional characteristics of the coding unit sample can be expressed as: F Dir (f subDir0 ,f subDir1 ,f subDir2 ,f subDir3 ,f curDir )="0,0,1,2,0".

[0088] To facilitate further understanding, Figure 3A Taking the coding unit sample 1 as an example, Figure 3B The relationship example diagram of the characteristics of the coding unit is shown as follows: Figure 3B As shown, the features of coding unit sample 1 include the division features and directional features of coding unit sample 1; wherein, the target division patterns of the four sub-block samples of coding unit sample 1 collectively form the division features of coding unit sample 1; the target intra-frame directional patterns of the four sub-block samples of coding unit sample 1, and the target intra-frame directional pattern of coding unit sample 1 collectively form the directional features of coding unit sample 1.

[0089] In step S214, for each coding unit sample, multiple division modes are tried for the coding unit sample, and the division mode with the minimum rate-distortion optimization is selected as the target division mode of the coding unit sample.

[0090] In step S215 , a target partitioning mode and corresponding features of each coding unit sample are determined.

[0091] In the process of offline statistical target features, the embodiment of the present application can determine the target partitioning mode used for subsequent partitioning for each coding unit sample by traversing and trying multiple partitioning modes. For example, for any coding unit sample, the embodiment of the present application can traverse and try multiple partitioning modes such as no partitioning class, quad partitioning class, horizontal partitioning class, and vertical partitioning class for the coding unit sample, and determine the rate-distortion optimization of the coding unit in various partitioning modes, thereby selecting the partitioning mode with the minimum rate-distortion optimization as the target partitioning mode for the coding unit sample. The value of the target partitioning mode of the coding unit sample can be defined as M, and its value range can refer to the description of the corresponding part above.

[0092] Based on the characteristics of the coding unit samples and the target division pattern of the coding unit samples, the embodiments of the present application can correspond the target division pattern and characteristics of each coding unit, thereby obtaining the target division pattern and corresponding characteristics of each coding unit sample. Since the embodiments of the present application use multiple coding unit samples to determine the target characteristics, and different coding unit samples may have different target division patterns and characteristics, the embodiments of the present application can obtain the characteristics corresponding to the coding unit samples under different target division patterns.

[0093] In a further optional implementation, the embodiment of the present application may save the target division mode and corresponding features of each coding unit into a feature file; thereby using the data in the feature file to count the target features of the coding unit that can skip subsequent division modes.

[0094] In an implementation example, after dividing a coding unit sample to obtain multiple sub-block samples of the coding unit sample, the embodiment of the present application can try multiple division modes and multiple intra-frame direction modes for each sub-block sample, thereby determining the target division mode and target intra-frame direction mode of each sub-block sample; then, try multiple division modes and multiple intra-frame direction modes for the coding unit sample, thereby determining the target division mode and target intra-frame direction mode of the coding unit sample; furthermore, the target division mode of each sub-block sample of the coding unit sample can form the division feature of the coding unit sample; the target intra-frame direction mode of each sub-block sample of the coding unit sample and the target intra-frame direction mode of the coding unit sample can form the directional feature of the coding unit sample. After forming the feature of the coding unit sample based on the division feature and directional feature of the coding unit sample, the embodiment of the present application can determine the feature corresponding to the coding unit sample in the target division mode, and save the feature corresponding to each coding unit sample in the target division mode into a feature file. Combined with Figure 3A Example, Figure 3C An example diagram of a feature file is shown. Assuming that there are n coding unit samples, the feature file can store the target division mode and corresponding features of coding unit sample 1, the target division mode and corresponding features of coding unit sample 2, and so on. The target division mode and corresponding features of coding unit sample n. In one example, the form of the features of each coding unit can be combined Figure 3B Example shown.

[0095] In step S216, according to the target division mode and the corresponding features of each coding unit sample, the probability distribution of the target division mode corresponding to each coding unit sample under different features is statistically analyzed.

[0096] Based on the multiple coding unit samples used to determine the target feature in the embodiments of the present application, after determining the target partition mode and corresponding features for each coding unit sample, the embodiments of the present application can statistically calculate the probability distribution of various partition modes corresponding to the coding unit samples under different features. In other words, based on the target partition mode and corresponding features for each coding unit sample, the embodiments of the present application can determine the probability of selecting various partition modes as the target partition mode for the coding unit sample under a certain feature, thereby obtaining the probability distribution of the target partition modes corresponding to the coding unit samples under different features.

[0097] In some embodiments, when a feature file is used to store the target partition mode and corresponding features of each coding unit sample, the feature file may record the features and target partition mode of each coding unit sample at different QPs (the features of multiple coding unit samples recorded in the feature file may be different, and the target partition modes of the multiple coding unit samples may also be different). Thus, embodiments of the present application may statistically calculate the probability distribution of various partition modes as the target partition mode under different features of the coding unit samples to obtain the probability distribution of the target partition modes corresponding to the coding unit samples under different features. In other words, the probability of selecting various partition modes as the target partition mode under different features of the coding unit samples.

[0098] As an optional implementation, the features based on the coding unit include division features and direction features. The division features F of each coding unit sample can be recorded in the feature file. split (f subSplit0 ,f subSPlit1 ,f subSplit2 ,f subSplit3 ), directional feature F Dir (f subDir0 ,f subDir1 ,f subDir2 ,f subDir3 ,f curDir ), and the target division mode M of the coding unit sample; Thus, the embodiment of the present application can be based on the records in the feature file, in different F split (f subsplit0 ,f subSplit1 ,f subsplit2 ,f subSplit3 ) value, and different F Dir (f subDir0 ,f subDir1 ,f subDir2 ,f subDir3 ,f curDir ) values, the probability distribution of the target division mode M is statistically analyzed respectively, so as to obtain the probability distribution of the target division mode corresponding to the coding unit samples under different division features, and the probability distribution of the target division mode corresponding to the coding unit samples under different direction features.

[0099] In a further optional implementation, the embodiment of the present application can store the probability distribution of the target partition mode corresponding to the coding unit samples under different partition features in the probability distribution set Map0 (F split ,PV M ). Among them, Map0(F split ,PV M ) M Represents: a specific partition feature F of the coding unit samplesplit In this example, for example, in the probability distribution set Map0(F split ,PV M ), a partition feature F split (f subSplit0 ,f subSplit1 ,f subSplit2 ,f subSplit3 ) = "3,3,3,3", the probability distribution of the target partition mode corresponding to this partition feature is PV M ={0%, 4%, 2%, 93%, 1%}, then when the values ​​of the target partition mode M are set to {0, 1, 2, 3, 4} in sequence, the probability distribution PV of the target partition mode is M This can be expressed as follows: based on the above-mentioned division characteristics of the coding unit samples, the probability of a division mode with a value of 0 being the target division mode is 0%, the probability of a division mode with a value of 1 being the target division mode is 4%, the probability of a division mode with a value of 2 being the target division mode is 2%, the probability of a division mode with a value of 3 being the target division mode is 93%, and the probability of a division mode with a value of 4 being the target division mode is 1%. The meaning of each division mode value can be found in the description of the corresponding section above and will not be elaborated here.

[0100] In an example, when using four different QPs (22, 27, 32, 37) to encode 10 frames of video sample sequences of six different types, the probability distribution set Map0 (F split ,PV M ) stored in the partition feature F split (f subSplit0 ,f subSplit1 ,f subSplit2 ,f subSplit3 ) has 5x5x5x5 possible values.

[0101] In a further optional implementation, the embodiment of the present application can store the probability distribution of the target division mode corresponding to the coding unit sample under different directional features in the probability distribution set Map1 (F Dir ,PV M ). Among them, Map1(F Dir ,PV M PV in 0 M Represents: a specific direction feature F of the coding unit sample DirIn this case, various partitioning modes are selected as the probability distribution of the target partitioning mode. In one example, for example, in the probability distribution set Map1 (F Dir ,PV M ), a certain directional feature F Dir (f subDir0 ,f subDir1 ,f subDir2 ,f subDir3 ,f curDir ) = "1,1,1,1,1", the probability distribution of the target division pattern corresponding to this directional feature is PV M ={0%,1%,3%,1%,95%}, then the probability distribution PV of the target partitioning pattern M It can be expressed as: based on the above-mentioned directional characteristics of the coding unit sample, the probability that the division mode with a value of 0 is the target division mode is 0%, the probability that the division mode with a value of 1 is the target division mode is 1%, the probability that the division mode with a value of 2 is the target division mode is 3%, the probability that the division mode with a value of 3 is the target division mode is 1%, and the probability that the division mode with a value of 4 is the target division mode is 95%.

[0102] In an example, when using four different QPs (22, 27, 32, 37) to encode 10 frames of video frame samples for six different types of video sample sequences, the probability distribution set Map1 (F Dir ,PV M ) has a total of 4x4x4x4x4 directional features.

[0103] In step S217, according to the probability distribution of the target division modes corresponding to the coding unit samples under different features, the feature corresponding to the preset probability threshold when the probability of the target division mode being a non-subsequent division mode is determined is greater than the feature, and the target feature is formed based on the determined feature.

[0104] After obtaining the probability distribution of target partitioning patterns corresponding to coding unit samples under different characteristics, to determine the target features corresponding to the subsequent partitioning patterns for which the coding unit can skip, embodiments of the present application can determine the target features based on the above probability distribution. In a further optional implementation, the target features can be recorded in a target feature set, which can be a collection of target features that record the subsequent partitioning patterns for which the coding unit can skip.

[0105] In some embodiments, the embodiment of the present application may determine, based on the above-mentioned probability distribution, that the probability that the target division mode of the coding unit sample is a non-subsequent division mode is greater than the feature corresponding to the preset probability threshold, thereby obtaining the feature corresponding to the coding unit sample when skipping the subsequent division mode from the above-mentioned probability distribution (that is, the feature corresponding to the coding unit being able to skip the subsequent division mode), and then forming the target feature based on the determined feature. Optionally, the embodiment of the present application may aggregate the determined features to obtain a target feature set. It should be noted that the subsequent division mode referred to in the embodiment of the present application may be a division mode after the coding unit is divided into sub-blocks (for example, a division mode after the coding unit performs quad division), including horizontal division and vertical division.

[0106] In some embodiments, based on the probability distribution of the target partitioning modes corresponding to the coding unit samples under different features, the embodiments of the present application can confirm the probability of selecting various partitioning modes as the target partitioning modes for the coding unit samples under different features; thus, the embodiments of the present application can traverse all features of the coding unit samples, and then for the currently traversed feature, determine the probability that the target partitioning mode corresponding to the feature is a non-subsequent partitioning mode. If the probability is greater than a preset probability threshold, the currently traversed feature can be used as the target feature; by traversing all features in the coding unit samples in this way, the embodiments of the present application can obtain the features corresponding to the probability that the target partitioning mode is a non-subsequent partitioning mode greater than the preset probability threshold, thereby forming the target features that can skip the subsequent partitioning mode in the embodiments of the present application. Further, the target features can be aggregated to form a target feature set.

[0107] As an optional implementation, the characteristics based on the coding unit may include division characteristics and direction characteristics, and the target characteristics determined in the embodiment of the present application may include: the target division characteristics and target direction characteristics corresponding to the coding unit skipping the subsequent division mode.

[0108] In a further optional implementation, the subsequent division mode based on the coding unit may include horizontal division and vertical division, and the target division feature determined by the embodiment of the present application may include a first target division feature of the coding unit skipping the horizontal division mode, and a second target division feature of the coding unit skipping the vertical division mode; the target direction feature determined by the embodiment of the present application may include a first target direction feature of the coding unit skipping the horizontal division mode, and a second target direction feature of the coding unit skipping the vertical division mode.

[0109] Furthermore, a target feature set may be formed based on the target features. The target feature set determined in the embodiment of the present application may include:

[0110] The target split feature set SkipHorSplit_SplitFeaturesSet of the coding unit skipping horizontal splitting may be referred to as the first target split feature set for ease of explanation; the first target split feature set may record the first target split feature of the coding unit skipping horizontal splitting mode;

[0111] The target direction feature set SkipHorSplit_DirFeaturesSet of the coding unit skipping horizontal splitting is referred to as the first target direction feature set for ease of explanation. The first target direction feature set records the first target direction feature of the coding unit skipping horizontal splitting mode.

[0112] The target partition feature set SkipVerSplit_SplitFeaturesSet of the coding unit skipping vertical partitioning is referred to as the second target partition feature set for ease of explanation; the second target partition feature set can record the second target partition feature of the coding unit skipping vertical partitioning mode;

[0113] The target direction feature set SkipVerSplit_DirFeaturesSet of the coding unit skipping vertical splitting is called the second target direction feature set for the convenience of explanation; the second target direction feature set records the second target direction feature of the coding unit skipping vertical splitting mode.

[0114] The following will introduce the determination process of the first target segmentation feature, the first target direction feature, the second target segmentation feature, and the second target direction feature respectively.

[0115] As an optional implementation, Figure 4 The flowchart of the optional method for determining the first target segmentation feature provided by the embodiment of the present application is shown as an example. Figure 4 As shown, the method flow may include the following steps.

[0116] In step S410 , all the partition features in the probability distribution set of the partition features are traversed.

[0117] The probability distribution set Map0(F split ,PV M ) records the probability distribution of the target partitioning mode corresponding to the coding unit samples under different partitioning features. The embodiment of the present application can map Map0(F split ,PV M ) to traverse all the partition features in .

[0118] In step S411 , the probability that the target partition mode corresponding to the traversed partition feature is a non-horizontal partition mode is determined.

[0119] When traversing Map0(F split ,PV M ) in the process of traversing all the partition features, for the partition features traversed, the embodiment of the present application can determine the partition feature in Map0(F split ,PV M ) is the probability that the target partition mode M is a non-horizontal partition mode, that is, P(M is not equal to horizontal partition).

[0120] Optional,

[0121] Here, i≠3 indicates that the target partition mode M is not a horizontal partition with a value of 3. The target partition mode M has a value ranging from 0 to 4. The meaning of each value can be referred to the description of the corresponding part above.

[0122] In step S412, it is determined whether the determined probability is greater than a preset probability threshold. If so, step S413 is executed; if not, step S414 is executed.

[0123] The embodiment of the present application may set a preset probability threshold, and the specific value of the preset probability threshold may be determined according to the actual situation, such as 90%. After determining the probability that the target partitioning mode corresponding to the currently traversed partitioning feature is non-horizontal partitioning, the embodiment of the present application may compare the determined probability with the preset probability threshold; if the determined probability is greater than the preset probability threshold, the traversed partitioning feature is deemed to be a partitioning feature that can skip the horizontal partitioning mode, and the embodiment of the present application may execute step S413; if the determined probability is not greater than the preset probability threshold, the currently traversed partitioning feature is deemed not to be a partitioning feature that can skip the horizontal partitioning mode, and the embodiment of the present application may continue to traverse Map0(F split ,PV M ) until all partition features are traversed.

[0124] In step S413, the traversed segmentation feature is determined as the first target segmentation feature.

[0125] When the probability that the target partitioning mode corresponding to the traversed partitioning feature is a non-horizontal partitioning mode is greater than a preset probability threshold, the embodiment of the present application may determine the traversed partitioning feature as the first target partitioning feature. Further, if the first target partitioning feature is recorded in the first target partitioning feature set, the embodiment of the present application may save the traversed partitioning feature to the first target partitioning feature set, so that the first target partitioning feature that enables the coding unit to skip the subsequent horizontal partitioning mode is saved in the first target partitioning feature set.

[0126] In step S414, it is determined whether all the partition features in the probability distribution set of the partition features have been traversed. If so, step S415 is executed; if not, step S416 is executed.

[0127] In step S415, the process ends.

[0128] In step S416, the next traversed segmentation feature is determined, and the process returns to step S411.

[0129] When the probability that the target partitioning mode corresponding to the traversed partitioning feature is a non-horizontal partitioning mode is not greater than the preset probability threshold, the embodiment of the present application may be in Map0(F split ,PV M ) is not traversed, the next traversed ... split ,PV M ) is not traversed, the next traversed partition feature is determined and the process returns to step S411.

[0130] As can be seen, the embodiments of the present application can determine the division feature corresponding to the probability distribution of the target division mode corresponding to the coding unit samples under different division features, when the probability of the target division mode being non-horizontal division is greater than the preset probability threshold, and form a first target division feature based on the determined division feature. Furthermore, the first target division feature can be recorded in the first target division feature set, so that the embodiments of the present application can form the first target division feature set.

[0131] As an optional implementation, the method for determining the first target direction feature can be the same as Figure 4 The method flow shown is implemented in the same way, except that: when determining the first target direction feature, the embodiment of the present application is based on the probability distribution set Map1 (F Dir ,PV M ) traverses the direction feature in order to determine the probability that the target division mode corresponding to the traversed direction feature is a non-horizontal division mode; compares the determined probability with a preset probability threshold; if the determined probability is greater than the preset probability threshold, the traversed direction feature can be used as the first target direction (for example, the traversed direction feature is saved in the first target direction feature set); if the determined probability is not greater than the preset probability threshold, the next traversed direction feature is determined, and so on until Map1(F Dir ,PV M) have been traversed. That is, the embodiment of the present application can determine the directional feature corresponding to the probability of the target division mode being non-horizontal division greater than the preset probability threshold based on the probability distribution of the target division mode corresponding to the coding unit samples under different directional features, and form a first target directional feature based on the determined directional feature. Further, the first target directional feature can be recorded in the first target directional feature set, so that the embodiment of the present application can form a first target directional feature set.

[0132] As an optional implementation, the method for determining the second target segmentation feature can be the same as Figure 4 The method flow shown is implemented in the same way, except that: when determining the second target segmentation feature, the embodiment of the present application is a probability distribution set Map0 (F split ,PV M ) is traversed, and the probability that the target partitioning mode corresponding to the partitioning feature is a non-vertical partitioning mode is determined, that is, P(M is not equal to vertical partitioning); thereby comparing the determined probability with a preset probability threshold; if the determined probability is greater than the preset probability threshold, the traversed partitioning feature can be used as a second target partitioning feature (for example, the traversed partitioning feature is saved in a second target partitioning feature set) so as to serve as a partitioning feature for the coding unit to skip subsequent vertical partitioning modes; if the determined probability is not greater than the preset probability threshold, then the next traversed partitioning feature is determined, and so on until Map0(F split ,PV M ) have been traversed.

[0133] That is, embodiments of the present application can determine, based on the probability distribution of target partition modes corresponding to coding unit samples under different partition features, the partition feature corresponding to the probability that the target partition mode is non-vertical partitioning is greater than a preset probability threshold, and form a second target partition feature based on the determined partition feature. Furthermore, the second target partition feature can be recorded in a second target partition feature set, thereby forming a second target partition feature set in embodiments of the present application.

[0134] Optional, Here, i≠4 indicates that the target partition mode M is not a vertical partition with a value of 4.

[0135] As an optional implementation, the method for determining the second target direction feature can be the same as Figure 4 The method flow shown is implemented in the same way, except that: when determining the second target direction feature, the embodiment of the present application is based on the probability distribution set Map1 (F Dir ,PV M) traverses the directional features in the traversed directional features, thereby determining the probability that the target division mode corresponding to the traversed directional features is a non-vertical division mode; comparing the determined probability with a preset probability threshold; if the determined probability is greater than the preset probability threshold, the traversed directional features can be used as the second target directional features (for example, the traversed directional features are saved in the second target directional feature set), so as to serve as directional features that enable the coding unit to skip subsequent vertical division modes; if the determined probability is not greater than the preset probability threshold, determining the next traversed directional features, until Map1(F Dir ,PV M ) have been traversed. That is, the embodiment of the present application can determine the directional feature corresponding to the probability of the target division mode being non-vertical division greater than the preset probability threshold based on the probability distribution of the target division mode corresponding to the coding unit samples under different directional features, and form a second target directional feature based on the determined directional feature. Further, the second target directional feature can be recorded in the second target directional feature set, so that the embodiment of the present application can form a second target directional feature set.

[0136] It should be noted that the target features determined in the embodiment of the present application include the first target division feature, the first target direction feature, the second target division feature and the second target direction feature, which is only an optional implementation method. The embodiment of the present application may also select at least one of the first target division feature, the first target direction feature, the second target division feature and the second target direction feature for determination; for example, when determining the first target division feature, the embodiment of the present application may decide whether the coding unit can skip the subsequent horizontal division mode based on the division feature of the coding unit; when determining the first target direction feature, the embodiment of the present application may decide whether the coding unit can skip the subsequent horizontal division mode based on the direction feature of the coding unit; when determining the second target division feature, the embodiment of the present application may decide whether the coding unit can skip the subsequent vertical division mode based on the division feature of the coding unit; when determining the second target direction feature, the embodiment of the present application may decide whether the coding unit can skip the subsequent vertical division mode based on the direction feature of the coding unit.

[0137] Based on the target features introduced above, in the process of video encoding, an embodiment of the present application can determine the features of the coding unit after dividing the coding unit of the video frame into sub-blocks, thereby matching the features of the coding unit with the target features; and then, based on the matching results, decide whether the coding unit can skip the subsequent division mode, so as to avoid the need to try multiple division modes for the coding unit when the coding unit can skip the subsequent division mode, thereby reducing the encoding complexity of the coding unit and improving the encoding efficiency.

[0138] It should be explained that the inventors of the present application took into account that the target division mode and target intra-frame direction mode selected by the encoder for the coding unit are both the most consistent with the texture characteristics of the coding unit, and the texture characteristics of the coding unit are also related to the texture characteristics of the sub-blocks obtained by dividing the coding unit. Therefore, there is a relationship between the target division mode of the coding unit and the encoding information of the sub-blocks divided by the coding unit. When encoding a coding unit, the encoder will first divide the coding unit to obtain the sub-blocks of the coding unit, and then try to perform subsequent division modes such as horizontal division and vertical division on the coding unit; and the sub-blocks divided by the coding unit are respectively recursively used to obtain encoding information for further encoding of the sub-blocks. Therefore, before the subsequent division mode of the coding unit is subjected to rate-distortion optimization processing, the embodiment of the present application has already obtained the encoding information of the sub-blocks of the coding unit. Therefore, the encoding information of the sub-blocks of the coding unit can be used to form the characteristics of the coding unit, thereby helping the encoder decide whether the coding unit can skip the subsequent division mode.

[0139] As an optional implementation, Figure 5 An optional flow chart of the division decision method of the coding unit provided in the embodiment of the present application is exemplarily shown. Optionally, the method flow can be implemented by an encoder, for example, an encoder provided in a terminal device or a server; Figure 5 The method flow may include the following steps.

[0140] In step S510 , the coding unit is divided to obtain a plurality of sub-blocks.

[0141] During the encoding process of a video frame, after the video frame is divided into a series of coding tree units, the coding tree units can be further divided into coding units for actual encoding. For any coding unit to be encoded, the embodiment of the present application can divide the coding unit to obtain multiple sub-blocks of the coding unit.

[0142] As an optional implementation, when the maximum allowable division level of the multi-type tree structure is 1, the embodiment of the present application can divide the coding unit through a quadripartition method to obtain multiple sub-blocks (for example, one coding unit is divided into 4 sub-blocks through quadripartition). In other possible implementations, when the maximum allowable division level of the multi-type tree structure is greater than 1, the embodiment of the present application can divide the coding unit through a quadripartition method or a multi-type tree method to obtain multiple sub-blocks.

[0143] In step S511 , coding information for encoding each sub-block is determined.

[0144] After obtaining multiple sub-blocks of the coding unit, each sub-block can be further encoded, so that for each sub-block, the embodiment of the present application can determine the encoding information for encoding the sub-block. In some embodiments, the encoding information of the sub-block may include a target partition mode and a target intra-frame direction mode of the sub-block. As an optional implementation, for each sub-block, the embodiment of the present application can try multiple partition modes for the sub-block, thereby selecting the partition mode with the minimum rate-distortion optimization as the target partition mode of the sub-block; at the same time, multiple intra-frame direction modes can be tried for the sub-block, and the intra-frame direction mode with the minimum rate-distortion optimization can be selected as the target intra-frame direction mode of the sub-block.

[0145] In step S512, the characteristics of the coding unit are determined according to the coding information of each sub-block.

[0146] In an embodiment of the present application, the characteristics of a coding unit may be used to determine whether the coding unit skips a subsequent splitting mode, which may be a splitting mode after the coding unit is split into sub-blocks. As an optional implementation, the subsequent splitting mode may include a horizontal splitting mode and a vertical splitting mode.

[0147] In some embodiments, the characteristics of a coding unit may include a division characteristic and a directional characteristic of the coding unit. As an optional implementation, embodiments of the present application may obtain the division characteristics of the coding unit based on the target division pattern of each sub-block of the coding unit. For example, the target division pattern of each sub-block of the coding unit is mapped to a string to obtain the division characteristics of the coding unit.

[0148] As an optional implementation, the embodiment of the present application can try multiple intra-frame direction modes for the coding unit and select the intra-frame direction mode with the lowest rate-distortion optimization as the target intra-frame direction mode for the coding unit. Therefore, the embodiment of the present application can obtain the directional characteristics of the coding unit based on the target intra-frame direction modes of each sub-block of the coding unit and the target intra-frame direction mode of the coding unit. For example, the target intra-frame direction modes of each sub-block of the coding unit and the target intra-frame direction mode of the coding unit are mapped to a string to obtain the directional characteristics of the coding unit.

[0149] Optionally, the features of the encoding unit may be in the form of a feature vector.

[0150] In step S513, before performing rate-distortion optimization processing of the subsequent division mode on the coding unit, it is determined whether the characteristics of the coding unit match the target characteristics. If so, step S514 is executed; if not, step S515 is executed.

[0151] Different from the method of trying multiple division modes for the coding unit and then selecting the division mode with the minimum rate-distortion optimization as the subsequent division mode used by the coding unit, the embodiment of the present application can decide whether the coding unit can skip the subsequent division mode by matching the characteristics of the coding unit with the target characteristics before performing the rate-distortion optimization of the above-mentioned subsequent division mode on the coding unit.

[0152] In some embodiments, if the target feature is recorded in a target feature set, the embodiments of the present application may determine whether the feature of the coding unit is recorded in the target feature set before performing rate-distortion optimization processing on the coding unit in a subsequent division mode, thereby determining whether the feature of the coding unit matches the target feature. The target feature set records the feature corresponding to the coding unit skipping the subsequent division mode.

[0153] In other embodiments, if the encoder has sufficient computing power, the embodiments of the present application can also calculate the target features corresponding to the coding unit skipping the subsequent division mode in real time online (the method for determining the target features can refer to the description of the corresponding part of the previous text), so as to determine whether the features of the coding unit match the target features calculated in real time online before performing rate-distortion optimization processing on the coding unit for the subsequent division mode.

[0154] That is to say, using the target feature set to pre-record the target features and determining whether the features of the coding unit are recorded in the target feature set is only an optional implementation of step S513. The embodiment of the present application may also not pre-set or record the target features, for example, by using real-time online calculation of the target features to implement step S513.

[0155] In step S514, the decision coding unit skips the subsequent partitioning mode.

[0156] If the characteristics of the coding unit match the target characteristics (for example, the characteristics of the coding unit are recorded in the target feature set), the embodiment of the present application can decide that the coding unit can skip the subsequent division mode, thereby avoiding the need to try multiple division modes for the coding unit, reducing the coding complexity of the coding unit, and thus improving the coding efficiency.

[0157] In step S515, rate-distortion optimization processing of subsequent division modes is performed on the coding unit, so as to select a division mode with the minimum rate-distortion optimization from multiple division modes as the subsequent division mode used by the coding unit.

[0158] The features of the coding unit do not match the target features (for example, the features of the coding unit are not recorded in the target feature set). In an embodiment of the present application, the coding unit may be subjected to rate-distortion optimization processing for subsequent division modes, thereby selecting the division mode with the smallest rate-distortion optimization as the subsequent division mode used by the coding unit, thereby performing subsequent division on the coding unit.

[0159] The coding unit division decision method provided in the embodiment of the present application can divide the coding unit to obtain multiple sub-blocks when encoding the coding unit, and determine the coding information used to encode each sub-block; thereby, the embodiment of the present application can determine the characteristics of the coding unit based on the coding information of each sub-block; and then before performing rate-distortion optimization processing of the subsequent division mode on the coding unit, the embodiment of the present application can determine whether the characteristics of the coding unit match the target characteristics, and the target characteristics are the characteristics corresponding to the coding unit skipping the subsequent division mode; if the characteristics of the coding unit match the target characteristics, the embodiment of the present application can decide that the coding unit skips the subsequent division mode, thereby avoiding the need to try multiple division modes for the coding unit in the future, reducing the coding complexity of the coding unit, and thus improving the coding efficiency.

[0160] It should be further introduced that, after obtaining the sub-blocks of the coding unit by horizontal binary division, the inventor of the present application has considered using a Bayesian classifier to predict whether the coding unit needs to be further vertically divided (such as vertical binary division, vertical trisection division) in the future, so as to skip the subsequent vertical division of the coding unit when the coding unit does not need to be vertically divided in the future, so as to reduce the coding complexity of the coding unit.

[0161] The above Bayesian classifier actually uses a binary classification problem to decide whether the coding unit can skip vertical division. Its implementation process can be shown as follows:

[0162] After performing horizontal binary partitioning on the coding unit, a target intra-frame direction mode and a target partitioning mode for further encoding of the sub-block are determined, and the target intra-frame direction mode and the target partitioning mode of the sub-block are combined into a feature vector F, which is used as a training data for the Bayesian classifier; whether the target partitioning mode of the coding unit is vertical partitioning is used as the corresponding training label. For example, the training label can be defined as T0 and T1, where T0 indicates that the coding unit does not select vertical partitioning after trying multiple partitioning modes, and T1 indicates that the coding unit selects vertical partitioning after trying multiple partitioning modes;

[0163] Using the above training data and training labels, the histogram estimation method is used to estimate the conditional probability P(F|T1) of the feature vector when the target division mode of the coding unit is vertical division, the conditional probability P(F|T0) of the feature vector when the target division mode of the coding unit is non-vertical division, the prior probability P(T1) when the target division mode of the coding unit is vertical division, and the prior probability P(T0) when the target division mode of the coding unit is non-vertical division;

[0164] When using the Bayesian classifier to predict whether a coding unit can subsequently skip vertical splitting, the current feature vector F of the coding unit, as well as the conditional probabilities P(F|T1) and P(F|T0) and prior probabilities P(T1) and P(T0) obtained by the histogram estimation method can be determined. Then, the binary classification problem expressed by the following formula is used to decide whether the coding unit can subsequently skip vertical splitting:

[0165]

[0166] Among them, τ is the classification cost factor, which can be an adjustable parameter.

[0167] However, although the above method can use the Bayesian classifier to predict whether the coding unit can skip the vertical division mode in the future, thereby reducing the coding complexity of the coding unit to a certain extent, it has certain limitations: the feature vector F of the coding unit depends on the coding information of the sub-block after the coding unit completes the horizontal division (that is, the target intra-frame direction mode and target division mode of the sub-block further encoded), and the horizontal division belongs to the division of the multi-type tree. If the multi-type tree is a multi-type tree structure that only allows one layer of division at most, the coding unit cannot be divided based on the multi-type tree after obtaining the sub-block through horizontal division, which results in the loss of the coding information of the sub-block. At this time, the above conditional probability will no longer apply to the multi-type tree structure.

[0168] That is to say, when the maximum allowable division level of the multi-type tree is 1 (i.e., Max MTT Depth = 1), after the coding unit obtains the sub-block through horizontal division, it has completed the maximum allowable division level of the multi-type tree, and the sub-block cannot be further encoded, so the coding information for further encoding of the sub-block cannot be obtained. Therefore, the above-mentioned method of using the Bayesian classifier cannot be applied to the multi-type tree structure that only allows a maximum of one layer of division, and there are limitations in use.

[0169] Therefore, when the maximum allowed division level of the multi-type tree is 1, the embodiment of the present application can determine the characteristics of the coding unit based on the coding information of the sub-block after the coding unit is quad-divided; at this time, the multi-type tree structure that only allows a maximum of one division does not limit the number of divisions of the coding unit in the quadtree. Therefore, after the coding unit is quad-divided, the embodiment of the present application can also determine the coding information for further encoding the sub-block based on the multi-type tree structure that only allows a maximum of one division, thereby providing feature support for deciding whether the coding unit can skip subsequent division modes. Therefore, the solution provided by the embodiment of the present application can be applicable to the multi-type tree structure that only allows a maximum of one division, and can improve the applicability of the solution provided by the embodiment of the present application.

[0170] It should be noted that the coding unit division decision method provided in the embodiment of the present application is achieved by matching the characteristics of the coding unit with the target characteristics when deciding whether the coding unit can skip the subsequent division mode. This is different from the binary classification problem of the Bayesian classifier introduced above.

[0171] In the embodiment of the present application, when the coding unit is quad-divided to obtain sub-blocks, the characteristics of the coding unit can be determined based on the coding information further encoded by the sub-blocks. Therefore, after the coding unit is quad-divided, the embodiment of the present application can also determine the coding information of the sub-block based on a multi-type tree structure that allows only one layer of division at most, thereby providing feature support for deciding whether the coding unit can skip subsequent division modes. Therefore, the solution provided by the embodiment of the present application can be applicable to a multi-type tree structure that allows only one layer of division at most, which can improve the applicability of the solution provided by the embodiment of the present application.

[0172] It can be seen that the coding unit division decision method provided in the embodiment of the present application can reduce the coding complexity of the coding unit, improve the coding efficiency, and has high applicability.

[0173] In some embodiments, the characteristics based on the coding unit may include the division characteristics and directional characteristics of the coding unit, and the target characteristics corresponding to the subsequent division mode skipped by the coding unit may include: the target division characteristics and target directional characteristics corresponding to the subsequent division mode skipped by the coding unit. Therefore, before performing rate-distortion optimization processing on the subsequent division mode of the coding unit, the embodiment of the present application may determine whether the division characteristics of the coding unit match the target division characteristics, and / or whether the directional characteristics of the coding unit match the target directional characteristics; if the division characteristics of the coding unit match the target division characteristics, or the directional characteristics of the coding unit match the target directional characteristics, the embodiment of the present application may decide that the coding unit skips the subsequent division mode; if the division characteristics of the coding unit do not match the target division characteristics, and the directional characteristics of the coding unit do not match the target directional characteristics, the embodiment of the present application may perform rate-distortion optimization processing on the subsequent division mode of the coding unit to determine the subsequent division mode used by the coding unit.

[0174] In some further embodiments, the subsequent division mode based on the coding unit may include a horizontal division mode (such as a horizontal binary and horizontal trifurcated mode), and a vertical division mode (such as a vertical binary and vertical trifurcated mode), and the target division feature corresponding to the coding unit skipping the subsequent division mode may include: the first target division feature of the coding unit skipping the horizontal division mode, and the second target division feature of the coding unit skipping the vertical division mode; the target direction feature corresponding to the coding unit skipping the subsequent division mode may include: the first target direction feature of the coding unit skipping the horizontal division mode, and the second target direction feature of the coding unit skipping the vertical division mode.

[0175] In a further optional implementation, based on the target features being recorded in the target feature set, the target feature set may be further subdivided into:

[0176] A first target partition feature set, recording a first target partition feature of a coding unit skipping horizontal partitioning mode;

[0177] A first target direction feature set, recording first target direction features of a coding unit skipping horizontal division mode;

[0178] A second target partition feature set, recording a second target partition feature of a coding unit skipping vertical partition mode;

[0179] The second target direction feature set records the second target direction feature of the coding unit skipping vertical division mode.

[0180] As an optional implementation, an embodiment of the present application may decide whether the coding unit can skip the subsequent horizontal division mode based on the first target division feature (for example, the division feature recorded in the first target division feature set) and the first target directional feature (for example, the directional feature recorded in the first target directional feature set); and, based on the second target division feature (for example, the division feature recorded in the second target division feature set) and the second target directional feature (for example, the directional feature recorded in the second target directional feature set), decide whether the coding unit can skip the subsequent vertical division mode.

[0181] It should be noted that the texture features of the coding unit are also related to the texture features of the sub-blocks obtained by dividing the coding unit, so the target division mode of the coding unit is related to the target division mode and target intra-frame direction mode of the sub-blocks into which the coding unit is divided. When encoding a coding unit, the encoder first divides the coding unit into sub-blocks, and then attempts to divide the coding unit horizontally and vertically; and each sub-block divided by the coding unit adopts a recursive method of trying multiple division modes and intra-frame direction modes to obtain the target division mode and target intra-frame direction mode of the sub-block. Therefore, before performing the rate-distortion optimization processing of the horizontal division and vertical division of the coding unit, the embodiment of the present application has obtained the target division mode, target intra-frame direction mode and target intra-frame direction mode of the sub-block of the coding unit. Therefore, the target division mode of the sub-block can form the division feature of the coding unit, and the target intra-frame direction mode of the sub-block and the target intra-frame direction mode of the coding unit can form the directional feature of the coding unit, which are respectively used to help the encoder decide whether the coding unit can skip subsequent horizontal or vertical division.

[0182] Optional, Figure 6 Another optional flow chart of the coding unit division decision method provided in the embodiment of the present application is shown as an example. Figure 6 The method flow may include the following steps.

[0183] In step S610 , the coding unit is divided to obtain a plurality of sub-blocks.

[0184] In step S611 , a target partition mode and a target intra direction mode for each subblock, and a target intra direction mode for the coding unit are determined.

[0185] In step S612, the target partition mode of each subblock is mapped to the partition feature of the coding unit; and the target intra-frame direction mode of each subblock and the target intra-frame direction mode of the coding unit are mapped to the direction feature of the coding unit.

[0186] When a video encoder encodes a coding unit, if the coding unit has completed the rate-distortion optimization process (i.e., the coding unit is divided into multiple sub-blocks, and based on the rate-distortion optimization process, the target division mode and target intra-frame direction mode of each sub-block are determined), then according to the target division mode and target intra-frame direction mode of each sub-block of the coding unit, the embodiment of the present application can obtain the division characteristics and direction characteristics of the coding unit.

[0187] In step S613, before performing rate-distortion optimization processing in the horizontal split mode on the coding unit, it is determined whether the split feature of the coding unit matches the first target split feature. If so, step S614 is executed; otherwise, step S615 is executed.

[0188] As an optional implementation, the embodiment of the present application may determine whether the division feature of the coding unit is recorded in the first target division feature set.

[0189] In step S614, the coding unit decides to skip the horizontal split mode.

[0190] In step S615 , it is determined whether the directional feature of the encoding unit matches the first target directional feature. If so, step S614 is executed; if not, step S616 is executed.

[0191] As an optional implementation, the embodiment of the present application may determine whether the directional feature of the encoding unit is recorded in the first target directional feature set.

[0192] In step S616, the rate-distortion optimization processing of the horizontal division mode is performed on the coding unit, and the selection of the final division mode of the coding unit is participated in.

[0193] As an optional implementation, before performing rate-distortion optimization processing on the coding unit in the horizontal split mode (horizontal binary and horizontal trifurcated modes), the embodiment of the present application may search for the split features of the coding unit in the first target split feature set (i.e., the target split feature set SkipHorSplit_SplitFeaturesSe in which the coding unit skips horizontal split); if the split features of the coding unit exist in the first target split feature set, it may be decided that the coding unit skips the horizontal split mode; if the split features of the coding unit do not exist in the first target split feature set, the first target directional feature set (i.e., the target directional feature set SkipHorSplit_DirFeaturesSet in which the coding unit skips horizontal split) may be searched for the directional features of the coding unit; if the directional features of the coding unit exist in the first target directional feature set, it may be decided that the coding unit skips the horizontal split mode; if the directional features of the coding unit still do not exist in the first target directional feature set, the rate-distortion optimization processing of the horizontal split mode may be performed on the coding unit.

[0194] That is to say, if the division feature of the coding unit is recorded in the first target division feature set, or the directional feature of the coding unit is recorded in the first target directional feature set, the coding unit is decided to skip the horizontal division mode; otherwise, when the division feature of the coding unit is not recorded in the first target division feature set, and the directional feature of the coding unit is not recorded in the first target directional feature set, it is determined to perform rate-distortion optimization processing of the horizontal division mode on the coding unit.

[0195] In step S617, before performing rate-distortion optimization processing in the vertical partition mode on the coding unit, it is determined whether the partition feature of the coding unit matches the second target partition feature. If so, step S618 is executed; otherwise, step S619 is executed.

[0196] As an optional implementation, the embodiment of the present application may determine whether the division feature of the coding unit is recorded in the second target division feature set.

[0197] In step S618, the decision coding unit skips the vertical split mode.

[0198] In step S619 , it is determined whether the directional feature of the encoding unit matches the second target directional feature. If so, step S618 is executed; if not, step S620 is executed.

[0199] As an optional implementation, the embodiment of the present application may determine whether the directional feature of the encoding unit is recorded in the second target directional feature set.

[0200] In step S620, rate-distortion optimization processing of the vertical division mode is performed on the coding unit, and the final division mode of the coding unit is selected.

[0201] It can be seen that before the embodiment of the present application performs rate-distortion optimization processing on the vertical split mode (vertical binary and vertical trifurcated modes) of the coding unit, it can search for whether the split feature of the coding unit exists in the second target split feature set (i.e., the target split feature set SkipVerSplit_SplitFeaturesSet for the coding unit to skip vertical split); if the split feature of the coding unit exists in the second target split feature set, it can be decided that the coding unit skips the vertical split mode; if the split feature of the coding unit does not exist in the second target split feature set, it can search for whether the directional feature of the coding unit exists in the second target directional feature set (i.e., the target directional feature set SkipVerSplit_DirFeaturesSet for the coding unit to skip vertical split); if the directional feature of the coding unit exists in the second target directional feature set, it can be decided that the coding unit skips the vertical split mode; if the directional feature of the coding unit exists in the second target directional feature set, it can be decided that the coding unit skips the vertical split mode; if the directional feature of the coding unit still does not exist in the second target directional feature set, the rate-distortion optimization processing of the vertical split mode can be performed on the coding unit.

[0202] That is to say, if the division feature of the coding unit is recorded in the second target division feature set, or the directional feature of the coding unit is recorded in the second target directional feature set, the coding unit is decided to skip the vertical division mode; otherwise, when the division feature of the coding unit is not recorded in the second target division feature set, and the directional feature of the coding unit is not recorded in the second target directional feature set, it is determined to perform rate-distortion optimization processing of the vertical division mode on the coding unit.

[0203] It should be noted that the simultaneous use of the first target division feature, the first target direction feature, the second target division feature and the second target direction feature to decide whether the coding unit can skip the subsequent division mode is only an optional implementation example. The embodiment of the present application may also support the use of at least one of the first target division feature, the first target direction feature, the second target division feature and the second target direction feature to decide whether the coding unit can skip horizontal division or vertical division.

[0204] To verify the effectiveness of the solution provided by the embodiment of the present application in video coding, the embodiment of the present application can test the coding unit division decision method provided by the embodiment of the present application on the VVC official reference platform, thereby quickly deciding whether the coding unit can skip horizontal or vertical division within the frame based on the target features of offline statistics. Testing the solution provided by the embodiment of the present application on the VVC official reference platform can be carried out under the general test conditions of JEVT, and the main configuration parameters can be shown in Table 2 below.

[0205]

[0206]

[0207] Table 2

[0208] In terms of the settings of the encoder, the embodiment of the present application uses the settings in the default All-Intra configuration. The video sequences used in the test are the five types of HEVC standard test sequences recommended by JVET, a total of 17 sequences. The resolutions of the test video sequences from A to E range from 1920x1080 to 416x240. The coding performance is mainly evaluated by two indicators, BDBR (Bjotegaard Delta Bit rate) and TS, and the coding performance of the algorithm is evaluated based on the original VTM13.0 encoder. Among them, BDBR represents the bit rate difference between the two encoding methods under the same objective quality, which is obtained by encoding the same video at four QP values ​​(22, 27, 32, 37) and calculating the bit rate and PSNR. BDBR can comprehensively reflect the bit rate and quality of the video. The larger its value, the higher the bit rate of the proposed fast algorithm compared to the original encoder, and the worse the compression performance of the algorithm. TS is used to measure the degree to which the fast algorithm reduces the encoding time based on the original encoder. It is calculated as follows:

[0209]

[0210] Among them, T p is the total encoding time after embedding the fast algorithm into VTM13.0, T o is the total encoding time of the original encoder.

[0211] Under the above conditions, after testing the coding unit division decision method provided by the embodiment of the present application on the VVC official reference platform, on average, the scheme provided by the embodiment of the present application increased the BDBR by 0.31% overall, indicating that the bit rate did not increase significantly, that is, the compression performance of the encoder was effectively guaranteed, and the encoding time was reduced by 15.76% compared with the original encoder, which greatly reduced the encoding complexity and improved the encoding efficiency.

[0212] The embodiment of the present application can use the encoder to complete the division of the coding unit sample into sub-block samples, and determine the characteristics of the coding unit sample based on the encoding information of each sub-block sample of the coding unit sample; then use the offline statistical method to estimate the probability distribution of the target division mode of the coding unit sample under different characteristics, and then determine the target characteristics (such as the target feature set) of the coding unit skipping the subsequent division mode such as the horizontal or vertical division mode based on the above probability distribution, and embed the target characteristics (such as the target feature set) into the encoder. When the coding unit is actually encoded, the embodiment of the present application can determine the characteristics of the coding unit based on the encoding information of each sub-block of the coding unit after dividing the coding unit into sub-blocks; thereby matching the characteristics of the coding unit with the target characteristics embedded in the encoder. If the characteristics of the coding unit match the target characteristics, it can be decided that the coding unit skips the subsequent division mode such as horizontal or vertical division; otherwise, the subsequent division of the coding unit is performed in accordance with the rate-distortion optimization processing method of the subsequent division mode such as horizontal or vertical division of the coding unit.

[0213] Furthermore, the embodiment of the present application can use quadripartitioning to divide the coding unit to obtain sub-blocks; and then use the coding information of the sub-blocks to determine the characteristics of the coding unit. Therefore, even in the case of MaxMTTDepth=1, the embodiment of the present application can continue to attempt the next step of encoding on the sub-blocks of the coding unit after the coding unit is quadripartitioned. Therefore, the coding information of the sub-blocks can be used to support the determination of the characteristics of the coding unit when MaxMTTDepth=1, so that the solution provided by the embodiment of the present application can be applicable to the case of MaxMTTDepth=1, thereby improving the applicability of the solution provided by the embodiment of the present application.

[0214] Based on the division decision method of the coding unit provided in the embodiment of the present application, the embodiment of the present application provides a corresponding decoding method; as an optional implementation, Figure 7A The following is an exemplary flowchart of an optional decoding method provided in an embodiment of the present application. The method can be implemented by a decoder, for example, a decoder provided in a terminal device or a server. Figure 7A The method flow may include the following steps.

[0215] In step S710, a code stream to be decoded is obtained.

[0216] In step S711, a subsequent division decision result of the coding unit in the video frame after being divided into sub-blocks is obtained from the code stream.

[0217] In an embodiment of the present application, the subsequent division decision result can be determined based on the matching result between the characteristics of the coding unit and the target characteristics, and the target characteristics are the characteristics corresponding to the coding unit skipping the subsequent division mode; and the characteristics of the coding unit can be determined according to the coding information of each sub-block of the coding unit, and the coding unit can be divided into multiple sub-blocks.

[0218] In step S712, if the subsequent partitioning decision result indicates that the coding unit skips the subsequent partitioning mode after partitioning the sub-blocks, it is determined that the coding unit skips the subsequent partitioning mode.

[0219] In an embodiment of the present application, a subsequent division decision result of a coding unit in a video frame after being divided into sub-blocks may be carried in a code stream. The subsequent division decision result may be determined based on the division decision method for the coding unit provided in an embodiment of the present application when encoding the video frame. For the corresponding content, please refer to the description of the corresponding part above. Thus, if the subsequent division decision result indicates that the coding unit skips the subsequent division mode (e.g., horizontal or vertical division mode) after being divided into sub-blocks, the embodiment of the present application may determine that the coding unit skips the subsequent division mode, so that when decoding the video frame, the coding unit is no longer divided horizontally or vertically after being divided into sub-blocks.

[0220] In the video transcoding scenario based on cloud services, the embodiment of the present application provides a corresponding cloud transcoding method; as an optional implementation, Figure 7B The following is an exemplary flowchart of the transcoding method provided in the embodiment of the present application, which can be applied to the cloud service platform. Figure 7B The method flow may include the following steps.

[0221] In step S721, a cloud transcoding request is obtained.

[0222] In some embodiments, individual or enterprise users can utilize the media transcoding service provided by the cloud service platform to transcode videos in scenarios such as live broadcast, video on demand, and video calls. In this case, the cloud service platform can obtain a cloud transcoding request from the individual or enterprise user, which can be used to request that a video in a first format be transcoded into a second format, thereby achieving video format transcoding.

[0223] In step S722 , the video in the first format is decoded to obtain a decoded video.

[0224] After receiving the cloud transcoding request, the cloud service platform can transcode the video in the first format into the second format; during the transcoding process, the cloud service platform can decode the video in the first format, and then encode the decoded video in the second format, thereby obtaining the transcoded video in the second format.

[0225] As an optional implementation, if the video in the first format is encoded using the division decision method of the encoding unit provided in the embodiment of the present application, then when the cloud service platform decodes the video in the first format, it can use the decoding method provided in the embodiment of the present application to decode the video in the first format to obtain a decoded video. The relevant content of the decoding method provided in the embodiment of the present application can refer to the description of the corresponding part above and will not be repeated here. In other possible implementations, the embodiment of the present application may also support decoding the video in the first format in a traditional manner.

[0226] In step S723, when encoding the decoded video into the second format, the division decision method of the coding unit is used to determine the division mode of the coding unit, and video encoding is implemented based on the determined division mode to obtain the encoded video in the second format.

[0227] Based on the coding unit division decision method provided in the embodiments of the present application, when encoding the decoded video into the second format, the cloud service platform can use the coding unit division decision method provided in the embodiments of the present application to determine the coding unit division mode, and implement video encoding based on the determined division mode, thereby obtaining the encoded video in the second format. For the relevant content of the coding unit division decision method provided in the embodiments of the present application, please refer to the description of the corresponding part above, and will not be repeated here.

[0228] In step S724 , the video in the second format is output.

[0229] After transcoding the video to obtain the second format, the cloud service platform can output the video in the second format so that individual users or corporate users can obtain the transcoded video in scenarios such as live broadcast, on-demand, and video calls.

[0230] The embodiment of the present application can utilize at least the coding unit division decision method provided by the embodiment of the present application in the cloud transcoding scenario of cloud service to reduce the coding complexity and improve the coding efficiency in the cloud transcoding scenario.

[0231] An embodiment of the present application further provides an electronic device, such as an image encoding device (such as a terminal device or server with an encoder), or a decoding device (such as a terminal device or server with a decoder). Figure 8 The block diagram of the electronic device provided by the embodiment of the present application is shown as an example. Figure 8 As shown, the electronic device may include: at least one processor 81 , at least one communication interface 82 , at least one memory 83 and at least one communication bus 84 .

[0232] In an embodiment of the present application, the number of processor 81, communication interface 82, memory 83, and communication bus 84 is at least one, and the processor 81, communication interface 82, and memory 83 communicate with each other through the communication bus 84. Optionally, the communication interface 82 can be an interface of a communication module for network communication. Optionally, the processor 81 may be a CPU, a GPU (Graphics Processing Unit), an NPU (embedded neural network processor), an FPGA (Field Programmable Gate Array), a TPU (tensor processing unit), an AI chip, an application-specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement an embodiment of the present application. The memory 83 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.

[0233] Among them, the memory 83 stores one or more computer-executable instructions, and the processor 81 calls one or more computer-executable instructions to execute the division decision method of the coding unit provided in the embodiment of the present application, or the decoding method provided in the embodiment of the present application, or the transcoding method provided in the embodiment of the present application.

[0234] An embodiment of the present application also provides a storage medium, which can store one or more computer-executable instructions. When the one or more computer-executable instructions are executed, they can implement the coding unit division decision method provided in the embodiment of the present application, or the decoding method provided in the embodiment of the present application, or the transcoding method provided in the embodiment of the present application.

[0235] An embodiment of the present application also provides a computer program, which, when executed, implements the partition decision method provided in the embodiment of the present application, or the decoding method provided in the embodiment of the present application, or the transcoding method provided in the embodiment of the present application.

[0236] The above describes a plurality of embodiment schemes provided by the embodiments of the present application. The various optional modes introduced in the various embodiment schemes can be combined with each other and cross-referenced without conflict, thereby extending a variety of possible embodiment schemes, which can all be considered as the embodiment schemes disclosed and disclosed in the embodiments of the present application. Although the embodiments of the present application are disclosed as above, the present application is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present application, and therefore the scope of protection of the present application should be based on the scope defined by the claims.

Claims

1. A method for determining the division of coding units, wherein: include: Dividing the coding unit to obtain a plurality of sub-blocks; Determining coding information for encoding each sub-block; Determining, based on the coding information of each sub-block, a feature of the coding unit, wherein the feature of the coding unit is used to decide whether the coding unit skips a subsequent division mode; the feature of the coding unit includes a division feature and a direction feature of the coding unit; Before performing rate-distortion optimization processing of a subsequent division mode on the coding unit, determining whether a division feature of the coding unit matches a target division feature corresponding to the coding unit skipping a subsequent division mode, and / or whether a directional feature of the coding unit matches a target directional feature corresponding to the coding unit skipping a subsequent division mode; If the division feature of the coding unit matches the target division feature, or the direction feature of the coding unit matches the target direction feature, it is decided that the coding unit skips the subsequent division mode.

2. The method according to claim 1, wherein The determining of the coding information for encoding each sub-block includes: Determining a target partition mode and a target intra-frame direction mode for each sub-block; The determining, according to the coding information of each sub-block, the characteristic of the coding unit includes: A division feature of the coding unit is obtained according to the target division mode of each subblock; and a direction feature of the coding unit is obtained according to the target intra-frame direction mode of each subblock and the target intra-frame direction mode of the coding unit.

3. According to the method according to claim 1, the subsequent division mode of the coding unit includes a horizontal division mode and a vertical division mode; the target division feature includes a first target division feature that the coding unit skips the horizontal division mode, and a second target division feature that the coding unit skips the vertical division mode; the target direction feature includes a first target direction feature that the coding unit skips the horizontal division mode, and a second target direction feature that the coding unit skips the vertical division mode.

4. The method according to claim 3, wherein: Before performing the rate-distortion optimization processing of the subsequent division mode on the coding unit, determining whether the division feature of the coding unit matches the target division feature, and / or whether the directional feature of the coding unit matches the target directional feature includes: Before performing the rate-distortion optimization process in the horizontal partitioning mode on the coding unit, determining whether the partitioning feature of the coding unit matches the first target partitioning feature; If the division feature of the coding unit does not match the first target division feature, determining whether the direction feature of the coding unit matches the first target direction feature; If the division feature of the coding unit matches the target division feature, or the direction feature of the coding unit matches the target direction feature, then deciding that the coding unit skips the subsequent division mode includes: If the division feature of the coding unit matches the first target division feature, or the direction feature of the coding unit matches the first target direction feature, it is decided that the coding unit skips the horizontal division mode.

5. The method according to claim 3, wherein: Before performing the rate-distortion optimization processing of the subsequent division mode on the coding unit, determining whether the division feature of the coding unit matches the target division feature, and / or whether the directional feature of the coding unit matches the target directional feature includes: Before performing rate-distortion optimization processing in a vertical partitioning mode on the coding unit, determining whether a partitioning feature of the coding unit matches the second target partitioning feature; If the division feature of the coding unit does not match the second target division feature, determining whether the direction feature of the coding unit matches the second target direction feature; If the division feature of the coding unit matches the target division feature, or the direction feature of the coding unit matches the target direction feature, then deciding that the coding unit skips the subsequent division mode includes: If the division feature of the coding unit matches the second target division feature, or the direction feature of the coding unit matches the second target direction feature, it is decided that the coding unit skips the vertical division mode.

6. The method according to any one of claims 1 to 5, wherein: The target segmentation feature and the target direction feature are recorded in a target feature set, wherein the target feature set includes a first target segmentation feature set, a first target direction feature set, a second target segmentation feature set, and a second target direction feature set; The first target partition feature set records the first target partition feature of the coding unit skipping horizontal partition mode; The first target direction feature set records the first target direction feature of the coding unit skipping the horizontal division mode; the second target division feature set records the second target division feature of the coding unit skipping the vertical division mode; The second target direction feature set records the second target direction feature of the coding unit skipping vertical division mode.

7. The method according to any one of claims 1 to 5, wherein: Also includes: determining a plurality of coding unit samples; For each coding unit sample, determining a plurality of sub-block samples obtained by dividing the coding unit sample; For each coding unit sample, determine coding information of each sub-block sample of the coding unit sample; For each coding unit sample, determining a feature of the coding unit sample according to coding information of each sub-block sample of the coding unit sample; For each coding unit sample, try multiple division modes of the coding unit sample, and select the division mode with the minimum rate distortion optimization as the target division mode of the coding unit sample; According to the target division mode and corresponding features of each coding unit sample, the probability distribution of the target division mode corresponding to each coding unit sample under different features is statistically analyzed; According to the probability distribution, a target feature is determined.

8. The method according to claim 7, wherein: Determining the target feature according to the probability distribution includes: According to the probability distribution of the target division modes corresponding to the coding unit samples under different features, it is determined that the probability when the target division mode is a non-subsequent division mode is greater than the feature corresponding to the preset probability threshold, and the target feature is formed based on the determined feature.

9. The method according to claim 8, wherein The characteristics of the coding unit samples include division characteristics and direction characteristics of the coding unit samples; The encoding information of the sub-block samples includes a target partition mode and a target intra-frame direction mode of the sub-block samples; The determining the feature of the coding unit sample according to the coding information of each sub-block sample of the coding unit sample includes: Determining a division feature of the coding unit sample according to a target division mode of each sub-block sample of the coding unit sample; And, determining a directional feature of the coding unit sample according to the target intra-frame directional mode of each sub-block sample of the coding unit sample and the target intra-frame directional mode of the coding unit sample.

10. The method according to claim 9, wherein: The determining, based on the probability distribution of the target division modes corresponding to the coding unit samples under different features, that the probability of the target division mode being a non-subsequent division mode is greater than a feature corresponding to a preset probability threshold, and forming the target feature based on the determined feature includes: Determine, based on the probability distribution of target partition modes corresponding to the coding unit samples under different partition features, a partition feature corresponding to a probability when the target partition mode is non-horizontal partitioning is greater than a preset probability threshold, and form a first target partition feature based on the determined partition feature; and, determining, based on the probability distribution of target division modes corresponding to the coding unit samples under different directional features, a directional feature corresponding to a predetermined probability threshold when the probability of the target division mode being non-horizontal division is greater than that of the target division mode, and forming a first target directional feature based on the determined directional feature; and, determining, based on probability distributions of target partition modes corresponding to coding unit samples under different partition features, a partition feature corresponding to a probability that the target partition mode is non-vertical partitioning is greater than a preset probability threshold, and forming a second target partition feature based on the determined partition feature; Furthermore, according to the probability distribution of the target division modes corresponding to the coding unit samples under different directional features, it is determined that the probability when the target division mode is non-vertical division is greater than the directional feature corresponding to the preset probability threshold, and a second target directional feature is formed based on the determined directional feature.

11. A decoding method, wherein: include: Get the code stream to be decoded; Parsing the code stream to obtain a subsequent division decision result of the coding unit in the video frame after the coding unit is divided into sub-blocks; The subsequent division decision result is determined based on a matching result between a division feature of the coding unit and a target division feature corresponding to a mode in which the coding unit skips subsequent division, and / or a matching result between a directional feature of the coding unit and a target directional feature corresponding to a mode in which the coding unit skips subsequent division; and the feature of the coding unit is determined based on coding information of each sub-block of the coding unit, wherein the coding unit is divided to obtain a plurality of sub-blocks; If the subsequent division decision result indicates that the coding unit skips the subsequent division mode after dividing the sub-blocks, it is determined that the coding unit skips the subsequent division mode; wherein, if the division feature of the coding unit matches the target division feature, or the directional feature of the coding unit matches the target directional feature, then the subsequent division decision result indicates that the coding unit skips the subsequent division mode after dividing the sub-blocks.

12. A transcoding method, wherein: Applied to a cloud service platform, the method includes: Obtain a cloud transcoding request, where the cloud transcoding request is used to request transcoding of a video in a first format into a second format; Decoding the video in the first format to obtain a decoded video; When encoding the decoded video into a second format, using the coding unit division decision method according to any one of claims 1 to 10 to determine a division mode of the coding unit, and performing video encoding based on the determined division mode to obtain an encoded video in the second format; Output the video in the second format.

13. An electronic device, wherein: The method comprises at least one memory and at least one processor, wherein the memory stores one or more computer-executable instructions, and the processor calls the one or more computer-executable instructions to implement the division decision method of the coding unit according to any one of claims 1 to 10, or the decoding method according to claim 11, or the transcoding method according to claim 12.

14. A storage medium, wherein: The storage medium stores one or more computer-executable instructions, and when the one or more computer-executable instructions are executed by the processor, the division decision method of the coding unit according to any one of claims 1 to 10 is implemented, or the decoding method according to claim 11, or the transcoding method according to claim 12.

15. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the coding unit division decision method according to any one of claims 1 to 10, or the decoding method according to claim 11, or the transcoding method according to claim 12.

Images