Cu partitioning method, electronic device, and computer-readable storage medium

Abstract

The application provides a CU division method, an electronic device and a computer readable storage medium. The CU division method comprises: dividing an original image to obtain a plurality of coding tree units (CTUs); dividing each CTU into an image block one-dimensional array comprising a plurality of image blocks in the granularity of an image block; performing visual attention mechanism calculation on the image blocks in the image block one-dimensional array to obtain a CU division depth corresponding to each image block in the image block one-dimensional array; and performing CU division on the CTU according to the CU division depth corresponding to each image block.

Description

Technical Field

[0001] This invention relates to the field of communications, and more particularly to a CU partitioning method, an electronic device, and a computer-readable storage medium. Background Technology

[0002] As people's material and spiritual living standards continue to improve, their demand for high-definition and even ultra-high-definition video is becoming increasingly urgent. How to reduce bandwidth consumption while ensuring video quality has gradually become a research focus. Compared to the previous generation of advanced video coding standards, H.264 / Advanced Video Coding (AVC), H.265 / High Efficiency Video Coding (HEVC) has significantly improved compression rates by introducing quadtree partitioning structures for coding units (CUs) and multi-angle intra-frame prediction techniques. Summary of the Invention

[0003] This invention provides a CU partitioning method, an electronic device, and a computer-readable storage medium.

[0004] In a first aspect, the present invention provides a method for partitioning a coding unit (CU), comprising:

[0005] The original image is divided into multiple coding tree units (CTUs).

[0006] Each CTU is divided into a one-dimensional array of image blocks, which includes multiple image blocks, using image blocks as the granularity.

[0007] Visual attention mechanism is calculated on the image blocks in the one-dimensional array of image blocks to obtain the CU segmentation depth corresponding to each image block in the one-dimensional array of image blocks;

[0008] The CTU is divided into CUs according to the CU division depth corresponding to each image block.

[0009] In some embodiments, the CTU is divided into multiple layers according to the depth value of the CTU, and the step of dividing the CTU into CUs according to the CUs corresponding to each image block includes:

[0010] Based on the CU division depth corresponding to each image block, the CTU is divided into CUs layer by layer in depth order.

[0011] Further, the step of dividing the CTU layer by layer according to the depth order of the CUs corresponding to each image block includes:

[0012] Starting from the layer with current depth i = 0, iterate through and count the number Ni of image blocks with CU partitioning depth greater than i corresponding to all image blocks in the CTU; where i represents the current depth, and i and Ni are natural numbers;

[0013] If Ni is greater than the preset CU partitioning threshold αi at the current depth i, then the current CU in the CTU is partitioned, and the CU partitioning at the next depth continues; where αi is greater than 0.

[0014] Otherwise, terminate the CU partitioning of the CTU.

[0015] In some embodiments, the step of performing visual attention mechanism calculations on the image blocks in the one-dimensional array of image blocks to obtain the CU segmentation depth corresponding to each image block in the one-dimensional array of image blocks includes:

[0016] Expand the dimension of the one-dimensional array of image blocks to obtain the expanded input array;

[0017] The input array is subjected to visual attention mechanism calculation to obtain a first calculation result;

[0018] The first calculation result is used to perform a fully connected layer calculation to obtain the second calculation result;

[0019] The second calculation result is used to calculate the softmax layer to obtain the CU partitioning depth corresponding to each image block in the one-dimensional array of image blocks.

[0020] In some embodiments, expanding the dimension of the one-dimensional array of image patches includes:

[0021] The one-dimensional array of image blocks is linearly projected and positional encoding information is added to obtain the expanded input array.

[0022] In some embodiments, the step of performing visual attention mechanism calculation on the image blocks in the one-dimensional array of image blocks to obtain the CU segmentation depth corresponding to each image block in the one-dimensional array of image blocks further includes:

[0023] The visual attention mechanism is trained through at least one round of backpropagation to obtain the depth prediction values ​​of each image patch.

[0024] The depth prediction values ​​of each image patch with the minimum loss are determined by the loss function and used as the CU partitioning depths corresponding to each image patch in the one-dimensional array of image patches.

[0025] Preferably, the image block is a 4*4 pixel block.

[0026] Preferably, the CTU is a 64*64 pixel block.

[0027] In a second aspect, the present invention provides an electronic device, the electronic device comprising:

[0028] One or more processors;

[0029] A memory having stored one or more programs that, when executed by one or more processors, cause the one or more processors to implement the CU partitioning method according to any one of the first aspects;

[0030] One or more I / O interfaces are connected between the processor and the memory and configured to enable information interaction between the processor and the memory.

[0031] Thirdly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the CU partitioning method according to any one of the first aspects.

[0032] The video encoding and decoding CU partitioning method proposed in this invention avoids the encoding complexity caused by traversing all CUs in traditional methods through reasonable visual attention mechanism calculation. Moreover, compared with traditional convolutional neural networks (CNNs), it further reduces the computational complexity of neural networks and improves the encoding speed. Through the learning of the attention mechanism, it can effectively guarantee the encoding quality of video, while improving the high real-time performance and reliability of the H265 / HEVC protocol. Attached Figure Description

[0033] Figure 1 This is a flowchart of a CU partitioning method provided in an embodiment of the present invention.

[0034] Figure 2 This is a flowchart of the traditional coding method.

[0035] Figure 3 This is a schematic diagram of a ViT network structure provided in an embodiment of the present invention.

[0036] Figure 4 This is a diagram showing the relationship between the size and depth of the CU partitioning provided in this embodiment of the invention.

[0037] Figure 5 This is a schematic diagram of the CTU (64×64) partitioning depth map provided in an embodiment of the present invention.

[0038] Figure 6 This is a schematic diagram of an electronic device provided in an embodiment of the present invention.

[0039] Figure 7This is a schematic diagram of a computer-readable storage medium provided in an embodiment of the present invention. Detailed Implementation

[0040] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0041] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no inherent meaning. Therefore, "module," "part," or "unit" may be used interchangeably.

[0042] With the introduction of new coding techniques in HEVC, the coding time complexity has increased several times compared to H.264 / AVC. The CU quadtree-based partitioning process requires traversing all possible CU partitioning results in the current Coding Tree Unit (CTU) (64×64 pixels), then calculating the Rate Distortion Cost (RDC) for each partitioning method, and finally selecting the CU partitioning method with the smallest RDC for encoding. While this process can reduce the bitrate, it accounts for 80% of the coding time, and the coding complexity has been forced to increase several times. Therefore, finding the optimal CU partitioning method more efficiently in the HEVC standard is crucial for accelerating the execution efficiency of H.265 / HEVC encoding.

[0043] Since the introduction of the Vision Transformer (ViT) mechanism, the patch-based attention mechanism has gained widespread attention in the field of image and video applications due to its advantages over traditional convolutional neural networks in terms of required training data, computational complexity, and actual performance.

[0044] To address the conflict between encoding bitrate and encoding complexity, the inventors proposed a ViT-based video encoding / decoding CU partitioning method based on the ViT attention mechanism.

[0045] In a first aspect, embodiments of the present invention provide a method for partitioning a coding unit (CU), such as... Figure 1 As shown, the method includes the following steps:

[0046] In step S100, the original image is divided to obtain multiple coding tree units (CTUs);

[0047] In step S200, each CTU is divided into a one-dimensional array of image blocks, which includes multiple image blocks, at the granularity of image blocks;

[0048] In step S300, visual attention mechanism calculation is performed on the image blocks in the one-dimensional array of image blocks to obtain the CU segmentation depth corresponding to each image block in the one-dimensional array of image blocks;

[0049] In step S400, the CTU is divided into CUs according to the CU division depth corresponding to each image block.

[0050] This invention provides a video codec CU block partitioning method based on a visual attention mechanism. First, the original image is divided into multiple CTUs. Then, the current CTU pixel block (e.g., 64×64 pixels) is further divided into multiple image patches according to the smallest CU partitioning unit, meaning each patch is the smallest CU partitioning unit.

[0051] Using a patch as the basic unit, each patch is encoded using the pixel values ​​it contains. All patches in a CTU are arranged into a one-dimensional array, with each patch as the unit. For example, a 64×64 pixel CTU can be divided into 256 4×4 pixel patches. These 256 patches, originally arranged in a matrix, are stretched into a linearly arranged one-dimensional array. It's important to note that "one-dimensional" refers to the patch dimension, not the pixel dimension. For pixels, the dimension is 256×16, where 256 represents the number of patch blocks and 16 represents the number of pixel values ​​in each patch block.

[0052] Next, the encoded information is linearly transformed and projected, positional encoding information is added, and then input into the VisionTransformer. After a series of calculations by the VisionTransformer's attention mechanism, the CU partitioning depth corresponding to each patch is obtained. Based on the CU partitioning depth corresponding to each patch, the current CTU is partitioned into CUs. This method is used to partition the CUs of each CTU in the original image, thus achieving CU partitioning of the entire original image.

[0053] like Figure 2 The diagram shows the overall flowchart of the H.265 / HEVC protocol encoding. The dashed box in the diagram represents the part of the invention that uses the ViT network to replace the traditional algorithm. As can be seen from the diagram, the invention uses the ViT network to replace the traditional calculation method of finding the optimal CU partitioning through loop traversal.

[0054] Compared to traditional H265 / HEVC, the video codec CU partitioning method based on ViT (Vision Transformer) avoids the encoding complexity caused by traversing all CUs in traditional methods through reasonable training and learning. Moreover, compared with traditional CNN, it further reduces the computational complexity of neural networks and improves the encoding speed. Through the learning of the attention mechanism, it can effectively guarantee the encoding quality of video, while improving the high real-time performance and reliability of the H265 / HEVC protocol.

[0055] In some embodiments, the CTU is divided into multiple layers according to the depth value of the CTU, and the step of dividing the CTU into CUs according to the CUs corresponding to each image block includes:

[0056] Based on the CU division depth corresponding to each image block, the CTU is divided into CUs layer by layer in depth order.

[0057] Further, the step of dividing the CTU layer by layer according to the depth order of the CUs corresponding to each image block includes:

[0058] Starting from the layer with current depth i = 0, iterate through and count the number Ni of image blocks with CU partitioning depth greater than i corresponding to all image blocks in the CTU; where i represents the current depth, and i and Ni are natural numbers;

[0059] If Ni is greater than the preset CU partitioning threshold αi at the current depth i, then the current CU in the CTU is partitioned, and the CU partitioning at the next depth continues; where αi is greater than 0.

[0060] Otherwise, terminate the CU partitioning of the CTU.

[0061] The CU partitioning method in this invention performs traversal statistics and CU partitioning at each depth corresponding to each patch. This replaces the traditional method of iterating through each CU partitioning method, thus avoiding the computational complexity caused by iterating through all CU partitioning methods and improving coding speed.

[0062] In this embodiment of the invention, a preset CU partitioning threshold is set for each depth as a criterion for determining whether CU partitioning is needed at the current depth.

[0063] For example, the illumination depth of the CTU ranges from 0 to 3, divided into four layers from top to bottom: 64×64 → 32×32 → 16×16 → 8×8, where each layer has a preset CU partitioning threshold αi. Based on the CU partitioning depth calculated using the aforementioned visual attention mechanism, the number of patch blocks with a CU partitioning depth exceeding the current layer is counted within each layer, serving as the predicted patch depth value for that current depth. If the predicted patch depth value for the current layer is greater than the preset CU partitioning threshold, the current CU is considered separable and is divided into four layers. After partitioning one layer, the predicted patch depth value for the next layer is counted and compared with the corresponding CU partitioning threshold, and the CU partitioning step is executed. This process is repeated recursively, layer by layer from top to bottom in depth order, to partition the CTU until the predicted patch depth value at a certain depth exceeds the CU partitioning threshold, at which point the CU partitioning of the CTU ends.

[0064] It should be noted that the preset CU partitioning threshold is not limited to comparing with the number of predicted patch depth values. More comparison rules can be added, or comparisons can be made with other metrics. For example, the proportion of predicted patch depth values ​​to the total number of patches in the current layer, or comparing the patch depth with the actual depth of the currently traversed CU.

[0065] In some embodiments, the step of performing visual attention mechanism calculations on the image blocks in the one-dimensional array of image blocks to obtain the CU segmentation depth corresponding to each image block in the one-dimensional array of image blocks includes:

[0066] Expand the dimension of the one-dimensional array of image blocks to obtain the expanded input array;

[0067] The input array is subjected to visual attention mechanism calculation to obtain a first calculation result;

[0068] The first calculation result is used to perform a fully connected layer calculation to obtain the second calculation result;

[0069] The second calculation result is used to calculate the softmax layer to obtain the CU partitioning depth corresponding to each image block in the one-dimensional array of image blocks.

[0070] In some embodiments, expanding the dimension of the one-dimensional array of image patches includes:

[0071] The one-dimensional array of image blocks is linearly projected and positional encoding information is added to obtain the expanded input array.

[0072] Figure 3This is a diagram of the video codec CU partitioning structure based on ViT in this invention. The left side shows a schematic diagram of dividing the CTU (64×64) into image patches of size 4×4. The right side shows that the divided image patches are first flattened into one-dimensional inputs, then linearly projected, and after adding positional encoding information, they are input into the Vision Transformer. After the visual attention mechanism between each patch is calculated, the output of each patch becomes the input of the subsequent fully connected layer. After passing through the softmax layer, the CU partitioning depth of each patch is obtained and mapped to the depth label of the image patch.

[0073] In some embodiments, the step of performing visual attention mechanism calculation on the image blocks in the one-dimensional array of image blocks to obtain the CU segmentation depth corresponding to each image block in the one-dimensional array of image blocks further includes:

[0074] The visual attention mechanism is trained through at least one round of backpropagation to obtain the depth prediction values ​​of each image patch.

[0075] The depth prediction values ​​of each image patch with the minimum loss are determined by the loss function and used as the CU partitioning depths corresponding to each image patch in the one-dimensional array of image patches.

[0076] In this embodiment of the invention, the CU partitioning depth corresponding to each image block in the one-dimensional array of image blocks can be used as the label corresponding to each image block. After calculating the CU partitioning depth corresponding to each image block through the visual attention mechanism, the Vision Transformer, fully connected layer, and softmax layer can be trained through backpropagation to obtain the depth prediction value of each image block. The predicted value is compared with the true label, and the depth prediction value of each image block is evaluated through the softmax layer loss function. If the loss has reached the minimum, it is considered to have reached the optimal value.

[0077] Preferably, the image block is a 4*4 pixel block divided according to the smallest unit of CU.

[0078] Preferably, the CTU is a 64*64 pixel block.

[0079] Figure 4This is a table showing the correspondence between CU partition size and CU partition depth. The CU partition size can be any one of 64×64, 32×32, 16×16, 8×8, or 4×4, and the corresponding CU partition depths are 0, 1, 2, 3, and 4, respectively.

[0080] Figure 5 This is a schematic diagram of a CU partitioning method in a CTU (64×64), where the corresponding CU blocks and partitioning depths are marked in the corresponding positions.

[0081] During the training and learning of the visual attention mechanism, we can also predict the CU division depth of each patch based on ViT, and then statistically analyze the predicted CU depth values ​​of the patch blocks to re-divide the CTUs from top to bottom.

[0082] Basic deep learning operations that can be used in embodiments of the present invention include, but are not limited to: ViT (VisionTransformer), deconvolution, Rectified Linear Function (ReLU), S-growing curve (Sigmoid), Full-Connection, Reshape, etc.

[0083] It should be noted that, in addition to using the ViT network structure, residual networks (ResNet, Residual Network), CNN, and feature compression and excitation residual networks (SE-ResNet) can also be used to replace ViT to achieve CU partitioning.

[0084] The CU partitioning method provided in this invention can be applied to all places where video encoding is required, such as video processing units (VPUs), built-in algorithms of video codec chips, smart cockpits, video compression, and video transmission.

[0085] The following describes a specific application of the CU partitioning method described in the first aspect of the present invention with reference to an embodiment.

[0086] Example 1

[0087] Figure 3 The overall process of this invention is described. The input is a CTU (64×64) video coding tree unit in H.265 / HEVC, and the output is the partitioning depth of each patch block after the CTU is divided into patches (4×4). Finally, the CU partitioning is completed by statistically analyzing the corresponding depths of the patches blocks in the CTU.

[0088] The embodiments of the present invention mainly include steps such as dataset and label preparation, Vision Transformer training, Vision Transformer inference, and CU statistical partitioning. The implementation details of each step will be described in detail below:

[0089] Step 1. Dataset and Label Preparation

[0090] The raw data can be obtained from publicly available datasets such as Vimeo90K, REDS4, and VID4. First, the raw images are divided into 64×64 blocks to obtain CTUs (Cost-to-Units). Then, for each CTU block, the CU (Cumulative Unit) partitioning method is exhaustively traversed, and the RDC (Rate-Distortion Cost) is calculated sequentially. The CU partitioning method with the lowest rate-distortion cost is selected. Finally, the CTU blocks are divided into 4×4 patches, and each patch is assigned the depth (depth label) of its corresponding CU block.

[0091] Step 2. Vision Transformer Training

[0092] The patch block data and corresponding labels obtained in each CTU are stretched into one-dimensional data. The stretched patch block data of each CTU can be represented as input = [patch_0, patch_1, patch_2, ..., patch_255], where patch_0, patch_1, patch_2, and patch_255 are the patch block encoded data, and the input dimension is 256×16. The stretched label corresponding to the patch block can be represented as label = [label_0, label_1, label_2, ..., label_255], where label_0, label_1, label_2, and label_255 represent the partition depth of the CU to which patch_0, patch_1, patch_2, and patch_255 belong, respectively. After linear projection and the addition of positional encoding information, the input dimension becomes 256×768. This input is then fed into the Vision Transformer for attention mechanism calculation, yielding the intermediate output ViT_output = [ViT_0, ViT_1, ViT_2, ..., ViT_255]. This ViT_output is then used as the output of subsequent fully connected and softmax layers, resulting in the depth of each patch block: depth = [dep_0, dep_1, dep2, ..., dep_255]. Finally, this depth is compared with the true label, and backpropagation is used to train the Vision Transformer, fully connected layers, and softmax layers. The softmax layer loss function is the multi-class cross-entropy loss.

[0093]

[0094] Where C=5 represents the number of depth classifications, and in this invention, the depth label can take values ​​of 0, 1, 2, 3, and 4, and p(xi) represents the probability of each possible result at each depth.

[0095] Step 3. Vision Transformer Inference

[0096] After training with Vision Transformer, fully connected layer, and softmax layer, in the inference phase, as described in the training steps of Vision Transformer, the one-dimensional data stretched from the patches of the current CTU block is used as input, and after passing through Vision Transformer, fully connected layer, and softmax layer, the depth corresponding to each patch block is obtained.

[0097] Step 4. CU Statistical Partitioning

[0098] Set the partitioning thresholds α0, α1, α2, and α3, and recursively traverse and statistically partition the CTU from top to bottom (64×64 → 32×32 → 16×16 → 8×8).

[0099] 1) First, count the number of patch depth prediction values ​​greater than depth 0 inside the top-level (64×64) CTU block (depth 0, size 64×64). If the proportion of patch depth prediction values ​​is greater than α0, then divide the current CTU block into four parts; otherwise, end the CU partitioning.

[0100] 2) For all 32×32 blocks (CTU) (depth 1, size 32×32), if the number of patch depth predictions greater than depth 1 is greater than α1, then the current CU block is divided into four parts; otherwise, the current CU partitioning ends.

[0101] 3) For all 16×16 blocks (CTU) (depth 2, size 16×16), if the number of patch depth predictions greater than depth 2 is greater than α2, then the current CU block is divided into four parts; otherwise, the current CU partitioning ends.

[0102] 4) For all 8×8 blocks (CTU) (depth 3, size 8×8), if the number of patch depth predictions greater than depth 3 is greater than α3, then the current CU block is divided into four parts; otherwise, the current CU partitioning ends.

[0103] The final partitioning method of CU in CTU is obtained and used in subsequent video encoding and decoding processes.

[0104] Secondly, embodiments of the present invention provide an electronic device, such as... Figure 6 As shown, it includes:

[0105] One or more processors 501;

[0106] The memory 502 stores one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the CU partitioning method as described in any of the first aspects above.

[0107] One or more I / O interfaces 503 are connected between the processor and the memory and configured to enable information exchange between the processor and the memory.

[0108] Among them, processor 501 is a device with data processing capabilities, including but not limited to central processing unit (CPU); memory 502 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); I / O interface (read-write interface) 503 is connected between processor 501 and memory 502, and can realize information interaction between processor 501 and memory 502, including but not limited to data bus (Bus).

[0109] In some embodiments, the processor 501, memory 502, and I / O interface 503 are interconnected via bus 504, and thus connected to other components of the computing device.

[0110] Thirdly, embodiments of the present invention provide a computer-readable storage medium, such as... Figure 7 As shown, a computer program is stored on a computer-readable storage medium, and when the computer program is executed by a processor, it implements any of the CU partitioning methods in the first aspect above.

[0111] As mentioned above, while H.265 / HEVC significantly improves compression ratio compared to its predecessor, H.264 / AVC, its encoding complexity has also increased several times over. The computational method for traversing all CU partitions and calculating the rate-distortion cost (RDC) to select the optimal CU partition is a major source of time consumption in H.265 / HEVC encoding. This paper proposes a ViT-based CU partitioning method for video encoding and decoding. Taking the smallest basic block of the CTU partition as input, it outputs the partition depth of each corresponding smallest basic block. Through the calculation of an image block attention mechanism, it improves video encoding efficiency, making real-time, high-quality video encoding possible.

[0112] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0113] In hardware implementations, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0114] The preferred embodiments of the present invention have been described above with reference to the accompanying drawings, but this does not limit the scope of the invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and spirit of the present invention should be within the scope of the present invention.

Claims

1. A method for partitioning a coding unit (CU), comprising: The original image is divided into multiple coding tree units (CTUs). Each CTU is divided into a one-dimensional array of image blocks, which includes multiple image blocks, using image blocks as the granularity. Visual attention mechanism is calculated on the image blocks in the one-dimensional array of image blocks to obtain the CU segmentation depth corresponding to each image block in the one-dimensional array of image blocks; The CTU is divided into CUs according to the CU division depth corresponding to each image block; The step of performing visual attention mechanism calculations on the image blocks in the one-dimensional array of image blocks to obtain the CU partitioning depth corresponding to each image block in the one-dimensional array of image blocks includes: Expand the dimension of the one-dimensional array of image blocks to obtain the expanded input array; The input array is subjected to visual attention mechanism calculation to obtain a first calculation result; The first calculation result is used to perform a fully connected layer calculation to obtain the second calculation result; The second calculation result is used to calculate the softmax layer to obtain the CU partitioning depth corresponding to each image block in the one-dimensional array of image blocks.

2. The CU partitioning method according to claim 1, wherein, The CTU is divided into multiple layers based on its depth value. The process of dividing the CTU into CUs based on the depth of each image block includes: Based on the CU division depth corresponding to each image block, the CTU is divided into CUs layer by layer in depth order.

3. The CU partitioning method according to claim 2, wherein, The step of dividing the CTU into CUs layer by layer in depth order according to the CUs corresponding to each image block includes: Starting from the layer with current depth i = 0, iterate through and count the number Ni of image blocks with CU partitioning depth greater than i corresponding to all image blocks in the CTU; where i represents the current depth, and i and Ni are natural numbers; If Ni is greater than the preset CU partitioning threshold αi at the current depth i, then the current CU in the CTU is partitioned, and the CU partitioning at the next depth continues; where αi is greater than 0. Otherwise, terminate the CU partitioning of the CTU.

4. The CU partitioning method according to claim 1, wherein, Expanding the dimension of the one-dimensional array of image blocks includes: The one-dimensional array of image blocks is linearly projected and positional encoding information is added to obtain the expanded input array.

5. The CU partitioning method according to claim 1, wherein, The step of performing visual attention mechanism calculation on the image blocks in the one-dimensional array of image blocks to obtain the CU segmentation depth corresponding to each image block in the one-dimensional array of image blocks further includes: The visual attention mechanism is trained through at least one round of backpropagation to obtain the depth prediction values ​​of each image patch. The depth prediction values ​​of each image patch with the minimum loss are determined by the loss function and used as the CU partitioning depths corresponding to each image patch in the one-dimensional array of image patches.

6. The CU partitioning method according to any one of claims 1 to 5, wherein, The image block is a 4*4 pixel block divided according to the smallest unit of CU.

7. The CU partitioning method according to any one of claims 1 to 5, wherein, The CTU is a 64*64 pixel block.

8. An electronic device, the electronic device comprising: One or more processors; A memory having stored one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the CU partitioning method according to any one of claims 1 to 7; One or more I / O interfaces are connected between the processor and the memory and configured to enable information interaction between the processor and the memory.

9. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the CU partitioning method according to any one of claims 1 to 7.

Images