A method, device, equipment and medium for dividing coding units

Abstract

The present invention discloses a coding unit division method, the method comprising: S1: using a preset number of division modes to divide the coding unit to obtain sub-blocks of each division mode; S2: for each division mode, predicting the optimal coding cost of the division mode according to the prediction parameters of the sub-blocks of the division mode without division; S3: selecting the sub-block corresponding to the smallest optimal coding cost from the sub-blocks of each division mode; S4: when the size information of the selected sub-block is greater than the minimum limit value, using a preset number of division modes to continue dividing the selected sub-block to obtain the sub-blocks of each division mode, and continuing to perform the process of step S2. Compared with the layer-by-layer recursive method used in the prior art, the present invention directly predicts the optimal coding cost after complete recursion according to the prediction parameters of the sub-blocks without division each time, without obtaining it after layer-by-layer recursion, and is simple to implement, which can reduce the computational complexity of the entire coding process.

Description

Technical Field

[0001] The present invention relates to the field of video coding technology, and in particular to a coding unit division method, device, equipment and medium. Background Art

[0002] Video coding requires first dividing the image into CU (Coding Unit) blocks of different sizes, using larger CU blocks in areas with less image details and smaller CU blocks in areas with rich details, and then selecting the division path with the lowest coding cost for encoding based on the CU block.

[0003] In the prior art, the process of selecting the partition path of the CU block utilizes a deep search method, that is, recursively searching layer by layer to traverse all paths to find the optimal partition path.

[0004] In this way, although it is possible to ensure that the partitioning path with the lowest encoding cost is found, the computational complexity is very high and real-time encoding of high-resolution scenes cannot be achieved. Summary of the invention

[0005] The purpose of the present invention is to propose a coding unit division method, device, equipment and medium to address the deficiencies of the above-mentioned prior art, and this purpose is achieved through the following technical solutions.

[0006] A first aspect of the present invention provides a coding unit division method, the method comprising:

[0007] S1: Divide the coding unit using a preset number of division modes to obtain sub-blocks of each division mode;

[0008] S2: for each partitioning mode, predicting the optimal coding cost of the partitioning mode according to the prediction parameters of the sub-blocks of the partitioning mode without partitioning;

[0009] S3: Select the sub-block corresponding to the minimum optimal coding cost from the sub-blocks of each partition mode;

[0010] S4: When the size information of the selected sub-block is greater than the minimum limit value, the selected sub-block is further divided using a preset number of division modes to obtain sub-blocks of each division mode, and step S2 is continued.

[0011] A second aspect of the present invention provides a coding unit division device, the device comprising:

[0012] A division module, used to divide the coding unit using a preset number of division modes to obtain sub-blocks of each division mode;

[0013] A prediction module, configured to predict, for each partitioning mode, an optimal coding cost of the partitioning mode according to prediction parameters of the sub-blocks of the partitioning mode without partitioning;

[0014] A selection module, used for selecting a sub-block corresponding to a minimum optimal coding cost from the sub-blocks of each partitioning mode;

[0015] The judgment module is used to continue to divide the selected sub-block using a preset number of division modes when the size information of the selected sub-block is greater than the minimum limit value, obtain sub-blocks of each division mode, and continue to execute the process of the prediction module.

[0016] The third aspect of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the first aspect when executing the program.

[0017] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the steps of the method described in the first aspect above.

[0018] Based on the coding unit division method and device described in the first aspect and the second aspect, the present invention has the following beneficial effects:

[0019] First, the initial coding unit is divided to obtain sub-blocks of various division modes, and the optimal coding costs of various division modes are predicted by using the prediction parameters of the sub-blocks of various division modes without division, and the sub-block corresponding to the smallest optimal coding cost is selected. When the size of the sub-block does not meet the minimum limit value, the sub-block is continued to be divided and the optimal coding cost is predicted until the size of the selected sub-block meets the minimum limit value. Compared with the layer-by-layer recursive method used in the prior art, the present invention directly predicts the optimal coding cost after complete recursion based on the prediction parameters of the sub-block without division, without obtaining it after layer-by-layer recursion, and is simple to implement, which can reduce the computational complexity of the entire coding process. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0021] Figure 1 This is a flow chart of an embodiment of a coding unit division method according to an exemplary embodiment of the present invention;

[0022] Figure 2 The present invention is based on Figure 1A schematic diagram of the coding unit division process shown in the illustrated embodiment;

[0023] Figure 3 It is a structural schematic diagram of a coding unit division device according to an exemplary embodiment of the present invention;

[0024] Figure 4 It is a schematic diagram of the hardware structure of an electronic device according to an exemplary embodiment of the present invention;

[0025] Figure 5 The figure is a schematic diagram of the structure of a storage medium according to an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0026] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0027] The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the" and "the" used in the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0028] It should be understood that although the terms first, second, third, etc. may be used in the present invention to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0029] At present, the division process of CU of each size is to use various division modes for the first level division, and then continue to use various division modes for the second level division of the blocks obtained by the first level division, and so on, until the size of the divided block meets the minimum limit value, and then recursively find the division path with the lowest encoding cost from back to front.

[0030] It can be seen that this layer-by-layer recursive division method has high computational complexity, and it is difficult to achieve real-time encoding of high-resolution scenes in terms of both hardware and software.

[0031] In order to solve the above technical problems, the present application proposes a coding unit division method, that is, firstly, a preset number of division modes are used to divide the coding unit to obtain sub-blocks of each division mode, and then for each division mode, the optimal coding cost of the division mode is predicted according to the prediction parameters of the sub-blocks of the division mode when not divided, and from the sub-blocks of each division mode, the sub-block corresponding to the smallest optimal coding cost is selected, and when the size information of the selected sub-block is greater than the minimum limit value, the selected sub-block is further divided using a preset number of division modes to obtain sub-blocks of each division mode, and the process of predicting the optimal coding cost of the division mode according to the prediction parameters of the sub-blocks of the division mode when not divided is continued for each division mode.

[0032] The beneficial effects that can be achieved based on the technical solution described above are as follows:

[0033] First, the initial coding unit is divided to obtain sub-blocks of various division modes, and the optimal coding costs of various division modes are predicted by using the prediction parameters of the sub-blocks of various division modes without division, and the sub-block corresponding to the smallest optimal coding cost is selected. When the size of the sub-block does not meet the minimum limit value, the sub-block is continued to be divided and the optimal coding cost is predicted until the size of the selected sub-block meets the minimum limit value. Compared with the layer-by-layer recursive method used in the prior art, the present invention directly predicts the optimal coding cost after complete recursion based on the prediction parameters of the sub-block without division, without obtaining it after layer-by-layer recursion, and is simple to implement, which can reduce the computational complexity of the entire coding process.

[0034] Figure 1 FIG. 1 is a flow chart of an embodiment of a coding unit division method according to an exemplary embodiment of the present invention. The method can be applied to any type of electronic device, such as Figure 1 As shown, the coding unit division method includes the following steps:

[0035] Step 101: Divide a coding unit using a preset number of division modes to obtain sub-blocks in each division mode.

[0036] In one possible implementation, the preset number of partition modes may include a horizontal two-partition mode (bth), a vertical two-partition mode (btv), a horizontal I-shaped partition mode (eqth), a vertical I-shaped partition mode (eqtv), and a quadtree partition mode (qt).

[0037] It should be additionally explained that the sizes of the sub-blocks obtained by each division mode may be the same or different.

[0038] Step 102: For each partitioning mode, predict the optimal coding cost of the partitioning mode according to the prediction parameters of the sub-blocks of the partitioning mode without partitioning.

[0039] Among them, the predicted optimal coding cost is actually the coding cost after complete recursion using the existing technology, while in the present invention, it is predicted based on the prediction parameters when the sub-blocks are not divided, and there is no need to obtain it after complete recursion, thus saving the calculation required for the entire recursive process.

[0040] In one possible implementation, prediction parameters of the sub-blocks of the partitioning pattern without partitioning are obtained, and the prediction parameters are input into a trained neural network model, so that the neural network model predicts the optimal coding cost of the partitioning pattern according to the prediction parameters and outputs it.

[0041] Optionally, obtaining prediction parameters of sub-blocks without division includes: obtaining coding costs and coding prediction information of sub-blocks without division, and obtaining size information of the sub-blocks and size information of the image to which the sub-blocks belong, and then using the obtained coding costs, coding prediction information, size information of the sub-blocks, size information of the image to which the sub-blocks belong, and preset coding quality parameters as prediction parameters.

[0042] The coding cost of the sub-block without division refers to the coding cost of the sub-block without division mode (no_split). The coding prediction information of the sub-block without division includes the intra-frame optimal prediction mode and prediction angle, or the inter-frame optimal prediction mode and the corresponding reference frame and motion vector.

[0043] Those skilled in the art will appreciate that the process of obtaining the coding cost and coding prediction information of a sub-block without division may be implemented using relevant technologies, and this application does not specifically limit this.

[0044] It should be noted that the training process of the neural network model includes: obtaining training samples, the training samples including prediction information and actual coding costs of blocks of different sizes, and then using the training samples to train a pre-built neural network model.

[0045] The blocks of different sizes in the training samples may come from P frames and I frames.

[0046] During training, the loss function is defined as follows:

[0047]

[0048] In the above formula, cost p [k] represents the optimal encoding cost of the kth partitioning mode output by the neural network model prediction. r[k] represents the actual coding cost of the kth partitioning mode of the sample.

[0049] Step 103: Select the sub-block corresponding to the minimum optimal coding cost from the sub-blocks of each division mode.

[0050] Step 104: when the size information of the selected sub-block is greater than the minimum limit value, the selected sub-block is further divided using a preset number of division modes to obtain sub-blocks of each division mode, and the process of step 102 is continued.

[0051] The minimum limit value includes the minimum value of the length and the width. As long as one of the data of the sub-block size information is less than or equal to the corresponding minimum value, the division is terminated.

[0052] In an exemplary scenario, Figure 2 As shown, the initial coding unit CU uses 5 division modes (horizontal two-division mode bth, vertical two-division mode btv, horizontal I-shaped division mode eqth, vertical I-shaped division mode eqtv, and quadtree division mode qt) for the first level division, and then predicts the optimal coding cost of each division mode according to the prediction parameters of the sub-blocks of each division mode when not divided, and obtains 5 optimal coding costs. The minimum optimal coding cost is selected from these 5 optimal coding costs, that is, the optimal coding cost of the bth mode is the minimum. When the size information of the bth1 block and the bth2 block is greater than the minimum limit value, the five division modes are used for the second level division, and the optimal coding cost of each division mode is continued to be predicted according to the prediction parameters of the sub-blocks of each division mode when not divided, and 5 optimal coding costs are obtained. The minimum optimal coding cost is continued to be selected from these 5 optimal coding costs, and so on, until the division is terminated when the size information of the sub-block is less than or equal to the minimum limit value.

[0053] It can be seen that at each level of division, only the sub-blocks of one division pattern need to be divided, while the sub-blocks of each division pattern need to be divided, so the computational complexity can be reduced.

[0054] At this point, the above Figure 1The division process shown first divides the initial coding unit to obtain sub-blocks of various division modes, and predicts the optimal coding costs of various division modes by using the prediction parameters of the sub-blocks of various division modes without division, and selects the sub-block corresponding to the smallest optimal coding cost. When the size of the sub-block does not meet the minimum limit value, the sub-block continues to be divided and the optimal coding cost is predicted until the size of the selected sub-block meets the minimum limit value. Compared with the layer-by-layer recursive method used in the prior art, the present invention directly predicts the optimal coding cost after complete recursion based on the prediction parameters of the sub-block without division each time, without obtaining it after layer-by-layer recursion, and is simple to implement, which can reduce the computational complexity of the entire coding process.

[0055] Corresponding to the embodiment of the aforementioned coding unit division method, the present invention also provides an embodiment of a coding unit division device.

[0056] Figure 3 FIG. 4 is a schematic diagram of a coding unit division device according to an exemplary embodiment of the present invention. The device is used to execute the coding unit division method provided in any of the above embodiments. Figure 3 As shown, the coding unit division device includes:

[0057] A division module 310 is used to divide the coding unit using a preset number of division modes to obtain sub-blocks in each division mode;

[0058] A prediction module 320, configured to predict, for each partitioning mode, an optimal coding cost of the partitioning mode according to prediction parameters of the sub-blocks of the partitioning mode without partitioning;

[0059] A selection module 330, configured to select a sub-block corresponding to a minimum optimal coding cost from the sub-blocks of each division mode;

[0060] The judgment module 340 is used to continue dividing the selected sub-block using a preset number of division modes when the size information of the selected sub-block is greater than the minimum limit value, obtain sub-blocks of each division mode, and continue to execute the process of the prediction module.

[0061] In one possible implementation, the prediction module 320 is specifically used to obtain prediction parameters of the sub-blocks of the partitioning pattern without partitioning; input the prediction parameters into a trained neural network model so that the neural network model predicts the optimal coding cost of the partitioning pattern according to the prediction parameters and outputs it.

[0062] In a possible implementation, the prediction module 320 is specifically used to obtain the coding cost and coding prediction information of the sub-block without division in the process of obtaining the prediction parameters of the sub-block of the division mode without division; obtain the size information of the sub-block and the size information of the image to which the sub-block belongs; and use the obtained coding cost, coding prediction information, size information of the sub-block, size information of the image to which the sub-block belongs, and preset coding quality parameters as the prediction parameters.

[0063] In a possible implementation, the device further includes ( Figure 3 Not shown):

[0064] The training module is used to obtain training samples, wherein the training samples include prediction information and actual coding costs of blocks of different sizes; and use the training samples to train a pre-built neural network model.

[0065] In a possible implementation, the preset number of division modes include a horizontal two-division mode, a vertical two-division mode, a horizontal I-shaped division mode, a vertical I-shaped division mode, and a quadtree division mode.

[0066] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0067] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0068] An embodiment of the present invention further provides an electronic device corresponding to the coding unit division method provided in the above embodiment, so as to execute the above coding unit division method.

[0069] Figure 4This is a hardware structure diagram of an electronic device according to an exemplary embodiment of the present invention, and the electronic device includes: a communication interface 601, a processor 602, a memory 603 and a bus 604; wherein the communication interface 601, the processor 602 and the memory 603 communicate with each other through the bus 604. The processor 602 can execute the coding unit division method described above by reading and executing the machine executable instructions corresponding to the control logic of the coding unit division method in the memory 603. The specific content of the method is referred to the above embodiment, and will not be repeated here.

[0070] The memory 603 mentioned in the present invention can be any electronic, magnetic, optical or other physical storage device, and can contain storage information, such as executable instructions, data, etc. Specifically, the memory 603 can be RAM (Random Access Memory), flash memory, storage drive (such as hard disk drive), any type of storage disk (such as optical disk, DVD, etc.), or similar storage medium, or a combination thereof. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 601 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used.

[0071] The bus 604 may be an ISA bus, a PCI bus or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 603 is used to store programs, and the processor 602 executes the programs after receiving execution instructions.

[0072] The processor 602 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 602. The above processor 602 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be combined and executed.

[0073] The electronic device provided in the embodiment of the present application and the coding unit division method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, operated or implemented therein.

[0074] The present application also provides a computer-readable storage medium corresponding to the coding unit division method provided in the above embodiment. Figure 5 As shown, the computer-readable storage medium shown is a CD 30 on which a computer program (ie, a program product) is stored. When the computer program is executed by a processor, the coding unit division method provided in any of the aforementioned embodiments will be executed.

[0075] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0076] The computer-readable storage medium provided by the above-mentioned embodiments of the present application and the coding unit division method provided by the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0077] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art not disclosed by the present invention. The description and examples are to be regarded as exemplary only, and the true scope and spirit of the present invention is indicated by the following claims.

[0078] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0079] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A coding unit division method, characterized in that: The method comprises: S1: Divide the coding unit using a preset number of division modes to obtain sub-blocks of each division mode; S2: for each partitioning mode, predicting the optimal coding cost of the partitioning mode according to the prediction parameters of the sub-blocks of the partitioning mode without partitioning; S3: Select the sub-block corresponding to the minimum optimal coding cost from the sub-blocks of each partition mode; S4: When the size information of the selected sub-block is greater than the minimum limit value, the selected sub-block is further divided using a preset number of division modes to obtain sub-blocks of each division mode, and the process of step S2 is continued.

2. The method according to claim 1, characterized in that The predicting the optimal coding cost of the partitioning mode according to the prediction parameters of the sub-blocks of the partitioning mode without partitioning includes: Obtain prediction parameters of the sub-blocks of the partitioning mode without partitioning; The prediction parameters are input into a trained neural network model so that the neural network model predicts and outputs the optimal coding cost of the partitioning mode according to the prediction parameters.

3. The method according to claim 2, characterized in that The obtaining of prediction parameters of the sub-blocks of the division mode without division includes: Obtaining coding cost and coding prediction information of the sub-block without division; Acquire size information of the sub-block and size information of the image to which the sub-block belongs; The acquired coding cost, coding prediction information, size information of the sub-block, size information of the image to which the sub-block belongs, and preset coding quality parameters are used as the prediction parameters.

4. The method according to claim 2, characterized in that: The method also includes a training process of the neural network model: Acquire training samples, wherein the training samples include prediction information and actual coding costs of blocks of different sizes; The training samples are used to train a pre-built neural network model.

5. The method according to claim 1, characterized in that The preset number of division modes include a horizontal two-division mode, a vertical two-division mode, a horizontal I-shaped division mode, a vertical I-shaped division mode, and a quadtree division mode.

6. A coding unit division device, characterized in that: The device comprises: A division module, used to divide the coding unit using a preset number of division modes to obtain sub-blocks of each division mode; A prediction module, configured to predict, for each partitioning mode, an optimal coding cost of the partitioning mode according to prediction parameters of the sub-blocks of the partitioning mode without partitioning; A selection module, used for selecting a sub-block corresponding to a minimum optimal coding cost from the sub-blocks of each partitioning mode; The judgment module is used to continue dividing the selected sub-block using a preset number of division modes when the size information of the selected sub-block is greater than the minimum limit value, obtain sub-blocks of each division mode, and continue to execute the process of the prediction module.

7. The device according to claim 6, characterized in that The prediction module is specifically used to obtain prediction parameters of the sub-blocks of the partitioning pattern without partitioning; input the prediction parameters into a trained neural network model so that the neural network model predicts the optimal coding cost of the partitioning pattern according to the prediction parameters and outputs it.

8. The device according to claim 7, characterized in that The prediction module is specifically used to obtain the coding cost and coding prediction information of the sub-block without division in the process of obtaining the prediction parameters of the sub-block of the division mode without division; obtain the size information of the sub-block and the size information of the image to which the sub-block belongs; and use the obtained coding cost, coding prediction information, size information of the sub-block, size information of the image to which the sub-block belongs, and preset coding quality parameters as the prediction parameters.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images