Video encoding device and video encoding method
By integrating CUs and optimizing block sizes, the video encoding device reduces header information, enhancing encoding efficiency in HEVC without additional processing complexity.
Patent Information
- Application Number
- JP2023218676
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2014-03-06
- Filing Date
- 2023-12-25
- Publication Date
- 2024-12-05
- Estimated Expiration
- 2035-02-25
AI Technical Summary
In HEVC, the increasing number of CU blocks leads to an increase in coded header information, which deteriorates encoding efficiency.
A video encoding device that integrates multiple CUs into one CU based on encoding information, reducing the amount of code for header information in the CU and PU layers by limiting basic block sizes and optimizing block combinations.
This approach suppresses the amount of code for header information in the CU and PU layers, improving encoding efficiency without increasing processing complexity.
Smart Images

Figure 0007595145000001 
Figure 0007595145000002 
Figure 0007595145000003
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a video encoding device and a video encoding method that divide an input image into blocks and encode the blocks. [Background technology]
[0002] In recent years, with the development of multimedia applications, it has become common to handle all media information, including images, audio, and text, in a unified manner. In addition, since digitized images contain huge amounts of data, image information compression technology is indispensable for storage and transmission. On the other hand, standardization of compression technology is also important for interoperability of compressed image data. For example, video compression technology standards include H.261, H.263, and H.264 from the International Telecommunication Union Telecommunication Standardization Sector (ITU-T), and MPEG-1, MPEG-3, MPEG-4, and MPEG-4AVC from the International Organization for Standardization (ISO / IEC). Currently, ITU-T and ISO / IEC are working together to standardize a next-generation video coding method called High Efficiency Video Coding (HEVC).
[0003] In such video coding, each picture to be coded is divided into blocks, which are coding units, and the amount of information is compressed by reducing the redundancy in the time direction and the space direction for each block. In intra-frame predictive coding, which aims to reduce spatial redundancy, a predicted image is generated from pixel information of surrounding coded blocks, and a difference image between the obtained predicted image and the block to be coded is obtained. In inter-frame predictive coding, which aims to reduce temporal redundancy, motion is detected and a predicted image is generated for each block by referring to a previous or following picture that has already been coded, and a difference image between the obtained predicted image and the block to be coded is obtained. The amount of information is compressed by performing orthogonal transform processing such as discrete cosine transform and quantization processing on these obtained difference images, and generating a code string using variable length coding and arithmetic coding.
[0004] Fig. 1 is a conceptual diagram for explaining combinations of block sizes defined in the HEVC standard. In HEVC (Non-Patent Document 1), as a coding unit (hereinafter referred to as "Coding Unit: CU"), any size can be selected and used from four types of block sizes, 64 x 64 pixels, 32 x 32 pixels, 16 x 16 pixels, and 8 x 8 pixels, as shown in Fig. 1.
[0005] Furthermore, when the CU size is set to 32x32 pixels as a prediction unit (hereinafter referred to as "Prediction Unit: PU"), which is a unit into which a CU is further divided and which generates a predicted image in intra-frame predictive coding and inter-frame predictive coding, any size can be selected and used from eight types of block sizes, such as 32x32 pixels, 16x32 pixels, and 16x16 pixels, as shown in Figure 1. For example, high coding efficiency is achieved by using a small block size for an image in which the movement of the captured object is complex, and a large block size for an image in which the movement of the captured object is simple.
[0006] Further, as an orthogonal transform unit (hereinafter referred to as "Transform Unit: TU"), which is a unit into which a CU is divided and performs the above-mentioned orthogonal transform processing and quantization processing, if the CU size is, for example, 32x32 pixels, any size can be selected from four block sizes of 32x32 pixels, 16x16 pixels, 8x8 pixels, and 4x4 pixels, as shown in Figure 1. For example, high coding efficiency is achieved by using a small block size for images with characteristics that differ in small areas and a large block size for images with characteristics that are similar in wide areas. [Prior art documents] [Non-patent literature]
[0007] [Non-Patent Document 1] ITU-T H.265 : High efficiency video coding (04 / 2013) Summary of the Invention [Problem to be solved by the invention]
[0008] In HEVC, four types of block sizes can be selected for CU. However, the more blocks in a CU are added during the encoding process, the more the amount of header information to be coded in the CU layer increases. As a result, the encoding efficiency may decrease.
[0009] The present disclosure has been made in consideration of the above-mentioned problems, and provides a video coding device that reduces the amount of coding of header information in the CU layer and PU layer, and improves coding efficiency without increasing the amount of processing. [Means for solving the problem]
[0010] The video coding device according to the present disclosure is a video coding device that codes a current picture to generate a code string, and includes: a block division unit that, in operation, (i) divides the current picture into a plurality of coding units and (ii) divides each of the plurality of coding units into at least one prediction unit; and a residual coding unit that, in operation, generates a residual component by calculating the difference between a predicted image generated for each prediction unit and the current picture. When a first size of a first coding unit of the plurality of coding units is 2N×2N, the same as a basic block size, a second size of the prediction unit in the first coding unit is any one of 2N×2N, 2N×N, and N×2N; and when a first size of the first coding unit of the plurality of coding units is 2N×2N, which is different from the basic block size, the second size of the prediction unit in the first coding unit is any one of 2N×2N, 2N×N, N×2N, and N×N.
[0011] The video coding method according to the present disclosure is a video coding method for coding a current picture to generate a code string, the method comprising: dividing the current picture into a plurality of coding units; dividing each of the plurality of coding units into at least one prediction unit; generating a residual component by calculating a difference between a predicted image generated for each prediction unit and the current picture; when a first size of a first coding unit of the plurality of coding units is 2N×2N, the same as a basic block size, a second size of the prediction unit in the first coding unit is any one of 2N×2N, 2N×N, and N×2N; and when a first size of the first coding unit of the plurality of coding units is 2N×2N, which is different from the basic block size, the second size of the prediction unit in the first coding unit is any one of 2N×2N, 2N×N, N×2N, and N×N. Effect of the Invention
[0012] As described above, the video encoding device of the present invention is able to reduce the amount of coding of header information in the CU layer and PU layer, thereby making it possible to improve the coding efficiency without increasing the amount of processing.
[0013] In addition, the present disclosure can be realized not only as such a video encoding device, but also as a program or integrated circuit that performs processing equivalent to each of the means included in such a video encoding device. [Brief description of the drawings]
[0014] [Figure 1] FIG. 1 is a conceptual diagram for explaining combinations of block sizes defined in the HEVC standard. [Diagram 2] 1 is a block diagram showing a configuration of a video coding device 100 according to a first embodiment. [Diagram 3] FIG. 2 is a conceptual diagram for explaining combinations of block sizes according to the first embodiment. [Figure 4] 11 is a flowchart showing an integrated determination process according to the first embodiment. [Figure 5A]FIG. 1 is a conceptual diagram showing an integrated determination process according to the first embodiment; [Figure 5B] FIG. 1 is a conceptual diagram showing an integrated determination process according to the first embodiment; [Figure 6] 10 is a conceptual diagram for explaining combinations of block sizes that appear as a result of performing the integrated determination process according to the first embodiment. FIG. [Figure 7] 13 is a flowchart showing an integrated determination process according to the second embodiment. [Figure 8A] FIG. 13 is a conceptual diagram showing an integrated determination process according to the second embodiment; [Figure 8B] FIG. 13 is a conceptual diagram showing an integrated determination process according to the second embodiment; [Figure 9] FIG. 11 is a conceptual diagram for explaining conditions for the integrated determination process according to the second embodiment. [Figure 10] FIG. 11 is a conceptual diagram for explaining combinations of block sizes that appear as a result of performing the integrated determination process according to the second embodiment. [Figure 11] 13 is a flowchart showing an integrated determination process according to the third embodiment. [Figure 12A] FIG. 13 is a conceptual diagram showing an integrated determination process according to the third embodiment. [Figure 12B] FIG. 13 is a conceptual diagram showing an integrated determination process according to the third embodiment. [Figure 12C] FIG. 13 is a conceptual diagram showing an integrated determination process according to the third embodiment. [Figure 13] FIG. 13 is a conceptual diagram for explaining combinations of block sizes that appear as a result of performing the integrated determination process according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0015] Hereinafter, the embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters or duplicate explanation of substantially the same configuration may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to facilitate understanding by those skilled in the art.
[0016] It should be noted that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.
[0017] (Embodiment 1) The video coding device 100 according to the first embodiment is realized by a microprocessor mounted on, for example, a video camera, a digital camera, a video recorder, a mobile phone, a mobile information terminal, a personal computer, or the like. The video coding device 100 performs a coding process of video data according to HEVC, which is a video compression standard. The video coding device 100 according to the present disclosure integrates a plurality of CUs into one CU based on coding information of the CU (Coording Unit) and the PU (Prediction Unit), and assigns one header information to the integrated CU. As a result, it is possible to suppress the coding amount of the header information of the CU layer and the PU layer, and it is possible to improve the coding efficiency without increasing the processing amount.
[0018] The configuration and operation of the video coding device 100 according to the first embodiment will be described below with reference to the drawings.
[0019] [1-1. Configuration of video encoding device] The following describes the configuration of the video coding device 100. Fig. 2 is a block diagram showing the configuration of the video coding device 100 according to the first embodiment.
[0020] 2, the video coding device 100 includes a picture memory 101, a basic block division unit 102, a basic block unit processing loop unit 111, an integrated determination unit 107, and a code string generation unit 108. The basic block unit processing loop unit 111 of the video coding device 100 includes a prediction residual coding unit 103, a prediction residual decoding unit 104, a picture buffer 105, a prediction processing unit 106, a difference calculation unit 109, and an addition calculation unit 110. With this configuration, the video coding device 100 divides an image input in units of pictures into basic blocks, and then performs coding processing in units of the divided basic blocks to output a code string.
[0021] Next, each unit constituting the video encoding device 100 will be described in detail.
[0022] The picture memory 101 inputs image signals in picture units in the order of display on the display device. The picture memory 101 rearranges and stores the input image signals in picture units in the order of encoding. When the picture memory 101 receives a read command from the basic block division unit 102, it outputs to the basic block division unit 102 the picture to be encoded, which is the input image signal related to the read command.
[0023] The basic block division unit 102 as a division unit divides the picture to be coded, which is sequentially input from the picture memory 101, into coding units and outputs the divided pictures. The basic block division unit 102 divides the picture into basic blocks, which are basic units of processing in the basic block unit processing loop unit 111. The video coding device 100 limits the size of the basic block to 16×16 pixels, which is smaller than 64×64 pixels and 32×32 pixels defined in HEVC. The basic block includes one or more CUs, which are coding units defined in the HEVC standard. FIG. 3 is a conceptual diagram for explaining combinations of each block size according to the first embodiment. As shown in the column of "basic CU size" in FIG. 3, the basic CU size can be 16×16 pixels or 8×8 pixels for a basic block of 16×16 pixels. One CU with a size of 16×16 pixels is included in a basic block of 16×16 pixels. Four CUs with a size of 8×8 pixels are included in a basic block of 16×16 pixels. Hereinafter, these CUs included in a basic block are referred to as "basic CUs". The basic block division unit 102 divides an input picture into basic blocks and selects the size of the basic CUs. In general, the basic block division unit 102 selects small-sized basic CUs when the pixel configuration of the input picture is complex, and selects large-sized basic CUs when the pixel configuration of the input picture is simple. Note that the same explanation can be applied even when some of the sizes of basic CUs shown in the example of FIG. 3 are not used, or when a basic CU with a size not shown in the example of FIG. 3 but smaller than a basic block is used.
[0024] The basic block division unit 102 sequentially outputs the picture divided into basic blocks (the size of the basic CU has been selected) to the prediction processing unit 106 and the difference calculation unit 109.
[0025] The prediction processing unit 106 performs prediction processing using either intra-screen prediction or inter-screen prediction for each basic block based on pictures divided into basic blocks sequentially input from the basic block division unit 102. The prediction processing unit 106 performs prediction processing for each PU (hereinafter referred to as "basic PU"), which is a prediction unit obtained by further dividing a basic CU. Specifically, as shown in the "basic PU size" column in FIG. 3, when the size of a basic CU is 16×16 pixels, it is divided into one basic PU of 16×16 pixels, two basic PUs of 16×8 pixels, or two basic PUs of 8×16 pixels. However, the basic PU sizes of 16×8 pixels and 8×16 pixels are used only when inter-screen prediction is selected. On the other hand, when the size of a basic CU is 8×8 pixels, it is divided into one basic PU of 8×8 pixels, two basic PUs of 8×4 pixels, two basic PUs of 4×8 pixels, or four basic PUs of 4×4 pixels. However, the basic PU sizes of 8×4 pixels and 4×8 pixels are used only when inter-screen prediction is selected. The basic PU size of 4×4 pixels is used only when intra-screen prediction is selected. In general, the prediction processing unit 106 selects a small-sized basic PU when the pixel configuration of the input basic CU is complex, while selecting a large-sized basic PU when the pixel configuration of the input basic CU is simple. Note that the same explanation can be applied even when some sizes of the basic PUs described in the example of FIG. 3 are not used, or when a basic PU that is not described in the example of FIG. 3 and is smaller than the basic CU is used.
[0026] When intra prediction is used, the prediction processor 106 performs prediction processing for a block in a picture to be coded using a reconstructed image signal of a neighboring block that has already been coded in the same picture. The reconstructed image signal is a signal generated by an addition calculation unit 110, which will be described later. The prediction processor 106 performs prediction processing by selecting one intra prediction mode that generates a predicted image that is most similar to the pixel configuration of the block to be coded from among a plurality of intra prediction methods (intra prediction modes).
[0027] On the other hand, when using inter-picture prediction, the prediction processing unit 106 performs prediction processing using a reconstructed image signal of another picture that has already been encoded and is stored in the picture buffer 105. Specifically, the prediction processing unit 106 searches for an area having a pixel configuration that is most similar to the pixel configuration of the block to be encoded from among the reconstructed images of the other picture that has already been encoded. Then, the prediction processing unit 106 generates a predicted image by determining which picture's reconstructed image to refer to (hereinafter, information on the picture to be referred to is referred to as "reference picture information") and how far the reconstructed image at a position in the reference picture is shifted from the position corresponding to the block to be encoded (hereinafter, information indicating the amount of positional shift is referred to as "motion vector information").
[0028] The difference calculation unit 109 generates a difference image signal that is a difference value between an input image signal in basic PU units selected based on a basic CU in a basic block input from the basic block division unit 102, and a prediction image signal in basic PU units input from the prediction processing unit 106. The difference calculation unit 109 outputs the generated difference image signal to the prediction residual coding unit 103.
[0029] The predictive residual coding unit 103, which is a residual coding unit, performs an orthogonal transform process on the difference image signal input from the difference calculation unit 109, and performs a quantization process on the orthogonal transform coefficients of each frequency component obtained. As a result, the predictive residual coding unit 103 generates a residual coefficient signal. The predictive residual coding unit 103 performs an orthogonal transform process and a quantization process for each TU (Transform Unit) (hereinafter referred to as a "basic TU"), which is an orthogonal transform unit obtained by further dividing a basic CU. Specifically, as shown in the "basic TU size" column in FIG. 3, when the size of a basic CU is 16×16 pixels, it can be divided into one basic TU of 16×16 pixels or four basic TUs of 8×8 pixels. On the other hand, when the size of a basic CU is 8×8 pixels, it can be divided into one basic TU of 8×8 pixels or four basic TUs of 4×4 pixels. In the example shown in FIG. 3, when a basic PU is determined, a basic TU is uniquely assigned, so there is no need to select the size of the basic TU. This makes it possible to significantly reduce the amount of processing. The same explanation can be applied even if some of the basic TU sizes described in the example of Figure 3 are not used, or if a basic TU size not described in the example of Figure 3 that is smaller than the basic PU is used.
[0030] The prediction residual decoding unit 104 uses, as a processing unit, the basic TU processed by the prediction residual coding unit 103. The prediction residual decoding unit 104 performs inverse quantization processing on the residual coefficient signal input from the prediction residual coding unit 103, and further performs inverse orthogonal transform processing to generate a reconstructed residual image signal.
[0031] The addition calculation unit 110 generates a reconstructed image signal by adding, in basic PU units, the reconstructed differential image signal input from the prediction residual decoding unit 104 and the predicted image signal input from the prediction processing unit 106.
[0032] The picture buffer 105 stores the reconstructed image signal input from the addition calculation unit 110. The reconstructed image signal stored in the picture buffer 105 is referenced in inter prediction processing of a picture to be coded temporally after the current picture to be coded.
[0033] The integration determination unit 107, which functions as an integration unit, determines whether or not to integrate basic CUs belonging to a plurality of basic blocks to be integration-determined into one integrated coding unit (hereinafter, referred to as "integrated CU") at the time when a series of processes of the basic block unit processing loop unit 111 is completed for the plurality of basic blocks to be integration-determined, and generates an integration determination result signal. That is, the integration determination unit 107 determines whether or not to integrate a plurality of basic CUs belonging to an N×N pixel (e.g., 32×32 pixel) region formed by a plurality of basic blocks into one integrated CU. The integration determination unit 107 has a first operation mode in which the basic CUs are not integrated and are output to the code string generation unit 108 as they are, and a second operation mode in which a plurality of basic CUs belonging to the N×N pixel region are integrated into one new CU and output to the code string generation unit 108. The integration determination unit 107 switches between the first operation mode and the second operation mode according to the integration determination result. That is, the integration determination unit 107 integrates multiple basic CUs included in an N×N pixel region into a single new CU when (1) all of the multiple basic CUs and basic PUs belonging to the N×N pixel region have the same block size, and (2) the prediction information of the multiple basic PUs belonging to the N×N pixel region is the same.
[0034] The code string generation unit 108 generates a code string by performing variable length coding and arithmetic coding on the residual coefficient signal input from the prediction residual coding unit 103 and other coding information signals required for the decoding process, in units of integrated CUs or units of basic CUs, in accordance with the integration determination result signal input from the integration determination unit 107. That is, the code string generation unit 108 performs variable length coding and arithmetic coding on the coding information set for the new integrated CU and the residual coefficients for the multiple basic CUs before integration that belonged to the new integrated CU, to generate a code string for the new integrated CU. When the integrated CU is output from the integration determination unit 107, the code string for the integrated CU is generated by performing variable length coding and arithmetic coding on the coding information set for the integrated CU and the residual coefficients for the multiple basic CUs before integration that belonged to the integrated CU.
[0035] As described above, in the video encoding device 100, the basic block size is limited to 16×16 pixels. This makes it possible to limit the selectable combinations of basic CU / basic PU / basic TU to only three combinations when intra prediction is selected, and to only six combinations when inter prediction is selected, as shown in Fig. 3. This makes it possible to significantly reduce the amount of processing required to select an optimal combination in a series of processes in the basic block unit processing loop unit 111.
[0036] In the above embodiment, the basic block size is limited to 16×16 pixels, but the present disclosure is not limited thereto. That is, the basic block size may not be limited, and an optimal combination may be selected from the combinations in FIG. 1 defined in the HEVC standard. However, in this case, the amount of processing will be enormous compared to when the basic block size is limited. Alternatively, the basic block size may not be limited to 16×16 pixels, but may be limited to other block sizes such as 32×32 pixels. For example, when limited to a block size of 32×32 pixels, the number of options for the block sizes of CUs and PUs belonging to the basic block size increases, so that the coding efficiency can be improved. On the other hand, the amount of processing for selecting the optimal block sizes of CUs and PUs increases. Therefore, a large block size may be used as the basic block size in a video encoding device that can tolerate an increase in the amount of processing, while a small block size may be used as the basic block size in a video encoding device that cannot tolerate an increase in the amount of processing.
[0037] [1-2. Operation of the integrated judgment unit] The integration determination unit 107 according to the first embodiment defines an integration area including a plurality of basic blocks. The integration determination unit 107 performs integration determination processing for all basic blocks included in the integration area when a series of processing steps of the basic block unit processing loop unit 111 is completed.
[0038] A method for determining whether or not to integrate basic CUs belonging to a plurality of basic blocks into one integrated CU in the integration determination unit 107 will be specifically described with reference to Figs. 4, 5A, and 5B. Fig. 4 is a flowchart showing the integration determination process according to the first embodiment. Figs. 5A and 5B are conceptual diagrams showing the integration determination process according to the first embodiment. Fig. 4 shows the process when the size of the basic block is 16x16 pixels and the size of the integrated area is 32x32 pixels. At this time, the integrated area includes four basic blocks. Note that the size of the integrated area may be a size other than 32x32 pixels depending on the size of the basic block, as long as it is larger than the size of the basic block.
[0039] First, the integration determination unit 107 determines whether or not all of the four basic blocks included in the integrated region are composed of basic CUs of 16×16 pixels and basic PUs of 16×16 pixels (S301).
[0040] If the condition in S301 is not satisfied (No in S301), the basic CUs in the consolidated region are not consolidated, as shown in FIG. 5A.
[0041] On the other hand, when the condition of S301 is satisfied (Yes in S301), the integrated determination unit 107 determines whether the prediction information of the four basic PUs in the integrated area is all the same (S302). Specifically, in the case of intra-screen prediction, the integrated determination unit 107 determines whether at least the intra-screen prediction modes of the four basic PUs in the integrated area are all the same. That is, when all basic PUs included in the integrated area are intra-screen prediction, the integrated determination unit 107 switches between the first operation mode and the second operation mode using whether at least the intra-screen prediction modes are all the same as the prediction information. On the other hand, in the case of inter-screen prediction, the integrated determination unit 107 determines whether at least the motion vector information and the reference picture information of the four basic PUs in the integrated area are all the same. That is, when all basic PUs included in the integrated area are inter-screen prediction, the integrated determination unit 107 switches between the first operation mode and the second operation mode using whether at least the motion vector information and the reference picture information are all the same as the prediction information.
[0042] If the condition in S302 is not satisfied (No in S301), the basic CUs in the consolidated area are not consolidated, as shown in FIG. 5A.
[0043] On the other hand, if the condition of S302 is satisfied (Yes in S301), the integration determination unit 107 integrates the four basic CUs of 16×16 pixels into one integrated CU of 32×32 pixels, as shown in FIG. 5B (S303).
[0044] Fig. 6 is a conceptual diagram for explaining combinations of block sizes that appear as a result of performing the integration determination process. These block sizes are the targets of variable length coding and arithmetic coding in the code string generation unit 108. Comparing with Fig. 3, it can be seen that an integrated CU with a CU size of 32x32 pixels, a PU size of 32x32 pixels, and a TU size of 16x16 pixels is added as a result of the integration determination process.
[0045] As described above, in the video encoding device 100 according to the first embodiment, when a plurality of basic CUs and basic PUs belonging to a combined region (a region of N×N pixels consisting of a plurality of basic blocks) all have the same block size and the prediction information of all the basic PUs included in the combined region is the same, the combined determination unit 107 combines the plurality of basic CUs included in the combined region into one new CU, and generates a code string based on the new combined CU.
[0046] If the integration process shown in FIG. 4 is not performed, even if the basic PU has the same prediction information, it is necessary to describe the prediction information of each basic PU individually in the code string, which results in unnecessary coding of the header information of the CU layer and the PU layer. On the other hand, if the integration process is performed, it is only necessary to describe one integrated prediction information in the code string, which makes it possible to suppress the coding amount of the header information of the CU layer and the PU layer, and to improve the coding efficiency without increasing the amount of processing. Note that the integration determination unit 107 integrates only the CU and the PU in the integrated CU, while leaving the TU as it is before integration. This makes it possible to convert to an integrated CU by only changing the header information of the CU layer and the PU layer, without needing to reconstruct the residual coefficient signal after integration.
[0047] (Embodiment 2) Next, a video coding device 100 according to the second embodiment will be described with reference to the drawings. Note that the configuration of the video coding device 100 is similar to that described in the first embodiment, and therefore the description will be omitted.
[0048] The video coding device 100 according to the second embodiment differs from the video coding device 100 according to the first embodiment in the integrated determination process performed by the integrated determination unit 107.
[0049] Fig. 7 is a flowchart showing the integration determination process according to the second embodiment. Fig. 8A and Fig. 8B are conceptual diagrams showing the integration determination process according to the second embodiment. Fig. 7 shows the process when the size of the basic block is 16 x 16 pixels and the size of the integrated area is 32 x 32 pixels. In this case, four basic blocks are included in the integrated area.
[0050] First, the integration determination unit 107 determines whether or not all four basic blocks included in the integration region are composed of 16×16 pixel basic CUs and 16×16 pixel basic PUs (S301). If the condition of S301 is not satisfied (No in S301), the basic CUs in the integration region are not integrated as shown in FIG. 8A. On the other hand, if the condition of S301 is satisfied (Yes in S301), the integration determination unit 107 determines whether or not the prediction information of only the basic PUs that constitute the combination that can be integrated for the four basic PUs in the integration region is the same (S502 in FIG. 7). Details of the determination in S502 in FIG. 7 will be described with reference to FIG. 9.
[0051] 9 is a conceptual diagram for explaining the conditions of the integration judgment process according to the second embodiment. As shown in FIG. 9, when the prediction information of the four basic PUs is all the same, the integration judgment unit 107 integrates the four basic PUs into one integrated PU of 32×32 pixels. Also, even if the prediction information of the four basic PUs is not all the same, when the prediction information of two sets of basic PUs adjacent to each other on the left and right is the same, the integration judgment unit 107 integrates the two basic PUs into two integrated PUs, a PU of 32×16 pixels obtained by integrating the two basic PUs on the top side and a PU of 32×16 pixels obtained by integrating the two basic PUs on the bottom side. Also, even if the prediction information of the four basic PUs is not all the same, when the prediction information of two sets of basic PUs adjacent to each other on the top and bottom is the same, the integration judgment unit 107 integrates the two basic PUs into two integrated PUs, a PU of 16×32 pixels obtained by integrating the two basic PUs on the left side and a basic PU of 16×32 pixels obtained by integrating the two PUs on the right side. Note that the 32×16 pixel combined PU and the 16×32 pixel combined PU can be selected for combination only in the case of inter prediction, and only the 32×32 pixel combined PU can be selected for combination in the case of intra prediction.
[0052] In short, in the integration determination process according to the second embodiment, when multiple basic PUs belonging to an integration region (region of N×N pixels) are divided into a group of two adjacent basic PUs, and when the prediction information of the basic PUs belonging to each group is the same, the multiple basic PUs included in the integration region are integrated into one new PU. In this case, the integration determination unit 107 integrates four basic CUs of 16×16 pixels into one integrated CU of 32×32 pixels, as shown in FIG. 8B (S303 in FIG. 7).
[0053] Fig. 10 is a conceptual diagram for explaining combinations of block sizes that appear as a result of performing the integration determination process. These block sizes are the targets of variable length coding and arithmetic coding in the code string generation unit 108. Comparing with Fig. 3, it can be seen that an integrated CU having a CU size of 32x32 pixels, a PU size of 32x32 pixels, and a TU size of 16x16 pixels, as well as an integrated CU having a CU size of 32x32 pixels, a PU size of 32x16 pixels or 16x32 pixels, and a TU size of 16x16 pixels are added as a result of the integration determination process.
[0054] As described above, in the video encoding device 100 according to the second embodiment, when multiple basic PUs belonging to an integrated region (region of N×N pixels) are divided into groups of two adjacent basic PUs, if the prediction information of the basic PUs belonging to each divided group is the same, the integration determination unit 107 integrates multiple basic CUs included in the integrated region into one new CU. Then, a code string is generated based on the new integrated CU.
[0055] For example, when the prediction information of all basic PUs included in the upper half of the integrated region (N×N pixel region) is the same information and the prediction information of all basic PUs included in the lower half of the integrated region is the same information, the integrated determination unit 107 integrates the multiple basic CUs included in the integrated region into one new CU. Alternatively, when the prediction information of all basic PUs included in the left half of the integrated region (N×N pixel region) is the same information and the prediction information of all basic PUs included in the right half of the integrated region is the same information, the integrated determination unit 107 integrates the multiple basic CUs included in the integrated region into one new CU.
[0056] In the merging process according to the first embodiment, only when the prediction information of the four basic PUs is the same, the merging process is performed to merge the four basic PUs into a merged CU. In contrast, in the merging process according to the second embodiment, even when the prediction information of two sets of basic PUs is the same, the PUs are merged into a merged CU, so that it is possible to merge more CUs. This makes it possible to further reduce the amount of coding of the header information of the CU layer and the PU layer, and to improve the coding efficiency without increasing the amount of processing.
[0057] In addition, the integration determination unit 107 integrates only the CU and the PU in the integrated CU, while leaving the TU as it is before integration. This makes it possible to convert to an integrated CU by only changing the header information of the CU layer and the PU layer without needing to reconstruct the residual coefficient signal after integration.
[0058] (Embodiment 3) Next, a video coding device 100 according to a third embodiment will be described with reference to the drawings. Note that the configuration of the video coding device 100 is similar to that described in the first embodiment, and therefore the description will be omitted.
[0059] The video coding device 100 according to the third embodiment differs from the video coding device 100 according to the first and second embodiments in the integrated determination process performed by the integrated determination unit 107.
[0060] The integration determination unit 107 according to the third embodiment defines two-stage integration areas, that is, integration area 1 and integration area 2, as integration areas including a plurality of basic blocks. The integration determination unit 107 performs integration determination processing for all basic blocks included in the integration area when a series of processes in the basic block unit processing loop unit 111 is completed.
[0061] A method of determining whether or not to integrate basic CUs belonging to a plurality of basic blocks into one integrated CU in the integration determination unit 107 will be specifically described with reference to FIG. 11, FIG. 12A, FIG. 12B, and FIG. 12C. FIG. 11 is a flowchart showing the integration determination process according to the third embodiment. FIG. 12A, FIG. 12B, and FIG. 12C are conceptual diagrams showing the integration determination process according to the third embodiment. FIG. 11 shows the process when the basic block is 16×16 pixels, the integrated area 1 is 32×32 pixels, and the integrated area 2 is 64×64 pixels. The integrated area 2 includes the integrated area 1. In this case, the integrated area 1 includes four basic blocks. The integrated area 2 includes four integrated areas 1. That is, the integrated area 2 includes 16 basic blocks. Note that, as the size of the integrated area, a size other than 32×32 pixels and 64×64 pixels may be used according to the size of the basic block as long as it is larger than the size of the basic block.
[0062] First, the integration determination unit 107 determines whether or not all of the four basic blocks included in the integrated area 1 are composed of basic CUs of 16×16 pixels and basic PUs of 16×16 pixels (S801).
[0063] If the condition in S801 is not satisfied (No in S801), the basic CUs in consolidated area 1 are not consolidated, as shown in FIG. 12A.
[0064] On the other hand, when the condition of S801 is satisfied (Yes in S801), the integrated determination unit 107 determines whether the prediction information of the four basic PUs in the integrated region 1 is all the same (S802). Specifically, in the case of intra prediction, it determines whether at least the intra prediction modes of the four basic PUs in the integrated region 1 are all the same. Also, in the case of inter prediction, it determines whether at least the motion vector information and the reference picture information of the four basic PUs in the integrated region 1 are all the same.
[0065] If the condition in S802 is not satisfied (No in S802), the basic CUs in the consolidated area 1 are not consolidated, as shown in FIG. 12A.
[0066] On the other hand, if the condition of S802 is satisfied (Yes in S802), the integration determination unit 107 integrates the four basic CUs of 16×16 pixels into one integrated CU1 of 32×32 pixels (S803).
[0067] The integration determination unit 107 performs a series of processes from S801 to S803 for all four integrated regions 1 belonging to the integrated region 2 (S804). That is, when the series of processes from S801 to S803 have not been completed for all integrated regions 1 in the integrated region 2 (No in S804), the integration determination unit 107 performs a series of operations from S801 to S803 for the integrated region 1 for which the processes have not been completed. As a result, when integration processing has not been performed for any integrated region 1, as shown in FIG. 12A, all of the 16 basic CUs in the integrated region 2 are not integrated. On the other hand, when only some of the four integrated regions 1 are integrated, only some basic CUs in the integrated region 2 are integrated into an integrated CU1 of 32×32 pixels as shown in FIG. 12B.
[0068] Next, when the series of processes from S801 to S803 have been completed for all integrated areas 1 within the integrated area 2 (YES in S804), the integration determination unit 107 determines whether all CUs have been integrated into integrated CU1 for the four integrated areas 1 belonging to the integrated area 2 (S805).
[0069] If the condition of S805 is not satisfied (No in S805), the integration judgment unit 107 ends the integration judgment process.
[0070] When the condition of S805 is satisfied (Yes in S805), the integrated determination unit 107 determines whether the prediction information of the four integrated PUs 1 in the integrated region 2 is all the same (S806).
[0071] If the condition in S806 is not satisfied (No in S806), the integration judgment unit 107 ends the integration judgment process.
[0072] If the condition of S806 is satisfied (Yes in S806), the merger determination unit 107 further merges the four 32×32 pixel merged CU1s into one 64×64 pixel merged CU2 (S807), as shown in FIG. 12C. The video encoding device 100 limits the size of the basic block to 16×16 pixels, which is smaller than the 64×64 pixels and 32×32 pixels defined in HEVC. This enables the merger into a 32×32 pixel merged CU1 and a 64×64 pixel merged CU2.
[0073] Fig. 13 is a conceptual diagram for explaining combinations of block sizes that appear as a result of performing the integration determination process. These block sizes are the targets of variable length coding and arithmetic coding in the code string generation unit 108. Comparing with Fig. 3, it can be seen that the integration determination process adds an integrated CU1 having a CU size of 32x32 pixels, a PU size of 32x32 pixels, and a TU size of 16x16 pixels, and an integrated CU2 having a CU size of 64x64 pixels, a PU size of 64x64 pixels, and a TU size of 16x16 pixels.
[0074] As described above, in the video encoding device 100 according to the third embodiment, the integration determination unit 107 integrates multiple basic CUs included in the integrated region 1 into one new integrated CU1 when multiple basic CUs and basic PUs belonging to the integrated region 1 (a region of N×N pixels consisting of multiple basic blocks) all have the same block size and the prediction information of all basic PUs included in the integrated region 1 is the same. Furthermore, when all CUs in the integrated region 2 (a region consisting of multiple integrated regions 1) are integrated into the integrated CU1 and the prediction information of all integrated PUs in the integrated region 2 is the same, the integration determination unit 107 integrates multiple basic CUs included in the integrated region 2 into one new integrated CU2. Then, a code string is generated based on the new integrated CU.
[0075] In the merging process according to the first embodiment, a maximum of four basic CUs are merged. In contrast, in the merging process according to the third embodiment, a maximum of 16 basic CUs are merged, making it possible to merge more CUs into one CU. This makes it possible to further reduce the amount of coding of the header information of the CU layer and the PU layer, and to improve the coding efficiency without increasing the amount of processing.
[0076] In addition, the integration determination unit 107 integrates only CUs and PUs in integrated CU1 and integrated CU2, while leaving TUs as they are before integration. This makes it unnecessary to reconstruct residual coefficient signals after integration, and makes it possible to convert to integrated CU1 and integrated CU2 by simply changing the header information of the CU layer and the PU layer.
[0077] (Other embodiments) As described above, the first to third embodiments have been described as examples of the technology disclosed in the present application. However, the technology in the present disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made. In addition, it is also possible to combine the components described in the first to third embodiments to create new embodiments.
[0078] Therefore, other embodiments will be exemplified below.
[0079] The integrated determination processes described in the first to third embodiments are not limited to being used individually. That is, any two or more of the integrated determination processes described in the first to third embodiments may be used in combination. For example, in the flowchart of the third embodiment shown in FIG. 11, both or one of step S802 and step S805 may be replaced with step S502 in the flowchart of the second embodiment shown in FIG.
[0080] Furthermore, a program having the same functions as the means included in the video encoding device 100 shown in the above-mentioned first to third embodiments may be recorded on a recording medium such as a flexible disk. This makes it possible to easily implement the processes shown in the above-mentioned embodiments in an independent computer system. Note that the recording medium is not limited to a flexible disk, and may be an optical disk, an IC card, a ROM cassette, or any other medium capable of recording a program.
[0081] Furthermore, functions equivalent to those of the units included in the video encoding device 100 shown in the first to third embodiments may be realized as an LSI, which is an integrated circuit. These units may be integrated into a single chip so as to include some or all of the units. Depending on the degree of integration, the LSI may be called an IC, a system LSI, a super LSI, or an ultra LSI.
[0082] The method of integration is not limited to LSI, but may be realized by a dedicated circuit or a general-purpose processor. It is also possible to use a field programmable gate array (FPGA) that can be programmed after LSI manufacturing, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells inside the LSI.
[0083] Furthermore, if an integrated circuit technology that can replace LSIs and the like emerges as a result of advances in semiconductor technology or other derived technologies, it is of course possible to use that technology to integrate functional blocks.
[0084] It should be noted that the above-described embodiments are intended to illustrate the technology of the present disclosure, and various modifications, substitutions, additions, omissions, and the like can be made within the scope of the claims or their equivalents. [Industrial Applicability]
[0085] The present disclosure can be applied to a video encoding device that encodes each picture constituting an input image and outputs the encoded video data, for example, a video camera, a digital camera, a video recorder, a mobile phone, a personal digital assistant, a personal computer, and the like. [Explanation of symbols]
[0086] 100 Video encoding device 101 Picture Memory 102 Basic block division part 103 Prediction residual coding unit 104 Prediction residual decoding unit 105 Picture Buffer 106 Prediction processing unit 107 Integrated Judgment Department 108 Code string generator 109 Difference calculation section 110 Addition operation unit 111 Basic block unit processing loop
Claims
1. A video encoding device that encodes a target picture to generate a code string, comprising: In operation, a block division unit that (i) divides the current picture into a plurality of coding units, and (ii) divides each of the plurality of coding units into at least one prediction unit; A residual coding unit that generates a residual component by calculating a difference between a prediction image generated for each prediction unit and the target picture in operation, When a first size of a first coding unit of the plurality of coding units is 2N×2N, which is the same as a basic block size, a second size of a prediction unit in the first coding unit is any one of 2N×2N, 2N×N, and N×2N; When a first size of the first coding unit of the plurality of coding units is 2N×2N, which is different from the basic block size, the second size of the prediction unit in the first coding unit is any one of 2N×2N, 2N×N, N×2N, and N×N. A video encoding device.
2. 1. A video encoding method for encoding a target picture to generate a code string, comprising: Dividing the current picture into a plurality of coding units, and dividing each of the plurality of coding units into at least one prediction unit; generating a residual component by calculating a difference between a predicted image generated for each prediction unit and the target picture; When a first size of a first coding unit of the plurality of coding units is 2N×2N, which is the same as a basic block size, a second size of a prediction unit in the first coding unit is any one of 2N×2N, 2N×N, and N×2N; When a first size of the first coding unit of the plurality of coding units is 2N×2N, which is different from the basic block size, the second size of the prediction unit in the first coding unit is any one of 2N×2N, 2N×N, N×2N, and N×N. A video coding method.
Citation Information
Patent Citations
Method and apparatus for determining a quantization parameter predictor from multiple adjacent quantization parameters.
JP2013531942A
Video information encoding method and decoding method
JP2013545372A
Image decoding device, image encoding device, image decoding method, and image encoding method
WO2013108330A1
Method and devices for encoding a sequence of images into a scalable video bit-stream, and decoding a corresponding scalable video bit-stream
WO2013128010A2
Image decoding device and image decoding method
WO2013136678A1
