Image decoding device, image decoding method, and image decoding program

Abstract

To provide a technique for improving encoding efficiency by performing block division suitable for image encoding and decoding.SOLUTION: A block division unit includes: a four division unit that generates four blocks by dividing each of a horizontal direction and a vertical direction of a target block in half to realize four division of the target block; and a 2-3 division unit that generates two or three blocks by dividing the target block in recursive division into two or three in the horizontal direction or the vertical direction. The 2-3 division unit prohibits horizontal division of the target block in a case in which the target block is horizontally divided, when the divided target block exceeds a right side of a picture boundary, and prohibits vertical division of the target block in a case in which the target block is vertically divided when the divided target block exceeds a lower side of the picture boundary.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a technique for encoding and decoding an image in divided block units.

Background Art

[0002] In image encoding and decoding, an image is divided into blocks, which are sets of a predetermined number of pixels, and processed in block units. At this time, by dividing into appropriate block units, the efficiency of intra prediction, inter prediction, orthogonal transformation, entropy encoding, etc. is improved, and as a result, the encoding efficiency is improved. By dividing into appropriate block units, the efficiency of intra prediction, inter prediction, orthogonal transformation, entropy encoding, etc. is improved, and as a result, the encoding efficiency is improved. By dividing into appropriate block units, the efficiency of intra prediction, inter prediction, orthogonal transformation, entropy encoding, etc. is improved, and as a result, the encoding efficiency is improved. As a result, the encoding efficiency is improved.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Non-Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] If the blocks are not divided into appropriate sizes and shapes, the encoding efficiency will decrease. In particular, at the picture edge, blocks including pixels at positions exceeding the picture boundary have inappropriate sizes and shapes, resulting in a decrease in encoding efficiency. At the picture edge, blocks including pixels at positions exceeding the picture boundary have inappropriate sizes and shapes, resulting in a decrease in encoding efficiency. As a result, the encoding efficiency is reduced.

[0006] The present invention has been made in view of such a situation, and its object is to perform image encoding and decoding To provide a technology that improves coding efficiency by performing block partitioning suitable for encoding. It is located there. [Means for solving the problem]

[0007] In one aspect of the present invention that solves the above problems, the block division part is a recursive division Divide the elephant block in half horizontally and vertically to create four blocks. The division section and the target block in the recursive division are divided into two or three horizontally or vertically. It includes a 2-3 division section that divides to produce two or three blocks, and the 2-3 division section is If the target block is divided horizontally, the divided target block will be at the picture boundary. If it exceeds the right side, horizontal division of the target block is prohibited, and the target block is If it is divided vertically, the divided target block will extend below the picture boundary. This prohibits dividing the target block vertically. [Effects of the Invention]

[0008] According to the present invention, block division suitable for image encoding and decoding becomes possible, and encoding Efficiency can be improved. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram of an image encoding device according to the first embodiment. [Figure 2] This is a block diagram of an image decoding device according to the first embodiment. [Figure 3] This is a flowchart illustrating the division into tree blocks and the division within tree blocks. [Figure 4] This diagram shows how an input image is divided into tree blocks. [Figure 5]This is a diagram for explaining the z-scan. [Figure 6] This is a diagram for explaining the splitting of tree blocks. [Figure 7] This is a flowchart for explaining the processing of each split block when the tree block is split into four. [Figure 8] This is a flowchart for explaining the processing of each split block when the tree block is split into two or three. [Figure 9] This is a diagram showing the relationship between the tree block and the picture boundary. [Figure 10] This is a diagram showing the relationship between the picture boundary and the pixels. [Figure 11] This is a flowchart for explaining the block splitting in the first embodiment. [Figure 12] This is a diagram showing the block splitting in the first embodiment. [Figure 13] This is a diagram showing the syntax regarding block splitting in the first embodiment. [Figure 14] This is a flowchart for explaining the block splitting in the second embodiment. [Figure 15] This is a diagram showing the block splitting in the second embodiment. [Figure 16] This is a flowchart for explaining the block splitting in the third embodiment. [Figure 17] This is a diagram showing the block splitting in the third embodiment. [Figure 18] This is a flowchart for explaining the block splitting in the third embodiment. [Figure 19] This is a diagram showing the block splitting in the fourth embodiment. [Figure 20] This is a flowchart for explaining the block splitting in the fourth embodiment. [Figure 21] This is a diagram showing the block splitting in the fifth embodiment. [Figure 22] This is a diagram for explaining an example of the hardware configuration of the encoding / decoding device in the first embodiment. [Modes for carrying out the invention]

[0010] Embodiments of the present invention provide a technology for encoding and decoding images in divided block units. To provide. (First Embodiment) Regarding the image encoding device 100 and image decoding device 200 according to the first embodiment of the present invention Let me explain. In the first embodiment, when dividing a block into two or three parts, the block Limit the partitioning.

[0011] Figure 1 is a block diagram of the image encoding device 100 according to the first embodiment. This only represents the data flow related to the image signal, and does not include image data such as motion vectors or prediction modes. The data flow related to additional information other than signals is not shown in the diagram. The image encoding device 100 has At least one screen's worth of image signal is input.

[0012] The block division unit 101 divides the image into encoding target blocks, which will be the units of encoding processing. The image signal within the block to be encoded is supplied to the residual signal generation unit 103. The division unit 101 evaluates the degree of matching of the predicted image by processing the image signal of the block to be encoded. This is supplied to the predictive image generation unit 102.

[0013] The block division unit 101 recursively divides the image into rectangles of a predetermined size and encodes the blocks to be encoded. A lock is generated. The block partitioning unit 101 divides the target block in the recursive partitioning into four parts. A four-part division section that generates four blocks, and a recursive division that divides the target block into two. It also includes a 2-3 division section that divides into 3 parts to generate 2 or 3 blocks. The detailed operation of the split section 101 will be described later.

[0014] The predictive image generation unit 102 receives the image signal of the block to be encoded from the block division unit 101 and The decoded image signal is supplied from the decoded image memory 108. The predictive image generation unit 102 supplies Using the signal, intra-prediction (in-screen prediction) and inter-prediction ( Inter-screen prediction is performed to generate a predicted image signal. In intra-prediction, the target block is encoded. In the same picture (encoded picture), the encoded blocks adjacent to the block to be encoded The image signals of the blocks are supplied from the decoded image memory 108 to the predictive image generation unit 102. The predictive image generation unit 102 then receives this image signal and the block division unit 101. Using the image signal of the block to be encoded, a predicted image signal is generated. , an encoded picture (reference image) that is chronologically before or after the encoded picture. The image signal is supplied from the decoded image memory 108 to the predictive image generation unit 102. The predictive image generation unit 102 combines this image signal with the coded pairs supplied from the block division unit 101. Using the elephant block, the degree of agreement is evaluated by block matching, etc., and the amount of movement is indicated. The motion vector is calculated. The predictive image generation unit 102 uses this motion vector to generate a motion from a reference image. The predictive image generation unit 102 then generates the predicted image signal. The predicted image signal is supplied to the residual signal generation unit 103.

[0015] The residual signal generation unit 103 combines the image signal to be encoded with the image generated by the prediction image generation unit 102. The predicted signal is subtracted from the residual signal to generate a residual signal, which is then supplied to the orthogonal transformation / quantization unit 104.

[0016] The orthogonal transformation / quantization unit 104 orthogonally transforms the residual signal supplied from the residual signal generation unit 103. The orthogonal transformation and quantization unit 104 encodes the orthogonally transformed and quantized residual signal. It is supplied to the quantization unit 105 and the inverse quantization / inverse orthogonal transformation unit 106.

[0017] The encoding unit 105 receives orthogonal transform and quantized data from the orthogonal transform and quantization unit 104. It generates an encoded bitstream corresponding to the residual signal. The encoding unit 105 also generates an encoded bitstream corresponding to each component Additional information such as motion vectors, prediction modes, and block division information supplied from the component elements. Then, it generates the corresponding encoded bitstream. The encoding unit 105 then encodes The bitstream is output from the image encoding device 100.

[0018] The inverse quantization / inverse orthogonal transformation unit 106 receives orthogonal transformations supplied from the orthogonal transformation / quantization unit 104. The quantized residual signal is inversely quantized and then inversely orthogonal transformed to obtain the residual signal. The signal conversion unit 106 supplies the residual signal to the decoded image signal superimposition unit 107.

[0019] The decoded image signal superimposition unit 107 combines the predicted image signal generated by the predicted image generation unit 102 with the predicted image signal. The residual signal obtained in the inverse quantization / inverse orthogonal transform unit 106 is superimposed to generate a decoded image, and the decoding The image is stored in the image memory 108. The decoded image signal superimposition unit 107 superimposes the decoded image. A filtering process is applied to reduce block distortion and other issues caused by the numbering process, and the decoded image is stored in memory 10. It may be stored in 8.

[0020] Figure 2 is a block diagram of the image decoding device 200 according to Embodiment 1. Figure 2 shows an image This only represents the data flow related to the signal, and does not include image signals such as motion vectors or prediction modes. The data flow for additional information other than that is not shown in the diagram. The image decoding device 200 has a code A numbered bitstream is input.

[0021] The decoding unit 201 decodes the supplied encoded bitstream and performs orthogonal transformations and quantum transformations. The converted residual signal is supplied to the block division unit 202. The decoding unit 201 also processes the motion... Additional information such as the operator, prediction mode, and block division information is supplied to each component, and additional information It is used for reporting and corresponding processing.

[0022] The block division unit 202 decodes the blocks to be decoded based on the decoded block division information. Determine the shape, and then reverse the orthogonal transform and quantized residual signals of the determined decoded block. This is supplied to the quantization / inverse orthogonal transformation unit 203.

[0023] The block division unit 202, based on the decoded block division information, determines a predetermined size The image is recursively divided into rectangles to generate blocks to be decoded. Block division unit 202 This is a four-part division section that divides the target block in the recursive partition into four parts to generate four blocks, and , dividing the target block in recursive partitioning into 2 or 3 blocks It includes a 2-3 division section that generates the block. The detailed operation of the block division section 202 will be described later. ru.

[0024] The inverse quantization / inverse orthogonal transformation unit 203 inversely transforms the supplied orthogonal transformed / quantized residual signal. Substitution and inverse orthogonal transformation are performed to obtain the residual signal. This is then supplied to the decoded image signal superposition unit 205.

[0025] The predictive image generation unit 204 predicts from the decoded image signal supplied from the decoded image memory 206. An image signal is generated and supplied to the decoded image signal superimposition unit 205.

[0026] The decoded image signal superimposition unit 205 superimposes the predicted image signal generated by the predicted image generation unit 204 and the inverse The residual signal obtained in the quantization / inverse orthogonal transform unit 203 is superimposed with the decoded image signal to generate the image signal. Furthermore, the decoded image signal superimposition unit 205 stores the decoded image signal in the decoded image memory 206. Furthermore, the decoded image signal superimposition unit 205 reduces block distortion and other issues caused by encoding in the decoded image. A filtering process may be applied to the decoded image before storing it in the image memory 206. The decoded image signal superimposition unit 205 outputs the decoded image from the image decoding device 200.

[0027] Next, regarding the operation of the block division unit 101 in the image encoding device 100, Figure 3 is used. Let me explain. Figure 3 shows that the block division unit 101 divides the image into tree blocks, and within This demonstrates the process of dividing a section into blocks.

[0028] First, the input image is divided into tree blocks of a predetermined size (S1000). Therefore, the tree block will be 128x128 pixels. However, the tree block will be this size It is not limited to this; any size and aspect ratio can be used as long as it is a rectangle. Also, encoding The size of the tree block may be predetermined between the device and the decoding device. Furthermore, The encoding device determines the size of the tree block and records it in the encoded bitstream, and The encoding device uses the size of the tree block recorded in the encoded bitstream. This is also acceptable. Figure 4 shows how the input image is divided into tree blocks. The locks are encoded in raster scan order, that is, from left to right and top to bottom.

[0029] The inside of the tree block is further divided into rectangular blocks. The inside of the tree block is shown in the diagram. Encode in the z-scan order shown in 5. The z-scan order is top left, top right, bottom left, bottom right, The order is shown. Within a tree block, division is possible into 4, 2, or 3 parts. .

[0030] The block is divided into four sections by halving it horizontally and vertically, as shown in Figure 6(a). This is done by dividing and generating four blocks.

[0031] The division of a block into two or three parts is done by dividing it horizontally or vertically. When dividing a block horizontally into two, divide it in half as shown in Figure 6(b), and then divide the two blocks into two sections. Generate a block. Also, if you divide the block into three horizontally, as shown in Figure 6(c) Divide it into a 1:2:1 ratio to generate three blocks. Meanwhile, divide the block vertically into two. In this case, divide it in half as shown in Figure 6(d) to generate two blocks. Also, vertically When dividing a block into three parts, divide it into a 1:2:1 ratio as shown in Figure 6(e), and the three blocks Generates.

[0032] Referring again to Figure 3, the operation of the block division unit 101 will be explained. First, the tree block Determine whether or not to divide the inside of the box into four halves horizontally and vertically (S1001). ).

[0033] To determine the optimal solution from multiple conditions, including whether or not to divide the block into four parts: There is an existing method called RD optimization (Rate-Distortion Optimization). In optimization, the coding cost is calculated from the code size and coding distortion. Then, under multiple conditions... For each encoded case, calculate the encoding cost and determine the case where the encoding cost is minimized. Select the option to divide the block into four parts. The encoding cost is calculated for both the combined encoding cost and the encoding cost when the block is not divided into four parts. This is done by selecting the case where the value is minimized. The optimal case is determined from multiple conditions. To achieve this, methods other than RD optimization may be used.

[0034] If it is determined that the inside of the tree block should be divided into four parts (S1001:YES), the tree block The inside of the block is divided into four sections (S1002). The process of further dividing the four sections will be described later. As described (Figure 7).

[0035] If it is determined that the inside of the tree block should not be divided into four parts (S1001:NO), the tree block Determine whether to divide the inside of the box into two or three sections (S1003).

[0036] If it is determined that the inside of the tree block should be divided into two or three parts (S1003:YES) Then, determine whether or not to divide in the vertical direction (S1004).

[0037] If the direction of division is determined to be vertical (S1004:YES), the tree block inside Determine whether or not to divide it into two parts (S1005).

[0038] If it is determined that the inside of the tree block should be divided into two (S1005:YES), the tree block The inside of the block is divided vertically into two parts (S1006). On the other hand, the inside of the tree block is divided into three parts. If it is determined that (S1005:NO), the inside of the tree block is divided vertically into three parts. S1007). The process of subdividing blocks that have been divided vertically into two or three sections will be discussed later. As described (Figure 8).

[0039] If the direction of division is determined to be horizontal (S1004:NO), the inside of the tree block Determine whether or not to divide it into two parts (S1008).

[0040] If it is determined that the inside of the tree block should be divided into two (S1008:YES), the tree block The inside of the block is divided horizontally into two parts (S1009). On the other hand, the inside of the tree block is divided into three parts. If it is determined that (S1008:NO), the inside of the tree block is divided into three horizontal sections. S1010). The process of subdividing blocks that have been divided into two or three sections horizontally will be discussed later. As described (Figure 8).

[0041] If it is determined that the inside of the tree block should not be divided into two or three parts (S1003:NO) , terminate the block splitting process without splitting the inside of the tree block (S1011) ).

[0042] Next, let's consider the division of the tree block into four parts, both horizontally and vertically. The processing of each block is explained using the flowchart in Figure 7.

[0043] First, decide whether or not to divide the inside of the block into four equal parts horizontally and vertically. do (S1101).

[0044] If it is decided to divide the inside of the block into four again (S1101:YES), the inside of the block Divide it into four parts again (S1102).

[0045] If it is decided not to divide the inside of the block into four again (S1101:NO), the inside of the block Determine whether to divide it into two or three parts (S1103).

[0046] If it is determined that the inside of the block should be divided into two or three parts (S1103: YES), then division Determine whether the direction is vertical or not (S1104).

[0047] If the direction of division is determined to be vertical (S1104: YES), the inside of the block is divided into two. Determine whether or not to divide (S1105).

[0048] If it is determined that the inside of the block should be divided into two (S1105: YES), the inside of the block will be lowered. Divide it into two perpendicular sections (S1106). On the other hand, if it is decided to divide the inside of the block into three sections ( S1105:NO), divide the inside of the block into three vertical sections (S1107).

[0049] If the direction of division is determined to be horizontal (S1104:NO), the inside of the block is divided into two. To decide whether or not to do it (S1108).

[0050] If it is determined that the inside of the block should be divided into two (S1108: YES), the inside of the block will be filled with water. Divide it into two horizontal sections (S1109). On the other hand, if it is decided to divide the inside of the block into three sections ( S1108:NO), divide the inside of the block into three horizontal sections (S1110).

[0051] If it is determined that the inside of the block will not be divided into two or three parts (S1103:NO), The block partitioning process is terminated without further subdividing the inside of the block (S1111).

[0052] For each of the four divided blocks, the process shown in the flowchart in Figure 7 is executed recursively. The contents of the four divided blocks are encoded in z-scan order.

[0053] Next, when the tree block is divided vertically into two or three sections, each of the divided blocks... The process of checking will be explained using the flowchart in Figure 8.

[0054] When a tree block is divided vertically into two or three sections, each of the divided sections will Then, a decision is made as to whether or not to divide the inside of the block into two or three parts again (S1201).

[0055] If it is determined that the inside of the block should be divided into two or three parts (S1201: YES), Determine whether the direction is vertical or not (S1202).

[0056] If the direction of division is determined to be vertical (S1202:YES), the inside of the block is divided into two. Determine whether or not to divide (S1203).

[0057] If it is determined that the inside of the block should be divided into two (S1203:YES), the inside of the block will be lowered. Divide it into two perpendicular sections (S1204). On the other hand, if it is decided to divide the inside of the block into three sections ( S1203:NO), divide the inside of the block into three vertical sections (S1205).

[0058] If the direction of division is determined to be horizontal (S1202:NO), the inside of the block is divided into two. To decide whether or not to do so (S1206).

[0059] If it is determined that the inside of the block should be divided into two (S1206: YES), the inside of the block will be filled with water. Divide it into two horizontal sections (S1207). On the other hand, if it is decided to divide the inside of the block into three sections ( S1206:NO), divide the inside of the block into three horizontal sections (S1208).

[0060] If it is determined that the inside of the block will not be divided into two or three parts again (S1201:NO), The block partitioning process is terminated without further subdividing the inside of the block (S1209).

[0061] For each divided block when the vertical direction is divided into two or three sections, see the flow in Figure 8. The process shown in the chart is executed recursively. Inside the two-part or three-part block The characters are encoded from left to right.

[0062] Similarly, for each divided block when the horizontal direction is divided into two or three sections, see Figure The process shown in flowchart 8 is executed recursively. The blocks are divided into two or three sections. The inside of the character "ク" is encoded from top to bottom.

[0063] Furthermore, the process of subdividing the divided blocks when a tree block is split was explained. However, the parent block does not have to be a tree block. For example, a tree block (128x1 If you divide a 28-pixel block into four sections, and then further divide each of those four sections (64x64 pixels) Furthermore, the above process is also applied to the subdivision of the subdivided blocks.

[0064] Recursive block partitioning can also be limited by defining the number of partitions. Furthermore, the number of divisions may be predetermined between the encoding device and the decoding device. The encoding device determines the number of divisions and records it in the encoded bitstream, and the decoding device Alternatively, the configuration may use the number of divisions recorded within the encoded bitstream.

[0065] Next, we will explain block division at the edges of the screen. Figure 9 shows an image in tree blocks. This shows the relationship with the picture boundaries when the image is divided into sections. As shown in Figure 9, the size of the image is Since the size of the tree block is not necessarily an integer multiple, the tree block at the edge of the screen is This may include parts inside and outside the picture, separated by the picture boundary. The tree blocks 1001, 1002, and 1003 are shown as locks. In this case, see Figure 10. As shown, the portion of the image that goes beyond the picture boundary is the same as the outermost pixels within the image. Treat them as such. Pixels 1011, 1012, and 1013 are shown as the outermost pixels on the screen.

[0066] Pixel 1011 is located within tree block 1001 and is the upper rightmost pixel on the screen. It is plain. Pixels to the right of pixel 1011, which are outside the picture boundary and outside the screen, are pixel 101 Treat it as the same as 1.

[0067] Pixel 1012 is located within tree block 1002 and is the bottom leftmost pixel on the screen. It is plain. Pixels below pixel 1012, which are outside the picture boundary and outside the screen, are pixel 101 Treat it as the same as 2.

[0068] Pixel 1013 is located within tree block 1003 and is the bottom rightmost pixel on the screen. It is plain. The area to the right, below, and to the lower right of pixel 1013 is outside the picture boundary. The pixel is treated as the same as pixel 1013.

[0069] Then, when dividing a block into two or three parts, the block division is restricted. This allows the blocks at the edges of the screen to be divided into appropriate shapes, improving encoding efficiency. It is possible.

[0070] The block division limitation applies when dividing a block into two or three parts at the edge of the screen. In other words, the process of dividing into two or three parts in Figure 3 (S1004 to S1010) This is replaced by the process described below. Also, the process of dividing into two or three parts (S) in Figure 7 Lines 1104 through S1110 are replaced by the process described below. Furthermore, in Figure 8 The process of dividing into two or three parts (S1202 to S1208) is performed in the process described below. It changes.

[0071] The limitations on block partitioning will be explained using Figure 11. First, all of the block partitioning Regarding combinations, it is decided whether or not to restrict block partitioning (S1301). All possible combinations of division include dividing a block horizontally into two, and dividing the block vertically into two. If the block is divided into two in the direction, if the block is divided into three horizontally, if the block is divided into three vertically These are the four cases when the image is divided. Furthermore, whether or not block division is restricted depends on whether the image crosses the picture boundary. This is determined by whether or not the pixels at the obtained position are divided. For example, dividing the block horizontally into two parts. If dividing the block results in the division of pixels beyond the picture boundary, then divide the block horizontally. Restrict the division into two parts.

[0072] Here, restricting the division of the block horizontally means that the block is horizontal This means prohibiting division in any direction. It also means prohibiting dividing a block vertically. Restricting this means prohibiting the division of blocks vertically. Similarly, restricting the division of a block into two means that the block cannot be divided into two. It means prohibiting doing so. It also means restricting the division of a block into three parts. This means that dividing the block into three parts is prohibited.

[0073] Next, we determine whether or not to restrict block partitioning for all combinations of block partitioning. S1302). If block partitioning is restricted for all combinations (S1302:YES) , do not divide the block (S1314). On the other hand, restrict block division for all combinations. If not (S1302:NO), determine whether or not to restrict horizontal block division. (S1303).

[0074] If horizontal block division is restricted (S1303:YES), proceed to the next process (S 1306). On the other hand, if horizontal block division is not restricted (S1303:NO), Determine whether or not to restrict block division in the perpendicular direction (S1304).

[0075] If you want to restrict vertical block division (S1304:YES), proceed to the next step (S 1310). On the other hand, if vertical block division is not restricted (S1304:NO), Determine whether or not to split the lock vertically (S1305).

[0076] If the direction of block division is determined to be vertical (S1305:YES), the block Determine whether to limit the division to three (S1306). On the other hand, the direction of block division is horizontal. If it is determined to be a direction (S1305: NO), proceed to the next process (S1310).

[0077] If you want to limit the division of the block to three parts (S1306:YES), divide the block vertically into two parts. Divide (S1308). On the other hand, if the division of the block into three parts is not restricted (S1306:NO) Then, a decision is made as to whether or not to divide the block into two (S1307).

[0078] If it is determined that the block should be divided into two (S1307:YES), the block will be divided vertically. Divide into 2 (S1308). On the other hand, if it is decided to divide the block into 3 (S1307: NO), divide the block into three vertical sections (S1309).

[0079] If you want to restrict vertical division (S1304:YES), and the direction of block division If it is determined to be horizontal (S1305:NO), then whether or not to restrict the division of the block into three parts. Determine (S1310).

[0080] If you want to limit the division of the block to three parts (S1310:YES), divide the block horizontally into two parts. Divide (S1312). On the other hand, if the division of the block into three parts is not restricted (S1310: NO) Then, a decision is made as to whether or not to divide the block into two (S1311).

[0081] If it is determined that the block should be divided into two (S1311:YES), the block will be divided horizontally. Divide into 2 (S1312). On the other hand, if it is decided to divide the block into 3 (S1311: NO), divide the block into three horizontal sections (S1313).

[0082] In other words, if pixels beyond the picture boundary are divided by block division, Restrict block division in that direction.

[0083] Let's explain with a concrete example. Currently, at the bottom edge of the screen, the tree block is not divided into four sections. (S1001:NO), the inside of the tree block is divided into 2 or 3 (S1003:Y ES). Figure 12(a) shows that the tree block is separated by the picture boundary into parts inside and outside the screen. This shows how it includes the part. In this case, when the tree block is divided into all combinations, Then, it is decided whether or not to restrict block partitioning (S1301).

[0084] For all possible combinations of tree block divisions, see Figure 12(a) or As shown in Figure 12(d), when the block is divided vertically into two parts as shown in Figure 12(a), Since pixels beyond the picture boundary are split, this block splitting is limited. Furthermore, as shown in Figure 12(b), if the block is divided vertically into three parts, it will exceed the picture boundary. Since the pixels at the specified position are divided, this block division is limited. On the other hand, as shown in Figure 12(c) When the block is divided horizontally into two sections, pixels located beyond the picture boundary are divided. Since this does not restrict the block division, the block division is not restricted. Similarly, as shown in Figure 12(d) When the image is divided into three horizontal sections, pixels beyond the picture boundary are not divided, so It does not restrict block partitioning.

[0085] Therefore, block partitioning is not restricted by all combinations of block partitioning (S1302: NO). Also, there is no restriction on horizontal block division (S1303: NO), and vertical block division is not restricted. Restrict lock partitioning (S1304:YES). However, do not restrict the partitioning of blocks into three parts. (S1310: NO). Therefore, determine whether or not to divide the block into two (S1311). If we now decide to divide the block into two (S1311:YES), then the block Divide it into two horizontal sections (S1312).

[0086] These limitations on block division result in blocks having the correct shape, because they are off-screen. In a block containing pixels, the pixel values ​​of the parts outside the screen are constant. Therefore, in that block Changes in pixel values ​​within the lock screen area are relative to blocks that do not include pixels outside the screen. It is extremely small. Therefore, there is little need to encode fine pixel changes. Consequently, off-screen By grouping pixels into blocks as much as possible, the code size is reduced, improving encoding efficiency. It can be improved.

[0087] This block division limitation also applies to the right edge of the screen. All combinations of tree blocks... The cases where the lock is split are shown in Figures 12(e) to 12(h), respectively. Furthermore, as shown in Figure 12(e), when the block is divided horizontally into two, it crosses the picture boundary. The pixels at the specified position are divided. Also, as shown in Figure 12(f), the block is divided horizontally into three parts. When divided, pixels beyond the picture boundary are divided. Therefore, these blocks Limit the number of blocks. These block division limitations can improve encoding efficiency. ru.

[0088] Next, the operation of the block division unit 202 of the image decoding device 200 will be described. The block division section 202 is processed in the same way as the block division section 101 in the image encoding device 100 described above. The block is divided according to the procedure. The block division unit 101 selects the block division pattern. It selects and outputs the selected block division information. Meanwhile, the block division unit 202 performs encoding The blocks are divided using the block division information decoded from the stream. The limitations on segmentation are the same as those for the image encoding device 100 described above.

[0089] Syntax relating to block division in the first embodiment (composition of the encoded bitstream) The syntax rules are shown in Figure 13. In Figure 13, QT() is the syntax used for dividing a block into four parts. MTT() represents a block, and MTT() represents the syntax for splitting a block into two or three parts. The image encoding device 100 encodes according to this syntax, and the image decoding device 200 Decrypt according to this syntax.

[0090] First, whether or not to divide the block into four parts is represented by QTflag. If it is divided into four parts, QTflag=1. If you do not want to divide it into four parts, set QTflag=0. If you do divide it into four parts (QTflag=1), each of the four divided blocks If it can be further divided into 4 parts (QTvalid=1), the process of dividing into 4 parts is performed recursively. If not present (QTflag=0), whether to split into two or three parts is indicated by MTTflag. When dividing into 3 parts (MTTflag=1), whether or not to divide vertically is represented by vertical_flag, 2 Whether or not to split is indicated by BTflag. vertical_flag=1 for vertical splitting, horizontal splitting. To divide in a direction, set vertical_flag=0. To divide into two, set BTflag=1, then divide into three. If you want to split it, set BTflag=0. For each block that has been split into 2 or 3, split it into 2 again. Alternatively, if it is possible to divide it into three parts (MTTvalid=1), recursively perform the 2-part or 3-part division process.

[0091] Here, the variable QTvalid indicates whether each of the four divided blocks can be further divided into four. Let me explain. QTvalid is defined for each of the four divided blocks. If no pixels on the screen are included, QTvalid=0. In all other cases, QTvalid=1. Yes.

[0092] Furthermore, it is possible to determine whether each block, which has been divided into two or three parts, can be further divided into two or three parts. The variable MTTvalid is explained below. MTTvalid is calculated for each block when the division is into two or three parts. It is defined as follows: If the block to be divided into two or three sections does not contain any pixels in the screen, MTTvalid=0. In all other cases, MTTvalid=1.

[0093] In this embodiment, the vertical_flag is unnecessary because it restricts the direction of block division. Therefore, it is also acceptable to omit the vertical_flag in Figure 13.

[0094] Due to these limitations on block division, it is difficult to divide blocks at the edges of the screen into appropriate shapes. This allows for improved encoding efficiency. Furthermore, it is suitable for image encoding and decoding. Locking and splitting is possible. (Second Embodiment) A second embodiment of the present invention, consisting of an image encoding apparatus and an image decoding apparatus, will be described below. In the second embodiment, when the depth of block division reaches the limit depth, block division This restricts the configuration. The rest of the configuration is the same as in the first embodiment, so its description is omitted.

[0095] Here, the depth of block division will be explained. In the first embodiment, block After dividing it into two or three parts, recursively apply the following to each of the two or three divided blocks: The process of dividing into two or three parts was explained. In this process, the first division into two parts or The 3-part division process is defined as having a depth of 0. Also, the division by the first 2-part or 3-part division process The second 2-part or 3-part division process for each block is defined as depth 1, and the second 2 A third 2-part or 3-part division for each block that has been divided by the splitting or 3-part division process. The division process is defined as having a depth of 2, and the depth is defined similarly for subsequent divisions. Furthermore, block division is restricted. A predetermined depth is set and defined as the limiting depth.

[0096] The block division limitation applies when dividing a block into two or three parts at the edge of the screen. In other words, the process of dividing into two or three parts in Figure 3 (S1004 to S1010) This is replaced by the process described below. Also, the process of dividing into two or three parts (S) in Figure 7 Lines 1104 through S1110 are replaced by the process described below. Furthermore, in Figure 8 The process of dividing into two or three parts (S1202 to S1208) is performed in the process described below. It changes.

[0097] The limitations on block division will be explained using Figure 14. First, the block is divided into two parts. When dividing into three parts, it is determined whether the depth of the block division has reached the limit depth (S1 401).

[0098] If the limit depth has not been reached (S1401:NO), it is decided not to restrict block partitioning. (S1402). On the other hand, if the limit depth has been reached (S1401: YES), block For all combinations of block partitions, it is decided whether or not to restrict the block partition (S13 01). Since S1301 is the same as in the first embodiment, its description is omitted.

[0099] Next, we determine whether or not to restrict block partitioning for all combinations of block partitioning. S1302). The process from S1302 onward is the same as in the first embodiment, so the explanation is omitted. do.

[0100] In other words, block division divides pixels at positions beyond the picture boundary, and blocks If the block partitioning depth reaches the limit depth, the block partitioning will be restricted.

[0101] Let's explain with a concrete example. Currently, at the bottom edge of the screen, the tree block is not divided into four sections. (S1001:NO), divide the inside of the tree block into two or three parts (S1003:Y ES). Also, the limit depth is set to 1. Figure 12(a) shows the tree block as picture boundary. This shows the view including the parts inside and outside the screen, separated by a divider. This tree block can be divided into two or three sections. When dividing, pixels located beyond the picture boundary are divided by vertical division. However, the depth is 0 and has not reached the limit depth of 1 (S1401:NO). Therefore, it is decided not to restrict block partitioning (S1402). Thus, all blocks No restrictions on division (S1302: NO), no restrictions on horizontal block division (S1303 :NO), does not restrict vertical block division (S1304:NO). And, block Determine whether or not to divide the block vertically (S1305). Now, divide the block vertically If it is decided to divide (S1305:YES), the division of the block into three parts is not restricted. (S1306: NO) Then, we decide whether or not to divide the block into two (S1307). If it is decided to divide the block into two (S1307:YES), then the block will be divided vertically. Divide it into two parts in the direction (S1308). This is shown in Figure 12(a).

[0102] Next, divide each of the two vertically divided blocks into two or three parts. The depth is divided. At this point, the depth is 1, and the limit depth of 1 has been reached (S1401:YES). Therefore, for all combinations of block partitioning, we can determine whether or not to restrict the block partitioning. Decide (S1301).

[0103] For the case where the block is divided into all possible combinations, see Figures 15(a) to 1 As shown in 5(d). When the block is divided vertically into two parts as shown in Figure 15(a), the picture Since pixels beyond the boundary are divided, this block division is limited. Similarly, Figure As shown in 15(b), when the block is divided vertically into three parts, the position beyond the picture boundary Since the pixels are divided, this block division is limited. On the other hand, as shown in Figure 15(c) When a block is divided horizontally into two sections, pixels beyond the picture boundary are not divided. Therefore, this block division is not restricted. Similarly, the blocks can be divided into water as shown in Figure 15(d). When divided into three sections horizontally, pixels beyond the picture boundary are not divided, so this block Do not restrict splitting.

[0104] Therefore, block partitioning is not restricted by all combinations of block partitioning (S1302: NO). Also, there is no restriction on horizontal block division (S1303: NO), and vertical block division is not restricted. Lock partitioning is restricted (S1304:YES). However, the 3-part partitioning of a block is not restricted. (S1310: NO). Therefore, determine whether or not to divide the block into two (S1311). If we now decide to divide the block into two (S1311:YES), then the block Divide it into two horizontal sections (S1312).

[0105] These limitations on block division result in blocks of appropriate size and shape. Therefore, in blocks that include pixels outside the screen, the pixel values ​​of the parts outside the screen are constant. Therefore, changes in the pixel values ​​of the portion of that block within the screen will affect the block that does not include pixels outside the screen. In contrast, it is relatively small. Therefore, there is little need to encode fine pixel changes. Consequently By grouping pixels outside the screen into blocks as much as possible, the code size is reduced. This can improve the efficiency of the numbering process.

[0106] This block division limitation also applies to the right edge of the screen. The tree block can be divided into two. When dividing into 3 parts, the depth is 0 and does not reach the limit depth of 1, so the block division is not limited to 2 Divide or 3-part. When further dividing a divided block, the depth is 1 and there is a limit to the depth. Since it has reached 1, we determine whether or not to restrict block division. Then, the block is water When dividing the image into two horizontally, and when dividing the block into three horizontally, the image boundary will not cross the picture boundary. The pixels at the obtained positions are divided. Therefore, these block divisions are limited. By limiting the block division, encoding efficiency can be improved.

[0107] In this embodiment, the block division depth is defined for 2 divisions or 3 divisions. This may also be defined for a 4-part division. In this embodiment, the depth of the block division This restricts block division. This prevents blocks from exceeding the picture boundaries they contain. The position can be restricted by the number or proportion of pixels. In other words, these values ​​can be predetermined values. Configure it to limit the block partition if it is greater than the specified value. In addition, these values The values ​​may also be different for each block division depth. This allows for the number of pixels outside the screen and Blocks with a small proportion of pixels will be split, while blocks with a large number or proportion of pixels outside the screen will not be split. To do so. The limiting depth of block division and pixels located beyond the picture boundary contained within the block. The values ​​related to the block division limitations, such as the number and proportion, are determined by the encoding device in the encoded bitstory. The system is configured to record the data within the bitstream, and the decoding device uses the values ​​recorded within the encoded bitstream. That's fine.

[0108] In the first embodiment, block division is restricted regardless of the depth of block division. On the other hand, In this embodiment, block division is limited according to the depth of block division. This allows, Blocks with a small proportion of off-screen pixels are divided, while blocks with a large proportion of off-screen pixels are divided. To avoid splitting, divide the blocks at the edges of the screen into appropriate sizes and shapes. This allows for improved encoding efficiency. Furthermore, it is suitable for image encoding and decoding. Block division is possible. (Third embodiment) A third embodiment of the present invention, comprising an image encoding apparatus and an image decoding apparatus, will be described below. In the third embodiment, the blocks are divided according to the number of pixels located beyond the picture boundary. This controls the following. The other configurations are the same as in the first embodiment, so their description is omitted.

[0109] Block division control is applied when dividing a block into two or three parts at the edge of the screen. In other words, the process of dividing into two or three parts in Figure 3 (S1004 to S1010) This is replaced by the process described below. Also, the process of dividing into two or three parts (S) in Figure 7 Lines 1104 through S1110 are replaced by the process described below. Furthermore, in Figure 8 The process of dividing into two or three parts (S1202 to S1208) is performed in the process described below. It changes.

[0110] The control of block partitioning will be explained using Figure 16. First, all of the block partitioning For each combination, count the number of pixels at positions that cross the picture boundary (S1601). The total number of block divisions is the number of combinations when a block is divided horizontally into two sections. If the block is divided vertically into two parts, if the block is divided horizontally into three parts, the block is divided vertically These are four parts when the direction is divided into three sections.

[0111] Then, for both the case where the block is divided into two and the case where the block is divided into three, The division direction that maximizes the number of pixels at positions beyond the picture boundary is determined (S1602).

[0112] Next, we decide whether or not to divide the block into two (S1603).

[0113] If it is decided to divide it into two (S1603:YES), the portion determined in S1602 Divide into two in the direction of division (S1604). On the other hand, if it is decided to divide into three (S1603:N O) Divide into three sections in the division direction determined in S1602 (S1605).

[0114] In other words, the blocks are designed so that the largest number of pixels located beyond the picture boundaries are included within the block. It controls the direction of lock splitting.

[0115] Here, let's explain with a specific example. Currently, the image is 1920x1080 pixels, and the tree block is The resolution will be 128x128 pixels. Also, the tree block will not be divided into four parts (S1101:NO). The inside of the block is divided into two or three parts (S1103:YES). Then, Figure 17(a As shown in the image, there are 72 pixels vertically beyond the picture boundary at the bottom edge of the screen. At this time, when the tree block is divided into all possible combinations, the number of blocks in each block The number of pixels beyond the Kucha boundary is counted (S1601).

[0116] For each case where the tree block is divided into all possible combinations, see Figure 17(a) or As shown in Figure 17(d). When the block is divided vertically into two parts as shown in Figure 17(a), The left (500) and right (501) sides of the divided block each have 4608 pixels. On the other hand, if the block is divided horizontally into two parts as shown in Figure 17(b), the divided block The upper part (510) has 1024 pixels, and the lower part (511) has 8192 pixels. In other words, pict The maximum number of pixels beyond the boundary occurs at the lower 8192 when divided horizontally. These are pixels. Therefore, the division direction that maximizes the number of pixels is the horizontal direction (S1602).

[0117] Similarly, if the block is divided vertically into three parts as shown in Figure 17(c), then the divided parts The left side (520) and right side (522) of the block each have 2304 pixels, and the center (52 1) has 4608 pixels. On the other hand, as shown in Figure 17(d), the block is divided into three horizontal sections. If so, the upper part of the divided block would have 0 pixels, the middle (531) would have 5120 pixels, and the lower part would have 0 pixels. (532) has 4096 pixels. In other words, the number of pixels at the position beyond the picture boundary is the maximum. This refers to the central 5120 pixels when divided horizontally. Therefore, the maximum number of pixels is The dividing direction is horizontal (S1602).

[0118] Finally, it is decided whether or not to divide the block into two (S1603). If it is decided to divide it into two, If this occurs (S1603:YES), the block is divided into two horizontally, which is the determined division direction. (S1604). On the other hand, if it is decided to divide it into three parts (S1603: NO), then it is decided. The block is divided into three sections horizontally, which is the direction of division (S1605).

[0119] In this embodiment, when dividing a block into two or three parts, the part that crosses the picture boundary The direction of block division is controlled so that the most pixels of a given location are included within the block. In a block that includes pixels outside the screen, the pixel values ​​of the parts outside the screen are constant, so This is because there is little need to encode changes in pixels. Therefore, such pixels can be created. By combining them into a single block, the amount of code is reduced, improving encoding efficiency. It can be made to happen.

[0120] Furthermore, not only the direction of block division but also the number of divisions may be determined. As such, the maximum number of pixels beyond the picture boundary is 8192 pixels in the case of a 2-way split. In the case of a 3-way division, it is 5120 pixels. In other words, pixels located beyond the picture boundary are... Since the most common case within a lock is a 2-part division, we divide the block into two.

[0121] This means that, compared to a two-way division, a three-way division divides the block closer to the edge of the block. Therefore, a 3-way division tends to result in fewer pixels beyond the picture boundary compared to a 2-way division. Yes. Therefore, at the edges of the screen, the direction of division should be determined so that the block is not divided into three parts, but into two parts. This makes it possible to reduce the processing involved in dividing a block into three parts, and the block division process It can be made faster.

[0122] Block division control is applied when dividing a block into two or three parts at the edge of the screen. In other words, the process of dividing into two or three parts in Figure 3 (S1004 to S1010) This is replaced by the process described below. Also, the process of dividing into two or three parts (S) in Figure 7 Lines 1104 through S1110 are replaced by the process described below. Furthermore, in Figure 8 The process of dividing into two or three parts (S1202 to S1208) is performed in the process described below. It changes.

[0123] The control of block partitioning will be explained using Figure 18. First, all of the block partitioning For each combination, the number of pixels beyond the picture boundary is counted (S1651). The total number of block divisions is the case where the block is divided horizontally into two, and the case where the block is divided horizontally into two. These are the two parts obtained by dividing the block vertically into two sections.

[0124] Next, let's consider the case where the block is divided into two, and the number of pixels at the position beyond the picture boundary is the maximum. The division direction is determined (S1652). Then, the blocks are divided in the determined division direction. Divide (S1654).

[0125] The syntax for block partitioning in this embodiment is shown in Figure 13 as vertical_flag This is omitted. In this embodiment, this controls the direction of block division, and ve This is because rtical_flag becomes unnecessary. Furthermore, in this embodiment, the number of block divisions You may decide to do so, or you may choose not to divide the block into three parts. In that case, BTflag is Since this is essential, it may be omitted. The block division of this embodiment The input can be the same as in Figure 13.

[0126] In the case of 1920x1080 pixels, pixels located beyond the picture boundary at the right edge of the screen. It does not exist. However, if the width of the image is not an integer multiple of the size of the tree block, Since there are pixels located beyond the Kucha boundary, block division is controlled in the same way as described above.

[0127] In this embodiment, the number of pixels beyond the picture boundary determines the block division method. Direction is controlled. This is done by blocking the proportion of pixels that are located beyond the picture boundary. You can also control the direction of the division.

[0128] This control over block division allows the blocks at the edges of the screen to be divided into appropriate sizes and shapes. This allows for improved encoding efficiency. Furthermore, image encoding and decoding... It allows for block partitioning suitable for this purpose. (Fourth embodiment) A fourth embodiment of the present invention, specifically an image encoding apparatus and an image decoding apparatus, will be described below. In the fourth embodiment, a case in which the picture boundary differs from that of the previously described embodiment will be explained. The rest of the configuration is the same as in the first embodiment, so we will omit the explanation.

[0129] Now, let's consider a pattern where the picture boundary is different from that in Figure 12. However, the picture boundary is different. Even if this happens, the decision on whether or not to restrict block division remains the same as in the case of Figure 12. Excluding the n. For example, suppose in Figure 12(a), the picture boundary is located slightly above. Even in that case, pixels beyond the picture boundary are divided, so block division is controlled. It is limited. In other words, Figure 12(a) shows that even if the picture boundary changes vertically, the block portion The decision to limit the percentage remains unchanged. This is also true for Figure 12(b). Similarly, Figures 12(e) and 12(f) show that even when the picture boundary changes in the left-right direction, the block division remains the same. The judgment result of restricting the split remains unchanged.

[0130] Ultimately, the patterns where the picture boundaries are different from those in FIG. 12 are FIGS. 19(a) to 19(d). The picture boundary in FIG. 19(a) is located above FIG. 12(c). Similarly, the picture boundary in FIG. 19(b) is located above FIG. 12(d). Likewise, the picture boundaries in FIGS. 19(c) and FIG. 19(d) are located to the left of FIGS. 12(g) and 12(h).

[0131] Both FIGS. 19(a) and 19(b) are horizontal splits. However, in the case of a 2 - split, the pixels at positions beyond the picture boundary are not split, while in the case of a 3 - split, the pixels at positions beyond the picture boundary are split. That is, only in the case of a 3 - split, the block split is restricted.

[0132] Both FIGS. 19(c) and 19(d) are vertical splits. However, in the case of a 2 - split, the pixels at positions beyond the picture boundary are not split, while in the case of a 3 - split, the pixels at positions beyond the picture boundary are split. That is, only in the case of a 3 - split, the block split is restricted.

[0133] That is, the determination of restricting the block split differs between a 2 - split and a 3 - split. This is because, compared to a 2 - split, in a 3 - split, the block is split at a position closer to the edge of the block. In a 3 - split, there is a tendency for the pixels at positions beyond the picture boundary to be split more than in a 2 - split. Therefore, at the edge of the screen, it is possible to determine whether to restrict the direction to a 2 - split instead of a 3 - split of the block. Thereby, the processing related to the 3 - split of the block can be reduced, and the processing of the block split can be speeded up.

[0134] The block division restriction is applied when dividing a block into two or three parts at the screen edge. That is, the division process (from S1004 to S1010) of dividing into two or three parts in FIG. 3 is replaced by the process described below. Also, the division process (from S1104 to S1110) of dividing into two or three parts in FIG. 7 is replaced by the process described below. Furthermore, the division process (from S1202 to S1208) of dividing into two or three parts in FIG. 8 is replaced by the process described below. Regarding the block division restriction, it will be described using FIG. 20. FIG. 20 is in a form where some of the processes in FIG. 11 are omitted. Therefore, the same step numbers as in FIG. 11 may be used for the description and some may be omitted.

[0135] First, for all combinations of block division, it is determined whether to restrict the block division (S1901). All combinations of block division refer to two cases: when the block is divided into two parts horizontally and when the block is divided into two parts vertically. Also, whether to restrict the block division is determined by whether the pixels at positions beyond the picture boundary are divided. For example, if when the block is divided into two parts horizontally, the pixels at positions beyond the picture boundary are divided, then dividing the block into two parts horizontally is restricted. The processes after S1302 are the same as in FIG. 11 except that the process for dividing into three parts is eliminated, so the description is omitted.

[0136]

[0137]

[0138]

[0139] 901). Now, if we divide the block horizontally into two, the position beyond the picture boundary Since the pixels are not divided, horizontal division is not restricted. Also, the blocks are divided vertically. When divided into two, pixels beyond the picture boundary are divided, resulting in a vertical division. The division is restricted. Therefore, block division is restricted for all combinations of block divisions. No (S1302: NO). And, without restricting horizontal block division (S1303 :NO), because it restricts vertical block division (S1304:YES). In other words, The lock is divided into two horizontal sections (S1312).

[0138] Now, consider a pattern where the picture boundary is further up or to the left compared to Figure 19. In this case, in every pattern, pixels located beyond the picture boundary are split. Furthermore, the division of blocks into two or three parts may always be restricted at the edges of the screen.

[0139] This control over block division allows the blocks at the edges of the screen to be divided into appropriate sizes and shapes. This allows for improved encoding efficiency. Furthermore, image encoding and decoding... It allows for block partitioning suitable for this purpose. (Fifth embodiment) The image coding apparatus and image decoding apparatus according to the fifth embodiment of the present invention will be described below. In the fifth embodiment, the limitation of block division at the lower right edge of the screen will be described. The rest of the configuration is the same as in the first embodiment, so we will omit the explanation.

[0140] As shown in Figure 9, the tree block at the edge of the screen is separated by the picture boundary within the screen. This may include parts outside the screen. In particular, tree block 1001 and the image at the right edge of the screen. Compared with the off-screen part included in the tree block 1002 at the lower end of the surface, the off-screen part included in the tree block 1003 at the lower right end of the surface tends to be larger. Therefore, due to the block splitting limitation, there is a large room for improving the coding efficiency. When the tree block is split, the block splitting is restricted for the split blocks included in the tree block. When the tree block is split, the block splitting is restricted for the split blocks included in the tree block. In the split blocks included in the tree block, the block splitting is restricted.

[0141] The restriction of block splitting at the lower right end of the surface will be described using FIG. 20. The block at the lower right end of the surface corresponds to the tree block 1003 in FIG. 9.

[0142] First, for all combinations of block splitting, it is determined whether to restrict the block splitting (S1901). All combinations of block splitting refer to two cases: when the block is split into two in the horizontal direction and when the block is split into two in the vertical direction. Also, whether to restrict the block splitting is determined by whether the pixels at positions exceeding the picture boundary are split. For example, when the block is split into two in the horizontal direction, if the pixels at positions exceeding the picture boundary are split, splitting the block into two in the horizontal direction is restricted.

[0143] Now, since it is the block at the lower right end of the surface, for all combinations of block splitting, the pixels at positions exceeding the picture boundary are split. If block splitting is restricted for all combinations (S1302: YES), the block is not split (S1314).

[0144] Here, a specific example will be described using FIG. 21. FIGS. 21(a) and 21(b) show all combinations of block splitting in the tree block at the lower right end of the surface. Now, in both Figure 21(a) and Figure 21(b), pixels located beyond the picture boundary are divided. This is done. If block partitioning is restricted for all combinations (S1302:YES), Do not split the block (S1314).

[0145] Figure 21(c) shows the tree block divided into four sections along the same picture boundary as Figure 21(a). This is an example. Each of the four divided blocks is scanned in z-scan order: blocks 601, 602, 6 Let's assume 03,604. We will now explain each step of block partitioning with respect to Figure 21(c).

[0146] Figure 21(c) is divided into blocks by the following steps. First, the process in Figure 3 Next, the image is divided into tree blocks (S1000), and the inside of the tree block is divided into four sections. The system determines this (S1001: YES) and divides the tree block into four parts (S1002). 4 divisions For each of the blocks 601, 602, 603, and 604, the process shown in Figure 7 is performed. .

[0147] In Figure 7, it is determined that the interior of the divided block is not to be divided into four again (S1101: NO), determine that the inside of the block is divided into two or three parts (S1103: YES). Figure The process of dividing into two or three parts in step 7 (S1104 to S1110) corresponds to the process in Figure 20. It will be replaced.

[0148] In Figure 20, block partitioning is restricted for all combinations of block partitioning. Determine whether or not (S1901). Since it is a block at the bottom right corner of the screen, For all combinations of lock splitting, pixels located beyond the picture boundary are split. If block partitioning is restricted for all combinations (S1302:YES), the block No division is performed (S1314). This completes the block division shown in Figure 21(c).

[0149] Figure 21(d) shows that the tree block is divided into four sections by a different picture boundary than in Figure 21(c). This is an example. This example shows a complex block division that combines the embodiments described above. Of the four divided blocks, blocks 602 and 603 are further divided into blocks. It is done. After dividing the tree block into four (S1002), each of the four divided blocks is 6 Each step of block partitioning is described for each of the following sections: 01, 602, 603, and 604.

[0150] Block 601 performs the process shown in Figure 7. In Figure 7, the interior of the divided block is reprocessed. It was determined that it should not be divided into four parts (S1101:NO), and the inside of the block was to be divided into two or three parts. It is determined to be (S1103: NO). Block 601 is not subdivided internally. The splitting process is terminated (S1111).

[0151] Block 602 performs the process shown in Figure 7. In Figure 7, the interior of the divided block is reprocessed. If it is determined that it is not divided into 4 parts (S1101:NO), the inside of the block is divided into 2 or 3 parts. The determination is made (S1103:YES). Processing of 2 division or 3 division in Figure 7 (S110 Steps 4 through S1110 are replaced by the process shown in Figure 20.

[0152] In Figure 20, block partitioning is restricted for all combinations of block partitioning. Determine whether or not (S1901). Now, if the block is divided into two horizontally, Since pixels beyond the boundary are divided, horizontal division is limited. When the lock is divided vertically into two parts, pixels beyond the picture boundary are not divided. Therefore, vertical division is not restricted. Thus, all combinations of block divisions are possible. And, block division is not restricted (S1302:NO). And, horizontal block division Restrict (S1303:YES). In other words, the block is divided vertically into two parts (S13 08). In the end, block 602 is vertically divided by division 612.

[0153] Block 603 performs the process shown in Figure 7. In Figure 7, the interior of the divided block is reprocessed. If it is determined that it is not divided into 4 parts (S1101:NO), the inside of the block is divided into 2 or 3 parts. The determination is made (S1103:YES). Processing of 2 division or 3 division in Figure 7 (S110 Steps 4 through S1110 are replaced by the process shown in Figure 18.

[0154] In Figure 18, for all combinations of block divisions, the portion that crosses the picture boundary is Count the number of pixels in the block (S1651). The total number of block division combinations is... There are two cases: one where the block is divided horizontally into two parts, and another where the block is divided vertically into two parts. Now, in block 603, as shown in Figures 17(a) and 17(b), pict The number of pixels beyond the boundary is maximized when the map is divided horizontally. Therefore, When a block is divided into two, the portion with the maximum number of pixels beyond the picture boundary is... The division direction is determined to be horizontal (S1652). Then, the blocks are divided in the determined division direction. It is divided into two parts (S1654). In the end, block 603 is horizontally divided by division 623.

[0155] Block 604 performs the process shown in Figure 7. In Figure 7, the interior of the divided block is reprocessed. If it is determined that it is not divided into 4 parts (S1101:NO), the inside of the block is divided into 2 or 3 parts. The determination is made (S1103:YES). Processing of 2 division or 3 division in Figure 7 (S110 Steps 4 through S1110 are replaced by the process shown in Figure 20.

[0156] In Figure 20, block partitioning is restricted for all combinations of block partitioning. Determine whether or not (S1901). Now, divide the block horizontally into two by division 624. When splitting, pixels beyond the picture boundary are split, so horizontal splitting is Restrict. Also, if the block is divided vertically into two by division 614, pict Since pixels beyond the boundary are divided, vertical division is restricted. For all combinations of the split, pixels located beyond the picture boundary are split. If block partitioning is restricted for all combinations (S1302:YES), the block is divided Do not divide (S1314).

[0157] This control over block division allows the blocks at the edges of the screen to be divided into appropriate sizes and shapes. This allows for improved encoding efficiency. Furthermore, image encoding and decoding... It allows for block partitioning suitable for this purpose.

[0158] In all the embodiments described above, the object controlling the block division is the picture boundary. This is defined as a position beyond a certain point. This means that an arbitrary boundary is defined, and the position beyond that point is blocked. You may also control the pixel division. Furthermore, you can use arbitrary boundaries, such as pixels that are heavier than surrounding pixels. We define pixels with high importance and control block division for positions beyond their boundaries. It is also acceptable to do so. Furthermore, the position beyond any arbitrary boundary is not limited to the bottom or right edge of the screen, but also to the top edge of the screen. It may be at the left edge, or it doesn't have to be at the edge. In that case, even if it's not at the edge of the screen, the block can be placed there. It can be divided into appropriate sizes and shapes, thereby improving encoding efficiency.

[0159] All of the embodiments described above may be combined in any way.

[0160] In all the embodiments described above, the encoded bitst output by the image encoding device Reem is specified to be decodeable according to the encoding method used in the embodiment. It has a data format. The encoded bitstream is HDD, SSD, flat Provided by recording on a recording medium that can be read by a computer, such as a flash memory or optical disc. You may do so, or you may provide it from the server via a wired or wireless network. Therefore, the image decoding device corresponding to this image encoding device, regardless of the means of provision, this specific data It can decode encoded bitstreams of data formats.

[0161] In order to exchange encoded bitstreams between the image encoding device and the image decoding device, When a wired or wireless network is used, the data format should be appropriate for the transmission method of the communication channel. The encoded bitstream may be converted and transmitted. In that case, the image encoding device will output Convert the encoded bitstream into encoded data in a data format suitable for the transmission method of the communication channel. A transmitting device that converts and transmits data to the network, and a device that receives encoded data from the network and encodes it. A receiving device is provided that restores the data into a numbered bitstream and supplies it to an image decoding device.

[0162] The transmitting device has a memory that buffers the encoded bitstream output by the image encoding device. The packet processing unit packets the encoded bitstream, and the network It includes a transmitting unit that transmits packetized encoded data. The receiving device connects the network A receiving unit that receives encoded data that has been packetized via a receiving unit, and a receiving unit that receives the encoded data It uses memory to process the encoded data and generates an encoded bitstream by packet processing. , including a packet processing unit that provides packets to an image decoding device.

[0163] In order to exchange encoded bitstreams between the image encoding device and the image decoding device, When a wired or wireless network is used, in addition to the transmitting device and receiving device, Even if a relay device is provided that receives encoded data transmitted by a transmitting device and supplies it to a receiving device, Good. The relay device is a receiving unit that receives packetized encoded data transmitted by the transmitting device. And, a memory for buffering the received encoded data, and the packetized encoded data and It includes a transmitting unit that transmits to the network. Furthermore, the relay device includes a packetized encoded data A receiving packet processing unit that processes the data into packets to generate an encoded bitstream, and a coding A recording medium for storing encoded bitstreams and a transmission medium for packetizing encoded bitstreams. It may include a signal packet processing unit.

[0164] Furthermore, by adding a display unit to the configuration that displays the image decoded by the image decoding device, the display It can also be used as a device. In that case, the display unit is generated by the decoded image signal superimposition unit 205, and The decoded image signal stored in the image memory 206 is read out and displayed on the screen.

[0165] Furthermore, by adding an imaging unit to the configuration and inputting the captured image into an image encoding device, It can also be used as a device. In that case, the imaging unit inputs the captured image signal into the block division unit 101. To exert force.

[0166] Figure 22 shows an example of the hardware configuration of the encoding / decoding device of the present invention. The encoding / decoding device This includes the configuration of an image encoding device and an image decoding device according to embodiments of the present invention. The encoding / decoding device 9000 consists of a CPU 9001, a codec IC 9002, and an I / O interface. -Face 9003, Memory 9004, Optical Disc Drive 9005, Network It has an interface 9006 and a video interface 9009, and each part is connected to bus 9010 It is connected by.

[0167] The image encoding unit 9007 and the image decoding unit 9008 are typically connected to the codec IC 9002. It is implemented as follows. The image coding process of the image coding device according to an embodiment of the present invention is an image code This is performed by the numbering unit 9007, and is an image decoding in an image decoding device according to an embodiment of the present invention. The processing is performed by the image encoding unit 9007. The I / O interface 9003 is For example, this is achieved via a USB interface, and external keyboard 9104, mouse 9 Connects to 105, etc. CPU9001 receives input via I / O interface 9003. Based on the user's input, the encoding / decoding device 9 performs the action desired by the user. Control 000. User operation via keyboard 9104, mouse 9105, etc. This includes selecting whether to perform encoding or decoding, setting the encoding quality, and encoding stream. This includes input / output destinations for software, image input / output destinations, etc.

[0168] When the user wishes to play back images recorded on the disk recording medium 9100 The optical disc drive 9005 encodes video from the inserted disc recording medium 9100. The readout stream is read, and the readout encoded stream is coded via bus 9010. The image is sent to the image decoding unit 9008 of IC9002. The image decoding unit 9008 processes the input encoded video. The image decoding process in the image decoding apparatus according to the embodiment of the present invention is applied to the stream. The process is executed, and the decoded image is sent to the external monitor 9103 via the video interface 9009. Send. Furthermore, the encoding / decoding device 9000 has a network interface 9006. via network 9101, it connects to external distribution servers 9106 and mobile terminals 9107. It is possible to continue using the image recorded on the disk recording medium 9100. If you wish to play back images recorded on the receiver 9106 or mobile terminal 9107, The network interface 9006 receives data from the input disk recording medium 9100. Instead of reading the encoded bitstream, the encoded stream is read from network 9101. Obtain it. Also, if the user wishes to play back the image recorded in memory 9004. In this case, the encoded stream recorded in memory 9004 is used in the embodiment of the present invention. The image decoding process is performed in the image decoding device.

[0169] The image captured by the user with the external camera 9102 is encoded and recorded in memory 9004. If operation is desired, the video interface 9009 receives images from camera 9102. The image is then sent via bus 9010 to the image encoding unit 9007 of codec IC 9002. The image encoding unit 9007 processes the image input via the video interface 9009. The image encoding process is performed in the image encoding device according to an embodiment of the present invention, and the encoded bit Create a bitstream. Then, encode the bitstream and send it via bus 9010. Send to memory 9004. The user changes to memory 9004 and writes to disk storage medium 9100. If you wish to record the stream, the optical disc drive 9005 is inserted The encoded stream is written to the inserted disk recording medium 9100.

[0170] Hardware configurations that have an image encoding device but no image decoding device, or hardware configurations that have an image decoding device It is also possible to realize a hardware configuration that does not include an image encoding device. The hardware configuration is such that, for example, the codec IC9002 is the image encoding unit 9007, or This is achieved by replacing each of the image decoding units 9008 with their respective components.

[0171] The above encoding and decoding processes are performed using hardware-based transmission, storage, and receiving devices. It would certainly be fine to implement it as such, but ROM (Read-Only Memory) and flash Firmware stored in memory, etc., and software on computers, etc. It's okay to make it happen. The firmware program, the software program... It may be recorded on a recording medium that can be read by a computer, etc., and provided via a wired or wireless network. It can be provided from the server through the work, or data from terrestrial or satellite digital broadcasting. It can also be offered as a broadcast.

[0172] The present invention has been described above based on embodiments. The embodiments are illustrative and their respective components The combination of constituent elements and each processing process can be varied in many ways, and such variations Those skilled in the art will understand that the examples also fall within the scope of the present invention. [Explanation of Symbols]

[0173] 100 Image encoding device, 101 Block division unit, 102 Predictive image generation unit, 103 Residual signal generation unit, 104 Orthogonal transformation / quantization unit, 105 Encoding unit, 10 6 Inverse quantization / inverse orthogonal transform section, 107 Decoded image signal superposition section, 108 Decoded image memo R, 200 Image decoding device, 201 Decoding unit, 202 Block division unit, 2 03 Inverse quantization / inverse orthogonal transformation unit, 204 Predictive image generation unit, 205 Decoded image signal weighting Tatami mat section, 206 decoded image memory.

Claims

1. An image decoding device that decodes images in divided block units, A decoding unit that decodes block division information including a flag indicating whether to divide the first block into two or three parts, The system includes a block division unit that recursively divides the image into rectangles of a predetermined size based on the block division information to generate the first block, The aforementioned block division section is A four-part division unit in which the first block in the recursive partition is divided in half horizontally and vertically to form four blocks, and four blocks are generated as the second block, The recursive partitioning includes a 2-3 division section that divides the first block in the horizontal or vertical direction into two or three parts to generate two or three blocks as a third block, The aforementioned 2-3 division section prohibits dividing the first block with a horizontal division line and permits dividing it with a vertical division line if, when the first block is divided into two by a horizontal division line, the divided third block extends beyond the right side of the picture boundary, and when the first block is divided into two by a vertical division line, the divided third block does not extend below the picture boundary. The aforementioned 2-3 division section prohibits dividing the first block with a vertical division line and permits dividing it with a horizontal division line if, when the first block is divided with a vertical division line, the divided third block extends below the picture boundary, and when the first block is divided with a horizontal division line, the divided third block does not extend to the right of the picture boundary. The aforementioned 2-3 division section prohibits dividing the first block into three parts if the divided third block extends beyond the picture boundary, assuming the first block is divided into three parts by a horizontal or vertical dividing line. The aforementioned 2-3 division section prohibits dividing the 128x128 pixel tree block obtained by dividing the image into two using horizontal and vertical division lines if the tree block extends beyond the lower right side of the picture boundary. The decoding unit, under the condition that the first block is not to be divided into three parts by horizontal or vertical dividing lines, does not decode the flag. An image decoding device characterized by the following features.

2. An image decoding method that decodes an image in divided block units, A decoding step of decoding block partitioning information which includes a flag indicating whether to partition the first block into two or three parts, The method includes a block division step of recursively dividing the image into rectangles of a predetermined size based on the block division information to generate the first block, The aforementioned block partitioning step is: A four-part division step in which the first block in the recursive partition is divided in half horizontally and vertically to form four blocks, and four blocks are generated as the second block, The recursive partitioning includes a two- or three-part division step in which the first block is divided horizontally or vertically into two or three parts to generate two or three blocks as a third block, The 2-3 division step prohibits dividing the first block with a horizontal dividing line and permits dividing it with a vertical dividing line if, when the first block is divided with a horizontal dividing line, the divided third block extends beyond the right side of the picture boundary, and when the first block is divided with a vertical dividing line, the divided third block does not extend below the picture boundary. The 2-3 division step prohibits dividing the first block with a vertical dividing line and permits dividing it with a horizontal dividing line if, when the first block is divided with a vertical dividing line, the divided third block extends below the picture boundary, and when the first block is divided with a horizontal dividing line, the divided third block does not extend to the right of the picture boundary. The aforementioned 2-3 division step prohibits dividing the first block into three parts if, after dividing the first block into three parts by a horizontal or vertical dividing line, the resulting third block extends beyond the picture boundary. The above 2-3 division step prohibits dividing the tree block into two using horizontal and vertical division lines if the 128x128 pixel tree block obtained by dividing the image extends beyond the lower right side of the picture boundary. The decoding step does not decode the flag under the condition that the first block is not divided into three parts by horizontal or vertical dividing lines. An image decoding method characterized by the following:

3. An image decoding program that decodes an image in divided block units, A decoding step of decoding block partitioning information which includes a flag indicating whether to partition the first block into two or three parts, The computer is instructed to perform a block division step, which involves recursively dividing the image into rectangles of a predetermined size based on the block division information to generate the first block, The aforementioned block partitioning step is: A four-part division step in which the first block in the recursive partition is divided in half horizontally and vertically to form four blocks, and four blocks are generated as the second block, The recursive partitioning includes a two- or three-part division step in which the first block is divided horizontally or vertically into two or three parts to generate two or three blocks as a third block, The 2-3 division step prohibits dividing the first block with a horizontal dividing line and permits dividing it with a vertical dividing line if, when the first block is divided with a horizontal dividing line, the divided third block extends beyond the right side of the picture boundary, and when the first block is divided with a vertical dividing line, the divided third block does not extend below the picture boundary. The 2-3 division step prohibits dividing the first block with a vertical dividing line and permits dividing it with a horizontal dividing line if, when the first block is divided with a vertical dividing line, the divided third block extends below the picture boundary, and when the first block is divided with a horizontal dividing line, the divided third block does not extend to the right of the picture boundary. The aforementioned 2-3 division step prohibits dividing the first block into three parts if, after dividing the first block into three parts by a horizontal or vertical dividing line, the resulting third block extends beyond the picture boundary. The above 2-3 division step prohibits dividing the tree block into two using horizontal and vertical division lines if the 128x128 pixel tree block obtained by dividing the image extends beyond the lower right side of the picture boundary. The decoding step does not decode the flag under the condition that the first block is not divided into three parts by horizontal or vertical dividing lines. An image decoding program characterized by the following features.

4. An image encoding device that encodes an image in divided block units, A block division unit that recursively divides the image into rectangles of a predetermined size to generate a first block, The system comprises an encoding unit that encodes block division information including a flag indicating whether the first block is to be divided into two or three parts, and generates a bitstream, The aforementioned block division section is A four-part division unit in which the first block in the recursive partition is divided in half horizontally and vertically to form four blocks, and four blocks are generated as the second block, The recursive partitioning includes a 2-3 division section that divides the first block in the horizontal or vertical direction into two or three parts to generate two or three blocks as a third block, The aforementioned 2-3 division section prohibits dividing the first block with a horizontal division line and permits dividing it with a vertical division line if, when the first block is divided into two by a horizontal division line, the divided third block extends beyond the right side of the picture boundary, and when the first block is divided into two by a vertical division line, the divided third block does not extend below the picture boundary. The aforementioned 2-3 division section prohibits dividing the first block with a vertical division line and permits dividing it with a horizontal division line if, when the first block is divided with a vertical division line, the divided third block extends below the picture boundary, and when the first block is divided with a horizontal division line, the divided third block does not extend to the right of the picture boundary. The aforementioned 2-3 division section prohibits dividing the first block into three parts if the divided third block extends beyond the picture boundary, assuming the first block is divided into three parts by a horizontal or vertical dividing line. The aforementioned 2-3 division section prohibits dividing the 128x128 pixel tree block obtained by dividing the image into two using horizontal and vertical division lines if the tree block extends beyond the lower right side of the picture boundary. The encoding unit does not encode the flag under the condition that the first block is not divided into three by horizontal or vertical dividing lines. An image coding device characterized by the following:

5. An image encoding method that encodes an image in divided block units, A block division step involves recursively dividing the image into rectangles of a predetermined size to generate a first block, The method includes an encoding step of encoding block division information, which includes a flag indicating whether the first block is to be divided into two or three parts, to generate a bitstream, The aforementioned block partitioning step is: A four-part division step in which the first block in the recursive partition is divided in half horizontally and vertically to form four blocks, and four blocks are generated as the second block, The recursive partitioning includes a two- or three-part division step in which the first block is divided horizontally or vertically into two or three parts to generate two or three blocks as a third block, The 2-3 division step prohibits dividing the first block with a horizontal dividing line and permits dividing it with a vertical dividing line if, when the first block is divided with a horizontal dividing line, the divided third block extends beyond the right side of the picture boundary, and when the first block is divided with a vertical dividing line, the divided third block does not extend below the picture boundary. The 2-3 division step prohibits dividing the first block with a vertical dividing line and permits dividing it with a horizontal dividing line if, when the first block is divided with a vertical dividing line, the divided third block extends below the picture boundary, and when the first block is divided with a horizontal dividing line, the divided third block does not extend to the right of the picture boundary. The aforementioned 2-3 division step prohibits dividing the first block into three parts if, after dividing the first block into three parts by a horizontal or vertical dividing line, the resulting third block extends beyond the picture boundary. The above 2-3 division step prohibits dividing the tree block into two using horizontal and vertical division lines if the 128x128 pixel tree block obtained by dividing the image extends beyond the lower right side of the picture boundary. The encoding step, under the condition that the first block is not divided into three by horizontal or vertical dividing lines, does not encode the flag. An image encoding method characterized by the following.

6. An image encoding program that encodes an image in divided block units, A block division step involves recursively dividing the image into rectangles of a predetermined size to generate a first block, The computer is made to perform an encoding step of encoding block division information including a flag indicating whether the first block is to be divided into two or three parts, thereby generating a bitstream. The aforementioned block partitioning step is: A four-part division step in which the first block in the recursive partition is divided in half horizontally and vertically to form four blocks, and four blocks are generated as the second block, The recursive partitioning includes a two- or three-part division step in which the first block is divided horizontally or vertically into two or three parts to generate two or three blocks as a third block, The 2-3 division step prohibits dividing the first block with a horizontal dividing line and permits dividing it with a vertical dividing line if, when the first block is divided with a horizontal dividing line, the divided third block extends beyond the right side of the picture boundary, and when the first block is divided with a vertical dividing line, the divided third block does not extend below the picture boundary. The 2-3 division step prohibits dividing the first block with a vertical dividing line and permits dividing it with a horizontal dividing line if, when the first block is divided with a vertical dividing line, the divided third block extends below the picture boundary, and when the first block is divided with a horizontal dividing line, the divided third block does not extend to the right of the picture boundary. The aforementioned 2-3 division step prohibits dividing the first block into three parts if, after dividing the first block into three parts by a horizontal or vertical dividing line, the resulting third block extends beyond the picture boundary. The above 2-3 division step prohibits dividing the tree block into two using horizontal and vertical division lines if the 128x128 pixel tree block obtained by dividing the image extends beyond the lower right side of the picture boundary. The encoding step, under the condition that the first block is not divided into three by horizontal or vertical dividing lines, does not encode the flag. An image encoding program characterized by the following features.

7. A storage method for generating a bitstream according to the image encoding method described in claim 5 and storing the bitstream in a recording medium.

8. A transmission method for generating a bitstream according to the image encoding method described in claim 5, and transmitting the bitstream.

Images