Image decoding device, image decoding method, and image decoding program

Abstract

To provide a technique for improving encoding efficiency by performing block division suitable for image encoding and decoding.SOLUTION: A block division unit includes: a four division unit that generates four blocks by dividing each of a horizontal direction and a vertical direction of a target block in half to realize four division of the target block; and a 2-3 division unit that generates two or three blocks by dividing the target block in recursive division into two or three in the horizontal direction or the vertical direction. The 2-3 division unit prohibits horizontal division of the target block in a case in which the target block is horizontally divided, when the divided target block exceeds a right side of a picture boundary, and prohibits vertical division of the target block in a case in which the target block is vertically divided when the divided target block exceeds a lower side of the picture boundary.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a technique for encoding and decoding an image in units of divided blocks. [Background technology]

[0002] In image encoding and decoding, an image is divided into blocks, which are groups of a predetermined number of pixels. Processing is performed in units of locks. In this case, by dividing it into appropriate block units, The efficiency of prediction, inter-prediction, orthogonal transformation, entropy coding, etc. is improved, resulting in This improves the encoding efficiency. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2015-526008 [Non-patent literature]

[0004] [Non-Patent Document 1] JVET, Versatile Video Coding (Draft 2), July 2018 Summary of the Invention [Problem to be solved by the invention]

[0005] If the blocks are not divided into appropriate sizes and shapes, the coding efficiency will decrease. At the edge of the screen, blocks containing pixels beyond the picture boundary are of an inappropriate size and shape. This resulted in a decrease in coding efficiency.

[0006] The present invention has been made in view of the above circumstances, and its object is to provide an image encoding and decoding method. To provide a technology for improving coding efficiency by dividing a signal into blocks suitable for coding. is located. [Means for solving the problem]

[0007] In one aspect of the present invention that solves the above problem, the block division unit Divide the elephant block into four halves horizontally and vertically to generate four blocks. The division part and the target block in the recursive division are divided into two or three parts horizontally or vertically. and a 2-3 division unit that divides the data to generate two or three blocks, the 2-3 division unit If the target block is divided horizontally, the divided target block is If it exceeds the right side, the target block is prohibited from being divided horizontally. If the vertical division is performed, the divided target block may exceed the bottom of the picture boundary. In this case, dividing the target block vertically is prohibited. [Effects of the Invention]

[0008] According to the present invention, block division suitable for encoding and decoding of an image is possible, and encoding Efficiency can be improved. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram of an image encoding device according to a first embodiment. [Figure 2] FIG. 1 is a block diagram of an image decoding device according to a first embodiment. [Figure 3] 10 is a flowchart illustrating division into tree blocks and division within tree blocks. [Figure 4] FIG. 10 is a diagram illustrating how an input image is divided into tree blocks. [Figure 5]FIG. 1 is a diagram illustrating a z-scan. [Figure 6] FIG. 10 is a diagram illustrating division of a tree block. [Figure 7] 10 is a flowchart illustrating processing of each divided block when a tree block is divided into four. [Figure 8] 10 is a flowchart illustrating processing of each divided block when a tree block is divided into two or three. [Figure 9] FIG. 10 is a diagram illustrating the relationship between tree blocks and picture boundaries. [Figure 10] FIG. 1 is a diagram illustrating the relationship between picture boundaries and pixels. [Figure 11] 4 is a flowchart illustrating block division according to the first embodiment. [Figure 12] FIG. 2 is a diagram illustrating block division according to the first embodiment. [Figure 13] FIG. 2 is a diagram showing syntax related to block division in the first embodiment. [Figure 14] 10 is a flowchart illustrating block division according to the second embodiment. [Figure 15] FIG. 10 is a diagram illustrating block division in the second embodiment. [Figure 16] 13 is a flowchart illustrating block division according to the third embodiment. [Figure 17] FIG. 11 is a diagram illustrating block division in the third embodiment. [Figure 18] 13 is a flowchart illustrating block division according to the third embodiment. [Figure 19] FIG. 13 is a diagram illustrating block division in the fourth embodiment. [Figure 20] 13 is a flowchart illustrating block division according to the fourth embodiment. [Figure 21] FIG. 13 is a diagram illustrating block division in the fifth embodiment. [Figure 22] FIG. 2 is a diagram illustrating an example of a hardware configuration of a coding / decoding device according to a first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] The embodiment of the present invention provides a technique for encoding and decoding an image in units of divided blocks. Provide. (First embodiment) Regarding the image encoding device 100 and the image decoding device 200 according to the first embodiment of the present invention, In the first embodiment, when a block is divided into two or three parts, Limit block splitting.

[0011] FIG. 1 is a block diagram of an image coding device 100 according to the first embodiment. It only shows the data flow related to the image signal, and does not include image information such as motion vectors and prediction modes. The flow of data related to additional information other than the signal is not shown. At least one screen's worth of image signals is input.

[0012] The block division unit 101 divides an image into blocks to be coded, which are units of coding processing. , the image signal in the block to be coded is supplied to the residual signal generator 103. The division unit 101 divides the image signal of the block to be coded into The predicted image is supplied to the predicted image generation unit 102.

[0013] The block division unit 101 recursively divides an image into rectangles of a predetermined size, and divides the image into blocks to be coded. The block division unit 101 divides the target block into four parts in the recursive division. The 4-part division generates four blocks by dividing the target block into two or less blocks in the recursive division. and a 2-3 division unit that divides the data into two or three blocks. The detailed operation of the dividing section 101 will be described later.

[0014] The predicted image generating unit 102 receives the image signal of the block to be coded from the block dividing unit 101. , and a decoded image signal is supplied from the decoded image memory 108. The predicted image generation unit 102 receives the supplied Using the signal, intra prediction (intra-picture prediction) or inter prediction ( Intra prediction is performed to generate a predicted image signal. and the coded block adjacent to the coding target block in the same picture (coding picture). The image signal of the block is supplied from the decoded image memory 108 to the predicted image generation unit 102 . The predicted image generating unit 102 receives this image signal and the block division unit 101. A predicted image signal is generated using the image signal of the block to be coded. , a coded picture (reference picture) that is chronologically before or after the coded picture The image signal is supplied from the decoded image memory 108 to the predicted image generation unit 102. The predicted image generating unit 102 compares this image signal with the coding pair supplied from the block dividing unit 101. The degree of match is evaluated by block matching, etc., and the motion is calculated. The predicted image generating unit 102 calculates a motion vector from the reference image based on the motion vector. The predicted image generator 102 performs compensation to generate a predicted image signal. The predicted image signal is supplied to a residual signal generator 103 .

[0015] The residual signal generator 103 receives the image signal to be coded and the predicted image generated by the predictive image generator 102. The prediction signal is subtracted from the residual signal to generate a residual signal, which is supplied to the orthogonal transform and quantization unit 104 .

[0016] The orthogonal transform and quantization unit 104 performs orthogonal transformation on the residual signal supplied from the residual signal generation unit 103. The orthogonal transform and quantization unit 104 encodes the orthogonally transformed and quantized residual signal. The resulting signal is supplied to the inverse quantization and inverse orthogonal transformation unit 105 and the inverse quantization and inverse orthogonal transformation unit 106 .

[0017] The encoding unit 105 receives the orthogonally transformed and quantized image data supplied from the orthogonal transform and quantization unit 104. The encoding unit 105 generates an encoded bit stream corresponding to the residual signal. Regarding additional information such as motion vectors, prediction modes, and block division information provided by the component elements, Then, the encoding unit 105 generates a corresponding encoded bit stream. The bitstream is output from the image encoding device 100.

[0018] The inverse quantization and inverse orthogonal transformation unit 106 receives the orthogonal transform signal supplied from the orthogonal transform and quantization unit 104. The quantized residual signal is inverse quantized and inverse orthogonal transformed to obtain the residual signal. The cross-conversion unit 106 supplies the residual signal to a decoded image signal superimposition unit 107 .

[0019] The decoded image signal superimposing unit 107 superimposes the predicted image signal generated by the predicted image generating unit 102. The residual signal obtained by the inverse quantization and inverse orthogonal transformation unit 106 is superimposed to generate a decoded image. The decoded image signal superimposing unit 107 stores the coded image in the image memory 108. The image is then filtered to reduce block distortion and other artifacts caused by the decoding process. It may be stored in 8.

[0020] FIG. 2 is a block diagram of an image decoding apparatus 200 according to the first embodiment. It only shows the flow of data related to signals, and image signals such as motion vectors and prediction modes The flow of data related to additional information other than the above is not shown. The encoded bitstream is input.

[0021] The decoding unit 201 decodes the supplied coded bit stream and performs orthogonal transformation and quantization. The decoding unit 201 then supplies the motion vector-encoded residual signal to the block division unit 202. Additional information such as vector, prediction mode, and block division information is supplied to each component. It is used for processing the information.

[0022] The block division unit 202 divides the block to be decoded based on the decoded block division information. The shape of the orthogonally transformed and quantized residual signal of the determined block to be decoded is inversely The result is supplied to the quantization and inverse orthogonal transformation unit 203 .

[0023] The block division unit 202 divides a predetermined size of The block division unit 202 recursively divides the image into rectangles to generate blocks to be decoded. is a 4-partitioning part that divides the target block into 4 parts in the recursive division to generate 4 blocks. ,The target block in the recursive division is divided into two or three blocks. The detailed operation of the block dividing unit 202 will be described later. do.

[0024] The inverse quantization and inverse orthogonal transformation unit 203 inversely quantizes the orthogonally transformed and quantized residual signal. The residual signal is then supplied to the decoded image signal superimposing unit 205.

[0025] The predicted image generating unit 204 generates a predicted image from the decoded image signal supplied from the decoded image memory 206. An image signal is generated and supplied to a decoded image signal superimposing unit 205 .

[0026] The decoded image signal superimposing unit 205 superimposes the predicted image signal generated by the predicted image generating unit 204 and the inverse The residual signal obtained by the quantization and inverse orthogonal transformation unit 203 is superimposed on the residual signal to generate a decoded image signal. Furthermore, the decoded image signal superimposing unit 205 stores the decoded image signal in the decoded image memory 206 . The decoded image signal superimposing unit 205 reduces block distortion and the like caused by coding in the decoded image. The decoded image may then be subjected to a filtering process to store the decoded image in the decoded image memory 206. The decoded image signal superimposing unit 205 outputs the decoded image from the image decoding device 200.

[0027] Next, the operation of the block division unit 101 in the image coding device 100 will be explained with reference to FIG. FIG. 3 shows how the block division unit 101 divides an image into tree blocks, This shows the operation of dividing a part into blocks.

[0028] First, the input image is divided into tree blocks of a predetermined size (S1000). Here, the tree block is 128x128 pixels. The size is not limited to the above, and any size and aspect ratio may be used as long as it is rectangular. The size of the treeblock may be predetermined between the device and the decoding device. The encoding device determines the size of the treeblock, records it in the encoded bitstream, and decodes it. The encoding device uses the treeblock size recorded in the encoded bitstream. Fig. 4 shows how an input image is divided into tree blocks. The blocks are coded in raster scan order, ie, left to right, top to bottom.

[0029] The inside of the tree block is further divided into rectangular blocks. The image is encoded in the z-scan order shown in Figure 5. The z-scan order is top left, top right, bottom left, bottom right, The division within a tree block can be 4-, 2-, or 3-way. .

[0030] The blocks are divided into four parts, each divided in half horizontally and vertically as shown in Figure 6(a). This is done by dividing the byte into four blocks.

[0031] The division of a block into two or three parts is done by dividing it horizontally or vertically. When dividing a block into two in the horizontal direction, it is divided into two halves as shown in Figure 6(b). Also, when dividing the block into three parts horizontally, as shown in Figure 6(c), Divide the image into 1:2:1 and generate three blocks. Divide the image into two blocks vertically. In this case, the image is divided in half as shown in Figure 6(d) to generate two blocks. When dividing the blocks into three, the ratio is 1:2:1 as shown in Figure 6(e), and the three blocks are Generate.

[0032] Referring again to FIG. 3, the operation of the block division unit 101 will be described. It is determined whether the inside of the block is to be divided into four halves in both the horizontal and vertical directions (S1001 ).

[0033] How to determine the optimal case based on multiple criteria, including whether to divide a block into four There is an existing method called RD optimization (Rate-Distortion Optimization). In optimization, the coding cost is calculated from the code amount and coding distortion. For each encoding, the encoding cost is calculated, and the encoding cost is minimized. In other words, the decision whether to divide a block into four or not is made by The coding cost is calculated when the block is divided into four and when it is not divided into four. This is done by selecting the case where the cost is the smallest. To achieve this, methods other than RD optimization may be used.

[0034] If it is determined that the inside of the tree block is to be divided into four (S1001: YES), The block is divided into four parts (S1002). The process of re-dividing the divided blocks will be explained later. (Figure 7).

[0035] If it is determined that the inside of the tree block is not to be divided into four (S1001: NO), It is determined whether the inside of the block is to be divided into two or three (S1003).

[0036] If it is determined that the tree block is to be divided into two or three parts (S1003: YES) Then, it is determined whether the division direction is the vertical direction (S1004).

[0037] If the split direction is determined to be vertical (S1004: YES), It is determined whether to divide the image into two (S1005).

[0038] If it is determined that the inside of the tree block is to be divided into two (S1005: YES), The inside of the tree block is divided into two parts vertically (S1006). On the other hand, the inside of the tree block is divided into three parts. If it is determined that the tree block is divided into three parts vertically (S1005: NO), S1007). The re-division process of blocks divided into two or three vertically will be explained later. (Figure 8).

[0039] If the dividing direction is determined to be horizontal (S1004: NO), the inside of the tree block is divided into It is determined whether to divide it into two (S1008).

[0040] If it is determined that the inside of the tree block is to be divided into two (S1008: YES), The inside of the tree block is divided into two parts horizontally (S1009). On the other hand, the inside of the tree block is divided into three parts. If it is determined that the tree block is divided into three parts horizontally (S1008: NO), S1010). The re-division process of blocks divided into two or three horizontally will be explained later. (Figure 8).

[0041] If it is determined that the tree block is not to be divided into two or three parts (S1003: NO) The block division process is terminated without dividing the inside of the tree block into blocks (S1011 ).

[0042] Next, the tree block is divided into four halves horizontally and vertically. The processing of each block will be described with reference to the flowchart in FIG.

[0043] First, decide whether to divide the block into four halves again horizontally and vertically. (S1101).

[0044] If it is determined that the block interior is to be divided into four again (S1101: YES), is again divided into four (S1102).

[0045] If it is determined that the block interior is not to be divided into four again (S1101: NO), It is determined whether to divide the image into two or three parts (S1103).

[0046] If it is determined that the block is to be divided into two or three parts (S1103: YES), It is determined whether the direction to be scanned is the vertical direction (S1104).

[0047] If the dividing direction is determined to be vertical (S1104: YES), the block is divided into two parts. It is determined whether or not to divide (S1105).

[0048] If it is determined that the block interior is to be divided into two (S1105: YES), the block interior is divided vertically. Divide the block into two in the perpendicular direction (S1106). On the other hand, if it is determined that the block interior should be divided into three ( S1105: NO), and then divide the inside of the block vertically into three parts (S1107).

[0049] If the dividing direction is determined to be horizontal (S1104: NO), the block is divided into two. It is determined whether or not to do so (S1108).

[0050] If it is determined that the inside of the block is to be divided into two (S1108: YES), the inside of the block is divided into two. Divide the block into two in the horizontal direction (S1109). On the other hand, if it is determined that the block interior should be divided into three ( S1108: NO), and then divide the inside of the block horizontally into three parts (S1110).

[0051] If it is determined that the block is not to be divided into two or three parts (S1103: NO), The block division process ends without dividing the inside of the block again (S1111).

[0052] The process shown in the flowchart in Figure 7 is executed recursively for each of the four divided blocks. The four divided blocks are coded in z-scan order.

[0053] Next, when the tree block is divided into two or three vertically, The check processing will be described with reference to the flowchart in FIG.

[0054] When a tree block is divided into two or three blocks vertically, Then, it is determined whether the inside of the block should be divided into two or three again (S1201).

[0055] If it is determined that the block is to be divided into two or three parts (S1201: YES), It is determined whether the direction to be scanned is the vertical direction (S1202).

[0056] If the division direction is determined to be vertical (S1202: YES), the block is divided into two parts. It is determined whether or not to divide (S1203).

[0057] If it is determined that the block interior is to be divided into two (S1203: YES), the block interior is divided vertically. Divide the block into two in the perpendicular direction (S1204). On the other hand, if it is determined that the block interior should be divided into three ( S1203: NO), and then divide the inside of the block vertically into three parts (S1205).

[0058] If the dividing direction is determined to be horizontal (S1202: NO), the block is divided into two. It is determined whether or not to do so (S1206).

[0059] If it is determined that the inside of the block is to be divided into two (S1206: YES), the inside of the block is divided into two. Divide it into two in the horizontal direction (S1207). On the other hand, if it is determined that the inside of the block should be divided into three ( S1206: NO), and then divide the inside of the block horizontally into three parts (S1208).

[0060] If it is determined that the block interior will not be divided into two or three again (S1201: NO), The block division process ends without further dividing the inside of the block (S1209).

[0061] When dividing the image vertically into two or three blocks, the flow chart in Fig. 8 is used for each divided block. The process shown in the chart is executed recursively. Inside the block divided into two or three are encoded from left to right.

[0062] Similarly, for each divided block when dividing horizontally into two or three, The process shown in the flowchart in 8 is executed recursively. The inside of the block is coded from top to bottom.

[0063] In addition, we have explained how to re-divide the divided blocks when the tree block is divided. However, the parent block does not have to be a treeblock. For example, a treeblock (128x1 When a block (28 pixels) is divided into four and each of the four blocks (64x64 pixels) is further divided Then, the above process is applied to the division of the further divided blocks.

[0064] The number of recursive block divisions may be limited by defining the number of divisions. Furthermore, the number of divisions may be determined in advance between the encoding device and the decoding device. The encoding device determines the number of divisions and records it in the encoded bitstream, and the decoding device The number of divisions recorded in the coded bitstream may be used.

[0065] Next, we will explain how to divide an image into blocks at the edge of the screen. As shown in Figure 9, the image size is The size of the tree block is not necessarily an integer multiple of the size of the tree block, so the tree block at the edge of the screen is It may contain both on-screen and off-screen parts across the picture boundary. The locks are tree blocks 1001, 1002, and 1003. In this case, the locks are shown in Figure 10. As shown in the figure, the area outside the picture boundary is the same as the outermost pixel on the screen. The pixels 1011, 1012, and 1013 are shown as the outermost pixels within the screen.

[0066] Pixel 1011 is in tree block 1001 and is the pixel located at the top right corner of the screen. The pixels to the right of pixel 1011 that are outside the picture boundary are pixels 101 Treated as the same as 1.

[0067] Pixel 1012 is located in tree block 1002 and is the pixel located at the bottom left of the screen. The pixels below pixel 1012 that are outside the picture boundary are pixels below pixel 1012. Treated as the same as 2.

[0068] Pixel 1013 is in tree block 1003 and is the pixel located at the bottom right of the screen. The pixels to the right, below, and to the lower right of pixel 1013 are outside the picture boundary. The pixel is treated as the same as pixel 1013.

[0069] Then, when dividing a block into two or three, the block division is restricted. This allows the blocks at the edges of the screen to be divided into appropriate shapes, improving coding efficiency. can be done.

[0070] Block division restrictions are applied when dividing a block into two or three blocks at the edge of the screen. In other words, the process of dividing into two or three parts (S1004 to S1010) in FIG. , is replaced by the process described below. Also, the process of dividing into two or three in FIG. 1104 to S1110) are replaced by the processing described below. The process of dividing into two or three parts (S1202 to S1208) is replaced by the process described below. Replace.

[0071] The restriction on block division will be explained with reference to FIG. 11. First, all of the block divisions It is determined whether or not to restrict block division for the combination (S1301). All combinations of block division are the combinations of dividing a block into two horizontally and vertically. If you divide the block into two in the horizontal direction, you can divide the block into three in the vertical direction. In addition, whether or not to restrict block division depends on whether or not the picture boundary is crossed. For example, if a block is divided into two horizontally, If dividing the pixels beyond the picture boundary, the block is divided horizontally. Limit splitting into two.

[0072] Here, restricting the division of blocks horizontally means dividing blocks horizontally. This means that dividing a block vertically is prohibited. Restricting this means that vertical division of blocks is prohibited. Similarly, restricting the division of a block into two means that the division of a block into two It also means that dividing a block into three parts is prohibited. This means that dividing a block into thirds is prohibited.

[0073] Next, it is determined whether or not to restrict block division for all combinations of block division ( S1302). When restricting block division in all combinations (S1302: YES) , the block is not divided (S1314). On the other hand, block division is restricted in all combinations. If not (S1302: NO), it is determined whether or not to restrict block division in the horizontal direction. (S1303).

[0074] If horizontal block division is restricted (S1303: YES), proceed to the next process (S On the other hand, if the horizontal block division is not restricted (S1303: NO), the vertical It is determined whether or not to restrict block division in the vertical direction (S1304).

[0075] If vertical block division is restricted (S1304: YES), proceed to the next process (S On the other hand, if vertical block division is not restricted (S1304: NO), It is determined whether the lock is to be divided vertically (S1305).

[0076] If the direction of block division is determined to be vertical (S1305: YES), It is determined whether or not to limit the division into three blocks (S1306). If it is determined to be a direction (S1305: NO), the process proceeds to the next step (S1310).

[0077] When limiting the division of a block to three (S1306: YES), divide the block vertically into two. On the other hand, if the division of the block into three is not restricted (S1306: NO), Then, it is determined whether or not the block should be divided into two (S1307).

[0078] If it is determined that the block should be divided into two (S1307: YES), the block is divided vertically. Divide the block into two (S1308). On the other hand, if it is determined that the block should be divided into three (S1307: NO), and then divide the block vertically into three parts (S1309).

[0079] If vertical division is restricted (S1304: YES), and the direction of block division If it is determined that the direction is horizontal (S1305: NO), it is determined whether to limit the division of the block into three parts. The determination is made (S1310).

[0080] When limiting the division of a block to three (S1310: YES), divide the block into two horizontally. On the other hand, if the division of the block into three is not restricted (S1310: NO), Then, it is determined whether or not the block should be divided into two (S1311).

[0081] If it is determined that the block is to be divided into two (S1311: YES), the block is divided horizontally. Divide the block into two (S1312). On the other hand, if it is determined that the block should be divided into three (S1311: NO), and then divide the block horizontally into thirds (S1313).

[0082] In other words, if a pixel beyond a picture boundary is divided by block division, Restrict block division in the direction of

[0083] Here, we will explain a specific example. At the bottom of the screen, the tree block is not divided into four parts. (S1001: NO), the inside of the tree block is divided into two or three parts (S1003: Y ES). Figure 12(a) shows the treeblocks divided into on-screen and off-screen parts across the picture boundary. In this case, if the tree block is divided into all combinations, Then, it is determined whether or not to restrict block division (S1301).

[0084] For all combinations of treeblock division, the results are shown in Fig. 12(a) to As shown in Fig. 12(a), if the block is divided into two vertically, This block division is restricted because pixels beyond the picture boundary are divided. As shown in Figure 12(b), if the block is divided into three parts vertically, the picture boundary is crossed. Since the pixel at the position where the pixel is located is divided, this block division is restricted. If the block is divided into two horizontally as shown, the pixels beyond the picture boundary are divided. Similarly, as shown in Figure 12(d), If you divide the block into thirds horizontally, the pixels beyond the picture boundary will not be divided. There are no restrictions on block division.

[0085] Therefore, block division is not restricted in all combinations of block division (S1302: In addition, the horizontal block division is not restricted (S1303: NO), and the vertical block division is restricted (S1304: NO). Restrict lock division (S1304: YES). And do not restrict division of blocks into 3. (S1310: NO). Therefore, it is determined whether or not to divide the block into two (S1311). Now, if we decide to divide the block into two (S1311: YES), we divide the block into Divide it into two parts horizontally (S1312).

[0086] These block division restrictions ensure that blocks have the correct shape because In a block containing pixels, the pixel values ​​outside the screen are constant. The pixel value change in the part of the screen that is locked is relative to the block that does not include pixels outside the screen. Therefore, there is little need to encode small pixel changes. By grouping pixels into blocks as close as possible, the amount of code is reduced and coding efficiency is improved. It can be improved.

[0087] This restriction on block division also applies to the right edge of the screen. The cases where the lock is divided are shown in Figure 12(e) to Figure 12(h). As shown in Fig. 12(e), if the block is divided into two in the horizontal direction, the picture boundary is crossed. The pixel at the position where the pixel is located is divided. Also, as shown in Figure 12(f), the block is divided into three parts horizontally. When dividing, pixels beyond the picture boundary are divided. This restriction on block division can improve coding efficiency. do.

[0088] Next, the operation of the block division unit 202 of the image decoding device 200 will be described. The block dividing unit 202 performs the same processing as the block dividing unit 101 in the image encoding device 100 described above. The block division unit 101 selects a block division pattern. On the other hand, the block division unit 202 selects the coded block and outputs the selected block division information. The blocks are divided using the block division information decoded from the bit stream. The block division restrictions are the same as those of the image coding device 100 described above.

[0089] The syntax for block division in the first embodiment (the structure of the coded bit stream) The syntax rules are shown in Figure 13. In Figure 13, QT() is the syntax for dividing a block into four parts. MTT() represents the syntax for dividing a block into two or three parts. The image coding device 100 performs coding in accordance with this syntax, and the image decoding device 200 performs coding in accordance with this syntax. Decode according to this syntax.

[0090] First, whether to divide the block into four or not is indicated by QTflag. If you divide it into four, QTflag=1, If not divided into four, set QTflag=0. If divided into four (QTflag=1), each block divided into four If it is possible to divide it into four parts (QTvalid=1), it will be divided into four parts recursively. If there is no QTflag (QTflag=0), whether to split into two or three is indicated by MTTflag. When dividing into three parts (MTTflag=1), whether to divide vertically is indicated by vertical_flag, and Whether to split or not is indicated by BTflag. To split vertically, set vertical_flag=1, and to split horizontally, set If you want to split it in the vertical direction, set vertical_flag=0. If you want to split it in two, set BTflag=1. When dividing, set BTflag=0. Each block divided into two or three is further divided into two. Or if it can be divided into three parts (MTTvalid=1), it will recursively divide it into two or three parts.

[0091] Here, the variable QTvalid indicates whether each of the four divided blocks can be further divided into four. QTvalid is defined for each block divided into four. Blocks divided into four If the pixel in the screen is not included in QTvalid, QTvalid = 0. Otherwise, QTvalid = 1. become.

[0092] Also, each block divided into two or three can be further divided into two or three. The variable MTTvalid that indicates the validity of each block divided into two or three is explained below. If the pixel in the screen is not included in the block to be divided into two or three, MTTvalid = 0. In all other cases, MTTvalid = 1.

[0093] In this embodiment, the direction of block division is restricted, so vertical_flag is not necessary. Therefore, vertical_flag in FIG. 13 may be omitted.

[0094] Due to these block division restrictions, it is difficult to divide the blocks at the edge of the screen into appropriate shapes. This allows for improved coding efficiency. Lock splitting is possible. (Second embodiment) An image encoding device and an image decoding device according to a second embodiment of the present invention will be described. In the second embodiment, when the depth of the block division reaches the limit depth, the block division is stopped. The rest of the configuration is the same as in the first embodiment, so a description thereof will be omitted.

[0095] Here, the depth of block division will be explained. After dividing into two or three blocks, recursively divide each block into two or three blocks. The process of dividing the image into two or three parts has been explained. The three-division process is defined as depth 0. Also, the first two-division or three-division process The second division into two or three parts for each block is defined as depth 1. The third division into two or three parts for each block divided by the division or trisection process The division process is defined as depth 2, and the depth is defined in the same way. The depth at which the hole is to be cut is determined in advance and defined as the limit depth.

[0096] Block division restrictions are applied when dividing a block into two or three blocks at the edge of the screen. In other words, the process of dividing into two or three parts (S1004 to S1010) in FIG. , is replaced by the process described below. Also, the process of dividing into two or three in FIG. 1104 to S1110) are replaced by the processing described below. The process of dividing into two or three parts (S1202 to S1208) is replaced by the process described below. Replace.

[0097] The restrictions on block division will be explained using Figure 14. First, when a block is divided into two or When dividing the image into three parts, it is judged whether the depth of the block division reaches the limit depth (S1 401).

[0098] If the limit depth has not been reached (S1401: NO), it is determined that there is no limit to block division. On the other hand, if the limit depth is reached (S1401: YES), the block For all combinations of block division, it is determined whether or not to restrict block division (S13 01). S1301 is the same as in the first embodiment, so the explanation will be omitted.

[0099] Next, it is determined whether or not to restrict block division for all combinations of block division ( S1302). The process from S1302 onwards is the same as in the first embodiment, so the explanation will be omitted. do.

[0100] In other words, the pixels beyond the picture boundary are divided by the block division, and the blocks are When the depth of block division reaches the limit depth, the block division is limited.

[0101] Here, we will explain a specific example. At the bottom of the screen, the tree block is not divided into four parts. (S1001: NO), divide the inside of the tree block into two or three parts (S1003: Y ES). The limit depth is 1. Figure 12(a) shows the tree block at the picture boundary. This shows how the tree block includes both the inside and outside of the screen, separated by a square. When splitting, the vertical split splits the pixels beyond the picture boundary. However, the depth is 0, which does not reach the limit depth of 1 (S1401: NO). Therefore, it is decided that there is no restriction on block division (S1402). No restriction on division (S1302: NO), no restriction on horizontal block division (S1303 : NO), and does not restrict vertical block division (S1304: NO). It is determined whether the block is to be divided vertically (S1305). If it is determined that the block should be divided (S1305: YES), there is no restriction on dividing the block into three parts. If the result of step S1306 is NO, it is determined whether or not to divide the block into two (step S1307). If it is determined that the block should be divided into two (S1307: YES), the block is divided vertically. The image is divided into two parts in the same direction (S1308). This is shown in FIG. 12(a).

[0102] Next, of the blocks divided vertically into two, the left block is divided into two or three. At this time, the depth is 1, which means that the limit depth of 1 has been reached (S1401: YES). Therefore, for all combinations of block division, we can decide whether to restrict block division or not. Decide (S1301).

[0103] When the blocks are divided into all combinations, the results are shown in Fig. 15(a) to Fig. 1 As shown in Fig. 15(a), when a block is divided into two vertically, the picture Since pixels beyond the pixel boundary are split, this block split is restricted. As shown in 15(b), if the block is divided into three parts vertically, the position beyond the picture boundary On the other hand, as shown in Figure 15(c), If a block is divided into two horizontally, the pixels beyond the picture boundary will not be divided. Therefore, we do not restrict this block division. Similarly, we divide the blocks into water as shown in Figure 15(d). When dividing the image horizontally into thirds, the pixels beyond the picture boundary are not divided. Do not limit block splitting.

[0104] Therefore, block division is not restricted in all combinations of block division (S1302: In addition, the horizontal block division is not restricted (S1303: NO), and the vertical block division is restricted (S1304: NO). Lock division is restricted (S1304: YES). And division of blocks into three is not restricted. (S1310: NO). Therefore, it is determined whether or not to divide the block into two (S1311). Now, if we decide to divide the block into two (S1311: YES), we divide the block into Divide it into two parts horizontally (S1312).

[0105] These block division restrictions ensure that blocks have the appropriate size and shape. Therefore, in a block that includes pixels outside the screen, the pixel values ​​of the outside part of the screen are constant. Therefore, the change in pixel value of the part of the block that is on the screen is not reflected in the block that does not include pixels outside the screen. Therefore, there is little need to encode small pixel changes. By grouping pixels outside the screen into blocks as much as possible, the amount of coding is reduced. This can improve the encoding efficiency.

[0106] This restriction on block division also applies to the right edge of the screen. When dividing into 3, the depth is 0 and the limit depth is 1, so the block division is not limited and the limit is 2. Divide or divide into 3. When dividing a divided block further, the depth is 1 and the limit depth is Since the number of blocks has reached 1, it is determined whether to limit block division. When dividing a block into two parts horizontally and dividing a block into three parts horizontally, the picture boundary is crossed. The pixels at the positions where the pixel is located are divided. Therefore, these block divisions are restricted. Block division restrictions can improve coding efficiency.

[0107] In this embodiment, the depth of block division is defined as division into two or three. This may be defined for four divisions. In this embodiment, the depth of block division is This limits the block division. The number or ratio of pixels at the position may be limited. In other words, these values ​​may be predetermined values. If the value is larger than 1, the block splitting is limited. , may be set to different values ​​for each depth of block division. Blocks with a small percentage are divided, and blocks with a large number or percentage of pixels outside the screen are not divided. The pixels beyond the picture boundary that the block contains and the limit depth of the block division are The values ​​that limit the block division, such as the number and ratio of The decoder uses the values ​​recorded in the coded bitstream. That's fine.

[0108] In the first embodiment, block division is restricted regardless of the depth of block division. In this embodiment, block division is restricted according to the depth of block division. Blocks with a small percentage of off-screen pixels are split, and blocks with a large percentage of off-screen pixels are split. Therefore, it is important to divide the blocks at the edge of the screen into appropriate sizes and shapes. This allows for improved coding efficiency. It can be divided into blocks. (Third embodiment) An image encoding device and an image decoding device according to a third embodiment of the present invention will be described. In the third embodiment, the block division is performed according to the number of pixels beyond the picture boundary. The rest of the configuration is the same as in the first embodiment, so a description thereof will be omitted.

[0109] Block division control is applied when dividing the block into two or three at the edge of the screen. In other words, the process of dividing into two or three parts (S1004 to S1010) in FIG. , is replaced by the process described below. Also, the process of dividing into two or three in FIG. 1104 to S1110) are replaced by the processing described below. The process of dividing into two or three parts (S1202 to S1208) is replaced by the process described below. Replace.

[0110] The control of block division will be explained with reference to FIG. 16. First, all of the block divisions For each combination, the number of pixels beyond the picture boundary is counted (S1601). All combinations of block division are the combinations of the blocks when the block is divided into two horizontally. If you divide a block into two vertically, if you divide a block into three horizontally, There are four when divided into three directions.

[0111] And for each case where the block is divided into two and three, The division direction that maximizes the number of pixels beyond the picture boundary is determined (S1602).

[0112] Next, it is determined whether or not the block should be divided into two (S1603).

[0113] If it is determined that the data should be divided into two (S1603: YES), the data is divided into two parts as determined in S1602. On the other hand, if it is determined that the image should be divided into three parts (S1603:N O), and then divide it into three in the division direction determined in S1602 (S1605).

[0114] In other words, the block is divided so that the pixels beyond the picture boundary are included in the block as much as possible. Controls the direction of lock splitting.

[0115] Here, we will explain a specific example. Now, the image is 1920x1080 pixels, and the tree block is The pixel size is 128x128. Also, the tree block is not divided into four (S1101: NO). The inside of the block is divided into two or three parts (S1103: YES). ), there are 72 vertical pixels at the bottom of the screen that extend beyond the picture boundary. At this time, when the tree block is divided into all combinations, the peaks in each block are The number of pixels at positions beyond the texture boundary is counted (S1601).

[0116] When the treeblock is divided into all combinations, the results are shown in Fig. 17(a) to As shown in Fig. 17(a), when the block is divided into two vertically, The left side (500) and the right side (501) of the divided block each have 4608 pixels. On the other hand, if the block is divided into two horizontally as shown in Figure 17(b), the divided block The upper side (510) is 1024 pixels, and the lower side (511) is 8192 pixels. The maximum number of pixels beyond the pixel boundary is the bottom 8192 pixels when dividing horizontally. Therefore, the division direction that maximizes the number of pixels is the horizontal direction (S1602).

[0117] Similarly, if the block is divided into three parts vertically as shown in Figure 17(c), the divided parts The left (520) and right (522) sides of the block each have 2304 pixels, and the center (52 1) has 4608 pixels. On the other hand, as shown in Figure 17(d), the block is divided into three horizontally. For example, the upper part of the divided block is 0 pixels, the center (531) is 5120 pixels, and the lower part is (532) is 4096 pixels. In other words, the maximum number of pixels beyond the picture boundary is The maximum number of pixels is 5120 pixels in the center when dividing the image horizontally. The division direction is the horizontal direction (S1602).

[0118] Finally, it is determined whether or not the block should be divided into two (S1603). If the result is "YES" (S1603), the block is divided into two in the horizontal direction, which is the determined division direction. On the other hand, if it is determined that the image should be divided into three parts (S1603: NO), the image is decided as follows. The block is divided into three in the horizontal direction, which is the division direction (S1605).

[0119] In this embodiment, when a block is divided into two or three parts, the part that exceeds the picture boundary is The direction of block division is controlled so that the most pixels in the block are included. In the block including pixels outside the screen, the pixel values ​​of the outside part are constant, so This is because there is little need to encode small pixel changes. By combining as many blocks as possible into one, the amount of code is reduced and coding efficiency is improved. It can be done.

[0120] Furthermore, not only the direction of division of the blocks but also the number of divisions of the blocks may be determined. As shown above, the maximum number of pixels beyond the picture boundary is 8192 pixels in the case of two divisions. In the case of three divisions, the number of pixels is 5120. In other words, the pixels beyond the picture boundary are black. Since the block is often divided into two, the block is divided into two.

[0121] This is because, compared to splitting into two, splitting into three blocks causes the block to be split closer to the edge of the block. The number of pixels beyond the picture boundary tends to be smaller in a three-division image than in a two-division image. Therefore, at the edge of the screen, the block is not divided into thirds, but rather into two. This reduces the processing required to divide the block into three parts, and the block division processing It can be made faster.

[0122] Block division control is applied when dividing the block into two or three at the edge of the screen. In other words, the process of dividing into two or three parts (S1004 to S1010) in FIG. , is replaced by the process described below. Also, the process of dividing into two or three in FIG. 1104 to S1110) are replaced by the processing described below. The process of dividing into two or three parts (S1202 to S1208) is replaced by the process described below. Replace.

[0123] The control of block division will be explained using FIG. 18. First, all of the block divisions For each combination, the number of pixels beyond the picture boundary is counted (S1651). All combinations of block division are the case of dividing a block into two horizontally and the case of dividing a block into two horizontally. These are the two blocks that are created when the block is divided vertically into two.

[0124] Next, when the block is divided into two, the maximum number of pixels beyond the picture boundary is Then, the division direction is determined so that the blocks are divided into two in the determined division direction (S1652). Divide (S1654).

[0125] The syntax for block division in this embodiment is as follows: vertical_flag in FIG. This is because in this embodiment, the direction of block division is controlled, and ve Furthermore, in this embodiment, the number of divisions of the block is It is also possible to determine the number of blocks, or not to divide the blocks into three. In that case, BTflag is invalid. Since this is an essential part, it may be omitted. The syntax may be the same as in FIG.

[0126] In the case of 1920x1080 pixels, the pixel beyond the picture boundary at the right edge of the screen However, if the image width is not an integer multiple of the treeblock size, Since there are pixels beyond the image boundary, block division is controlled in the same manner as described above.

[0127] In this embodiment, the block division method is determined based on the number of pixels beyond the picture boundary. This is because the percentage of pixels beyond the picture boundary determines the direction of the block. The direction of the division may also be controlled.

[0128] By controlling this block division, the blocks at the edge of the screen are divided into appropriate sizes and shapes. This allows for improved coding efficiency. This allows for block division suitable for (Fourth embodiment) An image encoding device and an image decoding device according to a fourth embodiment of the present invention will be described. In the fourth embodiment, a case where the picture boundary is different from that in the previous embodiment will be described. The rest of the configuration is the same as in the first embodiment, so a description thereof will be omitted.

[0129] Here, we consider a pattern in which the picture boundaries are different from those in Figure 12. However, Even if the result is different, the decision on whether to restrict block division is the same as in the case of Figure 12. For example, in Figure 12(a), if the picture boundary is located slightly above Even in this case, the pixels beyond the picture boundary are divided, so block division is restricted. In other words, even if the picture boundary changes vertically, the block division is The decision to limit the allocation remains unchanged. This is also the case in Figure 12(b). 12(e) and 12(f) show that even if the picture boundary changes horizontally, the block size is The decision to limit the rate remains unchanged.

[0130] In the end, the patterns with picture boundaries different from those in Figure 12 are those in Figures 19(a) to 19(d). The picture boundary in Figure 19(a) is located higher than that in Figure 12(c). The picture boundary in Fig. 9(b) is located above that in Fig. 12(d). Similarly, Fig. 19(c) The picture boundary in Figure 19(d) is located to the left of Figure 12(g) and Figure 12(h). are.

[0131] Figure 19(a) and Figure 19(b) are both horizontal divisions. In the case of dividing the image into three parts, the pixels beyond the picture boundary are not divided. The pixels beyond the limit are divided. In other words, the block division is limited to 3 divisions. do.

[0132] Figure 19(c) and Figure 19(d) are both vertical divisions. In the case of dividing the image into three parts, the pixels beyond the picture boundary are not divided. The pixels beyond the limit are divided. In other words, the block division is limited to 3 divisions. do.

[0133] In other words, the decision to limit block division is different for 2-division and 3-division. This is because the block is divided closer to the edge of the block in the 3-division method. In the case of splitting the image into two, the pixels beyond the picture boundary tend to be split. At the edge, it is possible to determine whether to restrict the direction of division into two blocks rather than dividing into three. This reduces the processing required to divide the block into three parts, and speeds up the block division processing. It can be made into

[0134] Block division restrictions are applied when dividing a block into two or three blocks at the edge of the screen. In other words, the process of dividing into two or three parts (S1004 to S1010) in FIG. , is replaced by the process described below. Also, the process of dividing into two or three in FIG. 1104 to S1110) are replaced by the processing described below. The process of dividing into two or three parts (S1202 to S1208) is replaced by the process described below. Replace.

[0135] The block division restriction will be explained using FIG. 20. FIG. 20 is a diagram showing a block division restriction in comparison with FIG. 11. Therefore, the same step numbers as in Figure 11 are used for the explanation. It may be omitted.

[0136] First, we decide whether to restrict block division for all combinations of block division. All combinations of block division are defined as the combinations of dividing a block into two blocks horizontally. When a block is divided, it is divided into two vertically. Whether or not to limit the number of pixels depends on whether or not the pixels beyond the picture boundary are split. For example, if a block is divided into two horizontally, the pixels beyond the picture boundary are divided. If the block is divided, the division into two blocks horizontally is restricted. The process is the same as in Figure 11 except that the process for dividing into three parts is eliminated, so the explanation is omitted. do.

[0137] Here, a specific example will be explained using FIG. 19(a). In the case of FIG. 19(a), For all combinations of block division, it is determined whether or not to restrict block division (S1 901). If the block is divided into two horizontally, the position beyond the picture boundary Since the pixels are not divided, the division in the horizontal direction is not restricted. When dividing into two, the pixels beyond the picture boundary are divided, so the vertical division Therefore, block division is restricted for all combinations of block division. (S1302: NO). And, the horizontal block division is not restricted (S1303 :NO), it limits vertical block division (S1304:YES). The lock is divided into two parts horizontally (S1312).

[0138] Now, consider a pattern where the picture boundary is further up or closer to the left than in Figure 19. In any pattern, pixels beyond the picture boundary are divided. , At the edges of the screen, the division of blocks into two or three parts may always be limited.

[0139] By controlling this block division, the blocks at the edge of the screen are divided into appropriate sizes and shapes. This allows for improved coding efficiency. This allows for block division suitable for (Fifth embodiment) An image encoding device and an image decoding device according to a fifth embodiment of the present invention will be described. In the fifth embodiment, a restriction on block division at the bottom right corner of the screen will be described. The rest of the configuration is the same as that of the first embodiment, so a description thereof will be omitted.

[0140] As shown in Figure 9, the tree blocks at the edge of the screen are separated by picture boundaries. In particular, the tree block 1001 at the right edge of the screen and the image The area at the bottom right of the screen is larger than the area outside the screen that includes the tree block 1002 at the bottom of the screen. The tree block 1003 tends to include more of the off-screen area. By restricting block division, there is a lot of room for improving coding efficiency. If the block is split, the split blocks contained in the tree block In this case, block division is restricted.

[0141] The restriction on block division at the bottom right corner of the screen will be explained using FIG. The block at the bottom end corresponds to treeblock 1003 in FIG.

[0142] First, we decide whether to restrict block division for all combinations of block division. All combinations of block division are defined as the combinations of dividing a block into two blocks horizontally. When a block is divided, it is divided into two vertically. Whether or not to limit the number of pixels depends on whether or not the pixels beyond the picture boundary are split. For example, if a block is divided into two horizontally, the pixels beyond the picture boundary are divided. If divided, restricts the horizontal division of a block into two parts.

[0143] Now, since it is a block at the bottom right corner of the screen, all combinations of block divisions In this case, pixels beyond the picture boundary are divided. If division is restricted (S1302: YES), the block is not divided (S1314).

[0144] Here, a specific example will be explained using Figure 21. Figure 21(a) and Figure 21(b) show the , which shows all combinations of block divisions for the treeblock at the edge of In both Figure 21(a) and Figure 21(b), the pixel beyond the picture boundary is divided. If you restrict block division for all combinations (S1302: YES), Do not split the block (S1314).

[0145] Figure 21(c) shows the tree block divided into four parts at the same picture boundary as Figure 21(a). This is an example where each of the four divided blocks is divided into blocks 601, 602, 603, and 604 in z-scan order. 03,604. Each step of the block division will be explained with reference to FIG. 21(c).

[0146] FIG. 21(c) is divided into blocks by the following steps. First, in the process of FIG. The image is divided into tree blocks (S1000), and each tree block is divided into four parts. (S1001: YES), and divide the tree block into four (S1002). For each of the blocks 601, 602, 603, and 604, the process shown in FIG. 7 is performed. .

[0147] In FIG. 7, it is determined that the inside of the divided block is not to be divided into four again (S1101: If the block is divided into two or three parts (S1103: YES), the block is divided into two or three parts (S1104: NO). The process of dividing into two or three parts in 7 (S1104 to S1110) is similar to the process in Fig. 20. Replaced.

[0148] In FIG. 20, block division is restricted for all combinations of block division. It is determined whether or not the block is currently at the bottom right corner of the screen (S1901). For all combinations of block division, pixels beyond the picture boundary are divided. If you want to restrict block division in all combinations (S1302: YES), No division is performed (S1314). This completes the block division process shown in FIG.

[0149] Figure 21(d) shows a tree block divided into four parts at a different picture boundary than Figure 21(c). This example shows a complex block division that combines the above-mentioned embodiments. Of the four divided blocks, blocks 602 and 603 are further divided into blocks. After dividing the tree block into four (S1002), each of the four divided blocks is Each step of block division will be explained for each of 01, 602, 603, and 604.

[0150] Block 601 performs the process shown in FIG. 7. In FIG. 7, the inside of the divided block is reconstructed. It is determined that the block is not divided into four parts (S1101: NO), and the block is not divided into two or three parts. The block 601 is not divided into blocks (S1103: NO). The block division process ends (S1111).

[0151] Block 602 performs the process shown in FIG. 7. In FIG. 7, the inside of the divided block is reconstructed. It is determined that the block is not divided into four (S1101: NO), and the block is divided into two or three. (S1103: YES) 4 to S1110) are replaced with the processing of FIG.

[0152] In FIG. 20, block division is restricted for all combinations of block division. If the block is divided into two in the horizontal direction, the pixel count is determined (S1901). Since pixels beyond the block boundary are split, horizontal splitting is limited. If the block is divided into two vertically, the pixels beyond the picture boundary will not be divided. Therefore, the vertical division is not restricted. Therefore, block division is not restricted (S1302: NO). (S1303: YES). In other words, the block is divided into two vertically (S13 08). Eventually, block 602 is vertically divided by division 612.

[0153] Block 603 performs the process shown in FIG. 7. In FIG. 7, the inside of the divided block is reconstructed. It is determined that the block is not divided into four (S1101: NO), and the block is divided into two or three. (S1103: YES) 4 to S1110) are replaced with the processing of FIG.

[0154] In FIG. 18, for all combinations of block division, the position beyond the picture boundary is The number of pixels in the block division is counted (S1651). There are two types: one where the block is divided into two horizontally, and one where the block is divided into two vertically. Now, in block 603, as shown in Figure 17(a) and Figure 17(b), The number of pixels beyond the pixel boundary is greatest when dividing horizontally. When a block is divided into two, the number of pixels beyond the picture boundary is the maximum. The dividing direction is determined to be horizontal (S1652). Then, the blocks are divided in the determined dividing direction. The block 603 is divided into two parts (S1654). As a result, the block 603 is divided horizontally by the division 623.

[0155] Block 604 performs the process shown in FIG. 7. In FIG. 7, the inside of the divided block is reconstructed. It is determined that the block is not divided into four (S1101: NO), and the block is divided into two or three. (S1103: YES) 4 to S1110) are replaced with the processing of FIG.

[0156] In FIG. 20, block division is restricted for all combinations of block division. Now, the block is divided into two parts horizontally by division 624 (S1901). When dividing, the pixels beyond the picture boundary are divided, so the horizontal division is In addition, when the block is divided into two vertically by the division 614, the picture Vertical division is limited because pixels beyond the block boundary are divided. For all combinations of block division, pixels beyond the picture boundary are divided. If you want to restrict block division for all combinations (S1302: YES), divide the block. Do not divide (S1314).

[0157] By controlling this block division, the blocks at the edge of the screen are divided into appropriate sizes and shapes. This allows for improved coding efficiency. This allows for block division suitable for

[0158] In all the embodiments described above, the object of block division control is picture boundaries. This means that an arbitrary boundary is defined and the position beyond it is blocked. It is also possible to control the block division. The pixels with high importance are determined, and the positions beyond the boundary are used as the targets for controlling block division. Furthermore, the position beyond any boundary is not necessarily the bottom or right edge of the screen, but may be the top edge of the screen. It can be the left edge or the right edge, or it doesn't have to be the edge. In that case, you can place a block even if it's not on the edge of the screen. The image can be divided into appropriate sizes and shapes, and the coding efficiency can be improved.

[0159] All of the above-described embodiments may be combined in multiple ways.

[0160] In all the embodiments described above, the coded bitstream output by the image coding device The stream is specified so that it can be decoded according to the encoding method used in the embodiment. The encoded bitstream can be stored on HDD, SSD, flash drive, Provided by recording it on a computer-readable recording medium such as flash memory or optical disk. It may be provided by a server via a wired or wireless network. Therefore, an image decoding device corresponding to this image encoding device can use this specific data regardless of the providing means. It is possible to decode coded bitstreams in the data format.

[0161] In order to exchange coded bitstreams between an image coding device and an image decoding device, When a wired or wireless network is used, the data format appropriate for the transmission mode of the communication path In this case, the image coding device may convert the output bitstream into The coded bit stream is converted into coded data in a data format suitable for the transmission mode of the communication channel. a transmitting device that converts the encoded data into a digital signal and transmits it to a network; and a receiving device for restoring the encoded bit stream to a coded bit stream and supplying the coded bit stream to the image decoding device.

[0162] The transmitting device includes a memory for buffering the coded bit stream output by the image coding device. a packet processing unit for packetizing the coded bit stream; and a transmitting unit for transmitting the packetized encoded data. a receiving unit that receives packetized coded data via a packetizer; and a memory for packet processing of the coded data to generate a coded bit stream. and a packet processing unit for providing the packet to an image decoding device.

[0163] In order to exchange coded bitstreams between an image coding device and an image decoding device, When a wired or wireless network is used, in addition to the transmitting device and the receiving device, Even if a relay device is provided to receive the coded data transmitted by the transmitting device and supply it to the receiving device, The relay device has a receiving section that receives packetized coded data transmitted from the transmitting device. a memory for buffering the received coded data; and a transmitting unit for transmitting the packetized coded data to the network. a reception packet processing unit that processes the received data into packets to generate an encoded bit stream; a recording medium for storing the encoded bitstream and a transmission medium for packetizing the encoded bitstream. The communication packet processing unit may also include a communication packet processing unit.

[0164] In addition, by adding a display unit for displaying the image decoded by the image decoding device to the configuration, In this case, the display unit may be a device that displays the decoded image signal generated by the decoded image signal superimposing unit 205. The decoded image signal stored in the decoded image memory 206 is read out and displayed on the screen.

[0165] In addition, by adding an imaging unit to the configuration and inputting the captured image to the image encoding device, In this case, the imaging unit inputs the captured image signal to the block division unit 101. To exert effort.

[0166] FIG. 22 shows an example of the hardware configuration of the encoding / decoding device of the present application. The present invention includes the configurations of an image encoding device and an image decoding device according to the embodiments of the present invention. The encoding / decoding device 9000 includes a CPU 9001, a codec IC 9002, an I / O interface interface 9003, memory 9004, optical disk drive 9005, network interface The interface 9006 and the video interface 9009 are connected to each other via a bus 9010. are connected by

[0167] The image encoding unit 9007 and the image decoding unit 9008 are typically implemented by a codec IC 9002 and The image encoding process of the image encoding device according to the embodiment of the present invention is implemented as an image encoding The image decoding process is executed by the decoding unit 9007 in the image decoding device according to the embodiment of the present invention. The encoding process is performed by the image encoding unit 9007. The I / O interface 9003 For example, a USB interface can be used to connect an external keyboard 9104 and mouse 9 The CPU 9001 receives input via the I / O interface 9003. The encoding / decoding device 9 executes the operation desired by the user based on the user's operation. 000. As a user operation using the keyboard 9104, mouse 9105, etc. The CODEC allows you to select whether to perform encoding or decoding functions, set the encoding quality, and There are input / output destinations for frames, input / output destinations for images, etc.

[0168] When the user desires to play back images recorded on the disk recording medium 9100 The optical disc drive 9005 reads the encoded video from the inserted disc recording medium 9100. The read encoded stream is sent to the decoder via bus 9010. The image decoding unit 9008 of the block IC 9002 receives the coded image data and sends it to the image decoding unit 9008. The image decoding process in the image decoding device according to the embodiment of the present invention is performed on the bit stream. The decoded image is then displayed on an external monitor 9103 via a video interface 9009. The encoding / decoding device 9000 also includes a network interface 9006. , and connects to an external distribution server 9106 and a mobile terminal 9107 via a network 9101. The user can change the image recorded on the disk recording medium 9100 and use it on the distribution server. If you want to play back the images recorded on the server 9106 or the mobile terminal 9107, The network interface 9006 receives the code from the input disk recording medium 9100. Instead of reading the encoded bitstream, the network 9101 Also, when the user desires to play back the images recorded in the memory 9004, In this case, the embodiment of the present invention is applied to the coded stream recorded in the memory 9004. The image decoding device according to the present invention performs image decoding processing.

[0169] The user captures an image using an external camera 9102 and encodes it into memory 9004. When an operation is desired, the video interface 9009 receives an image from the camera 9102. The image data is input and sent to the image encoding unit 9007 of the codec IC 9002 via the bus 9010. The image encoding unit 9007 encodes the image input via the video interface 9009. The image encoding process is performed in the image encoding device according to the embodiment of the present invention, and the encoded bits are The encoded bitstream is then sent to the memory via bus 9010. The user changes the memory 9004 and codes it on the disk recording medium 9100. If it is desired to record the encrypted stream, the optical disc drive 9005 may be The encoded stream is written to the inserted disc recording medium 9100.

[0170] A hardware configuration that has an image encoding device but does not have an image decoding device, or a hardware configuration that has an image decoding device However, it is also possible to realize a hardware configuration that does not include an image coding device. The hardware configuration is, for example, a codec IC 9002, an image encoding unit 9007, or This is realized by replacing the image decoding unit 9008 with the image decoding unit 9009.

[0171] The above encoding and decoding processes are carried out by hardware-based transmission, storage, and reception devices. It can be realized as a ROM (read only memory) or flash memory. firmware stored in memory, etc., and software in computers, etc. The firmware program and software program may be implemented in a computer. The information may be provided by recording it on a computer-readable recording medium, or by providing it over a wired or wireless network. It can be provided from a server via a network, or it can be provided as terrestrial or satellite digital broadcasting data. It may also be provided as a digital broadcast.

[0172] The present invention has been described above based on the embodiments. The embodiments are merely examples, and the respective structures thereof are not intended to be limiting. The fact that various variations are possible in the combination of components and treatment processes, and that such variations It will be understood by those skilled in the art that the embodiments are within the scope of the present invention. [Explanation of symbols]

[0173] 100 image encoding device, 101 block division unit, 102 predicted image generation unit, 103 residual signal generation unit, 104 orthogonal transformation and quantization unit, 105 encoding unit, 10 6 inverse quantization and inverse orthogonal transformation unit, 107 decoded image signal superposition unit, 108 decoded image memo 200 image decoding device, 201 decoding unit, 202 block division unit, 2 03 inverse quantization and inverse orthogonal transformation unit, 204 predicted image generation unit, 205 decoded image signal overlap Folding section, 206 decoded image memory.

Claims

1. An image decoding device that decodes an image in units of divided blocks, a decoding unit that decodes block division information including a first flag indicating whether a first block is to be divided into four, a second flag indicating whether the first block is to be divided into horizontal or vertical division lines, and a third flag indicating whether the first block is to be divided into two or three; a block dividing unit that recursively divides the image into rectangles of a predetermined size based on the block division information to generate the first blocks, The block division unit a quartering unit that divides the first block in the recursive division into four halves in both the horizontal and vertical directions to generate four blocks as second blocks; a 2-3 division unit that divides the first block in the recursive division into two or three blocks in a horizontal or vertical direction to generate two or three blocks as a third block; the 2-3 division unit prohibits the division of the first block into two along the horizontal division line and permits the division of the first block into two along the vertical division line when, if the first block is divided into two along a horizontal division line, the divided third block extends beyond the right side of a picture boundary, and if the first block is divided into two along a vertical division line, the divided third block does not extend beyond the lower side of the picture boundary; the 2-3 division unit prohibits dividing the first block into two along the vertical division line and permits dividing the first block into two along the horizontal division line if, when the first block is divided into two along a vertical division line, the divided second block extends below a picture boundary, and when the first block is divided into two along a horizontal division line, the divided third block does not extend to the right side of the picture boundary; the 2-3 division unit prohibits dividing the first block into three if, when the first block is divided into three along a division line in a horizontal direction or a vertical direction, the divided third block exceeds a picture boundary; the 2-3 division unit prohibits dividing the tree block into two along a horizontal division line and a vertical division line when the tree block of 128x128 pixels obtained by dividing the image exceeds the lower right side of a picture boundary; the decoding unit, when decoding the first flag and determining that the first flag is false, does not decode the second flag under a condition that division of the target block by a horizontal or vertical division line is prohibited; the decoding unit does not decode the third flag under a condition that division of the first block into three by horizontal or vertical division lines is prohibited. An image decoding device comprising:

2. An image decoding method for decoding an image in units of divided blocks, comprising: a decoding step of decoding block division information including a first flag indicating whether the first block is to be divided into four, a second flag indicating whether the first block is to be divided into horizontal or vertical division lines, and a third flag indicating whether the first block is to be divided into two or three; a block division step of recursively dividing the image into rectangles of a predetermined size based on the block division information to generate the first blocks, The block division step includes: a quartering step of dividing the first block in the recursive division into four halves in both the horizontal and vertical directions to generate four blocks as second blocks; a 2-3 division step of dividing the first block in the recursive division into two or three blocks in a horizontal or vertical direction to generate two or three blocks as a third block; the 2-3 division step prohibits the first block from being divided into two by the horizontal division line and permits the first block from being divided into two by the vertical division line if the divided third block exceeds the right side of the picture boundary when the first block is divided into two by the horizontal division line and the divided third block does not exceed the lower side of the picture boundary when the first block is divided into two by the vertical division line; the 2-3 division step prohibits dividing the first block into two along the vertical division line and permits dividing the first block into two along the horizontal division line if, when the first block is divided into two along the vertical division line, the divided second block extends below the picture boundary, and when the first block is divided into two along the horizontal division line, the divided third block does not extend beyond the right side of the picture boundary; the 2-3 division step prohibits dividing the first block into three if the divided third block exceeds a picture boundary when the first block is divided into three by a division line in a horizontal or vertical direction; the 2-3 division step prohibits dividing the tree block into two along a horizontal division line and a vertical division line if the tree block of 128x128 pixels obtained by dividing the image exceeds the lower right side of the picture boundary; the decoding step includes, when the first flag is decoded and the first flag is false, not decoding the second flag under a condition that division of the target block by a division line in a horizontal direction or a vertical direction is prohibited; the decoding step does not decode the third flag under a condition that division of the first block into three by a division line in a horizontal or vertical direction is prohibited.

1. An image decoding method comprising:

3. An image decoding program for decoding an image in units of divided blocks, a decoding step of decoding block division information including a first flag indicating whether the first block is to be divided into four, a second flag indicating whether the first block is to be divided into horizontal or vertical division lines, and a third flag indicating whether the first block is to be divided into two or three; a block division step of recursively dividing the image into rectangles of a predetermined size based on the block division information to generate the first blocks; The block division step includes: a quartering step of dividing the first block in the recursive division into four halves in both the horizontal and vertical directions to generate four blocks as second blocks; a 2-3 division step of dividing the first block in the recursive division into two or three blocks in a horizontal or vertical direction to generate two or three blocks as a third block; the 2-3 division step prohibits the first block from being divided into two by the horizontal division line and permits the first block from being divided into two by the vertical division line if the divided third block exceeds the right side of the picture boundary when the first block is divided into two by the horizontal division line and the divided third block does not exceed the lower side of the picture boundary when the first block is divided into two by the vertical division line; the 2-3 division step prohibits dividing the first block into two along the vertical division line and permits dividing the first block into two along the horizontal division line if, when the first block is divided into two along the vertical division line, the divided second block extends below the picture boundary, and when the first block is divided into two along the horizontal division line, the divided third block does not extend beyond the right side of the picture boundary; the 2-3 division step prohibits dividing the first block into three if the divided third block exceeds a picture boundary when the first block is divided into three by a division line in a horizontal or vertical direction; the 2-3 division step prohibits dividing the tree block into two along a horizontal division line and a vertical division line if the tree block of 128x128 pixels obtained by dividing the image exceeds the lower right side of the picture boundary; the decoding step includes, when the first flag is decoded and the first flag is false, not decoding the second flag under a condition that division of the target block by a division line in a horizontal direction or a vertical direction is prohibited; the decoding step does not decode the third flag under a condition that division of the first block into three by a division line in a horizontal or vertical direction is prohibited. An image decoding program comprising:

4. An image encoding device that encodes an image in units of divided blocks, a block dividing unit that recursively divides the image into rectangles of a predetermined size to generate first blocks; an encoding unit that generates a bitstream by encoding block division information including a first flag indicating whether the first block is to be divided into four, a second flag indicating whether the first block is to be divided into a horizontal division line or a vertical division line, and a third flag indicating whether the first block is to be divided into two or three, The block division unit a quartering unit that divides the first block in the recursive division into four halves in both the horizontal and vertical directions to generate four blocks as second blocks; a 2-3 division unit that divides the first block in the recursive division into two or three blocks in a horizontal or vertical direction to generate two or three blocks as a third block; the 2-3 division unit prohibits the division of the first block into two along the horizontal division line and permits the division of the first block into two along the vertical division line when, if the first block is divided into two along a horizontal division line, the divided third block extends beyond the right side of a picture boundary, and if the first block is divided into two along a vertical division line, the divided third block does not extend beyond the lower side of the picture boundary; the 2-3 division unit prohibits dividing the first block into two along the vertical division line and permits dividing the first block into two along the horizontal division line if, when the first block is divided into two along a vertical division line, the divided second block extends below a picture boundary, and when the first block is divided into two along a horizontal division line, the divided third block does not extend to the right side of the picture boundary; the 2-3 division unit prohibits dividing the first block into three if, when the first block is divided into three by a division line in a horizontal direction or a vertical direction, the divided third block exceeds a picture boundary; the 2-3 division unit prohibits dividing the tree block into two along a horizontal division line and a vertical division line when the tree block of 128x128 pixels obtained by dividing the image exceeds the lower right side of a picture boundary; the encoding unit, when encoding the first flag and the first flag is false, does not encode the second flag under a condition that division of the target block by a division line in a horizontal direction or a vertical direction is prohibited; the encoding unit does not encode the third flag under a condition that division of the first block into thirds along a division line in a horizontal direction or a vertical direction is prohibited. An image encoding device comprising:

5. An image coding method for coding an image in units of divided blocks, comprising: a block division step of recursively dividing the image into rectangles of a predetermined size to generate first blocks; an encoding step of encoding block division information including a first flag indicating whether the first block is to be divided into four, a second flag indicating whether the first block is to be divided into a horizontal division line or a vertical division line, and a third flag indicating whether the first block is to be divided into two or three, to generate a bitstream; The block division step includes: a quartering step of dividing the first block in the recursive division into four halves in both the horizontal and vertical directions to generate four blocks as second blocks; a 2-3 division step of dividing the first block in the recursive division into two or three blocks in a horizontal or vertical direction to generate two or three blocks as a third block; the 2-3 division step prohibits the first block from being divided into two by the horizontal division line and permits the first block from being divided into two by the vertical division line if the divided third block exceeds the right side of the picture boundary when the first block is divided into two by the horizontal division line and the divided third block does not exceed the lower side of the picture boundary when the first block is divided into two by the vertical division line; the 2-3 division step prohibits dividing the first block into two along the vertical division line and permits dividing the first block into two along the horizontal division line if, when the first block is divided into two along the vertical division line, the divided second block extends below the picture boundary, and when the first block is divided into two along the horizontal division line, the divided third block does not extend beyond the right side of the picture boundary; the 2-3 division step prohibits dividing the first block into three if the divided third block exceeds a picture boundary when the first block is divided into three by a division line in a horizontal or vertical direction; the 2-3 division step prohibits dividing the tree block into two along a horizontal division line and a vertical division line if the tree block of 128x128 pixels obtained by dividing the image exceeds the lower right side of the picture boundary; the encoding step, when the first flag is encoded and the first flag is false, does not encode the second flag under a condition that division of the target block by a division line in a horizontal direction or a vertical direction is prohibited; the encoding step does not encode the third flag under a condition that division of the first block into three by a division line in a horizontal direction or a vertical direction is prohibited.

1. An image encoding method comprising:

6. An image encoding program for encoding an image in units of divided blocks, a block division step of recursively dividing the image into rectangles of a predetermined size to generate first blocks; an encoding step of encoding block division information including a first flag indicating whether the first block is to be divided into four, a second flag indicating whether the first block is to be divided into a horizontal division line or a vertical division line, and a third flag indicating whether the first block is to be divided into two or three, to generate a bitstream; The block division step includes: a quartering step of dividing the first block in the recursive division into four halves in both the horizontal and vertical directions to generate four blocks as second blocks; a 2-3 division step of dividing the first block in the recursive division into two or three blocks in a horizontal or vertical direction to generate two or three blocks as a third block; the 2-3 division step prohibits the first block from being divided into two by the horizontal division line and permits the first block from being divided into two by the vertical division line if the divided third block exceeds the right side of the picture boundary when the first block is divided into two by the horizontal division line and the divided third block does not exceed the lower side of the picture boundary when the first block is divided into two by the vertical division line; the 2-3 division step prohibits dividing the first block into two along the vertical division line and permits dividing the first block into two along the horizontal division line if, when the first block is divided into two along the vertical division line, the divided second block extends below the picture boundary, and when the first block is divided into two along the horizontal division line, the divided third block does not extend beyond the right side of the picture boundary; the 2-3 division step prohibits dividing the first block into three if the divided third block exceeds a picture boundary when the first block is divided into three by a division line in a horizontal or vertical direction; the 2-3 division step prohibits dividing the tree block into two along a horizontal division line and a vertical division line if the tree block of 128x128 pixels obtained by dividing the image exceeds the lower right side of the picture boundary; the encoding step, when the first flag is encoded and the first flag is false, does not encode the second flag under a condition that division of the target block by a division line in a horizontal direction or a vertical direction is prohibited; the encoding step does not encode the third flag under a condition that division of the first block into three by a division line in a horizontal direction or a vertical direction is prohibited.

1. An image encoding program comprising:

7. A storage method for generating a bitstream according to the image coding method of claim 5 and storing the bitstream on a recording medium.

8. A transmission method for generating a bitstream according to the image coding method of claim 5 and transmitting the bitstream.

Images