Image encoding device and method, and image decoding device and method
Patent Information
- Authority / Receiving Office
- HK · HK
- Patent Type
- Patents
- Current Assignee / Owner
- JVC KENWOOD CORP
- Filing Date
- 2024-03-07
- Publication Date
- 2026-07-10
AI Technical Summary
In the existing image encoding and decoding process, inappropriate segmentation of block size and shape leads to reduced encoding efficiency and increased processing volume.
An image encoding device and a decoding device are used to recursively divide an image into rectangular blocks of a specified size through a block segmentation unit, and the segmentation is restricted to continuous division in the same direction, including four-segmentation and two-segmentation, to prevent the block shape from being too long and thin, and to optimize the encoding and decoding process of block segmentation information.
It improves coding efficiency, reduces processing volume, optimizes memory area requirements for intra-frame and inter-frame prediction, and simplifies encoding and decoding processes.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[0001] This application is a divisional application of the invention patent application with application number 201780063661.0, application date June 13, 2017, entitled "Image Encoding Apparatus and Method, Image Decoding Apparatus and Method, and Storage Medium". Technical Field
[0002] This invention relates to a technique for segmenting an image into blocks and encoding and decoding the segmented blocks as units. Background Technology
[0003] In image encoding and decoding, the image is divided into blocks, which are sets of pixels of a specified number, and encoding and decoding are performed on a block-by-block basis. By performing appropriate block segmentation, the efficiency of intra-frame prediction, inter-frame prediction, orthogonal transform, entropy coding, etc., can be improved, resulting in improved coding efficiency.
[0004] Prior art literature
[0005] Patent documents
[0006] Patent Document 1: Japanese Patent Publication No. 2015-526008. Summary of the Invention
[0007] Encoding efficiency will decrease if blocks are not segmented into appropriate sizes and shapes. Furthermore, the amount of processing required in subsequent encoding and decoding will increase if blocks are not segmented into appropriate sizes and shapes.
[0008] This embodiment was made in view of the following situation, and its purpose is to provide a technique for improving coding efficiency by performing block segmentation that is suitable for image encoding and decoding.
[0009] To address the aforementioned problems, one aspect of this embodiment relates to an image encoding apparatus that segments an image into blocks and encodes the segments as units. The image encoding apparatus includes: a block segmentation unit (101) that recursively segments the image into rectangles of a predetermined size to generate encoding object blocks; and an encoding unit (105) that encodes the block segmentation information of the encoding object blocks. The block segmentation unit includes: a four-segmentation unit that performs four-segmentation on the recursively segmented object blocks in both the horizontal and vertical directions to generate four blocks; and a two-segmentation unit that performs two-segmentation on the recursively segmented object blocks in either the horizontal or vertical direction to generate two blocks. If the previous recursively segmented block was a two-segment, the two-segmentation unit prohibits segmenting the current recursively segmented object block in the same direction as the block was segmented in the previous recursively segmented block.
[0010] Another embodiment of this invention is also an image encoding device. This device divides an image into blocks and encodes each block as a unit. The image encoding device includes: a block segmentation unit (101) that recursively divides the image into rectangles of a predetermined size to generate encoding object blocks; and an encoding unit (105) that encodes the block segmentation information of the encoding object blocks. The block segmentation unit includes: a four-segmentation unit that divides the recursively segmented object blocks into four parts horizontally and vertically to generate four blocks; and a two-segmentation unit that divides the recursively segmented object blocks into two parts horizontally or vertically to generate two blocks. If the previous recursive segmentation was two parts and the previous recursive segmentation was four parts, the block segmentation unit prohibits further segmentation of the object blocks.
[0011] Another embodiment of this invention is an image encoding method. This method segments an image into blocks and encodes each block as a unit. The image encoding method includes: a block segmentation step, recursively segmenting the image into rectangles of a predetermined size to generate encoding object blocks; and an encoding step, encoding the block segmentation information of the encoding object blocks. The block segmentation step includes: a four-segmentation step, dividing the recursively segmented object blocks into four parts horizontally and vertically to generate four blocks; and a two-segmentation step, dividing the recursively segmented object blocks into two parts horizontally or vertically to generate two blocks. In the two-segmentation step, if the previous recursively segmented block was a two-segment, it is prohibited to segment the object blocks in the current recursive segment in the same direction as the direction in which the block was segmented in the previous recursively segmented block.
[0012] Another embodiment of this invention is an image decoding apparatus. This apparatus segments an image into blocks and decodes the images on a block-by-block basis. The image decoding apparatus includes: a decoding unit (201) that decodes block segmentation information obtained from segmenting the image; and a block segmentation unit (202) that generates a decoded object block based on the decoded recursive block segmentation information. The block segmentation unit includes: a four-segmentation unit that divides the recursively segmented object block into four blocks in both the horizontal and vertical directions to generate four blocks; and a two-segmentation unit that divides the recursively segmented object block into two blocks in either the horizontal or vertical direction to generate two blocks. If the previous recursive segmentation resulted in two segments, the decoding unit does not decode a flag indicating whether the object block in the current recursive segmentation is segmented in the same direction as the block was segmented in the previous recursive segmentation.
[0013] Another embodiment of this invention is also an image decoding device. This device segments an image into blocks and decodes the images on a block-by-block basis. The image decoding device includes: a decoding unit (201) that decodes the block segmentation information of the blocks obtained from segmenting the image; and a block segmentation unit (202) that generates a decoding object block based on the decoded recursive block segmentation information. The block segmentation unit includes: a four-segmentation unit that divides the recursively segmented object block into four blocks in both the horizontal and vertical directions to generate four blocks; and a two-segmentation unit that divides the recursively segmented object block into two blocks in either the horizontal or vertical direction to generate two blocks. In the case where the previous recursive segmentation was two segments and the previous recursive segmentation was four segments, the decoding unit does not decode a flag indicating whether to further segment the object block.
[0014] Another embodiment of this method is an image decoding method. This method segments an image into blocks and decodes the images on a block-by-block basis. The image decoding method includes: a decoding step, which decodes the block segmentation information of the segments obtained from the segmented image; and a block segmentation step, which generates a decoded object block based on the decoded recursive block segmentation information. The block segmentation step includes: a four-segmentation step, which divides the object block in the recursive segmentation into four blocks in both the horizontal and vertical directions to generate four blocks; and a two-segmentation step, which divides the object block in the recursive segmentation into two blocks in either the horizontal or vertical direction to generate two blocks. In the decoding step, if the previous recursive segmentation resulted in two segments, a flag indicating whether the object block in the current recursive segmentation is segmented in the same direction as the block was segmented in the previous recursive segmentation is not decoded.
[0015] Furthermore, any combination of the above-mentioned constituent elements, or the content of this embodiment transformed into methods, apparatus, systems, recording media, computer programs, etc., is also valid as a form of this embodiment.
[0016] According to this embodiment, it is possible to provide image encoding and decoding that can perform block segmentation suitable for image encoding and decoding, improve encoding efficiency, and reduce processing volume. Attached Figure Description
[0017] Figure 1 This is a structural diagram of the image encoding apparatus according to the first embodiment;
[0018] Figure 2 This is a structural diagram of the image decoding apparatus according to the first embodiment;
[0019] Figure 3 This is a flowchart illustrating the segmentation of tree blocks and the segmentation within tree blocks;
[0020] Figure 4This is a diagram illustrating the case where the input image is segmented into tree blocks;
[0021] Figure 5 This is a diagram illustrating the z-scan.
[0022] Figure 6 It is a diagram that divides the tree block into four parts in both the horizontal and vertical directions;
[0023] Figure 7 It is a diagram that divides a tree block into two parts horizontally;
[0024] Figure 8 It is a diagram that divides a tree block into two parts vertically;
[0025] Figure 9 This is a flowchart illustrating the processing of each segment after dividing a tree block into four parts horizontally and vertically.
[0026] Figure 10 This is a flowchart illustrating the processing of each segment after horizontally dividing a tree block into two parts;
[0027] Figure 11 This diagram illustrates the further subdivision of a tree block after two horizontal divisions.
[0028] Figure 12 This is a flowchart illustrating the processing of each segment after a tree block is vertically divided into two parts.
[0029] Figure 13 This diagram illustrates the further division of a tree block after two vertical divisions.
[0030] Figure 14 This is a diagram illustrating an example of syntax related to block segmentation in the first embodiment;
[0031] Figure 15 This is a diagram illustrating intra-frame prediction;
[0032] Figure 16 This is a diagram illustrating inter-frame prediction;
[0033] Figure 17 This is a diagram illustrating an example of syntax related to block segmentation in the second embodiment;
[0034] Figure 18 This is a diagram illustrating another example of the syntax related to block segmentation in the second embodiment;
[0035] Figure 19 This is a diagram illustrating an example of syntax related to block segmentation in the third embodiment;
[0036] Figure 20This diagram illustrates the scenario where the interior of a block that has been divided in the horizontal or vertical direction is further divided in the same direction.
[0037] Figure 21 This is a diagram illustrating an example of syntax related to block segmentation in the fourth embodiment;
[0038] Figure 22 This is a diagram showing the division of a tree block into four parts within each of the two subdivisions. Detailed Implementation
[0039] The embodiments of the present invention provide an image encoding technique that segments an image into rectangular blocks and encodes / decodes the segmented blocks.
[0040] (First Embodiment)
[0041] The image encoding apparatus 100 and image decoding apparatus 200 according to Embodiment 1 of the present invention will be described. In Embodiment 1, when block segmentation is performed recursively, segmentation is restricted to be performed continuously in the same direction.
[0042] Figure 1 This is a structural diagram of the image encoding apparatus 100 according to the first embodiment. Here, in Figure 1 Only the data stream related to the image signal is shown. Each component provides additional information other than the image signal, such as motion vectors or prediction patterns, to the coded bit string generation unit 105 to generate corresponding coded data. The data stream related to the additional information is not shown.
[0043] The block segmentation unit 101 segments the image into encoding target blocks, which are processing units for encoding, and provides the image signals within the encoding target blocks to the residual signal generation unit 103. In addition, the block segmentation unit 101 provides the image signals of the encoding target blocks to the prediction image generation unit 102 in order to evaluate the consistency of the predicted image.
[0044] The block segmentation unit 101 recursively segments the image into rectangles of a predetermined size to generate coded object blocks. The block segmentation unit 101 includes: a four-segmentation unit that divides the recursively segmented object blocks into four parts horizontally and vertically to generate four blocks; and a two-segmentation unit that divides the recursively segmented object blocks into two parts horizontally or vertically to generate two blocks. Detailed operation of the block segmentation unit 101 will be described later.
[0045] The prediction image generation unit 102 generates a prediction image signal by performing intra-frame prediction or inter-frame prediction based on a prediction mode using the decoded image signal provided from the decoded image memory 108. Image signals within the encoded target block provided from the block segmentation unit 101 are used for evaluation of intra-frame and inter-frame prediction. In intra-frame prediction, a prediction image signal is generated using the image signal of the encoded target block provided from the block segmentation unit 101 and the image signals of surrounding encoded blocks that are close to the encoded target block in the same image as the encoded target block provided from the decoded image memory 108. In inter-frame prediction, for the image signal of the encoded target block provided from the block segmentation unit 101, an encoded image stored in the decoded image memory 108 that is located before or after the image (encoded image) containing the encoded target block in the time series is used as a reference image. Block consistency evaluation, such as block matching, is performed between the encoded image and the reference image to determine the motion vector representing the motion amount. Based on this motion amount, motion compensation is performed according to the reference image to generate a prediction image signal. The prediction image generation unit 102 provides the generated prediction image signal to the residual signal generation unit 103.
[0046] The residual signal generation unit 103 performs a subtraction operation on the encoded image signal and the prediction signal generated by the prediction image generation unit 102 to generate a residual signal, which is then provided to the orthogonal transformation / quantization unit 104.
[0047] The quadrature transform / quantization unit 104 performs quadrature transform / quantization on the residual signal provided from the residual signal generation unit 103, and provides the quadrature transform / quantization residual signal to the encoded bit string generation unit 105 and the inverse quantization / inverse quadrature transform unit 106.
[0048] The encoded bit string generation unit 105 generates an encoded bit string for the residual signal after orthogonal transformation / quantization provided by the orthogonal transformation / quantization unit 104. Additionally, the encoded bit string generation unit 105 generates corresponding encoded bit strings for additional information such as motion vectors, prediction modes, and block segmentation information.
[0049] The inverse quantization / inverse quadrature transformation unit 106 performs inverse quantization / inverse quadrature transformation on the residual signal obtained from the quadrature transformation / quantization unit 104. The residual signal after inverse quantization / inverse quadrature transformation is then provided to the decoded image signal overlay unit 107.
[0050] The decoded image signal overlay unit 107 overlays the predicted image signal generated by the predicted image generation unit 102 with the residual signal that has undergone inverse quantization and inverse quadrature transformation by the inverse quantization / inverse quadrature transformation unit 106 to generate a decoded image, which is then stored in the decoded image memory 108. Additionally, filtering processes to reduce block distortion caused by encoding are sometimes applied to the decoded image, and this is also stored in the decoded image memory 108.
[0051] Figure 2 This is a structural diagram of the image decoding apparatus 200 according to Embodiment 1. Here, in Figure 2 Only the data stream related to the image signal is shown. The bit string decoding unit 201 provides additional information other than the image signal, such as motion vectors or prediction modes, to each component and uses it for corresponding processing, but the data stream related to the additional information is not shown.
[0052] The bit string decoding unit 201 decodes the provided encoded bit string and provides the residual signal after orthogonal transformation / quantization to the block segmentation unit 202.
[0053] The block segmentation unit 202 determines the shape of the decoding target block based on the decoded block segmentation information, and provides the residual signal after orthogonal transformation / quantization of the determined decoding target block to the inverse quantization / inverse orthogonal transformation unit 203.
[0054] The block segmentation unit 202 recursively segments the image into rectangles of a predetermined size based on the decoded block segmentation information, generating decoded object blocks. The block segmentation unit 202 includes: a four-segmentation unit that divides the recursively segmented object blocks into four blocks in both the horizontal and vertical directions to generate four blocks; and a two-segmentation unit that divides the recursively segmented object blocks into two blocks in either the horizontal or vertical direction to generate two blocks. Detailed operation of the block segmentation unit 202 will be described later.
[0055] The inverse quantization / inverse quadrature transformation unit 203 performs inverse quadrature transformation and inverse quantization on the provided quadrature transformation / quantization residual signal to obtain the inverse quadrature transformation / inverse quantization residual signal.
[0056] The prediction image generation unit 204 generates a prediction image signal based on the decoded image signal provided from the decoded image memory 206, and provides it to the decoded image signal overlay unit 205.
[0057] The decoded image signal overlay unit 205 generates and outputs a decoded image signal by overlaying the predicted image signal generated by the predicted image generation unit 204 and the residual signal after inverse quadrature transformation / inverse quantization performed by the inverse quantization / inverse quadrature transformation unit 203, and stores it in the decoded image memory 206. Additionally, filtering processes to reduce block distortion caused by encoding are sometimes performed on the decoded image, and this is also stored in the decoded image memory 206.
[0058] The operation of the block segmentation unit 101 of the image encoding device 100 will be described in detail. Figure 3 This is a flowchart illustrating the segmentation of tree blocks and the segmentation within tree blocks.
[0059] First, the input image is divided into tree blocks of a specified size (S1000). For example, the tree block is set to 128 pixels × 128 pixels. However, the tree block is not limited to 128 pixels × 128 pixels; any size and shape can be used as long as it is rectangular. Alternatively, the size and shape of the tree block can be fixed between the encoding and decoding devices, or it can be configured such that the encoding device determines and records it in the encoded bitstream, and the decoding device uses the recorded block size. Figure 4 This indicates the case where the input image is segmented into tree blocks. The tree blocks are encoded and decoded in the raster scan order, i.e., from left to right and from top to bottom.
[0060] The interior of the tree block is further divided into rectangular blocks. The interior of each tree block is encoded and decoded in a z-scan order. Figure 5 This indicates the z-scan order. In a z-scan, encoding and decoding are performed in the order of top left, top right, bottom left, and bottom right. The internal segmentation of a tree block can be either quad-segmentation or bi-segmentation. Quad-segmentation involves splitting in both the horizontal and vertical directions. Bi-segmentation involves splitting in either the horizontal or vertical direction. Figure 6 It is a diagram that divides a tree block into four parts in both the horizontal and vertical directions. Figure 7 It is a diagram that divides a tree block into two parts horizontally. Figure 8 It is a diagram that divides a tree block into two parts vertically.
[0061] Refer again Figure 3 Determine whether to perform four-part division of the tree block in both horizontal and vertical directions (S1001).
[0062] If it is determined that the tree block's interior needs to be divided into four parts (S1001: Yes), the tree block's interior is divided into four parts (S1002), and each of the four parts in the horizontal and vertical directions is processed (S1003). The further division of the four-part blocks will be described later. Figure 9 ).
[0063] If it is determined that the internal structure of the tree block will not be divided into four parts (S1001: No), then it is determined whether the internal structure of the tree block will be divided into two parts (S1004).
[0064] If it is determined that the tree block is to be divided into two parts (S1004: Yes), determine whether to set the direction of the two parts to the horizontal direction (S1005).
[0065] If the direction of the two divisions is determined to be horizontal (S1005: Yes), the tree block is divided into two parts horizontally (S1006), and each part of the horizontally divided block is processed (S1007). The further division of the horizontally divided block will be described later. Figure 10 ).
[0066] If the direction of the two divisions is determined to be vertical rather than horizontal (S1005: No), the tree block is divided into two vertically (S1008), and each of the vertically divided blocks is processed (S1009). The further division of horizontally divided blocks will be described later. Figure 11 ).
[0067] If it is determined that the internal structure of the tree block will not be divided into two parts (S1004: No), the block division process ends without dividing the internal structure of the tree block (S1010).
[0068] Next, use Figure 9 The flowchart illustrates the processing of each block after the tree block is divided into four parts in both the horizontal and vertical directions.
[0069] Determine whether to further divide the block into four parts in both the horizontal and vertical directions (S1101).
[0070] If it is determined that the block will be divided into four parts again (S1101: Yes), the block will be divided into four parts again (S1102), and each of the four parts of the block in the horizontal and vertical directions will be processed (S1103).
[0071] If it is determined that the block should not be divided into four parts again (S1101: No), determine whether to divide the block into two parts (S1104).
[0072] If it is determined that the block is to be divided into two parts (S1104: Yes), determine whether to set the direction of the two parts to the horizontal direction (S1105).
[0073] If the direction of the two divisions is determined to be horizontal (S1105: Yes), the block is divided into two parts in the horizontal direction (S1106), and each part of the block divided in the horizontal direction is processed (S1107).
[0074] If the direction of the two divisions is determined to be vertical rather than horizontal (S1105: No), the block is divided into two parts in the vertical direction (S1108), and each part of the block divided in the vertical direction is processed (S1109).
[0075] If it is determined that the block should not be split into two parts (S1104: No), the block splitting process ends without splitting the block (S1110).
[0076] Execute on each of the four partitioned blocks Figure 9 The process is shown in the flowchart. The interior of the four-part block is also encoded and decoded in z-scan order.
[0077] Next, use Figure 10 The flowchart illustrates the processing of each block after horizontally dividing a tree block into two parts.
[0078] When a tree block is divided into two parts in the horizontal direction, for each part that is divided into two parts, it is first determined whether to divide the interior of the part into four parts in the horizontal and vertical directions (S1201).
[0079] If it is determined that the block is to be divided into four parts (S1201: Yes), the block is divided into four parts (S1202), and each part of the block divided into four parts in the horizontal and vertical directions is processed (S1203).
[0080] If it is determined that the block should not be divided into four parts (S1201: No), determine whether to divide the block into two parts again (S1204).
[0081] If it is determined that the block will be split into two parts again (S1204: Yes), the block will be split vertically (S1205), and each part of the block split vertically will be processed (S1206).
[0082] If it is determined that no further two-part division is required (S1204: No), the block division process ends without further division of the block's interior (S1207).
[0083] Figure 11 This describes the further subdivision of blocks after a horizontal split where the tree block is divided into two parts. Here, when the parent tree block is divided into two parts horizontally, the further subdivision of the subdivided blocks only allows vertical splits, which are automatically implemented vertically. Furthermore, when the parent tree block is divided into two parts, quadruple splits can be completely prohibited in the child blocks. This prevents blocks from being split in the same direction as the parent block, thus preventing horizontally elongated rectangular block splits and simplifying encoding / decoding processes.
[0084] Execute on each block that is divided into two in the horizontal direction Figure 10 The process is shown in the flowchart. The internal parts of the two divided blocks are also encoded and decoded in a top-to-bottom order.
[0085] Next, use Figure 12 The flowchart illustrates the processing of each block after it is vertically divided into two parts.
[0086] When a tree block is divided into two vertically, each of the two divided blocks first determines whether to divide the block into four parts in both the horizontal and vertical directions (S1301).
[0087] If it is determined that the block is to be divided into four parts (S1301: Yes), the block is divided into four parts (S1302), and each part of the block divided into four parts in the horizontal and vertical directions is processed (S1303).
[0088] If it is determined that the block should not be divided into four parts (S1301: No), determine whether to divide the block into two parts again (S1304).
[0089] If it is determined that two divisions will be performed again (S1304: Yes), the block is divided horizontally (S1305), and each of the blocks divided horizontally is processed (S1306).
[0090] If it is determined that no further two-part division is required (S1304: No), the block division process ends without further division of the block's interior (S1307).
[0091] Figure 13 This describes the further subdivision of a block when it is split vertically into two parts. Here, when the parent block is split vertically into two parts, the further subdivision of the split block only allows horizontal splits, which are automatically implemented horizontally. Furthermore, when the parent block is split into two parts, quadruple splits can be completely prohibited in the child blocks. This prevents blocks from being split in the same direction as the parent block, thus preventing block splits that are elongated and thinner in the vertical direction, simplifying encoding / decoding processes.
[0092] Perform on each block that is divided into two parts in the vertical direction Figure 12 The process is illustrated in the flowchart. The internal parts of the two divided blocks are also encoded and decoded in a left-right order.
[0093] Furthermore, the re-segmentation of the subdivided blocks when a tree block is segmented is explained, but the parent block may not be a tree block. For example, if a tree block (128×128) is divided into four parts, and the resulting four-part block (64×64) is further divided into four or two parts, the above processing should also be applied to the re-segmentation of the subdivided blocks.
[0094] Next, the operation of the block segmentation unit 202 of the image decoding apparatus 200 will be explained. Blocks are segmented in the same processing order as the block segmentation unit 101 of the image encoding apparatus 100. In the block segmentation unit 101 of the image encoding apparatus 100, a block segmentation pattern is selected, and the selected block segmentation information is output. In contrast, the syntax structure of the block segmentation unit 202 of the image decoding apparatus is different in the following ways: block segmentation is performed using the block segmentation information decoded from the encoded bitstream; and when decoding the block segmentation information from the encoded bitstream, information without options is not transmitted in the bitstream if further segmentation in the same direction is prohibited.
[0095] Figure 14 This section presents an example of the syntax (syntactic rules for encoding bitstreams) related to block segmentation in the first implementation. For internal segmentation of a tree block, a flag indicating whether to perform a quad-segmentation (4_division_flag) is first sent and received. If a quad-segmentation is performed (4_division_flag is 1), the tree block is quad-segmented and processing ends. Then, for the quad-segmented block, the process is repeated... Figure 14 The syntax shown further divides the internal structure. Without a four-part division (4_division_flag is 0), a flag indicating whether a two-part division (2_division_flag) is performed is sent and received. With a two-part division (2_division_flag is 1), a flag indicating the direction of the two divisions (2_division_direction) is also sent and received. 2_division_direction is 1 indicating a vertical division, and 2_division_direction is 0 indicating a horizontal division. Then, for the two-part blocks, the structure is again... Figure 14 The syntax shown further divides the block. Without performing two divisions (2_division_flag is 0), the processing ends without dividing the tree block.
[0096] This section explains the process of further subdividing the interior of blocks after four or two partitions. This subdivision of the block interior also utilizes... Figure 14 The syntax is shown, but it differs from the case of segmenting tree blocks in that there is a restriction on the segmentation direction when performing two segments. Specifically, when performing two segments on a tree block, and then further segmenting the interior of the resulting block, segmentation in the same direction as the initial two segments is prohibited. This prevents the resulting blocks from becoming elongated rectangles, thus preventing an increase in the memory area required for intra-frame or inter-frame prediction. Details regarding preventing an increase in memory area will be described later.
[0097] Alternatively, the number of divisions in the same direction can be counted, and divisions in the same direction can be restricted if a certain number of divisions are exceeded. For example, two divisions in the same direction can be allowed up to two times, but from the third time onwards, two divisions in the same direction can be prohibited.
[0098] exist Figure 14 In this example, a syntax is set to prioritize four-segmentation and send / receive information about whether to perform four-segmentation before information about whether to perform two-segmentation. Conversely, in the case of prioritizing two-segmentation, a syntax can also be set to send / receive information about whether to perform two-segmentation before information about whether to perform four-segmentation. This is because sending / receiving events that are more likely to occur based on probability first reduces the amount of code transmitted as a bitstream. That is, it is also possible to estimate in advance which is more likely to occur, four-segmentation or two-segmentation, and set a syntax to send / receive the more likely segmentation information first. For example, by sending / receiving whether four-segmentation or two-segmentation is prioritized in the image header information, the encoding device appropriately determines the priority segmentation number with higher coding efficiency, and the decoding device segments the interior of the tree block using a syntax based on the selected priority segmentation number.
[0099] In the image encoding apparatus 100 and the image decoding apparatus 200, segmented blocks are used to perform intra-frame prediction and inter-frame prediction. Both intra-frame prediction and inter-frame prediction involve copying pixels from memory.
[0100] Figure 15 (a)~ Figure 15 (d) represents an example of intra-frame prediction. Figure 15 (a) and Figure 15 (b) represents the prediction direction and mode number of the intra-frame prediction. Intra-frame prediction is as follows: Figure 15 (c) and Figure 15 As shown in (d), a predicted image of the encoded / decoded block is generated by copying pixels from already encoded / decoded pixels that are close to the encoded / decoded block. In intra-frame prediction, since the generation of encoded / decoded pixels is repeated from the generation of the predicted image on a block-by-block basis, the processing order becomes a sequence on a block-by-block basis. The smaller the block is divided, the greater the overall processing load. In addition, the more elongated the shape of the block is, the greater the processing of copying pixels from memory. Furthermore, since orthogonal transformations of the residual signals are performed during encoding / decoding, the more types of rectangle sizes there are, the more types of orthogonal transformations are required, resulting in an increase in circuit size. Therefore, when performing two divisions within the block, by limiting the two divisions to the same direction as the division method of the parent block, it is possible to prevent an increase in the memory area required for intra-frame prediction.
[0101] Figure 16An example of inter-frame prediction is shown. Inter-frame prediction generates a predicted image of the encoded / decoded object block by copying pixels from the pixels contained in the encoded / decoded image in block units. In inter-frame prediction, when copying pixels from the reference image in block units, it often becomes a structure that requires a management unit containing the necessary pixels for acquisition. Therefore, the smaller the block is divided, and the more elongated the shape of the block is, the greater the overall processing load. In addition, when performing fractional-precision motion compensation using interpolation filters on the reference image, it is necessary to copy several pixels plus pixels contained in the block. The smaller the block size, the greater the relative ratio of the additional pixels, and the greater the overall processing load. Therefore, when performing two divisions within the block, by limiting the two divisions to the same direction as the division direction of the parent block, it is possible to prevent an increase in the memory area required for inter-frame prediction.
[0102] (Second Implementation)
[0103] The image encoding apparatus and image decoding apparatus according to the second embodiment of the present invention will be described. In the second embodiment, the limitation on further subdividing the block interior when the block size is less than a predetermined size differs from that in the first embodiment; otherwise, the structure is the same as in the first embodiment. This prevents the overall processing load from increasing as the block interior is subdivided into smaller parts.
[0104] Figure 17 , Figure 18 This describes the syntax related to block segmentation in the second embodiment. It is different from the syntax of the first embodiment. Figure 14 The syntactic difference lies in the fact that initially, block splitting is only possible if the block size is larger than a specified size. Figure 17 In this case, blocks with more than 64 pixels can be divided into four or two segments.
[0105] Furthermore, when considering the difference in the number of pixels within blocks divided by four segments and two segments, such as Figure 18 As shown, quadruple splitting is allowed when the number of pixels within a block is greater than 64, and two-part splitting is allowed when the number of pixels within a block is greater than 32. This allows for high-precision control over the pixel count limit of the split blocks.
[0106] (Third Implementation)
[0107] The image encoding apparatus and image decoding apparatus according to the third embodiment of the present invention will be described. In the third embodiment, the limitation is that the block after being divided in the vertical direction is further divided in the vertical direction, which is different from the first embodiment; otherwise, the structure is the same as the first embodiment.
[0108] Typically, image pixel information is stored in one-dimensional memory according to the raster scan order. That is, in one-dimensional memory, horizontally oriented pixels are stored relatively close together, while vertically oriented pixels are stored relatively far apart. Therefore, horizontally oriented pixels are easy to access, but vertically oriented pixels are not. For example, in the case of a block of 16 pixels horizontally × 8 pixels vertically and a block of 8 pixels horizontally × 16 pixels vertically, the number of pixels is the same, but the memory range for storing pixels in the block of 8 pixels horizontally × 16 pixels vertically is larger than that in the block of 16 pixels horizontally × 8 pixels vertically. Therefore, when using motion compensation, pixel transfer requires more memory bandwidth.
[0109] Figure 19 This describes the syntax related to block segmentation in the third embodiment. It is different from the syntax of the first embodiment. Figure 14 The syntactic difference includes: it is forbidden to further divide the interior into two parts in the vertical direction only if the parent block is divided into two parts in the vertical direction.
[0110] Figure 20 This diagram illustrates a scenario where a block that has been divided horizontally or vertically is further subdivided in the same direction. For example... Figure 20 As shown, when the mother block is divided into two parts in the vertical direction, and then further divided into two parts internally, the horizontal division is automatically selected instead of the horizontal and vertical divisions.
[0111] (Fourth implementation)
[0112] The image encoding apparatus and image decoding apparatus according to the fourth embodiment of the present invention will be described. In the fourth embodiment, the difference from the first embodiment is that when the block is divided into two parts, and then the inner blocks of the divided parts are divided into four parts, further division of the inner blocks of the four-part division is prohibited. Otherwise, the structure is the same as that of the first embodiment.
[0113] Figure 21 Syntax related to block segmentation in the fourth embodiment. For example... Figure 22 As shown, after the parent block is divided into two parts, and the internal blocks of the divided blocks are further divided into four parts, 2_division_after_4_division_flag is 1, which prohibits all subsequent divisions.
[0114] This is because, in the case of a two-segment followed by a four-segment, since it's known that a four-segment wasn't chosen when the two-segment was selected, the likelihood of further block splitting after the two-segment followed by a four-segment is low. In such a case, choosing a four-segment from the beginning is sufficient. Furthermore, in the case of a two-segment followed by a four-segment, since the block shape has become rectangular, preventing further two-segmentation becomes complex. If subsequent block splitting is uniquely prohibited for blocks that have been four-segmented after two-segmentation, the process of determining whether block splitting is permissible becomes straightforward. By uniquely prohibiting subsequent block splitting for blocks that have been four-segmented after two-segmentation, the selection of whether or not to perform block splitting in the bitstream is eliminated, reducing the amount of code transmitted.
[0115] Alternatively, multiple block segmentation restriction methods from the first to the fourth embodiments can be combined.
[0116] The encoded bitstream of the image output by the image encoding device in the above embodiments has a specific data format so that it can be decoded according to the encoding method used in the embodiments, and the image decoding device corresponding to the image encoding device can decode the encoded bitstream of the specific data format.
[0117] To exchange encoded bitstreams between an image encoding device and an image decoding device, when using a wired or wireless network, the encoded bitstream can be converted into a data format suitable for the communication path before transmission. In this case, a transmitting device is provided that converts the encoded bitstream output by the image encoding device into encoded data in a data format suitable for the communication path and transmits it to the network; and a receiving device receives the encoded data from the network, restores it to the encoded bitstream, and provides it to the image decoding device.
[0118] The transmitting device includes: a memory for buffering the encoded bitstream output by the image encoding device; a packet processing unit for packetizing the encoded bitstream; and a transmitting unit for transmitting the packetized encoded data via a network. The receiving device includes: a receiving unit for receiving the packetized encoded data via a network; a memory for buffering the received encoded data; and a packet processing unit for packetizing the encoded data to generate an encoded bitstream and providing it to the image decoding device.
[0119] Alternatively, a display unit that displays the image decoded by the image decoding device can be added to the structure to serve as the display device. In this case, the display unit reads the decoded image signal generated by the decoded image signal overlay unit 205 and stored in the decoded image memory 206, and displays it on the screen.
[0120] Alternatively, the device can be configured as an imaging unit by adding an imaging unit to the structure and inputting the captured image into an image encoding device. In this case, the imaging unit inputs the captured image signal into the block segmentation unit 101.
[0121] The encoding and decoding related processes described above can be implemented using hardware transmission, storage, and reception devices, or through firmware stored in ROM (Read-Only Memory) or flash memory, or through computer software. These firmware or software programs can be provided by recording on a computer or other readable recording medium, provided from a server via wired or wireless networks, or provided as data broadcast via terrestrial wave or satellite digital broadcasting.
[0122] The present invention has been described above based on embodiments. These embodiments are illustrative, and various modifications can be made to the combination of the constituent elements and processing procedures. Those skilled in the art should understand that these modifications are also within the scope of the present invention.
[0123] Symbol Explanation
[0124] 100 Image encoding device, 101 Block segmentation unit, 102 Predictive image generation unit, 103 Residual signal generation unit, 104 Orthogonal transform / quantization unit, 105 Encoded bit string generation unit, 106 Inverse quantization / inverse orthogonal transform unit, 107 Decoded image signal overlay unit, 108 Decoded image memory, 200 Image decoding device, 201 Bit string decoding unit, 202 Block segmentation unit, 203 Inverse quantization / inverse orthogonal transform unit, 204 Predictive image generation unit, 205 Decoded image signal overlay unit, 206 Decoded image memory
[0125] Industrial availability
[0126] This invention enables the use of techniques for segmenting images into blocks and encoding and decoding the segmented blocks as units.
Claims
1. An image encoding apparatus for dividing an image into blocks and encoding the blocks as units, the image encoding apparatus being characterized in that it comprises: The block segmentation unit recursively segments the image into rectangles of a specified size to generate coded object blocks; The inter-frame prediction unit generates a prediction image signal using the image signal of the encoded object block provided by the block segmentation unit and the image signal of the encoded object block that exists in a picture different from the encoded object block provided by the decoded image memory. as well as The encoding department encodes the block segmentation information of the target block. The block segmentation portion includes: The four-part division process generates four blocks by dividing the object block in the recursive division into four parts in the horizontal and vertical directions. as well as The two-part division process splits the object block in the recursive division into two blocks in either the horizontal or vertical direction, thus generating two blocks. When the object block is smaller than a specified size, the two dividing sections restrict further division of the block's interior. When an object block is split into two parts, it is prohibited to split the object block in the same direction as the direction in which the parent block of the object block was split.
2. An image encoding method, comprising dividing an image into blocks and encoding the blocks as units, the image encoding method being characterized by comprising: The block segmentation step recursively segments the image into rectangles of a specified size to generate coded object blocks; The inter-frame prediction step generates a predicted image signal using the image signal of the encoded object block provided in the block segmentation step and the image signal of the encoded object block that exists in a picture different from the encoded object block provided from the decoded image memory. as well as The encoding step involves encoding the block segmentation information of the target block. The block segmentation step includes: The four-part division step divides the object block in the recursive division into four blocks in both the horizontal and vertical directions to generate four blocks. as well as The two-partitioning step divides the object block in the recursive partitioning process into two blocks in either the horizontal or vertical direction. In the two segmentation steps, if the object block is smaller than a specified size, further segmentation within the block is restricted. When an object block is split into two parts, it is prohibited to split the object block in the same direction as the direction in which the parent block of the object block was split.
3. An image decoding apparatus, which decodes images in units of blocks obtained by segmenting an image, characterized in that it comprises: The decoding unit decodes the block segmentation information of the blocks obtained from the segmented image; The block segmentation unit generates a decoded object block based on the decoded recursive block segmentation information; The inter-frame prediction unit generates a prediction image signal using the image signal of the decoded object block provided by the block segmentation unit and the image signal of the decoded object block that exists in a picture different from the decoded object block provided by the decoded image memory. as well as The block segmentation portion includes: The four-part division divides the object block in the recursive division into four blocks in the horizontal and vertical directions to generate four blocks. as well as The two-part division process splits the object block in the recursive division into two blocks in either the horizontal or vertical direction, thus generating two blocks. When the object block is smaller than a specified size, the two dividing sections restrict further division of the block's interior. When an object block is split into two parts, it is prohibited to split the object block in the same direction as the direction in which the parent block of the object block was split.
4. An image decoding method, wherein decoding is performed on a block-by-block basis obtained by segmenting an image, the image decoding method being characterized by comprising: The decoding step involves decoding the block segmentation information of the blocks obtained from the segmented image. The block segmentation step generates a decoded object block based on the decoded recursive block segmentation information; as well as The inter-frame prediction step generates a prediction image signal using the image signal of the decoded object block provided in the block segmentation step and the image signal of the decoded object block existing in a different image than the decoded object block provided from the decoded image memory. The block segmentation step includes: The four-part division step divides the object block in the recursive division into four blocks in both the horizontal and vertical directions to generate four blocks. as well as The two-partitioning step divides the object block in the recursive partitioning process into two blocks in either the horizontal or vertical direction. In the two segmentation steps, if the object block is smaller than a specified size, further segmentation within the block is restricted. When an object block is split into two parts, it is prohibited to split the object block in the same direction as the direction in which the parent block of the object block was split.
5. A method for storing a bitstream generated by the image encoding method according to claim 2 in a recording medium.
6. A transmission method for transmitting a bit stream generated by the image encoding method according to claim 2.