Image encoding apparatus and method, and image decoding apparatus and method

Abstract

This invention relates to an image encoding apparatus and method, and an image decoding apparatus and method. An image encoding apparatus is provided that divides an image into blocks and encodes the images in units of these blocks. A main signal block segmentation unit (101) divides the main signal of the image into rectangles of a predetermined size to generate main signal blocks. A secondary signal block segmentation unit (101) divides the secondary signal of the image into rectangles of a predetermined size to generate secondary signal blocks. A main signal prediction unit (102) predicts the main signal, and a secondary signal prediction unit (102) predicts the secondary signal. The secondary signal prediction unit (102) is capable of performing inter-component prediction of the secondary signal based on the encoded main signal, and the inter-component prediction is limited based on at least one of the size of the main signal block and the size of the secondary signal block.

Description

[0001] This application is a divisional application of the invention patent application with application number 201880020202.9, application date March 26, 2018, entitled "Image Encoding Apparatus and Method, Image Decoding Apparatus and Method, and Recording Medium". Technical Field

[0002] This invention relates to a technique for segmenting an image into blocks and encoding and decoding the segments as units. Background Technology

[0003] In image encoding and decoding, the image is divided into blocks, which are sets of a specified number of pixels, and encoding and decoding are performed on a block-by-block basis. By performing appropriate block segmentation, the coding efficiency of intra-frame prediction and inter-frame prediction is improved. In addition, in intra-frame prediction, the sub-signal is predicted from the decoded image of the main signal, thereby improving coding efficiency.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent document 1: Japanese Patent Application Publication No. 2013-90015. Summary of the Invention

[0007] The problem that the invention aims to solve

[0008] However, when predicting the secondary signal from the decoded image of the primary signal, the processing load increases, and a dependency relationship is created between the processing of the primary signal and the processing of the secondary signal, making parallel processing difficult.

[0009] The present invention was made in view of the following situation, and its object is to provide a technique for improving coding efficiency by performing block segmentation suitable for image encoding and decoding.

[0010] To address the aforementioned problems, one aspect of the present invention provides an image encoding apparatus that divides an image into blocks and encodes the blocks as units. The image encoding apparatus includes: a main signal block segmentation unit (101) that divides the main signal of the image into rectangles of a predetermined size to generate main signal blocks; a secondary signal block segmentation unit (101) that divides the secondary signal of the image into rectangles of a predetermined size to generate secondary signal blocks; a main signal prediction unit (102) that predicts the main signal; and a secondary signal prediction unit (102) that predicts secondary signals. The secondary signal prediction unit (102) is capable of inter-component prediction of secondary signals from the encoded main signal, and the inter-component prediction is limited based on at least one of the size of the main signal block and the size of the secondary signal block.

[0011] Another aspect of the present invention is an image encoding method. This method segments an image into blocks and encodes these blocks as units. The image encoding method includes: a main signal block segmentation step, which segments the main signal of the image into rectangles of a predetermined size to generate main signal blocks; a secondary signal block segmentation step, which segments the secondary signal of the image into rectangles of a predetermined size to generate secondary signal blocks; a main signal prediction step, which predicts the main signal; and a secondary signal prediction step, which predicts the secondary signal. The secondary signal prediction step is capable of inter-component prediction of the secondary signal from the encoded main signal, and the inter-component prediction is constrained based on at least one of the size of the main signal block and the size of the secondary signal block.

[0012] Another aspect of the present invention is an image decoding apparatus. This apparatus divides an image into blocks and decodes the image in units of these blocks. The image decoding apparatus includes: a main signal block segmentation unit (202) that divides the main signal of the image into rectangles of a predetermined size to generate main signal blocks; a secondary signal block segmentation unit (202) that divides the secondary signal of the image into rectangles of a predetermined size to generate secondary signal blocks; a main signal prediction unit (204) that predicts the main signal; and a secondary signal prediction unit (204) that predicts the secondary signal. The secondary signal prediction unit (204) is capable of inter-component prediction of the secondary signal from the decoded main signal, and the inter-component prediction is limited based on at least one of the size of the main signal block and the size of the secondary signal block.

[0013] Another aspect of the present invention is an image decoding method. This method segments an image into blocks and decodes the image on a block-by-block basis. The image decoding method includes: a main signal block segmentation step, which segments the main signal of the image into rectangles of a predetermined size to generate main signal blocks; a secondary signal block segmentation step, which segments the secondary signal of the image into rectangles of a predetermined size to generate secondary signal blocks; a main signal prediction step, which predicts the main signal; and a secondary signal prediction step, which predicts the secondary signal. The secondary signal prediction step is capable of inter-component prediction of the secondary signal from the decoded main signal, and the inter-component prediction is constrained based on at least one of the size of the main signal block and the size of the secondary signal block.

[0014] Furthermore, any combination of the above-mentioned constituent elements, or variations in the present invention as a method, apparatus, system, recording medium, computer program, etc., are also valid ways to implement the present invention.

[0015] According to the present invention, it is possible to provide image encoding and decoding that can perform block segmentation suitable for image encoding and decoding, improve encoding efficiency, and reduce processing volume. Attached Figure Description

[0016] Figure 1 This is a structural diagram of the image encoding apparatus according to the first embodiment;

[0017] Figure 2 This is a structural diagram of the image decoding apparatus according to the first embodiment;

[0018] Figure 3 This is a flowchart illustrating the segmentation of tree blocks and the segmentation within tree blocks;

[0019] Figure 4 This is a diagram representing the case where the input image is segmented into tree blocks;

[0020] Figure 5 This is a diagram illustrating the z-scan.

[0021] Figure 6 It is a diagram that divides a tree block into four parts in both the horizontal and vertical directions;

[0022] Figure 7 This is a diagram showing a tree block divided into two parts horizontally;

[0023] Figure 8 This is a diagram showing a tree block divided into two parts vertically.

[0024] Figure 9 This is a flowchart illustrating the processing of each segmented block when a tree block is divided into four parts in both the horizontal and vertical directions.

[0025] Figure 10 This is a flowchart illustrating the processing of each segmented block when a tree block is horizontally divided into two parts.

[0026] Figure 11 This diagram illustrates the further subdivision of a block after it has been divided into two parts in the horizontal direction.

[0027] Figure 12 This is a flowchart illustrating the processing of each segment after a tree block is vertically divided into two parts.

[0028] Figure 13 This diagram illustrates the further subdivision of a block after it has been divided into two parts in the vertical direction.

[0029] Figure 14 This is a diagram illustrating an example of syntax related to block segmentation in the first embodiment;

[0030] Figure 15 (a) to (d) are diagrams illustrating intra-frame prediction;

[0031] Figure 16 This is a diagram illustrating inter-frame prediction;

[0032] Figure 17(a) to (c) are diagrams illustrating the color difference format;

[0033] Figure 18 This is a diagram illustrating intra-frame prediction of luminance and chromatic aberration;

[0034] Figure 19 This is a flowchart illustrating intra-frame prediction of luminance and chromatic aberration;

[0035] Figure 20 (a) and (b) are diagrams illustrating the case where the size of the color difference block is larger than the size of the brightness block;

[0036] Figure 21 (a) and (b) are diagrams illustrating the case where the size of the color difference block is smaller than the size of the brightness block;

[0037] Figure 22 This is a diagram illustrating an example of the syntax for intra-frame chromatic difference prediction modes;

[0038] Figure 23 This is a diagram illustrating an example of replacing surrounding pixels to limit intra-frame chromatic aberration prediction;

[0039] Figure 24 Figures (a) to (c) illustrate intra-frame prediction of different luminance chromatic aberrations based on the size of the luma block. Detailed Implementation

[0040] The embodiments of the present invention provide an image encoding technique that divides an image into rectangular blocks and encodes and decodes the divided blocks.

[0041] (First Implementation)

[0042] The image encoding apparatus 100 and the image decoding apparatus 200 according to Embodiment 1 of the present invention will be described.

[0043] Figure 1 This is a structural diagram of the image encoding apparatus 100 according to the first embodiment. Here, in Figure 1 The diagram only shows the data stream related to the image signal. For additional information other than the image signal, such as motion vectors or prediction patterns, each component is provided to the coded bit string generation unit 105 to generate corresponding coded data, but the data stream for the additional information is not shown.

[0044] The block segmentation unit 101 segments the image into encoding target blocks, which are processing units for encoding, and provides the image signals within the encoding target blocks to the residual signal generation unit 103. In addition, the block segmentation unit 101 provides the image signals of the encoding target blocks to the prediction image generation unit 102 in order to evaluate the consistency of the prediction image.

[0045] The block segmentation unit 101 recursively segments the image into rectangles of a predetermined size to generate coded object blocks. The block segmentation unit 101 includes: a 4-segmentation unit that divides the recursively segmented object blocks into 4 blocks in both the horizontal and vertical directions to generate 4 blocks; and a 2-segmentation unit that divides the recursively segmented object blocks into 2 blocks in either the horizontal or vertical direction to generate 2 blocks. The detailed operation of the block segmentation unit 101 will be described later.

[0046] The prediction image generation unit 102 generates a prediction image signal based on the decoded image signal provided from the decoded image memory 108 and performs intra-frame prediction or inter-frame prediction based on a prediction mode. The image signal within the encoded target block provided from the block segmentation unit 101 is used for evaluation of intra-frame and inter-frame prediction. In intra-frame prediction, the prediction image signal is generated using the image signal of the encoded target block provided from the block segmentation unit 101 and the image signals of surrounding encoded blocks that exist in the same image as the encoded target block provided from the decoded image memory 108 and are close to the encoded target block. In inter-frame prediction, for the image signal of the encoded target block provided from the block segmentation unit 101, an encoded image stored in the decoded image memory 108 that is located before or after the image (encoded image) containing the encoded target block in the time series is used as a reference image. Block consistency evaluation, such as block matching, is performed between the encoded image and the reference image to determine the motion vector representing the motion amount. Based on this motion amount, motion compensation is performed from the reference image to generate the prediction image signal. The prediction image generation unit 102 provides the generated prediction image signal to the residual signal generation unit 103.

[0047] The residual signal generation unit 103 performs a subtraction operation on the encoded image signal and the prediction signal generated by the prediction image generation unit 102 to generate a residual signal, which is then provided to the orthogonal transformation / quantization unit 104.

[0048] The quadrature transform / quantization unit 104 performs quadrature transform / quantization on the residual signal provided from the residual signal generation unit 103, and provides the quadrature transformed and quantized residual signal to the encoded bit string generation unit 105 and the inverse quantization / inverse quadrature transform unit 106.

[0049] The encoded bit string generation unit 105 generates an encoded bit string for the residual signal after orthogonal transformation / quantization provided by the orthogonal transformation / quantization unit 104. Additionally, the encoded bit string generation unit 105 generates corresponding encoded bit strings for additional information such as motion vectors, prediction modes, and block segmentation information.

[0050] The inverse quantization / inverse quadrature transformation unit 106 performs inverse quantization and inverse quadrature transformation on the residual signal after quadrature transformation and quantization provided by the quadrature transformation / quantization unit 104, and provides the residual signal after inverse quantization and inverse quadrature transformation to the decoded image signal overlap unit 107.

[0051] The decoded image signal overlay unit 107 overlays the predicted image signal generated by the predicted image generation unit 102 with the residual signal that has undergone inverse quantization and inverse quadrature transformation by the inverse quantization / inverse quadrature transformation unit 106 to generate a decoded image, which is then stored in the decoded image memory 108. Alternatively, sometimes the decoded image may undergo filtering processing to reduce block distortion caused by encoding, and this processing is also stored in the decoded image memory 108.

[0052] Figure 2 This is a structural diagram of the image decoding apparatus 200 according to Embodiment 1. Here, Figure 2 The diagram only shows the data stream related to the image signal. The bit string decoding unit 201 provides additional information other than the image signal, such as motion vectors and prediction modes, to each component and uses it for corresponding processing. The data stream of the additional information is not shown.

[0053] The bit string decoding unit 201 decodes the provided encoded bit string and provides the residual signal after orthogonal transformation / quantization to the block segmentation unit 202.

[0054] The block segmentation unit 202 determines the shape of the target block to be decoded based on the decoded block segmentation information, and provides the residual signal after orthogonal transformation / quantization of the determined target block to the inverse quantization / inverse orthogonal transformation unit 203.

[0055] The block segmentation unit 202 recursively segments the image into rectangles of a predetermined size based on the decoded block segmentation information, generating decoded object blocks. The block segmentation unit 202 includes: a 4-segmentation unit that performs 4-segmentation on the recursively segmented object blocks in both the horizontal and vertical directions to generate 4 blocks; and a 2-segmentation unit that performs 2-segmentation on the recursively segmented object blocks in either the horizontal or vertical direction to generate 2 blocks. The detailed operation of the block segmentation unit 202 will be described later.

[0056] The inverse quantization / inverse quadrature transformation unit 203 performs inverse quadrature transformation and inverse quantization on the provided quadrature transformation and quantized residual signal to obtain the residual signal after inverse quadrature transformation and inverse quantization.

[0057] The prediction image generation unit 204 generates a prediction image signal based on the decoded image signal provided from the decoded image memory 206, and provides the prediction image signal to the decoded image signal overlay unit 205.

[0058] The decoded image signal overlay unit 205 generates and outputs a decoded image signal by overlaying the predicted image signal generated by the predicted image generation unit 204 and the residual signal that has undergone inverse quadrature transformation / inverse quantization by the inverse quantization / inverse quadrature transformation unit 203, and stores the signal in the decoded image memory 206. Additionally, sometimes filtering processing is performed on the decoded image to reduce block distortion caused by encoding, and this processing is also stored in the decoded image memory 206.

[0059] The operation of the block segmentation unit 101 of the image encoding device 100 will be described in detail. Figure 3 This is a flowchart illustrating the segmentation of tree blocks and the segmentation within tree blocks.

[0060] First, the input image is segmented into tree blocks of a specified size (S1000). For example, assume the tree block is 128 pixels × 128 pixels. However, the tree block is not limited to 128 pixels × 128 pixels; any size and shape can be used as long as it is rectangular. Alternatively, the size and shape of the tree blocks can be predetermined as fixed values ​​between the encoding and decoding devices, or the encoding device can determine and record them in the encoded bitstream, and the decoding device can use the recorded block size. Figure 4 This indicates the case where the input image is segmented into tree blocks. The tree blocks are encoded and decoded in a raster scan order from left to right and from top to bottom.

[0061] The interior of the tree block is further divided into rectangular blocks. The interior of each tree block is encoded and decoded in a z-scan order. Figure 5 This indicates the order of the z-scan. In a z-scan, encoding and decoding are performed in the order of top left, top right, bottom left, and bottom right. The internal segmentation of a tree block can be either 4-segmentation or 2-segmentation. 4-segmentation occurs in both horizontal and vertical directions. 2-segmentation occurs in either the horizontal or vertical direction. Figure 6 This is a diagram that divides a tree block into four parts in both the horizontal and vertical directions. Figure 7 This is a diagram showing a tree block divided into two parts horizontally. Figure 8 This is a diagram showing a tree block divided into two parts vertically.

[0062] Refer again Figure 3 Determine whether to perform 4-segmentation of the tree block in both horizontal and vertical directions (S1001).

[0063] If it is determined that the interior of the tree block needs to be divided into four parts (S1001: Yes), the interior of the tree block is divided into four parts (S1002), and each part of the block divided into four parts in the horizontal and vertical directions is processed (S1003). The process of further dividing the blocks after the four-part division will be described later. Figure 9 ).

[0064] If it is determined that the internal structure of the tree block will not be divided into 4 parts (S1001: No), then it is determined whether the internal structure of the tree block will be divided into 2 parts (S1004).

[0065] If it is determined that the tree block is to be divided into 2 parts (S1004: Yes), determine whether to set the direction of the 2-partition to the horizontal direction (S1005).

[0066] If the direction of the 2-splitting is determined to be horizontal (S1005: Yes), the tree block is 2-splitting internally in the horizontal direction (S1006), and various processes are performed on the block that has been 2-splitting in the horizontal direction (S1007). The process of further splitting the block after it has been 2-splitting in the horizontal direction will be described later. Figure 10 ).

[0067] If the direction of the 2-splitting is determined to be vertical rather than horizontal (S1005: No), the block is 2-splitting internally in the vertical direction (S1008), and the blocks that have been 2-splitting in the vertical direction are then processed (S1009). The process of further splitting blocks that have been 2-splitting in the horizontal direction will be described later. Figure 11 ).

[0068] If it is determined that the internal structure of the tree block will not be divided into two parts (S1004: No), the block division process ends without dividing the internal structure of the tree block (S1010).

[0069] Next, use Figure 9 The flowchart illustrates the processing of each block after the tree block is divided into 4 parts in the horizontal and vertical directions.

[0070] Determine whether to further divide the block into 4 parts in the horizontal and vertical directions (S1101).

[0071] If it is determined that the block is to be divided into 4 parts again (S1101: Yes), the block is divided into 4 parts again (S1102), and each part of the block that has been divided into 4 parts in the horizontal and vertical directions is processed (S1103).

[0072] If it is determined that the block should not be divided into 4 parts again (S1101: No), determine whether to divide the block into 2 parts (S1104).

[0073] If it is determined that the block is to be divided into two parts (S1104: Yes), determine whether to set the direction of the two-part division to the horizontal direction (S1105).

[0074] If the direction of the 2-splitting is determined to be horizontal (S1105: Yes), the block is divided into 2 parts in the horizontal direction (S1106), and each part of the block that has been divided into 2 parts in the horizontal direction is processed (S1107).

[0075] If the direction of the 2-segmentation is determined to be vertical rather than horizontal (S1105: No), the block is divided into 2 segments in the vertical direction (S1108), and the block after being divided into 2 segments in the vertical direction is processed (S1109).

[0076] If it is determined that the block should not be divided into two parts (S1104: No), the block splitting process ends without splitting the block (S1110).

[0077] Execute on each block after it has been divided into 4 Figure 9 The process is shown in the flowchart. The interior of the 4-segmented block is also encoded and decoded in z-scan order.

[0078] Next, use Figure 10 The flowchart illustrates the processing of each block after horizontal 2-segmentation of a tree block.

[0079] If the tree block is divided into 2 parts in the horizontal direction, first determine whether to divide the interior of each block into 4 parts in the horizontal and vertical directions (S1201).

[0080] If it is determined that the block is to be divided into 4 parts (S1201: Yes), the block is divided into 4 parts (S1202), and each part of the block that has been divided into 4 parts in the horizontal and vertical directions is processed (S1203).

[0081] If it is determined that the block should not be divided into 4 parts (S1201: No), determine whether to divide the block into 2 parts again (S1204).

[0082] If it is determined that the block will be split into two again (S1204: Yes), the block is split in the vertical direction (S1205), and each processing of the block that has been split into two in the vertical direction is performed (S1206).

[0083] If it is determined that no further 2-splitting is required (S1204: No), the block splitting process ends without further splitting of the block's interior (S1207).

[0084] Figure 11This describes the further subdivision of blocks after a horizontal 2-segmentation of a tree block. Here, when the parent tree block is horizontally 2-segmented, the further 2-segmentation of the resulting blocks only allows vertical 2-segmentation, automatically performing 2-segmentation in the vertical direction. Furthermore, when the parent tree block is 2-segmented, 4-segmentation can be completely prohibited in child blocks. Therefore, it is possible to prevent block subdivision in the same direction as the parent block, thus preventing horizontally elongated rectangular block subdivisions and simplifying encoding and decoding processes.

[0085] Figure 10 The flowchart illustrates the processing performed on each block after it has been divided into two parts horizontally. The internal parts of each block are also encoded and decoded in a top-to-bottom order.

[0086] Next, use Figure 12 The flowchart illustrates the processing of each block after a tree block is split into two parts in the vertical direction.

[0087] If the tree block is divided into 2 parts in the vertical direction, first determine whether to divide the interior of each block into 4 parts in the horizontal and vertical directions (S1301).

[0088] If it is determined that the block is to be divided into 4 parts (S1301: Yes), the block is divided into 4 parts (S1302), and each part of the block that has been divided into 4 parts in the horizontal and vertical directions is processed (S1303).

[0089] If it is determined that the block should not be divided into 4 parts (S1301: No), determine whether to divide the block into 2 parts again (S1304).

[0090] If it is determined that the block will be split into two again (S1304: Yes), the block is split in the horizontal direction (S1305), and each processing of the block that has been split into two in the horizontal direction is performed (S1306).

[0091] If it is determined that no further 2-splitting is required (S1304: No), the block splitting process ends without further splitting of the block's interior (S1307).

[0092] Figure 13This describes the further segmentation of blocks after a vertical 2-segmentation of a tree block. Here, when the parent tree block is vertically 2-segmented, the further 2-segmentation of the resulting blocks only allows horizontal 2-segmentation, which is automatically performed horizontally. Furthermore, when the parent tree block is 2-segmented, 4-segmentation can be completely prohibited in child blocks. This prevents block segmentation in the same direction as the parent block, thus preventing vertically elongated rectangular block segments and simplifying encoding and decoding processes.

[0093] Figure 12 The process shown in the flowchart is performed on each block that is divided into two parts in the vertical direction. The internal parts of each block are also encoded and decoded in a left-right order.

[0094] Furthermore, the re-segmentation of the segmented blocks when splitting a tree block is explained, but the parent block may not be a tree block. For example, if a tree block (128×128) is split into 4 parts, and the resulting 4-part block (64×64) is further split into 4 parts or 2 parts, the above processing is also applied to the re-segmentation of the resulting blocks.

[0095] Next, the operation of the block segmentation unit 202 of the image decoding apparatus 200 will be explained. Blocks are segmented in the same processing order as the block segmentation unit 101 of the image encoding apparatus 100. In the block segmentation unit 101 of the image encoding apparatus 100, a block segmentation mode is selected and the selected block segmentation information is output. In contrast, the block segmentation unit 202 of the image decoding apparatus differs in that it uses block segmentation information decoded from the encoded bitstream to segment blocks. Furthermore, when decoding the block segmentation information from the encoded bitstream, if further segmentation in the same direction is prohibited, a syntax structure is used to prevent the transmission of information without selection options within the bitstream.

[0096] Figure 14 This section presents an example of the syntax (syntactic rules for encoding bitstreams) related to block segmentation in the first implementation. Regarding segmentation within a tree block, a flag indicating whether to perform 4-division (4_division_flag) is first sent and received. If 4-division is performed (4_division_flag is 1), the tree block is segmented into 4 parts, and processing ends. Then, for the 4-division blocks, the process is repeated... Figure 14The syntax shown further divides the internal data. Without 4-division (4_division_flag is 0), a flag indicating whether 2-division is performed (2_division_flag) is sent and received. With 2-division (2_division_flag is 1), a flag indicating the direction of the 2-division (2_division_direction) is also sent and received. 2_division_direction is 1 indicating a vertical division, and 2_division_direction is 0 indicating a horizontal division. Then, for the 2-division blocks, the data is processed again using... Figure 14 The syntax shown further divides the block. Without performing a 2-division (2_division_flag is 0), the processing ends without dividing the tree block.

[0097] Here, we will explain the process of further subdividing the internal structure of blocks after 4- or 2-partitioning. The process of further subdividing the internal structure of blocks also uses... Figure 14 The syntax shown differs from that of segmenting tree blocks in the following way: there is a restriction on the segmentation direction during 2-segmentation. Specifically, when a tree block is 2-segmented, further segmentation within the resulting block is prohibited in the same direction as the initial 2-segmentation. This prevents the resulting blocks from becoming elongated rectangles, thus preventing an increase in memory bandwidth required for intra-frame and inter-frame prediction. Details regarding preventing memory bandwidth increases will be discussed later.

[0098] Alternatively, the number of 2-splitting operations in the same direction can be counted, and splitting in the same direction can be restricted if a certain number of operations are exceeded. For example, 2-splitting operations in the same direction can be allowed up to 2 times, but 2-splitting operations in the same direction can be prohibited from the third time onwards.

[0099] exist Figure 14 In this context, the syntax prioritizes 4-segmentation, sending and receiving information about whether to perform 4-segmentation before or after 2-segmentation. Conversely, when 2-segmentation is prioritized, the syntax can also prioritize sending and receiving information about whether to perform 2-segmentation before or after 4-segmentation. This is because sending and receiving events that are more probabilistically likely results in less coding as a bitstream. In other words, it can also be a syntax that pre-estimates which segmentation, 4-segmentation or 2-segmentation, is more likely to occur and sends the more likely segmentation information first. For example, by sending and receiving the header information of the image, the encoding device can adaptively determine the priority segmentation number with higher coding efficiency, and the decoding device can segment the interior of the tree block using a syntax based on the selected priority segmentation number.

[0100] In the image encoding apparatus 100 and the image decoding apparatus 200, intra-frame prediction and inter-frame prediction are performed using the segmented blocks. Both intra-frame prediction and inter-frame prediction involve copying pixels from memory.

[0101] Figure 15 (a)~ Figure 15 (d) represents an example of intra-frame prediction. Figure 15 (a) and Figure 15 (b) represents the prediction direction and mode number of the intra-frame prediction. For example... Figure 15 (c) and Figure 15 As shown in (d), in intra-frame prediction, a predicted image of the encoded / decoded block is generated by copying pixels from already encoded / decoded pixels that are close to the target block. In intra-frame prediction, since the generation of encoded / decoded pixels is repeated block-by-block from the start of the predicted image generation, the processing order becomes a sequence on a block-by-block basis. The smaller the segmentation within a block, the greater the overall processing load. Furthermore, the more elongated the shape of the block, the greater the processing required for pixel copying from memory. Additionally, since orthogonal transformations of the residual signals are performed during encoding and decoding, the more types of rectangle sizes there are, the more types of orthogonal transformations are required, resulting in an increase in circuit size. Therefore, when performing a 2-segmentation within a block, limiting the 2-segmentation to the same direction as the segmentation direction of the parent block can prevent an increase in the memory bandwidth required for intra-frame prediction.

[0102] Figure 16 This illustrates an example of inter-frame prediction. Inter-frame prediction generates a predicted image of the encoded / decoded target block by copying pixels from the pixels contained in the encoded / decoded image in blocks. In inter-frame prediction, when copying pixels from the reference image in blocks, the structure often becomes the configuration of the device required to acquire a memory management unit containing the necessary pixels. Therefore, the smaller the block is divided, and the more elongated the shape of the block becomes, the greater the overall processing load. Furthermore, when performing fractional-precision motion compensation using interpolation filters on the reference image, it is necessary to copy pixels containing a few pixels plus the pixels within the block. The smaller the block size, the larger the relative ratio of the added pixels, and the greater the overall processing load. Therefore, when performing 2-segmentation within the block, limiting the 2-segmentation to the same direction as the segmentation direction of the parent block can prevent an increase in the memory bandwidth required for inter-frame prediction.

[0103] Next, the relationship between the luminance signal and the chrominance signal in intra-frame prediction will be explained. Previously known formats for the relationship between the luminance and chrominance signals include 4:2:0, 4:2:2, and 4:4:4. Figure 17The 4:2:0 format shown in (a) samples two pixels for the luminance signal in both the horizontal and vertical directions, whereas it samples one pixel for the chrominance signal in both the horizontal and vertical directions. Since the human eye is more sensitive to the luminance signal than to the chrominance signal, the amount of information in the chrominance signal is reduced compared to the luminance signal. Figure 17 The 4:2:2 format shown in (b) samples the luminance signal with two pixels horizontally, and conversely, samples the chrominance signal with one pixel horizontally. Vertically, the luminance signal is sampled with two pixels vertically, and the chrominance signal with two pixels vertically. Figure 17 The 4:4:4 format shown in (c) samples two pixels in the horizontal / vertical direction for the luminance signal, while sampling two pixels in both the horizontal and vertical directions for the chrominance signal.

[0104] This embodiment is illustrated using the 4:2:0 format, the most widely used image coding format. The block segmentation units 101 and 202 include: a luminance block segmentation unit that segments the luminance signal of the image to generate luminance blocks; and a chrominance block segmentation unit that segments the chrominance signal of the image to generate chrominance blocks. In intra-frame prediction, the luminance signal and chrominance signal are segmented into blocks independently. That is, in intra-frame prediction, the size of the luminance block and the size of the chrominance block are determined independently. In intra-frame prediction, since the luminance signal and chrominance signal copy pixel values ​​from surrounding pixels respectively, segmenting the luminance signal and chrominance signal into blocks independently improves prediction efficiency. In contrast, for inter-frame prediction, the luminance signal and chrominance signal are processed together for block segmentation. That is, in inter-frame prediction, the size of the luminance block and the size of the chrominance block are the same. This is because in inter-frame prediction, it is not necessary to distinguish between luminance and chrominance during motion compensation.

[0105] The image prediction generation units 102 and 204 include a luminance signal prediction unit for predicting luminance signals and a chrominance signal prediction unit for predicting chrominance signals. To improve the prediction efficiency of chrominance signals in intra-frame prediction, the chrominance signal prediction unit performs luminance-chrominance intra-frame prediction based on the encoded and decoded pixels of the luminance signal. In luminance-chrominance intra-frame prediction, the primary signal is encoded / decoded before the secondary signal, and the encoded / decoded primary signal is used to predict the secondary signal. Here, in 4:2:0 and 4:2:2 formats, the luminance signal has more information than the chrominance signal; therefore, the luminance signal is used as the primary signal, and the chrominance signal as the secondary signal. In the 4:4:4 format, the luminance signal and chrominance signal have the same amount of information, but depending on other formats, the luminance signal is usually used as the primary signal, and the chrominance signal as the secondary signal.

[0106] Figure 18 This is a diagram illustrating intra-frame prediction of luminance and chromatic aberration. Figure 19This is a flowchart illustrating intra-frame prediction of luminance and chromatic aberration.

[0107] like Figure 18 As shown, the intra-frame prediction of luminance and chrominance difference is based on the correlation between the encoded and decoded peripheral pixels 12a and 12b of luminance block 10 and the encoded and decoded peripheral pixels 16a and 16b of chrominance block 14. Since the intra-frame prediction of luminance and chrominance difference predicts the chrominance signal, the peripheral pixels that are the objects of the correlation calculation are defined based on the peripheral pixels of the chrominance block to which the correlation is calculated. That is, the peripheral pixels of the luminance block located at the same position as the peripheral pixels determined for the chrominance block become the objects of the correlation calculation.

[0108] First, the correlation between the surrounding pixels of the encoded / decoded luminance signal and the surrounding pixels of the chrominance signal is calculated (S1901). Next, the encoded / decoded luminance signal of the encoding / decoding target block is downsampled (S1902). Here, multiple downsampling filter types can be prepared to select the filter type. For example, filters with different intensities or different numbers of taps can be prepared to select multiple filter types. The filter type can be automatically selected using the correlation between surrounding pixels, or the filter type can be encoded, decoded, and transmitted in the bitstream. Alternatively, if the correlation between surrounding pixels is not used to determine the downsampling filter for the luminance signal of the encoding / decoding target block, the processing order of steps S1901 and S1902 is different, and steps S1901 and S1902 can be processed in parallel.

[0109] Finally, the chromatic difference signal is predicted based on the correlation between surrounding pixels using the downsampled luminance signal (S1903). In the 4:2:0 format, downsampling is halved in both the horizontal and vertical directions. In the 4:2:2 format, downsampling is halved in the horizontal direction, but no downsampling is performed in the vertical direction. In the 4:4:4 format, no downsampling is performed in either the horizontal or vertical directions.

[0110] In intra-frame prediction of luma and chromatic aberration, the prediction processing of the chromatic aberration signal can begin after the luma signal of the encoded / decoded object block has been completed. Therefore, the timing at which the prediction processing of the chromatic aberration block can begin depends on the size of the luma block and the size of the chromatic aberration block.

[0111] Figure 20 (a) and Figure 20 (b) is a diagram illustrating intra-frame prediction of luminance and chrominance when the size of the chrominance block is larger than the size of the luma block. Figure 20 As shown in (a), the first to fourth luminance blocks 20a, 20b, 20c, and 20d after being divided into four sections have a pixel count of 16×16. Figure 20The number of pixels in color difference block 20e shown in (b) is 16×16.

[0112] Here, the comparison of the sizes of the luma and chroma blocks is not based on the number of pixels within the block, but rather on the area taken into account for the chroma format. That is, in the 4:2:0 format, since the area occupied by the luma block is half that of the chroma block, the luma block is smaller when both have 16×16 pixels. In the 4:2:0 format, when the luma block has 16×16 pixels and the chroma block has 8×8 pixels, both blocks occupy the same area, and the luma and chroma blocks have the same size.

[0113] To compare the sizes of luminance and chroma blocks by comparing the number of pixels within each block rather than by comparing their areas, the pixel count of the chroma block is converted to the pixel count of the luminance block using the ratio of the luminance signal to the chroma signal in the chroma format. In the 4:2:0 format, since the pixel count of the luminance signal is twice the pixel count of the chroma signal, the pixel count in the dimensions of the chroma block is multiplied by 2 to convert it to the pixel count in the dimensions of the luminance block. For example, in the 4:2:0 format, with a luminance block size of 16×16 and a chroma block size of 16×16, the converted size of the chroma block, based on the pixel count of the luminance block, is 32×32, meaning the chroma block is larger.

[0114] Intra-frame prediction can perform prediction processing on subsequent blocks after the decoding process is completed on a block-by-block basis. That is, after the decoding of the first luma block 20a is completed, the decoding of the second luma block 20b can be performed; after the decoding of the second luma block 20b is completed, the decoding of the third luma block 20c can be performed; and after the decoding of the third luma block 20c is completed, the decoding of the fourth luma block 20d can be performed.

[0115] When the size of the chromatic aberration block is larger than the size of the luma block, the peripheral pixels required for the prediction processing of chromatic aberration block 20e exist together with the luma pixels and chromatic aberration pixels before the decoding of the four luma blocks 20a, 20b, 20c, and 20d. Without waiting for the decoding of the four luma blocks 20a, 20b, 20c, and 20d, it is possible to calculate... Figure 19 The degree of correlation between the peripheral pixels of the luminance signal and the peripheral pixels of the chromatic difference signal in step S1901.

[0116] Next, after the decoding of the first luminance block 20a is completed, the following steps are executed: Figure 19Step S1902 involves downsampling the luminance signal. Without waiting for the decoding of the second luminance block 20b to complete, the pixel of the color difference block 20e corresponding to the position of the first luminance block 20a can be predicted. Similarly, after the decoding of the second luminance block 20b is completed, luminance signal downsampling is performed, and without waiting for the decoding of the third luminance block 20c to complete, the pixel of the color difference block 20e corresponding to the position of the second luminance block 20b is predicted. Furthermore, after the decoding of the third luminance block 20c is completed, luminance signal downsampling is performed, and without waiting for the decoding of the fourth luminance block 20d to complete, the pixel of the color difference block 20e corresponding to the position of the third luminance block 20c is predicted. Finally, after the decoding of the fourth luminance block 20d is completed, luminance signal downsampling is performed, and the pixel of the color difference block 20e corresponding to the position of the fourth luminance block 20d is predicted.

[0117] Figure 21 (a) and Figure 21 (b) is a graph illustrating intra-frame prediction of luminance chromatic aberration when the size of the chromatic aberration block is smaller than the size of the luminance block. Figure 21 The number of pixels in luminance block 21a shown in (a) is 16×16. Figure 21 The number of pixels in the first to fourth color difference blocks 21b, 21c, 21d, and 21e after the 4-segmentation shown in (b) is 4×4.

[0118] In the 4:2:0 format, since the area occupied by the luma block is half the area occupied by the chroma block, the chroma block is smaller when the luma block has 16×16 pixels and the chroma block has 4×4 pixels. To compare the sizes of the luma and chroma blocks by comparing the number of pixels within each block rather than by comparing their areas, in the 4:2:0 format, the pixel count of the chroma block is multiplied by 2 to convert it to the pixel count of the luma block. In the 4:2:0 format, with a luma block size of 16×16 and a chroma block size of 4×4, the converted size of the chroma block, based on the number of pixels in the luma block, is 8×8, indicating that the chroma block is smaller.

[0119] When the size of the chromatic difference block is smaller than the size of the luminance block, the peripheral pixels of the first chromatic difference block 21b, along with the luminance and chromatic difference pixels, can be utilized starting before the decoding of luminance block 21a. However, the peripheral pixels of the second to fourth chromatic difference blocks 21c, 21d, and 21e cannot be utilized if the decoding of luminance block 21a is not completed. That is, if the decoding of luminance block 21a is completed but the decoding of the first chromatic difference block 21b is not completed, then for the second chromatic difference block 21c, it is impossible to calculate... Figure 19The correlation between the peripheral pixels of the luminance signal and the peripheral pixels of the chrominance signal is calculated in step S1901. Similarly, for the third chrominance block 21d, if the decoding of luminance block 21a is completed but the decoding of the first and second chrominance blocks 21b and 21c is not completed, then the correlation between the peripheral pixels of the luminance signal and the peripheral pixels of the chrominance signal cannot be calculated for the third chrominance block 21d. Similarly, for the fourth chrominance block 21e, if the decoding of luminance block 21a is completed but the decoding of the first to third chrominance blocks 21b, 21c, and 21d is not completed, then the correlation between the peripheral pixels of the luminance signal and the peripheral pixels of the chrominance signal cannot be calculated for the fourth chrominance block 21e.

[0120] Thus, when the size of the chromatic aberration block is smaller than the size of the luma block, if intra-frame prediction of luminance-chromatic aberration is performed, there will be processing dependencies between the luma block and the chromatic aberration block, as well as within the chromatic aberration block itself, making parallel processing unsuitable. Therefore, when the size of the chromatic aberration block is smaller than the size of the luma block, intra-frame prediction of luminance-chromatic aberration is restricted. Methods for restricting intra-frame prediction of luminance-chromatic aberration include (1) syntactic restriction, (2) permutation of intra-frame chromatic aberration patterns, and (3) permutation of surrounding pixels.

[0121] Figure 22 Examples of syntax for representing intra-frame chromatic aberration prediction modes. When the chromatic aberration prediction mode number is 0, the same intra-frame prediction mode as the luma prediction mode is used for the chromatic aberration prediction mode. For example, when the luma prediction mode is a horizontal prediction mode, the chromatic aberration prediction mode is also a horizontal prediction mode. When the chromatic aberration prediction mode number is 1, the average value mode (DC mode) is used. The DC mode uses the average value of surrounding pixels for intra-frame prediction. When the chromatic aberration prediction mode number is 2, the luma-chromatic aberration intra-frame prediction mode is used.

[0122] As a method to restrict intra-frame prediction of luminance and chrominance differences, in the case of (1) syntactic restriction, mode 2 is prohibited because it represents intra-frame prediction of luminance and chrominance differences. That is, mode 2 is not transmitted, and the intra-frame chrominance mode is selected from mode 0 and mode 1.

[0123] As a method for limiting intra-frame prediction of luminance chromatic aberration, in the case of replacing the intra-frame chromatic aberration mode in (2), if the chromatic aberration prediction mode number is designated as mode 2, the vertical prediction mode is used instead of the luminance chromatic aberration intra-frame prediction mode. However, the replacement prediction mode is not limited to the vertical prediction mode, and may also be other prediction modes. In addition, it is more preferable that the luminance prediction mode of mode 0 is the same as the mode to be replaced in mode 2 and the mode to be used is not repeated.

[0124] As a method for limiting intra-frame prediction of luminance and chromatic aberration, in the case of (3) permuting surrounding pixels, the permutation is used to calculate Figure 19Step S1901 involves determining the correlation between the peripheral pixels of the luminance signal and the peripheral pixels of the chrominance signal. Figure 21 In the second to fourth color difference blocks 21c, 21d, and 21e, the correlation degree of surrounding pixels can also be calculated without waiting for the decoding of the luminance block 21a.

[0125] Figure 23 This illustrates an example of pixel permutation. For the left-hand peripheral pixels of the second color difference block 21c, pixels within the first color difference block 21b are typically used. However, using pixels within the first color difference block 21b requires waiting for the decoding of the luminance block 21a and the first color difference block 21b to complete. Therefore, the usable area 21f is used as the peripheral pixel of the second color difference block 21c without waiting for the decoding of the luminance block 21a. Similarly, for the third color difference block 21d, the usable area 21f is used as the peripheral pixel of the third color difference block 21d without waiting for the decoding of the luminance block 21a. Likewise, for the fourth color difference block 21e, the usable area 21f is used as the peripheral pixel of the fourth color difference block 21e without waiting for the decoding of the luminance block 21a.

[0126] Thus, in the first embodiment, when performing intra-frame prediction of luminance and chrominance differences, if the size of the chrominance block is smaller than the size of the luminance block, limiting the intra-frame prediction of luminance and chrominance differences can reduce the processing dependency between the luminance and chrominance blocks. This enables parallel processing of luminance and chrominance blocks, reducing the amount of encoding / decoding processing.

[0127] (Second Implementation)

[0128] The second embodiment of the present invention will be described. In the second embodiment, instead of the size relationship between the luma block and the chromaticity block, the size of each luma block and the size of each chromaticity block are evaluated independently to constrain the intra-frame prediction of luma and chromaticity. This is different from the first embodiment, but the structure and operation are the same as the first embodiment.

[0129] First, the limitations on the size of the luminance block will be explained. Figure 24 (a)~ Figure 24 (c) represents intra-frame prediction of different luminance chromatic aberrations based on the size of the luma patch. For example... Figure 24 of (a), Figure 24 As shown in (b), the first luminance block 24a corresponds to the position of the first chromatic aberration block 24e, the second luminance block 24b corresponds to the position of the second chromatic aberration block 24f, the third luminance block 24c corresponds to the position of the third chromatic aberration block 24g, and the fourth luminance block 24d corresponds to the position of the fourth chromatic aberration block 24h. Additionally, Figure 24 The luminance block 24i of (c) and Figure 24The positions of the first to fourth color difference blocks 24e, 24f, 24g, and 24h in (b) correspond.

[0130] Explain the dependency relationship between luminance blocks and chrominance blocks. For example... Figure 24 As shown in (a), when the size of the luminance block is small, if the decoding of the first luminance block 24a is completed, the decoding of the first chrominance block 24e can be performed. If the decoding of the second luminance block 24b is completed, the decoding of the second chrominance block 24f can be performed. If the decoding of the third luminance block 24c is completed, the decoding of the third chrominance block 24g can be performed. If the decoding of the fourth luminance block 24d is completed, the decoding of the fourth chrominance block 24h can be performed.

[0131] On the other hand, such as Figure 24 As shown in (c), if the luminance block 24i is not decoded, then the decoding of the first to fourth color difference blocks 24e, 24f, 24g, and 24h is impossible when the size of the luminance block is large.

[0132] When the segmentation of luma blocks and chroma blocks is determined independently, if the absolute size of the luma block is large, the probability that the size of the chroma block will be smaller than that of the luma block increases. Therefore, when the absolute size of the luma block is greater than a specified size, the intra-frame prediction of luma and chroma for the corresponding chroma block is restricted.

[0133] Similarly, if the absolute size of the chromatic aberration block is small, the probability that the size of the chromatic aberration block is smaller than the size of the luma block increases. Therefore, when the absolute size of the chromatic aberration block is below a specified size, the intra-frame prediction of the luma and chromatic aberration of the chromatic aberration block is limited.

[0134] The method for limiting intra-frame prediction of luminance and chromatic aberration is the same as that in the first embodiment.

[0135] Thus, in the second embodiment, intra-frame prediction of luminance and chrominance is restricted when the absolute size of the luminance block is greater than a threshold or when the absolute size of the chrominance block is less than a threshold. This allows for the prediction of cases where the size of the chrominance block is smaller than the size of the luminance block, restricting intra-frame prediction of luminance and chrominance, and probabilistically reducing the processing dependency between the luminance and chrominance blocks.

[0136] (Third Implementation)

[0137] The third embodiment of the present invention will be described. In the third embodiment, the chromatic difference blocks are divided in the block segmentation unit 101 such that the size of the chromatic difference blocks is not smaller than the size of the luminance blocks. This differs from the first embodiment, but the structure and operation are otherwise the same as in the first embodiment. When segmenting the chromatic difference blocks, the block segmentation unit 101 prohibits segmentation where the size of the chromatic difference blocks is smaller than the size of the luminance blocks. As a result, in intra-frame prediction of luminance and chromatic difference, the situation where the size of the chromatic difference blocks is smaller than the size of the luminance blocks is avoided, and parallel processing of luminance blocks and chromatic difference blocks is always possible.

[0138] The encoded bitstream of the image output by the image encoding device in the above embodiments has a specific data format so that it can be decoded according to the encoding method used in the embodiments, and the image decoding device corresponding to the image encoding device can decode the encoded bitstream of the specific data format.

[0139] In cases where wired or wireless networks are used to exchange encoded bitstreams between an image encoding device and an image decoding device, the encoded bitstream can be converted into a data format suitable for the transmission method of the communication path for transmission. In this case, a transmitting device is provided that converts the encoded bitstream output by the image encoding device into encoded data in a data format suitable for the transmission method of the communication path and transmits it to the network; and a receiving device receives the encoded data from the network, restores it to the encoded bitstream, and provides it to the image decoding device.

[0140] The transmitting device includes: a memory for buffering the encoded bitstream output by the image encoding device; a packet processing unit for packetizing the encoded bitstream; and a transmitting unit for transmitting the packetized encoded data via a network. The receiving device includes: a receiving unit for receiving the packetized encoded data via a network; a memory for buffering the received encoded data; and a packet processing unit for packetizing the encoded data to generate an encoded bitstream and providing it to the image decoding device.

[0141] Alternatively, by adding a display unit to the structure to display the image decoded by the image decoding device, it can also function as a display device. In this case, the display unit reads the decoded image signal generated by the decoded image signal overlay unit 205 and stored in the decoded image memory 206, and displays it on the screen.

[0142] Alternatively, the imaging device can be configured by adding an imaging unit to the structure and inputting the captured image into an image encoding device. In this case, the imaging unit inputs the captured image signal into the block segmentation unit 101.

[0143] The encoding and decoding processes described above can be implemented using hardware transmission, storage, and reception devices, or through firmware or computer software stored in ROM (Read-Only Memory) or flash memory. The firmware or software program can be provided by recording it on a computer or other readable recording medium, or by providing it from a server via wired or wireless networks, or by providing it as data broadcast via terrestrial wave or satellite digital broadcasting.

[0144] The present invention has been described above based on embodiments. These embodiments are illustrative, and various modifications can be made to the combination of these constituent elements and processing procedures; those skilled in the art should understand that such modifications are also within the scope of the present invention.

[0145] Symbol Explanation

[0146] 100 Image encoding device, 101 Block segmentation unit, 102 Predictive image generation unit, 103 Residual signal generation unit, 104 Orthogonal transform / quantization unit, 105 Encoded bit string generation unit, 106 Inverse quantization / inverse orthogonal transform unit, 107 Decoded image signal overlap unit, 108 Decoded image memory, 200 Image decoding device, 201 Bit string decoding unit, 202 Block segmentation unit, 203 Inverse quantization / inverse orthogonal transform unit, 204 Predictive image generation unit, 205 Decoded image signal overlap unit, 206 Decoded image memory.

[0147] Industrial availability

[0148] This invention utilizes a technique for segmenting an image into blocks and encoding and decoding the segmented blocks as units.

Claims

1. An image encoding apparatus for dividing an image into blocks, encoding the blocks as units, and generating a bitstream, the image encoding apparatus being characterized by comprising: The main signal block segmentation unit divides the main signal of the image into rectangles of a first predetermined size to generate main signal blocks; The sub-signal block segmentation unit divides the sub-signal of the image into rectangles of a second predetermined size to generate sub-signal blocks; The main signal prediction unit predicts the main signal. as well as Sub-signal prediction unit, predicts sub-signals. The main signal block and the sub-signal block are segmented independently in the case of intra-frame prediction. The primary signal is a luminance signal, and the secondary signal is a chromatic aberration signal. The sub-signal prediction unit is capable of inter-component prediction of the sub-signal from the encoded main signal. However, this inter-component prediction is prohibited when the size of the main signal block is a third predetermined size or larger. The sub-signal prediction unit downsamples the main signal according to the format between the main signal and the sub-signal during the inter-component prediction.

2. An image encoding method, comprising dividing an image into blocks, encoding the blocks as units, and generating a bitstream, the image encoding method being characterized by comprising: The main signal block segmentation step involves dividing the main signal of the image into rectangles of a first predetermined size to generate main signal blocks; The sub-signal block segmentation step divides the sub-signal of the image into rectangles of a second predetermined size to generate sub-signal blocks; The main signal prediction step is to predict the main signal. as well as Secondary signal prediction steps: Predict the secondary signal. The main signal block and the sub-signal block are segmented independently in the case of intra-frame prediction. The primary signal is a luminance signal, and the secondary signal is a chromatic aberration signal. In the sub-signal prediction step, inter-component prediction of the sub-signal from the encoded main signal can be performed. However, this inter-component prediction is prohibited when the size of the main signal block is a third predetermined size or larger. In the sub-signal prediction step, the main signal is downsampled according to the format between the main signal and the sub-signal in the inter-component prediction.

3. An image decoding apparatus, which decodes images in units of blocks obtained by segmenting an image, characterized in that it comprises: The main signal block segmentation unit divides the main signal of the image into rectangles of a first predetermined size to generate main signal blocks; The sub-signal block segmentation unit divides the sub-signal of the image into rectangles of a second predetermined size to generate sub-signal blocks; The main signal prediction unit predicts the main signal. as well as Sub-signal prediction unit, predicts sub-signals. The main signal block and the sub-signal block are segmented independently in the case of intra-frame prediction. The primary signal is a luminance signal, and the secondary signal is a chromatic aberration signal. The sub-signal prediction unit is capable of inter-component prediction of the sub-signal from the decoded main signal. However, this inter-component prediction is prohibited when the size of the main signal block is a third predetermined size or larger. The sub-signal prediction unit downsamples the main signal according to the format between the main signal and the sub-signal during the inter-component prediction.

4. An image decoding method, wherein decoding is performed on a block-by-block basis obtained by segmenting an image, the image decoding method being characterized by comprising: The main signal block segmentation step involves dividing the main signal of the image into rectangles of a first predetermined size to generate main signal blocks; The sub-signal block segmentation step divides the sub-signal of the image into rectangles of a second predetermined size to generate sub-signal blocks; The main signal prediction step is to predict the main signal. as well as Secondary signal prediction steps: Predict the secondary signal. The main signal block and the sub-signal block are segmented independently in the case of intra-frame prediction. The primary signal is a luminance signal, and the secondary signal is a chromatic aberration signal. In the sub-signal prediction step, inter-component prediction of the sub-signal from the decoded main signal can be performed. However, this inter-component prediction is prohibited when the size of the main signal block is a third predetermined size or larger. In the sub-signal prediction step, the main signal is downsampled according to the format between the main signal and the sub-signal in the inter-component prediction.

5. A method for storing a bit stream, characterized in that, The image encoding method of claim 2 is used to generate a bitstream; and the bitstream is stored.

6. A method for transmitting a bit stream, characterized in that, The image encoding method of claim 2 is used to generate a bitstream; and the bitstream is transmitted.

Images