Image encoding device and image encoding method

Abstract

An image encoding device (200) is provided with: a feature acquirer (240) that acquires a feature for each component; a ratio calculator (242) that calculates a feature ratio of a plurality of components according to the features thereof; a variance value acquirer (244) that acquires a variance value for one component; a flatness checker (246) that checks flatness according to the variance value; a target code amount controller (250) that determines a target code amount of data of each component according to the feature ratio and that corrects the target code amount determined according to the feature ratio according to the result of checking flatness; a frequency converter (210) that performs frequency conversion on the data of each component; a quantization processor (220) that quantizes the data of each component according to the target code amount; and an encoder (230) that encodes the data of each component.

Description

Image Encoding Device and Image Encoding Method

[0001] The present disclosure relates to an image encoding device and an image encoding method.

[0002] In Patent Document 1, for each group, a quantization step and an encoding method are determined so that the code length of a block including a plurality of groups does not exceed a predetermined value, and encoding processing is performed based on the determined quantization step and encoding method to generate encoded data. Patent Document 2 discloses that image data is compressed by either a first compression algorithm in which a scale factor is fixed or a second compression algorithm in which the scale factor is adjustable, and it is determined whether the amount of code is within a predetermined range.

[0003] Japanese Patent No. 6502739 Japanese Patent No. 4631629

[0004] With the conventional techniques described in Patent Document 1 and Patent Document 2, although the amount of code is controlled, image quality degradation may occur.

[0005] The present disclosure provides an image encoding device and an image encoding method capable of suppressing image quality degradation.

[0006] An image encoding apparatus according to one aspect of the present disclosure includes: a feature acquirer that acquires feature quantities of data for each of a plurality of components constituting pixels of an image in a processing target block among a plurality of blocks of the image; a ratio calculator that calculates the feature quantity ratio of the plurality of components according to the feature quantities; a variance acquirer that acquires the variance value of the data of one of the plurality of components in the processing target block for one of the plurality of components; a flatness determiner that performs a flatness determination of whether or not the processing target block is flat according to the variance value; a target code quantity controller that determines the target code quantity of the data of each of the plurality of components according to the feature quantity ratio and corrects the target code quantity determined according to the feature quantity ratio according to the result of the flatness determination; a frequency converter that performs a frequency conversion on the data of each of the plurality of components; a quantization processor that quantizes the data of the component after frequency conversion according to the target code quantity of the data of each of the plurality of components; and an encoder that encodes the data of the component after quantization for each of the plurality of components.

[0007] Furthermore, an image encoding method according to one aspect of the present disclosure includes the steps of: obtaining feature quantities of data for each of a plurality of components constituting pixels of an image in a processing target block among a plurality of blocks of the image; calculating a feature quantity ratio of the plurality of components according to the feature quantities; obtaining a variance value for the data of one of the plurality of components in the processing target block for one of the plurality of components; performing a flatness determination to determine whether the processing target block is flat or not according to the variance value; determining a target code amount for the data of each of the plurality of components according to the feature quantity ratio, and correcting the target code amount determined according to the feature quantity ratio according to the result of the flatness determination; performing a frequency transformation on the data of each of the plurality of components; quantizing the data of the component after frequency transformation according to the target code amount of the data of each of the plurality of components; and encoding the data of the component after quantization for each of the plurality of components.

[0008] An image encoding apparatus and image encoding method according to one aspect of this disclosure can suppress image quality degradation.

[0009] This is a block diagram showing the configuration of the image encoding device in the first reference example. This is a flowchart showing the operation of the image encoding device in the first reference example. This is a diagram showing a standard quantization table used for luminance data. This is a diagram showing a standard quantization table used for chrominance data. This is a conceptual diagram showing an example of compression of luminance data and chrominance data in the first reference example. This is a conceptual diagram showing mosquito noise caused by quantization errors. This is a block diagram showing the configuration of the image encoding device in the second reference example. This is a flowchart showing the operation of the image encoding device in the second reference example. This is a block diagram showing the configuration of the image encoding device in the embodiment. This is a flowchart showing the operation of the image encoding device in the embodiment. This is a conceptual diagram showing an example of calculating features for luminance data. This is a conceptual diagram showing an example of calculating features for chrominance data. This is a diagram showing a reference table between feature ratio and target code amount. This is a graph showing the relationship between feature ratio and compression ratio. This is a conceptual diagram showing a first example of calculating the variance value. This is a conceptual diagram showing a second example of calculating the variance value. This is a flowchart showing the flatness determination process. This is a flowchart showing the target code amount correction process. This is a conceptual diagram showing the target code amount correction process. This is a conceptual diagram showing a first method for determining the scale factor. This is a flowchart showing a first method for determining the scale factor. This is a conceptual diagram showing a second method for determining the scale factor. This is a flowchart showing the second method for determining the scale factor. This is a conceptual diagram showing the third method for determining the scale factor. This is a flowchart showing the third method for determining the scale factor. This is a conceptual diagram showing the fourth method for determining the scale factor. This is a flowchart showing the fourth method for determining the scale factor. This is a conceptual diagram showing the fifth method for determining the scale factor. This is a conceptual diagram showing the first compression example in the embodiment. This is a block diagram showing the configuration of the image processing apparatus in the embodiment. This is a block diagram showing the configuration of the image compressor in the embodiment. This is a flowchart showing the operation of the image compressor in the embodiment. This is a conceptual diagram showing the second compression example in the embodiment. This is a block diagram showing the configuration of the image decompressor in the embodiment. This is a diagram showing the image quality evaluation results.

[0010] In recent years, image processing devices have been increasing the resolution of images. This has led to an increase in the amount of data encoded in image data, which can cause problems such as transmission delays and increased memory usage. Quantizing image data can reduce the amount of data encoded, thereby mitigating these problems.

[0011] However, quantization of image data can lead to quantization errors and degradation of image quality. For example, when image data has a large number of features, the reduction of the image data's code amount to a certain level can result in a significant loss of features. On the other hand, noise that occurs in flat areas around edges is more noticeable and easily recognizable visually.

[0012] For example, an image encoding device according to one aspect of the present disclosure includes: a feature acquirer that acquires feature quantities of data for each of a plurality of components constituting pixels of an image in a processing target block among a plurality of blocks of the image; a ratio calculator that calculates the feature quantity ratio of the plurality of components according to the feature quantities; a variance acquirer that acquires the variance value of the data of one of the plurality of components in the processing target block for one of the plurality of components; a flatness determiner that performs a flatness determination of whether or not the processing target block is flat according to the variance value; a target code quantity controller that determines the target code quantity of the data of each of the plurality of components according to the feature quantity ratio and corrects the target code quantity determined according to the feature quantity ratio according to the result of the flatness determination; a frequency converter that performs a frequency conversion on the data of each of the plurality of components; a quantization processor that quantizes the data of each of the plurality of components after frequency conversion according to the target code quantity of the data of each of the plurality of components; and an encoder that encodes the data of each of the plurality of components after quantization.

[0013] This allows the image encoding device to adjust the target code size for each component of the data according to the feature ratio of multiple components. Therefore, the image encoding device can suppress significant feature loss. Furthermore, the image encoding device can adjust the target code size according to whether the block being processed is flat or not, that is, according to the likelihood of noise being noticeable. Therefore, the image encoding device can suppress the generation of noticeable noise. Thus, the image encoding device can suppress image quality degradation.

[0014] Furthermore, for example, the aforementioned plurality of components include two components: luminance and color difference.

[0015] This allows the image encoding device to adjust the target encoding amount for each component of luminance and chrominance according to the feature quantities of the data for that component. Therefore, the image encoding device can appropriately suppress image quality degradation for images where the feature quantities differ between luminance and chrominance.

[0016] Furthermore, for example, the aforementioned one component is a component of luminance.

[0017] This allows the image encoding device to perform flatness determination according to the variance value of the luminance component data. Therefore, the image encoding device can appropriately determine whether the block to be processed is flat or not according to components that are easily visible and are assumed to have relatively large feature quantities. Consequently, the image encoding device can appropriately suppress the generation of noticeable noise.

[0018] Furthermore, for example, the aforementioned plurality of components include three components: red, green, and blue.

[0019] This allows the image encoding device to adjust the target encoding amount for each component (red, green, and blue) according to the feature quantities of the data for that component. Therefore, the image encoding device can appropriately suppress image quality degradation for images where the feature quantities differ between red, green, and blue.

[0020] Furthermore, for example, the aforementioned multiple components include transparency.

[0021] This allows the image encoding device to reduce the memory capacity required to retain transparency information when blending multiple images using RGBA, as well as the delay that occurs during the transmission of that data.

[0022] Furthermore, for example, in determining the target code amount, the target code amount controller increases the target code amount of the data for each of the plurality of components as the proportion of the features of the data for that component in the feature ratio increases, and if it is determined that the block to be processed is flat, in correcting the target code amount, the corrected target code amount is increased for one component compared to the target code amount before correction, and the corrected target code amount is decreased for each of the plurality of components, excluding one component, compared to the target code amount before correction.

[0023] This allows the image encoding device to increase the target code size for each component as the feature size increases, and decrease the target code size as the feature size decreases. Furthermore, if the image encoding device determines that a block to be processed is flat, it can increase the target code size for the component used in the flatness determination and decrease the target code sizes for the other components. Therefore, the image encoding device can appropriately adjust the target code size according to the feature ratio and the results of the flatness determination.

[0024] Furthermore, for example, the variance value acquisition device acquires the variance value for each of the multiple subblocks in the processing target block, and the flatness determination device determines whether or not a subblock is flat according to the variance value acquired for each of the multiple subblocks, and performs the flatness determination according to the number of subblocks that are determined to be flat among the multiple subblocks.

[0025] This allows the image encoding device to determine whether a block to be processed is flat or not, according to the number of subblocks determined to be flat. In other words, the image encoding device can determine whether a block to be processed is flat or not, according to the size of the flat areas within that block. Therefore, the image encoding device can appropriately adjust the target code amount according to the size of the flat areas where noticeable noise may occur.

[0026] Furthermore, for example, the feature acquisition device acquires, for each of the plurality of components, the statistical value of the absolute difference between adjacent pixels in the data of that component as the feature of the data of that component.

[0027] This allows the image encoding device to acquire feature quantities that correspond to sharp changes between adjacent pixels. Therefore, the image encoding device can appropriately adjust the target code quantity according to the feature quantities that correspond to sharp changes between adjacent pixels.

[0028] Furthermore, for example, the feature extractor obtains the feature quantities of the data for each of the plurality of components using the Hadamard transform.

[0029] This allows the image encoding device to acquire feature quantities corresponding to the amount of edges obtained by the Hadamard transform. Therefore, the image encoding device can appropriately adjust the target code quantity according to the feature quantities corresponding to the amount of edges.

[0030] Furthermore, for example, the feature acquisition device acquires the feature quantities of the data for each of the plurality of components by obtaining information indicating the feature quantities of the data for that component from a device outside the image encoding device.

[0031] This allows the image encoding device to acquire features without calculating them. Therefore, the image encoding device can reduce the amount of computation it has to perform.

[0032] Furthermore, for example, the encoder multiplexes an identification code indicating a plurality of target code amounts determined for the plurality of components and a plurality of data encoded for the plurality of components into a stream and outputs the stream.

[0033] This allows the image encoding device to indicate the target code amount for each component of the data within the stream. Therefore, the image encoding device can assist in decoding the data for each component from the stream.

[0034] Furthermore, for example, the encoder multiplexes the multiple data encoded for the multiple components into a stream and outputs the stream, and outputs an identification code indicating the multiple target coding amounts determined for the multiple components separately from the stream.

[0035] This allows the image encoding device to specify the target code amount for each component of the data separately from the stream. Therefore, the image encoding device can provide information for decoding each component of the data separately from the stream.

[0036] Furthermore, for example, the quantization processor determines a scale factor that affects the quantization width for each of the plurality of components according to the target code amount, and quantizes the data of that component according to the scale factor.

[0037] This allows the image encoding device to adjust the scale factor used for quantizing the data of each component according to the target code size of that component's data. Therefore, the image encoding device can appropriately adjust the code size of the data of each component according to the target code size of that component's data.

[0038] Furthermore, for example, the quantization processor initializes the scale factor, quantizes the data according to the scale factor, obtains the predicted code amount of the data according to the data quantized according to the scale factor, updates the scale factor according to the comparison result between the predicted code amount and the target code amount, and determines the scale factor by repeating the quantization of the data, obtaining the predicted code amount, and updating the scale factor until the predicted code amount matches the target code amount.

[0039] This allows the image coding device to search for and determine a scale factor such that the predicted code amount fits the target code amount. Therefore, the image coding device can determine an appropriate scale factor for the target code amount.

[0040] Furthermore, for example, the quantization processor initializes the scale factor according to the features of the data.

[0041] This allows the image encoding device to appropriately initialize the scale factor during the search for the scale factor, thereby suppressing processing delays.

[0042] Furthermore, for example, the quantization processor updates the initial value of the scale factor if the difference between the scale factor determined by repeatedly updating the scale factor and the initial value of the scale factor is greater than a threshold, and does not update the initial value of the scale factor if the difference is less than or equal to the threshold.

[0043] This allows the image encoding device to update the initial value of the scale factor, which is determined according to the feature quantity based on the final scale factor, to an optimal value, thereby suppressing processing delays in subsequent processing.

[0044] Further, for example, when the difference between the scale factor determined by repeating the update of the scale factor and the initial value of the scale factor is greater than a first threshold value, the quantization processor increments a count value; when the difference is less than or equal to the first threshold value, the quantization processor does not increment the count value; when the count value is greater than a second threshold value, the quantization processor updates the initial value of the scale factor; and when the count value is less than or equal to the second threshold value, the quantization processor does not update the initial value of the scale factor.

[0045] As a result, the image encoding apparatus can update the initial value of the scale factor determined according to the feature amount at an appropriate update frequency based on the final scale factor, and thus can suppress processing delay in subsequent processing.

[0046] Further, for example, the quantization processor initializes a first scale factor, quantizes the data according to the first scale factor, obtains a first prediction code amount of the data according to the data quantized according to the first scale factor, updates the first scale factor according to a comparison result between the first prediction code amount and a target code amount, and repeats quantization of the data, acquisition of the first prediction code amount, and update of the first scale factor until the first prediction code amount conforms to the target code amount, thereby determining the first scale factor, determining a second scale factor according to the feature amount of the data, quantizing the data according to the second scale factor, obtaining a second prediction code amount of the data according to the data quantized according to the second scale factor, and determining one of the first scale factor and the second scale factor as the scale factor based on a comparison result between the first prediction code amount and the target code amount and a comparison result between the second prediction code amount and the target code amount.

[0047] As a result, the image encoding device can determine the scale factor by using both a method of searching for a scale factor such that the predicted code amount conforms to the target code amount and a method of determining the scale factor based on the feature amount. Therefore, the image encoding device can suppress the scale factor corresponding to the local solution in the search from being determined as the final scale factor.

[0048] Also, for example, the quantization processor initializes the first scale factor according to the feature amount of the data.

[0049] As a result, the image encoding device can appropriately initialize the scale factor in the search for the scale factor and suppress the processing delay.

[0050] Also, for example, an image encoding method according to an aspect of the present disclosure includes steps of: obtaining a feature amount of data of a component in a processing target block among a plurality of blocks of the image for each of a plurality of components constituting pixels of the image; calculating a feature amount ratio of the plurality of components according to the feature amount; obtaining a variance value regarding the data of the one component in the processing target block for one of the plurality of components; performing a flatness determination as to whether the processing target block is flat according to the variance value; determining a target code amount of the data of each of the plurality of components according to the feature amount ratio, and correcting the target code amount determined according to the feature amount ratio according to the result of the flatness determination; performing frequency conversion on the data of each of the plurality of components; quantizing the data of the component after frequency conversion for each of the plurality of components according to the target code amount of the data of the component; and encoding the data of the component after quantization for each of the plurality of components.

[0051] This makes it possible to adjust the target code size for each component of the data according to the feature ratio of multiple components. Therefore, it becomes possible to suppress significant feature loss. Furthermore, it becomes possible to adjust the target code size according to whether the processing block is flat or not, that is, according to the likelihood of noise being noticeable. Therefore, it becomes possible to suppress the generation of noticeable noise. Thus, it becomes possible to suppress image quality degradation.

[0052] The embodiments will be described below with reference to the drawings. Note that the embodiments described below are either comprehensive or specific examples. The numerical values, shapes, materials, components, arrangement and connection configurations of components, steps, and the order of steps shown in the following embodiments are examples only and are not intended to limit the scope of the claims.

[0053] Figure 1 is a block diagram showing the configuration of an image encoding device in the first reference example. The image encoding device 100 shown in Figure 1 encodes an image block by block. Each image is composed of multiple pixels, and each block is also composed of multiple pixels. A block may be a randomly accessible area called an MCU (Minimum Coded Unit). For example, a 16x8 pixel MCU can be used as a block. The image encoding device 100 performs fixed-length compression on each block of the image according to the target code amount.

[0054] Furthermore, each pixel in an image is composed of multiple components, such as luminance and chrominance. When the image encoding device 100 encodes an image block by block, it encodes each block of the image component by component. For example, the image encoding device 100 encodes the luminance data and chrominance data in a block independently.

[0055] Specifically, the image encoding device 100 comprises a frequency converter 110, a quantization processor 120, and an encoder 130. The quantization processor 120 further comprises a quantizer 121, a quantization table derivator 122, and a scale factor determinator 123. For example, these components are electrical circuits.

[0056] The frequency converter 110 performs frequency conversion on each component constituting a pixel, using the data of that component in the processing block. For example, DCT (Discrete Cosine Transform) is used for frequency conversion. As a result, for each component, the multiple pixel values ​​of that component in the processing block are converted into multiple frequency coefficients of that component in the processing block.

[0057] The quantization processor 120 quantizes the converted data of each component in the processing block according to the same fixed target code quantity for multiple components. This compresses the data.

[0058] Specifically, the scale factor determiner 123 determines the scale factor according to the target code amount, the transformed data, and the fixed-length coding algorithm so that the code amount of the data matches the target code amount. The quantization table derivator 122 derives a quantization table according to the scale factor. The quantizer 121 quantizes the transformed data according to the quantization table.

[0059] In a quantization table, the quantization width is defined for each frequency level. The scale factor affects the quantization width defined for each frequency level in the quantization table. For example, as the scale factor increases, the quantization width also increases. That is, the quantization width may have a monotonically increasing relationship with the scale factor, either in the narrow or broad sense. Alternatively, the quantization width may be proportional to the scale factor. Furthermore, a scale factor may be used such that as the scale factor increases, the quantization width decreases.

[0060] The encoder 130 encodes the quantized data of each component in the processing block into a stream. Huffman coding may be used for encoding. The encoder 130 may also feed back the code amount to the scale factor determiner 123. The scale factor determiner 123 may then re-determine the scale factor by updating it according to the code amount. The determination of the scale factor, derivation of the quantization table, quantization, and encoding may then be repeated.

[0061] This encodes the blocks to be processed according to the target code amount. The target code amount may be a register setting value set in the image encoding device 100. Specifically, a compression ratio such as 50% or 25% may be used as the target code amount. The compression ratio is the ratio of the capacity of the compressed data to the capacity of the uncompressed data. In the first reference example, the code amount of the data of each component in each block of the image is controlled according to the same fixed target code amount.

[0062] For example, the code amounts for luminance data and chrominance data are controlled according to the same fixed target code amount. Specifically, the scale factor for luminance data is determined according to the fixed target code amount. Then, the luminance data is quantized and encoded according to the scale factor for luminance data. Similarly, the scale factor for chrominance data is determined according to the same fixed target code amount. Then, the chrominance data is quantized and encoded according to the scale factor for chrominance data.

[0063] Figure 2 is a flowchart showing the operation of the image encoding device 100 shown in Figure 1. First, the image is divided into multiple blocks (S101). This division process (S101) may be performed by a divider (not shown) of the image encoding device 100, or by a frequency converter 110. After that, a loop processing (S102 to S106) is performed on a block-by-block basis.

[0064] In the block-based loop processing (S102-S106), the frequency converter 110 performs frequency conversion on the data of each component in the processing block (S102). Subsequently, the quantization processor 120 quantizes the converted data of each component in the processing block according to the scale factor (S103). Then, the encoder 130 encodes the quantized data of each component in the processing block (S104).

[0065] Here, for each component, if the amount of encoded data generated for that component in the processing block matches the target amount of code (Yes in S105), processing of that processing block is terminated. On the other hand, if the amount of encoded data generated does not match the target amount of code (No in S105), the quantization processor 120 updates the scale factor (S106).

[0066] For example, if the generated code amount is greater than the target code amount, the quantization processor 120 increases the scale factor. If the generated code amount is smaller than the target code amount, the quantization processor 120 decreases the scale factor. Then, quantization (S103), encoding (S104), and updating the scale factor (S106) are repeated until the generated code amount matches the target code amount.

[0067] As a result, the block to be processed is encoded according to the target code amount. The image encoding device 100 also performs quantization (S103), encoding (S104), and scaling factor update (S106) for each component. Meanwhile, the image encoding device 100 controls the generated code amount by using the same fixed target code amount for multiple components.

[0068] The quantization table derivator 122 may derive a quantization table by reflecting a scale factor in a standard quantization table. Alternatively, the standard quantization table may be defined for each component.

[0069] Figure 3A shows a standard quantization table used for luminance data. In the quantization table, the quantization width is defined for each frequency level. In Figure 3A, the numbers in the quantization table indicate the quantization width. Also, in the quantization table, the upper left corresponds to low frequencies and the lower right corresponds to high frequencies. Based on the fact that humans are insensitive to subtle changes, the quantization width is generally set to be small for low frequencies and large for high frequencies. This suppresses subjective image quality degradation and reduces the amount of code.

[0070] Figure 3B shows a standard quantization table used for color difference data. Similar to Figure 3A, the numbers in the quantization table in Figure 3B represent the quantization range. Humans perceive luminance and color difference differently. In accordance with these differences in human perception of luminance and color difference, a different quantization table from the standard quantization table for luminance data is defined as the standard quantization table for color difference data. However, the standard quantization table used for luminance data may also be used for color difference data.

[0071] Figure 4 is a conceptual diagram showing a compression example in the first reference example. Specifically, it shows an example of compression of luminance data and chrominance data in a 16x8 pixel MCU. Here, the YUV422 format is used, and the 16x8 pixel MCU contains 16x8 Y values ​​(luminance values), 8x8 Cb values ​​(blue chrominance values), and 8x8 Cr values ​​(red chrominance values). Furthermore, each Y value, Cb value, and Cr value is represented by 8 bits.

[0072] Therefore, the amount of luminance data (Y) in the MCU before compression is 16 × 8 × 8 = 1024 bits. The amount of code obtained by encoding this data with a compression ratio of 25% is 1024 × 25% = 256 bits. Similarly, the amount of chrominance data (Cb and Cr) in the MCU before compression is 16 × 8 × 8 = 1024 bits. The amount of code obtained by encoding this data with a compression ratio of 25% is 1024 × 25% = 256 bits.

[0073] In the first example, the same fixed target code value (compression ratio) is used for both luminance and chrominance. That is, the compression ratio and code value are controlled to be the same for both the luminance data (Y) and the chrominance data (Cb and Cr).

[0074] Furthermore, the 16x8 Y values ​​in a 16x8 pixel MCU may be divided into two sets, each consisting of 8x8 Y values. Frequency conversion, quantization, and encoding may then be performed in units of 8x8 values. Similarly, frequency conversion, quantization, and encoding may also be performed in units of 8x8 values ​​for the 8x8 Cb values ​​and 8x8 Cr values.

[0075] Figure 5 is a conceptual diagram illustrating mosquito noise caused by quantization errors. Image encoding and decoding processes involve frequency transformation, quantization, inverse quantization, and inverse frequency transformation. Specifically, frequency transformation and quantization are performed during image encoding, and inverse quantization and inverse frequency transformation are performed during image decoding. Here, an example is shown where frequency transformation, quantization, inverse quantization, and inverse frequency transformation are performed on an 8x8 pixel block of an image.

[0076] First, in the encoding process, the blocks of the input image undergo frequency transformation. Specifically, the multiple pixel values ​​that make up a block are decomposed into multiple frequency coefficients that make up the block, according to multiple basis sets of the frequency transformation.

[0077] Here, the 8x8 pixel values ​​that make up a block are converted into 8x8 frequency coefficients that make up a block (bottom left of Figure 5). Similar to the quantization tables shown in Figures 3A and 3B, in the converted block data, the top left corresponds to low frequencies and the bottom right corresponds to high frequencies. For example, if edges exist in the blocks of the input image, non-zero frequency coefficients exist not only in the low-frequency region but also in the mid-frequency and high-frequency regions.

[0078] Next, the converted data is quantized. Specifically, each frequency coefficient is quantized according to the corresponding quantization width in the quantization table. As a result, an 8x8 quantized frequency coefficient that makes up a block is obtained (bottom center of Figure 5). This compresses the data. In particular, in the high-frequency region, the frequency coefficients are greatly compressed by the large quantization width.

[0079] Subsequently, during the decoding process, the quantized data is dequantized. Specifically, each quantization frequency coefficient is dequantized according to the corresponding quantization width in the quantization table. As a result, an 8x8 frequency coefficient that constitutes a block is obtained (bottom right of Figure 5). This expands the data.

[0080] The data in a block after quantization and dequantization differs from the data in the block before quantization and dequantization. The data error between these blocks is caused by the rounding of numbers during quantization and is called the quantization error. In particular, in the high-frequency range, a large quantization width is used, resulting in a large quantization error.

[0081] Subsequently, an inverse frequency transform is performed on the inversely quantized data. This combines the multiple frequency coefficients that make up a block into multiple pixel values ​​that make up the block, according to the multiple basis sets of the frequency transform. Specifically, the 8x8 frequency coefficients that make up a block are transformed into 8x8 pixel values ​​that make up a block. This then yields the blocks of the reconstructed image.

[0082] On the other hand, irreversible algorithms can cause noise to occur. Specifically, in blocks of playback images, quantization errors can generate mosquito noise, leading to image quality degradation. Mosquito noise is particularly easily perceived visually in flat areas around edges. In addition, in moving images, the noise is more noticeable because it moves.

[0083] Figure 6 is a block diagram showing the configuration of the image encoding device in the second reference example. The image encoding device 100 shown in Figure 6 encodes an image block by block. Each image is composed of multiple pixels, and each block is also composed of multiple pixels. A block may be a unit of randomly accessible area called an MCU. For example, a 16x8 pixel MCU may be used as a block. The image encoding device 100 performs fixed-length or variable-length compression on each block of the image according to the target code amount.

[0084] Furthermore, each pixel in an image is composed of multiple components, such as brightness and color difference. When the image encoding device 100 encodes an image block by block, it encodes each block of the image component by component.

[0085] Specifically, the image encoding device 100 comprises a frequency converter 110, a quantization processor 120, an encoder 130, a feature extractor 140, a ratio calculator 142, and a target code amount controller 150. Furthermore, the quantization processor 120 comprises a quantizer 121, a quantization table derivator 122, and a scale factor determinator 123. For example, these components are electrical circuits.

[0086] The feature extractor 140 acquires the data feature quantities of each component constituting a pixel in the processing block. In this way, the feature extractor 140 acquires multiple feature quantities for each of multiple components. The data feature quantities may correspond to the complexity of the data. Note that, for each component, the data feature quantity of that component in the processing block may be expressed as the feature quantity of that component.

[0087] The ratio calculator 142 calculates the feature ratios of multiple components. Specifically, the ratio calculator 142 calculates the feature ratios of multiple components by taking the ratios of multiple features obtained by the feature extractor 140 for each component.

[0088] The target code amount controller 150 determines the target code amount of each component of data in the processing block according to the feature ratio. For example, for each component, the target code amount controller 150 increases the target code amount of the data for that component as the proportion of the features in the feature ratio increases. Note that for each component, the target code amount of the data for that component in the processing block may be expressed as the target code amount of that component.

[0089] The frequency converter 110 performs frequency conversion on the data of each component in the processing block. For example, a DCT is used for frequency conversion. As a result, for each component, the multiple pixel values ​​of that component in the processing block are converted into multiple frequency coefficients of that component in the processing block.

[0090] The quantization processor 120 quantizes the converted data of each component in the target block according to the target code amount of the data of that component in the target block. This compresses the data.

[0091] Specifically, the scale factor determinator 123 determines the scale factor according to the target code size, the transformed data, and the fixed-length coding algorithm so that the code size of the data fits the target code size. Here, the target code size is the target code size determined according to the feature ratio as the target code size of the data for each component.

[0092] The quantization table derivator 122 derives a quantization table for each component according to the scale factor for the data of that component. The quantizer 121 quantizes the converted data of each component according to the quantization table for the data of that component.

[0093] In a quantization table, the quantization width is defined for each frequency level. The scale factor affects the quantization width defined for each frequency level in the quantization table. For example, as the scale factor increases, the quantization width also increases. That is, the quantization width may have a monotonically increasing relationship with the scale factor, either in the narrow or broad sense. Alternatively, the quantization width may be proportional to the scale factor. Furthermore, a scale factor may be used such that as the scale factor increases, the quantization width decreases.

[0094] The encoder 130 encodes the quantized data of each component in the processing block into a stream. Huffman coding may be used for encoding.

[0095] Furthermore, the encoder 130 may feed back the code amount of the data for each component to the scale factor determiner 123. The scale factor determiner 123 may then re-determine the scale factor for each component by updating the scale factor for the data of that component according to the code amount of the data of that component. The determination of the scale factor, derivation of the quantization table, quantization, and encoding may then be repeated.

[0096] This encodes the blocks to be processed according to the target code weight. Here, the target code weight is the target code weight determined according to the feature ratio as the target code weight for the data of each component. Specifically, for each component, the compression ratio for the data of that component may be used as the target code weight for the data of that component. In particular, the code weight of the data of each component in each block of the image is controlled according to the target code weight of the data of that component in that block.

[0097] For example, for a block to be processed, the feature ratio of luminance data and chrominance data is obtained. Then, the target code mass for the luminance data and the target code mass for the chrominance data are determined according to the feature ratio of luminance data and chrominance data. Then, the scale factor for the luminance data and the scale factor for the chrominance data are determined according to the target code mass for the luminance data and the target code mass for the chrominance data, respectively.

[0098] The luminance data and chrominance data are then quantized and encoded according to scale factors for the luminance data and chrominance data, respectively. Therefore, the code values ​​of the luminance data and chrominance data are controlled according to separate, variable target code values.

[0099] Figure 7 is a flowchart showing the operation of the image encoding device 100 shown in Figure 6. First, the image is divided into multiple blocks (S111). This division process (S111) may be performed by a divider (not shown) of the image encoding device 100, or by a frequency converter 110. After that, a loop processing (S112 to S119) is performed on a block-by-block basis.

[0100] In the block-based loop processing (S112-S119), the feature extractor 140 acquires the feature quantities of the data for each component in the processing block (S112). Next, the ratio calculator 142 calculates the feature quantity ratio of multiple components (S113). Then, the target code quantity controller 150 determines the target code quantity of the data for each component in the processing block according to the feature quantity ratio (S114). The frequency converter 110 performs frequency conversion on the data for each component in the processing block (S115).

[0101] Subsequently, the quantization processor 120 quantizes the converted data of each component in the processing block according to the scale factor (S116). Then, the encoder 130 encodes the quantized data of each component in the processing block (S117).

[0102] Here, for each component, if the generated code amount of the encoded data for that component in the processing block matches the target code amount of the data for that component in the processing block (Yes in S118), processing of that processing block is terminated. On the other hand, if the generated code amount does not match the target code amount (No in S118), the quantization processor 120 updates the scale factor (S119).

[0103] For example, if the generated code amount is greater than the target code amount, the quantization processor 120 increases the scale factor. If the generated code amount is smaller than the target code amount, the quantization processor 120 decreases the scale factor. Then, quantization (S116), encoding (S117), and updating the scale factor (S119) are repeated until the generated code amount matches the target code amount.

[0104] In the above operation, for each component, feature acquisition (S112), target code size determination (S114), frequency conversion (S115), quantization (S116), encoding (S117), and scale factor update (S119) are performed. That is, for each component, feature values ​​are acquired and the target code size is determined. Then, the block to be processed is encoded according to the target code size of each component.

[0105] Furthermore, in the above operation, multiple target code values ​​are determined for each component according to the feature ratio of the multiple components. For example, a larger target code value is assigned to a component that has a larger proportion in the feature ratio. This makes it possible to assign a larger target code value to a component with more features, thereby suppressing significant feature loss.

[0106] On the other hand, noise such as mosquito noise and block noise is more noticeable in flat regions. In flat regions, it is assumed that there are few features and the feature amount is small. Therefore, it is possible that small feature amounts will be acquired and small target coding amounts will be assigned to flat regions. And when small target coding amounts are assigned to flat regions, it is possible that a lot of noticeable noise will be generated.

[0107] Furthermore, in flat regions, the feature size may be small for all components. Therefore, multiple components may occupy the same proportion in the feature ratio. Consequently, the target code size determined according to the feature ratio may not be appropriate for flat regions.

[0108] Figure 8 is a block diagram showing the configuration of the image encoding device in this embodiment. The image encoding device 200 shown in Figure 8 encodes an image block by block. Each image is composed of multiple pixels, and each block is also composed of multiple pixels. A block may be a unit of randomly accessible area called an MCU. For example, a 16x8 pixel MCU can be used as a block. The image encoding device 200 performs fixed-length or variable-length compression on each block of the image according to the target encoding amount.

[0109] Furthermore, each pixel in an image is composed of multiple components, such as luminance and chrominance. When the image encoding device 200 encodes an image block by block, it encodes each block of the image component by component. The multiple components may consist of two components, luminance and chrominance, or three components, red, green, and blue, corresponding to RGB. Alternatively, the multiple components may consist of four components, including transparency (alpha) in addition to the three components of red, green, and blue, corresponding to RGBA (Red, Green, Blue, Alpha).

[0110] Specifically, the image coding device 200 includes a frequency converter 210, a quantization processor 220, an encoder 230, a feature extractor 240, a ratio calculator 242, a variance extractor 244, a flatness selector 246, and a target code amount controller 250. The quantization processor 220 also includes a quantizer 221, a quantization table derivator 222, and a scale factor determiner 223. For example, these components are electrical circuits.

[0111] The feature extractor 240 acquires the data feature quantities of each component constituting a pixel in the processing block. In this way, the feature extractor 240 acquires multiple feature quantities for each of multiple components. The data feature quantities may correspond to the complexity of the data. Note that, for each component, the data feature quantity of that component in the processing block may be expressed as the feature quantity of that component.

[0112] For example, the feature extractor 240 may calculate a statistical value of the absolute difference between adjacent pixels in the data for each component as a feature of the data for that component. The statistical value may be a sum or an average. The average of the absolute difference between adjacent pixels can also be expressed as activity.

[0113] Furthermore, the feature extractor 240 may obtain the feature quantities of the data for each component using the Hadamard transform. For example, the feature extractor 240 may obtain the amount of edges obtained by applying the Hadamard transform to the data for each component as the feature quantity of the data for that component.

[0114] Furthermore, for example, the feature acquisition device 240 may acquire information indicating the feature quantities of the data for each component from an external device of the image encoding device 200. The external device may be a device that calculates feature quantities. Alternatively, the external device may be an imaging device that determines feature quantities based on imaging conditions. Alternatively, the external device may determine feature quantities according to the image type.

[0115] The ratio calculator 242 calculates the feature ratios of multiple components. Specifically, the ratio calculator 242 calculates the feature ratios of multiple components by taking the ratios of multiple features obtained by the feature extractor 240 for each component.

[0116] The feature ratio may be expressed for each component as the ratio of the feature amount of that component to the total feature amount of multiple components. In other words, the feature ratio may be expressed as multiple ratios corresponding to multiple components. Alternatively, the feature ratio may be expressed as a single ratio for two components, such as luminance and chrominance, as the ratio of the feature amount of one component to the feature amount of the other component, or as the ratio of the feature amount of one component to the total feature amount of all components.

[0117] The variance value retrieval unit 244 retrieves the variance value for one of the multiple components in the data of that component within the processing block. For example, the variance value indicates the dispersion of values ​​in the data of that one component. Specifically, the variance value may correspond to variance or standard deviation in mathematics.

[0118] Furthermore, one component used to obtain the variance value may be a component that is assumed to have a larger feature compared to the other components. Alternatively, one component used to obtain the variance value may be a component that is more easily visible and stands out compared to the other components.

[0119] For example, the variance value acquirer 244 may acquire a single variance value for the entire data of one component in the block to be processed. Alternatively, the variance value acquirer 244 may acquire variance values ​​for each subblock of the data of one component in the block to be processed. That is, the variance value acquirer 244 may acquire multiple variance values, each corresponding to multiple subblocks in the block to be processed.

[0120] The flatness determination unit 246 determines whether the block to be processed is flat or not, according to the variance value. For example, the flatness determination unit 246 may determine that the block to be processed is flat if the variance value obtained by the variance value acquisition unit 244 is smaller than a threshold.

[0121] Alternatively, the flatness determination unit 246 may determine whether a subblock is flat or not according to the variance value obtained for each of the multiple subblocks in the block to be processed. Then, the flatness determination unit 246 may determine whether the block to be processed is flat or not according to the number of subblocks that have been determined to be flat among the multiple subblocks.

[0122] Specifically, the flatness determination unit 246 may determine that a subblock is flat if the variance value obtained for each of the multiple subblocks in the block to be processed is smaller than the first threshold. Furthermore, the flatness determination unit 246 may determine that the block to be processed is flat if the number of subblocks determined to be flat among the multiple subblocks is greater than the second threshold.

[0123] The target code amount controller 250 determines the target code amount of each component of data in the processing block according to the feature ratio. For example, for each component, the target code amount controller 250 increases the target code amount of the data for that component as the proportion of the features in the feature ratio increases. Note that for each component, the target code amount of the data for that component in the processing block may be expressed as the target code amount of that component.

[0124] For example, the target code amount controller 250 determines the target code amount for each component according to the relationship between the proportion of the feature quantities of that component and the proportions of the feature quantities of the other components. Specifically, if the proportion of the feature quantities of the first component is greater than the proportion of the feature quantities of the second component, the first target code amount of the first component may be set to be greater than the second target code amount of the second component.

[0125] Furthermore, the target code amount controller 250 may maintain the total target code amount of data for multiple components in multiple blocks at a reference target code amount, and adjust the ratio of target code amounts between components for each block according to the feature ratio. Alternatively, the target code amount controller 250 may determine the target code amount corresponding to the feature ratio by referring to a reference table that associates feature ratios with target code amounts.

[0126] Furthermore, the target code amount controller 250 corrects the target code amount for each component, which is determined according to the feature ratio, according to the result of the flatness determination. For example, if the block to be processed is determined to be flat, the target code amount controller 250 increases the target code amount of one component and decreases the target code amounts of the other components. The component whose target code amount is increased may be a component that is assumed to have a large feature compared to the other components, a component that is easily visible and stands out, or a component used to obtain the variance value.

[0127] Furthermore, if it is determined that the block to be processed is flat, the target code amount controller 250 may add a predetermined first offset amount to the target code amount of one component, or subtract a predetermined second offset amount from the target code amounts of the other components.

[0128] Furthermore, if the block to be processed is determined to be flat, the target code amount controller 250 may correct the feature ratio and determine the target code amount for each component according to the corrected feature ratio. As a result, the target code amount is corrected from the target code amount determined according to the feature ratio before correction to the target code amount determined according to the feature ratio after correction. In other words, correcting the target code amount determined according to the feature ratio according to the result of the flatness determination corresponds to determining the target code amount according to the feature ratio and the result of the flatness determination.

[0129] The frequency converter 210 performs frequency conversion on the data of each component in the processing block. For example, a DCT is used for frequency conversion. As a result, for each component, the multiple pixel values ​​of that component in the processing block are converted into multiple frequency coefficients of that component in the processing block.

[0130] The quantization processor 220 quantizes the converted data of each component in the target block according to the target code amount of the data for that component in the target block. This compresses the data.

[0131] Specifically, the scale factor determinator 223 determines the scale factor according to the target code size, the transformed data, and the fixed-length coding algorithm so that the code size of the data fits the target code size. Here, the target code size is the target code size determined according to the feature ratio and the result of the flatness determination as the target code size of the data for each component.

[0132] The quantization table derivator 222 derives a quantization table for each component according to the scale factor for the data of that component. The quantizer 221 quantizes the converted data of each component according to the quantization table for the data of that component.

[0133] In a quantization table, the quantization width is defined for each frequency level. The scale factor affects the quantization width defined for each frequency level in the quantization table. For example, as the scale factor increases, the quantization width also increases. That is, the quantization width may have a monotonically increasing relationship with the scale factor, either in the narrow or broad sense. Alternatively, the quantization width may be proportional to the scale factor. Furthermore, a scale factor may be used such that as the scale factor increases, the quantization width decreases.

[0134] The encoder 230 encodes the quantized data of each component in the processing block into a stream. Huffman coding may be used for encoding.

[0135] Furthermore, the encoder 230 may feed back the code amount of the data for each component to the scale factor determinator 223. The scale factor determinator 223 may then re-determine the scale factor for each component by updating the scale factor for the data of that component according to the code amount of the data of that component. The determination of the scale factor, derivation of the quantization table, quantization, and encoding may then be repeated.

[0136] This encodes the blocks to be processed according to the target code weight. Here, the target code weight is the target code weight for each component of the data determined according to the feature ratio and the result of the flatness determination. Specifically, for each component, the compression ratio for the data of that component may be used as the target code weight for the data of that component. In particular, the code weight of the data of each component in each block of the image is controlled according to the target code weight of the data of that component in that block.

[0137] Furthermore, the reference target code amount may be set as a register setting value in the image encoding device 200. The target code amount controller 250 may then determine the target code amount by applying a gain or offset to the reference target code amount according to the feature ratio. The reference target code amount may be the same for multiple components, or it may be different for each component. Also, the reference target code amount may correspond to the total target code amount mentioned above.

[0138] For example, for a block to be processed, the feature ratio of luminance data and chrominance data is obtained. The result of a flatness determination based on the variance of the luminance data is also obtained. The target code mass for the luminance data and the target code mass for the chrominance data are then determined according to the feature ratio of luminance data and chrominance data, and the result of the flatness determination based on the variance of the luminance data. The scale factor for the luminance data and the scale factor for the chrominance data are then determined according to the target code mass for the luminance data and the target code mass for the chrominance data, respectively.

[0139] The luminance data and chrominance data are then quantized and encoded according to scale factors for the luminance data and chrominance data, respectively. Therefore, the code values ​​of the luminance data and chrominance data are controlled according to separate, variable target code values.

[0140] Furthermore, a general characteristic of images, especially natural images, is that changes in color difference tend to be smaller than changes in brightness. Therefore, noise based on quantization errors is less likely to occur in color difference. Also, a characteristic of human vision is that human vision is less sensitive to changes in color difference than to changes in brightness. Therefore, noise based on quantization errors is less likely to be visible in color difference. Considering these general characteristics of images and human vision, it may be beneficial to increase the target code amount of brightness data compared to color difference data.

[0141] Therefore, in the above configuration of the image encoding device 200, luminance data may be actively protected. Specifically, the target code amount controller 250 may determine the target code amounts for the luminance data and the chrominance data such that the target code amount for the luminance data is greater than the target code amount for the chrominance data.

[0142] In particular, as mentioned above, noise is more noticeable in flat regions. On the other hand, the target code amount determined according to the feature ratio may not be appropriate for flat regions. Therefore, if the block to be processed is determined to be flat, the target code amount for the luminance data may be increased by correcting the target code amount determined according to the feature ratio. This suppresses the generation of noticeable noise.

[0143] Figure 9 is a flowchart illustrating the operation of the image encoding device 200 shown in Figure 8. First, the image is divided into multiple blocks (S201). This division process (S201) may be performed by a divider (not shown) of the image encoding device 200, or by a frequency converter 210. After that, a block-by-block loop processing (S202 to S212) is performed.

[0144] In the block-based loop processing (S202-S212), the feature extractor 240 acquires the feature quantities of the data for each component in the block to be processed (S202). Next, the ratio calculator 242 calculates the feature quantity ratio of multiple components according to the feature quantities acquired for each component (S203). Next, the target code quantity controller 250 determines the target code quantity of the data for each component in the block to be processed according to the feature quantity ratio (S204).

[0145] In parallel with the acquisition of features (S202), the calculation of feature ratios (S203), and the determination of the target code size (S204), the variance value acquirer 244 acquires the variance value of one component of the data in the block to be processed (S205).

[0146] Subsequently, the flatness determination unit 246 performs a flatness determination according to the variance value to determine whether the block to be processed is flat or not (S206). If the block to be processed is determined to be flat (Yes in S206), the target code amount controller 250 corrects the target code amount of the data for that component in the block to be processed for each component (S207). On the other hand, if the block to be processed is determined to be not flat (No in S206), the target code amount controller 250 maintains the target code amount of the data for that component in the block to be processed without correction for each component.

[0147] Then, the frequency converter 210 performs frequency conversion on the data of each component in the block to be processed (S208).

[0148] Subsequently, the quantization processor 220 quantizes the converted data of each component in the processing block according to the scale factor (S209). Then, the encoder 230 encodes the quantized data of each component in the processing block (S210).

[0149] Here, for each component, if the generated code amount of the encoded data for that component in the processing block matches the target code amount of the data for that component in the processing block (Yes in S211), processing of that processing block is terminated. On the other hand, if the generated code amount does not match the target code amount (No in S211), the quantization processor 220 updates the scale factor (S212).

[0150] For example, if the generated code amount is larger than the target code amount, the quantization processor 220 increases the scale factor to increase the quantization width. If the generated code amount is smaller than the target code amount, the quantization processor 220 decreases the scale factor to decrease the quantization width. Then, quantization (S209), encoding (S210), and updating the scale factor (S212) are repeated until the generated code amount conforms to the target code amount.

[0151] In the above operation, the target code amount for each component of the data is adjusted according to the feature ratio of multiple components. Therefore, significant feature loss is suppressed. Furthermore, the target code amount is adjusted according to whether the block being processed is flat or not, that is, according to the likelihood of noise being noticeable. Therefore, the generation of noticeable noise is suppressed. Thus, image quality degradation is suppressed.

[0152] Figure 10A is a conceptual diagram showing an example of calculating feature quantities for luminance data. For example, the average of the absolute difference values ​​between adjacent pixels of luminance data is calculated as a feature quantity for luminance data. Figure 10A shows an equation for calculating the average absolute difference value between adjacent pixels of luminance data in a 16x8 pixel MCU in YUV422 format. The feature quantity acquirer 240 may calculate the average absolute difference value between adjacent pixels of luminance data as a feature quantity for luminance data according to the equation shown in Figure 10A.

[0153] In the equation in Figure 10A, for example, Yn(i,j) at n=0 represents the Y value of the pixel located at (i,j) in the left 8x8 pixel array, and Yn(i,j) at n=1 represents the Y value of the pixel located at (i,j) in the right 8x8 pixel array. Act_y in Figure 10A roughly corresponds, though not strictly, to the average absolute difference between adjacent pixels of luminance data.

[0154] Figure 10B is a conceptual diagram showing an example of calculating feature quantities for color difference data. For example, the average of the absolute difference values ​​between adjacent pixels in the color difference data is calculated as a feature quantity for the color difference data. Figure 10B shows an equation for calculating the average absolute difference value between adjacent pixels in color difference data for a 16x8 pixel MCU in YUV422 format. The feature quantity acquirer 240 may calculate the average absolute difference value between adjacent pixels in the color difference data as a feature quantity for the color difference data according to the equation shown in Figure 10B.

[0155] In the equation in Figure 10B, for example, Cb(i,j) represents the Cb value of the pixel located at (i,j) in the 8x8 pixel array corresponding to Cb, and Cr(i,j) represents the Cr value of the pixel located at (i,j) in the 8x8 pixel array corresponding to Cr. Act_c in Figure 10B roughly corresponds, though not strictly, to the average absolute difference between adjacent pixels of luminance data.

[0156] Figure 11A is a reference table showing the relationship between feature ratios and target code amounts. In this example, the feature ratio used corresponds to the ratio of the luminance data features to the chrominance data features. For example, if the ratio of the luminance data features to the chrominance data features is less than 1, then 0 is used as the feature ratio.

[0157] Furthermore, if the ratio is 1 or more but less than 2, 1 is used as the feature ratio. If the ratio is 2 or more but less than 3, 2 is used as the feature ratio. If the ratio is 3 or more but less than 4, 3 is used as the feature ratio. If the ratio is 4 or more but less than 5, 4 is used as the feature ratio. If the ratio is 5 or more but less than 6, 5 is used as the feature ratio. If the ratio is 6 or more but less than 7, 6 is used as the feature ratio. If the ratio is 7 or more, 7 is used as the feature ratio.

[0158] Then, the target code amounts for luminance data and chrominance data are associated with the feature ratio. In this example, the larger the feature ratio, the larger the target code amount for luminance data and the smaller the target code amount for chrominance data. The sum of the target code amounts for luminance data and chrominance data is constant regardless of the feature ratio. Also, for any feature ratio, the target code amount for luminance data is larger than the target code amount for chrominance data.

[0159] In the example in Figure 11A, similar to the example in Figure 4, the data size of the uncompressed luminance data and the data size of the uncompressed chrominance data are each 1024 bits, for a total of 2048 bits. In the reference table in Figure 11A, the ratio of the target code size to the uncompressed data size is shown as the compression ratio.

[0160] In addition, the reference table may associate only the target code amount, expressed in bits for luminance or chrominance, with the feature ratio, or it may associate only the compression ratio for luminance or chrominance with the feature ratio as information indicating the target code amount.

[0161] Figure 11B is a graph showing the relationship between feature ratio and compression ratio. The graph in Figure 11B corresponds to the values ​​shown in Figure 11A.

[0162] The larger the relative features of the luminance data, the larger the target encoding (compression ratio) for the luminance data, and the smaller the target encoding (compression ratio) for the chrominance data. Furthermore, for any feature ratio, the target encoding of the luminance data is larger than that of the chrominance data. The larger the relative features of the luminance data, the greater the difference between the target encoding (compression ratio) of the luminance data and the target encoding (compression ratio) of the chrominance data.

[0163] Figure 12 is a conceptual diagram showing a first example of calculating the variance value. In this example, the variance value represents the variance of multiple values ​​contained in the luminance data. Figure 12 also shows an equation for calculating the variance of multiple values ​​contained in 16x8 pixel luminance data in units of 8x8 pixels. In other words, in this example, two variance values ​​are calculated, corresponding to two subblocks.

[0164] In the equation in Figure 12, for example, Y0(i,j) represents the brightness value (Y value) of the pixel located at (i,j) in the left 8x8 pixel array, and Y1(i,j) represents the brightness value (Y value) of the pixel located at (i,j) in the right 8x8 pixel array. Furthermore, ave(Y0) represents the average of 8x8 values ​​in the left 8x8 pixel array, and ave(Y1) represents the average of 8x8 values ​​in the right 8x8 pixel array.

[0165] Then, dist_y[0] represents the variance of the 8x8 values ​​in the left 8x8 pixel, and dist_y[1] represents the variance of the 8x8 values ​​in the right 8x8 pixel.

[0166] Figure 13 is a conceptual diagram showing a second example of calculating the variance value. In this example, the variance value represents the variance of multiple values ​​contained in the luminance data. Figure 13 also shows an equation for calculating the variance of multiple values ​​contained in 16x8 pixel luminance data in units of 4x4 pixels. In other words, in this example, eight variance values ​​are calculated, each corresponding to one of the eight subblocks.

[0167] In the equation in Figure 13, for example, similar to the equation in Figure 12, Y0(i,j) represents the brightness value (Y value) of the pixel located at (i,j) in the left 8x8 pixel array, and Y1(i,j) represents the brightness value (Y value) of the pixel located at (i,j) in the right 8x8 pixel array. Furthermore, ave(Yxx) represents the average of the 4x4 values ​​in the 4x4 pixel array specified by Yxx. Finally, dist_y[n] represents the variance of the 4x4 values ​​in the 4x4 pixel array specified by n.

[0168] Figure 14 is a flowchart of the flatness determination process. Specifically, Figure 14 shows an example of the flatness determination process performed by the flatness determination unit 246. In this example, the variance value obtained in the example in Figure 12 or Figure 13 is used to determine whether or not the block to be processed is flat. In this example, multiple parameters expressed as FLAT_MODE, FLAT_THR_DIS, FLAT_THR_BLK4, and FLAT_THR_BLK8 are used.

[0169] FLAT_MODE is a parameter that indicates whether the flatness determination mode is 8x8 pixel unit mode or 4x4 pixel unit mode. If FLAT_MODE is 0, it indicates that the flatness determination mode is 8x8 pixel unit mode. If FLAT_MODE is 1, it indicates that the flatness determination mode is 4x4 pixel unit mode. FLAT_THR_DIS is a parameter that indicates a threshold for the variance value. If FLAT_THR_DIS is equal to 0, it may indicate that flatness determination is not performed.

[0170] FLAT_THR_BLK4 is a parameter that indicates a threshold for the number of subblocks determined to be flat out of eight 4x4 pixel subblocks. FLAT_THR_BLK8 is a parameter that indicates a threshold for the number of subblocks determined to be flat out of two 8x8 pixel subblocks.

[0171] In the example shown in Figure 14, the flatness detector 246 first initializes its variables. Specifically, the flatness detector 246 sets i=0, j=0, flat_flag=0, flat_count8=0, and flat_count4=0 (S231).

[0172] Here, i and j are variables used to specify subblocks. flat_count8 is a variable that indicates the number of subblocks that were determined to be flat out of two 8x8 pixel subblocks. flat_count4 is a variable that indicates the number of subblocks that were determined to be flat out of eight 4x4 pixel subblocks.

[0173] Next, the flatness detector 246 determines whether FLAT_MODE is 0 or not (S232). If FLAT_MODE is 0 (Yes in S232), the flatness determination mode is the 8x8 pixel unit mode. In this case, the flatness detector 246 determines whether dist_y[i] obtained according to the example in Figure 12 is smaller than FLAT_THR_DIS (S233).

[0174] Then, if dist_y[i] is less than FLAT_THR_DIS (Yes in S233), the flatness criterion 246 adds 1 to flat_count8 (S234). In other words, in this case, the flatness criterion 246 counts the subblock being judged as a flat subblock. Then, if dist_y[i] is not less than FLAT_THR_DIS (No in S233), the flatness criterion 246 does not add 1 to flat_count8 and does not count the subblock being judged as a flat subblock.

[0175] Next, the flatness detector 246 adds 1 to i (S235). Then, these processes (S233 to S235) are repeated in the range where i is less than 2.

[0176] Subsequently, the flatness detector 246 determines whether flat_count8 is greater than FLAT_THR_BLK8 (S236). If flat_count8 is greater than FLAT_THR_BLK8 (Yes in S236), the flatness detector 246 sets flat_flag = 1 (S237). If flat_count8 is not greater than FLAT_THR_BLK8 (No in S236), the flatness detector 246 sets flat_flag = 0 (S238).

[0177] If FLAT_MODE is not 0 (No in S232), that is, if FLAT_MODE is 1, the flatness determination mode is the 4x4 pixel unit mode. In this case, the flatness determination unit 246 determines whether dist_y[4*i+j] obtained according to the example in Figure 13 is smaller than FLAT_THR_DIS (S239).

[0178] Then, if dist_y[4*i+j] is less than FLAT_THR_DIS (Yes in S239), the flatness criterion 246 adds 1 to flat_count4 (S240). In other words, in this case, the flatness criterion 246 counts the subblock being judged as a flat subblock. Then, if dist_y[4*i+j] is not less than FLAT_THR_DIS (No in S239), the flatness criterion 246 does not add 1 to flat_count4 and does not count the subblock being judged as a flat subblock.

[0179] Next, the flatness detector 246 adds 1 to j (S241). Then, these processes (S239 to S241) are repeated in the range where j is less than 4.

[0180] Next, the flatness detector 246 adds 1 to i and initializes j to 0 (S242). Then, these processes (S239 to S242) are repeated in the range where i is less than 2.

[0181] Subsequently, the flatness detector 246 determines whether flat_count4 is greater than FLAT_THR_BLK4 (S243). If flat_count4 is greater than FLAT_THR_BLK4 (Yes in S243), the flatness detector 246 sets flat_flag = 1 (S244). If flat_count4 is not greater than FLAT_THR_BLK4 (No in S243), the flatness detector 246 sets flat_flag = 0 (S245).

[0182] Here, flat_flag indicates whether the block to be processed is flat or not. Specifically, if flat_flag is equal to 1, it indicates that the block to be processed is flat, and if it is equal to 0, it indicates that the block to be processed is not flat.

[0183] Figure 15 is a flowchart illustrating the target code amount correction process. Specifically, Figure 15 shows an example of the target code amount correction process, in which the target code amount controller 250 corrects the target code amount determined according to the feature ratio. In this example, the target code amount determined according to the example in Figure 11A is corrected according to the result of the flatness determination process performed according to the example in Figure 14. In this example, several parameters expressed as FLAT_WOFST, FLAT_WLMT, and CONST are used.

[0184] FLAT_WOFST is a parameter that indicates the offset of the feature ratio. If the block to be processed is determined to be flat, the offset indicated by FLAT_WOFST is added to the feature ratio in the example in Figure 11A.

[0185] FLAT_WLMT is a parameter that indicates the group to which the feature ratio offset is applied. If the block being processed is determined to be flat, the offset is added to the feature ratio, provided that the pre-correction value of the feature ratio is smaller than the value indicated by FLAT_WLMT.

[0186] CONST is a parameter that indicates an upper limit on the feature ratio. The corrected value of the feature ratio is clipped according to the upper limit indicated by CONST.

[0187] In the example shown in Figure 15, the target code amount controller 250 first determines whether flat_flag is equal to 1 (S251). Here, flat_flag is a variable set in Figure 14 and indicates whether the block to be processed is flat or not. Specifically, if flat_flag is equal to 1, it indicates that the block to be processed is flat, and if it is equal to 0, it indicates that the block to be processed is not flat.

[0188] Here, if flat_flag is not equal to 1 (No in S251), that is, if flat_flag is equal to 0 and the block to be processed is not flat, the target code amount controller 250 terminates the target code amount correction process. In this case, the target code amount controller 250 determines the target code amount according to the feature ratio without correcting the feature ratio. In other words, in this case, the target code amount controller 250 does not correct the target code amount.

[0189] On the other hand, if flat_flag is equal to 1 (Yes in S251), that is, if the block to be processed is flat, the target code amount controller 250 determines whether w_idx is smaller than FLAT_WLMT (S252). Here, w_idx is a variable that indicates the feature ratio and is determined according to the examples in Figures 10A, 10B and 11A. In other words, the target code amount controller 250 determines whether w_idx, which indicates the feature ratio, is smaller than FLAT_WLMT.

[0190] Here, if w_idx is not smaller than FLAT_WLMT (No in S252), the target code amount controller 250 terminates the target code amount correction process. In this case, the target code amount controller 250 determines the target code amount according to the feature ratio without correcting the feature ratio. That is, in this case, the target code amount controller 250 does not correct the target code amount.

[0191] On the other hand, if w_idx is smaller than FLAT_WLMT (Yes in S252), the target code amount controller 250 adds FLAT_WOFST to w_idx (S253). In other words, in this case, the target code amount controller 250 corrects the feature ratio by adding an offset to the feature ratio.

[0192] Next, the target code amount controller 250 determines whether w_idx is greater than CONST (S254). If w_idx is greater than CONST (Yes in S254), the target code amount controller 250 updates w_idx to CONST (S255). If w_idx is not greater than CONST (No in S254), the target code amount controller 250 does not update w_idx. As a result, w_idx is clipped according to CONST.

[0193] When the feature ratio corresponding to w_idx is corrected, the target code amount controller 250 determines the target code amount according to the corrected feature ratio. This process corresponds to correcting the target code amount determined according to the feature ratio before correction. In other words, the target code amount controller 250 corrects the target code amount by correcting the feature ratio.

[0194] Figure 16 is a conceptual diagram illustrating the target code amount correction process. In the example in Figure 16, the feature ratio is corrected from 1 to 4 in the same reference table as in the example in Figure 11A. Consequently, the target code amount is corrected. When the block to be processed is flat, the feature ratio is corrected as in the example in Figure 16, and consequently, the target code amount is corrected.

[0195] More specifically, if the block to be processed is flat, the feature ratio is adjusted so that the proportion of luminance features in the feature ratio increases. Consequently, the target code values ​​for luminance increase and chrominance decrease, thereby adjusting each target code value. This suppresses the generation of noticeable noise.

[0196] In the example described using Figures 15 and 16, the target code amount controller 250 corrects the target code amount determined according to the feature ratio by correcting the feature ratio. However, the target code amount controller 250 may also directly correct the target code amount determined according to the feature ratio according to a predetermined offset amount, etc., after determining the target code amount according to the feature ratio.

[0197] Figure 17 is a conceptual diagram illustrating the first method for determining the scale factor. In this first method, the scale factor is determined by a search method.

[0198] Specifically, the frequency converter 210 performs frequency conversion on the data in the image processing block. The scale factor determinator 223 initializes the scale factor and quantizes the converted data according to the scale factor. Then, the scale factor determinator 223 obtains the predicted code amount by predicting the code amount according to the quantized data.

[0199] If the target code amount and the predicted code amount do not match, the scale factor determinator 223 updates the scale factor and quantizes the transformed data again according to the scale factor. The scale factor determinator 223 then repeats quantization, code amount prediction, and scale factor updating until the target code amount and the predicted code amount match, thereby determining the scale factor.

[0200] Figure 18 is a flowchart of the first determination method shown in Figure 17. Figure 18 particularly shows the operation of the scale factor determination unit 223. Specifically, first the scale factor determination unit 223 initializes the scale factor (S301). For example, the scale factor determination unit 223 may initialize the scale factor using the average scale factor as the initial value. This suppresses the increase in the number of searches in the search method.

[0201] Then, the scale factor determinator 223 quantizes the transformed data according to the scale factor (S302). Alternatively, a quantizer 221 may quantize the transformed data according to the scale factor instead of the scale factor determinator 223.

[0202] Then, the scale factor determinist 223 obtains the predicted code amount by predicting the code amount according to the quantized data (S303). The scale factor determinist 223 may also obtain the predicted code amount as the code amount predicted by applying Huffman coding to the quantized data.

[0203] Alternatively, the encoder 230 may encode the quantized data instead of the scale factor determinator 223. The scale factor determinator 223 may then predict the code value by obtaining it from the encoder 230.

[0204] Next, the scale factor determinist 223 calculates the target code amount minus the predicted code amount (S304). Then, the scale factor determinist 223 determines whether the target code amount minus the predicted code amount satisfies the polarity and convergence conditions (S305). These conditions correspond to the condition that the target code amount and the predicted code amount match.

[0205] If the target code amount - predicted code amount does not satisfy the condition (No in S305), the scale factor determinator 223 updates the scale factor according to the polarity and magnitude of the target code amount - predicted code amount (S306). The scale factor determinator 223 then repeats quantization (S302), code amount prediction (S303), calculation of target code amount - predicted code amount (S304), and updating the scale factor (S306) until the target code amount - predicted code amount satisfies the condition.

[0206] For example, the scale factor determinist 223 determines that the condition is met if the target code amount minus the predicted code amount is a positive value and is less than or equal to a threshold.

[0207] Alternatively, the scale factor determinator 223 may determine that the target code amount - predicted code amount satisfies the condition if the target code amount - predicted code amount is a positive value and the amount of change associated with the update of the scale factor is less than or equal to a threshold. Alternatively, the scale factor determinator 223 may determine that the target code amount - predicted code amount satisfies the condition if the target code amount - predicted code amount is a positive value and the scale factor has been updated more than or equal to a threshold number of times.

[0208] Alternatively, the scale factor determinator 223 may reduce the amount of change in the scale factor by half each time the scale factor is updated, and update the scale factor until the amount of change in the scale factor reaches the minimum unit.

[0209] In updating the scale factor, if the target code amount minus the predicted code amount is a positive value, the scale factor determinator 223 may decrease the scale factor. If the target code amount minus the predicted code amount is a negative value, the scale factor determinator 223 may increase the scale factor. Furthermore, the scale factor determinator 223 may increase the amount of change in the scale factor as the absolute value of the target code amount minus the predicted code amount increases.

[0210] If the target code amount minus the predicted code amount satisfies the condition (Yes in S305), the scale factor determinist 223 determines that scale factor as the final scale factor.

[0211] Figure 19 is a conceptual diagram illustrating the second method for determining the scale factor. In the second method, the scale factor is initialized according to the data features in the image processing block. Specifically, the scale factor determiner 223 acquires the data features in the image processing block and initializes the scale factor according to these features. Otherwise, it is the same as the first method.

[0212] Figure 20 is a flowchart of the second determination method shown in Figure 19. In particular, Figure 20 shows the operation of the scale factor determination unit 223.

[0213] Specifically, first, the scale factor determinist 223 acquires the data features (S401). For example, the scale factor determinist 223 may acquire the data features in the same way as the feature acquirer 240, or it may acquire the data features from the feature acquirer 240. Alternatively, the scale factor determinist 223 may acquire the data features using different criteria and methods than the feature acquirer 240.

[0214] Next, the scale factor determinator 223 initializes the scale factor according to the features of the data (S402). For example, the scale factor determinator 223 may initialize the scale factor by using a smaller value as the initial value of the scale factor as the number of features increases. This suppresses the increase in the number of searches in the search method. Therefore, processing delay is suppressed and throughput performance is improved.

[0215] The subsequent processes (S403 to S407) are the same as the corresponding processes (S302 to S306) in the first determination method.

[0216] Figure 21 is a conceptual diagram illustrating a third method for determining the scale factor. In this third method, the scale factor finally determined by updating the scale factor is compared with the initial value of the scale factor. If the difference between them is greater than a predetermined threshold, the initial value of the scale factor corresponding to that feature is updated. On the other hand, if the difference is less than or equal to the threshold, the initial value of the scale factor is not updated. This initial value of the scale factor can be used for subsequent blocks that have equivalent features.

[0217] The initial value of the scale factor may be updated to the final determined scale factor value, or it may be updated to an intermediate value (average value) between the initial value before the update and the final determined scale factor value. Otherwise, it is the same as the second determination method.

[0218] Figure 22 is a flowchart illustrating the third determination method shown in Figure 21. Figure 22 particularly shows the operation of the scale factor determination unit 223.

[0219] Specifically, the scale factor determiner 223 performs the same processes (S501 to S507) as the corresponding processes (S401 to S407) in the second determination method until the final determination of the scale factor. After the final determination of the scale factor (Yes in S506), the scale factor determiner 223 compares the final determined scale factor with the initial value of the scale factor and compares the difference between them with a predetermined threshold (S508).

[0220] If the difference is greater than a predetermined threshold (Yes in S508), the scale factor determinator 223 updates the initial value of the scale factor corresponding to that feature (S509). If the difference is less than or equal to a predetermined threshold (No in S508), the scale factor determinator 223 does not update the initial value.

[0221] Furthermore, for example, even if the difference is determined to be greater than a predetermined threshold, the scale factor determinator 223 may not immediately update the initial value, but instead increment a count value corresponding to the number of times the difference was determined to be greater than the predetermined threshold. The scale factor determinator 223 may then update the initial value only when that count value exceeds the threshold corresponding to the predetermined number of times. The count value may be initialized to 0 at the timing when image encoding begins, and when the threshold is exceeded, etc.

[0222] This is expected to further suppress the increase in the number of searches in the search method for subsequent processing blocks that have equivalent feature quantities. Therefore, compared to the second decision method, processing delay is further suppressed, and further improvements in throughput performance are expected.

[0223] In addition, in other methods for determining the scale factor, the initial value of the scale factor may be updated according to the final value of the scale factor, similar to the third method.

[0224] Figure 23 is a conceptual diagram illustrating a fourth method for determining the scale factor. In an example of the fourth method, the scale factor determiner 223 comprises a first scale factor determiner 310, a second scale factor determiner 320, and a scale factor selector 330. For example, these components are electrical circuits.

[0225] The first scale factor determinist 310 determines the first scale factor using the same method as the first scale factor determination method. That is, the first scale factor determinist 310 determines the scale factor determined by the first scale factor determination method as the first scale factor. The first scale factor determinist 310 also obtains the code amount predicted according to the first scale factor as the first predicted code amount.

[0226] The second scale factor determinist 320 determines the second scale factor according to the data features in the image processing block. Next, the scale factor determinist 223 quantizes the transformed data according to the second scale factor. Then, the scale factor determinist 223 obtains a second predicted code value by predicting the code value of the quantized data according to the second scale factor.

[0227] The scale factor selector 330 determines the scale factor by selecting a scale factor from the first scale factor and the second scale factor according to the first predicted code amount and the second predicted code amount.

[0228] Figure 24 is a flowchart of the fourth determination method shown in Figure 23. In particular, Figure 24 shows the operation of the first scale factor determination unit 310, the second scale factor determination unit 320, and the scale factor selector 330 in the scale factor determination unit 223.

[0229] Specifically, the first scale factor determinator 310 performs the same processing (S601 to S606) as in the first determination method (S301 to S306). However, the scale factor and predicted code amount in the first determination method are to be read as the first scale factor and the first predicted code amount.

[0230] Furthermore, the second scale factor determinist 320 acquires data features (S607). For example, the second scale factor determinist 320 may acquire data features in the same way as the feature acquirer 240, or it may acquire data features from the feature acquirer 240. Alternatively, the second scale factor determinist 320 may acquire data features using different criteria and methods than the feature acquirer 240.

[0231] Next, the second scale factor determinator 320 determines the second scale factor according to the features of the data (S608). For example, the second scale factor determinator 320 may determine a smaller value for the second scale factor as the features become larger.

[0232] Then, the second scale factor determinist 320 quantizes the transformed data according to the second scale factor (S609). Alternatively, a quantizer 221 may quantize the transformed data according to the second scale factor instead of the second scale factor determinist 320.

[0233] Then, the second scale factor determinist 320 obtains a second predicted code amount by predicting the code amount according to the quantized data (S610). The second scale factor determinist 320 may also obtain the code amount predicted by applying Huffman coding to the quantized data as the second predicted code amount.

[0234] Alternatively, the encoder 230 may encode the quantized data instead of the second scale factor determinist 320. The second scale factor determinist 320 may then predict the code value by obtaining it from the encoder 230.

[0235] Next, the scale factor selector 330 determines the scale factor by selecting a scale factor from the first scale factor and the second scale factor according to the first predicted code amount and the second predicted code amount (S611).

[0236] For example, the scale factor selector 330 selects the first scale factor if the first predicted code amount better matches the target code amount than the second predicted code amount, and selects the second scale factor if the second predicted code amount better matches the target code amount than the first predicted code amount. Here, it may be defined that the closer the predicted code amount is to the target code amount while being less than or equal to the target code amount, the better it matches the target code amount.

[0237] In the fourth determination method, the scale factor is selected from a first scale factor based on the search method and a second scale factor based on the features. This suppresses the determination of the scale factor corresponding to a local minimum as the final scale factor in the search method.

[0238] Figure 25 is a conceptual diagram showing a fifth method for determining the scale factor. In the example of the fifth method, similar to the example of the fourth method, the scale factor determiner 223 comprises a first scale factor determiner 310, a second scale factor determiner 320, and a scale factor selector 330. In the fifth method, the first scale factor is initialized according to the data features in the image processing block. Specifically, the first scale factor determiner 310 initializes the first scale factor according to the features obtained by the second scale factor determiner 320. Otherwise, it is the same as the fourth method.

[0239] Figure 26 is a flowchart of the fifth determination method shown in Figure 25. In particular, Figure 26 shows the operation of the first scale factor determination unit 310, the second scale factor determination unit 320, and the scale factor selector 330 in the scale factor determination unit 223.

[0240] Specifically, first, the second scale factor determinist 320 acquires the data features (S701). For example, the second scale factor determinist 320 may acquire the data features in the same way as the feature acquirer 240, or it may acquire the data features from the feature acquirer 240.

[0241] Next, the first scale factor determinist 310 initializes the first scale factor according to the features of the data (S702). For example, the first scale factor determinist 310 may initialize the first scale factor by using a smaller value as the initial value for the first scale factor as the features increase. This suppresses the increase in the number of searches in the search method. Therefore, processing delay is suppressed and throughput performance is improved.

[0242] The subsequent processes (S703 to S711) are the same as the corresponding processes (S602 to S606 and S608 to S611) in the fourth determination method.

[0243] Furthermore, in the fifth decision method, the second scale factor and the second predicted code size are the same as the first scale factor and the first predicted code size in the search method.

[0244] Therefore, in the fifth decision method, the second scale factor may be interpreted as the initial first scale factor in the search method, and the second predicted code amount may be interpreted as the initial first predicted code amount in the search method. Furthermore, the process of selecting a scale factor in the fifth decision method may be interpreted as the process of selecting a scale factor from the initial first scale factor and the final first scale factor according to the initial first predicted code amount and the final first predicted code amount.

[0245] Figure 27 is a conceptual diagram showing a first compression example in this embodiment. Similar to the example in Figure 4, Figure 27 shows an example of compression of luminance data and chrominance data in a 16x8 pixel MCU. The overall compression ratio is 25%, and the total code size is 512 bits. In other words, the overall compression ratio and code size are the same as in the example in Figure 4.

[0246] On the other hand, in the example shown in Figure 27, the compression ratio of the luminance data (Y) is 37.5%, and the code size of the luminance data (Y) is 384 bits. Also, the compression ratio of the chrominance data (Cb and Cr) is 12.5%, and the code size of the chrominance data (Cb and Cr) is 128 bits. In other words, the compression ratio and code size of the luminance data (Y) are different from those of the chrominance data (Cb and Cr).

[0247] In this embodiment, the compression ratio and code size for the processing target block change according to the feature ratio of multiple components and the result of the flatness determination. For example, for each component, the larger the proportion of the feature of that component in the feature ratio, the larger the target code size determined, and the resulting compression ratio and code size for that component are also larger. Also, for example, if the processing target block is determined to be flat, the target code size for one component is corrected to be relatively larger, and as a result, the compression ratio and code size also become relatively larger.

[0248] The target code amount controller 250 may change the ratio of multiple target code amounts corresponding to multiple components depending on the processing block, and maintain the sum of the multiple target code amounts constant regardless of the processing block. Furthermore, the target code amount controller 250 may make the target code amount of the luminance data larger than the target code amount of the chrominance data. Also, for example, if the processing block is determined to be flat, the target code amount controller 250 may increase the target code amount of the luminance data and decrease the target code amount of the chrominance data.

[0249] Furthermore, the target code amount controller 250 may determine how much larger the target code amount of the luminance data should be than the target code amount of the chrominance data, according to the features of the luminance data and the features of the chrominance data.

[0250] Furthermore, the encoder 230 may encode the scale factor and include the encoded scale factor in the stream. For example, the image decoder decodes the quantized data and scale factor from the stream and performs dequantization on the quantized data according to the scale factor. The image decoder then reconstructs the data by performing an inverse frequency transform on the data after dequantization.

[0251] As described above, the image encoding device 200 determines the target code size according to the feature ratios of multiple components and the results of the flatness determination. Therefore, the image encoding device 200 can suppress image quality degradation associated with image data compression. In other words, the image encoding device 200 can reduce the code size of image data while suppressing image quality degradation.

[0252] Therefore, the image encoding device 200 can reduce the memory capacity required to store image data. This allows for miniaturization, cost reduction, and power consumption reduction of the device handling the image data. Furthermore, the strain on memory bandwidth during access to image data is alleviated, enabling playback of high-resolution and high-frame-rate video.

[0253] Figure 28 is a block diagram showing the configuration of the image processing apparatus in this embodiment. The image processing apparatus 400 shown in Figure 28 comprises an image input 401, image compressors 402 and 406, image decompressors 403 and 407, an image output 404, a drawing processor 405, a memory controller 408, and a memory 409. For example, these components are electrical circuits.

[0254] The image input device 401 acquires the input image. For example, the image input device 401 acquires the input image from a camera or image sensor, etc.

[0255] Each of the image compressors 402 and 406 corresponds to the image encoding device 200 and encodes the image block by block. In doing so, each of the image compressors 402 and 406 compresses the data in the block to be processed.

[0256] The memory controller 408 controls access to the memory 409 from each component based on the bus protocol, and controls the reading and writing of data to the memory 409.

[0257] Memory 409 is a memory built into the image processing device 400. For example, compressed image data is stored in memory 409 according to the control of memory controller 408. Compressed image data is also read from memory 409 and decompressed (decompressed) by image decompressors 403 and 407.

[0258] Each of the image decompressors 403 and 407 decodes the image block by block. In doing so, each of the image decompressors 403 and 407 decompresses the data in the block being processed.

[0259] The drawing processor 405 renders the image. The drawing processor 405 may edit the image or generate a graphic image.

[0260] The image output device 404 outputs an image. For example, the image output device 404 outputs an image to a display device or the like. The image output device 404 may output multiple images superimposed on each other.

[0261] Figure 29 is a block diagram showing the configurations of the image compressors 402 and 406 shown in Figure 28. Each of the image compressors 402 and 406 comprises a local buffer 510, a preprocessor 520, encoding engines 531-534, a postprocessor 540, and a request buffer 550. The local buffer 510 also includes a local arbiter 511. For example, these components are electrical circuits.

[0262] The feature extractor 240, ratio calculator 242, variance extractor 244, flatness selector 246, and target code amount controller 250 of the image coding device 200 may be included in the preprocessor 520. Furthermore, the frequency converter 210 and quantization processor 220 of the image coding device 200 may be included in the coding engines 531-534. Also, the encoder 230 of the image coding device 200 may be included in the coding engines 531-534 and the postprocessor 540.

[0263] The encoding engines 531 to 534 correspond to four sequences corresponding to the four components that make up the pixels of an image. Here, two of the four sequences, corresponding to luminance and chrominance, are used. Specifically, encoding engine 533 is used for luminance, and encoding engine 534 is used for chrominance.

[0264] The local buffer 510 receives the image to be compressed. The local arbiter 511 acquires control information and, according to that information, negotiates access to the local buffer 510. Specifically, it controls the simultaneous parallel processing of luminance data (Y) and chrominance data (C) in the image processing blocks.

[0265] The preprocessor 520 acquires luminance data and chrominance data from the local buffer 510. Then, the preprocessor 520 acquires feature quantities for the luminance data and feature quantities for the chrominance data according to the luminance data and chrominance data. Then, the preprocessor 520 calculates the feature ratio according to the feature quantities for the luminance data and feature quantities for the chrominance data. Here, the feature ratio is the ratio of the feature quantities of the luminance data to the feature quantities of the chrominance data.

[0266] Furthermore, the preprocessor 520 obtains the variance value of the luminance data according to the luminance data. Then, the preprocessor 520 performs a flatness determination according to the variance value.

[0267] Furthermore, the preprocessor 520 determines the target code amounts for the luminance data and the chrominance data according to the feature ratio and the flatness determination results. The preprocessor 520 may also derive the target code amounts for the luminance data and the chrominance data from the feature ratio and flatness determination results by referring to the reference table described above.

[0268] The encoding engine 533 obtains luminance data and the target code value of the luminance data from the preprocessor 520. The encoding engine 533 then performs frequency conversion on the luminance data, quantizes the converted luminance data according to the target code value of the luminance data, and encodes the quantized luminance data.

[0269] The encoding engine 534 obtains color difference data and the target code amount of the color difference data from the preprocessor 520. Then, the encoding engine 534 performs frequency conversion on the color difference data, quantizes the converted color difference data according to the target code amount of the color difference data, and encodes the quantized color difference data.

[0270] The post-processor 540 obtains encoded luminance data from the encoding engine 533 and encoded color difference data from the encoding engine 534. The post-processor 540 also obtains the feature ratio from the pre-processor 520 via the encoding engine 533 or the encoding engine 534. Alternatively, the post-processor 540 may obtain the feature ratio directly from the pre-processor 520 without going through the encoding engine 533 or the encoding engine 534. Here, the feature ratio may be a feature ratio corrected according to the result of the flatness determination.

[0271] The post-processor 540 then inserts the feature ratio identification code at the beginning of the encoded chrominance data. The post-processor 540 then concatenates the encoded chrominance data and encoded luminance data in that order. In this way, the post-processor 540 packs the feature ratio identification code, encoded chrominance data, and encoded luminance data and generates a stream containing them. In other words, the post-processor 540 multiplexes the feature ratio identification code, encoded chrominance data, and encoded luminance data into a stream.

[0272] The post-processor 540 then stores the stream in the request buffer 550. The post-processor 540 also performs request control for the stream.

[0273] The request buffer 550 stores a stream containing the identification code for the feature ratio, encoded color difference data, and encoded luminance data. The stream stored in the request buffer 550 is output to memory 409 or the like.

[0274] The operations performed by the preprocessor 520 may also be performed by the feature acquirer 240 or target code amount controller 250 of the image coding device 200. The operations performed by the coding engines 531 to 534 may also be performed by the frequency converter 210 or quantization processor 220 of the image coding device 200. The operations performed by the coding engines 531 to 534 and the postprocessor 540 may also be performed by the encoder 230 of the image coding device 200.

[0275] Figure 30 is a flowchart showing the operation of the image compressors 402 and 406, respectively, as shown in Figures 28 and 29.

[0276] First, the preprocessor 520 sets up a reference table in which the feature ratios of multiple components are associated with the target code amount for each component (S801). The preprocessor 520 may set up a reference table for each image. For example, the preprocessor 520 may set up a reference table for each frame that makes up a moving image.

[0277] Next, the preprocessor 520 acquires the features of the luminance data (S802). In parallel, the preprocessor 520 acquires the features of the chrominance data (S803). Then, the preprocessor 520 calculates the feature ratio, which is the ratio of the features of the luminance data to the features of the chrominance data, according to the features of the luminance data and the features of the chrominance data (S804).

[0278] In parallel with the acquisition of features and the calculation of feature ratios (S802-S804), the preprocessor 520 acquires the variance value of the luminance data (S805). Then, the preprocessor 520 performs a flatness determination to determine whether the block to be processed is flat or not, according to the variance value of the luminance data (S806).

[0279] Then, the preprocessor 520 determines the target code amount for the luminance data and the target code amount for the chrominance data according to the feature ratio and the result of the flatness determination (S807).

[0280] Next, the encoding engine 533 encodes the luminance data according to the target code amount of the luminance data (S808). Specifically, the encoding engine 533 performs frequency conversion on the luminance data, quantizes the converted luminance data according to the target code amount of the luminance data, and encodes the quantized luminance data.

[0281] In parallel, the encoding engine 534 encodes the color difference data according to the target code amount of the color difference data (S809). Specifically, the encoding engine 534 performs frequency conversion on the color difference data, quantizes the converted color difference data according to the target code amount of the color difference data, and encodes the quantized color difference data. Then, the post-processor 540 inserts the feature ratio identification code into the encoded color difference data (S810). Here, the feature ratio may be a feature ratio corrected according to the result of the flatness determination.

[0282] Subsequently, the post-processor 540 concatenates the encoded color difference data and the encoded luminance data in that order (S811).

[0283] Figure 31 is a conceptual diagram showing a second compression example in this embodiment. Similar to the example in Figure 27, Figure 31 shows an example of compression of luminance data and chrominance data in a 16x8 pixel MCU. The overall compression ratio is 25%, and the total code size is 512 bits. In other words, the overall compression ratio and code size are the same as in the example in Figure 27.

[0284] On the other hand, in the example shown in Figure 31, a 3-bit identification code is inserted into the color difference data. This further compresses the color difference data by 3 bits. For example, 3 bits of data in the high-frequency range of the color difference data may be deleted. The luminance data then follows the color difference data into which the identification code has been inserted.

[0285] The feature ratio identification code indicates the target code amount for both the luminance data and the chrominance data. When the generated code amount for both the luminance data and the chrominance data matches the target code amount, the identification code can indicate the position in the stream corresponding to the boundary between the luminance data and the chrominance data. Therefore, in this case, it becomes easier to separate the luminance data and the chrominance data from the stream during the decoding (decompression) process.

[0286] Furthermore, if the amount of generated code is less than the target amount during the encoding (compression) process, padding or other methods may be used to adjust the amount of generated code to match the target amount.

[0287] Furthermore, instead of using the identification code for the feature ratio corrected according to the flatness determination result, the identification code for the feature ratio before correction and the identification code for the flatness determination result may be used, or an identification code for a combination of the feature ratio before correction and the flatness determination result may be used. In any case, these identification codes indicate the target code amounts for the luminance data and chrominance data, respectively.

[0288] Figure 32 is a block diagram showing the configurations of the image decompressors 403 and 407 shown in Figure 28. Each of the image decompressors 403 and 407 comprises a request buffer 610, a preprocessor 620, decoding engines 631 and 632, a postprocessor 640, and a local buffer 650. The local buffer 650 also includes a local arbiter 651. For example, these components are electrical circuits.

[0289] The decoding engines 631 and 632 are composed of a decoder, an inverse quantizer, and an inverse frequency converter, etc., in order to decode the stream encoded by the image encoding device 200, which corresponds to the image compressors 402, 406, etc., and convert it into each component of the image. The decoder may be, for example, a component that decodes the Huffman-coded stream data according to the target code amount when encoded by the image encoding device 200.

[0290] The inverse quantizer may be a component that inversely quantizes the output of the decoder according to the scale factor (header information in the Huffman code) when encoded by the image encoding device 200. The inverse frequency converter may be, for example, an IDCT (Inverse Discrete Cosine Transform). This allows the inverse frequency converter to perform an inverse orthogonal transform on the inversely quantized frequency coefficients to reconstruct pixel data and the like corresponding to each component of the image before encoding.

[0291] Decoding engines 631 and 632 correspond to two sequences of luminance data and chrominance data in order to decode a stream packed with feature ratio identification codes, encoded chrominance data, and encoded luminance data by, for example, the post-processor 540 shown in Figure 29. Specifically, decoding engine 631 is used for luminance data, and decoding engine 632 is used for chrominance data. This parallel processing makes it possible to reduce the delay time until image output.

[0292] Furthermore, the feature ratio identified by the identification code may be a feature ratio corrected according to the result of the flatness determination.

[0293] Alternatively, the chromatic difference data and luminance data may be processed in series using the decoding engine 631 so that the luminance data is processed after the chromatic difference data has been processed. Alternatively, the processing order of the luminance data and chromatic difference data may be reversed. Furthermore, the preprocessor 620 may efficiently distribute each stream output from the request buffer 610 to the decoding engines 631 and 632 according to their priority, and decoding may be performed.

[0294] Furthermore, the decoding engine may consist of one or more decoding engines. The number of decoding engines may be determined within a range where the delay time until image output is acceptable.

[0295] The request buffer 610 stores, for example, a stream packed with feature ratio identification codes, encoded color difference data, and encoded luminance data, which is transferred from memory 409.

[0296] In other words, the request buffer 610 stores a stream in which data of multiple components that constitute an image, such as brightness, chrominance, red, green, blue, and transparency, has been encoded. At that time, the preprocessor 620 performs request control based on the control information input to the local buffer 650, prioritizing the streams of each component necessary to generate the output image and storing them in the request buffer 610.

[0297] The stream stored in the request buffer 610 is input to the preprocessor 620. For example, the preprocessor 620 receives a stream packed with a feature ratio identification code, encoded color difference data, and encoded luminance data. In this case, the preprocessor 620 determines the target code amount for the luminance data and the target code amount for the color difference data according to the feature ratio identification code by code separation as shown in Figure 32.

[0298] For example, the target code amount for luminance data and the encoded luminance data are input to the decoding engine 631. Similarly, the target code amount for chrominance data and the encoded chrominance data are input to the decoding engine 632. The preprocessor 620 (code separation) may, for example, derive the target code amount for luminance data and the target code amount for chrominance data from the identification code of the feature ratio by referring to a reference table such as the one shown in Figure 11A. The decoding engines 631 and 632 then decode (decompress) the data of each component corresponding to the image before encoding and input it to the postprocessor 640.

[0299] The post-processor 640 converts the data of each component of the expanded image into arbitrary transfer units and stores them in the local buffer 650.

[0300] As mentioned above, the local buffer 650 receives data for each component of the expanded image. The local arbiter 651 acquires control information and, according to that information, mediates access to the local buffer 650, generating and outputting the image to be output for each block.

[0301] The series of processes performed by the image decompressors 403 and 407 described above makes it possible to efficiently decompress the compressed stream and convert it into an image.

[0302] Figure 33 shows the image quality evaluation results. Figure 33 shows the image quality evaluation results for each of the following cases: (1) without target code amount correction processing, (2) with target code amount correction processing including flatness judgment in 8x8 pixel unit mode, and (3) with target code amount correction processing including flatness judgment in 4x4 pixel unit mode. Specifically, the PSNR for Y, Cb, and Cr in each case is shown. PSNR is the Peak Signal-to-Noise Ratio, and a higher PSNR indicates less noise.

[0303] For image quality evaluation, encoding was performed with a compression ratio of 25%. Furthermore, in the target code amount correction process, a correction process was performed to increase the target code amount of luminance for flat processing blocks. The image quality evaluation results show that the target code amount correction process increased the PSNR of luminance and decreased the PSNR of chrominance. This suppresses the generation of noticeable noise. The decrease in PSNR of chrominance is at an acceptable level based on human visual characteristics.

[0304] Therefore, by correcting the target code amount, noise such as mosquito noise and block noise is suppressed, and image quality degradation is minimized.

[0305] Here, primarily, two components, luminance and chrominance, are used as the multiple components that constitute a pixel. However, three components corresponding to RGB—red, green, and blue—may also be used. Furthermore, in addition to the three components of red, green, and blue, a transparency (alpha) component for blending multiple images may also be used. The four components of red, green, blue, and transparency correspond to RGBA. Even in such cases, the loss of features can be suppressed by determining the target code amount according to the feature quantity.

[0306] Furthermore, while features typically correspond to complexity, they may also correspond to feature size. Similarly, the feature ratio, which is the ratio of features, typically corresponds to the complexity ratio, but it may also correspond to the ratio of feature size.

[0307] Furthermore, in the above explanation, the target code amount controller 250 increases the target code amount of the component used for obtaining the variance value and determining flatness when it is determined that the block to be processed is flat. However, the target code amount controller 250 may also increase the target code amount of a component different from the component used for obtaining the variance value and determining flatness when it is determined that the block to be processed is flat.

[0308] For example, the variance value acquirer 244 may acquire the variance value of the color difference data, and the flatness determiner 246 may perform a flatness determination according to the variance value of the color difference data. Furthermore, if the target code amount controller 250 determines that the block to be processed is flat, it may increase the target code amount of the luminance data.

[0309] Furthermore, in the above explanation, the variance value acquirer 244 acquires the variance value of the data of one of the multiple components, but it is also possible to acquire the variance value for each of the multiple components. The flatness determiner 246 may perform a flatness determination for each of the multiple components. The target code amount controller 250 may increase the target code amount of the component that has been determined to be flat among the multiple components.

[0310] For example, if three components corresponding to RGB—red, green, and blue—are used, the variance value acquirer 244 may acquire variance values ​​for each component, and the flatness determiner 246 may perform a flatness determination for each component. The target code amount controller 250 may then increase the target code amount of the component among the three components that has been determined to be flat.

[0311] Furthermore, the above explanation includes an example where an identification code indicating the target code amount for each component is included in a stream containing multiple data for multiple components. However, the identification code does not have to be included in the stream. For example, the identification code may be output separately from the stream. The identification code may also be stored separately from the stream, or provided separately to the image decoding device. This can separate the data from the control parameters, potentially enabling more efficient operation.

[0312] Alternatively, by applying variable-length coding to the data of each component, the coded data can be represented as variable-length data, and the output of an identification code indicating the target code length may be omitted.

[0313] As described above, the image coding device 200 includes a feature extractor 240, a ratio calculator 242, a variance extractor 244, a flatness determiner 246, a target code amount controller 250, a frequency converter 210, a quantization processor 220, and an encoder 230.

[0314] The feature extractor 240 acquires the feature quantities of the data for each component that makes up the pixels of the image, within the processing block of the image. The ratio calculator 242 calculates the feature quantity ratio of multiple components according to the feature quantities. The variance value extractor 244 acquires the variance value for the data of one component in the processing block. The flatness determiner 246 performs a flatness determination to determine whether the processing block is flat or not.

[0315] The target code amount controller 250 determines the target code amount of each component of the data according to the feature ratio, and corrects the target code amount determined according to the feature ratio according to the result of the flatness determination.

[0316] The frequency converter 210 performs frequency conversion on the data of each component. The quantization processor 220 quantizes the data of each component after frequency conversion according to the target code amount of the data of that component. The encoder 230 encodes the data of each component after quantization.

[0317] As a result, the image encoding device 200 can adjust the target code amount for each component of the data according to the feature ratio of multiple components. Therefore, the image encoding device 200 can suppress significant loss of features. Furthermore, the image encoding device 200 can adjust the target code amount according to whether the block to be processed is flat or not, that is, according to how noticeable the noise is. Therefore, the image encoding device 200 can suppress the generation of noticeable noise. Thus, the image encoding device 200 can suppress image quality degradation.

[0318] For example, the multiple components that make up an image pixel may include two components: luminance and chrominance. This allows the image encoding device 200 to adjust the target code amount for each component of luminance and chrominance according to the feature quantities of the data for that component. Therefore, the image encoding device 200 can appropriately suppress image quality degradation for images where the feature quantities differ between luminance and chrominance.

[0319] Furthermore, for example, one of the components mentioned above may be a luminance component. This allows the image encoding device 200 to perform flatness determination according to the variance value of the luminance component data. Therefore, the image encoding device 200 can appropriately determine whether the block to be processed is flat or not according to a component that is easily visible and is assumed to have a relatively large feature quantity. Consequently, the image encoding device 200 can appropriately suppress the generation of noticeable noise.

[0320] Furthermore, for example, the multiple components that make up the pixels of an image may include three components: red, green, and blue. This allows the image encoding device 200 to adjust the target code amount for each of the red, green, and blue components according to the feature quantities of the data for that component. Therefore, the image encoding device 200 can appropriately suppress image quality degradation for images where the feature quantities differ between red, green, and blue.

[0321] Furthermore, for example, the multiple components that make up the pixels of an image may include a transparency component. This allows the image encoding device 200 to adjust the target coding amount of the transparency component (information) when blending multiple images using RGBA, according to the feature quantities of the data of that component. Therefore, the image encoding device 200 can appropriately suppress image quality degradation when blending multiple images. In addition, it can suppress the memory capacity required to hold the transparency component and the delay that occurs in the transmission of that data.

[0322] Furthermore, for example, the target code amount controller 250 may increase the target code amount for each component's data as the proportion of the data features of that component in the feature ratio increases.

[0323] Furthermore, if the block to be processed is determined to be flat, the target code amount controller 250 may make the corrected target code amount for one of the components described above larger than the target code amount before correction. In this case, the target code amount controller 250 may also make the corrected target code amount smaller than the target code amount before correction for one or more of the multiple components, excluding the one component described above.

[0324] As a result, the image encoding device 200 can increase the target code amount for each component as the feature quantity increases, and decrease the target code amount as the feature quantity decreases. Furthermore, if the image encoding device 200 determines that the block to be processed is flat, it can increase the target code amount of the component used for the flatness determination and decrease the target code amounts of the other components. Therefore, the image encoding device 200 can appropriately adjust the target code amount according to the feature quantity ratio and the result of the flatness determination.

[0325] Furthermore, for example, the variance value acquisition unit 244 may acquire variance values ​​for each of the multiple subblocks in the block to be processed. The flatness determination unit 246 may determine whether or not a subblock is flat according to the variance values ​​acquired for each subblock. The flatness determination unit 246 may then perform a flatness determination according to the number of subblocks that have been determined to be flat among the multiple subblocks.

[0326] As a result, the image encoding device 200 can determine whether or not a block to be processed is flat according to the number of subblocks determined to be flat. In other words, the image encoding device 200 can determine whether or not a block to be processed is flat according to the size of the flat area in the block to be processed. Therefore, the image encoding device 200 can appropriately adjust the target code amount according to the size of the flat area where noticeable noise may occur.

[0327] Furthermore, for example, the feature extractor 240 may acquire, for each component, the statistical value of the absolute difference between adjacent pixels in the data of that component as a feature of the data of that component. This allows the image encoding device 200 to acquire feature quantities that correspond to sharp changes between adjacent pixels, etc. Therefore, the image encoding device 200 can appropriately adjust the target code quantity according to the feature quantities that correspond to sharp changes between adjacent pixels, etc.

[0328] Furthermore, for example, the feature extractor 240 may acquire the feature quantities of the data for each component using the Hadamard transform. This allows the image encoding device 200 to acquire feature quantities corresponding to the amount of edges, etc., obtained by the Hadamard transform. Therefore, the image encoding device 200 can appropriately adjust the target code amount according to the feature quantities corresponding to the amount of edges, etc.

[0329] Furthermore, for example, the feature extractor 240 may obtain information indicating the feature quantities of the data for each component from an external device to the image encoding device 200. In this way, the feature extractor 240 may obtain the feature quantities of the data for each component. This allows the image encoding device 200 to obtain feature quantities without calculating them. Therefore, the image encoding device 200 can reduce its computational processing.

[0330] Furthermore, for example, the encoder 230 may multiplex an identification code indicating multiple target code amounts determined for multiple components, and multiple data encoded for multiple components, into a stream. The encoder 230 may then output a stream. This allows the image encoding device 200 to indicate the target code amount of the data for each component in the stream. Therefore, the image encoding device 200 can assist in decoding the data for each component from the stream.

[0331] Furthermore, for example, the encoder 230 may multiplex multiple data encoded for multiple components into a stream and output the stream, and output identification codes indicating multiple target code amounts determined for multiple components separately from the stream. This allows the image encoding device 200 to indicate the target code amount of the data for each component separately from the stream. Therefore, the image encoding device 200 can provide information for decoding the data for each component separately from the stream.

[0332] Alternatively, for example, the quantization processor 220 may determine a scale factor that affects the quantization width for each component according to the target code amount. Then, the quantization processor 220 may quantize the data of each component according to the scale factor.

[0333] This allows the image encoding device 200 to adjust the scale factor used for quantizing the data of each component according to the target code amount of the data for that component. Therefore, the image encoding device 200 can appropriately adjust the code amount of the data of each component according to the target code amount of the data for that component.

[0334] Furthermore, for example, the quantization processor 220 may first initialize the scale factor in determining the scale factor. Then, the quantization processor 220 may quantize the data according to the scale factor. Then, the quantization processor 220 may obtain the predicted code amount of the data according to the data quantized according to the scale factor. Then, the quantization processor 220 may update the scale factor according to the comparison result between the predicted code amount and the target code amount.

[0335] The quantization processor 220 may determine the scale factor by repeatedly quantizing the data, obtaining the predicted code amount, and updating the scale factor until the predicted code amount matches the target code amount.

[0336] This allows the image coding device 200 to search for and determine a scale factor such that the predicted code amount matches the target code amount. Therefore, the image coding device 200 can determine an appropriate scale factor for the target code amount.

[0337] Furthermore, for example, the quantization processor 220 may initialize the scale factor according to the features of the data. This allows the image coding device 200 to appropriately initialize the scale factor during the search for the scale factor, thereby suppressing processing delays.

[0338] Furthermore, for example, the quantization processor 220 may update the initial value of the scale factor if the difference between the scale factor determined by repeatedly updating the scale factor and the initial value of the scale factor is greater than a threshold. The quantization processor 220 does not need to update the initial value of the scale factor if the difference is less than or equal to the threshold. As a result, the image encoding device 200 can update the initial value of the scale factor, which is determined according to the feature quantities based on the final scale factor, to an optimal value, thereby suppressing processing delays in subsequent processing.

[0339] Furthermore, for example, the quantization processor 220 may increment the count value if the difference between the scale factor determined by repeatedly updating the scale factor and the initial value of the scale factor is greater than the first threshold. The quantization processor 220 does not need to increment the count value if the difference is less than or equal to the first threshold.

[0340] Furthermore, the quantization processor 220 may update the initial value of the scale factor if the count value is greater than the second threshold. However, the quantization processor 220 does not need to update the initial value of the scale factor if the count value is less than or equal to the second threshold.

[0341] As a result, the image encoding device 200 can update the initial value of the scale factor, which is determined according to the feature quantity, at an appropriate update frequency based on the final scale factor, thereby suppressing processing delays in subsequent processing.

[0342] Furthermore, for example, the quantization processor 220 may first initialize a first scale factor in determining the scale factor. Then, the quantization processor 220 may quantize the data according to the first scale factor. Then, the quantization processor 220 may obtain a first predicted code value of the data according to the data quantized according to the first scale factor. Then, the quantization processor 220 may update the first scale factor according to the comparison result between the first predicted code value and the target code value.

[0343] The quantization processor 220 may determine the first scale factor by repeatedly quantizing the data, obtaining the first predicted code amount, and updating the first scale factor until the first predicted code amount matches the target code amount.

[0344] Furthermore, the quantization processor 220 may determine a second scale factor according to the features of the data. The quantization processor 220 may then quantize the data according to the second scale factor. The quantization processor 220 may then obtain a second predicted code value of the data according to the data quantized according to the second scale factor.

[0345] Furthermore, the quantization processor 220 may determine one of the first scale factor and the second scale factor as the scale factor based on the comparison result between the first predicted code amount and the target code amount, and the comparison result between the second predicted code amount and the target code amount.

[0346] As a result, the image coding device 200 can determine the scale factor using both a method of searching for a scale factor that fits the predicted code amount to the target code amount, and a method of determining the scale factor based on features. Therefore, the image coding device 200 can suppress the determination of a scale factor corresponding to a local optimum in the search as the final scale factor.

[0347] Furthermore, for example, the quantization processor 220 may initialize the first scale factor according to the features of the data. This allows the image coding device 200 to appropriately initialize the scale factor during the search for the scale factor, thereby suppressing processing delays.

[0348] Although the embodiments of the image encoding device have been described above according to the embodiments, the embodiments of the image encoding device are not limited to these embodiments. Modifications that a person skilled in the art can conceive of may be made to the embodiments, and the multiple components in the embodiments may be combined arbitrarily.

[0349] For example, in the embodiment, a process performed by a specific component may be performed by another component instead of that specific component. Also, the order of multiple processes may be changed, or multiple processes may be executed in parallel. Furthermore, multiple modifications may be combined and applied. In addition, the first and second ordinal numbers used in the description may be replaced, removed, or newly assigned as appropriate. These ordinal numbers do not necessarily correspond to a meaningful order and may be used to identify elements.

[0350] Furthermore, the image encoding method, including the steps performed by each component of the image encoding device, may be executed by any device or system. For example, part or all of the image encoding method may be executed by a computer equipped with a processor, memory, and input / output circuits, etc. In this case, the image encoding method may be executed by the computer executing a program that causes the computer to execute the image encoding method.

[0351] Furthermore, the above program may be recorded on a non-temporary computer-readable recording medium such as a CD-ROM.

[0352] Furthermore, each component of the image encoding device may be composed of dedicated hardware, general-purpose hardware that executes the above-mentioned program, or a combination of these. The general-purpose hardware may consist of memory on which the program is stored, and a general-purpose processor that reads the program from memory and executes it. Here, the memory may be semiconductor memory or a hard disk, and the general-purpose processor may be a CPU.

[0353] Furthermore, dedicated hardware may consist of memory and a dedicated processor, etc. For example, a dedicated processor may refer to memory for recording data and execute the image encoding method described above.

[0354] Furthermore, each component of the image encoding device may be an electrical circuit. These electrical circuits may form a single electrical circuit as a whole, or they may be separate electrical circuits. Also, these electrical circuits may correspond to dedicated hardware, or they may correspond to general-purpose hardware that executes the above-mentioned programs, etc.

[0355] This disclosure is useful, for example, for encoding devices that encode images, and is applicable to digital cameras, digital video cameras, digital video recorders, and image processing systems, etc.

[0356] 100, 200 Image Encoder 110, 210 Frequency Converter 120, 220 Quantization Processor 121, 221 Quantizer 122, 222 Quantization Table Derivative 123, 223 Scale Factor Determinator 130, 230 Encoder 140, 240 Feature Extractor 142, 242 Ratio Calculator 150, 250 Target Code Value Controller 244 Variance Value Extractor 246 Flatness Determinator 310 First Scale Factor Determinator 320 Second Scale Factor Determinator 330 Scale Factor Selector 400 Image Processing Unit 401 Image Input Unit 402, 406 Image Compressor 403, 407 Image Decompressor 404 Image Output Unit 405 Drawing Processor 408 Memory Controller 409 Memory 510, 650 Local buffer 511, 651 Local arbiter 520, 620 Preprocessor 531, 532, 533, 534 Encoding engine 540, 640 Postprocessor 550, 610 Request buffer 631, 632 Decoding engine

Claims

1. An image encoding device comprising: a feature extractor that acquires feature quantities of the data of each of the multiple components constituting the pixels of an image in a processing target block among the multiple blocks of the image; a ratio calculator that calculates the feature quantity ratio of the multiple components according to the feature quantities; a variance value extractor that acquires the variance value of the data of one of the multiple components in the processing target block for one of the multiple components; a flatness determiner that performs a flatness determination of whether or not the processing target block is flat according to the variance value; a target code amount controller that determines the target code amount of the data of each of the multiple components according to the feature quantity ratio and corrects the target code amount determined according to the feature quantity ratio according to the result of the flatness determination; a frequency converter that performs a frequency conversion on the data of each of the multiple components; a quantization processor that quantizes the data of each of the multiple components after frequency conversion according to the target code amount of the data of each of the multiple components; and an encoder that encodes the data of each of the multiple components after quantization.

2. The image encoding apparatus according to claim 1, wherein the plurality of components include two components: luminance and color difference.

3. The image coding apparatus according to claim 2, wherein the one component is a luminance component.

4. The image encoding apparatus according to claim 1, wherein the plurality of components include three components: red, green, and blue.

5. The image coding apparatus according to claim 1, wherein the plurality of components include a transparency component.

6. The image encoding apparatus according to any one of claims 1 to 5, wherein the target code amount controller, in determining the target code amount, increases the target code amount of the data of each of the plurality of components as the proportion of the feature amount of the data of that component in the feature amount ratio increases, and when it is determined that the block to be processed is flat, in correcting the target code amount, increases the corrected target code amount for one component compared to the target code amount before correction, and decreases the corrected target code amount for each of the plurality of components, excluding one component, compared to the target code amount before correction.

7. The image encoding apparatus according to any one of claims 1 to 5, wherein the variance value acquirer acquires the variance value for each of the plurality of subblocks in the processing target block, the flatness determiner determines whether or not the subblock is flat according to the variance value acquired for each of the plurality of subblocks, and the flatness determination is performed according to the number of subblocks among the plurality of subblocks that are determined to be flat.

8. The image encoding apparatus according to any one of claims 1 to 5, wherein the feature acquisition device acquires, for each of the plurality of components, the statistical value of the absolute difference between adjacent pixels in the data of the component as the feature of the data of the component.

9. The image encoding apparatus according to any one of claims 1 to 5, wherein the feature acquisition device acquires the feature quantities of the data of each of the plurality of components using the Hadamard transform.

10. The image encoding device according to any one of claims 1 to 5, wherein the feature acquisition device acquires information indicating the feature of the data of each of the plurality of components from an external device of the image encoding device, thereby acquiring the feature of the data of each of the plurality of components.

11. The image encoding apparatus according to any one of claims 1 to 5, wherein the encoder multiplexes identification codes indicating a plurality of target code amounts determined for the plurality of components and a plurality of data encoded for the plurality of components into a stream and outputs the stream.

12. The image encoding apparatus according to any one of claims 1 to 5, wherein the encoder multiplexes a plurality of data encoded for the plurality of components into a stream and outputs the stream, and outputs an identification code indicating a plurality of target code amounts determined for the plurality of components separately from the stream.

13. The image encoding apparatus according to any one of claims 1 to 5, wherein the quantization processor determines a scale factor that affects the quantization width for each of the plurality of components according to the target code amount, and quantizes the data of the component according to the scale factor.

14. An image encoding method comprising: a step of obtaining feature quantities of the data of each of the multiple components constituting the pixels of an image in a processing target block among the multiple blocks of the image; a step of calculating the feature quantity ratio of the multiple components according to the feature quantities; a step of obtaining the variance value of the data of one of the multiple components in the processing target block for one of the multiple components; a step of performing a flatness determination to determine whether the processing target block is flat or not according to the variance value; a step of determining the target code amount of the data of each of the multiple components according to the feature quantity ratio, and correcting the target code amount determined according to the feature quantity ratio according to the result of the flatness determination; a step of performing a frequency transformation on the data of each of the multiple components; a step of quantizing the data of the component after frequency transformation according to the target code amount of the data of each of the multiple components; and a step of encoding the data of the component after quantization for each of the multiple components.

Images