Image encoding device and image encoding method
By acquiring the feature amount of the image block in the image encoding device and determining the target code amount, performing frequency conversion and quantization processing, the problem of image quality deterioration and code amount increase in the image encoding process is solved, and more efficient image data compression and deal with.
Patent Information
- Application Number
- CN202380035608.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-04-27
- Filing Date
- 2023-04-24
- Publication Date
- 2024-11-29
AI Technical Summary
The prior art tends to lead to deterioration of image quality during image encoding, and the increase in code volume leads to transmission delay and memory usage.
An image encoding device is designed to obtain the feature quantity of the image block through the feature quantity acquirer, the target code quantity determines the target code quantity of each component, the frequency converter performs frequency conversion, and the quantization processor quantizes according to the target code quantity, The encoder then encodes the code quantity of each component to suppress deterioration of the picture quality.
The deterioration of image quality is effectively suppressed, the code volume of image data is reduced, the memory capacity requirement and transmission delay are reduced, and the efficiency of the image processing device is improved.
Smart Images

Figure CN119054278A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to an image encoding device and an image encoding method. Background Art
[0002] Patent document 1 discloses that a quantization step size and a coding method are determined for each group so that the code length of a block including a plurality of groups does not exceed a specified value, and that coding processing is performed based on the determined quantization step size and coding method to generate coded data. Patent document 2 discloses that image data is compressed using one of a first compression algorithm with a fixed scale factor and a second compression algorithm with an adjustable scale factor, and that a determination is made as to whether the code amount is within a specified range.
[0003] Prior Art Literature
[0004] Patent Literature
[0005] Patent Document 1: Japanese Patent No. 6502739
[0006] Patent Document 2: Japanese Patent No. 4631629 Summary of the invention
[0007] Problems to be solved by the invention
[0008] According to the conventional techniques described in Patent Documents 1 and 2, although the code amount is controlled, image quality may deteriorate.
[0009] The present disclosure provides an image encoding device and an image encoding method capable of suppressing image quality degradation.
[0010] Means for solving problems
[0011] An image encoding device involved in one embodiment of the present disclosure comprises: a feature quantity acquirer, which acquires the feature quantity of data of the component in a processing target block among multiple blocks of the image for each component among multiple components of pixels constituting the image; a target code quantity determiner, which determines the target code quantity of the data of the component according to the feature quantity of the data of the component for each component among the multiple components; a frequency converter, which performs frequency conversion on the data of the component for each component among the multiple components; a quantization processor, which quantizes the data of the component after the frequency conversion according to the target code quantity of the data of the component for each component among the multiple components; and an encoder, which encodes the quantized data of the component for each component among the multiple components.
[0012] In addition, an image encoding method involved in one embodiment of the present invention includes: a step of obtaining, for each of a plurality of components of pixels constituting an image, a feature amount of data of the component in a processing object block among a plurality of blocks of the image; a step of determining, for each of the plurality of components, a target code amount of the data of the component according to the feature amount of the data of the component; a step of performing a frequency transformation on the data of the component for each of the plurality of components; a step of quantizing, for each of the plurality of components, the data of the component after the frequency transformation according to the target code amount of the data of the component; and a step of encoding, for each of the plurality of components, the data of the component after the quantization.
[0013] Effects of the Invention
[0014] An image encoding device and an image encoding method according to one embodiment of the present disclosure can suppress image quality degradation. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 This is a block diagram showing the structure of the image encoding device in the reference example.
[0016] Figure 2 It is a flowchart showing the operation of the image encoding device in the reference example.
[0017] Figure 3A This is a diagram showing a standard quantization table used for luminance data.
[0018] Figure 3B This is a diagram showing a standard quantization table used for chrominance data.
[0019] Figure 4 This is a conceptual diagram showing an example of compression of luminance data and color difference data in a reference example.
[0020] Figure 5 This is a conceptual diagram showing mosquito noise caused by quantization error.
[0021] Figure 6 This is a block diagram showing the structure of the image encoding device in the embodiment.
[0022] Figure 7 This is a flowchart showing the operation of the image encoding device according to the embodiment.
[0023] Fig. 8A This is a conceptual diagram showing an example of calculating the feature amount of brightness data.
[0024] Figure 8B This is a conceptual diagram showing an example of calculating the feature amount of color difference data.
[0025] Fig. 9 This is a conceptual diagram showing a first method for determining a scaling factor.
[0026] Fig.10 This is a flowchart showing a first method for determining a scaling factor.
[0027] Fig.11 This is a conceptual diagram showing the second method for determining the scale factor.
[0028] Fig.12 This is a flowchart showing the second method for determining the scale factor.
[0029] Fig.13 This is a conceptual diagram showing the third method for determining the scale factor.
[0030] Fig.14 This is a flowchart showing the third method for determining the scale factor.
[0031] Fig.15 This is a conceptual diagram showing the fourth method of determining the scale factor.
[0032] Fig.16 This is a flowchart showing the fourth method for determining the scaling factor.
[0033] Fig.17 This is a conceptual diagram showing the fifth method for determining the scale factor.
[0034] Fig.18 This is a flowchart showing the fifth method for determining the scaling factor.
[0035] Fig.19 This is a conceptual diagram showing a first compression example in the embodiment.
[0036] Fig. 20 It is a block diagram showing the structure of the image processing device in the embodiment.
[0037] Fig.21 It is a block diagram showing the structure of an image compressor in the embodiment.
[0038] Fig.22A This is a diagram showing a reference table of complexity ratio and target code amount.
[0039] Fig. 22B It is a graph showing the relationship between complexity ratio and compression ratio.
[0040] Fig.23 This is a flowchart showing the operation of the image compressor in the embodiment.
[0041] Fig.24 This is a conceptual diagram showing a second compression example in the embodiment.
[0042] Fig.25 This is a block diagram showing the structure of an image decompressor in an embodiment.
[0043] Fig.26 It is a diagram showing the image quality evaluation results. DETAILED DESCRIPTION
[0044] In recent years, the resolution of images in image processing devices has been increasing. This is accompanied by disadvantages such as increased image data code volume, transmission delay, and increased memory usage. By quantizing image data, the image data code volume is reduced, which can suppress such disadvantages. However, due to the quantization of image data, quantization errors sometimes occur, resulting in image quality degradation.
[0045] Thus, for example, an image encoding device according to one embodiment of the present invention comprises: a feature quantity acquirer for acquiring, for each of a plurality of components of pixels constituting an image, a feature quantity of data of the component in a processing target block among a plurality of blocks of the image; a target code quantity determiner for determining, for each of the plurality of components, a target code quantity of the data of the component in accordance with the feature quantity of the data of the component; a frequency converter for performing frequency conversion on the data of the component for each of the plurality of components; a quantization processor for quantizing, for each of the plurality of components, the data of the component after frequency conversion in accordance with the target code quantity of the data of the component; and an encoder for encoding, for each of the plurality of components, the data of the component after quantization.
[0046] Thus, the image coding apparatus can adjust the target code amount of the data of each component according to the feature amount of the data of the component. Therefore, the image coding apparatus can suppress the situation where the feature is greatly lost. Therefore, the image coding apparatus can suppress the degradation of image quality.
[0047] In addition, for example, the plurality of components include two components of brightness and color difference.
[0048] Thus, the image coding apparatus can adjust the target code amount of the data of each component of brightness and color difference according to the feature amount of the data of the component. Therefore, the image coding apparatus can appropriately suppress image quality degradation for images with different feature amounts between brightness and color difference.
[0049] In addition, for example, the plurality of components include three components of red, green, and blue.
[0050] Thus, the image coding apparatus can adjust the target code amount of the data of each component of red, green and blue according to the feature amount of the data of the component. Therefore, the image coding apparatus can appropriately suppress image quality degradation for images with different feature amounts between red, green and blue.
[0051] Additionally, for example, the plurality of components includes transparency.
[0052] This makes it possible for the image coding apparatus to suppress the memory capacity required to maintain transparency information when mixing a plurality of images using RGBA, and the delay generated in the transmission of the data.
[0053] In addition, for example, the feature quantity acquirer acquires, for each of the plurality of components, a statistical value of an absolute value of a difference between adjacent pixels in the data of the component as the feature quantity of the data of the component.
[0054] This allows the image coding apparatus to obtain a feature amount corresponding to a sharp change between adjacent pixels, etc. Therefore, the image coding apparatus can appropriately adjust the target code amount according to the feature amount corresponding to a sharp change between adjacent pixels, etc.
[0055] In addition, for example, the feature amount acquirer acquires the feature amount of the data of each of the plurality of components using Hadamard transform.
[0056] Thus, the image coding apparatus can obtain a feature amount corresponding to the edge amount obtained by Hadamard transform, etc. Therefore, the image coding apparatus can appropriately adjust the target code amount according to the feature amount corresponding to the edge amount, etc.
[0057] In addition, for example, the feature quantity acquirer acquires information representing the feature quantity of the data of each of the plurality of components from a device external to the image encoding device, thereby acquiring the feature quantity of the data of the component.
[0058] This allows the image coding apparatus to obtain the feature amount without calculating the feature amount, and thus can reduce the number of calculations.
[0059] Furthermore, for example, the encoder multiplexes identification codes indicating a plurality of target code amounts determined for the plurality of components and a plurality of data encoded for the plurality of components into a stream, and outputs the stream.
[0060] This allows the image coding apparatus to indicate the target code size of each component data in the stream. Therefore, the image coding apparatus can support decoding of each component data from the stream.
[0061] In addition, for example, the quantization processor determines a scale factor affecting a quantization width according to the target code amount for each of the plurality of components, and quantizes the data of the component according to the scale factor.
[0062] Thus, the image coding apparatus can adjust the scale factor used in quantizing the data of each component according to the target code amount of the data of the component. Therefore, the image coding apparatus can appropriately adjust the code amount of the data of the component according to the target code amount of the data of the component.
[0063] In addition, for example, the quantization processor initially sets the scaling factor, quantizes the data according to the scaling factor, obtains a predicted code amount of the data based on the data quantized according to the scaling factor, updates the scaling factor according to a comparison result between the predicted code amount and the target code amount, and repeatedly performs quantization of the data, acquisition of the predicted code amount, and updating of the scaling factor until the predicted code amount meets the target code amount, thereby determining the scaling factor.
[0064] Thus, the image coding apparatus can search and determine the scaling factor that makes the predicted code amount match the target code amount. Therefore, the image coding apparatus can determine an appropriate scaling factor for the target code amount.
[0065] In addition, for example, the quantization processor initially sets the scale factor according to the feature amount of the data.
[0066] This allows the image coding apparatus to appropriately initialize the scale factor when searching for the scale factor, thereby suppressing processing delay.
[0067] In addition, for example, the quantization processor updates the initial value of the scale factor when the difference between the scale factor determined by repeatedly updating the scale factor and the initial value of the scale factor is larger than a threshold value, and does not update the initial value of the scale factor when the difference is less than the threshold value.
[0068] Thus, the image encoding device can update the initial value of the scale factor determined according to the feature amount to an optimal value based on the final scale factor, and thus can suppress processing delay in subsequent processing.
[0069] In addition, for example, when the difference between the scale factor determined by repeatedly updating the scale factor and the initial value of the scale factor is larger than a first threshold value, the quantization processor increments the count value (increases the count value by 1); when the difference is smaller than the first threshold value, the count value is not incremented; when the count value is larger than a second threshold value, the initial value of the scale factor is updated; when the count value is smaller than the second threshold value, the initial value of the scale factor is not updated.
[0070] Thus, the image coding apparatus can update the initial value of the scale factor determined according to the feature amount at an appropriate update frequency based on the final scale factor, and thus can suppress processing delay in subsequent processing.
[0071] In addition, for example, the quantization processor initially sets a first scale factor, quantizes the data according to the first scale factor, obtains a first predicted code amount of the data based on the data quantized according to the first scale factor, updates the first scale factor according to a comparison result between the first predicted code amount and the target code amount, repeatedly quantizes the data, obtains the first predicted code amount, and updates the first scale factor until the first predicted code amount meets the target code amount, thereby determining the first scale factor, determining a second scale factor according to the feature amount of the data, quantizing the data according to the second scale factor, obtaining a second predicted code amount of the data based on the data quantized according to the second scale factor, and determining one of the first scale factor and the second scale factor as the scale factor based on a comparison result between the first predicted code amount and the target code amount and a comparison result between the second predicted code amount and the target code amount.
[0072] Thus, the image coding apparatus can determine the scale factor using both the method of searching for the scale factor that makes the predicted code amount match the target code amount and the method of determining the scale factor based on the feature amount. Therefore, the image coding apparatus can suppress the situation where the scale factor corresponding to the local solution being explored is determined as the final scale factor.
[0073] In addition, for example, the quantization processor initially sets the first scale factor according to the feature amount of the data.
[0074] This allows the image coding apparatus to appropriately initialize the scale factor when searching for the scale factor, thereby suppressing processing delay.
[0075] In addition, for example, an image encoding method involved in one embodiment of the present invention includes: a step of obtaining, for each of a plurality of components of pixels constituting an image, a feature amount of data of the component in a processing object block among a plurality of blocks of the image; a step of determining, for each of the plurality of components, a target code amount of the data of the component according to the feature amount of the data of the component; a step of performing a frequency transformation on the data of the component for each of the plurality of components; a step of quantizing, for each of the plurality of components, the data of the component after the frequency transformation according to the target code amount of the data of the component; and a step of encoding, for each of the plurality of components, the data of the component after the quantization.
[0076] Thus, the target code amount of the data of each component can be adjusted according to the feature amount of the data of the component. Therefore, it is possible to suppress the situation where the feature is greatly lost. Therefore, it is possible to suppress the degradation of image quality.
[0077] The following embodiments are described using the accompanying drawings. In addition, the embodiments described below all represent general or specific examples. The numerical values, shapes, materials, components, configuration positions and connection methods of components, steps, and the order of steps shown in the following embodiments are examples, and their purpose is not to limit the claims.
[0078] Figure 1 This is a block diagram showing the structure of the image encoding device in the reference example. Figure 1 The image encoding device 100 shown encodes an image per block. Each image is composed of a plurality of pixels, and a block is also composed of a plurality of pixels. A block may also be a unit of a randomly accessible area called an MCU (Minimum Coded Unit). For example, a 16×8 pixel MCU may be used as a block. The image encoding device 100 performs fixed-length compression on each block of the image according to the target code amount.
[0079] In addition, pixels of an image are composed of multiple components such as brightness and color difference. When encoding an image for each block, the image encoding device 100 encodes the block of the image for each component. For example, the image encoding device 100 encodes the brightness data and the color difference data in the block independently.
[0080] Specifically, the image encoding apparatus 100 includes a frequency transformer 110, a quantization processor 120, and an encoder 130. The quantization processor 120 includes a quantizer 121, a quantization table deriver 122, and a scale factor determiner 123. These components are, for example, circuits.
[0081] The frequency converter 110 performs frequency conversion on the data of each component constituting a pixel in the processing target block. For example, DCT (Discrete Cosine Transform) is used in the frequency conversion. Thus, for each component, a plurality of pixel values of the component in the processing target block are converted into a plurality of frequency coefficients of the component in the processing target block.
[0082] The quantization processor 120 quantizes the transformed data of each component in the processing target block according to a fixed target code amount that is the same in a plurality of components. As a result, the data is compressed.
[0083] Specifically, the scale factor determiner 123 determines the scale factor according to the target code amount, the transformed data and the fixed length coding algorithm so that the code amount of the data meets the target code amount. The quantization table deriver 122 derives the quantization table according to the scale factor. The quantizer 121 quantizes the transformed data according to the quantization table.
[0084] In the quantization table, the quantization width is specified for each frequency level. The scale factor affects the quantization width specified for each frequency level in the quantization table. For example, if the scale factor becomes larger, the quantization width also becomes larger. That is, the quantization width may also have a monotonically increasing relationship with the scale factor in a narrow sense or a broad sense. In addition, the quantization width may also be proportional to the scale factor. In addition, a scale factor may be used in which the quantization width becomes smaller if the scale factor becomes larger.
[0085] The encoder 130 encodes the quantized data of the component in the processing target block into a stream for each component. Huffman code may also be used in the encoding. In addition, the encoder 130 may also feed back the code amount to the scale factor determiner 123. In addition, the scale factor determiner 123 may also update the scale factor according to the code amount, thereby determining the scale factor again. In addition, the determination of the scale factor, the derivation of the quantization table, the quantization and the encoding may be repeated.
[0086] Thus, the processing object block is encoded according to the target code amount. The target code amount may also be a register setting value set by the image encoding device 100. Specifically, a compression rate such as 50% or 25% may be used as the target code amount. The compression rate is the ratio of the capacity of compressed data to the capacity of uncompressed data. In the reference example, the code amount of each component data in each block of the image is controlled according to the same fixed target code amount.
[0087] For example, the code amount of the brightness data and the code amount of the color difference data are controlled according to the same fixed target code amount. Specifically, the scale factor corresponding to the brightness data is determined according to the fixed target code amount. In addition, the brightness data is quantized and encoded according to the scale factor corresponding to the brightness data. In addition, the scale factor corresponding to the color difference data is determined according to the same fixed target code amount. In addition, the color difference data is quantized and encoded according to the scale factor corresponding to the color difference data.
[0088] Figure 2 Yes means Figure 1 Flowchart of the operation of the image coding apparatus 100 shown in FIG. First, an image is divided into a plurality of blocks (S101). The division process (S101) can be performed by a divider (not shown) of the image coding apparatus 100 or by the frequency converter 110. Thereafter, a loop process (S102 to S106) is performed in units of blocks.
[0089] In the block-based loop processing (S102 to S106), the frequency converter 110 performs frequency conversion on the data of the component in the processing target block for each component (S102). Thereafter, the quantization processor 120 quantizes the converted data of the component in the processing target block according to the scale factor for each component (S103). In addition, the encoder 130 encodes the quantized data of the component in the processing target block for each component (S104).
[0090] Here, when the generated code amount of the encoded data of each component in the processing target block meets the target code amount (S105: Yes), the processing of the processing target block is completed. On the other hand, when the generated code amount does not meet the target code amount (S105: No), the quantization processor 120 updates the scale factor (S106).
[0091] For example, when the amount of generated code is larger than the target amount of code, the quantization processor 120 increases the scale factor. On the other hand, when the amount of generated code is smaller than the target amount of code, the quantization processor 120 decreases the scale factor. In addition, quantization (S103), encoding (S104), and updating of the scale factor (S106) are repeated until the amount of generated code meets the target amount of code.
[0092] Thus, the processing target block is encoded according to the target code amount. In addition, the image encoding device 100 performs quantization (S103), encoding (S104), and updating of the scale factor (S106) for each component. On the other hand, the image encoding device 100 controls the generated code amount by using the same fixed target code amount in multiple components.
[0093] The quantization table deriver 122 may also derive the quantization table by reflecting the scale factor in the standard quantization table. Alternatively, the standard quantization table may be defined for each component.
[0094] Figure 3A This is a diagram showing a standard quantization table used for brightness data. In the quantization table, quantization widths are specified for each frequency level. Figure 3A In the quantization table, the values in the table represent the quantization amplitude. In the quantization table, the upper left corresponds to low frequencies and the lower right corresponds to high frequencies. Based on the fact that people are insensitive to small changes, the quantization amplitude is basically determined to be small for low frequencies and large for high frequencies. This suppresses subjective image quality degradation and reduces the amount of code.
[0095] Figure 3B is a diagram showing a standard quantization table used in color difference data. Figure 3A Likewise, in Figure 3BIn the quantization table, the numerical value represents the quantization amplitude. People have different perceptions of brightness and color difference. According to the difference in people's perception of brightness and color difference, a quantization table different from the standard quantization table for brightness data is specified as the standard quantization table for color difference data. In addition, the standard quantization table used for brightness data can also be used for color difference data.
[0096] Figure 4 This is a conceptual diagram showing a compression example in a reference example. Specifically, it shows a compression example of brightness data and color difference data in a 16×8 pixel MCU. Here, the YUV422 format is used, and in a 16×8 pixel MCU, there are 16×8 Y values (brightness values), 8×8 Cb values (blue color difference values), and 8×8 Cr values (red color difference values). In addition, 1 Y value, 1 Cb value, and 1 Cr value are represented by 8 bits.
[0097] Therefore, the data amount of the brightness data (Y) in the MCU before compression is 16×8×8=1024 bits. The code amount obtained by encoding the data at a compression rate of 25% is 1024×25%=256 bits. Similarly, the data amount of the color difference data (Cb and Cr) in the MCU before compression is 16×8×8=1024 bits. The code amount obtained by encoding the data at a compression rate of 25% is 1024×25%=256 bits.
[0098] In the reference example, the same fixed target code amount (compression rate) is used for brightness and color difference. That is, the compression rate is controlled to be the same between brightness data (Y) and color difference data (Cb and Cr), and the code amount is also controlled to be the same.
[0099] In addition, the 16×8 Y values in the 16×8 pixel MCU can also be divided into two sets each consisting of 8×8 Y values. In addition, frequency conversion, quantization, and encoding can also be performed in units of 8×8 values. In addition, regarding the 8×8 Cb value and the 8×8 Cr value, frequency conversion, quantization, and encoding can also be performed in units of 8×8 values.
[0100] Figure 5 This is a conceptual diagram showing mosquito noise generated by quantization error. In the encoding process and decoding process of an image, frequency conversion, quantization, inverse quantization, and inverse frequency conversion are performed. Specifically, in the encoding process of an image, frequency conversion and quantization are performed, and in the decoding process of an image, inverse quantization and inverse frequency conversion are performed. Here, an example of performing frequency conversion, quantization, inverse quantization, and inverse frequency conversion on a block of 8×8 pixels of an image is shown.
[0101] First, in the encoding process, a frequency transform is performed on a block of an input image. Specifically, a plurality of pixel values constituting the block are decomposed into a plurality of frequency coefficients constituting the block according to a plurality of bases of the frequency transform.
[0102] Here, the 8×8 pixel values constituting the block are transformed into the 8×8 frequency coefficients constituting the block ( Figure 5 ). Figure 3A and Figure 3B Similarly, in the quantization table shown, in the transformed data of the block, the upper left corresponds to the low frequency and the lower right corresponds to the high frequency. For example, if there is an edge in the block of the input image, there are non-zero frequency coefficients not limited to the low frequency domain, but also in the medium frequency domain and the high frequency domain.
[0103] Next, the transformed data is quantized. Specifically, each frequency coefficient is quantized according to the corresponding quantization amplitude in the quantization table. As a result, the 8×8 quantized frequency coefficients constituting the block are obtained ( Figure 5 ). As a result, the data is compressed. In particular, in the high frequency domain, the frequency coefficients are compressed more by a larger quantization amplitude.
[0104] Subsequently, in the decoding process, the quantized data is inversely quantized. Specifically, each quantized frequency coefficient is inversely quantized according to the corresponding quantization amplitude in the quantization table. As a result, the 8×8 frequency coefficients constituting the block are obtained ( Figure 5 Thus, the data is decompressed.
[0105] The data of the block after quantization and inverse quantization is different from the data of the block before quantization and inverse quantization. The error in the data between these blocks is the error caused by the rounding of the values due to quantization, which is called quantization error. In particular, in the high-frequency domain, the quantization error is large due to the use of a larger quantization amplitude.
[0106] After that, inverse quantization is performed to complete the inverse frequency transformation of the data. Thus, according to the multiple bases of the frequency transformation, the multiple frequency coefficients constituting the block are synthesized into multiple pixel values constituting the block. Specifically, the 8×8 frequency coefficients constituting the block are transformed into the 8×8 pixel values constituting the block. In addition, the block of the reproduced image is obtained thereby.
[0107] In a block of a reproduced image, mosquito noise may be generated due to a quantization error, resulting in image quality degradation. In particular, mosquito noise is easily visually recognized in a flat area around an edge.
[0108] Figure 6 This is a block diagram showing the structure of the image encoding device in this embodiment. Figure 6The image encoding device 200 shown encodes an image per block. Each image is composed of a plurality of pixels, and a block is also composed of a plurality of pixels. A block can also be a unit of a randomly accessible area called an MCU. For example, an MCU of 16×8 pixels can be used as a block. The image encoding device 200 performs fixed-length or variable-length compression on each block of the image according to the target code amount.
[0109] In addition, the pixels of the image are composed of multiple components such as brightness and color difference. When the image encoding device 200 encodes the image block for each component, the multiple components can be two components of brightness and color difference, or three components of red, green and blue corresponding to RGB. In addition, the multiple components can also be four components corresponding to RGBA (RedGreenBlueAlpha) with transparency (alpha) added to the three components of red, green and blue.
[0110] Specifically, the image encoding device 200 includes a frequency converter 210, a quantization processor 220, an encoder 230, a feature quantity acquirer 240, and a target code quantity determiner 250. In addition, the quantization processor 220 includes a quantizer 221, a quantization table deriver 222, and a scale factor determiner 223. These components are, for example, circuits.
[0111] The feature quantity acquirer 240 may acquire the feature quantity of data of each component constituting a pixel in the processing target block. The feature quantity of the data may correspond to the complexity of the data.
[0112] For example, the feature quantity acquirer 240 may calculate, for each component, a statistical value of the absolute value of the difference between adjacent pixels in the data of the component as the feature quantity of the data of the component. The statistical value may be either a total value or an average value. The average value of the absolute value of the difference between adjacent pixels may also be expressed as activity.
[0113] In addition, the feature quantity acquirer 240 may also use Hadamard transform to acquire the feature quantity of the data of each component. For example, the feature quantity acquirer 240 may also acquire the edge quantity obtained by applying Hadamard transform to the data of the component as the feature quantity of the data of the component.
[0114] In addition, for example, the feature quantity acquirer 240 may acquire information representing the feature quantity of data of each component from a device external to the image encoding device 200. The external device may be a device that calculates the feature quantity. Alternatively, the external device may be a camera device that determines the feature quantity based on the camera conditions. Alternatively, the external device may determine the feature quantity based on the image type.
[0115] The target code amount determiner 250 determines the target code amount of the data of each component according to the feature amount of the data of the component in the processing target block. For example, the target code amount determiner 250 increases the target code amount as the feature amount increases.
[0116] In addition, the target code quantity determiner 250 may also determine the target code quantity of the data of each component according to the relationship between the feature quantity of the data of the component and the feature quantities of other components. Specifically, when the feature quantity of the data of the first component is larger than the feature quantity of the data of the second component, the first target code quantity of the data of the first component may be larger than the second target code quantity of the data of the second component.
[0117] Alternatively, the target code amount determiner 250 may maintain the total target code amount of data of a plurality of components in a plurality of blocks as a reference code amount, and adjust the ratio of the target code amounts between the components for each block according to the feature amount.
[0118] The frequency converter 210 performs frequency conversion on the data of the component in the processing target block for each component. For example, DCT is used in the frequency conversion. Thus, for each component, a plurality of pixel values of the component in the processing target block are converted into a plurality of frequency coefficients of the component in the processing target block.
[0119] The quantization processor 220 quantizes the transformed data of each component in the processing target block according to the target code amount of the data of the component in the processing target block, thereby compressing the data.
[0120] Specifically, the scale factor determiner 223 determines the scale factor according to the target code quantity, the transformed data and the fixed length coding algorithm so that the code quantity of the data meets the target code quantity. Here, the target code quantity is the target code quantity determined according to the characteristic quantity of the data of each component as the target code quantity of the data of the component.
[0121] The quantization table deriving unit 222 derives a quantization table corresponding to the data of the component according to the scale factor corresponding to the data of the component. The quantizer 221 quantizes the transformed data of the component according to the quantization table corresponding to the data of the component.
[0122] In the quantization table, the quantization width is specified for each frequency level. The scale factor affects the quantization width specified for each frequency level in the quantization table. For example, if the scale factor becomes larger, the quantization width also becomes larger. That is, the quantization width may also have a monotonically increasing relationship with the scale factor in a narrow sense or a broad sense. In addition, the quantization width may also be proportional to the scale factor. In addition, a scale factor may be used in which the quantization width becomes smaller if the scale factor becomes larger.
[0123] The encoder 230 encodes the quantized data of each component in the processing target block as a stream. Huffman code may be used for encoding.
[0124] In addition, the encoder 230 may also feed back the code amount of the data of each component to the scale factor determiner 223. In addition, the scale factor determiner 223 may also update the scale factor corresponding to the data of each component according to the code amount of the data of the component, thereby re-determining the scale factor corresponding to the data of the component. In addition, the determination of the scale factor, the derivation of the quantization table, the quantization and the encoding may be repeated.
[0125] Thus, the processing object block is encoded according to the target code amount. Here, the target code amount is a target code amount determined according to the characteristic amount of the data of each component as the target code amount of the data of the component. Specifically, for each component, the compression rate corresponding to the data of the component can be used as the target code amount of the data of the component. In particular, according to the target code amount of the data of each component in each block of the image, the code amount of the data of the component in the block is controlled.
[0126] In addition, the reference target code amount can also be set to the image encoding device 200 as a register setting value. In addition, the target code amount determiner 250 can also assign a gain or bias to the reference target code amount according to the feature amount, thereby determining the target code amount. The reference target code amount can be the same between multiple components or different between each component. In addition, the reference target code amount can also correspond to the above-mentioned total target code amount.
[0127] In addition, for example, the feature amount of the luminance data in the processing target block and the feature amount of the color difference data in the processing target block are obtained. In addition, the target code amount of the luminance data and the target code amount of the color difference data are determined according to the feature amount of the luminance data and the feature amount of the color difference data, respectively. In addition, the scale factor corresponding to the luminance data and the scale factor corresponding to the color difference data are determined according to the target code amount of the luminance data and the target code amount of the color difference data, respectively.
[0128] In addition, the luminance data and the color difference data are quantized and encoded according to the scale factor corresponding to the luminance data and the scale factor corresponding to the color difference data, respectively. Therefore, the code amount of the luminance data and the code amount of the color difference data are controlled according to the individual variable target code amounts.
[0129] In addition, as a general feature of images, especially in natural images, there is a tendency that the change in color difference is smaller than the change in brightness. Therefore, in color difference, noise based on quantization error is not easy to generate. In addition, as a characteristic of human vision, human vision has the characteristic of being less sensitive to changes in color difference than to changes in brightness. Therefore, in color difference, noise based on quantization error is not easy to be visually recognized. Considering such general image characteristics and human visual characteristics, it is sometimes beneficial to make the target code amount of brightness data larger than that of color difference data.
[0130] Therefore, the luminance data can be actively protected in the above-mentioned configuration of the image coding apparatus 200. Specifically, the target code amount determiner 250 can determine the target code amounts of the luminance data and the chrominance data so that the target code amount of the luminance data is larger than that of the chrominance data.
[0131] Figure 7 Yes means Figure 6 Flowchart of the operation of the image coding apparatus 200 shown in FIG. First, an image is divided into a plurality of blocks (S201). The division process (S201) can be performed by a divider (not shown) of the image coding apparatus 200 or by the frequency converter 210. Thereafter, a loop process (S202 to S208) is performed in units of blocks.
[0132] In the block-based loop processing (S202 to S208), the feature quantity acquirer 240 acquires the feature quantity of the data of the component in the processing target block for each component (S202). Next, the target code quantity determiner 250 determines the target code quantity of the data of the component in the processing target block according to the feature quantity of the data of the component in the processing target block for each component (S203). The frequency converter 210 performs frequency conversion on the data of the component in the processing target block for each component (S204).
[0133] Thereafter, the quantization processor 220 quantizes the transformed data of the component in the processing target block according to the scale factor for each component (S205). Also, the encoder 230 encodes the quantized data of the component in the processing target block for each component (S206).
[0134] Here, when the generated code amount of the encoded data of the component in the processing target block meets the target code amount of the data of the component in the processing target block for each component (S207: Yes), the processing of the processing target block is completed. On the other hand, when the generated code amount does not meet the target code amount (S207: No), the quantization processor 220 updates the scale factor (S208).
[0135] For example, when the generated code amount is larger than the target code amount, the quantization processor 220 increases the scale factor in order to increase the quantization amplitude. In addition, when the generated code amount is smaller than the target code amount, the quantization processor 220 decreases the scale factor in order to decrease the quantization amplitude. In addition, quantization (S205), encoding (S206) and updating of the scale factor (S208) are repeated until the generated code amount meets the target code amount.
[0136] In the above-mentioned operation, for each component, the feature quantity is acquired (S202), the target code quantity is determined (S203), the frequency conversion (S204), the quantization (S205), the encoding (S206), and the updating of the scale factor (S208) are performed. That is, for each component, the feature quantity is acquired, and the target code quantity is determined according to the feature quantity. In addition, according to the target code quantity of each component, the processing target block is encoded.
[0137] Fig. 8A : is a conceptual diagram showing an example of calculating the characteristic amount of brightness data. For example, the average value of the absolute value of the difference between adjacent pixels of brightness data is calculated as the characteristic amount of brightness data. Fig. 8A , represents a mathematical formula for calculating the average absolute value of the difference between adjacent pixels of the brightness data in the MCU of 16×8 pixels in the YUV422 format. The feature quantity acquirer 240 may also be based on Fig. 8A The mathematical formula shown calculates the average absolute value of the difference between adjacent pixels of the brightness data as the feature value of the brightness data.
[0138] In addition, Fig. 8A In the mathematical formula, for example, when n=0, Yn(i, j) represents the Y value of the pixel at (i, j) in the 8×8 pixels on the left, and when n=1, Yn(i, j) represents the Y value of the pixel at (i, j) in the 8×8 pixels on the right. Although not strictly speaking, Fig. 8A act_y roughly corresponds to the average absolute value of the difference between adjacent pixels of the brightness data.
[0139] Figure 8B 2 is a conceptual diagram showing an example of calculating the characteristic amount of color difference data. For example, the average value of the absolute value of the difference between adjacent pixels of the color difference data is calculated as the characteristic amount of the color difference data. Figure 8B , represents a mathematical formula for calculating the average absolute value of the difference between adjacent pixels of color difference data in a 16×8 pixel MCU in the YUV422 format. The feature quantity acquirer 240 may also be based on Figure 8B The mathematical formula shown calculates the average absolute value of the difference between adjacent pixels of the color difference data as the feature value of the color difference data.
[0140] In addition, Figure 8BIn the mathematical formula, for example, Cb(i, j) represents the Cb value of the pixel located at (i, j) in the 8×8 pixels corresponding to Cb, and Cr(i, j) represents the Cr value of the pixel located at (i, j) in the 8×8 pixels corresponding to Cr. Although not strictly speaking, Figure 8B act_c roughly corresponds to the average absolute value of the difference between adjacent pixels of the brightness data.
[0141] Fig. 9 2 is a conceptual diagram showing a first method for determining a scaling factor. In the first method, the scaling factor is determined by a heuristic method.
[0142] Specifically, the frequency converter 210 performs frequency conversion on the data in the processing target block of the image. The scale factor determiner 223 performs initial setting of the scale factor and quantizes the transformed data according to the scale factor. In addition, the scale factor determiner 223 predicts the code amount according to the quantized data to obtain the predicted code amount.
[0143] When the target code quantity does not meet the predicted code quantity, the scale factor determiner 223 updates the scale factor and quantizes the transformed data again according to the scale factor. In addition, the scale factor determiner 223 repeatedly performs quantization, code quantity prediction, and scale factor update, and determines the scale factor until the target code quantity meets the predicted code quantity.
[0144] Fig.10 Yes means Fig. 9 Flowchart of the first determination method shown. Fig.10 In particular, the operation of the scale factor determiner 223 is shown. Specifically, first, the scale factor determiner 223 performs initial setting of the scale factor (S301). For example, the scale factor determiner 223 may use the average scale factor as the initial value to perform initial setting of the scale factor. Thus, the increase in the number of explorations in the heuristic method is suppressed.
[0145] In addition, the scale factor determiner 223 quantizes the transformed data according to the scale factor (S302). Instead of the scale factor determiner 223, the quantizer 221 may quantize the transformed data according to the scale factor.
[0146] In addition, the scale factor determiner 223 predicts the code amount according to the quantized data to obtain a predicted code amount (S303). The scale factor determiner 223 may obtain a predicted code amount by applying Huffman coding to the quantized data as the predicted code amount.
[0147] Alternatively, the quantized data may be encoded by the encoder 230 instead of the scale factor determiner 223. In addition, the scale factor determiner 223 may obtain the code amount from the encoder 230 to predict the code amount.
[0148] Next, the scale factor determiner 223 calculates the target code amount minus the predicted code amount (S304). In addition, the scale factor determiner 223 determines whether the target code amount minus the predicted code amount satisfies the polarity and convergence conditions (S305). This condition corresponds to the condition that the target code amount meets the predicted code amount.
[0149] When the target code amount minus the predicted code amount does not satisfy the condition (S305: No), the scale factor determiner 223 updates the scale factor according to the polarity and size of the target code amount minus the predicted code amount (S306). In addition, the scale factor determiner 223 repeatedly performs quantization (S302), code amount prediction (S303), target code amount minus predicted code amount calculation (S304), and scale factor update (S306) until the target code amount minus predicted code amount satisfies the condition.
[0150] For example, when the target code amount minus the predicted code amount is a positive value and is equal to or smaller than a threshold, the scale factor determiner 223 determines that the condition is satisfied.
[0151] Alternatively, when the target code amount minus the predicted code amount is a positive value and the amount of change associated with the update of the scale factor is less than a threshold value, the scale factor determiner 223 may determine that the target code amount minus the predicted code amount satisfies the condition. Alternatively, when the target code amount minus the predicted code amount is a positive value and the scale factor is updated more than a threshold number of times, the scale factor determiner 223 may determine that the target code amount minus the predicted code amount satisfies the condition.
[0152] In addition, the scale factor determiner 223 may also gradually reduce the amount of change in the scale factor by 1 / 2 each time the scale factor is updated, and update the scale factor until the amount of change in the scale factor reaches the minimum unit.
[0153] In the updating of the scale factor, when the target code amount minus the predicted code amount is a positive value, the scale factor determiner 223 may also reduce the scale factor. In addition, when the target code amount minus the predicted code amount is a negative value, the scale factor determiner 223 may also increase the scale factor. In addition, the scale factor determiner 223 may also increase the amount of change in the scale factor when the absolute value of the target code amount minus the predicted code amount is larger.
[0154] If the target code amount minus the predicted code amount satisfies the condition (S305: Yes), the scale factor determiner 223 determines the scale factor at this time as the final scale factor.
[0155] Fig.112 is a conceptual diagram showing a second method for determining a scale factor. In the second method, the scale factor is initially set according to the feature amount of the data in the processing target block of the image. Specifically, the scale factor determiner 223 obtains the feature amount of the data in the processing target block of the image, and initially sets the scale factor according to the feature amount. The rest is the same as the first method.
[0156] Fig.12 Yes means Fig.11 Flowchart of the second determination method shown. Fig.12 In particular, the operation of the scale factor determiner 223 is shown.
[0157] Specifically, first, the scale factor determiner 223 obtains the feature amount of the data (S401). For example, the scale factor determiner 223 may obtain the feature amount of the data in the same manner as the feature amount acquirer 240, or may obtain the feature amount of the data from the feature amount acquirer 240. Alternatively, the scale factor determiner 223 may obtain the feature amount of the data using a different reference and method from the feature amount acquirer 240.
[0158] Next, the scale factor determiner 223 performs initial setting of the scale factor according to the feature quantity of the data (S402). For example, the scale factor determiner 223 may also use a smaller value as the feature quantity increases as the initial value of the scale factor to perform initial setting of the scale factor. Thus, the number of explorations in the heuristic method is suppressed from increasing. Therefore, processing delay is suppressed and throughput performance is improved.
[0159] The subsequent processing (S403 to S407) is the same as the corresponding processing (S302 to S306) in the first determination method.
[0160] Fig.13 : is a conceptual diagram showing a third determination method of a scale factor. In the third determination method, the scale factor finally determined by updating the scale factor is compared with the initial value of the scale factor. In addition, when the difference between them is larger than a predetermined threshold value, the initial value of the scale factor corresponding to the feature quantity is updated. On the other hand, when the difference is less than the threshold value, the initial value of the scale factor is not updated. The initial value of the scale factor can be used for subsequent blocks having the same feature quantity.
[0161] The initial value of the scale factor may be updated to the value of the scale factor finally determined, or may be updated to the intermediate value (average value) between the initial value before update and the value of the scale factor finally determined. The rest is the same as the second determination method.
[0162] Fig.14 Yes means Fig.13 Flowchart of the third determination method shown. Fig.14 In particular, the operation of the scale factor determiner 223 is shown.
[0163] Specifically, the scale factor determiner 223 performs the same processing (S501 to S507) as the corresponding processing (S401 to S407) in the second determination method until the scale factor is finally determined. After the scale factor is finally determined (S506: Yes), the scale factor determiner 223 compares the finally determined scale factor with the initial value of the scale factor, and compares the difference between them with a predetermined threshold value (S508).
[0164] If the difference is larger than a predetermined threshold (S508: Yes), the scale factor determiner 223 updates the initial value of the scale factor corresponding to the feature value (S509). If the difference is smaller than a predetermined threshold (S508: No), the scale factor determiner 223 does not update the initial value.
[0165] In addition, for example, even when it is determined that the difference is larger than a predetermined threshold value, the scale factor determiner 223 may not immediately update the initial value, but may increment the count value corresponding to the number of times the difference is determined to be larger than the predetermined threshold value. Alternatively, the scale factor determiner 223 may update the initial value only when the count value exceeds the threshold value corresponding to the predetermined number of times. Furthermore, the count value may be initialized to 0 at the timing when encoding of the image starts, at the timing when the count value exceeds the threshold value, or the like.
[0166] As a result, the effect of suppressing the increase in the number of searches in the search method is expected to be further improved for subsequent processing target blocks having the same feature value. Therefore, compared with the second determination method, processing delay is further suppressed, and throughput performance is expected to be further improved.
[0167] In addition, in other determination methods of the scale factor, the initial value of the scale factor may be updated according to the final value of the scale factor, similarly to the third determination method.
[0168] Fig.15 2 is a conceptual diagram showing a fourth method for determining a scale factor. In the example of the fourth method, the scale factor determiner 223 includes a first scale factor determiner 310, a second scale factor determiner 320, and a scale factor selector 330. These components are, for example, circuits.
[0169] The first scale factor determiner 310 determines the first scale factor by the same method as the first scale factor determination method. That is, the first scale factor determiner 310 determines the scale factor determined by the first scale factor determination method as the first scale factor. In addition, the first scale factor determiner 310 obtains the code amount predicted according to the first scale factor as the first predicted code amount.
[0170] The second scale factor determiner 320 determines the second scale factor according to the feature amount of the data in the processing target block of the image. Next, the scale factor determiner 223 quantizes the transformed data according to the second scale factor. In addition, the scale factor determiner 223 predicts the code amount of the data quantized according to the second scale factor to obtain a second predicted code amount.
[0171] The scale factor selector 330 selects a scale factor from among the first scale factor and the second scale factor according to the first predicted code amount and the second predicted code amount, thereby determining the scale factor.
[0172] Fig.16 Yes means Fig.15 Flowchart of the fourth determination method shown. Fig.16 In particular, the operations of the first scale factor determiner 310 , the second scale factor determiner 320 , and the scale factor selector 330 in the scale factor determiner 223 are shown.
[0173] Specifically, the first scale factor determiner 310 performs the same processing (S601 to S606) as the processing (S301 to S306) in the first determination method, except that the scale factor and the predicted code amount in the first determination method are replaced with the first scale factor and the first predicted code amount.
[0174] In addition, the second scale factor determiner 320 obtains the feature amount of the data (S607). For example, the second scale factor determiner 320 may obtain the feature amount of the data in the same manner as the feature amount acquirer 240, or may obtain the feature amount of the data from the feature amount acquirer 240. Alternatively, the second scale factor determiner 320 may obtain the feature amount of the data using a different criterion and method from the feature amount acquirer 240.
[0175] Next, the second scale factor determiner 320 determines the second scale factor according to the feature amount of the data (S608). For example, the second scale factor determiner 320 may determine a smaller value as the feature amount increases as the second scale factor.
[0176] In addition, the second scale factor determiner 320 quantizes the transformed data according to the second scale factor (S609). Instead of the second scale factor determiner 320, the quantizer 221 may quantize the transformed data according to the second scale factor.
[0177] In addition, the second scale factor determiner 320 predicts the code amount according to the quantized data to obtain a second predicted code amount (S610). The second scale factor determiner 320 may also obtain a code amount predicted by applying Huffman code to the quantized data as the second predicted code amount.
[0178] Alternatively, the quantized data may be encoded by the encoder 230 instead of the second scale factor determiner 320. In addition, the second scale factor determiner 320 may obtain the code amount from the encoder 230 to predict the code amount.
[0179] Next, the scale factor selector 330 selects a scale factor from among the first scale factor and the second scale factor according to the first predicted code amount and the second predicted code amount, thereby determining the scale factor (S611).
[0180] For example, the scale factor selector 330 selects the first scale factor when the first predicted code amount is more consistent with the target code amount than the second predicted code amount, and selects the second scale factor when the second predicted code amount is more consistent with the target code amount than the first predicted code amount. Here, it can also be stipulated that if the predicted code amount is less than the target code amount and is closer to the target code amount, it is more consistent with the target code amount.
[0181] In the fourth determination method, a scale factor is selected from the first scale factor based on the heuristic method and the second scale factor based on the feature amount. This prevents the scale factor corresponding to the local solution from being determined as the final scale factor in the heuristic method.
[0182] Fig.17 : is a conceptual diagram showing a fifth method for determining a scale factor. In the example of the fifth method, as in the example of the fourth method, the scale factor determiner 223 includes a first scale factor determiner 310, a second scale factor determiner 320, and a scale factor selector 330. In the fifth method, the first scale factor is initially set according to the feature amount of data in the processing target block of the image. Specifically, the first scale factor determiner 310 initially sets the first scale factor according to the feature amount obtained by the second scale factor determiner 320. The rest is the same as the fourth method.
[0183] Fig.18 Yes means Fig.17 Flowchart of the fifth determination method shown. Fig.18In particular, the operations of the first scale factor determiner 310 , the second scale factor determiner 320 , and the scale factor selector 330 in the scale factor determiner 223 are shown.
[0184] Specifically, first, the second scale factor determiner 320 obtains the feature amount of data ( S701 ). For example, the second scale factor determiner 320 may obtain the feature amount of data in the same manner as the feature amount acquirer 240 , or may obtain the feature amount of data from the feature amount acquirer 240 .
[0185] Next, the first scale factor determiner 310 performs initial setting of the first scale factor according to the feature quantity of the data (S702). For example, the first scale factor determiner 310 may also use a smaller value as the feature quantity increases as the initial value of the first scale factor to perform initial setting of the first scale factor. Thus, the increase in the number of explorations in the heuristic method is suppressed. Therefore, processing delay is suppressed and throughput performance is improved.
[0186] The subsequent processing (S703 to S711) is the same as the corresponding processing (S602 to S606 and S608 to S611) in the fourth determination method.
[0187] In the fifth determination method, the second scale factor and the second predicted code amount are the same as the initial first scale factor and the first predicted code amount in the heuristic method.
[0188] Therefore, in the fifth determination method, the second scale factor can also be interpreted as the initial first scale factor in the heuristic method, and the second prediction code amount can also be interpreted as the initial first prediction code amount in the heuristic method. In addition, the process of selecting the scale factor in the fifth determination method can also be interpreted as the process of selecting the scale factor from the initial first scale factor and the final first scale factor according to the initial first prediction code amount and the final first prediction code amount.
[0189] Fig.19 This is a conceptual diagram showing a first compression example in this embodiment. Fig.19 and Figure 4 The example also shows the compression of brightness data and color difference data in a 16×8 pixel MCU. In addition, the overall compression rate is 25%, and the overall code volume is 512 bits. In other words, the overall compression rate and code volume are similar to Figure 4 Same as the example.
[0190] On the other hand, Fig.19In the example, the compression rate of the luminance data (Y) is 37.5%, and the code size of the luminance data (Y) is 384 bits. In addition, the compression rate of the color difference data (Cb and Cr) is 12.5%, and the code size of the color difference data (Cb and Cr) is 128 bits. In other words, the compression rate and code size of the luminance data (Y) are different from the compression rate and code size of the color difference data (Cb and Cr).
[0191] In this embodiment, the compression rate and the amount of code of the data vary according to the feature amount of the data of each component in the processing target block. For example, with respect to each component, the larger the feature amount of the data of the component in the processing target block, the larger the target amount of code is determined for the data of the component, and the larger the compression rate and the amount of code obtained as a result for the data of the component are also.
[0192] The target code amount determiner 250 may also change the ratio of the target code amounts corresponding to the multiple components according to the processing target block, and keep the total of the target code amounts constant regardless of which processing target block. In addition, the target code amount determiner 250 may make the target code amount of the luminance data larger than the target code amount of the color difference data. In addition, the target code amount determiner 250 may also determine to what extent the target code amount of the luminance data is larger than the target code amount of the color difference data according to the feature amount of the luminance data and the feature amount of the color difference data.
[0193] In addition, the encoder 230 may also encode the scale factor and include the encoded scale factor in the stream. For example, the image decoding device decodes the quantized data and the scale factor from the stream, and performs inverse quantization on the quantized data according to the scale factor. In addition, the image decoding device performs inverse frequency transformation on the inversely quantized data to restore the data.
[0194] As described above, the image coding apparatus 200 obtains the feature quantity for each component and determines the target code quantity according to the feature quantity. Therefore, the image coding apparatus 200 can suppress the image quality degradation associated with the compression of the image data. That is, the image coding apparatus 200 can suppress the image degradation and reduce the code quantity of the image data.
[0195] Therefore, the image coding device 200 can reduce the memory capacity for storing image data. In addition, the device for processing image data can be miniaturized, cost-reduced, and power consumption can be reduced. In addition, the pressure on the memory bandwidth when accessing image data can be alleviated, and dynamic images can be reproduced with high image quality and high frame rate.
[0196] Fig. 20 It is a block diagram showing the structure of the image processing device in this embodiment. Fig. 20The image processing device 400 shown includes an image input device 401, image compressors 402 and 406, image decompressors 403 and 407, an image output device 404, a rendering processor 405, a memory controller 408, and a memory 409. These components are, for example, circuits.
[0197] The image input device 401 obtains an input image. For example, the image input device 401 obtains an image input from a camera or an image sensor.
[0198] The image compressors 402 and 406 each correspond to the image encoding device 200 and encode the image for each block. At this time, the image compressors 402 and 406 each compress the data in the processing target block.
[0199] The memory controller 408 controls access from each component to the memory 409 based on the bus protocol, and controls reading and writing of data with respect to the memory 409 .
[0200] The memory 409 is a memory built into the image processing device 400. For example, the memory 409 stores compressed image data according to the control of the memory controller 408. In addition, the compressed image data is read out from the memory 409 and decompressed (decompressed) by the image decompressors 403 and 407.
[0201] The image decompressors 403 and 407 decode the image for each block. At this time, the image decompressors 403 and 407 decompress the data in the processing target block.
[0202] The rendering processor 405 renders an image. The rendering processor 405 can edit an image or generate a graphic image.
[0203] The image output device 404 outputs an image. For example, the image output device 404 outputs the image to a display device, etc. The image output device 404 may output a plurality of images in an overlapping manner.
[0204] Fig.21 Yes means Fig. 20 The block diagrams of the image compressors 402 and 406 are shown. The image compressors 402 and 406 each include a local buffer 510, a preprocessor 520, encoding engines 531 to 534, a postprocessor 540, and a request buffer 550. In addition, the local buffer 510 includes a local arbitrator 511. These components are, for example, circuits.
[0205] The feature quantity acquirer 240 and the target code amount determiner 250 of the image coding apparatus 200 may also be included in the preprocessor 520. In addition, the frequency converter 210 and the quantization processor 220 of the image coding apparatus 200 may also be included in the coding engines 531 to 534. In addition, the encoder 230 of the image coding apparatus 200 may also be included in the coding engines 531 to 534 and the post-processor 540.
[0206] Coding engines 531 to 534 correspond to four sequences corresponding to four components of pixels constituting an image. Here, two sequences corresponding to luminance and chrominance are used among the four sequences. Specifically, coding engine 533 is used for luminance, and coding engine 534 is used for chrominance.
[0207] The image to be compressed is input to the local buffer 510. The local arbiter 511 obtains control information and mediates access to the local buffer 510 according to the control information. Specifically, the local arbiter controls simultaneous and parallel processing of luminance data (Y) and color difference data (C) in a processing target block of the image.
[0208] The preprocessor 520 obtains the luminance data and the color difference data from the local buffer 510. In addition, the preprocessor 520 obtains the feature quantity of the luminance data and the feature quantity of the color difference data according to the luminance data and the color difference data. In addition, the preprocessor 520 calculates the complexity ratio according to the feature quantity of the luminance data and the feature quantity of the color difference data. Here, the complexity ratio is the ratio of the complexity of the luminance data to the complexity of the color difference data.
[0209] Furthermore, the preprocessor 520 determines the target code amount of the luminance data and the target code amount of the chrominance data according to the complexity ratio. The preprocessor 520 may also refer to a reference table described below to derive the target code amount of the luminance data and the target code amount of the chrominance data from the complexity ratio.
[0210] The encoding engine 533 obtains the luminance data and the target code size of the luminance data from the preprocessor 520. In addition, the encoding engine 533 performs frequency conversion on the luminance data, quantizes the converted luminance data according to the target code size of the luminance data, and encodes the quantized luminance data.
[0211] The encoding engine 534 obtains the color difference data and the target code amount of the color difference data from the preprocessor 520. In addition, the encoding engine 534 performs frequency conversion on the color difference data, quantizes the converted color difference data according to the target code amount of the color difference data, and encodes the quantized color difference data.
[0212] The post-processor 540 obtains the encoded luminance data from the encoding engine 533 and the encoded color difference data from the encoding engine 534. In addition, the post-processor 540 obtains the complexity ratio from the pre-processor 520 via the encoding engine 533 or the encoding engine 534. Alternatively, the post-processor 540 may obtain the complexity ratio directly from the pre-processor 520 without going through the encoding engine 533 or the encoding engine 534.
[0213] In addition, the post-processor 540 inserts the identification code of the complexity ratio into the front end of the encoded color difference data. In addition, the post-processor 540 connects the encoded color difference data and the encoded brightness data in the order thereof. Thus, the post-processor 540 encapsulates the identification code of the complexity ratio, the encoded color difference data, and the encoded brightness data, and generates a stream including them. That is, the post-processor 540 multiplexes the identification code of the complexity ratio, the encoded color difference data, and the encoded brightness data into a stream.
[0214] In addition, the post-processor 540 stores the stream in the request buffer 550. In addition, the post-processor 540 performs request control on the stream.
[0215] The request buffer 550 stores a stream in which the complexity ratio identification code, the encoded color difference data, and the encoded brightness data are packed. The stream stored in the request buffer 550 is output to the memory 409 or the like.
[0216] The operation performed by the preprocessor 520 may be performed by the feature quantity acquirer 240 or the target code amount determiner 250 of the image coding apparatus 200. The operation performed by the coding engines 531 to 534 may be performed by the frequency converter 210 or the quantization processor 220 of the image coding apparatus 200. The operation performed by the coding engines 531 to 534 and the postprocessor 540 may be performed by the encoder 230 of the image coding apparatus 200.
[0217] Fig.22A This is a diagram showing a reference table of complexity ratio and target code amount. In this example, a complexity ratio corresponding to the ratio of the complexity of luminance data to the complexity of chrominance data is used. For example, when the ratio of the complexity of luminance data to the complexity of chrominance data is less than 1, 0 is used as the complexity ratio.
[0218] In addition, when the ratio is greater than or equal to 1 and less than 2, 1 is used as the complexity ratio. When the ratio is greater than or equal to 2 and less than 3, 2 is used as the complexity ratio. When the ratio is greater than or equal to 3 and less than 4, 3 is used as the complexity ratio. When the ratio is greater than or equal to 4 and less than 5, 4 is used as the complexity ratio. When the ratio is greater than or equal to 5 and less than 6, 5 is used as the complexity ratio. When the ratio is greater than or equal to 6 and less than 7, 6 is used as the complexity ratio. When the ratio is greater than or equal to 7, 7 is used as the complexity ratio.
[0219] In addition, the target code amount of the brightness data and the target code amount of the color difference data are associated with the complexity ratio. In this example, the larger the complexity ratio, the larger the target code amount of the brightness data, and the smaller the target code amount of the color difference data. Regardless of the complexity ratio, the sum of the target code amount of the brightness data and the target code amount of the color difference data is fixed. In addition, regardless of the complexity ratio, the target code amount of the brightness data is larger than the target code amount of the color difference data.
[0220] exist Fig.22A In the example, Figure 4 and Fig.19 Similarly to the example, the data size of the uncompressed brightness data and the data size of the uncompressed color difference data are each 1024 bits, and the total is 2048 bits. Fig.22A In the reference table, the ratio of the target code amount to the uncompressed data amount is expressed as the compression rate.
[0221] In addition, in the reference table, only the target code amount represented by the number of bits for brightness or color difference can be corresponded to the complexity ratio, and only the compression rate for brightness or color difference can be corresponded to the complexity ratio as information representing the target code amount.
[0222] Fig. 22B It is a graph showing the relationship between complexity ratio and compression ratio. Fig. 22B The graph corresponds to Fig.22A The values shown.
[0223] The greater the relative complexity of the luminance data, the greater the target code volume (compression rate) of the luminance data, and the smaller the target code volume (compression rate) of the color difference data. In addition, regardless of the complexity ratio, the target code volume of the luminance data is greater than the target code volume of the color difference data. The greater the relative complexity of the luminance data, the greater the difference between the target code volume (compression rate) of the luminance data and the target code volume (compression rate) of the color difference data.
[0224] Fig.23 Yes means Fig. 20 and Fig.21 Flowchart of the operations of the image compressors 402 and 406 respectively shown.
[0225] First, the preprocessor 520 sets a reference table that establishes a correspondence between the complexity ratio between components and the target code amount for each component (S801). The preprocessor 520 may also set the reference table for each image. For example, the preprocessor 520 may also set the reference table for each frame constituting a dynamic image.
[0226] Next, the preprocessor 520 obtains the feature amount of the luminance data (S802). In addition, the preprocessor 520 obtains the feature amount of the color difference data in parallel (S803). In addition, the preprocessor 520 calculates a complexity ratio which is a ratio of the complexity of the luminance data to the complexity of the color difference data according to the feature amount of the luminance data and the feature amount of the color difference data (S804). In addition, the preprocessor 520 determines the target code amount of the luminance data and the target code amount of the color difference data according to the complexity ratio (S805).
[0227] Next, the encoding engine 533 encodes the luminance data according to the target code amount of the luminance data (S806). Specifically, the encoding engine 533 performs frequency transformation on the luminance data, quantizes the transformed luminance data according to the target code amount of the luminance data, and encodes the quantized luminance data.
[0228] The encoding engine 534 encodes the color difference data in parallel according to the target code amount of the color difference data (S807). Specifically, the encoding engine 534 performs frequency transformation on the color difference data, quantizes the transformed color difference data according to the target code amount of the color difference data, and encodes the quantized color difference data. In addition, the post-processor 540 inserts the identification code of the complexity ratio into the encoded color difference data (S808).
[0229] Thereafter, the post-processor 540 connects the encoded color difference data and the encoded brightness data in order ( S809 ).
[0230] Fig.24 : is a conceptual diagram showing the second compression example in this embodiment. Fig.24 In, with Fig.19 The example of 16×8 pixel MCU also shows the compression of brightness data and color difference data. In addition, the overall compression rate is 25%, and the overall code volume is 512 bits. In other words, the overall compression rate and code volume are similar to Fig.19 Same as the example.
[0231] On the other hand, Fig.24In the example, a 3-bit identification code is inserted into the color difference data. As a result, the color difference data is further compressed by an amount corresponding to 3 bits. For example, data in the high frequency domain of the color difference data is deleted by an amount corresponding to 3 bits. In addition, the brightness data follows the color difference data into which the identification code is inserted.
[0232] The identification code of the complexity ratio indicates the target code amount of each of the luminance data and the color difference data. When the generated code amount and the target code amount are consistent with each of the luminance data and the color difference data, the identification code can indicate the position corresponding to the division of the luminance data and the color difference data in the stream. Therefore, in this case, in the decoding process (decompression process), it is easy to separate the luminance data and the color difference data from the stream.
[0233] Furthermore, in the encoding process (compression process), when the generated code amount does not satisfy the target code amount, it is possible to make adjustments by padding or the like so that the generated code amount matches the target code amount.
[0234] Fig.25 Yes means Fig. 20 The block diagram of the image decompressors 403 and 407 is shown. The image decompressors 403 and 407 each include a request buffer 610, a preprocessor 620, decoding engines 631 and 632, a postprocessor 640, and a local buffer 650. In addition, the local buffer 650 includes a local arbitrator 651. These components are, for example, circuits.
[0235] The decoding engines 631 and 632 are composed of a decoder, an inverse quantizer, an inverse frequency converter, etc., in order to decode and transform the stream encoded by the image encoding device 200 corresponding to the image compressor 402, 406, etc. into each component of the image. The decoder may be a component that decodes the stream data obtained by performing Huffman encoding according to the target code amount when encoded by the image encoding device 200, for example.
[0236] The inverse quantizer may also be a component that inversely quantizes the output of the decoder according to the scale factor (header information in the Huffman code) when encoded by the image encoding device 200. The inverse frequency transformer may also use, for example, IDCT (Inverse Discrete Cosine Transform). Thus, the inverse frequency transformer can perform an inverse orthogonal transform on the inverse quantized frequency coefficients and restore the pixel data of each component of the image before encoding.
[0237] Decoding engines 631 and 632 are used, for example, to Fig.21The post-processor 540 shown decodes the stream formed by encapsulating the complexity ratio identification code, the encoded color difference data, and the encoded brightness data, and corresponds to two sequences of brightness data and color difference data. Specifically, a decoding engine 631 is used for brightness data, and a decoding engine 632 is used for color difference data. By processing them in parallel, the delay time until the image is output can be shortened.
[0238] In addition, the decoding engine 631 may be used to process the color difference data and the brightness data in series in such a manner that the brightness data is processed after the processing of the color difference data is completed. In addition, the processing order of the brightness data and the color difference data may be reversed. In addition, the preprocessor 620 may be used to effectively allocate each stream to the decoding engines 631 and 632 according to the priority of each stream output from the request buffer 610 and decode them. In addition, the decoding engine may be composed of more than one. The number of decoding engines may also be determined within the range of the delay time allowed until the image is output.
[0239] For example, a stream obtained by encapsulating the complexity ratio identification code, the encoded color difference data, and the encoded brightness data is transferred from the memory 409 and stored in the request buffer 610 .
[0240] That is, a stream obtained by encoding data of multiple components such as brightness, color difference, red, green, blue, and transparency constituting an image is stored in the request buffer 610. At this time, the preprocessor 620 performs request control in order to prioritize the streams of each component required for generating an output image and store them in the request buffer 610 based on the control information input to the local buffer 650.
[0241] The stream stored in the request buffer 610 is input to the preprocessor 620. For example, a stream in which the complexity ratio identification code, the encoded color difference data, and the encoded brightness data are packaged is input to the preprocessor 620. In this case, the preprocessor 620 Fig.25 The code separation shown determines the target code amount of brightness data and the target code amount of color difference data according to the identification code of the complexity ratio.
[0242] For example, the target code amount of luminance data and the encoded luminance data are input to the decoding engine 631. In addition, the target code amount of color difference data and the encoded color difference data are input to the decoding engine 632. The preprocessor 620 (code separation) can also refer to, for example, Fig.22A The reference table shown in FIG. 6 derives the target code amount of the luminance data and the target code amount of the color difference data from the identification code of the complexity ratio. In addition, the decoding engines 631 and 632 decode (decompress) the data of each component corresponding to the image before encoding, and input it to the post-processor 640.
[0243] The post-processor 640 converts the data of each component of the decompressed image into an arbitrary transfer unit and stores it in the local buffer 650 .
[0244] The data of each component of the decompressed image is input to the local buffer 650. The local arbiter 651 obtains the control information, mediates access to the local buffer 650 according to the control information, and generates and outputs the image to be output for each block.
[0245] Through the above-mentioned series of processing performed by the image decompressors 403 and 407, the compressed stream can be efficiently decompressed and converted into an image.
[0246] Fig.26 It is a diagram showing the image quality evaluation results. Fig.26 It shows the average PSNR and worst PSNR of multiple blocks in each of the 12 types of images when the target code amount is fixed and when the target code amount is variable and determined according to the feature amount. PSNR is the Peak Signal-to-Noise Ratio, and the higher it is, the less noise there is. Fig.26 In the figure, the underlined values correspond to PSNRs less than 30dB.
[0247] The feature quantity of brightness data and the feature quantity of color difference data differ depending on the type and block of the image. Therefore, by determining the target code quantity according to the feature quantity, the situation of extreme degradation of image quality is suppressed. In particular, by determining the target code quantity according to the feature quantity, the worst PSNR related to brightness is improved. Accompanying this, although the worst PSNR related to color difference deteriorates, it is maintained at more than 27dB, which is a level allowed by human visual characteristics.
[0248] Therefore, by determining the target code amount according to the feature amount, noise such as mosquito noise is suppressed, and subjective image quality degradation is suppressed.
[0249] Here, the two components of brightness and color difference are mainly used as the multiple components constituting the pixel, but the three components of red, green and blue corresponding to RGB can also be used. Furthermore, in addition to the three components of red, green and blue, a component of transparency (alpha) for mixing multiple images can also be used. The four components of red, green, blue and transparency correspond to RGBA. In such a case, by determining the target code amount according to the feature amount, it is also possible to suppress the situation where the feature is greatly lost.
[0250] In addition, the feature amount typically corresponds to complexity, but is not limited to complexity, and may also correspond to the size of the feature.
[0251] As described above, the image encoding apparatus 200 includes the feature quantity acquirer 240 , the target code quantity determiner 250 , the frequency converter 210 , the quantization processor 220 , and the encoder 230 .
[0252] The feature quantity acquirer 240 acquires the feature quantity of the data of each component of the pixels constituting the image in a processing target block among a plurality of blocks of the image. The target code quantity determiner 250 determines the target code quantity of the data of each component according to the feature quantity of the data of the component.
[0253] The frequency converter 210 performs frequency conversion on the data of each component. The quantization processor 220 quantizes the data of each component after the frequency conversion according to the target code amount of the data of the component. The encoder 230 encodes the quantized data of each component.
[0254] Thus, the image coding apparatus 200 can adjust the target code amount of the data of each component according to the feature amount of the data of the component. Therefore, the image coding apparatus 200 can suppress the situation where the feature is greatly lost. Therefore, the image coding apparatus 200 can suppress the degradation of image quality.
[0255] For example, the multiple components of the pixels constituting the image may include two components, namely, brightness and color difference. Thus, the image coding apparatus 200 can adjust the target code amount of the data of each component of brightness and color difference according to the feature amount of the data of the component. Therefore, the image coding apparatus 200 can appropriately suppress image quality degradation for an image having different feature amounts between brightness and color difference.
[0256] In addition, for example, the plurality of components constituting the pixels of the image may include three components of red, green, and blue. Thus, the image coding apparatus 200 can adjust the target code amount of the data of each component of red, green, and blue according to the feature amount of the data of the component. Therefore, the image coding apparatus 200 can appropriately suppress image quality degradation for images with different feature amounts between red, green, and blue.
[0257] In addition, for example, the multiple components of the pixels constituting the image may also include a transparency component. Thus, the image coding device can adjust the target code amount of the data of the component according to the feature amount of the data of the component when mixing multiple images using RGBA. Therefore, the image coding device 200 can appropriately suppress the degradation of the image quality when mixing multiple images. In addition, the memory capacity required to maintain the transparency component and the delay generated in the transmission of the data can be suppressed.
[0258] In addition, for example, the feature quantity acquirer 240 may acquire, for each component, a statistical value of the absolute value of the difference between adjacent pixels in the data of the component as the feature quantity of the data of the component. Thus, the image coding apparatus 200 can acquire a feature quantity corresponding to a sharp change between adjacent pixels, etc. Therefore, the image coding apparatus 200 can appropriately adjust the target code amount according to the feature quantity corresponding to a sharp change between adjacent pixels, etc.
[0259] In addition, for example, the feature quantity acquirer 240 may also use Hadamard transform to acquire the feature quantity of the data of each component. Thus, the image coding apparatus 200 can acquire the feature quantity corresponding to the edge quantity obtained by Hadamard transform. Therefore, the image coding apparatus 200 can appropriately adjust the target code quantity according to the feature quantity corresponding to the edge quantity.
[0260] In addition, for example, the feature quantity acquisition unit 240 may acquire information indicating the feature quantity of data of each component from a device external to the image coding apparatus 200. In addition, the feature quantity acquisition unit 240 may also acquire the feature quantity of data of each component. In this way, the image coding apparatus 200 can acquire the feature quantity without calculating the feature quantity. Therefore, the image coding apparatus 200 can reduce the calculation process.
[0261] In addition, for example, the encoder 230 may multiplex the identification codes indicating the target code amounts determined for the multiple components and the multiple data encoded for the multiple components into a stream. In addition, the encoder 230 may output the stream. Thus, the image encoding device 200 can indicate the target code amount of the data of each component in the stream. Therefore, the image encoding device 200 can support decoding of the data of each component from the stream.
[0262] In addition, for example, the quantization processor 220 may determine a scale factor affecting the quantization width according to the target code amount for each component. In addition, the quantization processor 220 may quantize the data of each component according to the scale factor.
[0263] Thus, the image coding apparatus 200 can adjust the scale factor used in quantizing the data of each component according to the target code amount of the data of the component. Therefore, the image coding apparatus 200 can appropriately adjust the code amount of the data of the component according to the target code amount of the data of the component.
[0264] In addition, for example, the quantization processor 220 may also initially set the scale factor in determining the scale factor. In addition, the quantization processor 220 may also quantize the data according to the scale factor. In addition, the quantization processor 220 may also obtain the predicted code amount of the data based on the data quantized according to the scale factor. In addition, the quantization processor 220 may also update the scale factor according to the comparison result of the predicted code amount and the target code amount.
[0265] In addition, the quantization processor 220 may repeatedly perform data quantization, obtain the predicted code amount, and update the scale factor until the predicted code amount matches the target code amount, thereby determining the scale factor.
[0266] Thus, the image coding apparatus 200 can search and determine the scale factor that makes the predicted code amount match the target code amount. Therefore, the image coding apparatus 200 can determine an appropriate scale factor for the target code amount.
[0267] In addition, for example, the quantization processor 220 may initialize the scale factor according to the feature amount of the data. Thus, the image coding apparatus 200 can appropriately initialize the scale factor when searching for the scale factor, and can suppress processing delay.
[0268] In addition, for example, the quantization processor 220 may update the initial value of the scale factor when the difference between the scale factor determined by repeatedly updating the scale factor and the initial value of the scale factor is larger than a threshold value. In addition, the quantization processor 220 may not update the initial value of the scale factor when the difference is less than the threshold value. Thus, the image encoding device 200 can update the initial value of the scale factor determined according to the feature amount to an optimal value based on the final scale factor, thereby suppressing processing delay in subsequent processing.
[0269] In addition, for example, the quantization processor 220 may increment the count value when the difference between the scale factor determined by repeatedly updating the scale factor and the initial value of the scale factor is greater than the first threshold value. In addition, the quantization processor 220 may not increment the count value when the difference is less than the first threshold value.
[0270] In addition, the quantization processor 220 may update the initial value of the scale factor when the count value is greater than the second threshold value. In addition, the quantization processor 220 may not update the initial value of the scale factor when the count value is less than the second threshold value.
[0271] Thus, the image coding apparatus 200 can update the initial value of the scale factor determined according to the feature amount at an appropriate update frequency based on the final scale factor, and thus can suppress processing delay in subsequent processing.
[0272] In addition, for example, the quantization processor 220 may also initially set the first scale factor in determining the scale factor. In addition, the quantization processor 220 may also quantize the data according to the first scale factor. In addition, the quantization processor 220 may also obtain a first predicted code amount of the data based on the data quantized according to the first scale factor. In addition, the quantization processor 220 may also update the first scale factor according to the comparison result between the first predicted code amount and the target code amount.
[0273] Alternatively, the quantization processor 220 may repeatedly perform quantization of data, acquisition of the first predicted code amount, and updating of the first scale factor until the first predicted code amount matches the target code amount, thereby determining the first scale factor.
[0274] In addition, the quantization processor 220 may determine the second scale factor according to the feature amount of the data. In addition, the quantization processor 220 may quantize the data according to the second scale factor. In addition, the quantization processor 220 may obtain the second prediction code quantity of the data based on the data quantized according to the second scale factor.
[0275] Furthermore, the quantization processor 220 may determine one of the first scale factor and the second scale factor as the scale factor based on a comparison result between the first predicted code amount and the target code amount and a comparison result between the second predicted code amount and the target code amount.
[0276] Thus, the image coding apparatus 200 can determine the scale factor using both the method of searching for the scale factor that makes the predicted code amount match the target code amount and the method of determining the scale factor based on the feature amount. Therefore, the image coding apparatus 200 can prevent the scale factor corresponding to the local solution being explored from being determined as the final scale factor.
[0277] In addition, for example, the quantization processor 220 may initialize the first scale factor according to the feature amount of the data. Thus, the image coding apparatus 200 can appropriately initialize the scale factor when searching for the scale factor, thereby suppressing processing delay.
[0278] Although the image coding apparatus has been described above according to the embodiments, the image coding apparatus is not limited to the embodiments. The embodiments may be modified as conceived by those skilled in the art, and a plurality of components in the embodiments may be arbitrarily combined.
[0279] For example, for a process performed by a specific constituent element in the embodiment, it is also possible to replace the specific constituent element and perform it by another constituent element. In addition, the order of multiple processes can be changed, and multiple processes can be performed in parallel. In addition, multiple variations can be combined and applied. In addition, for the first and second ordinal numbers used in the description, they can be appropriately replaced, removed, or newly assigned. These ordinal numbers do not necessarily have to correspond to a meaningful order, and can also be used to identify elements.
[0280] In addition, the image coding method including the steps performed by the components of the image coding device may be executed by any device or system. For example, part or all of the image coding method may be executed by a computer having a processor, a memory, an input / output circuit, etc. In this case, the image coding method may be executed by executing a program for causing the computer to execute the image coding method.
[0281] Furthermore, the above-mentioned program may be recorded on a non-volatile computer-readable recording medium such as a CD-ROM.
[0282] In addition, each component of the image coding device may be composed of dedicated hardware, general-purpose hardware that executes the above-mentioned program, etc., or a combination thereof. In addition, the general-purpose hardware may be composed of a memory that records the program and a general-purpose processor that reads the program from the memory and executes it. Here, the memory may be a semiconductor memory or a hard disk, etc., and the general-purpose processor may be a CPU, etc.
[0283] In addition, the dedicated hardware may be composed of a memory and a dedicated processor, etc. For example, the dedicated processor may refer to the memory for recording data and execute the above-mentioned image encoding method.
[0284] In addition, each component of the image coding device may be a circuit. These circuits may constitute one circuit as a whole, or each may be an individual circuit. In addition, these circuits may correspond to dedicated hardware or general-purpose hardware that executes the above-mentioned program, etc.
[0285] Industrial Applicability
[0286] The present disclosure is useful as, for example, an encoding device that encodes an image, and can be applied to a digital camera, a digital video camera, a digital video recorder, an image processing system, and the like.
[0287] Description of reference numerals:
[0288] 100, 200 Image encoding device
[0289] 110, 210 frequency converter
[0290] 120, 220 Quantization Processor
[0291] 121, 221 Quantizer
[0292] 122, 222 Quantization table exporter
[0293] 123, 223 scale factor determiner
[0294] 130, 230 encoder
[0295] 240 Feature acquisition device
[0296] 250 Target Code Size Determiner
[0297] 310 1st scale factor determiner
[0298] 320 2nd scale factor determiner
[0299] 330 Scale factor selector
[0300] 400 Image processing device
[0301] 401 Image Importer
[0302] 402, 406 Image Compressor
[0303] 403, 407 Image Decompressor
[0304] 404 Image Exporter
[0305] 405 Dependency Processor
[0306] 408 Memory Controller
[0307] 409 Memory
[0308] 510, 650 local buffer
[0309] 511, 651 local arbitrator
[0310] 520, 620 Preprocessor
[0311] 531, 532, 533, 534 encoding engines
[0312] 540, 640 postprocessor
[0313] 550, 610 Request Buffer
[0314] 631, 632 Decoding Engine
Claims
1. An image encoding device, comprising: a feature quantity acquirer for acquiring, for each of a plurality of components of pixels constituting an image, a feature quantity of data of the component in a processing target block among a plurality of blocks of the image; a target code amount determiner for determining, for each of the plurality of components, a target code amount of the data of the component according to the feature amount of the data of the component; A frequency converter for performing frequency conversion on the data of each of the plurality of components; A quantization processor, for each of the plurality of components, quantizes the data of the component after the frequency transformation according to the target code amount of the data of the component; as well as The encoder encodes the quantized data of each of the plurality of components.
2. The image encoding device according to claim 1, The multiple components include two components, namely, brightness and color difference.
3. The image encoding device according to claim 1, The multiple components include three components of red, green and blue.
4. The image encoding device according to claim 1, The plurality of components includes a component of transparency.
5. The image encoding device according to any one of claims 1 to 4, The feature quantity acquirer acquires, for each of the plurality of components, a statistical value of an absolute value of a difference between adjacent pixels in the data of the component as the feature quantity of the data of the component.
6. The image encoding device according to any one of claims 1 to 4, The feature quantity acquirer acquires the feature quantity of the data of each of the plurality of components using Hadamard transform.
7. The image encoding device according to any one of claims 1 to 4, The feature amount acquirer acquires, for each of the plurality of components, information indicating the feature amount of the data of the component from a device external to the image encoding device, thereby acquiring the feature amount of the data of the component.
8. The image encoding device according to any one of claims 1 to 4, The encoder multiplexes identification codes indicating a plurality of target code amounts determined for the plurality of components and a plurality of data encoded for the plurality of components into a stream, and outputs the stream.
9. The image encoding device according to any one of claims 1 to 4, The quantization processor determines a scale factor affecting a quantization amplitude according to the target code amount for each of the plurality of components, and quantizes the data of the component according to the scale factor.
10. The image encoding device according to claim 9, The quantization processor performs: Initially setting the scale factor, quantizing the data according to the scaling factor, Based on the data quantized according to the scale factor, a prediction code quantity of the data is obtained, According to the comparison result between the predicted code amount and the target code amount, the proportional factor is updated, The quantization of the data, the acquisition of the predicted code amount, and the updating of the scale factor are repeated until the predicted code amount matches the target code amount, thereby determining the scale factor.
11. The image encoding device according to claim 10, The quantization processor initially sets the scale factor according to the feature quantity of the data.
12. The image encoding device according to claim 11, The quantization processor updates the initial value of the scale factor when the difference between the scale factor determined by repeatedly updating the scale factor and the initial value of the scale factor is larger than a threshold value, and does not update the initial value of the scale factor when the difference is smaller than the threshold value.
13. The image encoding device according to claim 11, The quantization processor performs: When the difference between the scale factor determined by repeatedly updating the scale factor and the initial value of the scale factor is greater than a first threshold, the count value is incremented, and when the difference is less than the first threshold, the count value is not incremented, When the count value is larger than the second threshold, the initial value of the scale factor is updated, and when the count value is smaller than the second threshold, the initial value of the scale factor is not updated.
14. The image encoding device according to claim 9, The quantization processor performs: Initialize the first scale factor. quantizing the data according to the first scale factor, Based on the data quantized according to the first scale factor, a first prediction code quantity of the data is obtained, According to the comparison result between the first predicted code amount and the target code amount, the first proportional factor is updated, Repeating the quantization of the data, the acquisition of the first predicted code amount, and the updating of the first scale factor until the first predicted code amount matches the target code amount, thereby determining the first scale factor, Determining a second scale factor according to the feature quantity of the data, quantizing the data according to the second scale factor, Based on the data quantized according to the second scale factor, a second prediction code amount of the data is obtained, Based on a comparison result between the first predicted code amount and the target code amount and a comparison result between the second predicted code amount and the target code amount, one of the first scale factor and the second scale factor is determined as the scale factor.
15. The image encoding device according to claim 14, The quantization processor initially sets the first scale factor according to the feature amount of the data.
16. An image encoding method, comprising: For each of a plurality of components of pixels constituting an image, a step of acquiring a feature amount of data of the component in a processing target block among a plurality of blocks of the image; For each of the plurality of components, a step of determining a target code size of the data of the component according to the feature amount of the data of the component; For each of the plurality of components, a step of performing frequency transformation on the data of the component; For each of the plurality of components, a step of quantizing the data of the component after frequency conversion according to the target code amount of the data of the component; as well as For each of the plurality of components, a step of encoding the quantized data of the component.
