Image encoding device, its control method, and program

Abstract

To generate encoded data that is suitable for an image in which compression ratios in both of optically horizontal and vertical directions are different.SOLUTION: An image encoding device comprises: a DWT transformation unit that performs wavelet transformation of image data and generates a plurality of pieces of sub-band data; a determination unit that determines a quantization parameter for transformation coefficients in a plurality of sub-bands obtained by the DWT transformation unit; and an encoding unit that performs quantizing and encoding of the transformation coefficients in the sub-band data obtained by the DWT transformation unit, following the quantization parameter determined by the determination unit. Here, the determination unit determines such that a quantization parameter which is applied to a LH sub-band having the same decomposition level is smaller than a quantization parameter which is applied to a HL sub-band in a case where for example a compression ratio between an optically horizontal and vertical directions indicates a compression ratio in which an image in a horizontal direction is α (α<1) times relative to a vertical direction in the image obtained by an imaging unit.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an image encoding technique.

Background Art

[0002] Currently, digital imaging devices that record moving images, such as digital video cameras, are widespread. In recent years, a method of recording RAW images has been applied not only to still images but also to moving images. Although the amount of data required for recording RAW images is enormous, it is considered that corrections and degradations to the original image can be minimized, and the degree of freedom in image editing is high even after recording.

[0003] When recording a RAW moving image on a recording medium, it is desirable to be able to record a moving image for a sufficient period of time. Therefore, it is desirable to compress and encode the RAW moving image before recording. Generally, a RAW image is an image in a Bayer array in which pixels of each of the R, G, and B colors are arranged in a mosaic pattern. Since adjacent pixels in the Bayer array have different color components, the correlation between adjacent pixels is low. Therefore, it is difficult to obtain a high compression efficiency even if encoded as it is. Therefore, only pixels of the same color component are extracted from the RAW image to generate a plurality of single-component planes. Then, by performing encoding for each plane, a plane conversion technique that increases the correlation between pixels within the plane and improves the compression efficiency is generally used as one of the compression encoding methods.

[0004] Another well-known conventional compression coding method is H.264 (H.264 / MPEG-4 Part 10: Advanced Video Coding). In this method, data size is compressed for each block of a predetermined number of pixels within a frame, utilizing the temporal and spatial redundancy inherent in moving images. H.264 achieves compression coding by combining techniques such as motion detection and motion compensation for temporal redundancy, discrete cosine transform (DCT) for frequency conversion for spatial redundancy, and further quantization and entropy coding. However, beyond a certain point, block distortion inherent in DCT conversion becomes noticeable, and subjective image degradation becomes apparent.

[0005] Therefore, subband coding technology, which uses Discrete Wavelet Transform (DWT) to decompose the signal into frequency bands called subbands by applying low-pass and high-pass filtering in the horizontal and vertical directions respectively, is employed in formats such as JPEG2000. Compared to coding technology using DCT, subband coding has the advantage of being less prone to block distortion and having better compression characteristics at high compression levels.

[0006] For example, the technology described in Patent Document 1 efficiently compresses RAW data by separating frequency components from a RAW image using plane transform and DWT transform to generate multiple subbands, and then quantizing and encoding each subband. In Patent Document 1, regarding the relative magnitudes of the quantization parameters set for each subband, considering the characteristics of human vision, the quantization parameters are set smaller for subbands closer to low-frequency components and larger for subbands closer to high-frequency components. Therefore, since the HL subband and LH subband at the same resolution level are in equivalent frequency ranges, the relative magnitudes of the quantization parameters will also be set to be equivalent, resulting in similar horizontal and vertical quantization distortions. [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2019-4428 [Overview of the project] [Problems that the invention aims to solve]

[0008] However, when encoding and displaying a RAW image with different optical compression ratios in the horizontal and vertical directions using Patent Document 1, the image is decompressed in the direction of optical compression, which increases the quantization distortion in that direction. In this case, if the optical compression is performed in only one direction, either horizontal or vertical, the resulting decompression direction is also limited to one direction, increasing the quantization distortion in that direction. This can lead to problems where the distortion is visible as image quality degradation during display.

[0009] In view of the above-mentioned problems, the present invention aims to provide an image encoding device that suppresses the increase in quantization distortion in either the horizontal or vertical direction when displaying an image encoded with different optical compression ratios in the horizontal and vertical directions. [Means for solving the problem]

[0010] To solve this problem, for example, the image encoding apparatus of the present invention has the following configuration. That is, An image encoding device for encoding image data obtained by an imaging means, An acquisition means for acquiring information representing the optical horizontal and vertical compression ratios of the imaging means, A DWT transformation means that performs a wavelet transform on the aforementioned image data to generate multiple subband data, A determination means for determining quantization parameters for conversion coefficients in multiple subbands obtained by the DWT conversion means, The system comprises an encoding means that quantizes and encodes the conversion coefficients in the subband data obtained by the DWT conversion means according to the quantization parameters determined by the determination means, The aforementioned determination means is When the optical horizontal and vertical compression ratios show a compression ratio such that the horizontal image relative to the vertical image in the image obtained by the imaging means is α (α < 1), the quantization parameters applied to the LH subband at the same resolution level are determined to be smaller than the quantization parameters applied to the HL subband. [Effects of the Invention]

[0011] According to the present invention, it is possible to generate encoded data suitable for images with different optical horizontal and vertical compression ratios. [Brief explanation of the drawing]

[0012] [Figure 1] A block diagram of an image encoding device according to the first embodiment. [Figure 2] A diagram illustrating the plane transformation in the first embodiment. [Figure 3] This study aims to create subbands when discrete wavelets (DWTs) are subjected to three vertical and three horizontal filtering steps. [Figure 4] A diagram showing the relationship between subblocks and block lines in the first embodiment. [Figure 5] A diagram showing the relationship between the optical horizontal and vertical compression ratios α according to the first embodiment. [Figure 6] A diagram showing the direction of the signal components of the plane data according to the second embodiment. [Figure 7] A diagram showing the decomposition of the oblique signal components in the U, V, GH plane data according to the second embodiment, into horizontal and vertical directions. [Figure 8] Block diagram of the image decoding device in the first embodiment. [Figure 9] A program configuration diagram of an information processing device in a modified example of the first embodiment. [Figure 10] A flowchart illustrating the image encoding processing procedure. [Figure 11] A flowchart illustrating the image decoding process.

Best Mode for Carrying Out the Invention

[0013] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential for the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0014] [First Embodiment] FIG. 1 is a block configuration diagram of a main part related to encoding of an imaging device 100 in the first embodiment.

[0015] The imaging device 100 includes a control unit 150 that controls each processing unit shown in FIG. 1. This control unit 150 is composed of a CPU, a ROM that holds programs executed by the CPU, and a RAM used as a work area.

[0016] The imaging device 100 can be realized as, for example, a digital camera or a digital video camera. In addition, it can also be realized as any information processing terminal or image processing device such as a personal computer, a mobile phone, a smartphone, a PDA, a tablet terminal, or a portable media player. Note that FIG. 1 shows a configuration including an imaging unit 101 in consideration of the case where it functions as a digital camera or the like which is an imaging device. However, the source of the image to be encoded is not limited to the imaging unit, and may be a storage medium storing the image to be encoded. It should be understood that the application to the imaging device in the embodiment is only for the purpose of embodying and facilitating understanding.

[0017] [Imaging Unit] The imaging unit 101 includes a lens optical system capable of optical zoom, including an optical lens, aperture, focus control, and lens drive unit, and an image sensor such as a CCD image sensor or CMOS image sensor that converts light information from the lens optical system into electrical signals. The imaging unit 101 outputs RAW image data, obtained by converting the electrical signals obtained by the image sensor into digital signals, to the plane conversion unit 102. The imaging unit 101 also has a non-volatile memory (not shown) that holds optical horizontal and vertical compression ratio information as metadata, and supplies this metadata to the quantization parameter weight setting unit 112. The imaging unit 101 is capable of capturing images at a rate of, for example, 30 frames per second. The optical lens may be configured so that the lens unit is detachable from the imaging device, and different types of lenses can be attached. If the optical lens is detachable, the configuration may allow not only a normal lens unit with an optical horizontal and vertical compression ratio of 1, but also anamorphic lenses with different optical horizontal and vertical compression ratios.

[0018] [Plane conversion section] The plane conversion unit 102 receives RAW image data of a frame of interest from the Bayer array captured by the imaging unit 101. The plane conversion unit 102 then separates the single RAW image data into multiple planes, each composed of a single component. Figure 2 shows the plane formation diagram when the input RAW image data of the Bayer array is separated into four planes. In the Bayer array, adjacent 2x2 pixels are composed of one red component (R), one blue component (B), and two green components (G1, G2). The plane conversion unit 102 separates the RAW image data into an R plane composed only of the R component, a G1 plane composed only of the G1 component, a G2 plane composed only of the G2 component, and a B plane composed only of the B component. If the horizontal number of pixels in the RAW image data is represented as W and the vertical number of pixels as H, then these four planes will have a size of horizontal W / 2 pixels and vertical H / 2 pixels. As a result of the above, when focusing on a plane of a single color component, the correlation between adjacent pixels is high, making it easy to improve compression efficiency.

[0019] [Discrete Wavelet Transform (DWT) section] The discrete wavelet transform unit 103 performs frequency transformation on the planes sequentially output from the plane transform unit 102 to generate transformation coefficients. DWT transformation is generally a transformation that filters the entire image. Vertical and horizontal filtering can be performed when the number of pixel data corresponding to the number of taps of the filter used has accumulated in the buffer memory. Therefore, by performing DWT transformation on each line of plane data and then recursively applying DWT to the generated low-frequency subband LL, it is possible to process each subband in parallel.

[0020] Figure 3 shows the subband formation diagram obtained when DWT is performed three times, with vertical and horizontal filtering processes forming one set. In the figure, "L" and "H" represent low and high frequencies, respectively, and the order indicates the bandwidth resulting from horizontal filtering on the front and the bandwidth resulting from vertical filtering on the back. The number after "Lv" indicates the resolution level of the DWT. When DWT is performed two or more times, the subband LL, which is the low-frequency band obtained in the previous transformation, becomes the target of the transformation. Therefore, as the number of DWT executions increases, the size of both the horizontal and vertical subbands of the immediately preceding transformation becomes halved. Also, for this reason, subband LL remains in the final DWT, so the resolution level is not indicated as shown in the figure. In this embodiment, the discrete wavelet transform unit 103 sequentially performs DWT on the four planes generated from the frame of interest (RAW image) to be encoded, but multiple units may be provided to shorten the processing time. For example, if two discrete wavelet transform units 103 are provided in parallel, the burden related to DWT can be halved compared to the case of one unit, and the time required for the transformation can also be halved. Furthermore, if four discrete wavelet transform units 103 are provided in parallel, it is possible to reduce the time required for DWT to one-quarter compared to the case of one unit.

[0021] In this embodiment, the discrete wavelet transform unit 103 is described as sequentially outputting the conversion coefficients for one line of each subband to the quantization unit 104 each time a conversion coefficient for one line of each subband is generated in each subband.

[0022] [Quantization section] The quantization unit 104 quantizes the conversion coefficients input from the discrete wavelet transform unit 103, one coefficient at a time, using the quantization parameter Qp generated by the quantization control unit 106. The larger the value of the quantization parameter Qp, the smaller the value after quantization, allowing for a reduction in the code amount, but this parameter also leads to a significant degradation of image quality. Furthermore, the quantization of the conversion coefficients for the four planes may be performed plane by plane or in parallel for all planes. However, in this embodiment, the quantization unit 104 quantizes the conversion coefficients at the same subband and position in each plane using a common quantization parameter Qp, and supplies the quantization results to the encoding unit 105.

[0023] [Encoding section] The encoding unit 105 entropy encodes the transformation coefficients of each plane after quantization by the quantization unit 104 to generate encoded data, and outputs it.

[0024] [Recording Processing Unit] The recording processing unit 107 takes the encoded data output from the encoding unit 105, formats it in a predetermined recording format, and records it on the recording medium 108 as a file with the information necessary for decoding in its header. The recording processing unit 107 also stores metadata (compression ratio) obtained from the imaging unit 101 in this header. This is so that the image decoding device can interpolate the decoded image horizontally or vertically to generate a normal image. The recording processing unit 107 also stores information related to the initial values ​​set by the initial target code amount setting unit 114 and the initial quantization value setting unit 115 in this header. However, if the initial values ​​set by the initial target code amount setting unit 114 and the initial quantization value setting unit 115 are standardized between the encoding device and the decoding device, it is not necessary to include this information in the file header.

[0025] [Recording medium] The recording medium 108 is a recording medium composed of, for example, non-volatile memory, and is configured to be detachable from the imaging device 100.

[0026] [Initial settings] The initial target code amount setting unit 114 sets the target code amount at the start of encoding the frame of interest (the RAW image of interest). The initial quantization value setting unit 115 sets the quantization parameter Qp at the start of encoding the frame of interest. Generally, the various setting values ​​at the start of encoding are calculated by feedback control based on the encoding information of the previous plane.

[0027] [Quantization Control Unit] Next, the quantization control unit 106 will be described. The quantization control unit 106 controls the quantization parameter Qp so that the generated code amount of the frame of interest converges to the target code amount of the frame of interest. In this embodiment, the RAW image of the frame of interest is separated into four planes, a discrete wavelet transform is performed on each, and quantization and encoding are performed. The quantization parameter Qp used when quantizing the four planes is common if the subband types are the same, and the update position of the quantization parameter Qp is also the same. This is because the quantization parameter Qp used in common is updated based on the code amount of the frame of interest (for all four planes).

[0028] Figure 4 shows the unit of quantization control. The control unit of the quantization parameter Qp is explained using Figure 4. As mentioned above, the input image to be encoded is a RAW image of a Bayer array, and is separated into 4 planes by the plane transformation unit 102. Furthermore, DWT is applied to each plane. As mentioned above, the encoding unit is a line for each subband, but the quantization control unit is a collection of the encoding results of each subband at the same pixel position. That is, as shown in Figure 4, one line of subbands {Lv3HL, Lv3LH, Lv3HH} and subband {LL} at resolution level 3, two lines of subbands {Lv2HL, Lv2LH, Lv2HH} at resolution level 2, and four lines of subbands {Lv1HL, Lv1LH, Lv1HH} at resolution level 1 are used as a control unit by one Qp. In other words, this quantization control unit corresponds to the data of one color component in one line of a RAW image obtained by imaging real space. Hereafter, the set of corresponding transformation coefficients within each subband, which is this control unit, will be called a "block line".

[0029] The generated code amount holding unit 109 receives and stores the code amount of the encoded data when each block line is encoded, as notified by the encoding unit 105. The target code amount calculation unit 113 calculates the target code amount of a single block line based on the target code amount in the frame of interest and the total number of block lines.

[0030] The difference calculation unit 110 calculates the difference between the generated code amount and the target code amount for each block line, and further calculates the integrated difference amount, which is the cumulative value of the differences.

[0031] The quantization value calculation unit 111 calculates (updates) the quantization parameter Qp of the i-th block line of interest based on the accumulated difference amount D(i-1) notified by the difference calculation unit 110. Details will be described later.

[0032] [Calculation of quantized value] One known method for calculating quantization parameters is described in MPEG2 Test Model 5. According to Test Model 5, the quantization parameter Qp[i] of the pixel block of interest is calculated using the following equation (1), based on the initial quantization parameter Qini and ΣE(i-1), which represents the product of the difference between the code amount in each pixel block from the first pixel block to the immediately preceding i-1th pixel block and the target code amount per pixel block, assuming the pixel block of interest is the i-th block. Qp[i] = Qini + r ×ΣE[i-1] …(1) Here, r represents the control sensitivity of the quantization parameter. The larger the control sensitivity r, the more Qp[i] fluctuates depending on ΣE[i-1], improving the controllability of the code quantity, but also increasing the change in image quality. On the other hand, the smaller the control sensitivity r, the less Qp[i] depends on ΣE[i-1], and its fluctuation becomes smaller, reducing the change in image quality, but decreasing the controllability of the code quantity.

[0033] In this embodiment, a RAW image of one frame of a Bayer array is separated into R, G1, G2, and B planes. Values ​​0, 1, 2, and 3 are assigned to each plane to identify it, and these values ​​are represented as the variable pl. The i-th block line of each plane pl is represented as BL(pl,i). The code amount of the encoded data generated when the block line BL(pl,i) is encoded is defined as C(BL(pl,i)), and the target code amount of one block line is represented as TC.

[0034] At this time, the difference calculation unit 110 calculates the integrated value ΣE[i-1] of the difference between the generated code amount and the target code amount from the first block line to the i-1 block line immediately preceding the i-th block line of interest, according to the following equation (2). ΣE[i-1]=ΣΣ{TC-C(BL(pl,k))} …(2) Here, ΣΣ represents the sum of pl=0,1,2,3 and k=0,1,2,...,i-1.

[0035] The quantization value calculation unit 111 applies the integrated value ΣE[i-1] obtained in equation (2) above to equation (1) shown earlier to calculate the quantization parameter Qp of the i-th block line of interest. Then, the quantization value calculation unit 111 converts this quantization parameter Qp into the actual quantization parameters Qp[pl][sb] for each subband and notifies the quantization unit 104. Note that pl and sb indicate the type and resolution level of the corresponding plane and subband, respectively.

[0036] The method for calculating the quantization parameters of each plane and subband by the quantization value calculation unit 111 is described below. As shown in equation (3), the quantization value calculation unit 111 calculates Qp[pl][sb] by multiplying the matrix mtx that each plane and subband has and the optical horizontal and vertical compression ratio α set by the quantization parameter weight setting unit 112 by the quantization parameter Qp calculated in equation (1). Note that the optical horizontal and vertical compression ratio α set by the quantization parameter weight setting unit 112 is set using metadata recorded by the imaging unit 101. Qp[pl][sb] = Qp[i] × mtx[pl][sb] × α …(3) Furthermore, Qp[pl][sb] may be prepared as a preset value in advance instead of being calculated using equation (3), and the preset value may be switched according to the optical horizontal and vertical compression ratio. Generally, the mtx setting is configured to control the amount of code by setting Qp to be larger for high-frequency subbands and smaller for low-frequency subbands. This compresses the amount of code generated for high-frequency components of image data, which are difficult to perceive due to human visual characteristics, and improves coding efficiency. Therefore, the matrix mtx is set so that the quantization parameter Qp is larger for high-frequency subbands and smaller for low-frequency subbands. In addition, the matrix mtx is set so that the quantization parameter is the same for HL subbands and LH subbands of the same resolution level. In this embodiment, the matrix mtx is assumed to be set to 3LL:3HL:3LH:3HH:2HL:2LH:2HH:1HL:1LH:1HH=1:2:2:4:4:4:8:8:8:16.

[0037] Here, the optical horizontal and vertical compression ratio α will be explained with reference to Figures 5(a) to (c).

[0038] Figure 5(a) shows the image captured when the imaging unit 101 captures a subject using a lens with a 1:1 compression ratio, where the compression ratio is equal in the horizontal and vertical directions. When the subject is a circle, the horizontal length and vertical length of the subject have a 1:1 relationship, and similarly in the captured image, the horizontal length and vertical length also have a 1:1 relationship. Therefore, in the example of Figure 5(a), the optical compression ratio α in the horizontal and vertical directions is 1.

[0039] Next, we will explain Figure 5(b). Figure 5(b) shows the image captured when the imaging unit 101 captures a subject using a 2:1 lens.

[0040] The relationship between the horizontal and vertical lengths of the subject is 1:1, as shown in Figure 5(a). However, the relationship between the horizontal and vertical lengths in the captured image is 1 / 2:1 because the image is captured using a lens that optically compresses the horizontal dimension by half. Therefore, when the vertical dimension is set to 1 (reference), the horizontal length becomes 1 / 2, resulting in a horizontal optical compression ratio α of 1 / 2 and a vertical optical compression ratio α of 1.

[0041] Finally, let's explain Figure 5(c). Figure 5(c) shows the image captured when the imaging unit 101 captures an image of a subject using a 1:2 lens. The relationship between the horizontal length and the vertical length of the subject is 1:1, as in Figures 5(a) and (b). However, the relationship between the horizontal length and the vertical length in the captured image is 1:1 / 2 because imaging is performed using a lens that optically compresses the image vertically. Therefore, when the horizontal length is set to 1 (reference), the vertical length is 1 / 2, so the optical compression ratio α in the vertical direction is 1 / 2, and the optical compression ratio α in the horizontal direction is 1.

[0042] In this embodiment, α is obtained from metadata acquired by the quantization parameter weight setting unit 112 through communication with the lens unit attached to the imaging unit 101. However, it may also be obtained through user settings. For example, the user may select information on the compression ratios of the lenses in the horizontal and vertical directions via an operation unit and display unit (not shown), and the control unit 150 may acquire the compression ratios of the lenses selected by the user. Alternatively, it may be obtained from information such as whether the lenses have equal optical horizontal and vertical compression ratios or whether lenses with different compression ratios are attached.

[0043] Next, a specific example of the calculation process for quantization parameters calculated by the quantization value calculation unit 111, based on the optical horizontal and vertical compression ratios α set by the quantization parameter weight setting unit 112 for each subband in equation (3), will be explained again with reference to Figures 5(a) to (c).

[0044] First, let's explain the case where the compression ratio is equal in the optically horizontal and vertical directions, as shown in Figure 5(a). When encoding is applied to the image captured in Figure 5(a) and the encoded image is displayed, there is no decompression in either the horizontal or vertical direction, so there is no increase in unidirectional quantization distortion associated with the display. Therefore, when the quantization value calculation unit 111 calculates the quantization parameter Qp for each subband using equation (3), it calculates with α=1 for all subbands, including the HL and LH subbands, that is, without changing the weight for each subband. Thus, the relationship shown in equation (4) below is applied as the value of α when calculating the quantization parameter Qp for the HL and LH subbands. HL:LH=1:1 …(4)

[0045] Therefore, the ratio of the quantization parameters for each subband is 3LL:3HL:3LH:3HH:2HL:2LH:2HH:1HL:1LH:1HH=1:2:2:4:4:4:8:8:8:16.

[0046] Next, we will explain the case using a 2:1 lens as shown in Figure 5(b). In the case of Figure 5(b), The horizontal direction is compressed to half its size relative to the vertical direction when forming an image. Therefore, if encoding is performed with the same quantization parameter weighting settings for the sunband as in Figure 5(a), the quantization distortion will be similar in both the horizontal and vertical directions during encoding. However, when displaying the image, the horizontal direction is stretched by a factor of two, increasing the horizontal quantization distortion, which is then visible as a degradation in image quality. Therefore, it is desirable to suppress the increase in horizontal quantization distortion by making the quantization parameter Qp of the HL subband, which is related to the horizontal image quality, smaller compared to the other subbands. Accordingly, when the quantization value calculation unit 111 calculates the quantization parameter Qp for each subband using equation (3), if sb is the HL subband, the optical compression ratio α in the horizontal direction is set to 1 / 2 to calculate the quantization parameter. If sb is the LH subband, the optical compression ratio α in the vertical direction is set to 1 to calculate the quantization parameter Qp. Therefore, the relationship shown in equation (5) below is applied as the value of α when calculating the quantization parameter Qp for the HL and LH subbands. HL:LH=1 / 2:1…(5)

[0047] Thus, in the case of a lens where the image is compressed more horizontally than vertically, if the horizontal compression ratio to the vertical is A (A<1), then the ratio of the quantization parameters of each subband is 3LL:3HL:3LH:3HH:2HL:2LH:2HH:1HL:1LH:1HH=1:2×A:2:4:4×A:4:8:8×A:8:16. In the case of the 2:1 lens in Figure 5(b), the image is formed by compressing the image to half its size horizontally relative to the vertical, so the horizontal compression ratio to the vertical is A=1 / 2. Therefore, the ratio of the quantization parameters of each subband is 3LL:3HL:3LH:3HH:2HL:2LH:2HH:1HL:1LH:1HH=1:2×1 / 2:2:4:4×1 / 2:4:8:8×1 / 2:8:16. While we have discussed a 2:1 lens, in the case of a 1.33:1 lens (a lens that compresses the image horizontally to 1 / 1.33 relative to the vertical), A = 1 / 1.33, and the quantization parameters are determined using the weight coefficient of A. Alternatively, instead of obtaining information about the lens's compression ratio from the lens itself, the user could select the horizontal and vertical compression ratios of the lens (of the captured image), and A, which is the horizontal compression ratio relative to the vertical, could be determined from the compression ratios selected by the user.

[0048] Finally, let's explain the case using a 1:2 lens as shown in Figure 5(c). In the case of Figure 5(c), if the same quantization parameter weighting is set for the sunband as in Figure 5(a), the quantization distortion will be similar in the horizontal and vertical directions during encoding. However, when displaying, the vertical direction is stretched by a factor of two, increasing the quantization distortion in the vertical direction, and this distortion is visible as a degradation of image quality. Therefore, it is desirable to suppress the increase in vertical quantization distortion by making the quantization parameter Qp of the LH subband, which is related to vertical image quality, smaller compared to the other subbands. Accordingly, when the quantization value calculation unit 111 calculates the quantization parameter Qp for each subband using equation (3), if sb indicates the LH subband, the optical compression ratio α in the vertical direction is set to 1 / 2 to calculate the quantization parameter Qp. If sb is the HL subband, the optical compression ratio α in the horizontal direction is set to 1 to calculate the quantization parameter Qp. Therefore, the relationship shown in equation (6) below is applied as the value of α when calculating the quantization parameter Qp for the HL and LH subbands. HL:LH = 1:1 / 2…(6)

[0049] For lenses where the image is compressed more in the vertical direction than in the horizontal direction, if the compression ratio in the vertical direction relative to the horizontal direction is B (B<1), then the ratio of the quantization parameters of each subband is 3LL:3HL:3LH:3HH:2HL:2LH:2HH:1HL:1LH:1HH=1:2:2×B:4:4:4×B:8:8:8×B:16. In the case of the 1:2 lens in Figure 5(c), the image is compressed to 1 / 2 in the vertical direction relative to the horizontal direction before imaging, so the ratio of the vertical direction relative to the horizontal direction is B=1 / 2. Therefore, the ratio of the quantization parameters of each subband is 3LL:3HL:3LH:3HH:2HL:2LH:2HH:1HL:1LH:1HH=1:2:2×1 / 2:4:4:4×1 / 2:8:8:8×1 / 2:16. While we have discussed 1:2 lenses, in the case of a 1:1.5 lens (a lens that compresses the image to 1 / 1.5 in the vertical direction relative to the horizontal direction before forming an image), the quantization parameters are determined by setting the vertical compression ratio B = 1 / 1.5 relative to the horizontal direction. Alternatively, instead of obtaining compression ratio information from the lens, the user can select the horizontal and vertical compression ratios (of the captured image) of the lens, and the vertical compression ratio B relative to the horizontal direction can be determined from the compression ratios selected by the user.

[0050] Thus, depending on whether the data is compressed horizontally (Figure 5b) or vertically (Figure c), the subbands to which each subband's quantization parameters are weighted switch. When the data is compressed horizontally (Figure 5b), the HL subband is weighted by a horizontal compression ratio A relative to the vertical, so that the quantization parameter of the HL subband becomes smaller. When the data is compressed vertically (Figure 5c), the subbands are weighted by a vertical compression ratio B relative to the horizontal, so that the quantization parameter of the LH subband becomes smaller.

[0051] As described above, by changing the value of α set in the quantization parameter weight setting unit 112 for each subband according to the optical horizontal and vertical compression ratio, the increase in unidirectional quantization distortion that occurs during display can be suppressed. Note that when sb is the HH subband, unlike the HL and LH subbands, the quantization parameter Qp is calculated without changing the weight of the quantization parameter according to the optical horizontal and vertical compression ratio (α=1). The method of setting the quantization parameter weight for each subband has been explained using examples with 2:1 and 1:2 lenses, but the optical compression ratio of the horizontal and vertical is not limited to these. Also, the optical compression ratio α does not have to be the ratio of the horizontal and vertical, but a value close to the ratio of the optical horizontal and vertical may be applied.

[0052] [Image decoding device] Next, the image decoding device in the embodiment will be described.

[0053] Figure 8 is a block diagram of the main components of the image decoding device 800, which decodes encoded image data generated by the image encoding device (Figure 1) described above. Since encoding and decoding are two sides of the same coin, the configuration of the image decoding device and its processing details will be briefly explained below with reference to the same figure.

[0054] The image decoding device 800 has a control unit 850 that controls the entire device. This control unit 850 consists of a CPU, a ROM that holds the program executed by the CPU, and RAM used as a work area.

[0055] The header analysis unit 801 analyzes the file header of the encoded image data to be decoded and obtains the information necessary for decoding. This information includes the initial quantization value (corresponding to the initial quantization value setting unit 115 in Figure 1), the initial target code amount (corresponding to the initial target code amount setting unit 114 in Figure 1), and information representing the optical horizontal and vertical compression ratios, as well as other information necessary for decoding. Note that if the initial quantization value and initial target code amount are standardized between the encoding device and the decoding device, it is not necessary to include this information in the file header.

[0056] The quantization control unit 810, comprising the quantization parameter weight setting unit 811, the target code amount calculation unit 812, the quantization value calculation unit 813, the code amount holding unit 814, and the difference calculation unit 815, should be understood to have substantially the same functions as the configurations with the same names shown in Figure 1.

[0057] The header analysis unit 801 supplies the initial quantization values ​​obtained from the analysis to the quantization control unit quantization value calculation unit 812, the initial target code amount to the target code amount calculation unit 113, and information representing the optical horizontal and vertical compression ratios to the quantization parameter weight setting unit 811 and the output unit 807.

[0058] The decoding unit 802 decodes the encoded data following the header to obtain the quantization coefficients of the block lines in the subbands of each plane. The decoding unit 802 supplies the quantization coefficients of all planes obtained in the decoding process to the inverse quantization unit 803. Since the decoding process determines the amount of encoded data for each block line of each plane, the decoding unit 802 supplies information representing the code amount to the code amount holding unit 814 each time the code amount of a block line is determined.

[0059] As a result, during the process of decoding the blocklines, the quantization value calculation unit 813 can determine the quantization parameters of the next blockline according to equations (1) to (3) used by the encoding side as shown above.

[0060] The inverse quantization unit 803 inversely quantizes each plane, block line by block, according to the quantization parameters set by the quantization control unit 810, and obtains the DWT conversion coefficients. Naturally, the quantization parameters set in the inverse quantization unit 803 are the same as the quantization parameters used in the image encoding device. The inverse quantization unit 803 supplies the conversion coefficients after inverse quantization to the inverse DWT unit 804.

[0061] The inverse DWT unit 804 performs an inverse DWT (inverse wavelet transform) on the transformation coefficients input from the inverse quantization unit 803 and outputs the result to the memory 806. As a result, four planes are constructed in the memory 806 as the decoding process progresses.

[0062] The plane integration unit 806 performs integration processing of the four planes, generates a RAW image of the Bayer array, and supplies it to the output unit 807.

[0063] The output unit 807, for example, processes the input RAW image and generates a normal color image in which each pixel consists of three components: RGB. Then, the output unit 807 interpolates the generated color image horizontally and vertically according to the information indicating the horizontal and vertical compression ratio supplied by the header analysis unit 801, and outputs it to an external display device. However, if the horizontal and vertical compression ratio is 1:1, the output unit 807 does not perform interpolation. If the output destination of the output unit 807 is a printing device, the output unit 807 generates an image with color components such as YMCK and then performs horizontal and vertical interpolation.

[0064] During the above decoding process, while the decoding process is being performed on a block-line basis, the quantization value calculation unit 813 of the quantization control unit 810 updates the quantization parameters for performing inverse quantization for each plane's block line. The process for updating these quantization parameters is the same as that of the quantization value calculation unit 111 in the imaging device 100 in Figure 1, so a detailed explanation is omitted here.

[0065] [Modified version of the first embodiment] An example of implementing the process corresponding to the first embodiment described above by an application executed by an information processing device, such as a personal computer, will be described as a modification of the first embodiment.

[0066] Figure 9 is a block diagram of the information processing device. When the power to this device is turned ON, the CPU 901 executes the boot program stored in the ROM 902, performs hardware initialization, loads the OS (Operating System) stored in the HDD (Hard Disk) 904 into the RAM 903, and transfers control to the OS (executes it), thereby enabling the device to function as an information processing device. In other words, the display unit 906, keyboard 907, and mouse 908 function as interfaces with the user. The I / F 905 is an interface for communicating with external devices, typically a network interface, USB (Universal Serial Bus), etc. The RAW image data to be encoded in this embodiment is also input from an external source via this I / F 905. In the above configuration, when the user operates the keyboard 907 or mouse 908 to input a predetermined instruction, the CPU 901 loads an encoding application or a decoding application from the HDD 904 into the RAM 903 and executes it, enabling the device to function as an image encoding device or an image decoding device.

[0067] The following describes the processing when this device functions as an image encoding device, referring to the flowchart in Figure 10. Basically, when the CPU 901 functions as an image encoding device by executing a program, the CPU 901 performs processing corresponding to each processing unit in Figure 1, so please refer to the first embodiment for details.

[0068] Furthermore, the following explanation assumes that the HDD904 already contains RAW image data of the Bayer array to be encoded as a file. It also assumes that the header of that file contains information indicating the optical horizontal and vertical compression ratios at the time of capture.

[0069] In S101, CPU901 obtains information indicating the optical horizontal and vertical compression ratios from the RAW image file to be encoded. This information may also be entered by the user via the control panel.

[0070] Then, in S102, CPU901 takes RAW image data from the relevant file and loads it into RAM903.

[0071] In S103, the CPU 901 generates four planes, R, G1, G2, and B, from the RAW image data expanded in RAM 903 and stores them in RAM 903.

[0072] In S104, CPU901 initializes various parameters prior to encoding. This initialization process includes setting initial quantization values, initial target code amounts, etc. This initialization process also includes zeroing out the area that stores the cumulative sum of the code amounts of the block lines and the target block lines.

[0073] In S105, CPU901 performs DWT on each of the four planes. CPU901 stores the subbands obtained from this process in RAM903. The presence of four boxes in S105 in the flowchart of Figure 10 indicates that this process is performed on all four planes.

[0074] In S106, the CPU 901 sets the initial value of the variable i, which defines the order of the block lines, to "1". Then, in S107, the CPU 901 reads the DWT coefficients of the i-th block line to be encoded for each plane from the subbands obtained from the DWT stored in RAM 903 and quantizes them.

[0075] In S108, the CPU 901 encodes the quantization coefficients of the i-th block line of each plane and temporarily stores the generated encoded data in the RAM 903. At this time, the CPU 901 calculates the difference between the amount of encoded data of the i-th block line of each plane and the target block line, and accumulates and adds these differences (this is the process corresponding to equation (2)).

[0076] In S109, CPU901 updates the quantization parameters for the next block line to be encoded (this corresponds to the process in equation (3)).

[0077] Then, in S110, CPU901 determines whether all block lines have been encoded based on the value of variable i. If there are any unencoded block lines, CPU901 proceeds to S111 and increments variable i by 1. Then, CPU109 returns to S107 to encode the next block line.

[0078] On the other hand, once the encoding of all block lines is complete, the CPU 901 proceeds from S110 to S112. In S112, the CPU 901 creates a file header in the HDD 904 containing various information necessary for decoding (including information indicating the horizontal and vertical compression ratios). Then, following the file header, the CPU 901 outputs the encoded data stored in the RAM 903 in a pre-configured format, thereby creating an encoded image data file.

[0079] Next, the processing when this device functions as an image decoding device will be explained with reference to the flowchart in Figure 11. Basically, when the CPU 901 functions as an image encoding device by executing a decoding program, the CPU 901 performs the processing corresponding to each processing unit in Figure 8, so please refer to the first embodiment for details.

[0080] Furthermore, the following explanation describes the process from the point where an encoded image file is already stored in HDD904, through the decoding process, to the display unit 906 displaying the image.

[0081] In S201, CPU901 analyzes the file header of the encoded image file to be decoded and obtains information indicating the optical horizontal and vertical compression ratios. This information may also be entered by the user via the control panel.

[0082] Then, in S202, the CPU 901 initializes various parameters prior to decoding. This initialization process includes setting the initial quantization value, initial target code amount, etc. This initialization process also includes zeroing out the area in RAM 903 that stores the cumulative sum of the code amounts of the block lines and the target block lines. In the next S203, the CPU sets the initial value of the variable i, which defines the order of the block lines, to "1".

[0083] In S204, CPU901 decodes the i-th block line of each plane and obtains the quantization conversion coefficients. At this time, the code value of the i-th block line is determined, so CPU901 calculates the difference between that code value and the code value of the target block line and updates the value in the region shown earlier by adding it (this is the process corresponding to equation (2)).

[0084] In S205, CPU901 performs inverse quantization on the quantization coefficient of the i-th block line of each plane. If the variable i is "1", quantization is performed according to the quantization parameters obtained during the initialization process. If the variable i is anything other than "1", inverse quantization is performed on the previous block line using the updated quantization parameters obtained during the decoding process.

[0085] In S206, CPU901 performs inverse DWT on the transformation coefficients obtained from inverse quantization to obtain an image of block lines. CPU901 stores the obtained block line image data in RAM903. Note that inverse DWT requires that the transformation coefficients of the number of taps of the filters used are the same; if this requirement is not met, inverse DWT is not performed.

[0086] In S207, CPU901 updates the quantization parameters in preparation for decoding the next blockline, based on the cumulative sum of the difference between the code amount of the blockline and the code amount of the target blockline (this corresponds to the process in equation (3)). In other words, the quantization parameters for each subband are determined according to the optical horizontal and vertical compression ratios.

[0087] In S208, CPU901 determines whether it has finished decoding all blocklines. If there are undecoded blocklines, CPU901 proceeds to S209 and increments the variable i by "1". Then, CPU901 returns to S204 to decode the next blockline.

[0088] On the other hand, once all block lines have been decoded, CPU901 proceeds to process S208 through S211. In S210, CPU901 integrates the four planes reconstructed in RAM903 to generate a RAW image of the Bayer sequence. Then, CPU901 proceeds to process S211.

[0089] In S211, the CPU 901 processes the generated RAW image to produce a color image in which each pixel is composed of R, G, and B components. Then, the CPU 901 scales the generated color image horizontally or vertically according to the horizontal and vertical compression ratios obtained in S201 to produce an image with a horizontal and vertical scale of 1:1, which is output to the display unit 906.

[0090] As explained above, the same processing as the first embodiment described earlier can also be achieved by a computer program.

[0091] [Second Embodiment] Next, a second embodiment will be described. The apparatus configuration in this second embodiment is the same as that in Figure 1 of the first embodiment.

[0092] However, in this second embodiment, the plane conversion unit 102 converts the RAW image data into one luminance plane (Y plane) and three non-luminance planes. Furthermore, while in the first embodiment the quantization parameter weights for each subband in each plane were determined using only the optical horizontal and vertical compression ratios, in this second embodiment, an example is described in which the quantization parameters are determined by also taking into account the optical compression ratio and the characteristics of the signal component direction of each plane.

[0093] In this second embodiment, the plane conversion unit 102 converts the RAW image data into three planes other than luminance (U, V, GH planes) and the luminance plane (Y plane) shown in equation (10) according to the following equations (7) to (9). U = B - (G1 + G2) / 2 …(7) V = R - (G1 + G2) / 2 …(8) GH = G1 - G2 …(9) Y = (R+B+G1+G2) / 4 …(10)

[0094] In this second embodiment, each plane was transformed as described above as an example of plane transformation, but the transformation method is not limited to this.

[0095] Figures 6(a) to (d) show the direction of the signal components in each plane. Figures 6(a) to (d) focus on adjacent 2x2 pixels (R, G1, G2, B) in the Bayer array shown in Figure 2.

[0096] First, the U plane is the difference between the summation average of the G1 and G2 components and the B component, according to equation (7). Therefore, the direction represented by the signal component U is the diagonal direction drawn by a solid line connecting point P', which is the centroid of the B component, and point Q', which is the centroid of the signal obtained by summation average of the G1 and G2 components, as shown in Figure 6(a).

[0097] The V-plane, according to equation (8), is the difference between the summation average of the G1 and G2 components and the R component. The direction represented by the signal component V is the diagonal direction drawn by a solid line connecting point S', which is the centroid of the R component, and point T', which is the centroid of the signal obtained by summation average of the G1 and G2 components, as shown in Figure 6(b).

[0098] The GH plane is the difference between the G1 component and the G2 component, according to equation (9). Therefore, the direction represented by the signal component GH is the diagonal direction drawn by the solid line connecting the centroid position V' of the G1 component and the centroid position W' of the G2 component, as shown in Figure 6(c).

[0099] Finally, the Y plane is the summation average of the R, G1, G2, and B components, as shown in equation (10). Therefore, the centroid of the signal component Y is point Z', which is the center of each component, as shown in Figure 6(d). The signal component Y has no direction and is represented as a point.

[0100] In this second embodiment, the weight setting method for the quantization parameter weight setting unit 112 may be calculated according to the optical horizontal and vertical compression ratio α, as described in the first embodiment. However, considering that some of the signal components after plane transformation are oblique, setting the weights after decomposing them into horizontal and vertical directions allows for encoding that better considers the degree of influence of quantization distortion in the horizontal and vertical directions due to decomposition during display.

[0101] Figures 7(a) to (c) show the decomposition of the diagonal signal components in the U, V, and GH planes into horizontal and vertical directions. In Figure 7(a), point R' is the intersection point with the horizontal axis when a line is drawn perpendicularly from point P' in the horizontal direction relative to Figure 6(a). The line segment connecting two points p1 and p2 is denoted as "L(p1,p2)". Also, the angle ∠S'T'U' is set to θ, and the length of L(P'Q') is set to 1 as the base. In this case, the ratio of the lengths of line segments L(Q'R'), L(R'P'), and L(P'Q') is given by the following equation (11). L(Q'R') : L(R'P') : L(P'Q') = cosθ : sinθ : 1 …(11)

[0102] Next, Figure 7(b) corresponds to Figure 6(b). Let U' be the point where a line drawn perpendicular to the horizontal axis from point S' intersects with the horizontal axis. Let θ be the angle ∠S'T'U', and if we set the length of line segment L(S'T') to 1 as the base, then the ratio of the lengths of line segments L(T'U'), L(U'S'), and L(S'T') is given by equation (12). L(T'U') : L(U'S') : L(S'T') = cosθ : sinθ : 1 …(12)

[0103] Finally, Figure 7(c) corresponds to Figure 6(c). Let X' be the intersection point when a line is drawn perpendicular to the horizontal axis from point V' and a line is drawn perpendicular to the vertical axis from point W'. Let θ be the angle ∠V'W'X' for each line segment L(V'W'), and let the length of line segment L(W'X'), L(X'V'), and L(V'W') be set to 1. Then the ratio of the lengths of line segments L(W'X'), L(X'V'), and L(V'W') is given by equation (13). L(W'X') : L(X'V') : L(V'W') = cosθ : sinθ : 1 …(13)

[0104] Here, with reference to Figures 5(a) to (c), the method for setting α in the quantization parameter weight setting unit 112 for each plane and subband will be explained. First, in Figure 5(a), since the relationship between the horizontal and vertical lengths of the image to be encoded is one to one, cosθ and sinθ are 1 / √2 and 1 / √2, respectively, and equations (11) to (13) are expressed by the following equation (14). L(Q'R') : L(R'P') : L(P'Q') = L(T'U') : L(U'S') : L(S'T') = L(W'X') : L(X'V') : L(V'W') = 1 / √2:1 / √2:1 … (14)

[0105] Since the ratio of the lengths of the line segments in equation (14) is equivalent to the α set for each subband, equation (15) below is applied as the value of α for the HL, LH, and HH subbands in the U, V, and GH planes. HL:LH:HH=1 / √2:1 / √2:1 … (15) Furthermore, since the signal components in the Y plane do not have a specific direction, the following relationship (16) is applied to the α values ​​of the HL, LH, and HH subbands. HL:LH:HH=1:1:1 … (16)

[0106] Next, let's explain the case in Figure 5(b). In this case, the relationship between the horizontal and vertical lengths is 1 / 2 to 1, so cosθ and sinθ are 1 / √5 and 2 / √5, respectively. Therefore, equation (12) becomes equation (17). L(Q'R') : L(R'P') : L(P'Q') = L(T'U') : L(U'S') : L(S'T') = L(W'X') : L(X'V') : L(V'W') = 1 / √5:2 / √5:1 … (17)

[0107] Therefore, the following relationship (18) is applied to the values ​​of α in the HL, LH, and HH subbands in the U, V, and GH planes. HL:LH:HH=1 / √5:2 / √5:1 …(18) Furthermore, the following relationship (19) is applied to the α values ​​of the HL, LH, and HH subbands in the Y plane. HL:LH:HH=1 / 2:1:1…(19)

[0108] Finally, let's explain the case shown in Figure 5(c). In this case, the relationship between the horizontal length and the vertical length is 1:1 / 2, so cosθ and sinθ are 2 / √5 and 1 / √5, respectively. Therefore, equation (13) becomes equation (20). L(Q'R') : L(R'P') : L(P'Q') = L(T'U') : L(U'S') : L(S'T') = L(W'X') : L(X'V') : L(V'W') = 2 / √5:1 / √5:1 … (20)

[0109] Therefore, the following relationship (21) is applied to the values ​​of α in the HL, LH, and HH subbands in the U, V, and GH planes. HL:LH:HH=2 / √5:1 / √5:1…(21) Furthermore, the following relationship (22) is applied to the values ​​of α for the HL, LH, and HH subbands in the Y plane. HL:LH:HH = 1:1 / 2:1…(22)

[0110] As explained above, according to this second embodiment, by setting the weights of each plane and subband by taking into account not only the optical horizontal and vertical compression ratios but also the direction of the signal components of each plane, it becomes possible to perform encoding that takes into account the degree of influence of quantization distortion in the horizontal or vertical direction due to decompression during display.

[0111] In the first embodiment, the quantization parameter values ​​were assumed to be the same across planes for subbands of the same type and block lines at the same position. This is because the four planes in the first embodiment differ only in their color components and therefore have the same weights. In contrast, the second embodiment encodes the luminance Y plane and three chrominance planes, and the luminance plane is set to have a quantization parameter that is sufficiently smaller than that of the other chrominance planes.

[0112] Furthermore, the process corresponding to the second embodiment described above may be implemented by a computer executing a program, similar to the modification of the first embodiment.

[0113] Furthermore, although the above embodiment describes examples where the horizontal and vertical compression ratios are 2:1 or 1:2, other ratios are also acceptable. In that case, the quantization parameters set for the HL and LH subbands at the same resolution level should be set according to that ratio.

[0114] In the first embodiment, its modified form, and the second embodiment described above, the image to be encoded was a RAW image in Bayer array format, but other image formats may also be used. For example, when encoding a monochrome image, the plane conversion unit 102 is unnecessary because the image is originally composed of only a single luminance component. It may also be applied to color images represented by luminance and chrominance components such as YCbCr. In this case, the same processing as described above should be performed on the Y plane, Cb plane, and Cr plane.

[0115] Furthermore, in the above embodiment, a Bayer array in which 2x2 pixels are arranged in raster scan order as R, G1, G2, B was used as an example. However, in the case of Bayer arrays other than this arrangement, the quantization parameters should be determined according to that arrangement.

[0116] (Other examples) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0117] The disclosures herein include the following imaging devices, methods, and programs. (Item 1) An image encoding device for encoding image data obtained by an imaging means, An acquisition means for acquiring information representing the optical horizontal and vertical compression ratios when taking photographs, A transformation means that performs a wavelet transform on the aforementioned image data to generate multiple subband data, A determination means for determining quantization parameters for the conversion coefficients in multiple subbands obtained by the conversion means, The system comprises an encoding means that quantizes and encodes the conversion coefficients in the subband data obtained by the conversion means according to the quantization parameters determined by the determination means, The determination means determines the quantization parameters by weighting each subband based on the information representing the compression ratio obtained by the acquisition means. An image coding device characterized by the following: (Item 2) The determination means determines the quantization parameters by weighting them with a weighting coefficient corresponding to the information representing the compression ratio obtained by the acquisition means. The image encoding device according to item 1, characterized in that it is a picture encoding device. (Item 3) The determination means determines which of the plurality of subbands to weight based on the information representing the compression ratio obtained by the acquisition means. The image encoding device according to item 2, characterized in that it is a picture encoding device. (Item 4) The determination means switches whether to weight the LH subband or the HL subband according to the compression ratio obtained by the acquisition means. The image encoding device according to item 1, characterized in that it is a picture encoding device. (Item 5) The optical compression ratio in the horizontal and vertical directions is such that the horizontal compression ratio relative to the vertical is a first ratio (first ratio < 1), The determination means determines the quantization parameters such that the ratio of the quantization parameters applied to the HL subband to the quantization parameters applied to the LH subband at the same resolution level becomes the first ratio. The image encoding device according to item 1, characterized in that it is a picture encoding device. (Item 6) The determination means determines the quantization parameters such that, when the optical horizontal and vertical compression ratio is 1, the HL subband and LH subband at the same resolution level have the same quantization parameters. When the optical horizontal-to-vertical compression ratio is such that the horizontal compression ratio relative to the vertical is a first ratio (first ratio < 1), the quantization parameters are determined by weighting the HL subband by the first ratio, thereby making the quantization parameters applied to the HL subband smaller than those applied to the LH subband at the same resolution level. The image encoding device according to item 1, characterized in that it is a picture encoding device. (Item 7) The optical compression ratio in the horizontal and vertical directions is such that the vertical compression ratio relative to the horizontal is a second ratio (second ratio < 1), The determination means determines the quantization parameters such that the ratio of the quantization parameters applied to the LH subband to the HL subband at the same resolution level becomes the second ratio. The image encoding device according to item 1, characterized in that it is a picture encoding device. (Item 8) The determination means determines the quantization parameters such that, when the optical horizontal and vertical compression ratio is 1, the HL subband and LH subband at the same resolution level have the same quantization parameters. When the optical compression ratio between the horizontal and vertical directions is such that the vertical compression ratio relative to the horizontal is a second ratio (second ratio < 1), the quantization parameters are determined by weighting the LH subband by the second ratio, thereby making the quantization parameters applied to the LH subband smaller than those applied to the HL subband at the same resolution level. The image encoding device according to item 1, characterized in that it is a picture encoding device. (Item 9) The aforementioned image encoding device has a detachable optical lens unit. The acquisition means acquires information representing the optical horizontal and vertical compression ratios by communicating with the lens unit. The determination means determines the quantization parameters based on information representing the optical horizontal and vertical compression ratios obtained from the lens unit. An image encoding device according to any one of items 1 to 8, characterized by the above. (Item 10) The acquisition means acquires information representing the compression ratio selected by the user via the operation unit of the image encoding device. The determination means determines the quantization parameter based on the information obtained by the acquisition means, which represents the compression ratio selected by the user. An image encoding device according to any one of items 1 to 9, characterized by the features described herein. (Item 11) If the information representing the compression ratio selected by the user indicates a compression ratio that is compressed more horizontally than vertically, the determination means determines that the quantization parameter applied to the HL subband at the same resolution level is smaller than the quantization parameter applied to the LH subband. The image encoding device according to item 10, characterized in that it is a picture encoding device. (Item 12) The system further includes a plane transformation means that transforms the image obtained by the imaging means into a plurality of planes, each composed of a single component. The conversion means performs a wavelet transform on each plane obtained from the plane transformation means. An image encoding device according to any one of items 1 to 11, characterized by the features described herein. (Item 13) The imaging means outputs a RAW image of the Bayer array, The plane conversion means converts the RAW image into an R plane, a G1 plane, a G2 plane, and a B plane. The image encoding device according to item 12, characterized in that it is a picture encoding device. (Item 14) If the optical horizontal and vertical compression ratios represent a compression ratio such that the horizontal image in the image obtained by the imaging means is 1 / 2 the size of the vertical image, The determination means determines that the quantization parameter applied to the HL subband at the same resolution level is half the quantization parameter applied to the LH subband. The image encoding device described in item 13, characterized by the features described herein. (Item 15) If the optical horizontal and vertical compression ratios represent a compression ratio such that the vertical image in the image obtained by the imaging means is halved relative to the horizontal image, The determination means determines that the quantization parameter applied to the LH subband at the same resolution level is half the quantization parameter applied to the HL subband. The image encoding device described in item 13, characterized by the features described herein. (Item 16) The imaging means outputs a RAW image in a Bayer array where 2x2 pixels are arranged in raster scan order as R, G1, G2, B pixels. The plane conversion means converts the RAW image into the following color difference U, V, and GH planes and the luminance Y plane. U = B - (G1 + G2) / 2 V = R - (G1 + G2) / 2 GH = G1-G2 Y = (R+B+G1+G2) / 4 The image encoding device according to item 12, characterized in that it is a picture encoding device. (Item 17) If the optical horizontal and vertical compression ratios represent a compression ratio such that the horizontal image in the image obtained by the imaging means is 1 / 2 the size of the vertical image, The aforementioned determination means is The ratio of the quantization parameter values ​​set for the HL, LH, and HH subbands of the Y plane at the same resolution level is set to 1 / 2:1:1. The ratio of the quantization parameter values ​​set for the HL, LH, and HH subbands of the U, V, and GH planes at the same resolution level is set to 1 / √5:2 / √5:1. The image encoding device according to item 16, characterized in that it is a picture encoding device. (Item 18) If the optical horizontal and vertical compression ratios represent a compression ratio such that the vertical image in the image obtained by the imaging means is halved relative to the horizontal image, The aforementioned determination means is The ratio of the quantization parameter values ​​set for the HL, LH, and HH subbands of the Y plane at the same resolution level is set to 1:1 / 2:1. The ratio of the quantization parameter values ​​set for the HL, LH, and HH subbands of the U, V, and GH planes at the same resolution level is set to 2 / √5:1 / √5:1. The image encoding device according to item 16, characterized in that it is a picture encoding device. (Item 19) A control method for an image encoding device that encodes image data obtained by an imaging means, The acquisition process involves obtaining information representing the optical horizontal and vertical compression ratios during shooting, A transformation step of performing a wavelet transform on the aforementioned image data to generate multiple subband data, A determination step for determining quantization parameters for the conversion coefficients in multiple subbands obtained in the conversion step, The system comprises an encoding step which quantizes and encodes the conversion coefficients in the subband data obtained in the conversion step according to the quantization parameters determined in the determination step, The determination step determines the quantization parameters by weighting each subband based on the information representing the compression ratio obtained in the acquisition step. A control method for an image encoding device, characterized by the above. (Item 20) A program that, when read and executed by a computer, causes the computer to function as one of the means of the device described in any one of items 1 to 18.

[0118] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of symbols]

[0119] 100...Imaging device, 101...Imaging unit, 102...Plane transform unit, 103...Discrete wavelet transform unit, 104...Quantization unit, 105...Encoding unit, 106...Quantization control unit, 107...Recording processing unit, 108...Recording medium, 109...Generated code amount holding unit, 110...Difference calculation unit, 111...Quantization value calculation unit, 112...Quantization parameter weight setting unit, 113...Target code amount calculation unit, 114...Initial target code amount setting unit, 115...Initial quantization value setting unit

Claims

1. An image encoding device for encoding image data obtained by an imaging means, An acquisition means for acquiring information representing the optical horizontal and vertical compression ratios when taking photographs, A transformation means that performs a wavelet transform on the aforementioned image data to generate multiple subband data, A determination means for determining quantization parameters for the conversion coefficients in multiple subbands obtained by the conversion means, The system comprises an encoding means that quantizes and encodes the conversion coefficients in the subband data obtained by the conversion means according to the quantization parameters determined by the determination means, The determination means determines the quantization parameters by weighting each subband based on the information representing the compression ratio obtained by the acquisition means. An image coding device characterized by the following:

2. The determination means determines the quantization parameters by weighting them with a weighting coefficient corresponding to the information representing the compression ratio obtained by the acquisition means. The image coding apparatus according to feature 1.

3. The determination means determines which of the plurality of subbands to weight based on the information representing the compression ratio obtained by the acquisition means. The image encoding apparatus according to feature 2.

4. The determination means switches whether to weight the LH subband or the HL subband according to the compression ratio obtained by the acquisition means. The image coding apparatus according to feature 1.

5. The optical compression ratio in the horizontal and vertical directions is such that the horizontal compression ratio relative to the vertical is a first ratio (first ratio < 1), The determination means determines the quantization parameters such that the ratio of the quantization parameters applied to the HL subband to the quantization parameters applied to the LH subband at the same resolution level becomes the first ratio. The image coding apparatus according to feature 1.

6. The determination means determines the quantization parameters such that, when the optical horizontal and vertical compression ratio is 1, the HL subband and LH subband at the same resolution level have the same quantization parameters. When the optical horizontal-to-vertical compression ratio is such that the horizontal compression ratio relative to the vertical is a first ratio (first ratio < 1), the quantization parameters are determined by weighting the HL subband by the first ratio, thereby making the quantization parameters applied to the HL subband smaller than those applied to the LH subband at the same resolution level. The image coding apparatus according to feature 1.

7. The optical compression ratio in the horizontal and vertical directions is such that the vertical compression ratio relative to the horizontal is a second ratio (second ratio < 1), The determination means determines the quantization parameters such that the ratio of the quantization parameters applied to the LH subband to the HL subband at the same resolution level becomes the second ratio. The image coding apparatus according to feature 1.

8. The determination means determines the quantization parameters such that, when the optical horizontal and vertical compression ratio is 1, the HL subband and LH subband at the same resolution level have the same quantization parameters. When the optical compression ratio in the horizontal and vertical directions is such that the vertical compression ratio relative to the horizontal is a second ratio (second ratio < 1), the quantization parameters are determined by weighting the LH subband by the second ratio, thereby making the quantization parameters applied to the LH subband smaller than those applied to the HL subband at the same resolution level. The image coding apparatus according to feature 1.

9. The aforementioned image encoding device has a detachable optical lens unit. The acquisition means acquires information representing the optical horizontal and vertical compression ratios by communicating with the optical lens unit. The determination means determines the quantization parameters based on information representing the optical horizontal and vertical compression ratios obtained from the optical lens unit. The image coding apparatus according to feature 1.

10. The acquisition means acquires information representing the compression ratio selected by the user via the operation unit of the image encoding device. The determination means determines the quantization parameter based on the information obtained by the acquisition means, which represents the compression ratio selected by the user. The image coding apparatus according to feature 1.

11. If the information representing the compression ratio selected by the user indicates a compression ratio that is compressed more horizontally than vertically, the determination means determines that the quantization parameter applied to the HL subband at the same resolution level is smaller than the quantization parameter applied to the LH subband. The image encoding apparatus according to feature 10.

12. The system further includes a plane transformation means that transforms the image obtained by the imaging means into a plurality of planes, each composed of a single component. The conversion means performs a wavelet transform on each plane obtained from the plane transformation means. The image coding apparatus according to feature 1.

13. The imaging means outputs a RAW image of the Bayer array, The plane conversion means converts the RAW image into an R plane, a G1 plane, a G2 plane, and a B plane. The image encoding apparatus according to feature 12.

14. If the optical horizontal and vertical compression ratios represent a compression ratio such that the horizontal image in the image obtained by the imaging means is 1 / 2 the size of the vertical image, The determination means determines that the quantization parameter applied to the HL subband at the same resolution level is half the quantization parameter applied to the LH subband. The image encoding apparatus according to feature 13.

15. If the optical horizontal and vertical compression ratios represent a compression ratio such that the vertical image in the image obtained by the imaging means is halved relative to the horizontal image, The determination means determines that the quantization parameter applied to the LH subband at the same resolution level is half the quantization parameter applied to the HL subband. The image encoding apparatus according to feature 13.

16. The imaging means outputs a RAW image in a Bayer array where 2x2 pixels are arranged in raster scan order as R, G1, G2, B pixels. The plane conversion means converts the RAW image into the following color difference U, V, and GH planes and the luminance Y plane. U = B - (G1 + G2) / 2 V = R - (G1 + G2) / 2 GH = G1-G2 Y = (R+B+G1+G2) / 4 The image encoding apparatus according to feature 12.

17. If the optical horizontal and vertical compression ratios represent a compression ratio such that the horizontal image in the image obtained by the imaging means is 1 / 2 the size of the vertical image, The aforementioned determination means is The ratio of the quantization parameter values ​​set for the HL, LH, and HH subbands of the Y plane at the same resolution level is set to 1 / 2:1:

1. The ratio of the quantization parameter values ​​set for the HL, LH, and HH subbands of the U, V, and GH planes at the same resolution level is set to 1 / √5:2 / √5:

1. The image encoding apparatus according to feature 16.

18. If the optical horizontal and vertical compression ratios represent a compression ratio such that the vertical image in the image obtained by the imaging means is halved relative to the horizontal image, The aforementioned determination means is The ratio of the quantization parameter values ​​set for the HL, LH, and HH subbands of the Y plane at the same resolution level is set to 1:1 / 2:

1. The ratio of the quantization parameter values ​​set for the HL, LH, and HH subbands of the U, V, and GH planes at the same resolution level is set to 2 / √5:1 / √5:

1. The image encoding apparatus according to feature 16.

19. A control method for an image encoding device that encodes image data obtained by an imaging means, The acquisition process involves obtaining information representing the optical horizontal and vertical compression ratios during shooting, A transformation step of performing a wavelet transform on the aforementioned image data to generate multiple subband data, A determination step for determining quantization parameters for the conversion coefficients in multiple subbands obtained in the conversion step, The system comprises an encoding step which quantizes and encodes the conversion coefficients in the subband data obtained in the conversion step according to the quantization parameters determined in the determination step, The determination step determines the quantization parameters by weighting each subband based on the information representing the compression ratio obtained in the acquisition step. A control method for an image encoding device, characterized by the above.

20. A program that, when read and executed by a computer, causes the computer to perform each step of the control method described in claim 19.

Images