Encoding device, imaging device, control method, and program

The described filter processing method for RAW image data adjusts filter strength based on brightness and contrast to balance noise reduction and image quality, improving encoding efficiency while maintaining image quality.

JP7579075B2Active Publication Date: 2024-11-07CANON KK
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2020118738
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2020-07-09
Publication Date
2024-11-07
Estimated Expiration
2040-07-09

AI Technical Summary

Technical Problem

Existing methods for encoding RAW image data struggle to balance noise reduction and image quality, with Gaussian filters either improving encoding efficiency at the cost of image quality or maintaining quality at the expense of reduced noise reduction.

Method used

A filter processing method that adjusts filter strength based on brightness and contrast of RAW image data, applying different strengths to different areas to balance noise reduction and image quality, using a Gaussian filter with varying standard deviations based on feature information.

Benefits of technology

Achieves both effective noise reduction and preservation of image quality by adaptively adjusting filter strengths according to image characteristics, enhancing encoding efficiency while minimizing quality deterioration.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007579075000002
    Figure 0007579075000002
  • Figure 0007579075000003
    Figure 0007579075000003
  • Figure 0007579075000004
    Figure 0007579075000004
Patent Text Reader

Abstract

To provide a technique of applying filter processing so as to achieve both of the suppression of an image quality deterioration to RAW image data and the improvement of a noise reduction effect with satisfactory balance.SOLUTION: A coding device comprises: filter means of applying filter processing that reduces a high frequency component to RAW image data; acquisition means of acquiring first characteristic information indicating brightness of the RAW image data; determination means of determining a filter intensity of the filter processing of each pixel of the RAW image data on the basis of the first characteristic information of the RAW image data, in which the filter intensity of a first pixel having a first brightness is smaller than the filter intensity of a second pixel having a second brightness larger than the first brightness; and coding means of coding the RAW image data to which the filter processing is applied.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an encoding device, an imaging device, a control method, and a program. [Background technology]

[0002] Currently, there are digital cameras that can record unprocessed image data (RAW image data) immediately after it is obtained by an image sensor. RAW image data is recorded while maintaining a rich range of gradations without compromising color information, allowing for highly flexible editing. However, the amount of RAW image data is huge, which can cause a problem of squeezing free space on a recording medium. For this reason, RAW image data is generally recorded after reducing the amount of data by compression encoding. For example, Patent Document 1 discloses a method of separating RAW image data in a Bayer array into four plane data of R, G1, G2, and B and encoding the data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2003-125209 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the process of generating RAW image data, dark current noise, thermal noise, shot noise, etc. occur due to the characteristics of the image sensor and circuit, so noise is mixed into the RAW image data. This noise causes random fluctuations in pixel values, which may become high-frequency components and lead to a decrease in coding efficiency. Therefore, in order to improve coding efficiency, it is necessary to suppress noise before coding. One of the noise suppression methods is the Gaussian filter. If the filter strength of the Gaussian filter is large (strong), the noise reduction effect is improved and the coding efficiency is improved, but image quality degradation occurs due to a decrease in resolution. On the other hand, if the filter strength of the Gaussian filter is small (weak), image quality degradation is suppressed, but the noise reduction effect is reduced and coding efficiency is reduced.

[0005] The present invention has been made in consideration of the above-mentioned circumstances, and aims to provide a technique for applying filter processing to RAW image data in a balanced manner between suppressing image quality degradation and improving noise reduction effects. [Means for solving the problem]

[0006] In order to solve the above problems, the present invention provides a filter means for applying a filter process for reducing high frequency components to RAW image data, and 、 First characteristic information indicating brightness and a second feature indicating contrast. and acquiring means for acquiring the first characteristic information of the RAW image data. and the second characteristic information. A method for determining a filter strength of the filtering process for each pixel of the RAW image data based on the Step by step and an encoding unit that encodes the RAW image data to which the filtering has been applied. a filter strength for a first pixel having a first brightness and a first contrast is less than a filter strength for a second pixel having a second brightness greater than the first brightness and the first contrast; and a filter strength for a third pixel having a third brightness and a second contrast is greater than a filter strength for a fourth pixel having the third brightness and a third contrast greater than the second contrast. The present invention provides an encoding device comprising: Effect of the Invention

[0007] According to the present invention, it is possible to apply filter processing to RAW image data so as to achieve a good balance between suppressing deterioration in image quality and improving the noise reduction effect.

[0008] Other features and advantages of the present invention will become apparent from the accompanying drawings and the following detailed description of the preferred embodiments of the present invention. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a configuration of an imaging device 100 including an encoding device. [Diagram 2] FIG. 1 shows RAW image data of a Bayer array. [Diagram 3] FIG. 2 is a block diagram showing a detailed configuration of a filter processing unit 104. [Figure 4] FIG. 11 is a graph showing the relationship between brightness and contrast and filter strength. [Diagram 5] 13 is a flowchart of a filter process. [Figure 6] 5A and 5B are diagrams for explaining pixel values ​​of a pixel to be filtered and its surrounding pixels; [Figure 7] FIG. 11 is a graph showing the relationship between brightness and contrast and filter strength for recording moving images. [Figure 8] 10 is a flowchart of a selection process of a setting pattern of a filter strength according to a second embodiment. [Figure 9] 13 is a flowchart of a selection process of a setting pattern of a filter strength according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0011] [First embodiment] 1 is a block diagram showing the configuration of an image capture device 100 including an encoding device. The image capture device 100 is configured to reduce noise by performing a predetermined filter process on RAW image data obtained by capturing an image, and then encode and record the image data.

[0012] The control unit 101 controls each unit of the imaging device 100. The imaging unit 102 includes a lens optical system and an imaging element, and generates RAW image data by converting an electrical signal obtained by the imaging element into a digital signal. The RAW image data is configured in a Bayer array as shown in FIG. 2. The generated RAW image data is stored in a memory 108 via a memory I / F 107. The lens optical system includes an optical lens, an aperture, a focus control unit, and a lens driving unit, and is configured to enable optical zoom. The imaging element includes a CCD image sensor that converts optical information from the lens optical system into an electrical signal, a CMOS sensor, or the like.

[0013] The plane conversion unit 103 reads out the RAW image data stored in the memory 108 via the memory I / F 107, and converts the RAW image data into four independent plane data by separating the data into each color component (R, G1, G2, B). The plane conversion unit 103 then writes the four plane data into the memory 108 via the memory I / F 107. Note that the method of conversion into plane data is not limited to this, and for example, a method of approximately converting into a luminance plane (Y) and other planes (C0, C1, C2) as shown in the following formula 1 may be used. Y=(R+G1+G2+B) / 4 C0=RB C1=(G0+G1) / 2-(R+B) / 2 C2=G0-G1 (1)

[0014] The filter processing unit 104 reads out plane data constituting the RAW image data stored in the memory 108 via the memory I / F 107, performs a predetermined filter process for each pixel, and writes the result back to the memory 108. The filter process here is a low-pass filter process that removes noise that may be a high-frequency component. One of the low-pass filters is a Gaussian filter. The Gaussian filter is a technology that uses a Gaussian distribution to weight nearby pixel values ​​according to the distance from a pixel of interest to smooth noise. If the coordinates of the center pixel to be filtered are (0,0), the image data before filter process at the coordinates (i,j) is f(i,j), the kernel size of the filter is w, and the standard deviation is σ, the image data after filter process g(i,j) is expressed by the following formula 2. TIFF0007579075000001.tif38160

[0015] As shown in Equation 2, once the standard deviation σ is determined, the weighting coefficient for the surrounding pixel values ​​is uniquely determined for the Gaussian filter. In other words, the standard deviation σ is a parameter that represents the filter strength. In the following explanation, the standard deviation σ will be referred to as the filter strength. The details of the filter process will be described later.

[0016] The filter used in the filtering process of the filtering unit 104 is not limited to a Gaussian filter, and any filter that reduces high frequency components can be used. The parameter corresponding to the filter strength is appropriately selected according to the type of filter used in the filtering process.

[0017] The encoding processing unit 105 performs compression encoding using a predetermined compression method on the plain data after the filtering process by the filtering processing unit 104 to generate encoded data. Then, the encoding processing unit 105 records the generated encoded data on the recording medium 106. The compression method is not particularly limited, and any compression method for RAW image data can be used.

[0018] The recording medium 106 is a recording medium configured, for example, of a non-volatile memory. The memory I / F 107 arbitrates memory access requests from each unit of the imaging device 100 and controls reading and writing to the memory 108. The memory 108 is a storage area configured, for example, of a volatile memory, for storing various data output from each unit of the imaging device 100.

[0019] 3 is a block diagram showing a detailed configuration of the filtering processing unit 104. The filtering processing unit 104 includes a feature information generating unit 301, a filter strength determining unit 302, and a filtering processing executing unit 303.

[0020] The feature information generating unit 301 generates feature information regarding the plane data of each color component read out via the memory I / F 107, and outputs the generated feature information to the filter strength determining unit 302. The feature information generated here is of two types, lightness (first feature information) and contrast (second feature information), and serves as index values ​​for determining the filter strength. An example of a method for calculating the feature information is shown in Equation 3. In Equation 3, g0 to g8 are pixel values ​​shown in FIG. 6, and MAX is a function for finding the maximum value from among candidate values. Brightness=g0 Contrast=MAX(|g0-g1|,|g0-g2|,|g0-g3|, |g0-g4|,|g0-g5|,|g0-g6|, |g0-g7|,|g0-g8|) (3)

[0021] As shown in Equation 3, the brightness is the pixel value of the pixel to be filtered, and the contrast is the maximum value of the absolute difference between the pixel to be filtered and each of the eight pixels adjacent to the pixel to be filtered.

[0022] The method of calculating the brightness and contrast is not limited to that shown in Equation 3, and other calculation methods may be used.

[0023] Furthermore, when Y, C0, C1, and C2 are used as plane data (see formula 1), the feature information generating unit 301 may generate feature information such as brightness and contrast only for the luminance plane (Y). In this case, the filter strength determined based on the feature information of the luminance plane (Y) is also used in the filter processing of the other planes (C0, C1, and C2). Alternatively, in consideration of human visual characteristics, a filter strength greater than the filter strength determined based on the feature information of the luminance plane (Y) may be used as the filter strength for the other planes (C0, C1, and C2). For example, the filter strength for the other planes (C0, C1, and C2) may be generated by multiplying the filter strength determined based on the feature information of the luminance plane (Y) by a coefficient greater than 1.

[0024] Furthermore, even when R, G1, G2, and B are used as plane data, the feature information generating unit 301 may generate feature information only for a specific plane (e.g., R). In this case, the filter strength determined based on the feature information of the specific plane (e.g., R) is also used in the filtering process of the other planes (e.g., G1, G2, and B).

[0025] In this way, when generating feature information and determining filter strength only for a specific plane, the filter processing unit 104 only needs to include a feature information generating unit 301 and a filter strength determining unit 302 for processing one plane, which makes it possible to reduce the circuit scale.

[0026] Here, the relationship between the feature information (brightness and contrast) and the filter strength will be described. The RAW image data is subjected to a post-processing after recording, such as a gamma correction process and a tone curve correction process, to adjust the brightness level. The brightness level adjustment is generally performed by improving the brightness level of the dark area and lowering the brightness level of the bright area. When comparing a dark area with a small original brightness level with a bright area with a large original brightness level, the dark area has a larger fluctuation rate of pixel value even if the same brightness level adjustment is performed. If low-pass filter processing is performed with the same filter strength for the dark area and the bright area, the dark area has a larger fluctuation rate of pixel value, so the fluctuation of pixel value due to the filter processing is also amplified, and image quality degradation is easily noticeable. On the other hand, in the bright area with a small fluctuation rate of the brightness level, the fluctuation rate of pixel value is small, so image quality degradation is not easily noticeable. Therefore, the filter processing unit 104 controls the filter processing so that the filter strength is reduced in the dark area (i.e., an area with low brightness (dark)) where image quality degradation is easily noticeable, and the filter strength is increased in the bright area (i.e., an area with high brightness (bright)) where image quality degradation is not easily noticeable.

[0027] Furthermore, when low-pass filter processing is performed on edge portions, the steep changes in pixel values ​​are dulled, causing a drop in perceived resolution, making image quality degradation more noticeable. On the other hand, in flat portions, pixel changes are originally small and the drop in perceived resolution is difficult to visually recognize, so image quality degradation is less noticeable. Therefore, the filter processing unit 104 controls the filter processing so that the filter strength is reduced in edge portions (i.e., areas with high contrast) where image quality degradation is more noticeable, and the filter strength is increased in flat areas (i.e., areas with low contrast) where image quality degradation is less noticeable.

[0028] In this way, the degree to which visual degradation is noticeable varies depending on the image features, so if the filter strength is made uniform within the screen, only weak filter processing can be performed to maintain image quality, and coding efficiency cannot be significantly improved. On the other hand, the filter processing unit 104 of this embodiment performs filter processing while changing the filter strength according to the features within the screen, so that it is possible to achieve a good balance between suppression of image degradation and improvement of noise reduction effect (improvement of coding efficiency).

[0029] The filter strength determination unit 302 performs feature classification using the feature information (brightness and contrast) generated by the feature information generation unit 301, and determines filter strengths σ0 to σ5 according to the classification results. Then, the filter strength determination unit 302 outputs the determined filter strengths to the filter process execution unit 303.

[0030] Here, the feature classification will be described with reference to Fig. 4. In Fig. 4, TH0 is a brightness threshold for classifying a dark area and a bright area, and TH1 is a contrast threshold for classifying a flat area and a low contrast area. TH2 is a contrast threshold for classifying a low contrast area and a medium contrast area in a dark area, and TH3 is a contrast threshold for classifying a medium contrast area and a high contrast area in a dark area. TH4 is a contrast threshold for classifying a low contrast area and a high contrast area in a bright area. Each threshold is a parameter that can be arbitrarily set according to the bit depth of the image, etc.

[0031] In feature classification using brightness, brightness is classified into either dark or bright areas depending on the magnitude relationship between the brightness calculated by the feature information generating unit 301 and a threshold value TH0. In feature classification using contrast, contrast is classified into either flat, low contrast, medium contrast, or high contrast depending on the magnitude relationship between the contrast calculated by the feature information generating unit 301 and threshold values ​​TH1, TH2, TH3, and TH4.

[0032] The result of the feature classification (feature region) corresponds to the feature of image quality degradation. The filter strength applied to each feature region will be described. σ0 to σ5 shown in FIG. 4 are filter strengths applied to each feature region classified according to the magnitude relationship with each threshold value. An example of the magnitude relationship of the filter strength for each feature region is shown below. σ0<σ1≦σ2<σ3≦σ4<σ5 (σ0 to σ5 are parameters that can be set arbitrarily)

[0033] The contrast threshold will be further explained. As mentioned above, since the dark area often contains minute information, the minute information is easily lost when a low-pass filter is applied. Therefore, if the contrast threshold for the bright area and the dark area is set to the same value, the image quality degradation is not visible at the edge boundary of the bright area, but is visible at the minute edge boundary of the dark area. Since the feature amount of the area to be protected is different between the dark area and the bright area, the contrast threshold for the dark area (TH2, TH3) and the contrast threshold for the bright area (TH4) can be set separately. Also, the contrast is classified more finely in the dark area than in the bright area because the image quality degradation is more sensitive to minute changes in contrast in the dark area. On the other hand, when the contrast is below a certain value (TH1) (i.e., in the case of an almost flat area), the image quality degradation is difficult to be visually recognized regardless of the brightness, so the threshold for the dark area and the bright area is made common.

[0034] The filter process execution unit 303 performs a filter process using a Gaussian filter on the plain data read out via the memory I / F 107. In this filter process, the filter process execution unit 303 applies the filter strength (σ0 to σ5) determined by the filter strength determination unit 302 to the standard deviation σ in Equation 2. Then, the filter process execution unit 303 writes the data after the filter process to the memory 108 via the memory I / F 107.

[0035] Fig. 5 is a flowchart of the filter process for one piece of plane data executed by the filter processing unit 104. In the explanation of Fig. 5, the filter strength and the threshold value used for feature classification explained with reference to Fig. 4 are used as they are. When the plane data to be processed (e.g., R) stored in the memory 108 is input to the filter processing unit 104 under the control of the control unit 101, the process of this flowchart starts.

[0036] In S501 , the feature information generating unit 301 generates feature information (brightness and contrast) of a pixel to be filtered, and outputs the generated feature information to the filter strength determining unit 302 .

[0037] In S502, the filter strength determination unit 302 determines whether the contrast generated in S501 is larger than the threshold value TH1. If the contrast is smaller than TH1, the process proceeds to S503, and if not, the process proceeds to S504.

[0038] In S503, the filter strength determination unit 302 determines the filter strength to be σ5. In this way, for pixels having a contrast lower than TH1 (predetermined contrast threshold), the filter strength determination unit 302 determines the filter strength without taking into consideration the lightness (first feature information).

[0039] In S504, the filter strength determination unit 302 determines whether the brightness generated in S501 is smaller than the threshold value TH0. If the brightness is smaller than TH0, the process proceeds to S505, and if not, the process proceeds to S510.

[0040] In S505, the filter strength determination unit 302 determines whether the contrast generated in S501 is larger than a threshold value TH2. If the contrast is smaller than TH2, the process proceeds to S506, where the filter strength determination unit 302 determines the filter strength to be σ3. If the contrast is equal to or larger than TH2, the process proceeds to S507.

[0041] In S507, the filter strength determination unit 302 determines whether the contrast generated in S501 is larger than a threshold value TH3. If the contrast is smaller than TH3, the process proceeds to S508, where the filter strength determination unit 302 determines the filter strength to be σ1. If the contrast is equal to or larger than TH3, the process proceeds to S509, where the filter strength determination unit 302 determines the filter strength to be σ0.

[0042] In S510, the filter strength determination unit 302 determines whether the contrast generated in S501 is larger than a threshold value TH4. If the contrast is smaller than TH4, the process proceeds to S511, where the filter strength determination unit 302 determines the filter strength to be σ4. If the contrast is equal to or larger than TH4, the process proceeds to S512, where the filter strength determination unit 302 determines the filter strength to be σ2.

[0043] After the filter strength is determined by the processes of S502 to S512, in S513, the filter process execution unit 303 applies a Gaussian filter to the filtering target pixel using the determined filter strength.

[0044] In S514, the filter processing unit 104 determines whether or not the filter process has been completed for all pixels constituting the plane data to be processed (for example, R). If the filter process has not been completed for all pixels, the process returns to S501, and the processes of S501 to S513 are performed for the next pixel to be filtered. If the filter process has been completed for all pixels, the process of this flowchart ends.

[0045] As described above, according to the first embodiment, the imaging device 100 determines the filter strength of the filter process for each pixel of the RAW image data based on the brightness (first feature information) and contrast (second feature information) of the RAW image data. The relationship between the filter strength determined in this way and the brightness and contrast is, for example, as shown in Fig. 4, where σ0<σ1≦σ2<σ3≦σ4<σ5.

[0046] This makes it possible to apply filter processing to RAW image data in a well-balanced manner between suppressing deterioration in image quality and improving the noise reduction effect.

[0047] In the example of Fig. 4, feature classification is performed so as to change the filter strength into two stages for brightness and four stages (dark areas) or three stages (bright areas) for contrast. However, the number of stages (feature regions) of feature classification is not limited to the example of Fig. 4, and the number of stages (feature regions) may be increased or decreased. Also, it is not essential to consider both brightness and contrast, and for example, the imaging device 100 may determine the filter strength of each pixel based only on brightness (or only on contrast).

[0048] [Second embodiment] In the second embodiment, a configuration in which a method of determining a filter strength is changed between a still image and a moving image will be described. In the second embodiment, the basic configuration of the imaging device 100 is similar to that of the first embodiment. Below, differences from the first embodiment will be mainly described.

[0049] The image playback method differs between still images and videos. While still images are played back independently on a frame-by-frame basis, videos are played back continuously in sequences of multiple frames. Therefore, the display time per frame of a video is much shorter than that of a still image, so the degradation in image quality when filtering is performed at the same strength is less noticeable. Therefore, when compared at the same compression rate, the video recording mode can improve the coding efficiency with relatively little degradation in image quality by further increasing the filter strength and reducing noise compared to the still image recording mode.

[0050] Therefore, the image capturing apparatus 100 of this embodiment operates to hold setting patterns of filter strength corresponding to the still image recording mode and the moving image recording mode, and to switch the setting pattern to be used depending on the recording mode setting.

[0051] In this embodiment, the same RAW image data is output from the imaging unit 102 when recording still images and when recording moving images, and the filter strength setting pattern of FIG. 4 described in the first embodiment is used when recording still images.

[0052] Fig. 7(a) is a diagram showing a setting pattern of filter strength for a moving image recording mode. In Fig. 7(a), the magnitude of the filter strength corresponding to each characteristic region is different from that in Fig. 4. A0 to A5 respectively represent the correction amount of the filter strength corresponding to each characteristic region, and the correction amount is a positive value. Therefore, by using the setting pattern of Fig. 7(a) when recording a moving image, the filter strength becomes larger than when recording a still image.

[0053] 8 is a flowchart of a process for selecting a setting pattern of a filter strength according to the second embodiment. Note that the feature classification method and the method for executing the filter process are the same as those in the first embodiment, and therefore the description thereof will be omitted.

[0054] In S801, the filter strength determination unit 302 acquires the current recording mode via the control unit 101. The recording mode is a still image recording mode or a moving image recording mode.

[0055] In S802, the filter strength determination unit 302 determines whether the recording mode acquired in S801 is the still image recording mode. If it is the still image recording mode, the process proceeds to S803, and if not, the process proceeds to S804.

[0056] In S803, the filter strength determination unit 302 selects a setting pattern for still image recording (for example, the setting pattern shown in FIG. 4) as the setting pattern of the filter strength.

[0057] In S804, the filter strength determination unit 302 selects a setting pattern for moving image recording (for example, the setting pattern shown in FIG. 7(a)) as the setting pattern of the filter strength.

[0058] The setting pattern for video recording is not limited to the one shown in FIG. 7(a). As another example, the setting pattern shown in FIG. 7(b) can be used. The setting pattern in FIG. 7(b) differs from the setting pattern in FIG. 4 in the magnitude of the threshold used for feature classification. B0 to B5 in FIG. 7(b) each represent the correction amount of the threshold, and the correction amount is a positive value. Therefore, by using the setting pattern in FIG. 7(b) when recording a video, the feature region with a high filter strength becomes wider compared to when recording a still image (i.e., the filter strength tends to be high overall). Alternatively, as yet another example, a setting pattern in which both the filter strength and the threshold are changed compared to FIG. 4, as shown in FIG. 7(c), can be used.

[0059] As described above, according to the second embodiment, the imaging device 100 determines the filter strength of the filter process for each pixel of the RAW image data, further based on whether the RAW image data was generated by shooting a still image or a moving image. The relationship between the filter strength determined in this way and the brightness and contrast is, for example, as shown in Fig. 4 when the RAW image data was generated by shooting a still image, and is, for example, as shown in Fig. 7 when the RAW image data was generated by shooting a moving image. Therefore, in this embodiment, for some or all of the possible combinations of brightness and contrast, a higher filter strength is used when recording a moving image than when recording a still image.

[0060] This makes it possible to further improve the noise reduction effect when recording moving images, where degradation in image quality is less noticeable than when recording still images.

[0061] [Third embodiment] In the third embodiment, a configuration will be described in which a method for determining a filter strength is changed according to a compression ratio at the time of recording a moving image. In the third embodiment, the basic configuration of the imaging device 100 is similar to that of the first embodiment. Below, differences from the first embodiment will be mainly described.

[0062] When recording moving images, in order to guarantee the recording time, the amount of code is controlled so as not to exceed a certain bit rate. In general code amount control, a target code amount is set in a certain unit, and the quantization step is controlled so that the amount of generated code approaches the target code amount. If this quantization step is large, the coefficients of the data to be quantized are likely to become 0 after quantization, and the data cannot be restored by the inverse quantization process at the time of decoding. Data that cannot be restored due to such quantization errors appears as black crush as the compression rate is increased, causing deterioration of image quality.

[0063] Therefore, the imaging device 100 of this embodiment operates to suppress an increase in the quantization step used in quantization by increasing the filter strength during high compression to further reduce high frequency components within the screen.

[0064] Increasing the filter strength will cause a decrease in the sense of resolution, but the sense of resolution can be restored by performing sharpness correction in the subsequent development process. This is because the pixel amplitude is larger in edge areas than in dark areas, making it relatively difficult for the coefficients after quantization to become 0. In this way, taking into account the possibility of recovery in subsequent correction processes, there is a greater benefit in terms of image quality to suppressing areas where the coefficients after quantization become 0 and cannot be restored during decoding.

[0065] The imaging device 100 of this embodiment stores filter strength setting patterns corresponding to still image recording mode, low-compression video recording mode, and high-compression video recording mode, and operates to switch the filter strength setting pattern to be used depending on the recording mode setting.

[0066] In this embodiment, the setting pattern of Figure 4 is used as the setting pattern for still image recording, the setting pattern of Figure 7(a) is used as the setting pattern for low-compression video recording, and the setting pattern of Figure 7(c) is used as the setting pattern for high-compression video recording.

[0067] 9 is a flowchart of a selection process of a setting pattern of a filter strength according to the third embodiment. Note that the feature classification method and the method of executing the filter process are the same as those in the first embodiment, and therefore the description thereof will be omitted.

[0068] The processes of S901 to S903 are the same as those of S801 to S803 in Fig. 8. If it is determined in S902 that the recording mode is not the still image recording mode, the processes from S904 onwards are executed.

[0069] In S904 , the filter strength determination unit 302 acquires the current bit rate via the control unit 101 .

[0070] In S905, the filter strength determination unit 302 determines whether the bit rate acquired in S904 corresponds to the low compression moving image recording mode. For example, when the current bit rate is equal to or greater than the bit rate threshold, the filter strength determination unit 302 determines that the current bit rate corresponds to the low compression moving image recording mode. Alternatively, the filter strength determination unit 302 may determine whether the recording mode is the low compression moving image recording mode or the high compression moving image recording mode based on other information, such as a mode indicating the size of the file size after encoding. If it is the low compression moving image recording mode, the process proceeds to S906, and if not, the process proceeds to S907.

[0071] In S906, the filter strength determination unit 302 selects a setting pattern for low-compression moving image recording (for example, the setting pattern shown in FIG. 7(a)) as the setting pattern of the filter strength.

[0072] In S907, the filter strength determination unit 302 selects a setting pattern for high-compression moving image recording (for example, the setting pattern shown in FIG. 7(c)) as the setting pattern of the filter strength.

[0073] As described above, according to the third embodiment, when RAW image data is generated by shooting a moving image, the imaging device 100 determines the filter strength of the filter process for each pixel of the RAW image data, further based on the setting of the encoding compression rate. The relationship between the filter strength determined in this way and the brightness and contrast is, for example, as shown in FIG. 7(a) in the case of a low compression rate (first compression rate), and is, for example, as shown in FIG. 7(c) in the case of a high compression rate (second compression rate). Therefore, in this embodiment, a higher filter strength is used at the time of high compression than at the time of low compression in some or all of the possible combinations of brightness and contrast.

[0074] This reduces the possibility that the quantization coefficients will become 0 during high compression, making it possible to suppress deterioration in image quality.

[0075] [Other embodiments] In the above embodiment, information such as brightness and contrast is acquired for each pixel of the RAW image data, and the filter strength is determined for each pixel based on the acquired information for each pixel. However, instead of acquiring information such as brightness and contrast for each pixel, for example, the RAW image data may be divided into rectangular blocks of a predetermined size, and information may be acquired and the filter strength may be determined for each block.

[0076] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0077] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0078] 100: imaging device, 101: control unit, 103: plane conversion unit, 104: filter processing unit, 105: encoding processing unit, 106: recording medium, 107: memory I / F, 108: memory, 301: feature information generation unit, 302: filter strength determination unit, 303: filter processing execution unit

Claims

1. A filter means for applying a filter process to the raw image data to reduce high frequency components; an acquisition means for acquiring first feature information indicating brightness and second feature information indicating contrast for the raw image data; a determination means for determining a filter strength of the filtering process for each pixel of the raw image data based on the first feature information and the second feature information of the raw image data; an encoding means for encoding the raw image data to which the filtering has been applied; Equipped with a filter strength for a first pixel having a first brightness and a first contrast is less than a filter strength for a second pixel having a second brightness greater than the first brightness and the first contrast; a filter strength for a third pixel having a third brightness and a second contrast is greater than a filter strength for a fourth pixel having the third brightness and a third contrast higher than the second contrast; 13. An encoding device comprising:

2. The determining means determines the filter strength of the filtering process without taking the first feature information into consideration for pixels having a contrast lower than a predetermined contrast threshold.

2. The encoding device according to claim 1 .

3. when the third brightness is less than a predetermined brightness threshold, a filter strength of a fifth pixel having the third brightness and a fourth contrast higher than the second contrast and lower than the third contrast is less than a filter strength of the third pixel having the third brightness and the second contrast; If the third brightness is greater than the predetermined brightness threshold, the filter strength of the fifth pixel is equal to the filter strength of the third pixel.

3. The encoding device according to claim 1 or 2.

4. the determining means determines a filter strength of the filtering process for each pixel of the raw image data based on whether the raw image data is generated by capturing a still image or a moving image; The filter strength of the first pixel when the RAW image data is generated by capturing a moving image is greater than the filter strength of the first pixel when the RAW image data is generated by capturing a still image.

4. The encoding device according to claim 1, wherein the first and second inputs are input to the first and second inputs.

5. The determining means determines a filter strength of the filtering process for each pixel of the RAW image data based on whether the RAW image data is generated by capturing a still image or a moving image, When the RAW image data is generated by capturing a still image, a filter strength of a sixth pixel having a fourth brightness greater than the first brightness and less than the second brightness is equal to a filter strength of the first pixel; When the raw image data is generated by capturing a moving image, the filter strength of the sixth pixel is equal to the filter strength of the second pixel.

4. The encoding device according to claim 1, wherein the first and second inputs are input to the first and second inputs.

6. The determining means determines a filter strength of the filtering process for each pixel of the RAW image data based on whether the RAW image data is generated by capturing a still image or a moving image, The filter strength of the third pixel when the RAW image data is generated by capturing a moving image is greater than the filter strength of the third pixel when the RAW image data is generated by capturing a still image.

4. The encoding device according to claim 1, wherein the first and second inputs are input to the first and second inputs.

7. The determining means determines a filter strength of the filtering process for each pixel of the RAW image data based on whether the RAW image data is generated by capturing a still image or a moving image, When the RAW image data is generated by capturing a still image, a filter strength of a seventh pixel having the third brightness and a fifth contrast higher than the second contrast and lower than the third contrast is equal to a filter strength of the fourth pixel; When the raw image data is generated by capturing a moving image, the filter strength of the seventh pixel is equal to the filter strength of the third pixel.

4. The encoding device according to claim 1, wherein the first and second inputs are input to the first and second inputs.

8. When the RAW image data is generated by shooting a moving image, the determining means determines a filter strength of the filtering process for each pixel of the RAW image data further based on a setting of a compression rate of the encoding; The filter strength of the first pixel when the setting is a first compression ratio is smaller than the filter strength of the first pixel when the setting is a second compression ratio higher than the first compression ratio. Coding device according to any one of claims 1 to 7.

9. The determining means determines a filter strength of the filtering process for each pixel of the raw image data. Coding device according to any one of claims 1 to 8.

10. The determining means determines a filter strength of the filtering process for each block of the raw image data. Coding device according to any one of claims 1 to 8.

11. A coding device according to any one of claims 1 to 10, An imaging means for generating the RAW image data; An imaging device comprising:

12. A method for controlling an encoding device, comprising the steps of: a filtering step for applying a filtering process to the raw image data to reduce high frequency components; an acquisition step of acquiring first feature information indicating brightness and second feature information indicating contrast for the RAW image data; a determination step of determining a filter strength of the filtering process for each pixel of the raw image data based on the first feature information and the second feature information of the raw image data; an encoding step of encoding the raw image data to which the filtering has been applied; Equipped with a filter strength for a first pixel having a first brightness and a first contrast is less than a filter strength for a second pixel having a second brightness greater than the first brightness and the first contrast; a filter strength for a third pixel having a third brightness and a second contrast is greater than a filter strength for a fourth pixel having the third brightness and a third contrast higher than the second contrast; A control method comprising:

13. A program for causing a computer to function as each of the means of the encoding device according to any one of claims 1 to 10.

Citation Information

Patent Citations

  • High-efficiency encoding device

    JP1991192981A

  • Picture transmission equipment

    JP1995154792A

  • Image processor, electronic camera, and image processing program

    JP2003125209A

  • Image processing apparatus, image processings system, compression processing method, storage medium, and program

    JP2003219366A