Image processing device, image processing method and imaging apparatus

JP2023074897A5Active Publication Date: 2024-11-21CANON KK
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2021188074
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2021-11-18
Publication Date
2024-11-21
Estimated Expiration
2041-11-18

AI Technical Summary

Technical Problem

Existing image processing systems for electronic viewfinders (EVFs) exhibit differences in resolution and noise levels when switching between display modes, leading to inconspicuous image discrepancies.

Method used

The system applies gradation characteristics and noise reduction processes tailored to specific display modes, ensuring consistent image quality by adjusting gradation, noise, and sharpness to minimize perceptual differences.

Benefits of technology

This approach generates display image data that reduces noticeable discrepancies between display modes, enhancing user experience and usability by maintaining consistent resolution and noise levels.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

To provide an image processing device and image processing method which can generate display image data in which a difference in images between display modes is inconspicuous.SOLUTION: An image processing device for generating display image data according to a display mode from image data obtained by imaging. The image processing device, in a case where the gradation characteristics applied to the image data in a first display mode are the gradation characteristics that darken an image in comparison to the gradation characteristics applied to the image data in a second display mode, makes the intensity of noise reduction processing and sharpness processing applied to the image data different from the intensity in the second display mode.SELECTED DRAWING: Figure 5
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, an image processing method, and an imaging apparatus.

Background Art

[0002] In an electronic viewfinder (EVF), unlike an optical viewfinder, a processed image can be displayed. Therefore, for example, by displaying an image reflecting the current shooting conditions on the EVF, there is an advantage that the image obtained by shooting can be confirmed before shooting. Patent Document 1 discloses applying processing to enhance the visibility of the image displayed on the EVF.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When it is possible to select a plurality of display modes in which the processed image to be displayed on the EVF is different, such as whether to process the image or the type of processing, there may be a difference in the resolution of the displayed image, the amount of noise, etc. before and after switching the display mode.

[0005] An object of the present invention is to improve such problems of the prior art. In one aspect of the present invention, there is provided an image processing apparatus and an image processing method capable of generating display image data in which the difference in the image between display modes is not noticeable.

Means for Solving the Problems

[0006] The above objective is achieved by an image processing apparatus comprising: an acquisition means for acquiring image data obtained by shooting; and a generation means for generating display image data according to a display mode from the image data, wherein the generation means, in generating display image data, applies gradation characteristics according to a display mode to the image data, and if the gradation characteristics applied to the image data for a first display mode are gradation characteristics that darken the image more than the gradation characteristics applied to the image data for a second display mode, the noise reduction processing applied to the image data is made weaker than the noise reduction processing applied to the image data for a second display mode, and / or the sharpness processing applied to the image data is made stronger than the sharpness processing applied to the image data for a second display mode. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide an image processing apparatus and an image processing method that can generate display image data in which the differences between display modes are not noticeable. [Brief explanation of the drawing]

[0008] [Figure 1] Block diagram showing an example of the functional configuration of a digital camera as an example of an image processing apparatus according to the embodiment. [Figure 2] A schematic block diagram showing the image data generation process for display in the embodiment. [Figure 3] This figure shows an example of the input / output characteristics of RAW data and the tonal values ​​after development processing. [Figure 4A] This figure shows an example of the grayscale characteristics used in the setting priority mode. [Figure 4B] This figure shows an example of the grayscale characteristics used in visibility priority mode. [Figure 4C] A diagram comparing the gradation characteristics used in setting priority mode and the gradation characteristics used in visibility priority mode. [Figure 5] Flowchart relating to noise level adjustment process in the embodiment [Figure 6]Figure showing a modified example that limits the luminance range to which noise reduction processing is applied. [Figure 7] Flowchart for noise level adjustment process in another modified example [Modes for carrying out the invention]

[0009] The present invention will be described in detail below with reference to the attached drawings, based on exemplary embodiments thereof. Note that the following embodiments do not limit the invention to the claims. Furthermore, while multiple features are described in the embodiments, not all of them are essential to the invention, and the multiple features may be combined arbitrarily. In addition, in the attached drawings, the same or similar configurations are given the same reference numeral, and redundant descriptions are omitted.

[0010] The following description will focus on a configuration in which the present invention is applied to the EVF display of a digital camera. However, the present invention can also be implemented in any electronic device capable of generating display images. Such electronic devices include not only imaging devices, but also computer equipment (personal computers, tablet computers, media players, PDAs, etc.), mobile phones, smartphones, game consoles, dashcams, head-mounted display devices (HMDs), and the like. These are merely examples, and the present invention can be implemented in other electronic devices as well.

[0011] ■Digital camera configuration Figure 1 is a block diagram showing an example of the functional configuration of a digital camera (hereinafter referred to as "camera") 100, which is an example of an image processing apparatus according to an embodiment of the present invention.

[0012] The control unit 101 is a general term for the input devices (buttons, switches, dials, etc.) provided for the user to input various instructions to the camera 100. The input devices that make up the control unit 101 have names according to the function they are assigned to. For example, the control unit 101 includes a shutter release switch, a video recording switch, a shooting mode selection dial for selecting a shooting mode, a menu button, directional keys, and an OK key.

[0013] The release switch is a switch for still image recording, and the control circuit 104 recognizes the half-pressed state of the release switch as an instruction for preparation for shooting and the fully pressed state as an instruction for starting shooting. Also, when the video recording switch is pressed while in the shooting standby state, the control circuit 104 recognizes it as an instruction for starting video recording, and when it is pressed during video recording, the control circuit 104 recognizes it as an instruction for stopping recording.

[0014] Note that the functions assigned to the same input device may be variable. Also, the input device may be a software button or key using a touch display. Further, the operation unit 101 may include an input device corresponding to a non-contact input method such as voice input or gaze input.

[0015] The lens unit 102 has a plurality of optical lenses including one or more movable lenses and an aperture. The movable lenses are a focus lens, an anti-shake lens, a zoom lens, etc. The lens unit 102 also has a configuration for driving the movable lenses and the aperture, such as a motor and an actuator. In this embodiment, the lens unit 102 is integrated with the camera 100, and the operation of the lens unit 102 is controlled by the control circuit 104.

[0016] In the case of an interchangeable lens to which the lens unit 102 is detachable, a control circuit is provided in the lens unit 102. Then, the operation of the lens unit 102 is controlled by the control circuit 104 through communication with the control circuit in the lens unit 102.

[0017] The imaging device 103 may be a known CCD or CMOS color image sensor having a color filter of a primary color Bayer array, for example. The imaging device 103 has a pixel array in which a plurality of pixels are two-dimensionally arranged and a peripheral circuit for reading signals from the pixels. Each pixel accumulates charges corresponding to the incident light amount by photoelectric conversion. By reading out a signal having a voltage corresponding to the amount of charges accumulated during the exposure period from each pixel, a group of pixel signals (analog image signals) representing the subject image formed by the lens unit 102 on the imaging surface is obtained.

[0018] The control circuit 104 has one or more processors capable of executing a program. The control circuit 104 realizes the functions of the camera 100 by, for example, loading a program stored in the non-volatile memory 107 into the working memory 108 and executing it to control the operations of each functional block. In the following description, a part of the operations executed by the control circuit 104 may be executed by a dedicated circuit such as an ASIC.

[0019] The control circuit 104 applies predetermined image processing to the analog image signal read from the imaging device 103, generates image data for display or recording, detects feature regions, and acquires and / or generates various evaluation values.

[0020] Here, the image processing applied by the control circuit 104 includes preprocessing, color interpolation processing, correction processing, detection processing, data processing, evaluation value calculation processing, special effect processing, and the like. The preprocessing includes A / D conversion, signal amplification, reference level adjustment, defective pixel correction, and the like. The color interpolation processing is a process of interpolating the values of color components that cannot be obtained during shooting, and is also called demosaicing processing. The correction processing includes white balance adjustment, tone correction, correction of image degradation caused by optical aberrations of the lens unit 102 (image restoration), correction of the influence of peripheral light reduction of the lens unit 102, color correction, and the like. The detection processing includes detection of feature regions (such as face regions and human body regions) and their movements, and human recognition processing. The data processing includes processes such as synthesis, scaling, encoding and decoding, and generation of header information (data file generation). The evaluation value calculation processing includes processes such as generation of signals and evaluation values used for autofocus detection (AF), and generation of evaluation values used for automatic exposure control (AE). The special effect processing includes processes such as addition of a blur effect, change of color tone, and relighting. Note that these are examples of the processing applicable by the control circuit 104, and do not limit the processing applied by the control circuit 104.

[0021] The control circuit 104 can also perform AF processing and AE processing based on the generated evaluation values.

[0022] The non-volatile memory 107 is electrically rewritable. The non-volatile memory 107 stores the program executed by the control circuit 104, the settings of the camera 100, GUI (Graphical User Interface) data, and other similar information.

[0023] The working memory 108 is what is known as RAM (Random Access Memory). The working memory 108 is used to load programs executed by the control circuit 104 and to temporarily store variable values ​​and other information during program execution. The working memory 108 is also used as an image data buffer and as video memory for the display device 105.

[0024] The display device 105 may be, for example, a liquid crystal display or an organic EL display. The display device 105 functions as an electronic viewfinder (EVF). The display device 105 may be provided inside the housing of the camera 100, on the surface of the housing, or both. If the display device 105 is provided inside the housing of the camera 100, the camera 100 has an eyepiece for looking at the display device 105.

[0025] Note that the display device 105 may be an external device to the camera 100. If the display device 105 is an external device, the camera 100 outputs signals or data in a format that can be displayed by the connected display device 105.

[0026] The storage unit 106 is, for example, a semiconductor memory card. Recordable image data (video data or still image data) is recorded in the storage unit 106 as a data file in a predetermined format by the control circuit 104.

[0027] The control circuit 104 continuously performs the operations of shooting video, generating display image data from the video obtained from the shooting, and displaying it on the display device 105, thereby enabling the display device 105 to function as an EVF (Electronic Viewfinder).

[0028] In this embodiment, the EVF display mode can be selected from a setting priority mode, which applies processing to the display image data to reflect the current settings, and a visibility priority mode, which applies processing to the display image data to improve the visibility of the shooting scene. The display mode may be selected by the user, for example, through the operation unit 101, or it may be automatically selected by the control circuit 104 according to the conditions.

[0029] The Settings Priority mode displays an image in the EVF that reflects the current settings for processing (e.g., contrast correction, exposure correction, saturation correction, etc.) applied to the image obtained through shooting. This mode allows you to preview the state of the image that will be recorded if you give instructions at that moment.

[0030] Furthermore, the Visibility Priority mode displays an image in the EVF with visibility close to what you would see directly with your eyes, without reflecting the current settings for processing the image obtained through shooting. Visibility Priority mode allows you to use the EVF instead of observing the scene with the optical viewfinder or your eyes.

[0031] The EVF display mode can be changed at any time by the user, for example, through the operation of the control unit 101. Therefore, the user can change the display mode as needed while viewing the EVF. Note that other display modes may also be included as EVF display modes.

[0032] The control circuit 104 controls the process for generating display image data from RAW data according to the display mode. RAW data is data in which the signal or data of each pixel has a single color component value corresponding to the color filter. Analog image signals read from the image sensor 103 and digital image signals (image data) obtained by A / D conversion of analog image signals are considered RAW data.

[0033] When the image sensor 103 has a primary color Bayer array color filter, the signal or data of each pixel constituting the RAW data has a value for one of the color components: R (red), G (green), or B (blue). The process of converting the RAW data into a state where the signal or data of each pixel has values ​​for multiple color components necessary to represent a color image is called development. Development includes color interpolation.

[0034] ■Generation process of image data for EVF display The development process that the control circuit 104 applies to RAW data to generate image data for EVF display will be explained using Figure 2. In Figure 2, the series of development processes applied by the control circuit 104 are represented as processes realized by multiple functional blocks 202 to 204 and 206.

[0035] As described above, each pixel data constituting the RAW data 201 has only a value representing the intensity of one color component according to the color array of the color filter of the image sensor 103. In this embodiment, it is assumed that the image sensor 103 is provided with a primary color Bayer array color filter. In this case, each pixel data of the RAW data 201 has a value representing the intensity of one of the color components R (red), G (green), or B (blue), and does not have values ​​for the remaining two color components.

[0036] The white balance unit 202 performs white balance adjustment to correct color cast caused by the light source so that white subjects appear white in the image. Specifically, the white balance unit 202 plots the RGB data of each pixel that makes up the RAW data 201 to coordinates in a predetermined color space (e.g., xy color space). Then, the white balance unit 202 integrates the R, G, and B values ​​of the pixel data plotted near the blackbody radiation trajectory, which is likely to be the color of the light source in that color space, for each color component.

[0037] The white balance unit 202 calculates the white balance coefficient G / R to be applied to the R component and the white balance coefficient G / B to be applied to the B component from the integral values ​​of each color component. The white balance unit 202 performs white balance adjustment by applying the white balance coefficients to the R component and the B component.

[0038] The color interpolation unit 203 compensates for the color component values ​​that individual pixel data lacks through noise reduction processing and color interpolation processing. Through color interpolation processing, each pixel data comes to have the three components R, G, and B, and the image data comes to represent a color image.

[0039] Subsequently, the image data is converted from RGB format to YUV format by the matrix conversion unit 204. Furthermore, the color and brightness components are adjusted by the color and brightness adjustment unit 206 based on the color and brightness adjustment parameters 205. The color and brightness adjustment parameters 205 are determined, for example, based on user settings via the operation unit 101.

[0040] The noise reduction unit 307 applies noise reduction and sharpness processing according to the EVF display mode and shooting scene to adjust the noise level and resolution of the image data. Details of the operation of the noise reduction unit 307 will be described later.

[0041] The tone conversion unit 309 uses the tone characteristic adjustment curve (gamma curve) 307 acquired by the tone characteristic acquisition unit 308 to apply tone conversion to the image data and generate display image data. The tone conversion unit 309 applies different tone conversion characteristics depending on the EVF display mode and shooting scene. Details of the tone conversion characteristics applied by the tone conversion unit 309 will be described later.

[0042] The display image data 207 is converted back to RGB format as needed, depending on the type of display device 105, and then displayed.

[0043] When shooting bright scenes such as daytime in clear skies, setting the maximum display brightness of the display device 105 to high brightness reduces the difference between what is seen on the display device 105 (EVF) and what is seen directly with the eye. On the other hand, when shooting dark scenes such as nightscapes, setting the maximum display brightness of the display device 105 to low brightness reduces the difference between what is seen on the display device 105 (EVF) and what is seen directly with the eye.

[0044] Therefore, in this embodiment, the control circuit 104 automatically changes the maximum display brightness of the display device 105 according to the brightness of the scene being photographed, thereby reducing the difference between how it looks with the EVF and how it looks when viewed directly with the eye. The brightness of the scene can be determined based on the brightness evaluation value for AE processing generated by the control circuit 104. Alternatively, a separate sensor for detecting the brightness of the scene may be provided, and the control circuit 104 may make the determination based on the brightness obtained from this sensor.

[0045] ■ Differences in appearance depending on the display mode Next, we will explain the difference in appearance between "setting priority mode" and "visibility priority mode." Here, we consider the case where the input / output characteristic L41 shown in Figure 3 is selected in the operation unit 101 and the display image data 207 is displayed on the display device 105. Here, we assume that the RAW data is a 14-bit value and the gradation is an 8-bit value.

[0046] Figures 4A to 4C show the input / output characteristics, with the logarithm of the input RAW data on the horizontal axis and the I value of the output (details will be described later) on the vertical axis. The logarithm of the RAW data is normalized by the signal value corresponding to proper exposure, and expressed as the number of stops, with the signal value corresponding to proper exposure set to 0 stops.

[0047] The I value is the value of the luminance component I in the coordinate values ​​of the ICtCp color space, which is a perceptually uniform color space defined by ITU-R BT.2100. As defined in ITU-R BT.2100, the photoelectric transfer function (OETF) is applied during the conversion process from RGB values ​​to ICtCp values.

[0048] In this embodiment, the inverse EOTF (Electro-Optical Transfer Function) of the PQ (Perceptual Quontization) method, standardized in SMPTE ST2084, is used as the OETF. In other words, the I value in the figure corresponds to the display brightness of the display device 105, and can be converted to the display brightness of the display device 105 by applying the PQ EOTF.

[0049] Figure 4A shows an example of the conversion characteristics between RAW data (RGB values) and I values ​​in "Setting Priority Mode". L51 shows an example of the characteristics during daytime in clear weather, and L52 shows an example of the characteristics during nighttime scenes. In this embodiment, the I values ​​corresponding to the maximum display brightness of the display device 105 are set to approximately 680 during daytime in clear weather and approximately 450 during nighttime scenes. These are examples, and other combinations of values ​​where the I value is larger during daytime in clear weather than during nighttime scenes may also be used.

[0050] Figure 4B shows an example of the conversion characteristics between RAW data (RGB values) and I values ​​in "Visibility Priority Mode". L63 shows an example of the characteristics in bright daylight, and L64 shows an example of the characteristics in nighttime scenes. Characteristic L63 is a straight line L61 with a slope of approximately 75, where the I value at proper exposure is approximately the same as characteristic L51, but with the slope made gentler in the dark and saturated areas. Characteristic L64 is a straight line L62 with a slope of approximately 75, where the I value at maximum display brightness is approximately the same as characteristic L52, but with the slope made gentler in the dark and saturated areas.

[0051] The slope of the linearly changing portion of characteristics L63 and L64 (where the straight lines L61 and L62 are used as is) is approximately 75, and this is common to both characteristics. It is generally known that human sensory characteristics are proportional to the logarithmic intensity of a particular stimulus value. Therefore, characteristics L63 and L64, which are created based on the straight lines L61 and L62 having the same slope, can be said to satisfy a perceptually uniform sense of contrast. Note that although the slope of straight lines L61 and L62 is set to approximately 75, it can be selected within a range of, for example, 65 to 85.

[0052] The gradation conversion unit 309 applies gradation characteristics according to characteristic L51 or L52 to the image data when the EVF display mode is set priority mode. Furthermore, when the EVF display mode is set to visibility priority mode, the gradation conversion unit 309 applies gradation characteristics according to characteristic L63 or L64 to the image data.

[0053] The L63 characteristic has an I-value at proper exposure that is almost the same as the L51 characteristic, and there is no significant difference in I-value between the L51 and L63 characteristics in the brightness range surrounding the proper exposure. Therefore, in situations where the characteristics for sunny daytime are used, the brightness of the image hardly changes even if the EVF display mode is switched.

[0054] Figure 4C shows the characteristics L52 and L64 for "Setting Priority Mode" and "Visibility Priority Mode" during nighttime scenes. The characteristic L64 for "Visibility Priority Mode" during nighttime scenes has an I value approximately 100 lower at proper exposure (0 stops) than the characteristic L52 for "Setting Priority Mode" during nighttime scenes. This is because Visibility Priority Mode displays images that are closer to how they actually appear. If the I values ​​at proper exposure were the same, the display in Visibility Priority Mode would give the impression of being brighter than what actually appears.

[0055] Thus, when applying gradation characteristics that result in a darker image compared to other display modes, visual differences may arise between display modes beyond just apparent brightness. This point will be explained further.

[0056] ■ Adjustment of noise level and resolution in "Visibility Priority Mode" In this embodiment, the display brightness in night scenes in "visibility priority mode" is reduced by the gradation characteristics applied after the development process. This is because adjusting the image to be darker during the development process can lead to a decrease in the accuracy of evaluation values ​​and detection values ​​based on the developed image, and consequently, the accuracy of autofocus and automatic correction processing.

[0057] On the other hand, darkening the image after development reduces the amplitude of noise and shooting, thus changing the perceived noise and resolution of the image. Specifically, when switching between "Setting Priority Mode" and "Visibility Priority Mode" when shooting night scenes, the resolution of the EVF display in "Visibility Priority Mode" may appear poor, and it may become difficult to determine the peak of focus when using manual focus (MF). Also, while "Visibility Priority Mode" requires an EVF display that closely resembles what is actually seen, the resolution of the display may appear worse than what is seen directly with the eye when shooting night scenes.

[0058] In view of these points, in this embodiment, the noise adjustment unit 307 is used to make the perceived resolution and noise level of the EVF display in visibility priority mode similar to that of the EVF display in setting priority mode. Specifically, in the EVF display during night scenes in "visibility priority mode," the amount of noise in the image is adjusted by noise processing or sharpness processing.

[0059] The operation of the noise adjustment unit 307 and the grayscale conversion unit 309 will be explained below using the flowchart in Figure 5. In S801, the noise adjustment unit 307 checks whether the EVF display mode is "setting priority mode" or "visibility priority mode". The current EVF display mode can be checked, for example, by referring to the non-volatile memory 107 or the working memory 108. If the setting priority mode is selected, the noise adjustment unit 307 executes S802; if the visibility priority mode is selected, it executes S806.

[0060] In the setting priority mode, the noise adjustment unit 307 sequentially applies noise reduction (NR) processing with NR intensity A in S802, sharpness processing with sharpness intensity A in S803, and manual focus peaking detection processing using peaking threshold A in S804 to the image data. The intensity and threshold of the processing applied in S802 to S804 are determined according to shooting conditions such as exposure and sensitivity, and are not affected by the image display conditions (such as the EVF display mode and display brightness).

[0061] In S805, the tone conversion unit 309 applies tone conversion A to the image data. Tone conversion A is a tone conversion according to characteristics L51 or L52, for example, shown in Figure 4A. The tone conversion unit 309 can, for example, select characteristic L51 for daytime sunny days if the evaluation value of the scene brightness obtained from the image data (e.g., Bv value) is higher than a threshold, or characteristic L52 for nighttime scenes if it is below the threshold. The evaluation value can be obtained by the noise adjustment unit 307, the tone conversion unit 309, or by other processing performed by the control circuit 104.

[0062] In S806, the noise adjustment unit 307 determines whether the brightness of the scene exceeds a threshold. As an example, if the Bv value is greater than the threshold, the noise adjustment unit 307 determines that the brightness of the scene exceeds the threshold and executes S807; if the Bv value is less than or equal to the threshold, it executes S811.

[0063] The objective here is to distinguish between a daytime, sunny scene and a nighttime scene, so we determine thresholds that allow us to differentiate between these scenes. If necessary, different thresholds may be used for each scene.

[0064] In bright scenes, the noise adjustment unit 307 sequentially applies noise reduction (NR) processing with NR intensity A in S807, sharpness processing with sharpness intensity A in S808, and MF peaking detection processing using peaking threshold A in S809 to the image data. These processes are the same as those in S802-S804 in the settings priority mode. In other words, in bright scenes, the adjustment of noise level and perceived resolution is the same in both the visibility priority mode and the settings priority mode.

[0065] In S810, the tone conversion unit 309 applies tone conversion B to the image data. Tone conversion B is a tone conversion according to characteristics L63 and L64, for example, as shown in Figure 4B. If the tone conversion unit 309 determines in S805 that the Bv value is greater than the threshold, it applies tone conversion according to characteristic L63 for daytime sunny conditions, and if it determines that the Bv value is less than or equal to the threshold, it applies tone conversion according to characteristic L64 for nighttime scenes.

[0066] If the Bv value is determined to be below the threshold in S806, the noise adjustment unit 307 sequentially applies NR processing of NR intensity B in S807, sharpness processing of sharpness intensity B in S808, and MF peaking detection processing using peaking threshold B in S809 to the image data.

[0067] In the NR processing in S807, to retain more noise than in bright scenes and to suppress the decrease in resolution caused by noise reduction processing, NR intensity B is made weaker than NR intensity A. Also, in the sharpness processing in S808, to increase the sense of resolution compared to bright scenes, sharpness intensity B is made stronger than sharpness intensity A.

[0068] Furthermore, in the peaking process of the S809, peaking threshold B is set higher than peaking threshold A to make the peak of focus easier to see than in bright scenes, thereby narrowing the range of values ​​that are judged to be the peak.

[0069] Thus, when the EVF display mode is set to "Visibility Priority Mode" and a tonal characteristic that darkens the image at proper exposure is applied, processing is applied that increases the perceived noise and resolution compared to when a tonal characteristic that does not darken the image at proper exposure is applied. This prevents users from feeling that the resolution of the EVF display has deteriorated or that it has become difficult to determine the peak of focus when they switch the display mode from the setting priority mode to the visibility priority mode.

[0070] Furthermore, in visibility priority mode, the application of NR processing and sharpness processing with NR intensity B and sharpness intensity B applied in S811 and S812 can be limited to a specific display brightness range.

[0071] The application of NR processing for NR intensity B will be explained. In Figure 6(a), L91 shows the difference between the characteristics L52 of the "setting priority mode," which is used when the shooting scene is not bright, and the characteristics L64 of the "visibility priority mode."

[0072] Furthermore, it is possible to apply NR processing of NR intensity B to luminance ranges where the difference in I values ​​(i.e., display brightness) between display modes represented by L91 is greater than or equal to a threshold, and to apply NR processing of NR intensity A to other luminance ranges.

[0073] Figure 6(b) shows an example of the relationship between brightness and NR intensity when NR intensity B is used for brightness ranges where the I-value difference of the gradation characteristics is 50 or more, and NR intensity A is used for other brightness ranges. Here, L101 shows the characteristics of NR intensity A used in night scenes in "setting priority mode". Also, L102 shows an example of the characteristics of brightness ranges where NR intensity B is used in night scenes in "visibility priority mode".

[0074] The luminance range in which characteristic L91 shown in Figure 6(a) is 50 or higher corresponds to approximately -3 stops to +2.5 stops on the logarithmic scale of the RAW data on the horizontal axis. Therefore, for the NR processing characteristics, characteristic L102, which is NR intensity B, is used, which weakens the NR intensity to approximately 0.8 times that of NR intensity A shown in characteristic L101 in the same luminance range.

[0075] Similarly, for sharpness processing, sharpness intensity B can be applied only to luminance ranges where the difference in I-values ​​of the gradation characteristics is greater than or equal to a threshold, while sharpness intensity A can be applied to all other luminance ranges.

[0076] Here, we have described an example of adjusting the perceived noise and resolution by limiting the adjustment to a luminance range where the difference in I-values ​​between the input / output characteristics of RAW data (luminance of image data) and I-values ​​(values ​​corresponding to display luminance) is above a threshold. However, the luminance range for adjusting the perceived noise and resolution may be determined by other methods. For example, the target luminance range may be determined by other methods based on I-values, such as using the luminance range corresponding to the I-value of the mean ± standard deviation. Alternatively, it may be determined based on something other than the I-value, such as scene luminance or display luminance.

[0077] Furthermore, the noise reduction (NR) processing using NR intensity B in S811 and the sharpness processing using sharpness intensity B in S812 may be applied to the entire brightness range.

[0078] (modified version) In the flowchart in Figure 5, when the visibility priority mode is set, the intensity of NR processing and sharpness processing, as well as the threshold used in MF peaking detection processing, are changed according to the brightness of the shooting scene. However, these changes may also be made according to other conditions.

[0079] For example, S806 could be used to determine whether or not high resolution is required in the EVF display. If it is determined that high resolution is required in the EVF display, S811 could be executed; otherwise, S807 could be executed. A situation where high resolution is required in the EVF display is, for example, when the focus mode is set to manual focus (MF).

[0080] In this case, the gradation characteristics applied by the S810 when it is determined that high resolution is required in the EVF display do not necessarily have to be gradation characteristics that darken the image, such as characteristic L64. For example, the same gradation characteristics may be applied whether or not high resolution is required in the EVF display, and only the intensity of NR processing and sharpness processing may differ.

[0081] Furthermore, in the flowchart in Figure 5, when the shooting scene is not bright in "Visibility Priority Mode," three settings are changed: NR intensity, sharpness intensity, and peaking threshold. However, it is not always necessary to change all of these.

[0082] For example, the effects of the present invention can be obtained by changing one or more of these parameters, such as changing only the NR intensity. Figure 7 is a flowchart showing the details of the operation of the noise adjustment unit 307 and the grayscale conversion unit 309 when only the NR intensity is changed, similar to Figure 5. In Figure 7, blocks that perform the same processing as in Figure 5 are given the same reference numerals as in Figure 5.

[0083] If the noise adjustment unit 307 determines in S806 that the Bv value is below the threshold, it performs NR processing of NR intensity B in S811. Subsequently, the noise adjustment unit 307 executes S808 and S809, applying the same sharpness processing and MF peaking detection processing as when the Bv value was not determined to be below the threshold in S806.

[0084] Thus, even when changing only one of the NR intensity or sharpness intensity, the luminance range to which it is applied may be the entire range or a specific part of it.

[0085] As described above, according to this embodiment, in an image processing device that generates display image data by applying gradation characteristics according to the display mode, when applying gradation characteristics that reduce the perceived resolution of the image, a process is applied to enhance the perceived resolution of the image. Therefore, it is possible to suppress differences in perceived resolution of the image due to differences in display modes, and to suppress changes in usability due to display modes.

[0086] (Other embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0087] The present invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of Symbols]

[0088] 100...Digital camera, 101...Operation unit, 104...Control circuit, 105...Display device, 107...Non-volatile memory, 108...Working memory

Claims

1. an acquisition means for acquiring image data obtained by photographing; generating means for generating display image data according to a display mode from the image data; The generating means In generating the display image data, a gradation characteristic corresponding to the display mode is applied to the image data; The gradation characteristics applied to the image data in the first display mode are gradation characteristics that darken the image compared to the gradation characteristics applied to the image data in the second display mode. applying a weaker noise reduction process to the image data than the noise reduction process applied to the image data for the second display mode; and / or sharpening the image data to be displayed in the second display mode; 1. An image processing device comprising:

2. The generating means further comprises: The gradation characteristics applied to the image data in the first display mode are gradation characteristics that darken the image compared to the gradation characteristics applied to the image data in the second display mode. a threshold value in a manual focus peaking detection process applied to the image data is set to be larger than a threshold value in a manual focus peaking detection process applied to the image data for the second display mode; 2. The image processing device according to claim 1, wherein:

3. 3. The image processing device according to claim 1, wherein a gradation characteristic that darkens the image compared to the gradation characteristic applied to the image data in the second display mode is applied to the image data when the brightness of the scene being photographed in the first display mode is below a threshold.

4. The generating means further comprises: When manual focus is set in the first display mode, the image is displayed more clearly than when manual focus is not set. Applying less noise reduction to the image data; and / or Increasing the sharpness processing applied to the image data; 4. The image processing device according to claim 1, wherein the image processing device is a computer.

5. 5. The image processing device according to claim 1, wherein the generating means makes the intensity of the noise reduction processing and / or the sharpness processing in the first display mode different from the intensity of the noise reduction processing and / or the sharpness processing in the second display mode for a specific brightness range of the image data.

6. 6. The image processing apparatus according to claim 1, wherein the gradation characteristics applied to the image data are gradation characteristics that represent the relationship between a logarithmic value of the image data and an I value in an ICtCp color space.

7. The image processing device described in claim 5, characterized in that the gradation characteristics applied to the image data are gradation characteristics that represent the relationship between the logarithmic value of the image data and the I value in the ICtCp color space, and the specific luminance range is a luminance range in which the difference between the gradation characteristics applied to the image data for the second display mode and the I value is greater than or equal to a threshold value.

8. 8. The image processing device according to claim 7, wherein the gradation characteristic applied to the image data for the first display mode is a gradation characteristic in which the I value changes linearly with respect to a change in the logarithmic value of the image data in a specific luminance range including a proper exposure.

9. An imaging element; an image processing device according to any one of claims 1 to 8, which processes image data obtained by photographing using the image sensor; An imaging device comprising:

10. 10. The imaging device according to claim 9, wherein the display image data generated by the image processing device is used to display on a display device that functions as an electronic viewfinder.

11. Further comprising an operating means, 11. The imaging device according to claim 9, wherein the first display mode and the second display mode are switchable in response to an operation of the operating means.

12. An image processing method executed by an image processing device, Acquiring image data obtained by photographing; generating display image data according to a display mode from the image data; The generating step comprises: generating the image data for display includes applying noise reduction processing, sharpness processing, and gradation characteristics according to the display mode to the image data; The gradation characteristics applied to the image data in the first display mode are gradation characteristics that darken the image compared to the gradation characteristics applied to the image data in the second display mode, the noise reduction processing applied to the image data is weaker than the noise reduction processing applied to the image data for the second display mode; and / or the sharpness processing applied to the image data is stronger than the sharpness processing applied to the image data for the second display mode; An image processing method comprising:

13. A program for causing a computer to function as each of the means included in the image processing device according to any one of claims 1 to 8.