Image processing apparatus, image processing method, and program

Abstract

To provide a technique capable of achieving desired display luminance even in a display mode in which smoothness is given priority.SOLUTION: An image processing apparatus according to the present invention includes an acquisition unit configured to acquire captured image data captured by an imaging unit, a setting unit configured to set any one of a plurality of display modes including a first display mode and a second display mode in which an upper limit of an exposure time of the imaging unit is shorter than that in the first display mode, a control unit configured to control the exposure time based on luminance of the captured image data, a processing unit configured to generate display image data to be displayed on a display unit based on the captured image data, and a display control unit configured to display the display image data on the display unit. In the second display mode, when the exposure time reaches the upper limit, at least one of the processing means and the display control means performs luminance adjustment processing for increasing the display luminance of the display image data on the display unit to display luminance substantially equal to the display luminance in the first display mode.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an image processing device, an image processing method, and a program, and more particularly to a technique for improving the visibility of a displayed image corresponding to a low-brightness scene. [Background technology]

[0002] A display mode has been proposed for cameras that controls the exposure time of the image sensor (the time it takes for the image sensor to accumulate charge in response to light irradiating the image sensor) so that a live view display (LV display) is performed at the desired display brightness. However, if the exposure time is long, the LV display will not display smooth movements (a jerky movement).

[0003] A display mode has also been proposed that prioritizes smoothness of the LV display and limits the exposure time to a predetermined time or less. However, because the exposure time is limited to a predetermined time or less, when shooting night scenes or using a lens with a dark (large) maximum F-number, the LV display will be darker than the desired display brightness.

[0004] Patent Document 1 discloses a technique for increasing the display brightness in response to a user's instruction, by performing image processing if the surroundings are dark. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-110220 Summary of the Invention [Problem to be solved by the invention]

[0006] However, there is an upper limit to the amount of increase in display luminance due to image processing, and therefore, even if the technique disclosed in Patent Document 1 is used, it may not be possible to achieve the desired display luminance.

[0007] An object of the present invention is to provide a technique that can achieve a desired display brightness even in a display mode that prioritizes smoothness. [Means for solving the problem]

[0008] A first aspect of the present invention is an image processing device comprising: an acquisition means for acquiring captured image data captured by an imaging unit; a setting means for setting one of a plurality of display modes including a first display mode and a second display mode in which the upper limit of the exposure time of the imaging unit is shorter than that of the first display mode; a control means for controlling the exposure time based on the luminance of the captured image data; a processing means for generating display image data to be displayed on a display unit based on the captured image data; and a display control means for displaying the display image data on the display unit, wherein, in the second display mode, when the exposure time reaches its upper limit, at least one of the processing means and the display control means performs a luminance adjustment process to increase the display luminance of the display image data on the display unit to a display luminance approximately equal to the display luminance in the first display mode.

[0009] A second aspect of the present invention includes an acquisition step of acquiring captured image data captured by an imaging unit, a setting step of setting one of a plurality of display modes including a first display mode and a second display mode in which an upper limit of an exposure time of the imaging unit is shorter than that of the first display mode, a control step of controlling the exposure time based on the luminance of the captured image data, a processing step of generating display image data to be displayed on a display unit based on the captured image data, and a display control step of displaying the display image data on the display unit, wherein in the second display mode: This is an image processing method characterized in that, when the exposure time reaches an upper limit, in at least one of the processing step and the display control step, a brightness adjustment process is performed to increase the display brightness of the display image data on the display unit to a display brightness approximately equal to the display brightness in the first display mode.

[0010] A third aspect of the present invention is a program for causing a computer to function as each of the means of the image processing device. [Effects of the Invention]

[0011] According to the present invention, it is possible to achieve a desired display brightness even in a display mode that prioritizes smoothness. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram of a digital camera. [Figure 2] FIG. 2 is a block diagram of an image processing unit. [Figure 3] 10 is a flowchart showing the operation of the digital camera. [Figure 4] 10 is a table showing specific examples of the rate of increase in display luminance. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, an embodiment of the present invention will be described. Note that an example in which the present invention is applied to an imaging device will be described, but the device to which the present invention can be applied is not limited to an imaging device. The present invention can be applied to various electronic devices (image processing devices) that can perform image processing on captured images (captured images). For example, the present invention can be applied to digital cameras, digital video cameras, personal computers, tablet terminals, smartphones, mobile phones, game consoles, video see-through head-mounted displays, etc.

[0014] (Digital Camera) Fig. 1 is a block diagram showing the functional configuration of a digital camera 100 according to this embodiment. Each unit of the digital camera 100 shown in Fig. 1 may be implemented as hardware such as a circuit or a processor, or may be implemented by a program.

[0015] The control unit 101 is composed of at least one processor or circuit, and controls each unit of the digital camera 100. For example, the control unit 101 controls each unit of the digital camera 100 by reading a program from the recording medium 102, expanding it into the memory 103, and executing it.

[0016] The recording medium 102 is a non-volatile storage device, such as a Flash-ROM, that can electrically erase and record information (data). The recording medium 102 stores programs and constants for controlling each part of the digital camera 100. The recording medium 102 may also serve as a recording medium (such as a semiconductor memory card) for storing images obtained by capturing images (RAW data, image data after development processing, etc.).

[0017] The memory 103 is a volatile storage device such as a RAM or DRAM. The memory 103 is used as a storage area for programs that control each unit of the digital camera 100. The memory 103 is also used as a VRAM when displaying images on the display unit 106.

[0018] The lens unit 110 forms an optical image formed by light from a subject on the imaging plane of the lens unit 110. The lens unit 110 may be detachable from the digital camera 100. The lens unit 110 usually includes a plurality of lenses (an imaging lens group). For simplicity, only one lens is shown in 1. Lens unit 110 includes a control circuit (not shown), and this control circuit controls the state of lens unit 110 based on a drive signal input from control unit 101.

[0019] The imaging unit 104 is an imaging element such as a CCD or CMOS sensor, and acquires an analog image signal by converting an optical image formed on an imaging surface by the lens unit 110 into an electrical signal. The acquired analog image signal is converted into a digital image signal (RAW data) by an A / D converter (not shown). In this embodiment, the imaging unit 104 is a single-chip color imaging element equipped with a general primary color filter. The primary color filter has three types of color filters with dominant transmission wavelength bands (wavelength ranges of light that are primarily transmitted) near 650 nm, 550 nm, and 450 nm, respectively, arranged in a mosaic pattern (Bayer array). The imaging unit 104 uses the primary color filter to capture three color planes corresponding to the R (red) band, the G (green) band, and the B (blue) band, respectively. At this time, each photoelectric conversion element of the imaging unit 104 can only obtain light intensity corresponding to a single color plane. Note that the imaging unit 104 is not limited to a single-chip color imaging element. The imaging unit 104 may include peripheral circuits such as an amplifier circuit that processes signals obtained from each photoelectric conversion element.

[0020] The image processing unit 105 performs various image processing such as pixel interpolation, resizing, and color conversion on the RAW data output from the imaging unit 104 or image data read from the recording medium 102. The image processing unit 105 also performs arithmetic processing using the RAW data obtained by imaging to obtain information necessary for exposure control and distance measurement control. Based on the information obtained by the image processing unit 105, the digital camera 100 performs TTL (through-the-lens) AF (autofocus) processing, AE (autoexposure) processing, and EF (flash pre-flash) processing. TTL AWB (auto white balance) processing and the like are also performed.

[0021] Display unit 106 is a display device such as a liquid crystal display, and displays GUIs such as settings and messages of digital camera 100, menu screens, captured images, etc. Display unit 106 may be, for example, an electronic viewfinder (EVF) or a rear LCD display provided in digital camera 100, or an external display connected to digital camera 100. The electronic viewfinder, rear LCD display, and external display are all capable of live view display (LV display), which shows a subject in approximately real time.

[0022] The display unit 106 includes a display control circuit (not shown), which can change the maximum display luminance (upper limit display luminance) of the display unit 106 based on a drive signal input from the control unit 101. The maximum display luminance of the display unit 106 may be dynamically controlled in accordance with a Bv value (luminance value) obtained by photometry of the image capture scene, or may be controlled in accordance with the image capture mode. The photometry of the image capture scene may be performed based on an image signal obtained by capturing an image, or may be performed based on the output of a photometric sensor provided separately from the image capture unit 104.

[0023] The operation unit 107 is a user interface that accepts various user operations. When the operation unit 107 detects a user operation, it outputs a control signal corresponding to the user operation to the control unit 101. The operation unit 107 includes a release switch for issuing an instruction to start a shooting preparation operation and to start shooting (actual shooting), a mode selection switch for selecting an imaging mode or a display mode, direction keys, an enter key, and the like.

[0024] (Image processing unit) 2 is a block diagram showing the functional configuration of the image processing unit 105. The image capturing unit 104 has three types of The image processing unit 105 reads the RAW data 201 from the memory 103, performs development processing on the RAW data 201, and generates display image data 208.

[0025] The white balance unit 202 performs white balance processing on the RAW data 201, which is a color conversion process that enhances the reproduction of the white color of the subject. The white balance unit 202 plots each RGB data included in the RAW data 201 in a predetermined color space, such as an xy color space. The RGB data is data obtained from three or more photoelectric conversion elements corresponding to the three RGB color planes. The white balance unit 202 integrates the R, G, and B values ​​of the RGB data plotted near the locus of blackbody radiation, which is likely to be the light source color. The white balance unit 202 then calculates white balance coefficients for the R and B values ​​(G integral value / R integral value and G integral value / B integral value) from the calculated integrated values ​​and corrects each RGB data using the white balance coefficients. This reduces color casts caused by the light source and enhances the reproduction of the white color of the subject.

[0026] The color interpolation unit 203 performs noise reduction and color interpolation on the image data after white balance processing. Generally, the higher the ISO sensitivity of the imaging unit 104, the greater the amount of random noise. Therefore, the higher the ISO sensitivity, the stronger the noise reduction processing. The color conversion process is a process for obtaining values ​​of color components not included in pixel data (data corresponding to a single color plane obtained from one photoelectric conversion element). The color conversion process generates image data including R, G, and B values ​​for all pixels.

[0027] The matrix conversion unit 204 performs matrix conversion processing on the image data generated by the color interpolation unit 203. As a result, general color image data is obtained.

[0028] The color and brightness adjustment unit 206 performs color and brightness adjustment processing to adjust at least one of color and brightness on the color image data generated by the matrix conversion unit 204. The color and brightness adjustment processing includes, for example, contrast correction, exposure correction, saturation correction, sharpness correction, etc., which are performed using color and brightness adjustment parameters 205.

[0029] The display conversion processing unit 207 generates display image data 208 by converting at least one of the color gamut and gamma characteristics of the image data after the color luminance adjustment processing so that the color gamut and gamma characteristics match those of the display unit 106 .

[0030] The display image data 208 thus generated (an image corresponding to the display image data 208) is displayed on the display unit 106, thereby realizing LV display.

[0031] (display mode) In this embodiment, the control unit 101 sets one of a plurality of display modes including a visibility priority mode and a smoothness priority mode as the display mode for the LV display. The visibility priority mode is a display mode that prioritizes the visibility of the LV display and is a display mode in which the upper limit of the exposure time of the imaging unit 104 is relatively long. The smoothness priority mode is a display mode that prioritizes the smoothness of the LV display and is a display mode in which the upper limit of the exposure time of the imaging unit 104 is shorter than in the visibility priority mode. The plurality of settable display modes may include display modes other than the visibility priority mode and the smoothness priority mode.

[0032] ((Visibility priority mode)) In the visibility priority mode, the control unit 101 determines at least the ISO sensitivity and exposure time of the image capturing unit 104 based on the luminance of the captured image data (for example, RAW data) captured by the image capturing unit 104. The captured image data is not limited to RAW data. For example, the captured image data may be image data that has undergone white balance processing, noise reduction processing, color interpolation processing, matrix conversion processing, contrast correction, exposure correction, saturation correction, sharpness correction, color gamut conversion, or gamma conversion (gamma processing).

[0033] In the case of a high-brightness scene, such as a daytime scene, or when the F-number of the lens unit 110 is bright (small), a sufficient amount of light is irradiated onto the imaging unit 104 even if the ISO sensitivity is low or the exposure time is short. As a result, the display brightness of the LV display (display brightness of the displayed image data) that matches the exposure setting specified by the user can be achieved. On the other hand, in the case of a low-brightness scene, such as a night scene, or when the F-number of the lens unit 110 is dark (large), a low ISO sensitivity or a short exposure time does not allow a sufficient amount of light to be irradiated onto the imaging unit 104. Therefore, the display brightness of the LV display that matches the exposure setting cannot be achieved unless the ISO sensitivity is increased or the exposure time is extended. Increasing the ISO sensitivity increases noise and reduces the quality of the LV display. Increasing the accumulation time reduces the smoothness of movement in the LV display, resulting in a jerky LV display. For example, extending the accumulation time to 1 / 10 second results in a frame rate of 10 fps for the LV display. The exposure setting may be determined automatically by the digital camera 100 rather than being specified by the user.

[0034] ((Conventional smoothness priority mode)) Even in the smoothness priority mode, the control unit 101 controls at least one of the ISO sensitivity and the exposure time of the imaging unit 104 based on the luminance of the captured image data (e.g., RAW data) captured by the imaging unit 104. As described above, in the smoothness priority mode, the upper limit of the exposure time is shorter than in the visibility priority mode.

[0035] As mentioned above, in visibility priority mode, extending the accumulation time can result in jerky LV display. Jerky LV display results in poor subject tracking, increasing the risk that the user will miss a shutter opportunity when photographing a moving object. In smoothness priority mode, a shorter upper limit on exposure time than in visibility priority mode can be used to achieve smooth LV display even in low-brightness scenes. For example, if the upper limit on exposure time in smoothness priority mode is 1 / 60 second, smooth LV display can be achieved at a frame rate of 60 fps or higher. Note that the upper limit on exposure time in smoothness priority mode is not limited to 1 / 60 second, and may be 1 / 120 second, 1 / 30 second, or the like.

[0036] ((Smoothness priority mode of this embodiment)) In the conventional smoothness priority mode, it is not possible to set a long exposure time, so in low-brightness scenes, the exposure time is insufficient, resulting in underexposure and a live view display with poor visibility.

[0037] Therefore, in this embodiment, when the exposure time reaches its upper limit in the smoothness priority mode, at least one of the image processing unit 105 and the display unit 106 (display control circuit) performs a brightness adjustment process to increase the display brightness of the LV display. The brightness adjustment process increases the display brightness of the LV display to a level substantially equal to the display brightness in the visibility priority mode.

[0038] By doing so, it is possible to realize a desired display brightness even in the smoothness priority mode, and it is possible to achieve both smoothness and visibility in the smoothness priority mode.

[0039] Here, let's assume that the ISO sensitivity has reached its upper limit. In this case, the L The difference in display brightness between the V display and the LV display in the conventional smoothness priority mode is caused by the difference in exposure time. The following three methods (methods 1 to 3) can be considered to reduce this difference in display brightness.

[0040] First method: A method of performing brightness increasing processing on the RAW data 201 to increase brightness, such as by applying digital gain. Second method: A method of performing brightness increase processing on image data during development processing (or after development processing), for example, image data after color interpolation processing or matrix conversion processing. Third method: Increasing display brightness by controlling the display unit 106

[0041] In the first method, for example, a digital amplifier is used to apply digital gain to the RAW data 201. Because the brightness is increased before development processing, image quality degradation such as dark noise and color cast cannot be sufficiently reduced during development processing, and a low-quality LV display that makes it difficult to visually recognize the subject is displayed in low-brightness scenes.

[0042] In the second method, for example, brightness increasing processing is performed by at least one of the color brightness adjusting unit 206 and the display conversion processing unit 207. Since brightness increasing processing is performed after noise reduction processing by the color interpolation unit 203, it is possible to perform LV display with less noticeable noise than in the first method.

[0043] The brightness increasing process by the color brightness adjusting unit 206 is, for example, a process of switching a gamma curve for contrast correction according to the amount of increase in display brightness of the LV display to increase the display brightness to approximately the same as the display brightness in the visibility priority mode. The brightness increasing process by the color brightness adjusting unit 206 is not limited to this, and may be, for example, a process of applying digital gain to the image data.

[0044] In the brightness increasing process by the display conversion processing unit 207, for example, de-gamma processing is performed on the image data (image data after gamma processing) output from the color brightness adjustment unit 206, using the inverse characteristic (inverse gamma curve) of the gamma curve used by the color brightness adjustment unit 206. The brightness of the de-gamma processed image data (image data having linear characteristics in which the brightness changes linearly with changes in gradation value) is increased according to the increase in display brightness of the LV display to increase the display brightness to approximately the same as the display brightness in the visibility priority mode. Then, the image data after the brightness has been increased is subjected to the same gamma processing as the gamma processing performed by the color brightness adjustment unit 206 (gamma processing using the same gamma curve as the gamma curve used by the color brightness adjustment unit 206). The brightness increasing process by the display conversion processing unit 207 is not limited to this, and may be, for example, processing in which digital gain is applied to the image data.

[0045] In the third method, for example, the display setting of the display unit 106 is not changed, and the correspondence between the pixel values ​​(grayscale values) of the display image data and the display luminance is temporarily changed by the display control circuit of the display unit 106. This temporarily increases the display luminance corresponding to the pixel values ​​(grayscale values) of the display image data.

[0046] Although the bit precision of each process is not particularly limited, in this embodiment, it is assumed that the bit precision decreases from upstream to downstream in the process. The bit precision may be constant for a portion of the process from the upstream end to the downstream end. For example, it is assumed that the RAW data 201 is generated with 14-bit precision, the color luminance adjustment unit 206 and the display conversion processing unit 207 perform processing with 10-bit precision, and the display unit 106 performs processing with 8-bit precision. In order to suppress image quality degradation such as tone jumps, it is preferable to increase display brightness through processing with high bit precision.

[0047] For the above reasons, the second method is considered to be the most suitable from the viewpoint of suppressing image quality degradation such as noise and tone jumps. However, the second method excessively increases the display brightness. If the noise reduction processing is increased, dark noise may become more noticeable, and the visibility of the peaking processing results may deteriorate. Although the noise reduction processing may be strengthened according to the increase in display luminance using the second method, if the noise reduction processing is strengthened too much, the perceived resolution of the image may be lost. Furthermore, there is an upper limit to the amount of noise that can be reduced. Furthermore, if the noise reduction processing is strengthened according to the increase in display luminance, multiple noise reduction parameters corresponding to multiple increases in display luminance must be prepared, which increases the number of parameters stored in memory 103, requiring a large-capacity memory 103. Therefore, an upper limit is also set on the increase in display luminance using the second method.

[0048] In this embodiment, if the display brightness of the LV display does not increase to a level substantially equal to the display brightness in visibility priority mode even when the display brightness is increased by the upper limit increase amount using the second method, the display brightness is further increased using a third method.

[0049] FIG. 3 is a flowchart showing the operation according to this embodiment (the operation of the digital camera 100 in smoothness priority mode).

[0050] In step S301, the control unit 101 controls the image processing unit 105 to increase the display brightness of the LV display using the second method.

[0051] In step S302, the control unit 101 determines whether the increase in display brightness by the second method has reached an upper limit. Information on the upper limit of the increase is stored in advance in, for example, the recording medium 102. The determination in step S302 may be interpreted as a determination of whether the display brightness of the LV display has been increased to a level substantially equal to the display brightness in the visibility priority mode. If the increase in display brightness by the second method has reached the upper limit (if the display brightness of the LV display has not been increased to a level substantially equal to the display brightness in the visibility priority mode), the process proceeds to step S303. If the increase in display brightness by the second method has not reached the upper limit (if the display brightness of the LV display has been increased to a level substantially equal to the display brightness in the visibility priority mode), the operation of FIG. 3 ends.

[0052] In step S303, the control unit 101 controls the display unit 106 (display control unit) to increase the display brightness of the LV display using the third method.

[0053] In this embodiment, the image processing unit 105 performs a synthesis process to synthesize graphic data with captured image data so that a synthesized image in which a graphic is superimposed on an LV image (an image based on captured image data) that represents a subject in substantially real time is displayed on the display unit 106. The graphic is, for example, an OSD (On-Screen Display) image. Therefore, when the display brightness is increased using the third method, not only the display brightness of the LV image but also the display brightness of the graphic is increased. As a result, even if the display brightness of the LV image can be made substantially equal to the display brightness in the visibility priority mode, the display brightness of the graphic ends up being higher than the display brightness in the visibility priority mode. In this case, the bright graphic may cause the LV image to be perceived as dark, thereby reducing the visibility of the LV image.

[0054] Therefore, in this embodiment, when the process of step S303 is performed, the process of step S304 is performed. In step S304, the control unit 101 controls the image processing unit 105 according to the increase in display brightness by the third method to reduce the brightness of the graphic data. This makes it possible to keep the display brightness of the graphic constant before and after the increase in display brightness of the LV display (LV image) by the third method.

[0055] The above operation makes it possible to improve visibility in smoothness priority mode in a manner that is favorable from the viewpoint of suppressing deterioration in image quality such as noise and tone jumps, and from the viewpoint of suppressing an increase in memory capacity required for noise reduction processing.

[0056] 4 is a table showing specific examples of the increase (increase rate) in display luminance according to the second method and the increase (increase rate) in display luminance according to the third method. Although there is no particular limit to the upper limit of the increase rate in display luminance according to the second method, it is assumed here to be three times.

[0057] 4, the scene is sufficiently bright, and in smoothness priority mode, LV display can be performed at a display luminance substantially equal to that in visibility priority mode, without increasing the display luminance using the second or third method, etc. Therefore, in pattern a, the display luminance is not increased using the second or third method, etc.

[0058] In pattern b in Figure 4, the scene is somewhat dark, and in visibility priority mode, the exposure time is extended to 1 / 20 second to achieve a display brightness that matches the user's exposure setting. In smoothness priority mode, the exposure time can only be extended to 1 / 60 second, so unless the display brightness is increased using method 2 or 3, the display brightness of the LV display will be half (1 / 2) of the display brightness in visibility priority mode. Therefore, in pattern b, the display brightness is doubled using method 2.

[0059] In pattern c in Figure 4, the scene is very dark, and in visibility priority mode, the exposure time is extended to 1 / 10 second to achieve a display brightness that matches the user's exposure setting. In smoothness priority mode, the exposure time can only be extended to 1 / 60 second, so unless the display brightness is increased using methods such as the second or third method, the display brightness of the LV display will be 1 / 6 times that of the display brightness in visibility priority mode. Therefore, even in pattern c, the display brightness is increased using method 2. Although the display brightness needs to be increased by six times, the upper limit of the increase rate of display brightness using method 2 is three times, so the display brightness is increased by three times using method 2. Then, the display brightness is increased by two times using method 3.

[0060] As described above, according to this embodiment, it is possible to achieve a desired display brightness even in the smoothness priority mode, and it is possible to achieve both smoothness and visibility in the smoothness priority mode.

[0061] The various controls described above may or may not be performed by a single piece of hardware (e.g., a processor or circuit). The entire device may be controlled by multiple pieces of hardware (e.g., multiple processors, multiple circuits, or a combination of one or more processors and one or more circuits) sharing the processing.

[0062] The above processor is a processor in the broad sense, and includes general-purpose processors and dedicated processors. General-purpose processors include, for example, CPUs (Central Processing Units), MPUs (Micro Processing Units), and DSPs (Digital Signal Processors). Dedicated processors include, for example, GPUs (Graphics Processing Units), ASICs (Application Specific Integrated Circuits), and PLDs (Programmable Logic Devices). Programmable logic devices include, for example, FPGAs (Field Programmable Gate Arrays) and CPLDs (Complex Programmable Logic Devices).

[0063] Although the embodiments of the present invention have been described in detail, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Furthermore, each of the above-described embodiments merely represents one embodiment of the present invention, and each embodiment can be combined as appropriate.

[0064] (Other embodiments) The present invention can also be realized by a process in which a program that realizes one or more functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device read and execute the program, or by a circuit that realizes one or more functions.

[0065] The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) an acquisition means for acquiring captured image data captured by the imaging unit; a setting means for setting one of a plurality of display modes including a first display mode and a second display mode in which the upper limit of the exposure time of the imaging unit is shorter than that of the first display mode; a control means for controlling the exposure time based on the luminance of the captured image data; a processing means for generating display image data to be displayed on a display unit based on the captured image data; a display control means for displaying the display image data on the display unit; and In the second display mode, when the exposure time reaches an upper limit, at least one of the processing means and the display control means performs a brightness adjustment process to increase the display brightness of the display image data on the display unit to a display brightness substantially equal to the display brightness in the first display mode. 1. An image processing device comprising: (Configuration 2) The brightness adjustment process includes at least one of image processing of the captured image data by the processing means and control of the display unit by the display control means. 2. The image processing device according to configuration 1, (Configuration 3) In the second display mode, when the exposure time reaches an upper limit, the processing means performs the image processing; There is an upper limit to the amount of increase in display luminance due to the image processing, When the display luminance of the display image data does not increase to a display luminance substantially equal to the display luminance in the first display mode by the image processing alone, the display control means further performs the control. 3. The image processing device according to configuration 2. (Configuration 4) the processing means performs a synthesis process of synthesizing graphic data with the captured image data so that a synthesized image in which a graphic is superimposed on an image based on the captured image data is displayed on the display unit; When the control is performed by the display control means, the processing means reduces the brightness of the graphic data so that the control does not result in an increase in the display brightness of the graphic. 4. The image processing device according to configuration 2 or 3. (Configuration 5) The processing means realizes the image processing by switching a gamma curve used to generate the display image data in accordance with an increase amount of the display luminance of the display image data for increasing the display luminance to approximately the same as the display luminance in the first display mode. 5. The image processing device according to any one of configurations 2 to 4. (Configuration 6) In the image processing, the processing means performs degamma processing on the image data after gamma processing, and adjusts the luminance of the image data after the degamma processing to the display luminance in the first display mode. The display luminance of the display image data is increased in accordance with the amount of increase in the display luminance to increase the display luminance to a value substantially equal to the display luminance of the image data, and the gamma processing is performed on the image data after the luminance has been increased. 5. The image processing device according to any one of configurations 2 to 4. (method) an acquisition step of acquiring captured image data captured by the imaging unit; a setting step of setting one of a plurality of display modes including a first display mode and a second display mode in which an upper limit of an exposure time of the imaging unit is shorter than that of the first display mode; a control step of controlling the exposure time based on the luminance of the captured image data; a processing step of generating display image data to be displayed on a display unit based on the captured image data; a display control step of displaying the display image data on the display unit; and In the second display mode, when the exposure time reaches an upper limit, at least one of the processing step and the display control step performs a brightness adjustment process to increase the display brightness of the display image data on the display unit to a display brightness substantially equal to the display brightness in the first display mode. An image processing method comprising: (program) A program for causing a computer to function as each means of the image processing device according to any one of the first to sixth aspects. [Explanation of symbols]

[0066] 100: Digital camera 101: Control unit 104: Imaging unit 105: Image processing unit 106: Display unit

Claims

1. an acquisition means for acquiring captured image data captured by the imaging unit; a setting means for setting one of a plurality of display modes including a first display mode and a second display mode in which the upper limit of the exposure time of the imaging unit is shorter than that of the first display mode; a control means for controlling the exposure time based on the luminance of the captured image data; a processing means for generating display image data to be displayed on a display unit based on the captured image data; a display control means for displaying the display image data on the display unit; and In the second display mode, when the exposure time reaches an upper limit, at least one of the processing means and the display control means performs a brightness adjustment process to increase the display brightness of the display image data on the display unit to a display brightness substantially equal to the display brightness in the first display mode.

1. An image processing device comprising:

2. The brightness adjustment process includes at least one of image processing of the captured image data by the processing means and control of the display unit by the display control means.

2. The image processing device according to claim 1, wherein:

3. In the second display mode, when the exposure time reaches an upper limit, the processing means performs the image processing; There is an upper limit to the amount of increase in display luminance due to the image processing, When the display luminance of the display image data does not increase to a display luminance substantially equal to the display luminance in the first display mode by the image processing alone, the display control means further performs the control.

3. The image processing device according to claim 2.

4. the processing means performs a synthesis process of synthesizing graphic data with the captured image data so that a synthesized image in which a graphic is superimposed on an image based on the captured image data is displayed on the display unit; When the control is performed by the display control means, the processing means reduces the brightness of the graphic data so that the control does not result in an increase in the display brightness of the graphic.

3. The image processing device according to claim 2.

5. The processing means realizes the image processing by switching a gamma curve used to generate the display image data in accordance with an increase amount of the display luminance of the display image data for increasing the display luminance to approximately the same as the display luminance in the first display mode.

3. The image processing device according to claim 2.

6. In the image processing, the processing means performs de-gamma processing on the image data after gamma processing, increases the luminance of the image data after the de-gamma processing in accordance with an increase in the display luminance of the display image data for increasing the display luminance to a display luminance substantially equal to the display luminance in the first display mode, and performs the gamma processing on the image data after the luminance has been increased.

3. The image processing device according to claim 2.

7. an acquisition step of acquiring captured image data captured by the imaging unit; a setting step of setting one of a plurality of display modes including a first display mode and a second display mode in which an upper limit of an exposure time of the imaging unit is shorter than that of the first display mode; a control step of controlling the exposure time based on the luminance of the captured image data; a processing step of generating display image data to be displayed on a display unit based on the captured image data; a display control step of displaying the display image data on the display unit; and In the second display mode, when the exposure time reaches an upper limit, at least one of the processing step and the display control step performs a brightness adjustment process to increase the display brightness of the display image data on the display unit to a display brightness substantially equal to the display brightness in the first display mode. An image processing method comprising:

8. A program for causing a computer to function as each of the means of the image processing device according to any one of claims 1 to 6.

Images