Image processing apparatus, image processing method, computer program product, and computer readable storage medium

Abstract

The invention relates to an image processing apparatus, an image processing method, a computer program product, and a computer readable storage medium. An image processing apparatus according to the present disclosure includes: a processing unit configured to generate display image data to be displayed on a display unit based on captured image data; and a display control unit configured to display the display image data on a display unit, in a second display mode, in a case where an exposure time reaches an upper limit, brightness adjustment processing is performed using at least one of the processing unit and the display control unit, and brightness adjustment processing is performed using at least one of the processing unit and the display control unit. The brightness adjustment process is a process of increasing a display brightness of the display image data in the display unit to a display brightness substantially equal to a display brightness in a first display mode.

Description

Technical Field

[0001] This disclosure relates to image processing apparatus, image processing methods and programs, and particularly to techniques for improving the visibility of display images corresponding to low-brightness scenes. Background Technology

[0002] As a camera display mode, a display mode has been proposed that controls the exposure time of the imaging element (the charge accumulation time in the imaging element corresponding to the light radiated to the imaging element) to enable live view display (LV display) with desired display brightness. However, when the exposure time is long, LV display with non-smooth motion occurs (LV display with unstable motion).

[0003] A display mode prioritizing the smoothness of the LV display and limiting the exposure time to a predetermined time or less has also been proposed. However, because the exposure time is limited to a predetermined time or less, the LV display becomes darker than the desired display brightness when shooting night scenes or using dark lenses with large open f-numbers.

[0004] Japanese Patent Application Publication No. 2007-110220 discloses a technology that increases display brightness through image processing in response to user commands to increase display brightness when the surrounding environment is dark.

[0005] However, there is an upper limit to the increase in display brightness through image processing, so even using the technology disclosed in Japanese Patent Application Publication No. 2007-110220, it may not be possible to achieve the desired display brightness. Summary of the Invention

[0006] This disclosure provides a technique that enables the desired display brightness to be achieved even in display modes that prioritize smoothness.

[0007] A first aspect of this disclosure provides an image processing apparatus, comprising: an acquisition unit configured to acquire captured image data by a camera unit; a setting unit configured to set one of a plurality of display modes including a first display mode and a second display mode, wherein in the second display mode, an upper limit for the exposure time of the camera unit is shorter than that in the first display mode; a control unit configured to control the exposure time based on the brightness of the captured image data; a processing unit configured to generate display image data to be displayed on a display unit based on the captured image data; and a display control unit configured to display the display image data on the display unit, wherein, in the second display mode, when the exposure time reaches the upper limit, at least one of the processing unit and the display control unit performs a brightness adjustment process, the brightness adjustment process being a process of increasing the display brightness of the display image data in the display unit to a display brightness substantially equal to that in the first display mode.

[0008] A second aspect of this disclosure provides an image processing method, comprising: an acquisition step for acquiring captured image data by a camera unit; a setting step for setting one of a plurality of display modes including a first display mode and a second display mode, wherein in the second display mode, the upper limit of the exposure time of the camera unit is shorter than that in the first display mode; a control step for controlling the exposure time based on the brightness of the captured image data; a processing step for generating display image data to be displayed on a display unit based on the captured image data; and a display control step for displaying the display image data on the display unit, wherein, in the second display mode, when the exposure time reaches the upper limit, a brightness adjustment process is performed in at least one of the processing step and the display control step, the brightness adjustment process being a process of increasing the display brightness of the display image data in the display unit to a display brightness substantially equal to that in the first display mode.

[0009] A third aspect of this disclosure provides a computer program product, including a program that causes a computer to perform the steps of the above-described image processing method. A fourth aspect of this disclosure provides a computer-readable storage medium storing a program that causes a computer to perform the steps of the above-described image processing method.

[0010] The features of this disclosure will become apparent from the following description of embodiments with reference to the accompanying drawings. The following description of the embodiments is given by way of example. Attached Figure Description

[0011] Figure 1This is a block diagram illustrating a digital camera.

[0012] Figure 2 This is a block diagram of the image processing unit.

[0013] Figure 3 This is a flowchart illustrating the operation of a digital camera.

[0014] Figure 4 This is a table showing specific examples of the rate of increase in display brightness. Detailed Implementation

[0015] Embodiments of this disclosure will be described below. Note that although examples of applying this disclosure to camera devices will be described, the devices to which this disclosure can be applied are not limited to camera devices. This disclosure is applicable to various electronic devices (image processing devices) capable of performing image processing on captured images (captured images). For example, this disclosure is applicable to digital cameras, digital video cameras, personal computers, tablet terminals, smartphones, mobile phones, game consoles, and video see-through head-mounted displays, etc.

[0016] Digital camera

[0017] Figure 1 This is a block diagram illustrating the functional structure of a digital camera 100 according to this embodiment. Figure 1 The units of the digital camera 100 illustrated herein may be hardware such as circuits or processors, or may be implemented by a program.

[0018] The control unit 101 includes at least one processor or circuit and controls the various units of the digital camera 100. For example, the control unit 101 controls the various units of the digital camera 100 by reading a program from the recording medium 102, expanding the program in the memory 103, and executing the program.

[0019] Recording medium 102 is a non-volatile recording device that enables electrical erasure and recording of information (data), and is, for example, a flash ROM. Recording medium 102 stores programs and constants for controlling the various units of digital camera 100. Recording medium 102 can also be used as a recording medium (e.g., a semiconductor memory card) for storing images (RAW data and image data after image processing, etc.) acquired through photography.

[0020] The memory 103 is a volatile recording device, and is, for example, RAM or DRAM. The memory 103 serves as an expansion area for programs used to control the various units of the digital camera 100. The memory 103 also serves as VRAM when images are displayed on the display unit 106.

[0021] Lens unit 110 uses light from the subject to form an optical image on its imaging surface. Lens unit 110 may be detachable from digital camera 100. Although lens unit 110 typically includes multiple lenses (imaging lens group), for simplicity, Figure 1 Only one lens is illustrated. Lens unit 110 includes control circuitry (not illustrated), and the control circuitry controls the state of lens unit 110 based on drive signals input from control unit 101.

[0022] The imaging unit 104 is, for example, an imaging element such as a CCD or CMOS sensor, and acquires an analog image signal by converting the optical image formed by the lens unit 110 on the imaging surface into an electrical signal. The acquired analog image signal is converted into a digital image signal (RAW data) by an A / D converter (not shown). In this embodiment, it is assumed that the imaging unit 104 is a single-board color imaging element including a general primary color filter. In the primary color filter, three types of color filters with transmission dominant wavelength bands (wavelength bands of mainly transmitted light) near 650nm, 550nm, and 450nm are arranged in a mosaic pattern (in a Bayer arrangement). The imaging unit 104 uses the primary color filter to image the color planes of the three colors corresponding to the red (R) band, green (G) band, and blue (B) band, respectively. At this time, each photoelectric conversion element of the imaging unit 104 can obtain only the light intensity corresponding to a single color plane. Note that the imaging unit 104 is not limited to a single-board color imaging element. The camera unit 104 may include peripheral circuitry, such as amplifier circuitry for processing signals obtained from each photoelectric conversion element.

[0023] The image processing unit 105 performs various types of image processing on the RAW data output from the imaging unit 104 or the image data read from the recording medium 102, such as pixel interpolation, resizing, and color conversion. Furthermore, the image processing unit 105 performs calculations on the RAW data obtained through imaging to acquire information required for exposure control and rangefinder control. In the digital camera 100, based on the information obtained from the image processing unit 105, it performs through-lens (TTL) type autofocus (AF) processing, automatic exposure (AE) processing, and flash pre-emission (EF) processing. Additionally, it performs TTL type automatic white balance (AWB) processing, etc.

[0024] The display unit 106 is, for example, a display device such as a liquid crystal display (LCD), and displays GUIs such as settings, messages, and menu screens of the digital camera 100, as well as captured images. The display unit 106 may be, for example, an electronic viewfinder (EVF) or a rear LCD screen installed in the digital camera 100, or it may be an external display connected to the digital camera 100. It is possible to display the subject substantially in real-time using a live view display (LV display) of any of the electronic viewfinder, rear LCD screen, and external display.

[0025] Display unit 106 includes a display control circuit (not shown), and the display control circuit can change the maximum display brightness (upper limit display brightness) of display unit 106 based on a drive signal input from control unit 101. The maximum display brightness of display unit 106 can be dynamically controlled based on the Bv value (brightness value) obtained by metering the camera scene, or it can be controlled based on the camera mode. The metering of the camera scene can be based on the image signal obtained by camera capture, or it can be based on the output of a metering sensor separately from camera unit 104, etc.

[0026] The operation unit 107 is a user interface that receives various user operations. When a user operation is detected, the operation unit 107 outputs a control signal corresponding to the user operation to the control unit 101. The operation unit 107 includes a release switch for indicating the start of shooting preparation operation and the start of shooting (main shooting), a mode selection switch for selecting the video recording mode or display mode, directional keys, and an OK key, etc.

[0027] Image processing unit

[0028] Figure 2 This is a block diagram illustrating the functional structure of the image processing unit 105. The camera unit 104 includes primary color filters that arrange three types of color filters in a mosaic pattern. Therefore, the RAW data 201 (the captured image data by the camera unit 104) is a color mosaic image. The image processing unit 105 reads the RAW data 201 from the memory 103, performs image processing, and generates display image data 208.

[0029] White balance unit 202 performs white balance processing on RAW data 201. This white balance processing is a color conversion process used to increase the reproduction of white in the subject. White balance unit 202 plots each RGB data included in the RAW data 201 in a predetermined color space, such as the xy color space. RGB data is obtained from three or more photoelectric conversion elements corresponding to the color planes of the three RGB colors. White balance unit 202 integrates the R, G, and B values ​​of the RGB data plotted near the trajectory of blackbody radiation, which have high light source color probability. Then, white balance unit 202 calculates white balance coefficients (G integral value / R integral value and G integral value / B integral value) for the R and B values ​​based on the calculated integral values, and uses these white balance coefficients to correct each RGB data. As a result, color fogging caused by the light source can be reduced, thereby enhancing the reproduction of white in the subject.

[0030] The color interpolation unit 203 performs noise reduction and color interpolation processing on the image data after white balance processing. Typically, since random noise increases with the ISO sensitivity of the imaging unit 104, stronger noise reduction is performed as the ISO sensitivity increases. Color conversion processing is the process of obtaining values ​​for color components not included in the pixel data (data corresponding to a single color plane obtained from a photoelectric conversion element). Image data including the R, G, and B values ​​of all pixels is generated through color conversion processing.

[0031] The matrix transformation unit 204 performs matrix transformation processing on the image data generated by the color interpolation unit 203. As a result, general color image data is obtained.

[0032] Color brightness adjustment unit 206 performs color brightness adjustment processing on the color image data generated by matrix conversion unit 204, adjusting at least one of color and brightness. Color brightness adjustment processing may include, for example, contrast correction, exposure correction, saturation correction, and sharpness correction performed using color brightness adjustment parameters 205.

[0033] The display conversion processing unit 207 generates display image data 208 by converting at least one of the color gamut and gamma characteristics of the image data after color brightness adjustment processing to match the color gamut and gamma characteristics of the display unit 106.

[0034] The display image data 208 (the image corresponding to the display image data 208) generated in this way is displayed on the display unit 106, thereby realizing LV display.

[0035] Display mode

[0036] In this embodiment, the control unit 101 sets one of a plurality of display modes, including a visibility priority mode and a smoothness priority mode, as the LV display mode. The visibility priority mode prioritizes the visibility of the LV display and is a display mode with a relatively long upper limit for the exposure time of the camera unit 104. The smoothness priority mode prioritizes the smoothness of the LV display and is a display mode with a shorter upper limit for the exposure time of the camera unit 104 compared to the upper limit for the exposure time of the camera unit 104 in the visibility priority mode. The plurality of display modes that can be set may include display modes other than the visibility priority mode and the smoothness priority mode.

[0037] Visibility-first mode

[0038] In the visibility-priority mode, the control unit 101 controls at least one of the ISO sensitivity and exposure time of the camera unit 104 based on the brightness of the captured image data (e.g., RAW data) obtained by the camera unit 104. Here, the captured image data is not limited to RAW data. For example, the captured image data can be image data after white balance processing, noise reduction processing, color interpolation processing, matrix transformation processing, contrast correction, exposure correction, saturation correction, sharpness correction, color gamut conversion, or gamma conversion (gamma processing).

[0039] In high-brightness scenes such as daytime scenes, or when the F-number of lens unit 110 is small (bright), sufficient light is applied to the imaging unit 104 even when the ISO sensitivity is low or the exposure time is short. As a result, the display brightness (display brightness for displaying image data) of the LV display can be achieved to match the exposure setting specified by the user. On the other hand, in low-brightness scenes such as nighttime scenes, or when the F-number of lens unit 110 is large (dark), the imaging unit 104 is not illuminated with sufficient light when the ISO sensitivity is low or the exposure time is short. Therefore, without increasing the ISO sensitivity or extending the exposure time, the display brightness of the LV display suitable for the exposure setting cannot be achieved. Increasing the ISO sensitivity increases noise, thereby reducing the quality of the LV display. When the accumulation time is extended, the smoothness of motion in the LV display decreases, resulting in an uneven LV display. For example, when the accumulation time is extended to 1 / 10 of a second, the frame rate of the LV display becomes 10fps. Note that the exposure setting can be automatically determined by the digital camera 100 without being specified by the user.

[0040] Traditional smoothness-first mode

[0041] Similarly, in the smoothness-priority mode, the control unit 101 controls at least one of the ISO sensitivity and exposure time of the camera unit 104 based on the brightness of the captured image data (e.g., RAW data) obtained by the camera unit 104. As described above, the upper limit of the exposure time in the smoothness-priority mode is lower than the upper limit of the exposure time in the visibility-priority mode.

[0042] As mentioned above, in visibility priority mode, motion-uneven LV displays can be achieved by extending the accumulation time. In motion-uneven LV displays, subject tracking is poor, and there is a high risk of the user missing a photo opportunity when shooting moving subjects. In smoothness priority mode, smooth LV displays can also be achieved in low-light scenes by using a shorter exposure time as the upper limit compared to visibility priority mode. For example, in smoothness priority mode, when the upper limit of the exposure time is 1 / 60 of a second, smooth LV displays can be achieved at a frame rate of 60fps or higher. Note that the upper limit of the exposure time in smoothness priority mode is not limited to 1 / 60 of a second, and can also be 1 / 120 of a second or 1 / 30 of a second, etc.

[0043] Smoothness-first mode in this embodiment

[0044] In the traditional smoothness-first mode, because long exposure times cannot be set, the exposure time is insufficient, resulting in underexposure in low-light scenes and poor visibility in live view.

[0045] Therefore, in this embodiment, in the smoothness-priority mode, when the exposure time reaches its upper limit, at least one of the image processing unit 105 and the display unit 106 (display control circuit) performs a brightness adjustment process to increase the display brightness of the LV display. Through this brightness adjustment process, the display brightness of the LV display is increased to a level substantially equal to the display brightness in the visibility-priority mode.

[0046] Therefore, the desired display brightness can also be achieved in the smoothness priority mode, and both smoothness and visibility can be obtained in the smoothness priority mode.

[0047] Here, it is assumed that the ISO sensitivity has reached its limit. In this case, the difference in display brightness between the LV display in visibility priority mode and the LV display in conventional smoothness priority mode is caused by the difference in exposure time. As a method to reduce this difference in display brightness, there are three possible methods (methods one through three).

[0048] The first method is a brightness enhancement processing method, such as applying digital gain to RAW data 201 to increase brightness. The second method is a brightness enhancement processing method for image data (e.g., image data after color interpolation processing or matrix transformation processing) that has been processed during image processing (which can be interpreted as image data after image processing). The third method is a method of increasing display brightness by controlling the display unit 106.

[0049] In the first method, for example, a digital gain is applied to the RAW data 201 using a digital amplifier. Because the brightness is increased before image processing, image quality degradation such as dark noise and color haze cannot be sufficiently reduced during image processing, resulting in low-quality LV display where the subject is difficult to visually identify in low-light scenes.

[0050] In the second method, for example, brightness enhancement processing is performed by at least one of the color brightness adjustment unit 206 and the display conversion processing unit 207. Since the brightness enhancement processing is performed after noise reduction processing using the color interpolation unit 203, LV display with less noticeable noise can be achieved compared to the first method.

[0051] The brightness increase processing using the color brightness adjustment unit 206 is, for example, a process that switches the gamma curve used for contrast correction based on the amount of increase in display brightness of the LV display to be substantially equal to the display brightness in the visibility priority mode. The brightness increase processing using the color brightness adjustment unit 206 is not limited to this, and may, for example, be the processing of applying digital gain to image data.

[0052] In the brightness enhancement processing using the display conversion processing unit 207, for example, the image data output from the color brightness adjustment unit 206 (image data after gamma processing) undergoes degamma processing using the inverse characteristic (inverse gamma curve) of the gamma curve used by the color brightness adjustment unit 206. The brightness of the degamma-processed image data (image data having a linear characteristic where brightness changes linearly with respect to grayscale values) is increased according to the increase in display brightness of the LV display, which is substantially equal to the display brightness in the visibility priority mode. Then, the image data whose brightness has been increased undergoes the same gamma processing as that performed by the color brightness adjustment unit 206 (gamma processing using the same gamma curve as that used by the color brightness adjustment unit 206). The brightness enhancement processing using the display conversion processing unit 207 is not limited to this, and may, for example, be a process that applies digital gain to the image data.

[0053] In the third method, for example, the display settings of the display unit 106 are not changed, and the correspondence between the pixel values ​​(grayscale values) of the displayed image data and the display brightness is temporarily changed using the display control circuit of the display unit 106. As a result, the display brightness corresponding to the pixel values ​​(grayscale values) of the displayed image data is temporarily increased.

[0054] While there are no specific limitations on the bit precision of each process, in this embodiment, the pseudo-positioning precision decreases from upstream to downstream of the process. The bit precision can be constant in the portion from upstream to downstream of the process. For example, assuming RAW data 201 is generated with 14-bit precision, it is processed with 10-bit precision in the color brightness adjustment unit 206 and the display conversion processing unit 207, and with 8-bit precision in the display unit 106. To suppress image quality degradation such as tone jumps, it is preferable to increase display brightness through processing with high bit precision.

[0055] For the reasons stated above, the second method is considered the most suitable from the perspective of suppressing image quality degradation (such as noise and tone jumps). However, when the display brightness is excessively increased using the second method, dark noise may become noticeable, or the visibility of the peaking result may deteriorate. Although the noise reduction process can be enhanced based on the amount of increase in display brightness using the second method, image resolution is lost when the noise reduction process is excessively enhanced. There is also an upper limit to the amount of noise that can be reduced. Furthermore, when the noise reduction process is enhanced based on the amount of increase in display brightness, the number of parameters stored in the memory 103 increases, and a large-capacity memory 103 is required, since multiple noise reduction parameters corresponding to multiple increases in display brightness need to be prepared. Therefore, an upper limit is also set on the amount of increase in display brightness using the second method.

[0056] In this embodiment, even if the display brightness of the LV display does not increase to a level substantially equal to the display brightness in the visibility priority mode when the upper limit of the increase is increased by the second method, the display brightness is further increased by the third method.

[0057] Figure 3 This is a flowchart illustrating the operation (operation of digital camera 100 in smoothness priority mode) according to this embodiment.

[0058] In step S301, the control unit 101 controls the image processing unit 105 to increase the display brightness of the LV display through a second method.

[0059] In step S302, the control unit 101 determines whether the increase in display brightness via the second method has reached its upper limit. Information related to the upper limit of the increase is, for example, pre-stored in the recording medium 102. The determination in step S302 can be interpreted as a determination of whether the display brightness of the LV display has increased to a level substantially equal to the display brightness in the visibility priority mode. When the increase in display brightness via the second method has reached its upper limit (if the display brightness of the LV display has not yet increased to a level substantially equal to the display brightness in the visibility priority mode), the process proceeds to step S303. If the increase in display brightness via the second method has not yet reached its upper limit (if the display brightness of the LV display has already increased to a level substantially equal to the display brightness in the visibility priority mode), the process ends. Figure 3 The operation.

[0060] In step S303, the control unit 101 controls the display unit 106 (display control unit) to increase the display brightness of the LV display through a third method.

[0061] In this embodiment, the image processing unit 105 performs a synthesis process that combines graphic data with captured image data, such that the graphic is superimposed on the LV image (based on the captured image data) representing the subject and displayed on the display unit 106 in essentially real-time. The graphic is, for example, an OSD (On-Screen Display) image. Therefore, when the display brightness is increased by the third method, not only the display brightness of the LV image is increased, but also the display brightness of the graphic is increased. As a result, even when the display brightness of the LV image can be made substantially equal to the display brightness in the visibility priority mode, the display brightness of the graphic becomes higher than the display brightness in the visibility priority mode. In this case, since the graphic is bright, the LV image can be perceived as dark, thus reducing the visibility of the LV image.

[0062] Therefore, in this embodiment, after processing in step S303, processing in step S304 is performed. In step S304, the control unit 101 controls the image processing unit 105 to reduce the brightness of the graphic data based on the increase in display brightness achieved by the third method. Therefore, the display brightness of the graphic can remain constant before and after the increase in display brightness of the LV display (LV image) achieved by the third method.

[0063] By performing the above operations, visibility in smoothness-first mode can be appropriately improved from the perspectives of suppressing image quality degradation such as noise and tone jumps, and from the perspective of suppressing the increase in memory capacity required for noise reduction processing.

[0064] Figure 4This table shows specific examples of the increase in display brightness (rate of increase) through the second method and the increase in display brightness (rate of increase) through the third method. Although there is no specific upper limit to the rate of increase in display brightness through the second method, it is assumed here to be three times.

[0065] exist Figure 4 In Mode a, the scene is bright enough, and in the smoothness-priority mode, LV display can be performed with a brightness substantially equal to that in the visibility-priority mode without increasing the display brightness through a second or third method. Therefore, in Mode a, the display brightness is not increased through a second or third method.

[0066] exist Figure 4 In Mode b, the scene is slightly darker, and in the visibility priority mode, the exposure time is extended to 1 / 20 of a second to achieve a display brightness suitable for the user's exposure settings. In the smoothness priority mode, the exposure time can only be extended to a maximum of 1 / 60 of a second. Therefore, unless the display brightness is increased through a second or third method, the display brightness of the LV display becomes half (1 / 2) of the display brightness in the visibility priority mode. Therefore, in Mode b, the display brightness is doubled through the second method.

[0067] exist Figure 4 In Mode c, the scene is very dark, and in visibility priority mode, the exposure time is extended to 1 / 10 of a second to achieve display brightness matching the user's exposure settings. In smoothness priority mode, since the exposure time can only be extended to a maximum of 1 / 60 of a second, the display brightness of the LV display becomes 1 / 6 of the display brightness in visibility priority mode unless the display brightness is increased through a second or third method. Therefore, also in Mode c, the display brightness is increased through the second method. Although it would require a six-fold increase in display brightness, since the upper limit of the increase rate through the second method is three times, the display brightness is increased by three times through the second method. Then, the display brightness is doubled through the third method.

[0068] As described above, according to this embodiment, the desired display brightness can also be achieved in the smoothness priority mode, and both smoothness and visibility can be obtained in the smoothness priority mode.

[0069] Note that the various types of control described above can be performed by a single piece of hardware (e.g., a processor or circuitry) or by processing in other ways. Processing can be distributed among multiple pieces of hardware (e.g., multiple processors, multiple circuits, or a combination of one or more processors and one or more circuits), thereby performing control over the entire device.

[0070] Furthermore, the processors mentioned above are processors in a broad sense, encompassing both general-purpose and special-purpose processors. Examples of general-purpose processors include central processing units (CPUs), microprocessor units (MPUs), and digital signal processors (DSPs). Examples of special-purpose processors include graphics processing units (GPUs), application-specific integrated circuits (ASICs), and programmable logic devices (PLDs). Examples of PLDs include field-programmable gate arrays (FPGAs) and complex programmable logic devices (CPLDs).

[0071] The above embodiments (including variations) are merely examples. Any structures obtained by appropriately modifying or changing some structures of the embodiments within the scope of the subject matter of this disclosure are also included in this disclosure. This disclosure also includes other structures obtained by appropriately combining various features of the embodiments.

[0072] According to this disclosure, desired display brightness can also be achieved in a display mode that prioritizes smoothness.

[0073] Other embodiments

[0074] Embodiments of the present invention can also be implemented by providing software (including computer program products of computer programs) that performs the functions of the above embodiments to a system or device via a network or various storage media, and the computer (central processing unit (CPU) or microprocessor unit (MPU) of the system or device) reads and executes the computer program.

[0075] Although this disclosure has been described with reference to embodiments, it should be understood that this disclosure is not limited to the disclosed embodiments. The scope of the appended claims is to be interpreted in the broadest sense to include all such modifications and equivalent constructions and functions.

Claims

1. An image processing apparatus, comprising: An acquisition unit is configured to acquire image data captured by the camera unit; The setting unit is configured to set one of a plurality of display modes including a first display mode and a second display mode, wherein in the second display mode, the upper limit of the exposure time of the camera unit is shorter than that in the first display mode; A control unit configured to control the exposure time based on the brightness of the captured image data; A processing unit configured to generate display image data to be displayed on a display unit based on the captured image data; as well as A display control unit is configured to display the image data on the display unit. In the second display mode, when the exposure time reaches the upper limit, at least one of the processing unit and the display control unit performs a brightness adjustment process, which increases the display brightness of the display image data in the display unit to a display brightness that is substantially equal to the display brightness in the first display mode.

2. The image processing apparatus according to claim 1, wherein, The brightness adjustment process includes at least one of image processing of the captured image data using the processing unit and control of the display unit using the display control unit.

3. The image processing apparatus according to claim 2, wherein, In the second display mode, when the exposure time reaches the upper limit, the processing unit performs the image processing. The increase in display brightness through the image processing has an upper limit, and The display control unit performs the control if the display brightness of the displayed image data is not increased to a level substantially equal to the display brightness in the first display mode, solely through the image processing.

4. The image processing apparatus according to claim 2 or 3, wherein, The processing unit performs a compositing process that combines the graphic data with the captured image data, resulting in a composite image, where the graphic is superimposed on the image based on the captured image data, and displayed on the display unit. When the control is performed using the display control unit, the processing unit reduces the brightness of the graphic data to prevent the display brightness of the graphic from increasing due to the control.

5. The image processing apparatus according to claim 2 or 3, wherein, The processing unit performs the image processing by switching the gamma curve used in generating the display image data according to the amount of increase in display brightness of the display image data to be substantially equal to the display brightness in the first display mode.

6. The image processing apparatus according to claim 2 or 3, wherein, In the image processing, the processing unit performs degamma processing on the image data after gamma processing, increases the brightness of the image data after gamma processing according to the increase in display brightness of the displayed image data to be substantially equal to the display brightness in the first display mode, and performs the gamma processing on the image data after the brightness increase.

7. An image processing method, comprising: The acquisition step is used to acquire the image data captured by the camera unit; The setting steps are used to set one of a plurality of display modes, including a first display mode and a second display mode, wherein in the second display mode, the upper limit of the exposure time of the camera unit is shorter than that in the first display mode; A control step for controlling the exposure time based on the brightness of the captured image data; The processing step is used to generate display image data to be displayed on the display unit based on the captured image data; as well as The display control step is used to display the image data on the display unit. In the second display mode, when the exposure time reaches the upper limit, a brightness adjustment process is performed in at least one of the processing step and the display control step. The brightness adjustment process is a process that increases the display brightness of the display image data in the display unit to a display brightness that is substantially equal to the display brightness in the first display mode.

8. A computer program product comprising a program that causes a computer to perform the steps of the image processing method according to claim 7.

9. A computer-readable storage medium storing a program that causes a computer to perform the steps of the image processing method according to claim 7.

Images