Electronic device, control method, and program

Abstract

To display an OSD with suitable luminance.SOLUTION: A display control device includes display control means for performing control in a manner to make a display part display a graphic image together with an HDR image. The display control means makes the graphic image displayed with second luminance when the luminance of the HDR image is first luminance, and makes it displayed with fourth luminance higher than the second luminance when the luminance of the HDR image is third luminance higher than the first luminance.SELECTED DRAWING: Figure 12A

Description

[Technical Field]

[0001] The present invention provides Electronic device, control method, and program Regarding. [Background technology]

[0002] In recent years, advances in LED elements have led to the widespread adoption of HDR (High Dynamic Range) compatible displays, which can tighten blacks and further increase the upper limit of brightness (maximum brightness; peak brightness). HDR-compatible displays can display high dynamic range (HDR) image data as is, without compression. HDR can realistically reproduce (with good gradation) scenery such as clouds in a blue sky or neon lights in a night view, which would have lost contrast with conventional dynamic range (hereinafter referred to as SDR; Standard Dynamic Range). This type of HDR has been mainstream for videos recorded on Blu-ray, but it is expected to be applied to still images in the future.

[0003] Here, on the rear LCD of an imaging device, when a captured image is played back and displayed, an OSD (graphic image) such as shooting information (time, aperture value, ISO sensitivity, image name) and playback status information is displayed superimposed on the captured image. Patent Document 1 discloses that peak luminance and gamma are received from the display device, and if it is determined that the OSD brightness will be dark, the OSD is brightened in advance, or that it is determined whether the image or the OSD is more important, and the OSD is set to the optimum brightness for either. Patent Document 2 discloses that an SDR OSD (100 cd / m 2 When converting the OSD brightness gradation value to HDR data, the luminance gradation value is set to Diffuse White (the dominant white in the image (300-500 cd / m 2 ) is disclosed. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-057824 [Patent Document 2] International Publication No. 2016 / 038950 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the conventional techniques disclosed in the above-mentioned Patent Documents 1 and 2 have a problem in that the visibility of the OSD is poor when the OSD is displayed relatively brightly or darkly with respect to the rear image.

[0006] Therefore, an object of the present invention is to display an OSD at an appropriate brightness. [Means for solving the problem]

[0007] One aspect of the present invention is An electronic device having an imaging element and a display unit, a display control means for controlling the display unit to display a graphic image together with an HDR image; the display control means controls to display the graphic image at a second luminance when a luminance related to the HDR image is a first luminance, and controls to display the graphic image at a fourth luminance higher than the second luminance when the luminance related to the HDR image is a third luminance higher than the first luminance, The luminance of the HDR image is an upper limit luminance value corresponding to the dynamic range at the time of image capture set in the electronic device. and , and , which is the upper limit luminance corresponding to the gamma at the time of development set in the electronic device. The electronic device is characterized by the above. [Effects of the Invention]

[0008] According to the present invention, the OSD can be displayed at an appropriate brightness. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is an external view of a digital camera according to a first embodiment. [Figure 2] 1 is a block diagram showing the configuration of a digital camera according to a first embodiment. [Figure 3] FIG. 2 is a diagram showing an external device connection according to the first embodiment. [Figure 4A] 5 is a flowchart showing LV shooting mode processing according to the first embodiment. [Figure 4B] 10 is a flowchart showing a quick review display process according to the first embodiment. [Figure 5] FIG. 4 is a sequence diagram of a connection process according to the first embodiment. [Figure 6A] 5 is a flowchart showing a shooting menu process according to the first embodiment. [Figure 6B] 5 is a flowchart showing a shooting menu process according to the first embodiment. [Figure 7A] 1 is a flowchart showing HDR shooting processing according to the first embodiment. [Figure 7B] 1 is a flowchart showing HDR shooting processing according to the first embodiment. [Figure 7C] FIG. 2 is a diagram showing a CxCy plane according to the first embodiment. [Figure 7D] 1 is a flowchart showing HDR shooting processing according to the first embodiment. [Figure 7E] FIG. 2 is a diagram showing an example of a display image according to the first embodiment. [Figure 7F] FIG. 2 is a diagram showing an example of a display image according to the first embodiment. [Figure 7G] FIG. 4 is a diagram showing an example of a tone correction amount according to the first embodiment. [Figure 7H] FIG. 4 is a diagram showing an example of a tone correction amount according to the first embodiment. [Figure 8] FIG. 2 is a diagram showing an example of a file configuration according to the first embodiment. [Figure 9A] 5 is a flowchart showing a playback mode process according to the first embodiment. [Figure 9B]5 is a flowchart showing a playback mode process according to the first embodiment. [Figure 9C] 5 is a flowchart showing a playback mode process according to the first embodiment. [Figure 9D] 5 is a flowchart showing a playback mode process according to the first embodiment. [Figure 9E] 5 is a flowchart showing a playback mode process according to the first embodiment. [Figure 9F] 5 is a flowchart showing a playback mode process according to the first embodiment. [Figure 9G] 5 is a flowchart showing a playback mode process according to the first embodiment. [Figure 9H] 5 is a flowchart showing a playback mode process according to the first embodiment. [Figure 10A] 5 is a flowchart showing a playback mode process according to the first embodiment. [Figure 10B] 5 is a flowchart showing a playback mode process according to the first embodiment. [Figure 11A] 10 is a flowchart showing a playback menu process according to the first embodiment. [Figure 11B] FIG. 2 is a functional block diagram relating to RAW development processing according to the first embodiment. [Figure 12A] 5 is a flowchart showing an OSD luminance setting process according to the first embodiment. [Figure 12B] 5 is a flowchart showing an OSD luminance setting process according to the first embodiment. [Figure 12C] 5 is a flowchart showing an OSD luminance setting process according to the first embodiment. [Figure 13] FIG. 10 is a diagram showing an example of OSD information display. [Figure 14] FIG. 2 is a diagram showing an OSD information display according to the first embodiment. [Figure 15] FIG. 10 is a diagram showing the correspondence relationship between shooting conditions and luminance gradation values ​​according to the second embodiment. [Figure 16] FIG. 10 is a diagram showing a change in luminance according to the second embodiment. [Figure 17] FIG. 10 is a diagram showing an example of a display image according to the third embodiment. [Figure 18] 11 is a flowchart showing an OSD luminance setting process according to the third embodiment. [Figure 19] FIG. 10 is a diagram showing an example of a display image according to the fourth embodiment. [Figure 20] 10 is a flowchart showing an OSD luminance setting process according to the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the embodiments described below are examples of methods for realizing the present invention, and may be modified or changed as appropriate depending on the configuration of the device to which the present invention is applied and various conditions. Furthermore, the embodiments may be combined as appropriate.

[0011] (Embodiment 1) Preferred embodiments of the present invention will now be described with reference to the drawings. Figures 1A and 1B show external views of a digital camera 100 as an example of an apparatus to which the present invention can be applied. Figure 1A is a front perspective view of the digital camera 100, and Figure 1B is a rear perspective view of the digital camera 100.

[0012] The display unit 28 is located on the rear of the digital camera 100 and displays images and various information. The out-of-viewfinder display unit 43 is located on the top of the digital camera 100 and displays various settings of the digital camera 100, including shutter speed and aperture. The terminal cover 40 protects connectors (not shown) such as a connection cable that connects the digital camera 100 to an external device. The quick-return mirror 12 is raised and lowered by an actuator (not shown) in response to instructions from a system control unit 50 (described below). The communication terminal 10 is a communication terminal that allows the digital camera 100 to communicate with a lens unit 150 (described below; detachable). The eyepiece viewfinder 16 is a peer-type viewfinder that allows the user to check the focus and composition of an optical image of a subject captured through the lens unit 150 by observing a focusing screen 13 (described below). The lid 202 is a cover for a slot that stores a recording medium 200 (described below). The grip portion 90 is a holding portion shaped to be easily gripped with the user's right hand when holding the digital camera 100 .

[0013] The digital camera 100 also has a mode switch 60, a shutter button 61, a main electronic dial 71, a power switch 72, a sub electronic dial 73, a four-way key 74, a SET button 75, and an LV button 76. The digital camera 100 also has a zoom-in button 77, a zoom-out button 78, and a playback button 79. The digital camera 100 may have other operating members. The various operating members will be described later.

[0014] FIG. 2 is a block diagram showing an example of the configuration of the digital camera 100. As shown in FIG.

[0015] The lens unit 150 is a lens unit equipped with an interchangeable photographic lens. The lens 103 is usually composed of multiple lenses, but for simplicity's sake, only a single lens is shown in FIG. 2. The communication terminal 6 is a communication terminal through which the lens unit 150 communicates with the digital camera 100, and the communication terminal 10 is a communication terminal through which the digital camera 100 communicates with the lens unit 150. The lens unit 150 communicates with the system control unit 50 via these communication terminals 6 and 10. The lens unit 150 controls the aperture 1 via the aperture drive circuit 2 using the internal lens system control circuit 4. The lens unit 150 also adjusts the focus by displacing the position of the lens 103 via the AF drive circuit 3 using the lens system control circuit 4.

[0016] The AE (auto exposure) sensor 17 measures the brightness of the subject (subject light) through the lens unit 150 .

[0017] The focus detection unit 11 outputs defocus amount information to the system control unit 50. The system control unit 50 controls the lens unit 150 based on the defocus amount information to perform phase difference AF.

[0018] The quick return mirror 12 (hereinafter referred to as the mirror 12) is raised and lowered by an actuator (not shown) in response to an instruction from the system control unit 50 during exposure, live view photography, video photography, etc. The mirror 12 is a mirror for switching the light beam incident from the lens 103 between the viewfinder 16 side and the imaging unit 22 side. Normally, the mirror 12 is arranged to guide (reflect) the light beam to the viewfinder 16 (mirror down), but when photography or live view photography is performed, the mirror 12 is moved up and down. When view display is performed, the mirror 12 flips up and retreats from the light beam (mirror up) so as to guide the light beam to the imaging unit 22. The mirror 12 is also a half mirror whose center portion is capable of transmitting part of the light beam, and transmits part of the light beam so that it enters the focus detection unit 11 for focus detection.

[0019] By observing the focusing screen 13 through the pentaprism 14 and the viewfinder 16, the photographer can check the focus and composition of the optical image of the subject obtained through the lens unit 150.

[0020] The shutter 101 is a focal plane shutter that can freely control the exposure time of the imaging unit 22 under the control of the system control unit 50.

[0021] The imaging unit 22 is an imaging element that converts an optical image into an electrical signal and is configured with a CCD, CMOS element, etc. The A / D converter 23 converts the analog signal output from the imaging unit 22 into a digital signal.

[0022] The image processing unit 24 performs predetermined processing (pixel interpolation, resizing such as reduction, color conversion, etc.) on data from the A / D converter 23 or data from the memory control unit 15. The image processing unit 24 also performs predetermined arithmetic processing using the captured image data, and the system control unit 50 performs exposure control and distance measurement control based on the obtained arithmetic results. This allows TTL (through-the-lens) AF (autofocus) processing, AE processing, EF (flash pre-flash) processing, etc. to be performed. The image processing unit 24 also performs predetermined arithmetic processing using the captured image data, and TTL AWB (auto white balance) processing based on the obtained arithmetic results.

[0023] The output data from the A / D converter 23 is written into the memory 32 via the image processing unit 24 and the memory control unit 15, or directly via the memory control unit 15. The memory 32 stores image data obtained by the imaging unit 22 and converted into digital data by the A / D converter 23, as well as image data to be displayed on the display unit 28. The memory 32 has a storage capacity sufficient to store a predetermined number of still images and a predetermined period of moving images and audio.

[0024] The memory 32 also serves as a memory (video memory) for image display. The D / A converter 19 converts the image display data stored in the memory 32 into an analog signal and supplies it to the display unit 28. In this way, the display image data written to the memory 32 is displayed by the display unit 28 via the D / A converter 19. The display unit 28 displays an image on a display device such as an LCD in accordance with the analog signal from the D / A converter 19. The digital signals that have been A / D converted by the A / D converter 23 and stored in the memory 32 are D / A converted by the D / A converter 19 and sequentially transferred to and displayed on the display unit 28, thereby realizing the function of an electronic viewfinder and enabling through-image display (live view display). Hereinafter, the image displayed in live view display will be referred to as the "LV image."

[0025] The viewfinder display 41 displays, via the viewfinder display drive circuit 42, a frame (AF frame) indicating the distance measurement point for which autofocus is currently being performed, icons indicating the camera setting status, and the like.

[0026] Various settings of the digital camera 100 such as shutter speed and aperture are displayed on the outside viewfinder display 43 via an outside viewfinder display drive circuit 44 .

[0027] The digital output I / F 90 supplies the image display data stored in the memory 32 to the external device 300 as a digital signal. The image data for display is displayed on the external device 300 .

[0028] The nonvolatile memory 56 is an electrically erasable and recordable memory, such as an EEPROM. The nonvolatile memory 56 stores constants, programs, etc. for the operation of the system control unit 50. The programs referred to here are programs for executing various flowcharts described later in this embodiment.

[0029] The system control unit 50 is a control unit made up of at least one processor or circuit, and controls the entire digital camera 100. The system control unit 50 executes programs recorded in the nonvolatile memory 56 described above to realize each process of this embodiment, which will be described later. The system memory 52 is, for example, a RAM, and the system control unit 50 loads constants and variables for operation of the system control unit 50, programs read from the nonvolatile memory 56, and the like into the system memory 52. ​​The system control unit 50 also performs display control by controlling the memory 32, D / A converter 19, display unit 28, etc.

[0030] The system timer 53 is a timekeeping unit that measures the time used for various controls and the time of a built-in clock.

[0031] The power supply control unit 80 is composed of a battery detection circuit, a DC-DC converter, a switch circuit for switching between powered blocks, etc., and detects whether a battery is installed, the type of battery, and the remaining battery power. The power supply control unit 80 also controls the DC-DC converter based on the detection results and instructions from the system control unit 50, and supplies the required voltage for the required period to each unit, including the recording medium 200. The power supply unit 30 is composed of primary batteries such as alkaline batteries or lithium batteries, secondary batteries such as NiCd batteries, NiMH batteries, or Li batteries, an AC adapter, etc.

[0032] The recording medium I / F 18 is an interface with a recording medium 200 such as a memory card or a hard disk. The recording medium 200 is a recording medium such as a memory card for recording captured images, and is composed of a semiconductor memory, a magnetic disk, or the like.

[0033] The communication unit 54 transmits and receives video signals and audio signals to and from external devices connected wirelessly or via a wired cable. The communication unit 54 can also connect to a wireless LAN (Local Area Network) or the Internet. The communication unit 54 can also communicate with external devices via Bluetooth (registered trademark) or Bluetooth Low Energy. The communication unit 54 can transmit images (including LV images) captured by the imaging unit 22 and images recorded on the recording medium 200, and can receive image data and various other information from external devices.

[0034] The orientation detection unit 55 detects the orientation of the digital camera 100 with respect to the direction of gravity. Based on the orientation detected by the orientation detection unit 55, it is possible to determine whether an image captured by the imaging unit 22 was captured with the digital camera 100 held horizontally or vertically. The system control unit 50 can add orientation information corresponding to the orientation detected by the orientation detection unit 55 to the image file of the image captured by the imaging unit 22 (captured image), or rotate and record the image. An acceleration sensor, a gyro sensor, or the like can be used as the orientation detection unit 55. The acceleration sensor or gyro sensor of the orientation detection unit 55 can also be used to detect movement of the digital camera 100 (panning, tilting, lifting, whether it is stationary, etc.).

[0035] The operation unit 70 is used to input various operation instructions to the system control unit 50. The operation unit 70 includes various operation members as an input unit that accepts operations from the user (user operations). For example, the operation unit 70 includes a push button, a rotary dial, a touch sensor, etc. Specifically, the operation unit 70 includes a mode switch 60, a shutter button 61, a main electronic dial 71, a power switch 72, a sub electronic dial 73, and a four-way key 74. The operation unit 70 also includes a SET button 75, a LV button 76, a zoom in button 77, a zoom out button 78, and a playback button 79. The operation unit 70 also includes an AF-ON button 70b, a quick setting button 70c, an active frame switching button 70d, a menu button 70e, a function button 70f, and an info button 70g. Each operation member of the operation unit 70 is assigned a function appropriate for each situation by selecting and operating various function icons displayed on the display unit 28 or the external device 300, and functions as various function buttons. Examples of the function buttons include an end button, a back button, an image forward button, a jump button, a filter button, and an attribute change button. For example, when the menu button 70e is pressed, a menu screen allowing various settings to be configured is displayed on the display unit 28 or the external device 300. The user can intuitively make various settings using the menu screen displayed on the display unit 28 or the external device 300, the four-way key 74, and the SET button 75.

[0036] The mode selector switch 60 is an operating member for switching between various modes. The mode selector switch 60 switches the operating mode of the system control unit 50 between a still image recording mode, a video shooting mode, a playback mode, etc. Modes included in the still image recording mode include an auto shooting mode, an auto scene determination mode, a manual mode, an aperture priority mode (Av mode), a shutter speed priority mode (Tv mode), and a program AE mode. There are also various scene modes and custom modes that provide shooting settings for specific shooting scenes. The mode selector switch 60 allows the user to directly switch to one of these modes. Alternatively, after switching to a list screen of shooting modes with the mode selector switch 60, the user may selectively switch to one of the displayed modes using another operating member. Similarly, the video shooting mode may also include multiple modes.

[0037] The shutter button 61 is an operating member for issuing a shooting instruction. The shutter button 61 includes a first shutter switch 62 and a second shutter switch 64. The first shutter switch 62 is turned ON when the shutter button 61 is pressed halfway (a shooting preparation instruction) and generates a first shutter switch signal SW1. The system control unit 50 starts operations such as AF processing, AE processing, AWB processing, and EF processing in response to the first shutter switch signal SW1. The second shutter switch 64 is turned ON when the shutter button 61 is pressed fully (a shooting instruction) and generates a second shutter switch signal SW2. The system control unit 50 starts a series of shooting processing operations, from reading out signals from the imaging unit 22 to writing image data to the recording medium 200, in response to the second shutter switch signal SW2.

[0038] The main electronic dial 71 is a rotary operation member, and by turning the main electronic dial 71, settings such as shutter speed and aperture can be changed. The power switch 72 is an operation member that switches the power of the digital camera 100 on and off. The sub electronic dial 73 is a rotary operation member, and by turning the sub electronic dial 73, the selection frame can be moved, images can be advanced, and the like. The four-way key 74 is configured so that each of the up, down, left, and right sections can be pressed. Processing can be performed according to the section of the four-way key 74 that is pressed. The SET button 75 is a push button, and is mainly used to confirm a selection item, etc.

[0039] The LV button 76 is a button for switching live view (hereinafter referred to as LV) ON / OFF in still image shooting mode. In video shooting mode, the LV button 76 is used to instruct the start and stop of video shooting (recording). The enlarge button 77 is an operation button for switching enlargement mode ON / OFF in the live view display in shooting mode and for changing the enlargement ratio in the enlargement mode. In playback mode, the enlargement button 77 is used to zoom in on the playback image. The zoom button 78 functions as a zoom button for zooming in or increasing the magnification ratio. The zoom out button 78 is a button for decreasing the magnification ratio of an enlarged playback image, thereby reducing the displayed image. The playback button 79 is an operation button for switching between shooting mode and playback mode. Pressing the playback button 79 during shooting mode switches to playback mode, and the most recent image recorded on the recording medium 200 can be displayed on the display unit 28.

[0040] The AF-ON button 70b is a button for issuing an instruction to execute AF. The direction in which the AF-ON button 70b is pressed is parallel to the direction (optical axis) of subject light entering the imaging unit 22 from the lens 103.

[0041] The quick setting button 70c (hereinafter referred to as the Q button 70c) is a button for displaying a quick setting menu, which is a list of setting items that can be set in each operating mode. For example, when pressed while waiting to shoot in live view shooting, a list of setting items such as the electronic front curtain shutter, monitor brightness, WB for the live view screen, two-point magnification, and silent shooting is displayed in a row superimposed on the live view. The user can select any option from the displayed quick setting menu using the four-way key 74 and press the SET button 75 to change the setting for the selected setting item or switch to an operating mode.

[0042] The active frame switching button 70d is a button for switching the active enlargement position (frame) between the two enlarged positions during two-point enlargement processing. Different functions are assigned depending on the operation mode, and pressing it in playback mode can add a protection attribute to the displayed image.

[0043] The menu button 70e is a button for displaying on the display unit 28 or the external device 300 a menu screen on which various settings can be made.

[0044] The function buttons 70f are buttons to which different functions can be assigned. Each of the function buttons 70f is located at a position that allows it to be operated with the fingers (middle finger, ring finger, or little finger) of the right hand that holds the grip portion 90, and the pressing direction is parallel to the direction (optical axis) of subject light entering the imaging unit 22 from the lens 103.

[0045] The info button 70g is a button for switching the display of various information.

[0046] The touch panel 70a detects contact with the touch panel 70a. The touch panel 70a and the display unit 28 can be configured as an integrated unit. For example, the touch panel 70a is configured so that its light transmittance does not interfere with the display of the display unit 28, and is attached to the upper layer of the display surface of the display unit 28. Then, input coordinates on the touch panel 70a are associated with display coordinates on the display unit 28. This makes it possible to provide a GUI (Graphical User Interface) that allows the user to directly operate the screen displayed on the display unit 28. The system control unit 50 can detect the following operations or states on the touch panel 70a. A finger or pen that has not been touching the touch panel 70a touches the touch panel 70a again, that is, the start of touching (hereinafter referred to as touch-down). A state in which a finger or a pen is touching the touch panel 70a (hereinafter referred to as Touch-On) A finger or pen is moved while touching the touch panel 70a (hereinafter referred to as Touch-Move). The finger or pen that was touching the touch panel 70a is removed from the touch panel 70a, that is, the touch ends (hereinafter referred to as "touch-up"). A state in which nothing is touching the touch panel 70a (hereinafter referred to as "touch-off" Off)

[0047] When a touch down is detected, a touch on is also detected at the same time. After a touch down, a touch on is usually continued to be detected unless a touch up is detected. If a touch move is detected, a touch on is also detected at the same time. Even if a touch on is detected, a touch move is not detected unless the touch position has moved. When it is detected that all fingers or pens that were touching have touched up, a touch off is detected.

[0048] These operation states and the position coordinates of the finger or pen touching the touch panel 70a are notified to the system control unit 50 via the internal bus. The system control unit 50 then determines what kind of operation (touch operation) was performed on the touch panel 70a based on the notified information. Regarding touch-move, the movement direction of the finger or pen moving on the touch panel 70a can also be determined for each vertical and horizontal component on the touch panel 70a based on changes in the position coordinates. If a touch-move of a predetermined distance or more is detected, it is determined that a slide operation has been performed. An operation in which a finger is touched on the touch panel 70a, moved quickly for a certain distance, and then released is called a flick. In other words, a flick is an operation in which a finger is quickly traced across the touch panel 70a as if flicking it. If a touch-move of a predetermined distance or more at a predetermined speed or more is detected and a touch-up is then detected, it is determined that a flick has been performed (it can be determined that a flick occurred following a slide operation). Furthermore, a touch operation in which multiple points (for example, two points) are touched simultaneously and the touch positions are brought closer together is called a pinch in, and a touch operation in which the touch positions are moved farther apart is called a pinch out. Pinch out and pinch in are collectively called a pinch operation (or simply pinch). The touch panel 70a may be of any of a variety of touch panel types, including resistive film type, capacitive type, surface acoustic wave type, infrared type, electromagnetic induction type, image recognition type, and optical sensor type. There are types that detect a touch by contact with the touch panel, and types that detect a touch by the approach of a finger or pen to the touch panel, and either type is acceptable.

[0049] Further, the present invention is applicable not only to the imaging device main body but also to a control device (display control device) that communicates with an imaging device (including a network camera) via wired or wireless communication and remotely controls the imaging device. Examples of the device for remotely controlling the imaging device include devices such as smartphones, tablet PCs, and desktop PCs. By notifying the imaging device from the control device side of commands for causing the imaging device to perform various operations and settings based on the operations performed on the control device side and the processing performed on the control device side, the imaging device can be remotely controlled. Also, the live view image captured by the imaging device may be received via wired or wireless communication and displayed on the control device side.

[0050] In addition, in each of the above-described embodiments, the case where the present invention is applied to a digital camera has been described as an example, but this is not limited to this example. That is, the present invention is applicable to any device provided with a display unit, such as a PDA, a mobile phone terminal, a portable image viewer, a printer device provided with a display, a digital photo frame, a music player, a game machine, and an electronic book reader.

[0051] FIG. 3 is a diagram showing the digital camera 100 and an external device 300. When the digital camera 100 and the external device 300 are connected by a connection cable 302, the display unit 28 of the digital camera 100 turns off (no display), and the screen displayed on the display unit 28 is displayed on the display 301 of the external device 300.

[0052] <LV shooting mode process> FIG. 4A is a flowchart showing the LV shooting mode process in the digital camera 100 . This process is realized by expanding the program recorded in the nonvolatile memory 56 into the system memory 52 and executing it by the system control unit 50.

[0053] In S401, the system control unit 50 determines whether or not the HDR shooting mode is set. If the HDR shooting mode is set, the process proceeds to S402, and if not (if the SDR shooting mode is set), the process proceeds to S422.

[0054] In S402, the system control unit 50 determines whether or not the digital camera 100 is connected to the external device 300. If it is connected, the process proceeds to S403, and if not, the process proceeds to S404.

[0055] In S403, the system control unit 50 performs connection processing between the digital camera 100 and the external device 300. Details of the connection processing will be described later with reference to Fig. 5. In this embodiment, if the external device supports HDR connection, an HDR connection is established, and if it does not support HDR connection, an SDR connection is established.

[0056] In S404, the system control unit 50 (image processing unit 24) develops the image captured by the imaging unit 22 and converted into a digital signal by the A / D converter 23 in HDR (HDR image quality).

[0057] In S405, the system control unit 50 determines whether the display unit 28 or the external device 300 that displays the LV image is HDR-compatible (compatible with HDR display). If it is HDR-compatible, the system control unit 50 proceeds to S409, and if it is not, the system control unit 50 proceeds to S406.

[0058] In S406, the system control unit 50 determines whether the HDR assist display setting is "Assist 1" or "Assist 2." If the HDR assist display setting is "Assist 1," the process proceeds to S407; if it is "Assist 2," the process proceeds to S408. Here, HDR assist display is a function for displaying an image so that the user can check the gradation of the HDR image data by converting an HDR image into an SDR image while prioritizing a specific range of the HDR image. In this embodiment, the HDR assist display setting can be switched (changed) between two settings, "Assist 1" and "Assist 2." "Assist 1" is a setting that prioritizes high brightness, and "Assist 2" is a setting that prioritizes low brightness. The HDR assist display setting can be changed by user operation. An HDR image is an image whose dynamic range is wider than a predetermined range. An SDR image is an image whose dynamic range is narrower than the predetermined range.

[0059] In S407, the system control unit 50 performs SDR conversion processing (processing for converting from HDR to SDR) on the HDR LV image developed in S404 in accordance with the Assist 1 setting, and displays the LV image in SDR (SDR image quality) on the display unit 28 or the external device 300. Note that the system control unit 50 also performs resizing processing to a size suitable for the display unit 28 or the external device 300.

[0060] In S408, the system control unit 50 performs SDR conversion processing (processing for converting from HDR to SDR) on the HDR LV image developed in S404 in accordance with the Assist 2 setting, and displays the LV image in SDR on the display unit 28 or the external device 300. Note that the system control unit 50 also performs resizing processing to a size suitable for the display unit 28 or the external device 300, similar to S407.

[0061] In S409, the system control unit 50 displays the HDR LV image developed in S404 as is on the display unit 28 or the external device 300. Note that, similar to S407, the system control unit 50 performs resizing processing to a size suitable for the display unit 28 or the external device 300. This shall be the case.

[0062] In S410, the system control unit 50 determines whether or not the menu button 70e has been pressed. If it has been pressed, the process proceeds to S411, and if not, the process proceeds to S412.

[0063] In S411, the system control unit 50 performs a shooting menu process, and then the process proceeds to S412. Details of the shooting menu process will be described later with reference to Figures 6A and 6B.

[0064] In S412, the system control unit 50 determines whether the info button 70g has been pressed. If it has been pressed, the process proceeds to S413, and if not, the process proceeds to S414.

[0065] In S413, the system control unit 50 switches the display of shooting information, such as a histogram or highlight warning.

[0066] In S414, the system control unit 50 determines whether or not the shutter button 61 is half-pressed (SW1). If it is half-pressed, the process proceeds to S415, and if not, the process proceeds to S420.

[0067] In S415, the system control unit 50 performs the above-described AF processing / AE processing.

[0068] In S416, the system control unit 50 determines whether the shutter button 61 is fully pressed (SW2). If it is fully pressed, the process proceeds to S418, and if not, the process proceeds to S417.

[0069] In S417, the system control unit 50 determines whether or not the half-pressed (SW1) state of the shutter button 61 is being maintained. If it is being maintained, the process proceeds to S415, and if not, the process proceeds to S420.

[0070] In S418, the system control unit 50 performs HDR shooting processing. Details of the HDR shooting processing will be described later with reference to Figures 7A to 7H.

[0071] In S419, the system controller 50 performs a quick review display process, which will be described in detail later with reference to FIG.

[0072] In S420, the system control unit 50 determines whether or not the LV button 76 (movie button) has been pressed. If it has been pressed, the process proceeds to S421, and if not, the process proceeds to S438.

[0073] In S421, the system control unit 50 HEVC compresses the image HDR developed in S404 and records it as an HDR video file.

[0074] In S422, the system control unit 50 determines whether or not the digital camera 100 is connected to the external device 300. If it is connected, the process proceeds to S423, and if not, the process proceeds to S424.

[0075] In S423, the system control unit 50 performs connection processing between the digital camera 100 and the external device 300. Details of the connection processing will be described later with reference to Fig. 5. Here, because the SDR shooting mode is in effect, the digital camera 100 and the external device 300 are connected in SDR mode.

[0076] In S424, the system control unit 50 (image processing unit 24) develops the image captured by the imaging unit 22 and converted into a digital signal by the A / D converter 23 in SDR (SDR image quality).

[0077] In S425, the system control unit 50 displays the SDR LV image developed in S424 as is on the display unit 28 or the external device 300. Note that the system control unit 50 performs resizing processing to a size suitable for the display unit 28 or the external device 300, similar to S407.

[0078] The processing of S426 to S433 is the same as the processing of S410 to S417 described above, and therefore a description thereof will be omitted.

[0079] In S434, the system control unit 50 performs SDR shooting processing. The SDR shooting processing is performed in the same manner as the HDR shooting processing described later with reference to Figures 7A to 7H. In the case of SDR shooting processing, in S710, the system control unit 50 compresses the SDR image that was SDR-developed in S707 as the main image, rather than the HDR-developed image.

[0080] In S435, the system controller 50 performs a quick review display process, which will be described in detail later with reference to FIG.

[0081] In S436, the system control unit 50 determines whether or not the LV button 76 has been pressed. If it has been pressed, the process proceeds to S437, and if not, the process proceeds to S438.

[0082] In S437, the system control unit 50 compresses the image SDR developed in S424 using H264 and records it as an SDR video file.

[0083] In S438, the system control unit 50 determines whether or not the playback button 79 has been pressed. If it has been pressed, the process proceeds to S439, and if not, the process proceeds to S440.

[0084] In S439, the system control unit 50 performs playback mode processing. Details of the playback mode processing will be described later with reference to Figures 9A to 9H, 10A and 10B.

[0085] In S440, it is determined whether or not the LV shooting mode processing is to be ended. If it is to be ended, this processing flow is ended, and if not, the process proceeds to S401.

[0086] <Quick review display process (S419, S435)> 4B is a flowchart showing the quick review display process (S419, S435) according to this embodiment. This process is implemented by the system control unit 50 loading a program stored in the non-volatile memory 56 into the system memory 52 and executing it.

[0087] In S451, the system control unit 50 determines whether the quick review display is set to "Yes." If it is set to "Yes," the process proceeds to S452; if it is not set to "No," the process ends.

[0088] In S452, the system control unit 50 determines whether or not the image was captured in HDR shooting mode. If the image was captured in HDR shooting mode, the process proceeds to S453. If not (if the image was captured in SDR shooting mode), the process proceeds to S460.

[0089] In S453, the system control unit 50 determines whether the display unit 28 or external device 300 that performs the quick review display is HDR-compatible. If it is HDR-compatible, the process proceeds to S457; if it is not, the process proceeds to S454.

[0090] In S454, the system control unit 50 determines whether or not the image was taken using RAW still image shooting. If it is taken using RAW still image shooting, the process proceeds to S455. If it is not taken using RAW still image shooting (H EI F In the case of still image shooting, proceed to S456.

[0091] In S455, the system control unit 50 converts the display HDR image in the RAW image file to SDR (converts from HDR to SDR) and displays it in SDR on the display unit 28 or the external device 300. Note that the system control unit 50 may also perform resizing processing to a size suitable for the display unit 28 or the external device 300.

[0092] In S456, the system control unit 50 converts the display HDR image in the HEIF image file to SDR (converts from HDR to SDR) and displays it in SDR on the display unit 28 or the external device 300. Note that the system control unit 50 may also perform resizing processing to a size suitable for the display unit 28 or the external device 300, similar to S455.

[0093] In S457, the system control unit 50 determines whether or not the image was taken using RAW still image shooting, similar to S454. If it is taken using RAW still image shooting, the process proceeds to S458. If it is not taken using RAW still image shooting (H EI F In the case of still image shooting, proceed to S459.

[0094] In S458, the system control unit 50 displays the display HDR image in the RAW image file in HDR on the display unit 28 or the external device 300. Note that the system control unit 50 may perform resizing processing to a size suitable for the display unit 28 or the external device 300, similar to S455.

[0095] In S459, the system control unit 50 displays the display HDR image in the HEIF image file in HDR on the display unit 28 or the external device 300. Note that the system control unit 50 may perform resizing processing to a size suitable for the display unit 28 or the external device 300, similar to S455.

[0096] In S460, the system control unit 50 determines whether or not the image was captured using RAW still image shooting. If it was captured using RAW still image shooting, the process proceeds to S461, and if not (JPEG still image shooting), the process proceeds to S462.

[0097] In S461, the system control unit 50 displays the display SDR image in the RAW image file in SDR on the display unit 28 or the external device 300. Note that the system control unit 50 may also perform resizing processing to a size suitable for the display unit 28 or the external device 300, as in S455.

[0098] In S462, the system control unit 50 displays the display SDR image in the JPEG image file in SDR on the display unit 28 or the external device 300. Note that the system control unit 50 may also perform resizing processing to a size suitable for the display unit 28 or the external device 300, as in S455.

[0099] In S463, the system control unit 50 determines whether or not the shutter button 61 has been pressed (halfway down, SW1). If it has been pressed, this processing flow ends, and if not, the process proceeds to S464.

[0100] In S464, the system controller 50 determines whether the time for the quick review display (displayed by the quick review display process) has elapsed for a predetermined set time or more. If so, the process ends; if not, the process proceeds to S463. Note that the set time can be set and changed by the user.

[0101] <Connection process (S403, S423)> 5A is a sequence diagram showing the connection process (S403, S423) when the digital camera 100 is connected to the external device 300. In this embodiment, the description will be given assuming that the digital camera 100 and the external device 300 are connected.

[0102] In S501, the system control unit 50 controls the digital output I / F 90 to start transmitting a +5V signal to the external device 300. The transmitted +5V signal is transmitted to the external device 300 via a +5V signal line (not shown) of the connection cable 302. The external device 300 receives the +5V signal from the connection cable 302 and proceeds to S502.

[0103] In S502, the external device 300 detects that the digital camera 100 has been connected to the external device 300 via the connection cable 302, and the process proceeds to S503.

[0104] In S503, the external device 300 starts transmitting an HPD (Hot Plug Detect) signal to the digital camera 100 via an HPD signal line (not shown) of the connection cable 302. The digital output I / F 90 of the digital camera 100 receives the transmitted HPD signal via the connection cable 302. Upon receiving the HPD signal, the digital output I / F 90 notifies the system control unit 50 of the reception of the HPD signal, and the process proceeds to S504.

[0105] In S504, the system control unit 50 detects the connection of the external device 300 by receiving a notification of the reception of the HPD signal, and proceeds to S505.

[0106] In S505, the system control unit 50 controls the digital output I / F 90 to transmit an EDID (Extended Display Identification Data) request signal from the connection cable 302. The transmitted EDID request signal is transmitted to the external device 300 via an EDID signal line (not shown) of the connection cable 302. The external device 300 receives the EDID request signal and proceeds to S506.

[0107] In S506, the external device 300 transmits the EDID from an EDID signal line (not shown) of the connection cable 302. The digital output I / F 90 of the digital camera 100 receives the transmitted EDID via the connection cable 302. Upon receiving the EDID, the digital output I / F 90 notifies the system control unit 50 of the reception of the EDID, and the process proceeds to S507.

[0108] In S507, the system control unit 50 instructs the digital output I / F 90 to copy the EDID received in S506 to the memory 32. After the copy is complete, the system control unit 50 analyzes the EDID stored in the memory 32, determines the video signal that the external device 300 can accept (the capabilities of the external device 300), and proceeds to S508.

[0109] In S508, the system control unit 50 determines whether to output an HDR signal or an SDR signal. In this embodiment, the system control unit 50 outputs an HDR signal to the external device 300 if the digital camera 100 is set to HDR-enabled and the external device 300 is HDR-compatible, and outputs an SDR signal otherwise. Note that an example of an external device 300 being HDR-compatible is when the video signal that the external device 300 can accept, as determined in S507, includes an HDR signal.

[0110] In S509, the system control unit 50 instructs the digital output I / F 90 to start transmitting the signal determined in S508. Upon receiving the instruction to start transmission, the digital output I / F 90 starts transmitting the video signal (SDR signal or HDR signal) via the connection cable 302 and proceeds to S510.

[0111] In S510, the system control unit 50 outputs a video signal to the external device 300 via a TMDS (Transition Minimized Differential Signaling) signal line (not shown) of the connection cable 302. The external device 300 receives the video signal via the TMDS signal line (not shown) of the connection cable 302 and proceeds to S511.

[0112] In S511, the external device 300 analyzes the video signal received in S510, switches the driving of the display 301 to a setting that allows the video signal to be displayed, and proceeds to S512.

[0113] In S512, the external device 300 displays the video signal received in S510 on the display 301 of the external device 300.

[0114] 5B and 5C are diagrams showing the processing performed when the digital camera 100 and external device 300 are connected and video is displayed on the external device 300, and the video output is switched from an SDR image to an HDR image (or from an HDR image to an SDR image).

[0115] 5B is a sequence diagram showing the process of switching the video output from digital camera 100 to external device 300 from an SDR image to an HDR image. Here, it is assumed that connection between digital camera 100 and external device 300 has been completed.

[0116] In S521, the system control unit 50 instructs the digital output I / F 90 to transmit an SDR video signal (SDR signal), and the process proceeds to S522.

[0117] In S522, the system control unit 50 outputs the SDR video signal to the external device 300 via the TMDS signal line (not shown) of the connection cable 302. The external device 300 receives the SDR video signal via the TMDS signal line (not shown) of the connection cable 302 and proceeds to S523.

[0118] In S523, the external device 300 displays the SDR video signal received in S522 on the display 301 of the external device 300. While the digital camera 100 is outputting the SDR video signal, the processes of S521 to S523 are repeated, thereby displaying an SDR image on the display 301 of the external device 300. When the digital camera 100 switches the video output to the external device 300 from an SDR image to an HDR image, the processes of S524 and onwards are executed.

[0119] In S524, the system control unit 50 controls the digital output I / F 90 to instruct it to stop outputting the SDR video signal, and then the process proceeds to S525.

[0120] In S525, the system control unit 50 notifies the external device 300 of the intention to stop transmission (output) of the video signal via the TMDS signal line (not shown) of the connection cable 302. The external device 300 receives the notification of the intention to stop transmission of the SDR video signal via the TMDS signal line (not shown) of the connection cable 302, and proceeds to S526.

[0121] In S526, since the external device 300 has stopped receiving the video from the digital camera 100, the external device 300 stops displaying the video on the display 301 of the external device 300, and the process proceeds to S527.

[0122] In S527, the system control unit 50 instructs the digital output I / F 90 to transmit an HDR video signal, and then proceeds to S528.

[0123] In S528, the system control unit 50 outputs the HDR video signal via a TMDS signal line (not shown) of the connection cable 302. The external device 300 outputs the HDR video signal via the TMDS signal line (not shown) of the connection cable 302. The HDR video signal is received via an S signal line (not shown) and the process proceeds to S529.

[0124] In S529, the external device 300 analyzes the video signal received in S528, switches the driving of the display 301 to a setting for displaying the HDR video signal, and proceeds to S530.

[0125] In S530, the external device 300 displays the HDR video signal received in S528 on the display 301 of the external device 300. At this time, the processing time of S529 to S530 varies depending on the performance of the external device 300, and it may take about 1 to 5 seconds for the video to be displayed.

[0126] 5C is a sequence diagram showing the process of switching the video output from digital camera 100 to external device 300 from HDR images to SDR images. Here, it is assumed that connection between digital camera 100 and external device 300 has been completed.

[0127] In S541, the system control unit 50 instructs the digital output I / F 90 to transmit an HDR video signal (HDR signal), and proceeds to S542.

[0128] In S542, the system control unit 50 outputs the HDR video signal to the external device 300 via the TMDS signal line (not shown) of the connection cable 302. The external device 300 receives the HDR video signal via the TMDS signal line (not shown) of the connection cable 302 and outputs S5 4 Go to step 3 .

[0129] In S543, the external device 300 displays the HDR video signal received in S542 on the display 301 of the external device 300. While the digital camera 100 is outputting the HDR video signal, the processes of S541 to S543 are repeated, thereby displaying an HDR image on the display 301 of the external device 300. When the digital camera 100 switches the video output to the external device 300 from an HDR image to an SDR image, the processes of S544 and onwards are executed.

[0130] In S544, the system control unit 50 instructs the digital output I / F 90 to stop outputting the HDR video signal, and then proceeds to S545.

[0131] In S545, the system control unit 50 notifies the external device 300 to stop transmitting (outputting) the video signal via the TMDS signal line (not shown) of the connection cable 302. The external device 300 receives the notification to stop transmitting the HDR video signal via the TMDS signal line (not shown) of the connection cable 302, and proceeds to S546.

[0132] In S546, since the external device 300 has stopped receiving the video from the digital camera 100, the external device 300 stops displaying the video on the display 301 of the external device 300, and the process proceeds to S547.

[0133] In S547, the system control unit 50 instructs the digital output I / F 90 to transmit an SDR video signal, and then the process proceeds to S548.

[0134] In S548, the digital camera 100 outputs an SDR video signal via a TMDS signal line (not shown) of the connection cable 302. The external device 300 receives the SDR video signal via a TMDS signal line (not shown) of the connection cable 302 and proceeds to S549.

[0135] In S549, the external device 300 analyzes the video signal received in S548, switches the driving of the display 301 to a setting for displaying the SDR video signal, and returns to S5 5 Go to 0.

[0136] In the S550, the external device 300 is 4 The SDR video signal received in step 8 is displayed on the display 301 of the external device 300. At this time, the processing time of steps S549 to S550 varies depending on the performance of the external device 300, and it may take about 1 to 5 seconds for the video to be displayed.

[0137] <Shooting menu processing (S411, S427)> 6A and 6B are flowcharts showing details of the shooting menu processing according to this embodiment. This processing is realized by the system control unit 50 loading a program recorded in the nonvolatile memory 56 into the system memory 52 and executing it.

[0138] In S601, the system control unit 50 determines whether or not the HDR shooting mode is set. If the HDR shooting mode is set, the process proceeds to S603, and if not (if the SDR shooting mode is set), the process proceeds to S602.

[0139] In S602, the system control unit 50 displays a menu for normal SDR shooting. In S603, the system control unit 50 displays the menu with functions that are not used in conjunction with HDR shooting disabled (grayed out).

[0140] In S604, the system control unit 50 determines whether or not a setting item related to HDR shooting has been selected by the user. If selected, the process proceeds to S605, and if not, the process proceeds to S610.

[0141] In S605, the system control unit 50 determines whether the user has enabled the setting for whether to perform HDR shooting. If the setting has been enabled, the system control unit 50 proceeds to S606, and if not, the system control unit 50 proceeds to S607.

[0142] In S606, the system control unit 50 changes the setting for whether or not to perform HDR shooting to valid, and records the setting value in the system memory 52.

[0143] In S607, the system control unit 50 determines whether the HDR assist display setting has been changed by the user. If it has been changed, the process proceeds to S608; if not, the process proceeds to S609. Note that it is desirable not to be able to change the HDR assist display setting when the setting for whether to perform HDR shooting is disabled.

[0144] In S608, the system control unit 50 changes the setting for HDR assist display during shooting and records the setting value in the system memory 52. ​​There may be two or more setting values ​​(options) as variations when the HDR assist display setting is set to "ON."

[0145] In S609, the system control unit 50 determines whether or not an instruction to end the HDR setting has been given by the user. If an instruction to end the HDR setting has not been given, the system control unit 50 proceeds to S604, and if an instruction to end the HDR setting has been given, the system control unit 50 proceeds to S610.

[0146] In S610, the system control unit 50 determines whether or not the user has selected a setting item for the still image recording quality. If selected, the process proceeds to S611; if not, the process proceeds to S651 (FIG. 6B).

[0147] In S611, the system control unit 50 determines whether or not the HDR shooting mode is set. If the HDR shooting mode is set, the process proceeds to S612, and if not (if the SDR shooting mode is set), the process proceeds to S614.

[0148] In S612, the system control unit 50 displays a setting screen for HDR shooting. In this example, the system control unit 50 accepts a user selection of the recording image quality for HDR shooting. The user can select from among "RAW image," "HDR still image file," and "simultaneous output of two images: RAW and HDR still image file." The image size can also be selected from among "Large," which is close to the number of pixels read from the sensor, "Middle," which is slightly smaller, and "Small," which is even smaller. Furthermore, the compression rate for reducing (compressing) the file size can also be selected from among "high image quality (low compression rate)," "standard (high compression rate)," and "low image quality (high compression rate)."

[0149] In S614, the system control unit 50 displays a setting screen for SDR shooting. In S615, the system control unit 50 accepts the recording image quality for SDR shooting selected by the user. As described above, there are multiple options for the setting recording image quality for SDR shooting.

[0150] In S651 (FIG. 6B), the system control unit 50 determines whether or not the user has selected a setting item for the video recording quality. If selected, the process proceeds to S652; if not, the process proceeds to S657.

[0151] In S652, the system control unit 50 determines whether or not the HDR shooting mode is set. If the HDR shooting mode is set, the process proceeds to S653, and if not (if the SDR shooting mode is set), the process proceeds to S655.

[0152] In S653, the system control unit 50 displays a setting screen for HDR shooting. In S654, the system control unit 50 accepts a user selection of the recording image quality for HDR shooting. The user can select one of the following recording image quality settings for HDR shooting: "RAW video," "RAW video + proxy video," "HDR video file," and "simultaneous output of three videos: RAW + proxy video + HDR video file." Also, the user can select image sizes such as "8K," "4K," "FullHD," "HD," and "VGA." Furthermore, the user can select compression rates for reducing (compressing) the file size, such as "high image quality (low compression rate)" (e.g., ALL-I), "standard" (e.g., IPB), and "low image quality (high compression rate)." Other options include the frame rate and broadcasting system (e.g., NTSC / PAL).

[0153] In S655, the system control unit 50 displays a setting screen for SDR shooting. In S656, the system control unit 50 accepts a user selection of the recording image quality for SDR shooting. As with HDR shooting, multiple options are provided for the setting recording image quality for SDR shooting.

[0154] In S657, the system control unit 50 determines whether or not the user has selected a setting item for HDR output. If selected, the process proceeds to S658, and if not, the process proceeds to S660.

[0155] In S658, the system control unit 50 determines whether or not the HDR output setting has been enabled by the user. If it has been enabled, the process proceeds to S659; if not, the process proceeds to S660.

[0156] In S659, the system control unit 50 changes the HDR output setting to enabled and records the setting value in the system memory 52.

[0157] In S660, the system control unit 50 determines whether or not the user has selected a setting item for view assistance during playback. If selected, the process proceeds to S661; if not, the process proceeds to S662. Proceed to S663.

[0158] In S661, the system control unit 50 determines whether the view assist setting during playback has been enabled by the user. If it has been enabled, the process proceeds to S662, and if not, the process proceeds to S663.

[0159] In S662, the system control unit 50 changes the view assist setting during playback to valid, and records the setting value in the system memory 52.

[0160] In S663, the system control unit 50 determines whether or not the user has selected a setting item for SDR conversion during transfer. If selected, the process proceeds to S664; if not, the process proceeds to S665.

[0161] In S664, the system control unit 50 switches the SDR conversion setting during transfer between enabled and disabled.

[0162] In S665, the system control unit 50 determines whether or not the user has selected any other setting items. If selected, the process proceeds to S666; if not, the process proceeds to S667. In S666, the system control unit 50 performs processing associated with the other setting items.

[0163] In S667, the system control unit 50 determines whether or not an instruction to end the shooting menu processing has been received from the user. If an instruction to end the shooting menu processing has been received, this processing flow ends; if not, the process proceeds to S601.

[0164] <HDR shooting processing (S418)> 7A is a flowchart showing details of the HDR shooting process (S418) according to this embodiment. An example in which the image processing unit 24 performs HDR development on RAW data written to the memory 32 will be described.

[0165] Imaging devices such as digital cameras and digital video cameras have a white balance function that corrects the color tone of a captured image depending on the light source used at the time of shooting. The white balance function corrects for differences in color tone that vary depending on the light source (natural light sources such as sunny or cloudy days, or artificial light sources such as fluorescent lights or incandescent lights), so that the whiteness appears the same regardless of the light source.

[0166] In S701, the system control unit 50 acquires RAW data via the memory control unit 15.

[0167] In S702, the system control unit 50 performs a process of determining pixels that appear to be white (a process of determining whether a pixel is within a white search frame) based on the acquired RAW data.

[0168] In S703, the system control unit 50 calculates white balance coefficients necessary for white balance processing based on the result of S702. Details of the processing of S702 and S703 (calculation processing of white balance coefficients) will be described later using the flowchart in FIG.

[0169] In S704 to S706, the system control unit 50 calculates tone correction parameters necessary for the tone correction process. Details of the tone correction parameter calculation process will be described later using the flowchart in Fig. 7D.

[0170] In S707, the system control unit 50 performs development using the calculated white balance coefficients, tone correction parameters, and various HDR parameters. Note that the development parameters may also be a color matrix, camera OETF curve data, color adjustment parameters, noise reduction parameters, sharpness parameters, etc. As the camera OETF, for example, the PQ (Perceptual Quantum) standardized in ST.2084 may be used. a The inverse characteristics of the EOTF (Electro-Optical Transfer Function) of the ARIB standard are assumed, but the OOTF (Opto-Optical Transfer Function) on the camera side may also be combined. The HLG (Hybrid Log-Gamma) OETF developed in STD-B67 may also be used.

[0171] In S708, the system control unit 50 resizes the image developed in S707 to generate an MPF ​​(Multi Pixel Format) image for use as a two-screen comparison image (image for simple display), and compresses it.

[0172] In S709, the system control unit 50 further resizes the MPF image generated in S708 to generate thumbnail images to be used for index display (index playback) and the like, and compresses them.

[0173] In S710, the system control unit 50 compresses the HDR image developed in S707 as the main image. There are various possible compression methods, but for example, 10-bit YUV422 data can be compressed using H.265 (ISO / IEC 23008-2 HEVC).

[0174] In S711, the system control unit 50 determines whether RAW recording is to be performed based on the user settings (recording image quality settings). If RAW recording is to be performed, the process proceeds to S712, and if not, the process proceeds to S713.

[0175] In S712, the system control unit 50 compresses the RAW image and records it on the recording medium 200 via the recording medium I / F 18. The compressed RAW image is recorded as a file with a header attached. Several compression methods are possible, including lossless compression, which is lossless and reversible, and lossy compression, which is lossy but reduces file size. The header also records the processing results of S702, the histogram calculated in S704, and the face detection results calculated in S705 as detection metadata. The header also records HDR development parameters, such as the white balance coefficient and tone correction amount if the user has selected HDR shooting, as well as the MPF image for display generated in S708.

[0176] Note that development parameters will be SDR development parameters when shooting in SDR mode. However, if there is ample processing speed, such as in single-shot mode, it is also possible to generate SDR development parameters when shooting in HDR mode and record both (Figures 8A and 8C). Furthermore, when there is ample processing speed, such as in single-shot mode, it is also possible to create an SDR-quality main image, MPF image, and thumbnail image using SDR development parameters in addition to the HDR display image, and record the HDR display image and SDR display image in the same file (Figure 8D). Furthermore, when displaying thumbnails, since the images are small, it is sufficient to know what the image is like. Therefore, it is acceptable to create and save only the thumbnail image created in S709 as an SDR-developed image (Figure 8E). This configuration makes it possible to display only the thumbnail image even on products that do not support decoding of H.265, the HDR compression format.

[0177] In S713, the system control unit 50 compresses the developed HDR image, adds static metadata or dynamic metadata, and records the compressed image as a file on the recording medium 200 via the recording medium I / F 18. Examples of metadata include the x and y coordinates of the three primary colors and white point of the ray, the maximum luminance value and the minimum luminance value of the mastering display, etc. Static metadata includes the maximum content luminance value (Maximum Content Light Level) and the frame-average luminance level (Maximum Frame-average Light Level). Dynamic metadata includes metadata for dynamic tone mapping of color volume conversion defined in SMPTE ST 2094. Representing HDR characteristics with a PQ signal requires a minimum depth of 10 bits, but the conventional JPEG format only allows for 8 bits, so a new container for HDR still images must be adopted. For this reason, this embodiment uses a container in the High Efficiency Image File Format (HEIF). HEIF is an image file format developed by the Moving Picture Experts Group (MPEG) and defined in MPEG-H Part 12 (ISO / IEC 23008-12). HEIF is characterized by its ability to store not only the original image but also thumbnails, multiple temporally related images, and metadata such as EXIF ​​and XMP in a single file. Therefore, it can also store 10-bit image sequences encoded in HEVC, making it easy to reuse.

[0178] <White balance coefficient calculation process (S702, S703)> FIG. 7B is a flowchart showing the details of the white balance coefficient calculation process (S702, S703) according to this embodiment.

[0179] In S721, the system control unit 50 performs de-Bayering. RAW data contains only one of the R, G, and B signals for each pixel. To perform a white search, the data must be converted into color signals, so de-Bayering is performed. There are several de-Bayering methods for generating signals for all R, G, and B channels. For example, signals are generated by linear interpolation using a low-pass filter.

[0180] In S722, the system control unit 50 performs a process of subtracting the optical black component from the de-Bayered signal. Generally, RAW data is affected by noise, so optical black (OB) has a value other than 0. Therefore, it is necessary to subtract the OB component from the de-Bayered signal.

[0181] In S723, the system control unit 50 calculates color signals Cx and Cy from the acquired RGB signals using the following equation (1): where Cx represents the color temperature, Cy corresponds to the green correction amount, and Yi is the luminance value.

number

[0182] The CxCy plane is illustrated in Figure 7C. As shown in Figure 7C, an imaging device captures images of white in advance under conditions ranging from high color temperatures (such as daytime) to low color temperatures (such as dusk), and the color evaluation values ​​Cx and Cy are plotted on a coordinate system, thereby determining a white axis 1200 that serves as a reference for detecting white. Because there is some variation in the white color of an actual light source, processing is performed to add width to both sides of the white axis 1200 (S724). This (white axis with width) is called a white search frame 1201.

[0183] In S725, the system control unit 50 plots each pixel after de-Bayering in the CxCy coordinate system and determines whether it is within the white search frame 1201.

[0184] In S726, the system control unit 50 performs a light / dark exclusion process to limit the pixels included in the white search frame to be integrated in the luminance direction. This process is performed to prevent dark colors from being easily affected by noise, which would reduce the accuracy of the white balance coefficient calculation. Similarly, bright colors are performed to prevent sensor saturation in one of the channels, which would disrupt the balance of the R / G or B / G ratio and cause the color to deviate from the correct color, reducing the accuracy of the white balance coefficient calculation. In this case, while SDR targets brightness up to +1EV, HDR targets brightness up to +2EV, allowing for the calculation of white balance coefficients that are more optimized for HDR.

[0185] In S727, the system control unit 50 calculates the integral values ​​SumR, SumG, and SumB of the color evaluation values ​​from Cx and Cy that are within the white search frame and have been subjected to the light and dark exclusion process (white node analysis process).

[0186] In S728, the system control unit 50 calculates white balance coefficients WBCo_R, WBCo_G, and WBCo_B from the calculated integrated values ​​using the following equation (2): Note that the white balance coefficients (WB coefficients) may be calculated for the shooting mode (SDR shooting or HDR shooting) set by the user, or may be calculated for both SDR and HDR.

number

[0187] <Gradation correction parameter calculation process (S704 to S706)> FIG. 7D is a flowchart showing the details of the tone correction parameter calculation process (S704 to S706) according to this embodiment.

[0188] In S731, the system control unit 50 performs WB processing using the WB coefficients generated in the processing of S701 to S703 in FIG. 7A.

[0189] In S732, the system control unit 50 performs histogram detection. Specifically, the system control unit 50 applies the white balance gain value obtained in S731 to the entire image data. Furthermore, the system control unit 50 creates a histogram as luminance information from the pixel values ​​(image) that have undergone gamma correction processing. While a known lookup table method may be used for gamma correction processing, it is preferable to use the same gamma characteristics as those used in development. However, to save processing time and memory, simplified gamma characteristics, such as gamma characteristics approximated by a broken line, may also be used. Note that the edges of an image are generally unimportant and may be affected by reduced peripheral illumination depending on the imaging lens. Therefore, the histogram may be created excluding pixels at the edges.

[0190] In S733, the system control unit 50 performs preprocessing for face detection. This is to make it easier to detect faces by performing reduction processing, gamma processing, etc. on the image data. Note that various known methods can be applied to the preprocessing for face detection.

[0191] In S734, the system control unit 50 performs face detection processing on the preprocessed image data using a known method. The face detection processing results in a region where a face is detected (face region) and the reliability of the detection.

[0192] In S735, the system control unit 50 calculates a gradation correction amount (gradation correction amount (A)) for compensating for the exposure correction amount (reduction amount) as the first gradation correction amount. At this time, the gradation correction amount of the input / output characteristics is calculated so that dark areas of the image are properly exposed, while high-brightness pixels above a predetermined brightness level are not corrected (at least, the exposure correction amount is not completely compensated for). This makes it possible to further prevent bright areas from becoming overexposed after gradation correction. This gradation correction amount can be prepared in advance as a plurality of correction tables corresponding to the exposure correction amount.

[0193] In S736, the system control unit 50 determines whether a face has been detected. Specifically, the system control unit 50 determines that a face has been detected if there is a face area whose reliability is higher than a preset evaluation threshold among the face areas detected by the face detection process in S734. If a face has been detected, the system control unit 50 proceeds to S737; if not, the system control unit 50 proceeds to S741.

[0194] In S737, the system control unit 50 extracts a part of the detected face area as a face luminance acquisition area. The face luminance acquisition area is an area for acquiring the luminance of a bright part of the face, and there are no particular restrictions on the number or position of the face luminance acquisition areas.

[0195] In S738, the system control unit 50 calculates the average value for each type of R, G, and B pixel included in each face luminance acquisition area (calculating face luminance). Furthermore, a white balance gain value is applied to each average value of the RGB pixels in the same manner as in histogram detection, and after performing gamma correction, the result is converted to a luminance value Y using the following equation (3). Note that the white balance gain value applied in histogram detection and face detection is preferably the gain value used in WB processing for the same image data. Ideally, the luminance gamma should be the same as that used in development, but a simplified gamma characteristic, such as a gamma characteristic approximated by a broken line, may be used to save processing time and memory.

number

[0196] In S739, the system control unit 50 converts the luminance value calculated for each face luminance acquisition area in S738 into a value assumed for a proper exposure. This is a process to correct the fact that the face luminance is detected as lower than when the image data is captured at a lower exposure than the proper exposure. The luminance value may be converted to compensate for the exposure correction amount (reduction amount) determined by the exposure control, or may be converted using the gradation correction amount calculated in S735.

[0197] In S740, the system control unit 50 calculates a representative value of the luminance of the detected face. For example, the representative value may be a statistical value such as the maximum value based on the luminance values ​​of each face luminance acquisition area of ​​the detected face area.

[0198] In S741, the system control unit 50 detects a histogram feature amount. The histogram feature amount is, for example, a level (SD) to which pixels having a cumulative frequency of 1% from the dark side of the histogram belong, a level (HL) to which pixels having a cumulative frequency of 1% from the light side belong, etc.

[0199] In S742, the system control unit 50 converts the histogram feature values ​​calculated in S741 to values ​​that assume image capture at proper exposure. This is a process to correct the fact that the histogram feature values ​​are detected to be lower than when image capture is performed at proper exposure because the image data was captured at an exposure lower than proper exposure. The conversion of the brightness values ​​may be performed to compensate for the exposure correction amount (reduction amount) determined by exposure control, or may be performed using the gradation correction amount calculated in S735.

[0200] In S743, the system control unit 50 calculates a target correction amount. First, the system control unit 50 determines a target brightness level for the representative brightness value of the face or a histogram feature. Then, based on these target brightness levels and the minimum and maximum brightness values ​​in the image data, the system control unit 50 creates a lookup table (input / output characteristics) that defines output brightness levels for input brightness levels using spline interpolation or the like. Hereinafter, this lookup table will also be referred to as the gradation correction amount (B) or the second gradation correction amount.

[0201] Here, the target gradation correction amount may be different for HDR and SDR. Figure 7E shows an example displayed in SDR. Figure 7F shows an example displayed in HDR. The luminance value of the subject is the same in both cases, but for the background, the maximum luminance in SDR is 100 cd / m 2 whereas HDR is 100cd / m 2As a result, even if the subject's brightness value is the same, the HDR image may appear darker. This is called brightness contrast and is a phenomenon caused by the characteristics of human vision. In such cases, applying a correction amount that lifts the darker areas, as shown in Figure 7H, to the gradation correction amount calculated using SDR, as shown in Figure 7G, can produce visually pleasing results.

[0202] The target brightness level for the representative brightness value of the face or the histogram feature of the image data can be set to a fixed value that is empirically considered preferable, but different target brightness levels may be set depending on the representative brightness value or the value of the histogram feature. In this case, a lookup table that defines the relationship between the input level and the target brightness level can be prepared for each parameter (representative brightness value or histogram feature) that sets the target brightness level. The correction characteristics for achieving conversion to the target brightness level thus determined are obtained using a method such as spline interpolation, and saved as a lookup table (or relational expression) to which the gradation correction amount (B) is applied, if necessary.

[0203] In S744, the system control unit 50 combines the gradation correction amount (A) calculated in S735 and the gradation correction amount (B) calculated in S743. For example, the system control unit 50 first applies the gradation correction amount (A) to each input luminance level, and then applies the gradation correction amount (B) to the corrected luminance level to determine the luminance value resulting from this, and creates a lookup table of output luminance levels for each input luminance level.

[0204] In S745, the system control unit 50 performs a process (limiter process) to limit the upper limit of the composite correction amount (composite gradation correction amount) calculated in S744. Combining the gradation correction amount (A) and the gradation correction amount (B) results in a larger correction amount, which can lead to noticeable noise in the corrected image. Therefore, a limit is set on the overall correction amount. The limiter process can be implemented by preparing a table of the maximum correction amount allowed for each brightness value, and replacing output levels in the lookup table created in S744 that exceed the maximum correction amount with the output level corresponding to the maximum correction amount. The gradation correction amount may be calculated for the shooting mode (SDR shooting or HDR shooting) selected by the user, or may be calculated for both SDR and HDR.

[0205] Data Structure Figure 8A shows the structure of still RAW image data recorded on the recording medium 200 in the various recording processes described above. The file format exemplified below is the ISO base media file format defined in ISO / IEC 14496-12. Therefore, this file format has a tree structure with nodes called boxes. Furthermore, each box can have multiple boxes as child elements.

[0206] Image data (image file) 801 begins with a box ftyp (802) for describing the file type, a box moov (803) containing all metadata, a box mdat (808) containing the main media data of the track, and an other box (807). The moov box has, as child elements, a box uuid (804) storing MetaData (805) and a trak box (806) storing information referencing ImageData. MetaData contains image metadata such as the date and time the image was created, the shooting conditions, whether it was shot in HDR or SDR, and other shooting information. The mdat box has, as a child element, ImageData (809), which is the captured still image data.

[0207] Here, the RAW image captured in SDR and the RAW image captured in HDR are recorded in different formats in ImageData (809).

[0208] 8B shows ImageData (809) in which a RAW image captured in SDR is recorded. ImageData (809) contains a THM image (821) developed in SDR quality and compressed using JPEG, an MPF ​​image (822), the original image (823), a RAW image (824), and RAW development parameters (825).

[0209] 8C shows ImageData (809) containing only an HDR image as a display image during HDR shooting. ImageData (809) contains a THM image (826) developed in HDR quality and compressed with HEVC, an MPF ​​image (827), a target image (828), a RAW image (824), and RAW development parameters (825).

[0210] 8D shows ImageData (809) containing both an HDR image and an SDR image for display during HDR shooting. ImageData (809) contains a THM image (821) developed in SDR quality and compressed in JPEG, an MPF ​​image (822), a target image (823), and a THM image (826) developed in HDR quality and compressed in HEVC. ImageData (809) also contains an MPF ​​image (827), a target image (828), a RAW image (824), and RAW development parameters (825).

[0211] 8E shows the ImageData (809) where only the THM image taken during HDR shooting is an SDR image, and the MPF image and the actual image are HDR images. The ImageData (809) contains a THM image (821) developed to SDR quality and compressed in JPEG, an MPF ​​image (827) developed to HDR quality and compressed in HEVC, the actual image (828), a RAW image (824), and RAW development parameters (825).

[0212] The file format shown in this embodiment is an example, and other boxes may be included as necessary. Also, the display image may be included in a box within the moov (803) or in another box (807).

[0213] <<Playback mode processing (SDR) (S439)>> 9A is a flowchart showing details of the playback mode process (S439) using the display unit 28. This process is realized by the system control unit 50 loading a program recorded in the nonvolatile memory 56 into the system memory 52 and executing it.

[0214] In S901, the system control unit 50 determines whether or not index playback is being performed. If it is index playback, the process proceeds to S902, and if not (normal playback), the process proceeds to S903).

[0215] In S902, the system control unit 50 determines the number of images to be played (displayed). In S903, the system control unit 50 determines the images to be played. In S904, the system control unit 50 performs drawing processing of the images to be played. Details of the drawing processing will be described later with reference to FIG. 9B.

[0216] In S905, the system control unit 50 determines whether or not the rendering of all images to be displayed has been completed. If completed, the process proceeds to S906, and if not, the process proceeds to S903.

[0217] In S906, the system control unit 50 outputs an image to the display unit 28, and ends this processing flow.

[0218] The system control unit 50 may also perform the following operation reception process (not shown). First, the system control unit 50 determines whether the user has pressed the menu button 70e. If it is determined that the button has been pressed, the system control unit 50 performs menu display processing. Next, the system control unit 50 determines whether the user has pressed the shutter button 61. If it is determined that the button has been pressed, the system control unit 50 ends the playback mode and performs shooting mode processing. Furthermore, the system control unit 50 determines whether the user has operated the power switch 72 to turn off the power. If it is determined that the power has been turned off, the system control unit 50 performs power-off processing. Furthermore, the system control unit 50 determines whether the user has performed an image switching operation. If it is determined that an image switching operation has been performed, the system control unit 50 proceeds to S903 and performs playback processing for the next image.

[0219] FIG. 9B is a flowchart showing the details of the drawing process (S904).

[0220] In S911, the system control unit 50 acquires information about the image to be played. In S912, the system control unit 50 determines the image to be played (loaded image selection). Details of the process of determining the image to be played will be described later with reference to FIG. 9C.

[0221] In S913, the system control unit 50 reads the image to be played back from the recording medium 200. In S914, the system control unit 50 performs decompression processing of the image to be played back. In S915, the system control unit 50 collects luminance data for each pixel from the image data that has been decompressed in S914. This luminance data is, for example, luminance information, and is used for histogram processing, highlight warning processing, etc.

[0222] In S916, the system control unit 50 determines whether the image to be played back (loaded image) is an HDR image. If it is an HDR image, the process proceeds to S917, and if not (if it is an SDR image), the process proceeds to S920.

[0223] In S917, the system control unit 50 determines whether the setting for HDR assist display during playback is "Assist 1." If it is "Assist 1," the process proceeds to S918, and if not (if it is "Assist 2"), the process proceeds to S919.

[0224] In S918, the system control unit 50 performs SDR conversion (conversion from HDR to SDR) on the image expanded in S914 in accordance with the "Assist 1" setting.

[0225] In S919, the system control unit 50 performs "assist 2" on the image expanded in S914. " setting, and performs SDR conversion (conversion from HDR to SDR).

[0226] In S920, the system control unit 50 performs scaling processing on the image to a size suitable for the display unit 28. In S921, the system control unit 50 determines the layout of the generated image, lays out the image, and ends the drawing processing.

[0227] 9C to 9H are flowcharts showing the details of the read image selection process (S912).

[0228] In S926, the system control unit 50 determines whether the acquired image can be played back. If it can be played back, the process proceeds to S927, and if not, the process proceeds to S936.

[0229] In S927, the system control unit 50 determines whether the image to be played back is a still image. If it is a still image, the process proceeds to S928, and if not, the process proceeds to S935.

[0230] In S928, the system control unit 50 determines whether the image to be played back is a RAW image. If it is a RAW image, the process proceeds to S929, and if not, the process proceeds to S930.

[0231] In S929, the system control unit 50 determines whether the RAW image to be played back is a RAW image captured in HDR. If it is a RAW image captured in HDR, the process proceeds to S931; if not (if it is a RAW image captured in SDR), the process proceeds to S932. This determination is made using the metadata in the RAW file, as described with reference to FIGS. 8A to 8E.

[0232] In S930, the system control unit 50 determines whether the still image determined not to be a RAW image was captured in HDR. If it is HDR, proceed to S933; if not (SDR), proceed to S934. In this embodiment, images captured in HDR are recorded as HEIF files, and images captured in SDR are recorded as JPEG files, so whether the image is HDR or SDR is determined based on whether it is a HEIF file or a JPEG file. Note that metadata within the HEIF file may also be used to determine whether the image is HDR or SDR.

[0233] In S931, the system control unit 50 selects image data to be used for playback from a RAW image file captured in HDR.

[0234] In S932, the system control unit 50 selects image data to be used for playback from the RAW image file captured in SDR.

[0235] In S933, the system control unit 50 selects image data to be used for playback from the HDR-developed still image file.

[0236] In S934, the system control unit 50 selects image data to be used for playback from the SDR developed still image file.

[0237] In S935, the system control unit 50 selects image data (frames) to be displayed from the moving image file.

[0238] In S936, the system control unit 50 performs processing to hide the reproduced image. In this case, information indicating that reproduction is not possible is displayed on the display unit 28 to inform the user that the image cannot be reproduced.

[0239] FIG. 9D is a flowchart showing the details of the process (S931) of selecting image data to be used for playback from a RAW image file captured in HDR.

[0240] In S941, the system control unit 50 determines whether or not index playback is being performed. If index playback is being performed, the process proceeds to S942, and if not (normal playback), the process proceeds to S943.

[0241] In S942, the system control unit 50 determines whether the number of images reproduced in index reproduction is 36 or more. If it is 36 or more, the process proceeds to S945; if not, the process proceeds to S944. Note that in this embodiment, the determination is made based on whether the number is 36 or more, but the threshold number is one example. The threshold number may also be changed depending on the size of the display unit 28.

[0242] In S943, the system control unit 50 determines the "HDR main image for display (HEVC)" (828) as the image data (read image) to be played back.

[0243] In S944, the system control unit 50 determines the "HDR MPF image for display (HEVC)" (827) as the image data to be played back.

[0244] In S945, the system control unit 50 determines the "HDR THM image for display (HEVC)" (826) as the image data to be played back.

[0245] 9E is a flowchart showing another example of the process (S931) of selecting image data to be used for playback from a RAW image file captured in HDR. Specifically, this is a flowchart showing the details of the process of selecting image data to be used for playback from a RAW image file when the RAW image file captured in HDR has an SDR image for display.

[0246] In S951, the system control unit 50 determines whether or not index playback is being performed. If index playback is being performed, the process proceeds to S952, and if not (normal playback), the process proceeds to S953.

[0247] In S952, the system control unit 50 determines whether the number of images to be reproduced in the index reproduction is 36 or more. If it is 36 or more, the process proceeds to S955, and if not, the process proceeds to S954.

[0248] In S953 to S955, the system control unit 50 determines whether or not an SDR image is included in the RAW image file to be played back. This determination is made using the metadata in the RAW file, as described with reference to FIGS. 8A to 8E.

[0249] In S956, the system control unit 50 determines the "HDR main image for display (HEVC)" (828) as the image data to be used for playback.

[0250] In S957, the system control unit 50 determines the "SDR main image for display (JPEG)" (823) as the image data to be used for playback.

[0251] In S958, the system control unit 50 determines the "HDR MPF image for display (HEVC)" (827) as the image data to be used for playback.

[0252] In S959, the system control unit 50 determines the "SDR MPF image (JPEG) for display" (822) as the image data to be used for playback.

[0253] In S960, the system control unit 50 determines the "HDR THM image for display (HEVC)" (826) as the image data to be used for playback.

[0254] In S961, the system control unit 50 determines the "SDR THM image for display (JPEG)" (821) as the image data to be used for playback.

[0255] FIG. 9F is a flowchart showing details of the process (S933) of selecting image data to be used for playback from the HDR developed still image file.

[0256] In S971, the system control unit 50 determines whether or not index playback is being performed. If index playback is being performed, the process proceeds to S972, and if not (normal playback), the process proceeds to S973.

[0257] In S972, the system control unit 50 determines whether the number of images to be reproduced in the index reproduction is 36 or more. If it is 36 or more, the process proceeds to S975, and if not, the process proceeds to S974.

[0258] In S973, the system control unit 50 determines "HDR main image (HEVC)" (not shown) as the image data to be used for playback.

[0259] In S974, the system control unit 50 determines "HDR MPF image (HEVC)" (not shown) as the image data to be used for playback.

[0260] In S975, the system control unit 50 determines "HDR THM image (HEVC)" (not shown) as the image data to be used for playback.

[0261] FIG. 9G is a flowchart for selecting image data to be used for playback from a RAW image file captured in SDR.

[0262] In S981, the system control unit 50 determines whether or not index playback is being performed. If index playback is being performed, the process proceeds to S982, and if not (normal playback), the process proceeds to S983.

[0263] In S982, the system control unit 50 determines whether the number of images to be reproduced in the index reproduction is 36 or more. If it is 36 or more, the process proceeds to S985, and if not, the process proceeds to S984.

[0264] In S983, the system control unit 50 determines the "SDR main image for display (JPEG)" (823) as the image data to be used for playback.

[0265] In S984, the system control unit 50 determines the "SDR MPF image (JPEG) for display" (822) as the image data to be used for playback.

[0266] In S985, the system control unit 50 determines the "SDR THM image (JPEG) for display" (821) as the image data to be used for playback.

[0267] FIG. 9H is a flowchart showing the details of the process (S934) for selecting image data to be used for playback from the SDR developed still image file.

[0268] In S991, the system control unit 50 determines whether or not index playback is being performed. If it is index playback, the process proceeds to S992, and if not (normal playback), the process proceeds to S993.

[0269] In S992, the system control unit 50 determines whether the number of images to be reproduced in the index reproduction is 36 or more. If it is 36 or more, the process proceeds to S995, and if not, the process proceeds to S994.

[0270] In S993, the system control unit 50 determines "SDR original image (JPEG)" (not shown) as the image data to be used for playback.

[0271] In S994, the system control unit 50 determines "SDR MPF image (JPEG)" (not shown) as the image data to be used for playback.

[0272] In S995, the system control unit 50 determines "SDR THM image (JPEG)" (not shown) as the image data to be used for playback.

[0273] <<Playback mode processing (HDR) (S439)>> 10A is a flowchart showing details of the playback mode process (S439) using the external device 300. This process is realized by the system control unit 50 loading a program recorded in the nonvolatile memory 56 into the system memory 52 and executing it.

[0274] In S1001, the system control unit 50 determines whether or not the digital camera 100 is connected to the external device 300. If it is determined that the digital camera 100 is connected, the process proceeds to S1002, and if not, the process proceeds to S1005.

[0275] In S1002, the system control unit 50 determines whether the HDR setting for playback is enabled. The HDR setting for playback can be selected and set from "HDR playback," "HDR playback not enabled," and "linked to shooting mode." Furthermore, "HDR playback enabled" is a mode that outputs HDR images if the external device 300 supports HDR, regardless of whether the image being played is an HDR image or an SDR image. "HDR playback not enabled" is a mode that outputs SDR images. "Linked to shooting mode" is a mode that links the output during playback with the shooting mode. If "HDR shooting" is set to "enabled," HDR output is also enabled during playback. If "HDR shooting" is set to "disabled," SDR output is also enabled during playback. In S1002, if "HDR playback enabled" is selected, the process proceeds to S1003. If "HDR playback not enabled" is selected, the process proceeds to S1005. In addition, in the case of "linked shooting mode," if the "HDR shooting" set in S606 is "enabled," the process proceeds to S1003; if it is "disabled," the process proceeds to S1005.

[0276] In S1003, the system control unit 50 determines whether or not the external device 300 is an HDR-compatible display. If the external device 300 is an HDR-compatible display, the process proceeds to S1004; if not, the process proceeds to S1005.

[0277] In S1004, the system control unit 50 outputs the HDR video image (signal) to the external device 300.

[0278] In S1005, the system control unit 50 outputs the SDR video to the external device 300.

[0279] S1006 to S1008 and S1010 to S1011 are the same as S901 to S903 and S905 to S906 described in FIG. 9A, so the description will be omitted. 10B will be used later.

[0280] FIG. 10B is a flowchart showing the details of the drawing process (S1009) in the playback mode process using the external device 300.

[0281] S1021 to S1025, S1028, and S1029 are the same as S911 to S915, S920, and S921 described with reference to FIG. 9B, and therefore the description thereof will be omitted.

[0282] In S1026, the system control unit 50 determines whether the image to be played back is an HDR image. If it is an HDR image, the process proceeds to S1028, and if not (if it is an SDR image), the process proceeds to S1027.

[0283] In S1027, the system control unit 50 performs HDR conversion (conversion from SDR to HDR).

[0284] <Playback menu processing> 11A is a flowchart showing the details of the playback menu process according to this embodiment. This process is realized by the system control unit 50 loading a program recorded in the nonvolatile memory 56 into the system memory 52 and executing it.

[0285] In S1101, the system control unit 50 refers to the user settings in the RAW development setting items (not shown) and determines whether or not to perform RAW development. If RAW development is to be performed, the process proceeds to S1102, and if RAW development is not to be performed, the process proceeds to S1103.

[0286] In S1102, the system control unit 50 performs RAW development processing on the user-specified RAW image file. For example, the system control unit 50 performs HDR development and saves the image as a HEIF file, or performs SDR development and saves the image as a JPEG file. Details of the RAW development processing will be described later with reference to FIG. 11B.

[0287] In S1103, the system control unit 50 refers to the SDR conversion setting items (not shown) for the HDR file and determines whether to perform SDR conversion (conversion from HDR to SDR). If SDR conversion is to be performed, the process proceeds to S1104, and if SDR conversion is not to be performed, the process proceeds to S1105.

[0288] In S1104, the system control unit 50 performs SDR conversion on the user-specified HDR image file. Because HDR images are images generated in a color space such as PQ OETF and BT.2020 gamut, tone mapping and gamut mapping must be performed on the image to convert it to an SDR color space such as γ2.2 or sRGB. While any known technique may be used as a specific method, for example, tone mapping that aligns the correct exposure with SDR can result in a result with adjusted brightness compared to SDR.

[0289] In S1105, the system control unit 50 refers to the file transfer setting items (not shown) to determine whether or not to transfer the file. If the file is to be transferred, the process proceeds to S1106, and if the file is not to be transferred, the process proceeds to S1107.

[0290] In S1106, the system control unit 50 performs transfer processing on the user-specified image file. When transferring an HDR image file, if the receiving device can only display SDR, the SDR conversion shown in S1104 may be performed in the camera before the file is transferred as an SDR image file.

[0291] In S1107, the system control unit 50 determines whether or not to end the playback menu process. If the playback menu processing is to be ended, this processing flow is ended, otherwise the process proceeds to S1101.

[0292] FIG. 11B is a functional block diagram of each function that performs the RAW development process (S1102). The system control unit 50 reads a captured RAW image 1111 recorded on the recording medium 200, and the image processing unit 24 performs RAW development processing. Each pixel in the RAW image represents only the intensity of a single color plane. Note that RAW images are classified as RAW (SDR) when captured in SDR mode and RAW (HDR) when captured in HDR mode. Furthermore, during development, RAW (SDR) may be directly developed into SDR or HDR. Conversely, RAW (HDR) may be developed into HDR or SDR. The white balance unit 1112 performs processing to make white whiter. When RAW (HDR) is subjected to HDR development, white balance processing is performed using HDR white balance coefficients recorded in the file. When SDR development is performed, SDR white balance coefficients stored in the file are generated and used for white balance processing. Note that if both HDR and SDR white balance coefficients are recorded in the RAW image, the appropriate one may be used.

[0293] The color interpolation section 1113 generates a color image in which the R, G, and B color information is aligned at all pixels by interpolating noise reduction and color mosaic images. A basic color image is generated from the generated color image through the matrix conversion section 1114 and the gamma conversion section 1115. Thereafter, processing for adjusting the appearance of the image is performed on the color image by the color luminance adjustment section 1116. For example, image correction such as detecting a sunset scene and enhancing its saturation is performed according to the scene. Tone correction is also performed in the same manner. When developing RAW (HDR) into HDR, tone correction is performed using the tone correction amount for HDR stored in the file. When developing SDR, the tone correction amount for SDR is calculated using the face detection result and histogram recorded in the file, and tone correction is performed. If both the tone correction amounts for HDR and SDR are recorded in the RAW image, the necessary one may be used as appropriate.

[0294] For the image subjected to the desired color adjustment, a developed image is generated in which the high-resolution image is compressed by a method such as JPEG or HEVC by the compression section 1117 and recorded on a recording medium such as a flash memory by the recording section 1118. Since the above-mentioned HEIF container can store a plurality of images, images developed in SDR as well as images developed in HDR may be included and stored.

[0295] <OSD Luminance Setting Process> FIG. 12A is a flowchart showing an example of the OSD luminance setting process according to the present embodiment. This process is realized by expanding the program recorded in the non-volatile memory 56 into the system memory 52 and executing it by the system control unit 50. This process is a process of adjusting (setting) the luminance of the OSD when displaying the information of the OSD on the HDR display.

[0296] In S1201, the system control unit 50 determines whether the display (in this embodiment, the external device 300) where the OSD is to be displayed supports HDR display. If it supports HDR display, the process proceeds to S1202; otherwise, the process proceeds to S1206.

[0297] In S1202, the system control unit 50 determines whether or not there is one image to be displayed. If there is one image, the process proceeds to S1203, and if not (if there are multiple images), the process proceeds to S1204.

[0298] In S1203, the system control unit 50 determines the luminance gradation value (white signal value) of the OSD based on the image (one image) to be displayed. The details of this process will be described later with reference to FIG. 12B.

[0299] In S1204, the system controller 50 determines the luminance gradation values ​​of the OSD based on the images to be displayed. Details of the process of determining the luminance gradation values ​​of the OSD based on the images will be described later with reference to FIG. 12B.

[0300] In S1205, the system control unit 50 acquires the displayable luminance of the display. This is to determine whether the display is capable of displaying the OSD luminance gradation values ​​determined in S1203 or S1204. The displayable luminance of the display can be acquired, for example, by acquiring EDID information containing display luminance information of the display using a DDI (Display Data Channel). Note that it is a prerequisite that both the graphic board and the display support DDI. Note that if the display luminance information of the display cannot be acquired, a display luminance selected by the user from a prepared assumed display luminance may be acquired. The assumed display luminance is, for example, 1000 cd / m, which conforms to the VESA DisplayHDR standard. 2 , 600cd / m 2 , 400cd / m 2 and UltraHD PremiumLOGO compliant 1000cd / m 2 , 500cd / m 2 etc.

[0301] In S1206, the system control unit 50 determines the luminance gradation values ​​of the OSD based on the SDR converted image (image after SDR conversion; SDR image). For example, the maximum luminance gradation value among the luminance gradation values ​​of all pixels constituting the SDR converted image is calculated, and the maximum luminance gradation value is used to determine the luminance gradation value of the OSD. In the case of SDR, the color gamut is Rec. 709, the gamma is 2.2, etc., and therefore the luminance gradation value is a relative value. However, with the recent spread of HDR displays, the peak luminance when SDR display is performed on an HDR display is also becoming higher. If a conventional SDR display is performed here, for example, the luminance gradation value of white (white luminance) of 255 (8 bits) will be 1000 cd / m 2 This raises the concern that the same problem of excessive brightness as with HDR may occur. If the peak brightness of the SDR display can be determined from the EDID information, the OSD white brightness gradation value can be set to the SDR code value that corresponds to the brightness gradation value determined in the same way as for HDR (processing from S1202 onwards).

[0302] In S1207, the system control unit 50 adjusts the OSD luminance gradation value. Specifically, the system control unit 50 determines the smaller luminance gradation value of the OSD luminance gradation value determined in S1203 or S1204 or the displayable luminance of the display acquired in S1205 as the final OSD luminance gradation value.

[0303] In S1208, the system controller 50 determines the brightness gradation value of the gray characters in the OSD. The gray characters are OSD that indicate that the function displayed in the gray characters is disabled. The process of determining the brightness gradation value of the gray characters will be described later with reference to FIG. 12C.

[0304] In S1209, the system control unit 50 sets the determined luminance gradation values ​​(the luminance gradation values ​​of the OSD and the luminance gradation values ​​of the gray characters).

[0305] FIG. 12B is a flowchart showing the details of the process (S1203, S1204) for determining the luminance gradation value (white signal value) of the OSD in this embodiment.

[0306] In S1211, the system control unit 50 acquires the maximum brightness gradation value among all pixels that make up the display image. For example, the system control unit 50 analyzes a histogram of the brightness signal of the display image and acquires the signal value of the bin on the highest brightness side (maximum value of the brightness signal). If the maximum value is simply selected, an extremely low-frequency value such as the signal value of a bright spot will be acquired, which will have a low correlation with visual appearance. Therefore, for example, the system control unit 50 selects the signal value of the top 1% (high brightness side) of the histogram. Alternatively, the average value of the signal values ​​may be acquired instead of the maximum value. Furthermore, if there are multiple images to be displayed (in the case of S1204), the maximum luminance gradation value is acquired from each of the multiple images, and the maximum, minimum, average, median, mode, etc. of the acquired multiple luminance gradation values ​​are acquired.

[0307] In S1212, if the luminance gradation value acquired in S1211 is lower than a minimum luminance gradation value (predetermined value) stored in advance, the system control unit 50 determines the minimum luminance gradation value as the luminance gradation value of the OSD. If the OSD is displayed according to the maximum value acquired in S1211, the OSD will also be dark in a scene with only dark pixels (a low-key scene). For this reason, in this embodiment, a minimum luminance value is stored in advance as the luminance of the OSD, and if the maximum value acquired in S1211 is lower than the minimum luminance value, the minimum luminance value is set as the luminance value of the OSD. For example, the minimum luminance value is 100 cd / m, which is listed as a de facto standard in the Appendix of ITU-R BT.1886. 2 can be used.

[0308] In S1213, the system control unit 50 sets the luminance gradation value determined in the above process as the luminance gradation value of the OSD. That is, if the luminance of the OSD (graphic image) is L1 and the luminance of the display image (HDR image) is L2, L1 = f(L2) can also be considered as an increasing function in a broad sense. Note that the higher the luminance of the display image, the higher the luminance of the OSD should be. The luminance of the OSD may increase linearly in proportion to the luminance of the display image, or may increase in a stepped manner (step function).

[0309] FIG. 12C is a flowchart showing the details of the process (S1208) for determining the luminance gradation value of gray characters in the OSD.

[0310] In S1221, the system control unit 50 acquires the brightness gradation values ​​of white and gray during SDR display. For example, the system control unit 50 acquires 255 (8 bits) as the brightness gradation value of white during SDR display and 128 (8 bits) as the brightness gradation value of gray during SDR display.

[0311] In S1222, the system control unit 50 converts the values ​​acquired in S1221 into ICtCp values. Specifically, the SDR white and gray signal values ​​are converted from XYZ values ​​to ICtCp values ​​using the conversion formulas described in the publicly available Dolby White Paper Version 7.2 (ICtCp_DolbyWhitePaper.pdf).

[0312] In S1223, the system control unit 50 calculates the difference ΔI (perceptual difference) between the ICtCp value of white and the ICtCp value of gray. Note that since the luminance of SDR is relative luminance, the white of the text is always 100 cd / m 2 200cd / m 2 If you want to consider this as SDR character white, you can do this by linearly scaling the XYZ color space in the middle of the conversion formula to ICtCp.

[0313] In S1224, the system control unit 50 acquires the OSD luminance gradation value determined in S1207 for HDR.

[0314] In S1225, the system control unit 50 converts the value acquired in S1224 into ICtCp, similar to SDR.

[0315] In S1226, the system control unit 50 calculates the ICtCp value of the gray character. In HDR, if the OETF of the image is PQ, the code value and the luminance value correspond one-to-one, so the above-mentioned line Shape scaling is not necessary. If the image's OETF is HLG, the desired scaling is required, just as with SDR, because it is a relative value. In this case, the expected system gamma must also be taken into consideration. In order to achieve the same perceptual difference between white and gray in SDR in HDR, the ICtCp value for gray in HDR can be set to the ICtCp value obtained by subtracting the perceptual difference ΔI from the ICtCp value for white in HDR.

[0316] In S1227, the system control unit 50 calculates the luminance gradation value (Y) from the ICtCp value of the gray character in HDR calculated in this way via LMS, XYZ, and RGB.

[0317] In S1228, the system control unit 50 sets the luminance gradation value of gray characters on the OSD during HDR display.

[0318] 13A to 13C are diagrams showing display images according to a conventional method. Conventionally, when a captured image is played back and displayed on the rear LCD of an imaging device, an OSD (graphic image) such as shooting information (seconds, aperture value, ISO sensitivity, image name) and playback status information has been superimposed on the captured image.

[0319] For example, in the case of SDR images, as shown in Figure 13A, the maximum possible gradation value of an SDR image and the white of an OSD are generally set to the same 8-bit code, with a maximum value of 255. However, in the case of HDR images, particularly image data conforming to the PQ standard, the maximum gradation value that exists in an HDR image using a 10-bit code is generally not 1023. Therefore, if the maximum value of 1023 is assigned to the white of an OSD (although this depends on the performance of the display), the OSD will appear brighter and more dazzling than the image, as shown in Figure 13B. On the other hand, the background image will appear relatively dark, creating concerns that it will be difficult to see. Therefore, for example, if a gradation value of 520 is assigned to the white of the OSD, it will not be as dazzling, as shown in Figure 13C. Here, the value 520 is the value of 100 cd / m , which is listed as the de facto standard in the Appendix of ITU-R BT.1886. 2This is a PQ code value that corresponds to the brightness of the OSD. However, there is a concern that if the background image data is bright, it may appear relatively dark and difficult to see. Also, darkening the white of the OSD may make it appear gray, but in this case, gray often indicates a disabled function, so there is a concern that users may be confused.

[0320] 13D and 13E are diagrams showing display images according to this embodiment. As shown in FIG. 13D, the maximum luminance gradation value of the background image and the luminance gradation value of the white characters of the OSD (graphic image) are the same, thereby improving the visibility of the OSD. Furthermore, as shown in FIG. 13E, even if the maximum luminance of the background image differs for each image displayed, the visibility of the OSD can always be improved by matching the luminance of the white characters of the OSD to the maximum luminance of the background image at that time. As described above, according to the first embodiment, it is possible to make the OSD superimposed on the background image appear highly visible even on an HDR display capable of displaying high luminance.

[0321] Furthermore, the above-described processing maintains the perceptual difference between the white area (1401) of the OSD and the gray area (gray characters or symbols) (1402) shown in Fig. 14 (SDR display) even when the luminance gradation value of white changes upon switching to HDR display. This makes it possible to perceive gray luminance with the same sensation as in SDR.

[0322] (Embodiment 2) Next, a second embodiment of the present invention will be described. In the first embodiment, the luminance gradation value of the OSD is determined based on the maximum luminance gradation value of the background image. In contrast, the second embodiment differs from the first embodiment in that the luminance gradation value of the OSD is determined based on the shooting conditions of the background image.

[0323] Specifically, in the process of S1203 or S1204 in Fig. 12A, the luminance gradation value of the OSD is determined according to the input dynamic range setting at the time of image capture and the gamma setting at the time of development. 2However, if the input dynamic range is narrow, it is possible that the output will not fully use 1023. The maximum value of the output signal actually used depends on the gamma curve during development, and a gamma characteristic with high contrast will result in a higher maximum value of the output signal. Therefore, for example, there are four different maximum signal values ​​corresponding to different shooting conditions: a narrow or wide D-range setting in the shooting mode during shooting, and a high or standard contrast gamma setting (development mode) during development. Figure 15 shows an example of four maximum signal values ​​and maximum luminance values ​​determined by the shooting conditions. The maximum signal value corresponding to the shooting conditions can be assigned as the white luminance gradation value of the OSD.

[0324] As described above, in the second embodiment, the visibility of the OSD is improved by determining the white luminance gradation value of the OSD based on the image capture conditions, rather than the pixel values ​​of the image. For example, when multiple images are played back and viewed sequentially, even if the maximum luminance value of the content varies from image to image, the white luminance gradation value of the OSD does not change, making it possible to suppress flickering and improve visibility.

[0325] Here, when multiple images captured under different shooting conditions are viewed sequentially while being forwarded, there is a concern that flickering may occur. Therefore, as shown in FIG. 16, if the shooting conditions are different between the first and second images, the brightness of the OSD may be gradually changed (1601). Similarly, the brightness is gradually changed when advancing from the second to the third image (1602). However, if images are advanced at high speed by, for example, continuously pressing an image advance button (not shown), the brightness does not need to be gradually changed. In this case, for example, flickering can be reduced by maintaining the brightness of the first image and displaying the OSD at a brightness that corresponds to the shooting conditions of the image being displayed when the image advance button is released (1603).

[0326] (Embodiment 3) Next, a third embodiment of the present invention will be described. In the first and second embodiments, the luminance gradation value of the OSD white is set based on the pixel values ​​of the displayed image or the shooting conditions. In the third embodiment, when multiple images including HDR images and SDR images are multi-played in an index display or the like, the luminance gradation value of the OSD white is determined based on the multiple images, which is different from the first and second embodiments. That is, the processing in S1204 is different from the first embodiment described above.

[0327] 17A to 17C are diagrams showing an example of multi-playback display according to this embodiment. For example, among four 2×2 images, the top two are SDR images with a maximum luminance value of 100 cd / m 2 The two images in the bottom row are HDR images with a maximum brightness of 648 cd / m 2 In this case, if the brightness gradation value of the white of the OSD is matched to the HDR image as in FIG. 17A, the brightness of the OSD becomes too high relative to the brightness of the SDR image, reducing the visibility of the OSD and also reducing the visibility of the SDR image. Also, if the brightness gradation value of the white of the OSD is matched to the SDR image as in FIG. 17B, the brightness of the OSD becomes too low relative to the brightness of the HDR image, reducing the visibility of the OSD. Therefore, in the third embodiment, the brightness gradation values ​​of the white of multiple images are adjusted.

[0328] FIG. 18 is a flowchart showing an example of the OSD luminance setting process according to the third embodiment.

[0329] In S1801, the system control unit 50 acquires the white luminance gradation values ​​of a plurality of images. In S1802, the system control unit 50 calculates the median value of the white luminance gradation values ​​of all the acquired images. In S1803, the system control unit 50 sets the calculated median value as the white luminance gradation value of the OSD. In this way, as shown in FIG. 17C, SDR images, HDR images, and the like can be displayed. Since the image and OSD can be displayed in a balanced manner, visibility can be improved. Note that the method for calculating the white luminance gradation value from multiple images (S1802) is not limited to the median, and statistics such as the average value or the mode value may also be used.

[0330] As described above, in the third embodiment, when multiple images are displayed simultaneously (side by side), the white luminance gradation value of the OSD is determined based on the white luminance gradation value of all the images. By doing so, it becomes possible to improve the visibility of all of the SDR image, HDR image, and OSD, even in multi-display such as index playback, for example.

[0331] (Embodiment 4) Next, a fourth embodiment of the present invention will be described. In the first to third embodiments, one luminance gradation value of the OSD white is set for the entire screen. That is, in the first to third embodiments, one luminance gradation value of the OSD white is set based on the entire background. However, a subject has bright and dark areas, and an SDR image is often darker than an HDR image. Therefore, the fourth embodiment differs from the first to third embodiments in that an optimal luminance gradation value of the OSD white is set for each area within the plane of a single background image.

[0332] For example, the white luminance gradation value of the bottom OSD in Figure 19A is about the same luminance as the background image, so there is no problem with the visibility of the bottom OSD. However, the top OSD, which has the same white luminance gradation value as the bottom OSD, appears relatively dark because the background image is bright. In such cases, it is difficult to set a single optimal white luminance gradation value for a single image. Therefore, it is desirable to consider the characteristics of the image and set an optimal white luminance gradation value for the OSD for each region within the image plane.

[0333] FIG. 20 is a flowchart showing an example of the OSD luminance setting process according to the fourth embodiment.

[0334] In S2001, the system control unit 50 acquires a luminance distribution image (an image showing the luminance distribution) based on the display image. The luminance components of the display image may be used as the luminance distribution image. Since luminance components in the mid- to low-frequency band (a low-frequency band in which the spatial frequency is lower than a predetermined frequency) are less likely to cause image quality degradation such as tone jump in the processing described later, a process to reduce the high-frequency band (high-frequency band) of the luminance signal of the display image is performed in advance. There are several possible processes for reducing (suppressing) the high-frequency band, but for example, a filtering process that applies an LPF (low-pass filter) horizontally and vertically may be performed.

[0335] In S2002, the system control unit 50 uses the calculated luminance distribution image to determine the OSD luminance gradation value for each region (modify the OSD information display image). Note that the OSD luminance gradation value can be increased in regions with higher luminance in the luminance distribution image. In this case, the higher the luminance of the region, the higher the OSD luminance gradation value. There are several possible methods for achieving this, including computing the luminance distribution image of the display image and the OSD information display image using screen processing. Screen processing is a layer compositing method and can be performed using known techniques. Specifically, it can be performed by multiplying a display image with black and white inversion by an OSD information display image with black and white inversion, and then re-inverting the resulting result.

[0336] In S2003, the system control unit 50 sets the white luminance gradation value of the OSD calculated in S2002 to the actual luminance gradation value (code value). By doing so, it is possible to superimpose a bright OSD in the bright area (top edge) and a dark OSD in the dark area (bottom edge) within the display image, as shown in FIG. 19B.

[0337] As described above, according to the fourth embodiment, different regions of an image are processed according to the characteristics of the image. By setting the brightness gradation value of white on the OSD, it is possible to display OSD information with stable visibility regardless of the subject.

[0338] The present invention can also be realized by executing the following process: software (programs) that realize the functions of the above-described embodiments are supplied to a system or device via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or device reads and executes the programs.

[0339] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention.

[0340] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0341] 50: System control unit

Claims

1. An electronic device having an imaging element and a display unit, a display control means for controlling the display unit to display a graphic image together with an HDR image, the display control means controls the display of the graphic image at a second luminance when the luminance related to the HDR image is a first luminance, and controls the display of the graphic image at a fourth luminance when the luminance related to the HDR image is a third luminance higher than the first luminance, The luminance of the HDR image is an upper limit luminance corresponding to a dynamic range set in the electronic device at the time of image capture, and is also an upper limit luminance corresponding to a gamma set in the electronic device at the time of development. An electronic device characterized by:

2. the display control means gradually changes the brightness of the graphic image when switching the image to be displayed on the display unit; 2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

3. the display control means adjusts the second luminance or the fourth luminance based on a display luminance selected by a user.

3. The electronic device according to claim 1, wherein the first and second electrodes are electrically connected to the first and second electrodes.

4. the display control means adjusts the second luminance or the fourth luminance based on the displayable luminance acquired from the display unit.

3. The electronic device according to claim 1, wherein the first and second electrodes are electrically connected to the first and second electrodes.

5. the display control means determines the brightness of gray indicating invalidity in the graphic image during HDR display based on a difference between an upper limit brightness of the graphic image during SDR display and the brightness of gray, and based on the second brightness or the fourth brightness.

5. The electronic device according to claim 1, wherein the first and second electrodes are electrically connected to the first and second electrodes.

6. A method for controlling an electronic device having an imaging element and a display unit, a display control step of controlling the display unit to display a graphic image together with the HDR image, In the display control step, when a luminance related to the HDR image is a first luminance, the graphic image is displayed at a second luminance, and when the luminance related to the HDR image is a third luminance higher than the first luminance, the graphic image is displayed at a fourth luminance higher than the second luminance; The luminance of the HDR image is an upper limit luminance corresponding to a dynamic range set in the electronic device at the time of image capture, and is also an upper limit luminance corresponding to a gamma set in the electronic device at the time of development. A control method comprising:

7. A program for causing a computer to function as each of the means of the electronic device according to any one of claims 1 to 5.

Images